Outcome MappingMonitoring and Evaluating Policy Influencing8 December 2009Simon Hearn, s.hearn@odi.org.uk

Brief definition of OM

• A participatory method for planning, monitoring and evaluation

• Focused on changes in behaviour of those with whom the project or program works

• Oriented towards social & organizational learning

2

3

The Problem

Policy change can be:

• Complex (involve a confluence of actors and factors)

• Unstable (independent of project duration)

• Non-linear (unexpected, emergent, discontinuous)

• Two-way (intervention may change)

• Beyond control (but subject to influence)

• Incremental, cumulative (watersheds & tipping points)

5

1. Contribution, not Attribution

• Change processes involve interactions among multiple actors and factors

2. There is a limit to our influence

Sphere of Control

Sphere of Influence

Sphere of Interest

Inputs Activities Outputs Outcomes Impacts

3. Identify and focus on key agents of change

Develop enthusiasm to address topic

Learn in partnership

Develop awareness

and enthusiasm

Challenge existing beliefs

High

Gen

eral

leve

l of

alig

nmen

t

Low

Low HighInterest in specific topic

1. Map actors on the matrix

2. Identify which are the most influential

3. Who do you work with directly?

4. Outcomes as progressive behaviour changes

(Deep transformation)

(Active engagement)

(Early positive responses)

Love to see

Like to see

Expect to see

...policy change examples

• Discourse changes• Procedural changes• Content changes• Attitudinal changes• Behavioural changes

5. Use a broad range of strategies

causalpersuasiv

esupportiv

e

Aimed at the actor

Aimed at the actors

environment

6. M&E on three levels

12

Initiative Policy Actor

outcomes(behaviour changes in the actors)

implementation(interventions by the program)

relevance & viability(actions of the program)

Strategies

7. Can’t prove causation

• But can make a plausible estimate based on:– Timing: Did the change happen after the activity?– Logic: Is it reasonable to expect that these inputs

would contribute to the change?– Expert Judgment: Do knowledgeable people –

including those involved – agree with the contribution claim?

– Alternative Explanations: What other factors could explain the change?

8. Clarify intent from the outset

• Need a systematic approach to choosing areas for influencing, setting objectives and defining outcomes

• This will enable a systematic method for collecting monitoring information and evidence of change

Evaluating REGLAP

• RAPID Outcome Assessment– Outcome Mapping– Episode Studies– Most Significant Change

• Tracks key changes in target actors and surrounding environment

• Makes a judgement on the contribution of project activities on the changes

• Recognises external events

Evaluating REGLAP

1. Document review2. Interviews with project team3. Workshop with project team, key

stakeholders, external experts

Evaluating REGLAP

1. Clarify...– Policy objectives– Key policy actors targeted– Desired changes in and around these actors

2. Collect evidence of...– Activities undertaken and outputs of strategies– Internal changes within the programme– Changes in behaviour of key actors– Other changes in the policy environment

3. Map the evidence and discuss linkages

Evaluating REGLAP

• Strategy• Management • Outputs • Uptake• Outcomes and Impact

Somewhat

BP 0 1 2 3 4 5 6 7 8 9 10

Project

EE

Po

lic

y E

nvi

ron

me

nt

Po

licy C

han

ge

BP 0 1 2 3 4 5 6 7 8 9 10

Project

EE

Po

lic

y E

nvi

ron

me

nt

Po

licy C

han

ge

Step 1: Describe the policy environment

Before TodayBP 0 1 2 3 4 5 6 7 8 9 10

BP1

BP2

BP3

BP4

BP5

BP6

BP7

Project

EE

Po

lic

y E

nv

iro

nm

en

t

year/month

Po

licy

Ch

an

ge

Step 2: Identify key policy actors

Before TodayBP 0 1 2 3 4 5 6 7 8 9 10

BP1 0 5

BP2 0 4

BP3 0 8

BP4 0 7

BP5 0 4

BP6 0 9

BP7 0 3

Project

EE

Po

licy

En

viro

nm

ent

year/month

Po

licy C

han

ge

Before TodayBP 0 1 2 3 4 5 6 7 8 9 10

BP1 0 5

BP2 0 4

BP3 0 8

BP4 0 7

BP5 0 4

BP6 0 9

BP7 0 3

Project

EE

Po

licy

En

viro

nm

ent

year/month

Po

licy C

han

ge

Step 3: Describe the behaviour of the key actors

Before TodayBP 0 1 2 3 4 5 6 7 8 9 10

BP1 0 1 2 3,4 5

BP2 0 1 2,3 4

BP3 0 1 2 3 4,5,6 7 8

BP4 0 1 2 3 4 5 6 7

BP5 0 1 2 3 4

BP6 0 1,2 3,4,5 6,7,8 9

BP7 0 1, 2 3

Project

EE

Po

lic

y E

nv

iro

nm

en

t

year/month

Po

licy

Ch

an

ge

Step 4: Map the key changes in behaviour

Before TodayBP 0 1 2 3 4 5 6 7 8 9 10

BP1 0 1 2 3,4 5

BP2 0 1 2,3 4

BP3 0 1 2 3 4,5,6 7 8

BP4 0 1 2 3 4 5 6 7

BP5 0 1 2 3 4

BP6 0 1,2 3,4,5 6,7,8 9

BP7 0 1, 2 3

Project 0 1 2,3 4,5 6 7 8 9 10

EE 0 1 2 3,4 5 6 7 8 9 10 11

Po

lic

y E

nvi

ron

me

nt

year/month

Po

licy C

ha

ng

e

Before TodayBP 0 1 2 3 4 5 6 7 8 9 10

BP1 0 1 2 3,4 5

BP2 0 1 2,3 4

BP3 0 1 2 3 4,5,6 7 8

BP4 0 1 2 3 4 5 6 7

BP5 0 1 2 3 4

BP6 0 1,2 3,4,5 6,7,8 9

BP7 0 1, 2 3

Project 0 1 2,3 4,5 6 7 8 9 10

EE 0 1 2 3,4 5 6 7 8 9 10 11

Po

lic

y E

nvi

ron

me

nt

year/month

Po

licy C

ha

ng

eStep 5: Map project changes and external influences

Before TodayBP 0 1 2 3 4 5 6 7 8 9 10

BP1 0 1 2 3,4 5

BP2 0 1 2,3 4

BP3 0 1 2 3 4,5,6 7 8

BP4 0 1 2 3 4 5 6 7

BP5 0 1 2 3 4

BP6 0 1,2 3,4,5 6,7,8 9

BP7 0 1, 2 3

Project 0 1 2,3 4,5 6 7 8 9 10 11

EE 0 1 2 3,4 5 6 7 8 9 10 11

Po

licy

En

viro

nm

ent

year/month

Po

licy C

han

ge

Before TodayBP 0 1 2 3 4 5 6 7 8 9 10

BP1 0 1 2 3,4 5

BP2 0 1 2,3 4

BP3 0 1 2 3 4,5,6 7 8

BP4 0 1 2 3 4 5 6 7

BP5 0 1 2 3 4

BP6 0 1,2 3,4,5 6,7,8 9

BP7 0 1, 2 3

Project 0 1 2,3 4,5 6 7 8 9 10 11

EE 0 1 2 3,4 5 6 7 8 9 10 11

Po

licy

En

viro

nm

ent

year/month

Po

licy C

han

ge

Step 6: Determine level of influence

Influences:DirectIndirectExternal