1

Developing Indicators

Workshop for Børne - og ungdomsnetværket

CopenhagenWednesday 13 August, 2003

2

During this workshop we will

• Place the development of indicators in the context of M&E and LFA

• Clarify what we understand by indicators and their importance to improving organisational performance

• Examine the characteristics of a good indicator and the process of developing them

• Explore the particular challenges of developing indicators for rights-based work with children

• Identify ways in which the network can support members in developing indicators for their work with children

3

Monitoring and evaluation

• “Monitoring is the systematic and continuous assessment of the progress of a piece of work over time.”

• “ An evaluation is the assessment at one point in time of the impact of a piece of work and the extent to which the stated objectives have been achieved.”

4

Why monitor and evaluate?

• To improve performance• To improve day-to-day decision-making• To enhance impact• To provide early warning of problems• To empower stakeholders• To build understanding and capacity• To stimulate learning• To demonstrate & strengthen

accountability

5

The programme management cycle

• Situation analysis• Planning• Implementation• Monitoring• Review and evaluation

6

LFA and indicators

• LFA is a commonly used approach for project and programme planning, implementation and evaluation

• LFA structures thinking about links between objective setting, monitoring and evaluation

• A Logframe matrix is used to map out the ‘objective hierarchy’ Goal, Purpose, Objectives, Outputs and Activities

• Requires the identification of indicators and means of verification for each ‘level’

7

Key stages in the Logical Framework Approach

1. Establish the scope of the work2. Agree on the planning framework3. Undertake a situation analysis

• Problem analysis• Stakeholder analysis• Objectives analysis

4. Develop the implementation strategy• Complete the Logframe matrix• Establish indicators and MoV• Identify & analyse assumptions & risks

5. Develop the M&E framework

8

Logframe terms and example

Goal The broader development impact to which the project contributes

Improved community health on a sustainable basis

Purpose The development outcome expected at the end of the project

A clean, reliable and sustainable supply of water adequate for community needs

ComponentObjectives

The expected outcomes for each component of the project

1.Reticulated water supply established by (date)

2.Competent village water supply technicians in place by (date)

Outputs The direct measurable results of the project (largely under management’s control)

1.1 plan for water supply1.2 suitable header tank 1.3 operational pump

Activities The tasks carried out to implement the project & deliver the outputs

Conduct site survey; identify local labour sources; build header tank; construct pump.

9

The Logframe matrix

Project Description

IndicatorsMeans of Verification

Assumptions

Goal

Purpose

Objectives

Outputs

Activities

10

What is an indicator?

An indicator is the quantitative or qualitative evidence that will be used to assess progress towards an objective. Indicators provide the basis for monitoring progress and evaluating the achievement of outcomes. An objective may have more than one indicator.

11

Different types of indicators

• Process indicators • Impact indicators• Quantitative indicators• Qualitative indicators• Proxy indicators• Indices

12

SMART Properties of indicators

• Specific• Measurable (and unambiguous)• Attainable (and sensitive)• Relevant• Time-bound

13

The Logframe matrixProject Description

Example IndicatorsMeans of Verification

Goal Improved community health on a sustainable basis

Reduced water-borne disease rates.

Health recordsInterviews with community members

Purpose A clean, reliable and sustainable supply of water adequate for community needs

Water cleanliness at acceptable standardsWater availability with no supply shortages of more than (specified time).Supply breakdowns fixed within agreed standards.

Water cleanliness testsSite visits to examine supplyInterviews with community members

Objectives 1. Water supply established by (date)

2. Competent village water supply technicians in place by (date)

Water supply in place and being used by (date)Village technicians able and dealing with supply maintenance & problems.

Site visitsRecords of supply use.Testing of technicians’ ability to deal with problems.Technicians maintenance logs

Outputs 1.1 plan for water supply1.2 suitable header tank 1.3 operational pump

Workable plan produced and used for constructionTank and pump constructed to agreed standards

Examination of planSite visits to examine ‘hardware’

Activities Conduct site survey; identify local labour sources; build header tank; construct pump.

Work plan targets met Work planConstruction logSite visits

14

Developing indicators that address performance issues

• Establish goal, purpose and objectives in the Logframe

• Identify performance questions for all levels in the objective hierarchy

• Develop the indicators needed to answer the performance questions

• Develop the means of verification

15

Criteria for selecting indicators

• Does the indicator measure what you want to measure?

• Does the indicator yield data that it is essential to know (rather than just nice to know)?

• Does the indicator yield data that is useful for programme planning and management?

• Is the indicator worth the time and effort to measure?

16

Dimensions of change in child rights programming

Source: Theis, Joachim (2003) Rights Based Monitoring and Evaluation: A Discussion Paper, Save the Children

17

Evaluation and rights based programming

• Must gather the views & opinions of children & young people whilst recognising the ‘child’s best interests’

• Should be designed with and involve children and young people whenever possible

• Must address the ‘Three Pillars’, as appropriate

18

Evaluating children and young people’s participation

1. Impact of children & young people’s participation• Impact on the stated objectives• Impact on children• Impact on adults, communities and

institutions

2. Quality of children & young people’s participation

3. Effectiveness of children & young people’s participation

19

Demonstrating a commitment to children’s participation• Transformed power relations between children and

adults• Raised awareness and developed skills in children’s

participation among children and adults• Children’s civil rights (information, expression and

association) actively promoted throughout the organisation

• Obstacles overcome and space increased for decision making by children at all levels of society and in all institutions

• Children and adults supported to claim children’s rights

20

Capacity building - some ideas1. Arrange training courses and workshops2. Organise mentoring support 3. Support development of organisational strategies,

policies & standards4. Support development of organisational systems and

procedures5. Arrange exchange visits & info exchange6. Produce and distribute case-studies, annotated

reading lists, guidelines, tools, manuals & practice standards

7. Facilitate collaborative working on common issues8. Develop a list of resource persons

21

Useful documents used in developing this presentation• Theis, Joachim (2003) Rights Based Monitoring and Evaluation:

A Discussion Paper Save the Children. Available online at www.mande.co.uk

• Kirby, Perpetua and Sara Bryson (2002) Measuring the Magic?: Evaluating and researching young people’s participation in public decision making, London: Carnegie Young People Initiative. Available online at www.carnegie-youth.org.uk

• Children and Participation: Research, monitoring and evaluation with children and young people. Available online at www.scfuk.org.uk