2. participatory monitoring and evaluation

Participatory Monitoring and Evaluation


1

Dominique Senn, seecon international gmbh


Find this presentation and more on www.sswm.info

Copy it, adapt it, use it – but acknowledge the source!CopyrightIncluded in the SSWM Toolbox are materials from various organisations and sources. Those materials are open source. Following the open-source concept for capacity building and non-profit use, copying and adapting is allowed provided proper acknowledgement of the source is made (see below). The publication of these materials in the SSWM Toolbox does not alter any existing copyrights. Material published in the SSWM Toolbox for the first time follows the same open-source concept, with all rights remaining with the original authors or producing organisations.

To view an official copy of the the Creative Commons Attribution Works 3.0 Unported License we build upon, visit http://creativecommons.org/licenses/by/3.0. This agreement officially states that:You are free to:

• Share - to copy, distribute and transmit this document • Remix - to adapt this document. We would appreciate receiving a copy of any changes that you have made to improve this

document. Under the following conditions:

• Attribution: You must always give the original authors or publishing agencies credit for the document or picture you are using.

DisclaimerThe contents of the SSWM Toolbox reflect the opinions of the respective authors and not necessarily the official opinion of the funding or supporting partner organisations. Depending on the initial situations and respective local circumstances, there is no guarantee that single measures described in the toolbox will make the local water and sanitation system more sustainable. The main aim of the SSWM Toolbox is to be a reference tool to provide ideas for improving the local water and sanitation situation in a sustainable manner. Results depend largely on the respective situation and the implementation and combination of the measures described. An in-depth analysis of respective advantages and disadvantages and the suitability of the measure is necessary in every single case. We do not assume any responsibility for and make no warranty with respect to the results that may be obtained from the use of the information provided.

Copyright & Disclaimer

http://www.sswm.info/

http://creativecommons.org/licenses/by/3.0



Contents

1. Introduction2. Participatory Monitoring and Evaluation3. Steps of a PM&E Process4. Techniques and Tools for PM&E5. References

3




Outlining the Terms – Participation (1/3)

4

1. Introduction

PARTICIPATION

There is no agreement among planners and managers about what participation exactly is and why it is important.

Possible definition: “Participation is a process through which stakeholders influence and share control over development initiatives and the decisions and resources which affect them.” (THE WORLD BANK 1996)

Through participation, we lost ‚control‘ of the project and in so

doing gained ownership and sustainability.

(World Bank Task Manager, Source: THE WORLD BANK 1996)





5

1. Introduction

Why participation?

For many governments, the United Nations Agencies and non-governmental organisations (NGOs), participation is considered to be critical to programme planning and poverty alleviation as a means to seek sustainability and equity.

(RIFKIN and KANGERE 2002)

PARTICIPATION

Source: http://www.drmdk.com/PIx/blackboard-green-clean.png [Accessed: 06.09.2013]

Tell me, and I will forget.Tell me, and I will forget.Show me, and I may remember.Show me, and I may remember.INVOLVEINVOLVE me, and I will me, and I will UNDERSTANDUNDERSTAND..


http://www.drmdk.com/PIx/blackboard-green-clean.png

http://www.google.ch/url?sa=i&source=images&cd=&cad=rja&docid=9wYWBeXJ8Ar1wM&tbnid=qKh6HuY_5onTAM:&ved=0CAgQjRwwAA&url=http://www.drmdk.com/LymeDisease/index_LymeDisease.html&ei=Z6EpUsmpFovZsga1s4CIAw&psig=AFQjCNEEJa0bZ4aAWe9IoJ2dDch5lYSkjg&ust=1378546407432492



Why participation? (RIFKIN and KANGERE 2002)

• People know what works for them and professionals need to learn from people.

• People make contributions of resources (money, materials, labour) for these programmes.

• People become committed to activities that they have helped develop.

• People can develop skills, knowledge and experience that will aid them in their future work.


6

1. Introduction

PARTICIPATION




Outlining the Terms – Monitoring (1/3)

What is monitoring?Monitoring is a continuous process of collecting and analysing information to compare how well a project, programme or policy is being implemented against expected results. (INTERNATIONAL FEDERATION OF RED CROSS AND RED CRESCENT SOCIETIES 2007)

7

1. Introduction

MONITORING





8

1. Introduction

Sources: http://upload.wikimedia.org/wikipedia/commons/f/fd/Abandoned_pit_latrine_(2940958117).jpg [Accessed: 06.09.2013];http://petecodella.com/diy-social-media-monitoring-10001789.htm [Accessed: 05.09.2013]http://www.wsscc.org/sites/default/files/imagecache/gallery_dl/photo_toilets3_south_africa.jpg [Accessed: 06.09.2013]

Actual state of a project / programme /

policy Expected Results

What is monitoring?

Compare


http://upload.wikimedia.org/wikipedia/commons/f/fd/Abandoned_pit_latrine_(2940958117).jpg

http://petecodella.com/diy-social-media-monitoring-10001789.htm

http://www.wsscc.org/sites/default/files/imagecache/gallery_dl/photo_toilets3_south_africa.jpg

http://www.google.ch/url?sa=i&rct=j&q=monitoring&source=images&cd=&cad=rja&docid=FWPMCsB_jD6UKM&tbnid=7R3JvSmE_m3ySM:&ved=0CAUQjRw&url=http://petecodella.com/diy-social-media-monitoring-10001789.htm&ei=_5ooUsfhKaK10wXh74GoDg&bvm=bv.51773540,d.ZGU&psig=AFQjCNGJamMy0Rzlcbda4Om7jVVdSNAVIA&ust=1378478882561760




Why monitoring?Monitoring aims at providing managers and major stakeholders with regular feedback and early indications of progress or lack thereof in the achievement of intended results.

9

1. Introduction

MONITORING

Monitoring generally involves collecting and analysing data on implementation processes, strategies and results, and recommending corrective measures. (INTERNATIONAL FEDERATION OF RED CROSS AND RED CRESCENT SOCIETIES 2007)

The three central elements of monitoring. Source: http://img.bhs4.com/8a/7/8a723596707a70e55cda39bd9d170b8da41d1597_large.jpg [Accessed: 06.09.2013]


http://img.bhs4.com/8a/7/8a723596707a70e55cda39bd9d170b8da41d1597_large.jpg



Outlining the Terms – Evaluation (1/2)What is evaluation?Evaluation is the systematic and objective assessment of an ongoing or completed project, programme or policy, its design, implementation and results. Evaluation determines the relevance and fulfilment of objectives, efficiency, effectiveness, impact and sustainability. (INTERNATIONAL FEDERATION OF RED CROSS AND RED CRESCENT SOCIETIES 2007)

10

1. Introduction

EVALUATION

Fulfillment of

objectives

Efficiency

Impact

Sustainability




Outlining the Terms – Evaluation (2/2)

Why evaluation?An evaluation should provide information that is credible and useful, so that lessons learnt can be includedinto the decision-making process. (INTERNATIONAL FEDERATION OF RED CROSS AND RED CRESCENT SOCIETIES 2007)

11

1. Introduction

EVALUATION

Source: http://www.clearhorizon.com.au/wp-content/uploads/2009/02/theproject25percent.jpg [Accessed: 06.09.2013]


http://www.clearhorizon.com.au/wp-content/uploads/2009/02/theproject25percent.jpg



12

2. Participatory Monitoring and Evaluation

PARTICIPATORY

MONITORING

AND EVALUATION

Putting the Pieces Together




Participation, Monitoring and Evaluation in the Project Cycle

13


EVALUATION

STAKEHOLDER

PARTICIPATION




Definition

Participatory monitoring and evaluation (PM&E) is a process through which :•Stakeholders at various levels •Engage in monitoring or evaluating a particular project, program or policy•Share control over the content, the process and the results of the monitoring and evaluation activity and •Engage in taking or identifying corrective actions.

(WORLD BANK 2010)

14





Stakeholder Groups

The following stakeholder groups are typically involved in PM&E:

• The end users of project goods and services, including both men and women at the community level

• Intermediary organisations, including NGOs• Private sector businesses involved in the project• Government staff at all levels.

(RIETBERGEN-McCRACKEN et al. 1998)

15





Conventional vs. Participatory Monitoring and Evaluation

16


Conventional •Outside experts measure

performance against pre-set indicators

•Use of standardised procedures and tools

Participatory (RIETBERGEN-McCRACKEN et al. 1998)

•Project stakeholders (including local people) are active participants, not just source of information

•Stakeholders evaluate, outsiders facilitate

•Focus on building stakeholder capacity for analysis and problem-solving

•Process builds commitment to implementing any recommended corrective actions




The 4 Steps of a PM&E Process

17

3. Steps of a PM&E Process




Step 1: Planning the Process (Adapted from AUBEL 2004)

The planning phase of a PM&E process includes:•Identification of stakeholder groups to be involved in the planning of the PM&E process.•Stakeholders define the objectives of the PM&E, what will be monitored, how and by whom. This step includes the identification of relevant indicators. In some cases, a common set of indicators is developed, while in other instances different stakeholder groups develop their own sets of indicators.This requires a lengthy process of negotiation, contestation and collaborative decision-marking among various stakeholders.

18





Step 2: Gathering Data (1/2)

19


Examples for quantitative methods are community surveys, interviews and observations.

Data on implementation processes, strategies and results can be collected using both quantitative and qualitative methods.

Conducting interviews in Togo. Source: SPUHLER (2007)

Community Survey in Vietnam. Source: http://www.cbm.org/I-can-serve-the-community-better-362538.php# [Accessed:

06.09.2013]


http://www.cbm.org/I-can-serve-the-community-better-362538.php



Step 2: Gathering Data (2/2)

20


Examples for qualitative methods are participatory learning methods using visual (e.g. rich pictures, locality mapping), interviewing and group tools (e.g. focus groups, world café) and exercices.

World café: A group discussion technique which facilitates social learning and consensus finding in relaxed and open discussions.Picture Source: http://farm2.static.flickr.com/1421/624509210_ae8751d3a3.jpg [Accessed: 07.09.2013]

Locality mapping: Illustration of the water, nutrition and sanitation situation in Mumbai, India, from the viewpoint of the inhabitants.Source: KROPAC (2009)


http://farm2.static.flickr.com/1421/624509210_ae8751d3a3.jpg



Step 3: Analysing Data

While data analysis is often thought of as a rather mechanical and expert-driven task, PM&E should be an opportunity to actively involve various categories of program stakeholders in the critical analysis of successes and constraints and the formulation of conclusions and lessons learned.

21


Gathered Data

Discussion of the results.Source: http://www.fauna-flora.org/wp-content/uploads/participatory-data-analysis-cover-photo.jpg [Accessed: 07.09.2013]

Formulation of

conclusions and lessons

learnt.


http://www.fauna-flora.org/wp-content/uploads/participatory-data-analysis-cover-photo.jpg

http://www.fauna-flora.org/wp-content/uploads/participatory-data-analysis-cover-photo.jpg



Step 4: Sharing the Information and Defining Actions to Be TakenNot all stakeholders can be involved in M&E data collection and analysis. In step 4, the results of preceding M&E activities are shared with other stakeholders, and there is discussion of appropriate actions to be taken based on the findings.

22





A participatory approach to monitoring and evaluation will usually make use of a number of techniques and tools, selected and combined to suit the objectives of the M&E work and the resources available. Many of the techniques associated with Participatory Rural Appraisal (PRA), Beneficiary Assessment (BA) and SARAR have been used in the context of monitoring or evaluation.

23

4. Techniques and Tools for PM&E




Participatory Rural Appraisal (PRA) (1/2)PRA is an approach used in the field of international development. It aims to incorporate the knowledge and opinions of rural people in the planning and management of projects and programmes. PRA uses a large variety of techniques, including:•Group dynamics (e.g. learning contracts, role reversals, feedback sessions)•Sampling (e.g. transect walks, wealth ranking, social mapping)•Interviewing (e.g. focus group discussions, semi-structured interviews, triangulation)•Visualisation (e.g. Venn diagrams, matrix scoring, timelines)

24





Participatory Rural Appraisal (PRA) (2/2)

25


Social mapping: Maps usually drawn by a group of villagers either on the ground using chalk or on a large sheet of paper. The maps provide information about the physical characteristics of the community, about the socio-economic conditions and how the participants perceive their community.(http://web.worldbank.org/WBSITE/EXTERNAL/TOPICS/EXTPOVERTY/EXTISPMA/0,,contentMDK:20190393~menuPK:415131~pagePK:148956~piPK:216618~theSitePK:384329~isCURL:Y,00.html [Accessed: 07.09.2013])

Picture Source: http://youthdevelopmentvoice.files.wordpress.com/2011/09/pra1.jpg [Accessed: 07.09.2013]

Transect walk: Involves walking with the villagers through an area and discussing about different aspects of land-use and agro-ecological zones in the village observed during the walk.(MUKHERJEE 1993)

Picture Source: http://www.espa.ac.uk/october-2012-newsletter [Accessed: 07.09.2013]


http://web.worldbank.org/WBSITE/EXTERNAL/TOPICS/EXTPOVERTY/EXTISPMA/0,,contentMDK:20190393~menuPK:415131~pagePK:148956~piPK:216618~theSitePK:384329~isCURL:Y,00.html



http://youthdevelopmentvoice.files.wordpress.com/2011/09/pra1.jpg

http://www.espa.ac.uk/october-2012-newsletter



Beneficiary AssessmentBeneficiary assessment is a qualitative research tool used to improve the impact of development operations by gaining the views of intended beneficiaries regarding a planned or ongoing intervention. (SALMEN 1999)

The tools usually used are conversational interviewing and focus group discussions on changes and impacts.

26


Focus group discussion with water kiosk users and vendors in Lusaka, Zambia.Source: SENN (2012)




SARAR (1/2)SARAR is an education/training methodology for working with stakeholders at different levels to engage their creative capacities in planning, problem solving and evaluation. The acronym SARAR stands for the five attributes and capacities that are considered the minimum essentials for participation to be a dynamic and self-sustaining process.

27





SARAR (2/2)

28


S Self-esteem A sense of self-worth as a person as well as a valuable resource for development

A Associative strength

The capacity to define and work toward a common vision through mutual respect, trust, and collaborative effort

R Resourcefulness

The capacity to visualise new solutions to problems even against the odds, and the willingness to be challenged and take risks

A Action planning

Combining critical thinking and creativity to come up with new, effective, and reality-based plans in which each participant has a useful and fulfilling role

R ResponsibilityFor follow-through until the commitments made are fully discharged and the hoped-for benefits achieved




Further M&E Techniques• Visual self-evaluation tools• Testimonials• Photographing the evidence• Community records and indicators

29





30

AUBEL, J. (2004): Strategic Report 9, Participatory Monitoring & Evaluation for Hygiene Improvement, Beyond the Toolbox: What else is Required for Effective PM&E? A Literature Review. Washington, D.C.: Office of Health, Infectious Diseases and Nutrition, Bureau for Global Health U.S. Agency for International Development. URL: http://www.ehproject.org/PDF/Strategic_papers/SR-9%20Lit%20Rev.pdf [Accessed: 06.09.2013]

INTERNATIONAL FEDERATION OF RED CROSS AND RED CRESCENT SOCIETIES (Editor) (2007): Monitoring and Evaluation in a Nutshell. International Federation of Red Cross and Red Crescent Societies.

MUKHERJEE, N. (1993): Participatory Rural Appraisal. Methodology and Applications. New Delhi: Ashok Kumar Mittal.

RIETBERGEN-McCRACKEN, J.; NARAYAN, D. ; WORLD BANK (Editor) (1998): Participation and Social Assessment: Tools and Techniques. Washington: World Bank. URL: http://www.rmportal.net/library/content/tools/biodiversity-conservation-tools/putting-conservation-in-context-cd/participatory-approaches-resources/1-c.pdf/at_download/file [Accessed: 06.09.2013]

RIFKIN, S.B., KANGERE, M. (2002): What Is Participation? In: HARTLEY, S. (Editor) (2002): CBR – A Participatory Strategy in Africa. Based on the Proceedings of a Conference Uganda, September 2001. London: University College London and Centre for International Child Health. URL: http://participatory-pages.yolasite.com/resources/Community%20Participation%20basics%5B1%5D.pdf [Accessed: 06.09.2013]

SALMEN, L.F. (1999): Beneficiary Assessment Manual for Social Funds. Washington: The World Bank. URL: http://www-wds.worldbank.org/servlet/WDSContentServer/WDSP/IB/2000/11/30/000094946_00111505375712/Rendered/PDF/multi_page.pdf [Accessed: 07.09.2013]

THE WORLD BANK (Editor) (1996): The World Bank Participation Source Book. Washington: World Bank. URL: http://www.betuco.be/voorlichting/Participation%20Sourcebook%20-%20World%20Bank.pdf [Accessed: 06.09.2013]

THE WORLD BANK (Editor) (2010): Participatory Monitoring and Evaluation, in Topics: Participation and Civic Engagement. Washington D.C.: The World Bank. URL: http://web.worldbank.org/WBSITE/EXTERNAL/TOPICS/EXTSOCIALDEVELOPMENT/EXTPCENG/0,,contentMDK:20509352~menuPK:1278203~pagePK:148956~piPK:216618~theSitePK:410306,00.html [Accessed 06.09.2013]

6. References


Participatory Monitoring and Evaluation 31

“Linking up Sustainable Sanitation, Water Management & Agriculture”

SSWM is an initiative supported by:

Created by:

2. participatory monitoring and evaluation

Documents