guide for monitoring evaluation community based projects

8/19/2019 Guide for Monitoring Evaluation Community Based Projects

1/85

On Target

A Guide for

Monitoringand Evaluating

Community-Based

Projects


2/85

© Julie Smith


3/85


4/85

Published by the United Nations Educational, Scientific and Cultural Organisation7, place de Fontenoy, 75352 Paris 07 SP, France

© UNESCO 2009All rights reserved

The designations employed and the presentation of material throughout this publication do notimply the expression of any opinion whatsoever on the part of UNESCO concerning the legalstatus of any country, territory, city or area or of its authorities, or concerning the delimitation ofits frontiers or boundaries.

The author is responsible for the choice and the presentation of the facts contained in this workand for the opinions expressed therein, which are not necessarily those of UNESCO and do notcommit the Organisation.

Graphic Design by UNESCO

Printed in France

ED-2009/WS/12 – cld 263.9


5/85

3

Contents

Acknowledgements ................................................................ 4

Preface ................................................................................... 5

Acronyms .............................................................................. 6

Executive Summary .................................................................. 7

Section 1 Introduction ........................................................... 9

Section 2 Steps to setting up and planning the project

and its evaluation ................................................. 1 7Section 3 The programme logic model. What is it? ................. 25

Section 4 How to create a logic model ................................. 39

Section 5 Structuring and defi ning your evaluation ................... 45

Section 6 Research techniques and tools ............................... 53

Section 7 Participatory evaluation .......................................... 6 1

Section 8 Data analysis and dissemination .............................. 65

Section 9 References, Annotated Bibliography, Annexes ........ 69


6/85

4

Acknowledgements

This guide was produced under the leadership of UNESCO’s Division for the Promotion of BasicEducation. The project was managed and supervised by Mehboob Dada, Programme Coordinator.

This publication draws on the work and support of a vast number of individuals and organizations whohave been involved in the UNESCO initiative “Another Way to Learn…” In the first instance, the projectteam wishes to acknowledge the valuable assistance of Rhiannon Barker who drafted the document,designed to support the projects and the team in research and evaluation of a series of programmeswithin the framework of Another Way to Learn.

UNESCO wishes to express its gratitude to NGO project partners, who cooperated in providing UNESCOwith feedback for the development and support of this publication, as well as on the experiences andlessons learned. Without the cooperation of civil society partners the manual would not have beenpossible.

In particular, UNESCO wishes to acknowledge and thank: Sophie Jadin and Mao Kosal (Phare PonleuSelpak - Cambodia), Shanthi Ranganathan (TT Ranganathan Clinical Research Foundation - India),Dadi Pudumjee, Sanjoy Roy and Javita Narang (Ishara Puppet Theatre Trust and Salaam Balaak Trust -India), Rodney Grant and Sophia Greaves (Pinelands Creative Workshop - Barbados), Joseph Meharris(The Centre of Hope - Trinidad), Dulce Almonte and Jaime de La Rosa (Red Dominicana de Personasque Viven con VIH/SIDA (REDOVIH) - Dominican Republic), Susana Fergusson (PROCREAR - Colombia),Raquel Barros (Lua Nova - Brazil), José Carlos de Freitas Spinola, Cido Martins and Patricia Moura(Reciclazaro - Brazil).

UNESCO also wishes to thank particularly Carolina Cano and Mariana Kitsiona for their valuablecontributions and comments during the preparation of this guide, and other colleagues in UNESCO’sEducation Sector, at Headquarters, and in field offices responsible for the various countries concerned

who reviewed various drafts and provided their continued support to the successful implementation ofthese projects. UNESCO expresses its gratitude to Rosalind David for all her support and comments.

UNESCO would like to acknowledge and thank particularly Julie Smith for her cartoons, whichhumorously highlight the spirit and dedication required in assuring we learn and address key issuesof development concern.

It should be noted that information provided on the Project Logic Model draws heavily on “ProgrammeDevelopment and Evaluation”, a programme developed by the University of Wisconsin. Theirexcellent work is widely accessible on the World Wide Web (www.uwex.edu/ces/pdande/evaluation/evallogicmodel.html) (Copyright 1996 Board of Regents of the University of Wisconsin System, d/b/aDivision of Cooperative Extension of the University of Wisconsin-Extension.)

Finally, UNESCO acknowledges and thanks the European Commission and UNAIDS for the financialsupport they have been providing to UNESCO, which was essential to the development of this guide.

“The real voyage of discovery consists not in seeing new landscapes,but in having new eyes.”

Marcel Proust

http://www.uwex.edu/ces/pdande/evaluation/evallogicmodel.htmlhttp://www.uwex.edu/ces/pdande/evaluation/evallogicmodel.htmlhttp://www.uwex.edu/ces/pdande/evaluation/evallogicmodel.htmlhttp://www.uwex.edu/ces/pdande/evaluation/evallogicmodel.html


7/85

5

Preface

The value and importance of good quality monitoring and evaluation (M&E) is increasingly recognisedby a wide range of stakeholders; planners, funders, policy makers and communities who are supportinginterventions. Demonstrating the extent to which a project has been able to meet its planned objectiveshelps ensure that resources are used as effectively, efficiently and appropriately as possible. It is ourintention that this monitoring and evaluation guide will give those with limited experience increasedconfidence and skills to tackle the whole of the project cycle both in terms of project planning andrelated M&E activities. The guide looks at:

• The value of conducting a needs assessment and ensuring that project goals and vision are clearlylinked to identified needs.

• Ways of monitoring and evaluating projects both during project implementation (process evaluation)and at the stage of project completion (outcome/impact evaluation).

• The value of planning projects within a clearly constructed model or framework. The programmelogic model is put forward as an example of this.

• How to ensure information emerging from M&E is fed back into ongoing project design and futureplanning.

• How to ensure the stakeholders are active participants in all project related design, planning andresearch.

This publication has grown out of work supporting a range of vocational and entrepreneurial skillsdevelopment projects undertaken through non-formal education in Africa, South Asia, the Caribbeanand Latin America. Central to all of the projects within this programme are the creative and innovative

methods used to communicate in a meaningful way, to engage people and to encourage participation.The projects all focus on capacity building, empowerment and creating learning opportunities.UNESCO’s conceptualisation of these projects is based on the four pillars of education presented in1996 by the Organization’s Task Force on Education for the Twenty-first Century: “learning to know”,“learning to do”, “learning to live together” and “learning to be”. The Dakar Framework for Action(2000), which expresses the collective commitment of the international community to Education forAll (EFA), also reminds us that education should be geared to tapping into each individual’s talentsand potential.

The projects are all located in areas of considerable financial deprivation, where infrastructuralsupport is limited and communities and individuals find it hard to escape from the downward spiralof poverty and social exclusion. However, what characterises all these projects is their capacity forinnovation and their motivation and courage to move forward, constantly striving to reflect on and

learn from their experiences. It is our hope that this guide will help provide support and techniques toformalise and entrench the monitoring and evaluation process, enabling community-based projectsboth to demonstrate success and improve the effectiveness of interventions.

UNESCODivision for the Promotion of Basic EducationSection for Secondary, Science,Technical and Vocational Education


8/85

6

Acronyms

AIDS Acquired Immune Deficiency Syndrome

CBO Community Based Organization

EFA Education for All

EU European Union

HIV Human Immunodeficiency Virus

IDS Institute of Development Studies

M&E Monitoring and Evaluation

NGO Non governmental organization

PM&E Participatory monitoring and evaluation

PRA Participatory Rural Appraisal

SMART Specific, Measurable, Appropriate, Realistic, Time-Bound

UNAIDS Joint United Nations Programme on HIV and AIDS

UNESCO United Nations Educational, Scientific and Cultural Organization


9/85

7

Executive Summary

This guide has been developed as a “work in progress” resource that can be used to support themonitoring and evaluation of community development initiatives. It is hoped that the guide will helpproject staff to see that when thorough planning, monitoring and evaluation is undertaken and thecommunities are fully involved in this process, the quality and appropriateness of the end productis enhanced. Whilst this guide has been developed based on the experience of and lessons learnedfrom a specific programme of projects focused on sustainable livelihoods and aspects of healtheducation, much of the content is generic and will thus be applicable to a wide range of communityinitiatives. The key aims of the guide are to:

• Provide an overview of key aspects of quality research which can be used by project staff toconduct their own internal evaluations

• Present recommended models, frameworks and theoretical concepts that can be used inprojects for planning and carrying out research.

• Provide practical guidance on the use of the programme logic model and look at ways it canbe adapted and used within projects

• Provide an overview of basic research tools and methods, focusing particularly on qualitativeresearch methods

• Promote the use of participatory research techniques

“To be surprised, to wonder, is to begin to understand.”

José Ortega y Gasset


10/85

9

Section 1 Introduction

1

1.1 Bakground 10

1.2 How to use this guide 10

1.3 Who is the guide for? 10

Putting research in context 101.4 Why do we need to evaluate? 10

1.5 What are the key issues in evaluatingcommunity initiatives? 1 1

1.6 Participatory techniques 13

1.7 What should evaluation hope to achieve? 14

1.8 Why are frameworks and models helpful? 14

1.9 What characterises a successful evaluation? 14


11/85

10

1.3 Who is the guide for?This guide is intended primarily to be usedby those setting up, planning or workingwith community development projects. TheUNESCO supported projects, which werethe driving force behind the creation of thispublication, aimed to raise the quality of life ofmarginalised populations through education,poverty reduction and building sustainable

livelihoods. The lessons learned from thiswork have application for a wider range ofprojects and settings.

Putting researchin context1.4 Why do we need

to monitor andevaluate?

First, a definition of the terms monitoring andevaluation: For the purposes of this guidemonitoring relates to the ongoing review andcollection of data, which will help to determinewhether anticipated outcomes are being met.Monitoring is an essential part of evaluationand data collected feeds into the moreoverarching process of evaluation. Evaluation

thus refers to the systematic collection ofinformation conducted during or after aproject in order to make judgments abouteffectiveness against anticipated outcomesand to help inform decisions about futureinterventions.

Monitoring and evaluation efforts assistin determining whether a programme hasachieved its intended outcomes, so that we canbe accountable for the money we spend andmake the most effective use of our resources.

Although evaluations have traditionally beenrequired by funders to ensure that money iswell spent, the challenge remains to ensure

1.1 BackgroundThis guide was produced following theevaluation of a range of vocational andentrepreneurial skills developmentprojects undertaken through non-formaleducation that were supported by UNESCO4.A key finding of this assessment wasthat projects would benefit both from aclearer understanding of the advantages

of conducting their own research andfrom being supported and encouraged toproduce a research “framework”, designedat the outset of the project, and appliedthroughout the course of project life. Areview of the research components of allof the projects comprised concluded thatresearch should be an integral part of projectdesign and that more training and expertisewas needed by most projects in relation todeveloping research design and technique.

The use of frameworks (such as theprogramme logic model) helps to provide step-by-step guidance on project planning. In sodoing it not only helps clarify project activitiesand the thinking and assumptions on whichprojects are based, but also facilitates andenhances the evaluation process.

1.2 How to use this

guideThis guide is not intended as a definitivemanual to monitoring and evaluation – more asa starting point with introductory informationfor further advice and support. It is anticipatedthat each section can be printed or photocopiedfor wider distribution, as required.

4 See “Another Way to Learn… Case studies” summarises

the 17 projects that make up the programme, available athttp://unesdoc.unesco.org/images/0015/001518/151825e.pdf (Also available in Spanish) (accessed January 2008)

“Appreciation is like looking through a wide-angle lensthat lets you see the entire forest, not just the one tree limb you walked upon.”

Doc Childre

http://unesdoc.unesco.org/images/0015/001518/151825ehttp://unesdoc.unesco.org/images/0015/001518/151825e


12/85

11

11.5 What are the keyissues in evaluatingcommunity initiatives?Community initiatives are not easy to evaluate!There is no straightforward, conclusivetechnique that can be taken from a text book

and adapted for use on each project site.Project staff need to be innovative, to applycommon sense and use their knowledge of thesupporting population, the environment, thepolitical and cultural context, to ensure thatthe right questions are asked in the right way.

Community initiatives tend to be made up ofcomplex multiple interventions, taking place atmany different levels, often designed to bringabout different outcomes. There is typically amix of strategies, aimed at both individual and

community level. The initiatives employed rangefrom trying to change individual behaviour

that both project staff and the supportingpopulation recognise the value of ongoingmonitoring and evaluation and use it toimprove the effectiveness and quality of theirprogrammes.

Monitoring should be carried out on anongoing basis to ensure that the aims andobjectives of the project are being met andto readjust programming based on lessonslearned to date. Internal evaluations areimportant not only to measure effectiveness,efficiency and project progress but also tohelp develop project ownership on the part ofboth project staff and the beneficiary groups.External evaluation, whilst costly, and mostof the time therefore prohibitive to smallcommunity ventures, nonetheless offers adegree of independent scrutiny which may, on

occasion, be appropriate.

“Men occasionally stumble over the truth,but most of them pick themselves up

and hurry off as if nothing ever happened.”

Winston Churchill

“When we try to pick anything out

by itself, we fi nd it hitched to everythingelse in the universe.”

John Muir


13/85

12

through education and empowerment, tobroader-based projects focused on equity,social justice and intersectoral interventions.The use of theoretical frameworks and modelshas sought to provide greater rigour regardingthe way in which different variables arecollected and interpreted. At the same time,new thinking around evaluation increasinglyrecognises the multiplicity of interactions thattake place within most community settingsand challenges the more simplistic linearrelationship between cause and effect. Giventhe complex way in which most interventionsare conducted, the questions that have tobe asked to find out whether or not theseinterventions are effective need to be wellconstructed and tested. It is important tobe aware of the fact that outcomes are notalways anticipated and may be beneficial or

detrimental to the community. Those involvedin project monitoring and evaluation must havethe courage and conviction to highlight bothpositive and negative project outcomes.

There are many different techniques andmethods used to evaluate programmes. Itis this diversity that makes evaluation sucha powerful tool. Methods can depend on thequestions of interest, on the context, on theevaluator’s own philosophical position and onthe characteristics of the other people involved

in the programme. Evaluators can call on anycombination of stances to add depth and qualityto the evaluation. Sophisticated and costlydesigns can be a waste of resources if thequestion can be answered by simpler means.

Evaluators dealing with overly-complicatedtools that they do not understand will notproduce results that are useful and valid.

Today, most people accept that bothquantitative and qualitative approaches play avaluable role in evaluating programmes and itis often suggested that ‘pluralistic evaluation’,i.e. employing a range of both quantitativeand qualitative evaluation methods, providesthe most appropriate strategy for addressing

complex issues about what counts as asuccessful or a good outcome (Beattie,1995).

The evaluation of most communityinterventions will benefit from employing arange of research techniques to evaluate theproject at distinct stages. These stages are:

• the formation of programme aims,objectives and procedures – including initialneeds assessment (formative evaluation)• the process of programme implementation

(process evaluation)• the measurement of outcomes and impact(outcome or summative evaluation)

“Simplicity is the ultimate sophistication.”

Leonardo da Vinci

Concerning monitoring and evaluation of health promotion projects, a recent publication from aWHO European Working Group recommends that policymakers:

– Encourage the adoption of participatory approaches to evaluation that provide meaningfulopportunities for involvement.– Require that a minimum of 10% of the total financial resources for a health promotion initiative

be allocated to evaluation.– Ensure that a mixture of process and outcome information is used to evaluate all health promotioninitiatives.– Support the use of multiple methods to evaluate health promotion initiatives.– Support further research into the development of suitable approaches to evaluate healthpromotion initiatives.– Support the establishment of a training and education infrastructure to develop expertise in theevaluation of health promotion initiatives.– Create and support opportunities for sharing information on evaluation methods used in healthpromotion through conferences, workshops, networks and other means.

WHO (2006) Evaluation in health promotion. Principles and perspectives Edited by Rootman, I.; Goodstadt,

M; Hyndman, B; McQueen; Potvin, L; Springett, J. & Ziglio, E. WHO Regional Office for Europe , http://www.euro.who.int/eprise/main/WHO/InformationSources/Publications/Catalogue/20040130_1 (accessed January

2008)

http://www/http://www/


14/85

13

1Participatory monitoring and evaluation(PM&E) is an approach which involves localpeople, development agencies and policymakers deciding together how progressshould be measured and results acted upon(IDS, 1998). It is an increasingly popularmethodology, not only because it is costeffective, making use of local skills andresources, but also because it forces peopleto examine their assumptions about whatconstitutes progress, facing up to the conflictsand contradictions that can emerge. Theresulting data and analysis is felt to be morein tune with the views and aspirations of thosedirectly affected.

“PM&E is not just a matter of usingparticipatory techniques within a conventionalmonitoring and evaluation setting. It is

about radically rethinking who initiates andundertakes the process, and who learns orbenefits from the findings.” (IDS, 1998, p.2)

Examples of participatory techniques are givenin section 7.

1.6 Participatorytechniques

As a general rule, small scale communityprojects are working on low budgets withlimited capacity in terms of staff time andskills. Quantitative research, employingquestionnaire surveys and large scale datacollection, tends to be inappropriate in thistype of scenario. An alternative researchparadigm is to involve the beneficiaries associal actors in their own development.In other words, those who “benefit” fromthe project should be actively involvedin monitoring the project (participatorymonitoring) and evaluating the impact ofthe project (participatory evaluation). Thepeople with whom the project is working

can be regularly involved in the assessmentof factors such as behavioural change andempowerment in their communities. Throughcareful facilitation, participants themselveswill develop the “yardsticks” to assess howchange has taken place. With help from projectstaff they can develop “baselines” and then, ona regular basis, look at what has changed as aresult of interventions.


15/85

14

1.7 What shouldevaluation hope toachieve?An evaluation of a community-basedintervention aims to:

• Design the process for obtaining therequired information using a variety ofmethods

• Encourage the development of clearindicators, milestones and initiatives

• Collect and analyse data throughout theproject cycle related to desired objectives andoutcomes

• Determine what went wrong, if some or allof the objectives were not met

• Make any necessary adjustments based onexperience and lessons learned

• Provide feedback to everyone participatingin the programme and to local communities

• Demonstrate outcomes and lessonslearned to funding organisations and serviceproviders.

• Use the final results to inform the planningand implementation of future work in thetarget communities

1.8 Why areframeworks andmodels helpful?

When planning a programme, it can beuseful to work within a structured modelor framework. Frameworks are basicallyplanning tools, which can be used to helpshape the thinking and structure behindproject planning and evaluation. Theyhelp create and clarify project theories,assumptions and activities - thus ensuringthat the evaluation process is more effective,transparent and straightforward.

1.9 What characterisesa successfulevaluation?Successful evaluations will demonstrate thefollowing:

• Clear programme objectives, targets andtime-frame (section 3).

• Participation of project “beneficiaries” inproject planning, monitoring and evaluation(section 7).

• Shared understanding and ownership ofproject objectives and how these are to beachieved (section 3.2) by stakeholders andpartners.

• Manageable and realistic data collectionand analysis. The more complicated the toolsand methods employed, the more likely theyare to fail (sections 5 and 6).

• Harmonised data collection tools andinstruments with other systems in place(section 6.8).

• Adequate financial and human resourcesto carry out the required levels of monitoring

and evaluation. Where technical capacity is notadequate, training and technical assistanceneed to be part of the programme design(section 6).

• Relevance and transparency. Monitoringof programmes needs to be conducted in atransparent way and data should be locallydriven and locally owned (sections 4-7).

• Appropriate feedback loops to ensureresults inform future planning processes andprojects (section 6).

• Monitoring and evaluation should beculturally appropriate and pass ethicalstandards established in local and nationalguidance (section 6).


16/85

15

Notes:

......................................................................................................................................................................

......................................................................................................................................................................

......................................................................................................................................................................

......................................................................................................................................................................

......................................................................................................................................................................

......................................................................................................................................................................

......................................................................................................................................................................

......................................................................................................................................................................

......................................................................................................................................................................

......................................................................................................................................................................

......................................................................................................................................................................

......................................................................................................................................................................

......................................................................................................................................................................

......................................................................................................................................................................

......................................................................................................................................................................

......................................................................................................................................................................

......................................................................................................................................................................

......................................................................................................................................................................


17/85

16

......................................................................................................................................................................

......................................................................................................................................................................

......................................................................................................................................................................

......................................................................................................................................................................

......................................................................................................................................................................

......................................................................................................................................................................

......................................................................................................................................................................

......................................................................................................................................................................

......................................................................................................................................................................

......................................................................................................................................................................

......................................................................................................................................................................

......................................................................................................................................................................

......................................................................................................................................................................

......................................................................................................................................................................

......................................................................................................................................................................

......................................................................................................................................................................

......................................................................................................................................................................

......................................................................................................................................................................

......................................................................................................................................................................

......................................................................................................................................................................


18/85

17

Section 2 Steps tosetting up and planning

the project and itsevaluation

2

2.1 The Project Cycle 1 8

2.2 Writing the project proposal and securing funding 1 8

2.3 Carrying out a needs assessment 1 9

2.4 Developing a research framework 20


19/85

18

2.1 The ProjectPlanning Cycle

It is useful to think of the stages in a projectas being part of a cycle, where each stage inthe cycle impacts and influences subsequentstages. This is represented in the figure below.

The Project Cycle

When thinking about monitoring andevaluating a project, it is crucial that there iscommitment to feeding results back into theproject cycle and using the learning to shape,adapt and improve project work in the future.Feedback and learning should occur all theway through the project. This is represented bymulti-directional arrows in the diagram above.

2.2 Writingthe project proposaland securing funding

The first step in any project, having identifiedkey aims and objectives, is to write a proposalto secure funding. This guide does not set outto provide a comprehensive list of potentialsources of funding. However, as a first stepit may be worth browsing the internet andagency websites as well as consulting keyplayers in local government, communitygroups, national and international agencies. Insome cases, it may be appropriate to securefunding for the needs assessment prior toapplying for a larger pot of money for theactual intervention.

Feeding backlearning to improve

projecteffectiveness

Implementingand monitoring

the project

Evaluatingthe project

Planningthe project

Assessingneeds


20/85

19

2

2.3 Carrying outa needs assessment

Typically, if it has not already been done,a needs assessment is used to review theprogramme’s aims and objectives andproposed methods of working prior to projectimplementation. In planning the needsassessment, care is required to includedifferent stakeholders and different interestgroups within the community. As far aspossible, care should be taken to ensurethat the views of the most vocal, or morepowerful, members of the community will notdominate the discussion. Needs assessmentsalso provide an early opportunity to involvethe local community through a process ofconsultation.

A needs assessment should achieve thefollowing:• Enable a programme’s aims and objectivesto be specified more clearly.• Ensure the programme addressesthe issues and priorities identified by thecommunity itself.• Provide an initial check that issues thoughtto be important by outsiders reflect thepriorities of the community.• Determine which problems/issues

identified by a group within the communityrepresent wider concerns.• Provide information on the majorstakeholders in the community.• Indicate the extent to which the communityconcerned may be mobilised and a communitydevelopment approach, if appropriate,be adopted. In doing this, it is importantto assess community strengths. If the

community is cohesive, displaying high levelsof integration and involvement, pre-existingnetworks will be worth tapping into.• Check on perceptions, interpretationsand acceptability of promotion materials,messages, or other interventions.• Test the appropriateness ofimplementation approaches and procedures.

At this stage, a priority is to create a baselinefrom which change can be measured.Qualitative methods to gather data about thecommunity might include:

• in-depth interviews with key informants inthe target community, taking care to identifyand include leaders in any interest groups orrival factions. Interviews may take the form ofbaseline videos which beneficiaries may find

enjoyable both to make and participate in• focus group discussions withprofessionals working in the community andwith local people. These may be drawn fromlocal clubs, associations, institutions e.g.schools, community centres, women’s groups,farmers gathering at a market, etc.• inviting participation from members ofthe public e.g. using the media, mounting acitizen’s forum or public debate about theprogramme’s aims and objectives• participant observation at local events

and groups, talking to people informally, andrecording observations in field notes• systematic collection of the views oflarger cross-sections of people using semi-structured questionnaires which containopen questions allowing individuals to expressthemselves freely

(See also Section 7)


21/85

20

2.4 Developinga researchframeworkThis manual guide encourages monitoring andevaluation to be planned and initiated from theproject’s outset.4

Use of the logic model framework (outlined insection 3) is encouraged to help plan and carryout the next steps. The key stages involved are:

i. Identify Outcomes

From the project’s aims and objectives identifyclear short-term and long-term outcomes.Remember when doing this to be realistic. Intrying to secure funding it is tempting to be

over-ambitious about what can be achieved –this can result in perceptions of failure later,whilst much positive and beneficial work ismissed because inappropriate targets havebeen set (see section 5.6).

4 A more complete range of evaluation techniques isprovided in Annex 2.

ii. Indicators

For each of these outcomes, identifyappropriate indicators which can be used todemonstrate whether or not these outcomeshave been met (see section 5.6).

iii. Evaluation methods?

Identify strategies to ensure that the project issubject to continuous evaluation and scrutiny(see sections 5 and 6).

iv. Who will conduct theresearch?

Decide who will conduct the research – itmay be appropriate that different groups of

people carry out different bits of research, i.e.project staff may be responsible for one aspectof the work and members of the communityanother. Think about how the community willbe involved in the analysis.

v. Research tools

Identify appropriate research tools whichcan be used to collect necessary data (seesection 6).


22/85

21

2

vi. Evaluation Checklist

Go through the evaluation checklist, foundat the end of this section, ensuring that eachpoint has been covered.

vii. Prioritise

Prioritise and be realistic. Make sure thatwhatever you do has a value and a purposeand that the reasons for the research areclearly understood by project staff.


23/85

22

Ethics? Are you treating people with respect?

– Has an appropriate ethical framework been developed? This may include considerations ofprotecting the confidentiality and anonymity of responses.

– Who considered the ethical dimensions of the project and how ethical policies would be

implemented?

Are you being realistic?

– Are adequate resources available for the evaluation?

– Are the evaluators fully trained and competent in the techniques they are being asked to carryout? (This may be particularly important with internal evaluations – i.e. when project staff arebeing asked to carry out much of the evaluation themselves.)

– Do stakeholders agree that the resources earmarked for the evaluation are proportional to theoverall size of the project? i.e. +/- 10% of the total budget for the project.

– Do all stakeholders share similar views of what the evaluation aims to achieve?

– What has been done to ensure that funders and practitioners have a realistic view of thecommunities they are working with and their capacity to absorb and engage with differentkinds of evaluation activities?

– What has been done to ensure that the proposed project outcomes, and the timescale withinwhich changes might be expected, are compatible?

– Is the project goal/vision compatible with the anticipated outcomes?

– Has the evaluation taken into account the historical, political and social (local and national)context in which the programme is taking place?

– Does the evaluation address the theoretical assumptions on which the project is based?

– What are the political forces at play? How much political power is being wielded?

Summary: Evaluation good practice checklist

Why Bother?

– What is the purpose of the evaluation?

– Who is it serving?

– Does the design of the evaluation support the initiative taking place on the ground?

– Does the research generate new knowledge?

– Is the evaluation being used as a tool to help empower the individuals and communities itserves?


24/85

23

2

Dissemination

– How can evaluators be encouraged to report more fully on their evaluation activities?

– Will the findings be conveyed to all participants and stakeholders in meaningful, timely and

appropriate ways?

– Will the results show the failings as well as the achievements of the project?

Methods

– Is the evaluation participatory?

– Are the objectives and monitoring indicators SMART (Specific, Measurable, Appropriate,Realistic, Time-Bound)?

– Has the planning of the evaluation been treated as equally important as the data collection?

– Is the evaluation integrated into all stages of development and implementation?

– Is the chosen methodology the most appropriate in light of the project interventions? Be clearwhy!

– How is the quality of practice/application of methods ensured (different questions need to beasked if the research is commissioned or carried out internally)?

– Is the timeframe of the evaluation sufficient to measure the potential outcomes (some of whichmay be very long-term)?

– Are the methods used flexible enough to monitor unintended outcomes?


25/85

24

Notes:

......................................................................................................................................................................

......................................................................................................................................................................

......................................................................................................................................................................

......................................................................................................................................................................

......................................................................................................................................................................

......................................................................................................................................................................

......................................................................................................................................................................

......................................................................................................................................................................

......................................................................................................................................................................

......................................................................................................................................................................

......................................................................................................................................................................

......................................................................................................................................................................

......................................................................................................................................................................

......................................................................................................................................................................

......................................................................................................................................................................

......................................................................................................................................................................

......................................................................................................................................................................

......................................................................................................................................................................


26/85

25

Section 3 The programmelogic model3. What is it? 3

3.1 What is the programme logic model? 26

3.2 What are the benefi ts of using a programmelogic model? 27

3.3 What do programme logic models look like? 28

3.4 Familiarising yourself with terminology 30

3.41 Project vision/goal 30

3.42 Inputs 3 1

3.43 Outputs 3 1

3.44 Outcomes 32

3.45 Looking at your assumptions (theory) 33

3.46 Linkages – theory of action 33

3.47 External factors 34

3.48 Explaining the difference between outputsand outcomes 34

3.49 What is programme theory? 35

3.410 Example of linkages from a drug educationprogramme 35

3 N.B . the following material draws heavily on “Programme Development and Evaluation” a programme

developed by the University of Wisconsin providing an excellent interactive internet course on theprogramme logic model

( www.uwex.edu/ces/pdande/evaluation/evallogicmodel.html accessed January 2008 ).

http://www.uwex.edu/ces/pdande/evaluation/evallogicmodel.htmlaccessedJanuary2008http://www.uwex.edu/ces/pdande/evaluation/evallogicmodel.htmlaccessedJanuary2008


27/85

26

Diagram adapted from: http://www.uwex.edu/ces/pdande/evaluation/evallogicmodel.html (accessed January 2008)

3.1 What areprogramme logicmodels?Logic models have been described in anumber of ways, using a variety of names:

“a logic framework” “a log frame” “a planning tool”“a framework to help explain and record how

the programme works and the theories andassumptions underlying the work” “a sensible model showing how the project issupposed to work” “a model to show the logical relationshipsbetween the different components of theproject - it helps to provide a simplified picture

of the complete intervention”

The model links outcomes (short- and long-term) with programme activities and processesand the theoretical assumptions underlyingthe programme. The model provides aplanning framework, highlighting how theprogramme is expected to develop and in whatorder to stage activities, and looks at howdesired outcomes are achieved. It includesan analysis of the inputs necessary to get theproject up and running and also provides anopportunity to help develop indicators used tomonitor the progress of the project towardsanticipated goals and outcomes.

Typical components of a programme logicmodel are shown in the diagram below.

Time LineWhen does each stage of the evaluation need to happen?

EvaluationFocus - Collect Data - Analyze and Interpret - Report

What we do

Conductworkshops,meetingsDeliverservicesDevelopproducts,curriculum,resourcesTrainProvidecounselingAssessFacilitatePartnerWork withmedia

Who wereachParticipants

ClientsAgencies

Decision-makers

Customers

Satisfaction

Outputs

Activities Participation

Outcomes - Impact

Short Term Medium Term Long Term

What theshort termresults are

LearningAwareness

Knowledge

Attitudes

Skills

Opinions

Aspirations

Motivations

What themediumterm results

are Action

Behavior

Practice

Decision-making

Policies

Social Action

What theultimateimpact(s) is

ConditionsSocial

Economic

Civic

Environmental

Program Action – Logic Model

Inputs

What weinvest

Staff

VolunteersTime

Money

Research base

Materials

Equipment

Technology

Partners

Priorities

Consider:

Mission

VisionValuesMandatesResourcesLocal dynamicsCollaboratorsCompetitors

Intended

outcomes

Situation

Needs andassets

Symptomsversusproblems

Stakeholderengagement

Assumptions External Factors

http://www.uwex.edu/ces/pdande/evaluation/evallogicmodel.htmlhttp://www.uwex.edu/ces/pdande/evaluation/evallogicmodel.html


28/85

27

3

• Reviewing assumptionson which project is basedIn complex community projects which aimto bring about change through a variety ofdifferent channels (increasing knowledge,building local capacity and skills, buildingempowerment and self-confidence), themechanisms for achieving each planned

outcome and the underlying theories oftenremain obscure to project staff. The logicmodel should help everyone involved inthe project (particularly project staff andconcerned populations) to understand theanticipated mechanisms for bringing aboutchange.

3.2 What are thebenefits of usinga programme logicmodel?

There are many benefits of using such asystem, extending beyond the developmentof a coherent research framework. Anumber of observed benefits are detailedbelow:

• Programme planningand designDeveloping a logic model helps to clarifythinking and to review activities andoutcomes against project aims andobjectives. Once the project is underway,

the model can be used as an ongoing tool toensure activities are on-target and to makenecessary modifications.

• Fostering joint ownership andcommitment to project goalsIn an ideal world, the first stages of buildingup the logic model will be undertaken asa joint exercise, involving all key players inproject design and implementation. This willinclude project staff, funders, representativesof the concerned community and other key

players in local community infrastructure. Thisearly approach to joint working should helpestablish productive working relationships andshared understanding among the differentstakeholders. Moreover, it promotes a sharedvision and understanding of the underlyingtheories and assumptions of the project.


29/85

28

A detailed analysis, which explores everyaspect of project activity and projected impact,will result in more effective and appropriatelytargeted project interventions. This stageis also a good opportunity to clarify keydefinitions which are open to confusion and

misinterpretation such as empowerment andparticipation. (See Annex 3 - glossary)

• Ongoing evaluationThe model helps to focus on each componentof programme activity. Individual parts canbe broken down into activities and linked tooutcomes which can be assessed separately;charting progress towards interim andlong-term outcomes. In this way robustmechanisms for measuring outcomes aredeveloped. What happens? What works? Forwhom? It should allow the team to identify the

obstacles preventing the project from workingoptimally. It is also a chance for the team toassess how easy it will be for them to measurethe indicators they have selected.

Moreover, the process should help identifyways to measure more intangible interimoutcomes (e.g. levels of participation, extent ofempowerment, cohesion of groups, etc). Theprogress of more complex initiatives can thusbe charted and improvements made along theway, based on new and emerging information.

Are your project assumptions correct?

In HIV and AIDS programmes there is often an assumption (based loosely on the Theory ofBehaviour Change Model) that knowledge about ways in which HIV is transmitted will leaddirectly to behaviour change. Clearly, the real life situation is more complex; the human mind,being a sophisticated instrument, weighs up the relative advantages of behaviour change inrelation to a cost/benefit analysis. If a sex worker, for example, is likely to lose her customer byinsisting on the use of condoms, this will create a disincentive to adopting safe behaviours andwill weigh heavily in any cost-benefit analysis. High levels of self-esteem, on the other hand,constitute one factor that may improve the chances of successful negotiation around condomuse.

In the context of a drugs programme, it is apparent that drug use results from a complexinteraction of social, cultural, psychological and economic factors. If, for example, a projectis working in an area of high poverty and deprivation where a strong “drug culture” prevailsamongst an underclass of largely alienated young people, it is apparent that knowledge aboutthe dangers of drug use will not be the only factor that determines their behaviour. The senseof allegiance and camaraderie afforded by membership in the drug taking group may, in theshort-term, be more important to some young people than the longer-term negative impact on

health. To be most effective, projects must be aware of all the factors that influence humanbehaviour.

• ParticipationThe process of developing the model is basedon participatory techniques which requirestakeholders to work together to clarify theunderlying rationale for the programme andthe conditions under which success is most

likely to be achieved. In this way, changes aremore likely to be built on consensus building,following an open, transparent process – andless on personalities, politics and ideology. Astronger sense of project ownership is thusfostered amongst stakeholders.

3.3 What doprogramme logic

models look like?Logic models come in all shapes and sizesand are developed according to the particularneeds of each individual project. Some modelsbegin with very basic structural links betweenproject components but grow over time in linewith project development and the creativeinsight of the project team. They should berepresented in one pictorial diagram in orderto give an overall vision of the links betweenthe different components and processes,

which form the whole project.


30/85

29

3

Sometimes a logic model is built as a table with lists of itemsin the input, output, and outcome columns. (See section 3 fordefinitions of these terms.) The model may include limiteddirectional arrows to illustrate connections and relationships.It may include numbered lists to show order within a column orto indicate rows of connections across the columns.

Other logic models use boxes, with lines and arrows connectingthe boxes to illustrate the causal linkages.

Some logic models use circles and other shapes. Somecommunity groups have used metaphors such as oysters, trees,footprints, and octopuses.

Some logic models are simple; others are complex.

Some logic models show only parts of a full model: some don’tinclude assumptions, situation or external factors; some onlyinclude outputs and outcomes.

Diagrams taken from: http://www.uwex.edu/ces/lmcourse/interface/coop_M1_Overview.htm (accessed January 2008)

(See also Section 3.43 and 3.44)

Remember that the logic model is justa MODEL. In an effort to simplify andcommunicate using one page, you can producelogic models that abbreviate programmecomplexities. Most importantly, the logicmodel must be clear and comprehensiveto those who will use it. To capture theprogramme theory, the logic model needsto show the linkages between and amongelements.

A logic model is a tool for enhancing

programme performance. Often the process ofproducing the logic model is more importantthan a polished end result. Thinking through

the steps involved in the project, in a logicaland systematic way, often helps to identify gaps(be they structural, economic or theoretical).

The purpose for which the logic model is to beused dictates the level of detail employed andthe information included. When a logic modelis used specifically for research purposes,it can be adapted to include more focus onoutcomes. The look and level of detail differswidely. Community-based practitioners whoneed to measure outcomes may wish to

show greater detail in the outcome chain andstreamline the input and output components.

Inputs Outputs Outcomes

1

2

3

4

1a

b

2a

b

c

3a

b

http://www.uwex.edu/ces/lmcourse/interface/coop_M1_Overview.htmhttp://www.uwex.edu/ces/lmcourse/interface/coop_M1_Overview.htm


31/85

30

Key Points

• Think about who will use the logic model.Who needs to understand the value/purposeof the model: you, your staff, the community,funders, administrators, elected officials?

• Settle on a graphic representation that bestfits the user and use.

• Recognise that deciding on a single imagethat displays the programme theory is oftenthe most difficult part of developing and usinga logic model. Probably more important thanthe final image is the process undertaken toproduce it.

3.4 TerminologyFamiliarising yourself with the following termswill help in assembling a logic model.

3.41 Project vision/goal

This is the big vision - what are you trying todo for the community? It may link directlyto improved levels of health and well-being(reflected in reduced levels of morbidity andmortality). Alternatively, it may focus on

education or capacity- building: raising levelsof education, training or wealth. Communityprojects tend to be based on a number of corevalues (e.g. equity, social justice, freedom,participation).

It may be interesting to discuss the keyobstacles to achieving your goal amongst yourbeneficiary group. Some or all of the followingfactors may come into play:

– income inequity

– social status/class

– level of social exclusion and social justice

– social support networks

– education

– employment and working conditions

– physical environment

– personal health practices and coping skills

– child development issues

– health services

– nutrition

– role of funders (possible constraintsimposed)

The list is as long as you want to make it. Theexercise is important because it puts into

perspective the huge number of variables thatmay have an impact on the project, helpingto provide an understanding of the complexityof community interventions and of the limitedimpact that small scale interventions can have.

Whilst thinking about your larger goal, it isimportant to focus more specifically on theproject’s priorities. In order to monitor projecteffectiveness, it is essential to have clearlyarticulated aims and objectives. These willdemonstrate how the overall goal/vision is

being achieved. If the goal, for example, is toreduce incidence of HIV infection, the aims andobjectives may relate to improved knowledge ofmodes of transmission, changing perceptionsof HIV, reported behaviour change or enhancedself-confidence. Alternatively the aims andobjectives may relate simply to putting theinfrastructure in place with which to attainlonger term goals for the future (providingbuildings, training staff, etc). Aims andobjectives should relate directly to indicators(discussed in section 5.5).

It is important to clarify the role of fundersfrom the very beginning. Have the fundersimposed any specific obligations in relationto project activity, e.g. monitoring, meetings,involvement in project? Are these requirementsgoing to affect the timing of project activities?


32/85

31

3

3.43 Outputs

Outputs are the activities, services, events, and products that reach people (individuals, groups,agencies) who participate in or who are targeted by the project.

Outputs are what we do or what we offer. They include workshops, services, conferences,

community surveys, facilitation, in-house counselling, etc.

Inputs are flexible and will vary in style andcontent depending on the type of project beingundertaken. A drugs education project, forexample, will need different types of inputsdepending on whether it is based within formalsettings (schools, youth clubs) or is targetedmore loosely at street kids or transientpopulations (immigrants or seasonal workers).

It is impossible to single out any one input asmore important than any other. However, onearea is often overlooked due to pressures ofmore tangible project activities: the buildingand maintenance of strong and appropriatepartnerships. Partners may include localorganizations who relate directly or indirectlyto project activity, service providers, funders(national and international), government andclient groups.

3.42 Inputs

Inputs are the resources and contributionsthat you and others make to the effort.These include time, people (staff, volunteers,beneficiaries if involved), the community,money, materials, equipment, partnerships,research base, and technology, among otherthings.

Diagram taken from: http://www.uwex.edu/ces/lmcourse/interface/coop_M1_Overview.htm (accessed January 2008)


What we do

Conductworkshops,meetingsDeliverservicesDevelopproducts,curriculum,resourcesTrainProvidecounseling

AssessFacilitatePartnerWork withmedia

Outputs


Who we reach

ParticipantsClientsAgenciesDecision-makersCustomers

What weinvest

StaffVolunteersTimeMoneyResearchBaseMaterials

EquipmentTechnologyPartners

Inputs

http://www.uwex.edu/ces/lmcoursehttp://www.uwex.edu/ces/lmcourse/interface/coop_M1_Overview.htmhttp://www.uwex.edu/ces/lmcoursehttp://www.uwex.edu/ces/lmcourse/interface/coop_M1_Overview.htm


33/85

32

These outputs are intended to lead to specific outcomes.


Outcomes - Impact



LearningAwareness

Knowledge

Attitudes

Skills

Opinions

Aspirations

Motivations

What themedium termresults are

Action

Behavior

Practice

Decision-making

Policies

Social Action


Conditions

Social

Economic

Civic

Environmental

3.44 Outcomes

Outcomes are the direct results, benefits, advantages or disadvantages for individuals, families,groups, communities, organizations or systems. Examples include changes in knowledge; skilldevelopment; changes in behaviour, capacities or decision-making; and policy development.Outcomes can be short-term, medium-term or longer-term achievements. Outcomes may be

positive, negative, neutral, intended or unintended, and they can be measured throughout theproject lifecycle.

Impact in this model refers to the ultimate consequences or effects of the programme. Forexample, increased economic security, reduced rates of teenage smoking or improved air quality.Impact is synonymous here with the long-term outcome of your goal. It is at the farthest right onthe logic model graphic. Impact refers to the ultimate, longer-term changes in social, economic,civic or environmental conditions. In common usage impact and outcomes are often usedinterchangeably.

It is important when considering impacts to look not only at intended, but also unintended impacts.An example of this may be found in gender-based projects where the focus is on raising self-esteemand improving livelihood opportunities for women. Long-term impacts will relate to the improved

economic and social status of the women. The ramifications of empowering and strengthening theposition of women may be significant in terms of readdressing the balance of power and statuswithin the household, which may in turn challenge local assumptions of the status quo.

In one UNESCO-funded project in South Asia, poor young women were empowered to such anextent that they began to assert the right to choose their own husband and were subsequentlyostracised for flouting long established codes of behaviour. If such eventualities are consideredat a project’s inception, it may be possible to alleviate any negative repercussions which mayresult, and to support all members of the community through the subsequent transition.



34/85

33

3

3.45 Looking at yourassumptions (theory)

Assumptions are the principles, beliefs and

ideas we have about the programme andthe people involved, and the way we thinkthe programme will work. Assumptions gotowards creating underlying beliefs abouthow the project will work; they are validatedwith research and experience. Assumptionsunderlie and influence the programmedecisions we make. They may relate to:

• The problem or situation.

• The resources and staff.

• The way the programme will operate.

• What the programme expects to achieve.

• The knowledge base.

• The external environment.

• The internal environment.

• The participants: how they learn, theirbehaviour, motivations, etc.

In developing a logic model, we want to makeexplicit all the implicit assumptions we aremaking. They may not all be portrayed in theone-page graphic, but we do want to exploreand discuss them.

Think about and clarify your assumptions onall dimensions in your logic model. What doyou know? What are you assuming? Continueto check and clarify them as you proceed.Faulty or overlooked assumptions are often

the reason for poor results. It may help tobuild a reflection cycle into the process. Inother words, the logic model can be drawn

Assumptions

Diagram taken from: http://www.uwex.edu/ces/lmcourse/interface/

coop_M1_Overview.htm (accessed January 2008)

up at project inception and then reviewedon an agreed regular basis to re-evaluatethe assumptions made and look at what haschanged and why.

3.46 Linkages - theory of action

Directional arrows can be used in the logicmodel to show how different bits of the modellink up. Connections may be vertical andhorizontal, one-direction or two-directionaland show feedback loops.

It is the linkages - not just what is labelled asinput, output, or outcome - that give the modelits power. Drawing the connections is oftenmessy and time-consuming but necessary.

It is what helps us make sure we’ve addressedall the logical connections. Sometimes wesimplify and only include the primary linkages;otherwise the logic model may become toodifficult to read.

In the end, the final outcome theoretically linksback to the beginning. By project completion itis anticipated that the starting point will havebeen transformed. The large feedback arrowat the top right of the logic model is an attemptto illustrate this connection and the dynamicsof programming. Some people like to show

the circular flow of a logic model that explicitlyconnects the end to the beginning. In actuality,programme environments are dynamic andconstantly changing so the beginning rarelystays the same.

Many people refer to these linkages as ‘if-then’ relationships. Reading from left to right,a logic model portrays a series of ‘if-then’relationships. When developing a logic model,think about the underlying assumptions,i.e. how are the linkages made? Are the

assumptions made realistic and sound? Whatevidence or research do you have to supportyour assumptions?

Diagram taken from: http://www.uwex.edu/ces/lmcourse/interface/

coop_M1_Overview.htm (accessed January 2008)

http://www.uwex.edu/ces/lmcourse/interfacehttp://www.uwex.edu/ces/lmcourse/interfacehttp://www.uwex.edu/ces/lmcourse/interfacehttp://www.uwex.edu/ces/lmcourse/interface


35/85

34

3.48 Explaining the difference between outputs and outcomes

Understanding the difference between outputs and outcomes is important.

Outputs relate to what we do. Outcomes and impact refer to what difference is made.

In the past, there has been a focus on the outputs column - the ‘what we do’ and ‘who we reach’.Recording outputs helps inform clients, funders and community partners about the nature of projectactivity. Most projects do a reasonable job of describing and counting activities and the number of

people served. The harder questions focus on “What difference does it make?” This is a questionabout OUTCOMES and impact.

3.47 External factors

A programme does not sit in isolation, outsideor apart from its surrounding environment. Aprogramme is affected by and affects externalfactors. These include for example: culturalmilieu, climate, economic structure, housingpatterns, demographic patterns, politicalenvironment, background and experience ofprogramme participants, media influence,changing policies and priorities. All ofthese may have a major influence on theachievement of outcomes. We can’t ignorethem! Among the elements they may affectare:

• programme implementation

• the speed and degree to which change

occurs


What we doConductworkshops,meetingsDeliverservicesDevelopproducts,curriculum,resourcesTrainProvidecounselingAssessFacilitatePartnerWork withmedia

Who we reachParticipantsClientsAgenciesDecision-makersCustomers

Outputs


Outcomes - Impact



LearningAwareness

Knowledge

Attitudes

Skills

Opinions

AspirationsMotivations

What themedium termresults are

Action

Behavior

Practice

Decision-making

Policies

Social Action


Conditions

Social

Economic

Civic

Environmental

• participants and recipients

• staffing patterns and resources available

A simple example includes mass localmigration following the opening of a factoryin the local city with potential employmentopportunities; both project staff andbeneficiaries may be lost to such a process.Conversely, a successful project with thepromise of building sustainable livelihoodsfor the most vulnerable may encourage morepeople to migrate into the local area, perhapsputting more strain on project resources. Areflection cycle, as mentioned above (3.45)may provide a suitable forum for assessing theimpact of external factors (both expected andunexpected).



36/85

35

3

In some logic models you will see activitiesseparated from outputs, with activities oftendisplayed before outputs. In these models,outputs are typically designated as theaccomplishment or product of the activity. Forexample, the number of workshops actuallydelivered or the number of individuals who heardthe media message. The assumption is that theactivity needs to be delivered as intended beforethe expected outcomes can occur.

3.49 What is programmetheory?

A logic model shows the series of connectionsor logical relationships that are expectedto lead to desired results over time. Themodel is primarily a framework or planning

tool – however, there are a number of logicalassumptions within the framework whichare the theoretical basis of project work.This logical base has been described as theprogramme’s theory of action (Patton, 1997) ortheory of change (Weiss,1998)

A theory of change is a description of howand why a set of activities (be they part of ahighly focused programme or a comprehensiveinitiative) are expected to lead to early,intermediate and longer-term outcomes over a

specified period.

Theory may sound too academic for some but itreally just refers to the following:

• Expectations• Beliefs• Experience• Conventional wisdom

For example, the theory behind a drugintervention programme may be that byempowering and educating young people

about drug misuse, the project will provideindividuals with the knowledge and confidenceto choose not to take drugs or use them in aless harmful way – thus providing them withskills and strategies to follow different paths.

3.410 Examples of linkagesfrom a drug educationprogramme:

INPUT

Staff training in drugs education

Workshops involving staff and target group tofinalise training programme and content

OUTPUT

Conducting workshops for the group

concerned

OUTCOME

Short-term: learning, awareness, knowledge,attitudes

Medium/long-term: Improved health,

behaviour, economic situation

A closer analysis of the above relationshipshelps establish a feel for their strength androbustness. For example, as long as a drugsworkshop is well planned, structured andtargeted (your assumptions), an increasedlevel of knowledge about drugs at the end of aperformance is not an unrealistic expectation.More tenuous however are the mediumand long-term anticipated outcomes thateducational workshops will lead to changesin behaviour or indeed an improved health

and economic position. There are many moreexternal factors that come into play and thesecould be shown as arrows coming off the logicmodel. For example, external factors which mayact as an obstruction to individuals in adoptingsafer drug related behaviours may include:


37/85

36

Examples of factors which may influence drug taking behaviour

Building up the connections in this way, showing factors which may influence outcomes (manyof which may be outside immediate project control), helps to place the project within a realisticcontext.

Drugtaking

behaviour

Ease ofaccess to

drugs

Access toother socialand leisure

activities

Behaviourof peers

Level ofsocial

support

Level ofknowledge

Low selfesteem


38/85

37

Notes:

......................................................................................................................................................................

......................................................................................................................................................................

......................................................................................................................................................................

......................................................................................................................................................................

......................................................................................................................................................................

......................................................................................................................................................................

......................................................................................................................................................................

......................................................................................................................................................................

......................................................................................................................................................................

......................................................................................................................................................................

......................................................................................................................................................................

......................................................................................................................................................................

......................................................................................................................................................................

......................................................................................................................................................................

......................................................................................................................................................................

......................................................................................................................................................................

......................................................................................................................................................................

......................................................................................................................................................................


39/85


40/85

39

Section 4 How to createa logic model

4

Step 1: Determine the purpose of the logic model. Who will use it and for what? 40

Step 2: Involve others 40

Step 3: Set the boundaries for the logic model 40

Step 4: Understand the situation 40

Step 5: Write a situation statement 40

Step 6: Explore the research and knowledge base 42

Step 7: Now start to fi ll in the empty boxes 42


41/85

40

Step 2: Involve others• Who should participate?

• Who should facilitate?

Step 3: Set theboundaries for the logicmodel

• What will the logic model depict: A single,focused activity (for example a single puppetshow to educate about HIV and AIDS withina specific setting), or a more comprehensiveinitiative involving a number of different but

complementary activities? For example, anumber of puppet shows running alongsidethe distribution of leaflets and communityoutreach work carried out over a number ofmonths.

• What level of detail is needed?

• Who will use the logic model? How?

Step 4: Understandthe situationThe information needed for this step canbe obtained from the needs assessmentdescribed in section 2.3.

Step 5: Write

a situation statementThis statement will form the foundation of yourlogic model. It will be based on the followingquestions:

• What is the problem/issue?

• Why is this a problem? (What causes theproblem?)

• For whom does this problem exist and atwhat levels (individual, household, group,

community, society in general)?

Logic model development is a process.

• Time and practice are required before youcan use logic models effectively. The bestway to learn is to practice! There is a goodinteractive site for practicing logic modelslocated at http://www1.uwex.edu/ces/1mcourse/ The self-study module containedon this site is worth working through, asit takes you through the steps clearly andmethodically with numerous opportunitiesto practice.

• The process of constructing a logic model(rather than the finished product) maybe the most important aspect of logicmodel development. The process buildsunderstanding, consensus and clarity inthinking about the programme - all of which

are critical to the programme’s success.

• Logic models are refined and changed manytimes. Leave your logic model in a visibleplace, perhaps stuck on a wall, where itcan be altered and amended as the projectdevelops. Change the model as the projectdynamic begins to change and you learnmore about your programme.

Getting Started

Step 1: Determine thepurpose of the logic model, who will use itand for what?

• Why are you formulating a logic model?

• Is your purpose to fulfil an administrativerequirement? To show your funderswhat you are doing? To complete a grantproposal? To determine a work plan? Toevaluate your programme? To improvethe quality of your work? To involve allstakeholders in a participatory processto build project understanding andcommitment?

• Make sure that everyone working on the

model understands its value.

http://www1.uwex.edu/ces/1mhttp://www1.uwex.edu/ces/1m


42/85

41

4

Describing who is affected by the problemhelps in the subsequent assessment of whohas benefited.

Priority SettingFrom the situation statement comes prioritysetting. Once the situation and problem arefully analysed, priorities can be set. Seldom

can we undertake everything, so we have toprioritise. Several factors are pertinent here;these include your mission, values, resources,expertise, experience, history, what you knowabout the situation and what others are doingin relation to the problem. Priorities lead to theidentification of desired inputs and outcomes.

• Who has a stake in the problem? (Who cares whether it is resolved or not?)

• What do we know about the problem/issue/people that are involved? What research andexperience do we have?

A situation statement from a drug education project:

In a drug programme, for example, your problem (identified during a needs assessment) maybe that within a defined community (school, village, marketplace) 20% of young people agedbetween 13 and 20 are taking illegal drugs. Having identified the problem, you need to explore itin much more detail:

– What are the perceived benefits of this behaviour? (i.e. why do the young people take drugs?Perceived benefits may include a sense of membership to a group, a good feeling from the drug.)

– What are the negative repercussions of this behaviour? (short-term and long-term)

– Which groups do the drugs affect, e.g. young people, parents, carers, wider family, school? Thiswill help to decide who has a stake in altering the behaviour and may therefore be willing to getinvolved in the programme.

– Look at the impact from a number of different perspectives: social, psychological, economic.

– In order to answer these questions fully it is important to talk to stakeholders in the community(including the key target group: drug-taking young people) as well as gathering evidence fromother sources. Data collection methods may include discussions with managers of similarprogrammes, review of published articles or evaluation reports, talking to other professionals,etc.

The situation statement can be appended tothe logic model as text. It is good practice toinclude a few words on the far left side of thelogic model. These words should capture theessence of the project. What is the problem/issue? Too often, we design and implementprogrammes without fully considering andunderstanding the situation. The better weunderstand the situation and analyse theproblem, the easier it is to develop effectivelogic models.

The situation statement forms the basis of therest of your work and provides the foundationfrom which the logic model is built. The socialcontext of the project is open to constant fluxand change and so needs to be updated asappropriate.

The situation statement establishes a baselinefor comparison during project implementationand on programme completion. A descriptionof the problem and its symptoms provides a

guideline against which to monitor projectprogress.


43/85

42

• What is your end goal?

• What will be different?

• How will your target population, thewider community, the social and economicinfrastructure, the environment be differentas a result of the programme?

Agree on a simple statement describing theultimate end result that you are hoping toachieve. This end result is the same as yourgoal. Spend time clarifying and agreeing onyour long-term desired outcome.

Once you have agreed on your long-termoutcome (end result/goal), work backwardsacross the logic model. In order to achievethose outcomes what will your outputs be?

What inputs are needed? You can use thetemplate below, or the version included inannex 4, to practice.

Step 6: Explorethe res

guide for monitoring evaluation community based projects

Documents