m%20and%20 e milan
DESCRIPTION
TRANSCRIPT
![Page 1: M%20and%20 E Milan](https://reader036.vdocuments.mx/reader036/viewer/2022081413/547f28bab4af9f8b5a8b45a2/html5/thumbnails/1.jpg)
Program Monitoring and Program Monitoring and EvaluationEvaluation
Colorado State University,USA 14 September 2002
Milan K.Dinda Ph.D
![Page 2: M%20and%20 E Milan](https://reader036.vdocuments.mx/reader036/viewer/2022081413/547f28bab4af9f8b5a8b45a2/html5/thumbnails/2.jpg)
M I
L A
N -
CS
UM
I L
A N
- C
SU
Why the Need for M&E?
![Page 3: M%20and%20 E Milan](https://reader036.vdocuments.mx/reader036/viewer/2022081413/547f28bab4af9f8b5a8b45a2/html5/thumbnails/3.jpg)
MIL
AN
-CS
U
Why M&E? Why M&E?
Internal uses External uses Asses progress against targets Modify program design and
operation Identify staff needs Informed long range planning Make resource allocation
decisions Identify problems or
bottlenecks Develop a common
understanding of desiredoutcomes
Report to funding agency Respond to questions or
criticisms about the program Promote the program to
potential participants Seek additional and more
targeted funding Improve outreach, public
relations and marketing effort
![Page 4: M%20and%20 E Milan](https://reader036.vdocuments.mx/reader036/viewer/2022081413/547f28bab4af9f8b5a8b45a2/html5/thumbnails/4.jpg)
MIL
AN
-CS
U
Defining the Terms
![Page 5: M%20and%20 E Milan](https://reader036.vdocuments.mx/reader036/viewer/2022081413/547f28bab4af9f8b5a8b45a2/html5/thumbnails/5.jpg)
MIL
AN
-CS
UM
ILA
N-C
SU
What is Monitoring?What is Monitoring?
• A routine processroutine process used to determine the extent to which a program has been effectively implemented at different levels, in time and at what cost
• Basically an internal activity
• Carried out regularly, for example, monthly,quarterly, half-yearly or annually.
![Page 6: M%20and%20 E Milan](https://reader036.vdocuments.mx/reader036/viewer/2022081413/547f28bab4af9f8b5a8b45a2/html5/thumbnails/6.jpg)
MIL
AN
-CS
U
What is Evaluation?What is Evaluation?
• An episodic processepisodic process designed to determine the value or worth of a specific project or program.
• Could be both internal and external
• Links particular outputs or outcome directly to an intervention
![Page 7: M%20and%20 E Milan](https://reader036.vdocuments.mx/reader036/viewer/2022081413/547f28bab4af9f8b5a8b45a2/html5/thumbnails/7.jpg)
MIL
AN
-CS
U
M and E in a NutshellM and E in a Nutshell
Monitoring Monitoring EvaluationEvaluation
Routine, daily assessment of on-going activities and progress
Looks at what is being done
Tracks changes over time in inputs, processes and outputs
Episodic assessment of achievements
Examines what has been achieved
Measures outcomes/ effects and impact cased by interventions
![Page 8: M%20and%20 E Milan](https://reader036.vdocuments.mx/reader036/viewer/2022081413/547f28bab4af9f8b5a8b45a2/html5/thumbnails/8.jpg)
MIL
AN
-CS
U
Three Types of EvaluationThree Types of Evaluation
Process EvaluationProcess Evaluation
• Are activities actually implemented? (Content)
• Does the program reach its intended clientele? (Scope and Coverage)
• Are activities being implemented as planned with necessary inputs? (Timing)
![Page 9: M%20and%20 E Milan](https://reader036.vdocuments.mx/reader036/viewer/2022081413/547f28bab4af9f8b5a8b45a2/html5/thumbnails/9.jpg)
MIL
AN
-CS
U
Three Types of EvaluationThree Types of Evaluation
Outcome EvaluationOutcome Evaluation
• Examines short-term effects (based on program objectives)
• With HIV/AIDS interventions, immediate effects or outcomes are often relatedto behavior changes and underlying changes in knowledge, attitudes and beliefs (e.g., use of condom)
• Usually looks at “before and after situation”
![Page 10: M%20and%20 E Milan](https://reader036.vdocuments.mx/reader036/viewer/2022081413/547f28bab4af9f8b5a8b45a2/html5/thumbnails/10.jpg)
MIL
AN
-CS
UM
ILA
N-C
SU
Three Types of EvaluationThree Types of Evaluation
Impact EvaluationImpact Evaluation
• Assesses the longer-term results of a program • Takes a long time, are expensive and complex • Example: Reduction in the new infection of HIV/AIDS
![Page 11: M%20and%20 E Milan](https://reader036.vdocuments.mx/reader036/viewer/2022081413/547f28bab4af9f8b5a8b45a2/html5/thumbnails/11.jpg)
MIL
AN
-CS
U
Systems Approach to M&E
![Page 12: M%20and%20 E Milan](https://reader036.vdocuments.mx/reader036/viewer/2022081413/547f28bab4af9f8b5a8b45a2/html5/thumbnails/12.jpg)
MIL
AN
-CS
UM
ILA
N-C
SU
Systems Analysis to M&ESystems Analysis to M&E
Input OutputProcess Impact Outcome
![Page 13: M%20and%20 E Milan](https://reader036.vdocuments.mx/reader036/viewer/2022081413/547f28bab4af9f8b5a8b45a2/html5/thumbnails/13.jpg)
MIL
AN
-CS
UM
ILA
N-C
SU
Systems Analysis to M&ESystems Analysis to M&E
Inputs OutputProcess Impact Outcome
InputsInputs
• Resources invested in a program (supplies, people,
• 5 M’s: (Wo)Man, Machines, Money, Methods, Materials
![Page 14: M%20and%20 E Milan](https://reader036.vdocuments.mx/reader036/viewer/2022081413/547f28bab4af9f8b5a8b45a2/html5/thumbnails/14.jpg)
MIL
AN
-CS
UM
ILA
N-C
SU
Systems Analysis to M&ESystems Analysis to M&E
Inputs OutputProcess Impact Outcome
Process Process
• activities carried out to achieve objectives
• shows what is done and how well it is done, for example, how volunteers are recruited and trained to become peer counselors
![Page 15: M%20and%20 E Milan](https://reader036.vdocuments.mx/reader036/viewer/2022081413/547f28bab4af9f8b5a8b45a2/html5/thumbnails/15.jpg)
I C
O
M
P
I C
O
M
P
Systems Analysis to M&ESystems Analysis to M&E
Inputs OutputProcess Impact Outcome
Outputs Outputs
• immediate results of activities achieved (deliverables of the program)
• number of workshops conducted; MSMs reached by BCC materials; STI, VCT care; knowledge of HIV transmission
![Page 16: M%20and%20 E Milan](https://reader036.vdocuments.mx/reader036/viewer/2022081413/547f28bab4af9f8b5a8b45a2/html5/thumbnails/16.jpg)
MIL
AN
-CS
UM
ILA
N-C
SU
Systems Analysis to M&ESystems Analysis to M&E
Inputs OutputProcess Impact Outcome
Outcomes (Effects)Outcomes (Effects)
• changes observed among clients of the program
• changes in behavior or skills (e.g., increased usage of condoms among truck drivers; exchange of clean needles); reduction in STI trend; increase social support
![Page 17: M%20and%20 E Milan](https://reader036.vdocuments.mx/reader036/viewer/2022081413/547f28bab4af9f8b5a8b45a2/html5/thumbnails/17.jpg)
MIL
AN
-CS
UM
ILA
N-C
SU
Systems Analysis to M&ESystems Analysis to M&E
Inputs OutputProcess Impact Outcome
ImpactImpact
• changes in the longer term that occur in the community level as a result of a given program
• usually observed in community level (e.g., reduction in HIV/AIDS related mortality; decreased prevalence; coping capacity of the community)
![Page 18: M%20and%20 E Milan](https://reader036.vdocuments.mx/reader036/viewer/2022081413/547f28bab4af9f8b5a8b45a2/html5/thumbnails/18.jpg)
MIL
AN
-CS
UM
ILA
N-C
SU
Logical Framework and Systems Approach to M&E
![Page 19: M%20and%20 E Milan](https://reader036.vdocuments.mx/reader036/viewer/2022081413/547f28bab4af9f8b5a8b45a2/html5/thumbnails/19.jpg)
MIL
AN
-CS
UM
ILA
N-C
SU
What is a Logical Framework?What is a Logical Framework?
Management tool for strategic planning and project management
Systematically and logically sets out a program’s objectives, outputs and activities.
![Page 20: M%20and%20 E Milan](https://reader036.vdocuments.mx/reader036/viewer/2022081413/547f28bab4af9f8b5a8b45a2/html5/thumbnails/20.jpg)
MIL
AN
-CS
UM
ILA
N-C
SU
What is a Logical Framework?What is a Logical Framework?
Hierarchy of Aims
Goal (Impact):Goal (Impact): the ultimate end of the program to which the specific project will contribute (reduction of prevalence of HIV/AIDS among sex workers in Chiang Mai)
Purpose (Outcome/Effect):Purpose (Outcome/Effect): what the project is expected to achieve once it is completed within the allocated time (provision of counseling and clinic services to sex workers)
Outputs:Outputs: products or deliverables of activities (trained peer counselors; clinical services provided)
Activities (Inputs; Process):Activities (Inputs; Process): actions to be undertaken and the resources available to produce the outputs (upgrading of clinics)
![Page 21: M%20and%20 E Milan](https://reader036.vdocuments.mx/reader036/viewer/2022081413/547f28bab4af9f8b5a8b45a2/html5/thumbnails/21.jpg)
MIL
AN
-CS
UM
ILA
N-C
SU
Logframe and Systems Analysis Logframe and Systems Analysis
![Page 22: M%20and%20 E Milan](https://reader036.vdocuments.mx/reader036/viewer/2022081413/547f28bab4af9f8b5a8b45a2/html5/thumbnails/22.jpg)
MIL
AN
-CS
UM
ILA
N-C
SU
M & E Framework for HIV/AIDS Program
![Page 23: M%20and%20 E Milan](https://reader036.vdocuments.mx/reader036/viewer/2022081413/547f28bab4af9f8b5a8b45a2/html5/thumbnails/23.jpg)
MIL
AN
-CS
UM
ILA
N-C
SU
M&E Framework for HIV/AIDS ProgramM&E Framework for HIV/AIDS Program
OrganizationalResources
ExternalResources
TechnicalCompetence
ManagementProcesses
Multi-sectoralparticipation
Care andSupport
Counseling andPreventiveServices
IEC, BCC andAdvocacy
Socio-political,economic andcultural factors
Prevalence Statusof HIV/AIDS
Behavior andAttitude
Practices
Social/Community
Support, PolicyChanges
Inputs Process Outputs Outcomes Impact
Monitoring Evaluation
![Page 24: M%20and%20 E Milan](https://reader036.vdocuments.mx/reader036/viewer/2022081413/547f28bab4af9f8b5a8b45a2/html5/thumbnails/24.jpg)
MIL
AN
-CS
UM
ILA
N-C
SU
M & E Process
![Page 25: M%20and%20 E Milan](https://reader036.vdocuments.mx/reader036/viewer/2022081413/547f28bab4af9f8b5a8b45a2/html5/thumbnails/25.jpg)
MIL
AN
-CS
UM
ILA
N-C
SU
ProcessProcess
• Define the Purpose of M & E
• Define the Scope of M & E
• Develop Indicators
• Determine Data Sources and Methods
• Develop an Implementation Plan
• Disseminate and Utilize Results
![Page 26: M%20and%20 E Milan](https://reader036.vdocuments.mx/reader036/viewer/2022081413/547f28bab4af9f8b5a8b45a2/html5/thumbnails/26.jpg)
MIL
AN
-CS
UM
ILA
N-C
SU
Define the Purpose of M & EDefine the Purpose of M & E
Three W’sThree W’s What: What interventions to be monitored and evaluated?
Why: Is it a routine process or a short-term process for decision making? Is it to examine results at output, outcome or impact level?
Who: Who will use the results (e.g., NGO, program manager, CBD staff , community, donor)
![Page 27: M%20and%20 E Milan](https://reader036.vdocuments.mx/reader036/viewer/2022081413/547f28bab4af9f8b5a8b45a2/html5/thumbnails/27.jpg)
MIL
AN
-CS
UM
ILA
N-C
SU
Define the Scope of M & EDefine the Scope of M & E
Important: Review first your program goals and Important: Review first your program goals and objectivesobjectives
• Intervention Level (Coverage): Program level, community level
Causality Links: Cause and Effect of what?
Timetable: How long will the M & E be undertaken?
![Page 28: M%20and%20 E Milan](https://reader036.vdocuments.mx/reader036/viewer/2022081413/547f28bab4af9f8b5a8b45a2/html5/thumbnails/28.jpg)
MIL
AN
-CS
UM
ILA
N-C
SU
Develop Indicators Develop Indicators
• Marker of performance - to track progress in the performance of the HIV/AIDS interventions
• Operational measures - “if you can measure it you can manage it”
• Key Standards: QualityQuality (binary - yes/no measure, categorical); QuantityQuantity (rate, absolute number, average, proportion), CostCost and TimingTiming
![Page 29: M%20and%20 E Milan](https://reader036.vdocuments.mx/reader036/viewer/2022081413/547f28bab4af9f8b5a8b45a2/html5/thumbnails/29.jpg)
MIL
AN
-CS
UM
ILA
N-C
SU
Develop Indicators Develop Indicators
Characteristics of Good A Good Indicator Valid: It is applicable to the interventions being measured (percentage of condom usage)
Reliable: Bias-free and consistent
Specific: Identifies specific areas to be examined (users during the last 3 months in District 1)
Sensitive: Reflects changes in the situation (Increase in the number of condom users)
Operational: It is measurable or quantifiable (number, percentage)
![Page 30: M%20and%20 E Milan](https://reader036.vdocuments.mx/reader036/viewer/2022081413/547f28bab4af9f8b5a8b45a2/html5/thumbnails/30.jpg)
MIL
AN
-CS
UM
ILA
N-C
SU
Develop IndicatorsDevelop Indicators
Component IndicatorsInputs Number of personnel assigned to the program (Quantity)
Qualified personnel assigned (Quality) The total cost of program. Budget for counseling component (Cost) Timeline of the program (Timing)
Process Number of peer educators trained (Quantity) Number of peer leaders trained who passed the test (Quality) Cost per peer training (Cost) Period of training days (Timing)
Outputs Increase in the number of sex workers reached (Quantity) Percent of sex workers who understood HIV/AIDS risks (Quality) Added cost of increasing the numbers of peer educators (Cost) Frequency of follow-ups undertaken (Timing)
Outcomes Increase in condom use rate among sex workers (Quantity) Improvement in negotiation skills of sex workers (Quality) Costs associated with the increase in condom usage (Cost) Time required to achieve improvement in condom usage (Timing)
Impact Reduction in HIV/AIDS rate among sex workers (Quantity) Ability of sex workers to collectively negotiate for occupational safety
(Quality) Cost of achieving reduction in HIV/AIDS prevalence (Cost) Time required to achieve reduction in HIV/AIDS prevalence (Timing)
![Page 31: M%20and%20 E Milan](https://reader036.vdocuments.mx/reader036/viewer/2022081413/547f28bab4af9f8b5a8b45a2/html5/thumbnails/31.jpg)
MIL
AN
-CS
UM
ILA
N-C
SU
Determine Data Sources/Methods Determine Data Sources/Methods
• Program-based: Direct Interventions - routine program statistics, client surveys, FGDs, facility-based surveys, in-depth interviews, provider survey, program documents and reports, direct observations
• Community-based: Indirect Interventions - community-based surveys, FGDs, qualitative surveys, population census
• Context-based - parliamentary records, public libraries, media reports
![Page 32: M%20and%20 E Milan](https://reader036.vdocuments.mx/reader036/viewer/2022081413/547f28bab4af9f8b5a8b45a2/html5/thumbnails/32.jpg)
MIL
AN
-CS
UM
ILA
N-C
SU
Develop an Implementation Plan Develop an Implementation Plan
Tip: Review your Logframe (if you have)
• Who is doing what?
• When is it being done?
• How much time does it take?
• How much does it cost?
![Page 33: M%20and%20 E Milan](https://reader036.vdocuments.mx/reader036/viewer/2022081413/547f28bab4af9f8b5a8b45a2/html5/thumbnails/33.jpg)
MIL
AN
-CS
UM
ILA
N-C
SU
Disseminate and Utilize Results Disseminate and Utilize Results
• Utilize in Decision Making
• Discuss with program staff
• Advocacy
• Share to donors and other interested parties
• Share to stakeholders and communities
![Page 34: M%20and%20 E Milan](https://reader036.vdocuments.mx/reader036/viewer/2022081413/547f28bab4af9f8b5a8b45a2/html5/thumbnails/34.jpg)
MIL
AN
-CS
UM
ILA
N-C
SU
Principles of M & E
![Page 35: M%20and%20 E Milan](https://reader036.vdocuments.mx/reader036/viewer/2022081413/547f28bab4af9f8b5a8b45a2/html5/thumbnails/35.jpg)
MIL
AN
-CS
UM
ILA
N-C
SU
Principles (BLEND) Principles (BLEND)
• Bias-free • Logical and Clear
• Educational
• Non-threatening
• Diversity of views
![Page 36: M%20and%20 E Milan](https://reader036.vdocuments.mx/reader036/viewer/2022081413/547f28bab4af9f8b5a8b45a2/html5/thumbnails/36.jpg)
MIL
AN
-CS
UM
ILA
N-C
SU
Pitfalls to Avoid
![Page 37: M%20and%20 E Milan](https://reader036.vdocuments.mx/reader036/viewer/2022081413/547f28bab4af9f8b5a8b45a2/html5/thumbnails/37.jpg)
MIL
AN
-CS
UM
ILA
N-C
SU
Avoid Pitfalls Avoid Pitfalls
• Too scientific • Perfecting the design
• Ignoring Stories
• Probing only Success
• Getting Done With It