vamesp ii monitoring & evaluation elementary opening and introduction to the course
TRANSCRIPT
![Page 1: VAMESP II Monitoring & Evaluation Elementary Opening and Introduction to the Course](https://reader031.vdocuments.mx/reader031/viewer/2022013012/56649e9c5503460f94b9d30e/html5/thumbnails/1.jpg)
VAMESP II
Monitoring & Evaluation
Elementary
Opening and Introduction to the Course
![Page 2: VAMESP II Monitoring & Evaluation Elementary Opening and Introduction to the Course](https://reader031.vdocuments.mx/reader031/viewer/2022013012/56649e9c5503460f94b9d30e/html5/thumbnails/2.jpg)
VAMESP II
Getting to know each other?
Who am I ? What is my background? What do I do now? What is my involvement
with Monitoring and Evaluation
What are my expectations from this workshop / training activity
Illustration by Petra RÖhr-Rowendaal
![Page 3: VAMESP II Monitoring & Evaluation Elementary Opening and Introduction to the Course](https://reader031.vdocuments.mx/reader031/viewer/2022013012/56649e9c5503460f94b9d30e/html5/thumbnails/3.jpg)
VAMESP II
Objective of these sessions Review principles of M&E and methodologies of
implementation – Building awareness that M&E is more than just data
collection and progress reporting M&E is an integral part of good practice in project
management. Setting M&E in the context of regulatory and donor
requirements. Collaborative working on practical M&E issues
(logframes, developing indicators, preparing/planning for monitoring) as they affect course participants.
Introduce the planning tools required for robust M&E.
![Page 4: VAMESP II Monitoring & Evaluation Elementary Opening and Introduction to the Course](https://reader031.vdocuments.mx/reader031/viewer/2022013012/56649e9c5503460f94b9d30e/html5/thumbnails/4.jpg)
VAMESP II
Content – what will we discuss?
What is monitoring and evaluation? What must we do for GoV and Donors? What information do managers need? Planning for M&E using Logframe
approach? Selecting indicators for M&E? Developing an M&E Plan? Communicating M&E information.
![Page 5: VAMESP II Monitoring & Evaluation Elementary Opening and Introduction to the Course](https://reader031.vdocuments.mx/reader031/viewer/2022013012/56649e9c5503460f94b9d30e/html5/thumbnails/5.jpg)
VAMESP II
What method will we use? Advantages of participatory
learning
Brief presentations Discussion and information sharing Practical exercises Using the M&E Manual as a resource
Demands a collaborative and participative approach
Illustrations by Petra RÖhr-Rowendaal
![Page 6: VAMESP II Monitoring & Evaluation Elementary Opening and Introduction to the Course](https://reader031.vdocuments.mx/reader031/viewer/2022013012/56649e9c5503460f94b9d30e/html5/thumbnails/6.jpg)
VAMESP II
Monitoring & Evaluation
Elementary – National M & E System
Section 2
Legislation, Principles and Elements
![Page 7: VAMESP II Monitoring & Evaluation Elementary Opening and Introduction to the Course](https://reader031.vdocuments.mx/reader031/viewer/2022013012/56649e9c5503460f94b9d30e/html5/thumbnails/7.jpg)
VAMESP II
Framework for M&E of ODA investments
National Strategic objectives for ODA are based on: 10 year social and economic strategy A 5 year development plan The ODA Master Plan Comprehensive Poverty Reduction and
Growth Strategy Focal point for M&E of ODA is FERD in
the Ministry of Planning and Investment
![Page 8: VAMESP II Monitoring & Evaluation Elementary Opening and Introduction to the Course](https://reader031.vdocuments.mx/reader031/viewer/2022013012/56649e9c5503460f94b9d30e/html5/thumbnails/8.jpg)
VAMESP II
M&E Responsibilities Legal basis in Chap VI, Decree
131/2006/ND-CP Various levels of responsibility:
of Project management units of Line Agencies of Ministry of Planning and Investment
Various types of reporting Monthly, quarterly, annual and terminal
reporting
![Page 9: VAMESP II Monitoring & Evaluation Elementary Opening and Introduction to the Course](https://reader031.vdocuments.mx/reader031/viewer/2022013012/56649e9c5503460f94b9d30e/html5/thumbnails/9.jpg)
VAMESP II
Good Practice in M&E
Good practice principles of M&E Reporing and monitoring systems country owned
and led Donors aim to rely on country owned reporting and
monitoring systems Reporting and monitoring systems support country’s
own information needs M&E systems should be simple and harmonised Information should be shared and transparent Evaluation missions should be planned and
implemented in partnership
![Page 10: VAMESP II Monitoring & Evaluation Elementary Opening and Introduction to the Course](https://reader031.vdocuments.mx/reader031/viewer/2022013012/56649e9c5503460f94b9d30e/html5/thumbnails/10.jpg)
VAMESP II
Principles of the Vietnam M&E system
Usefulness Impartiality and independence Credibility Participation Harmonisation Scheduled monitoring and evaluation program Professionally designed evaluations Cost effectiveness Report communicate & feedback results Using results to manage
![Page 11: VAMESP II Monitoring & Evaluation Elementary Opening and Introduction to the Course](https://reader031.vdocuments.mx/reader031/viewer/2022013012/56649e9c5503460f94b9d30e/html5/thumbnails/11.jpg)
VAMESP II
Elements of the Vietnam M&E system Glossary of M&E terms Purpose and design Indicators that are S M A R T Measures Tools and pro-formas Analysis and interpretation Reporting Sharing information
![Page 12: VAMESP II Monitoring & Evaluation Elementary Opening and Introduction to the Course](https://reader031.vdocuments.mx/reader031/viewer/2022013012/56649e9c5503460f94b9d30e/html5/thumbnails/12.jpg)
VAMESP II
Monitoring flow chart
Chủ DA
![Page 13: VAMESP II Monitoring & Evaluation Elementary Opening and Introduction to the Course](https://reader031.vdocuments.mx/reader031/viewer/2022013012/56649e9c5503460f94b9d30e/html5/thumbnails/13.jpg)
VAMESP II
National Monitoring System-Progress
Quaterly Progress AMT thought Project Owner
PMU PMU PMU PMU
Quaterly Progress AMT thought Project Owner
PMU PMU PMU PMU
Quaterly Portfolio
PMT
Line Agency
Line Agency
Donor Donor
Quaterly Portfolio
PMT
Feedback & Banchmarking
Feedback&Benchmarking
National Agencies
![Page 14: VAMESP II Monitoring & Evaluation Elementary Opening and Introduction to the Course](https://reader031.vdocuments.mx/reader031/viewer/2022013012/56649e9c5503460f94b9d30e/html5/thumbnails/14.jpg)
VAMESP II
PMU
Annual PlanDisbursement & Outputs
Monitoring data:Actual disbursement
Monitoring data:Actual outputs
Ø Monthly progress report to MPI, MoF & line agency for Gp A projectsØ Quarterly progress report to line agency, MPI and donorØ Quarterly technical report to related technical ministriesØ Bi-annual national outcome indicators to line agency, MPI and donor
Monitoring data:National outcome indicators
M&
E R
epor
t
ODA monitoring - PMU
![Page 15: VAMESP II Monitoring & Evaluation Elementary Opening and Introduction to the Course](https://reader031.vdocuments.mx/reader031/viewer/2022013012/56649e9c5503460f94b9d30e/html5/thumbnails/15.jpg)
VAMESP II
Line
age
ncy
(Min
istr
y or
Pro
vinc
e)
Aggregate annual PlanDisbursement & outputs
Aggregated data:Actual disbursement
Aggregated data:Actual outputs
Ø Quarterly summary progress reports to MPI, MoF and donorsØ Quarterly feedback report to PMUsØ Bi-annual provincial or sector national indicators to MPI and donors
Analysed data:National outcome indicators
M&
E R
epor
t
ODA monitoring – line agency
![Page 16: VAMESP II Monitoring & Evaluation Elementary Opening and Introduction to the Course](https://reader031.vdocuments.mx/reader031/viewer/2022013012/56649e9c5503460f94b9d30e/html5/thumbnails/16.jpg)
VAMESP II
ODA monitoring – MPIN
atio
nal O
DA
Man
agem
ent A
genc
ies
Ø Monthly summary exceptions report to PM for cabinet meetingØ Quarterly summary progress reports to PM and donorsØ Quarterly feedback report to line agencies, PMUs and donorsØ Quarterly lessons learned to national M&E websiteØ Bi-annual summary of national indicators to PM and CG meeting
Dat
a Ex
chan
ge &
OD
A W
ebsi
te
National ODA PlanDisbursement & outputs
Consolidated data:Actual disbursement
Consolidated data:Actual outputs
Consolidated data:National outcome indicators
![Page 17: VAMESP II Monitoring & Evaluation Elementary Opening and Introduction to the Course](https://reader031.vdocuments.mx/reader031/viewer/2022013012/56649e9c5503460f94b9d30e/html5/thumbnails/17.jpg)
VAMESP II
Monitoring & Evaluation
Elementary – Foundations
Section 3
Communication
![Page 18: VAMESP II Monitoring & Evaluation Elementary Opening and Introduction to the Course](https://reader031.vdocuments.mx/reader031/viewer/2022013012/56649e9c5503460f94b9d30e/html5/thumbnails/18.jpg)
VAMESP II
Communication process
ChannelMessage ReceiverSender
Context
The Communications Process
Feedback
![Page 19: VAMESP II Monitoring & Evaluation Elementary Opening and Introduction to the Course](https://reader031.vdocuments.mx/reader031/viewer/2022013012/56649e9c5503460f94b9d30e/html5/thumbnails/19.jpg)
VAMESP II
Communication Process To get your message across to others
Communication process involves Sender Message Channel Receiver Feedback Context
Message is successful when the sender and the receiver perceive it in same way
![Page 20: VAMESP II Monitoring & Evaluation Elementary Opening and Introduction to the Course](https://reader031.vdocuments.mx/reader031/viewer/2022013012/56649e9c5503460f94b9d30e/html5/thumbnails/20.jpg)
VAMESP II
Communication process – The Sender
To be an effective communicator Establish credibility Display knowledge of the subject Display knowledge of the audience Know the context of message delivery
Know your audience Messages will be mis-understood if you
do not understand who you are communicating to!!
![Page 21: VAMESP II Monitoring & Evaluation Elementary Opening and Introduction to the Course](https://reader031.vdocuments.mx/reader031/viewer/2022013012/56649e9c5503460f94b9d30e/html5/thumbnails/21.jpg)
VAMESP II
Communication process – The Message
Written, oral and non-verbal communications are affected by:
Sender’s tone Method of organisation Validity of the argument What is communicated and what is left out Your individual style
Messages have intellectual & emotional parts
Intellect gives us ability to reason Emotion allows us to present motivational appeals, which
change minds
![Page 22: VAMESP II Monitoring & Evaluation Elementary Opening and Introduction to the Course](https://reader031.vdocuments.mx/reader031/viewer/2022013012/56649e9c5503460f94b9d30e/html5/thumbnails/22.jpg)
VAMESP II
Communication process – The Channel
Messages are conveyed through channels: Verbal face-to-face meetings Telephone and video-conferencing Written – letters, emails, memos and
reports
![Page 23: VAMESP II Monitoring & Evaluation Elementary Opening and Introduction to the Course](https://reader031.vdocuments.mx/reader031/viewer/2022013012/56649e9c5503460f94b9d30e/html5/thumbnails/23.jpg)
VAMESP II
Communication process – The Receiver
Messages are delivered to an audience Message will prompt actions or reactions
Audience brings ideas and feelings to communication process These influence understanding of the
message A successful communicator considers
these before delivering the message
![Page 24: VAMESP II Monitoring & Evaluation Elementary Opening and Introduction to the Course](https://reader031.vdocuments.mx/reader031/viewer/2022013012/56649e9c5503460f94b9d30e/html5/thumbnails/24.jpg)
VAMESP II
Communication process – Feedback & Context
Audience – provides feedback Verbal and non-verbal reactions
Monitor feedback It measures if the audience is
understanding the message Context – The situation in which the
message is delivered Surrounding environment or broader
culture
![Page 25: VAMESP II Monitoring & Evaluation Elementary Opening and Introduction to the Course](https://reader031.vdocuments.mx/reader031/viewer/2022013012/56649e9c5503460f94b9d30e/html5/thumbnails/25.jpg)
VAMESP II
Removing barriers Communication barriers can emerge
at any stage of the communication process
Aim is to lessen frequency of these barriers at each stage
Communication should be: Clear Concise Accurate Well planned
![Page 26: VAMESP II Monitoring & Evaluation Elementary Opening and Introduction to the Course](https://reader031.vdocuments.mx/reader031/viewer/2022013012/56649e9c5503460f94b9d30e/html5/thumbnails/26.jpg)
VAMESP II
Removing barriers at all stages If lengthy, disorganised or contains
errors: messages will be misinterpreted
Poor verbal or body language: Confuses the message
Barriers in context – come from too much information too fast
Understand your audience culture – converse and deliver your message to the particular audience
![Page 27: VAMESP II Monitoring & Evaluation Elementary Opening and Introduction to the Course](https://reader031.vdocuments.mx/reader031/viewer/2022013012/56649e9c5503460f94b9d30e/html5/thumbnails/27.jpg)
VAMESP II
Communication Models
What is a communication model Drawings Charts Diagrams Pictograms Schematics Cartoons
![Page 28: VAMESP II Monitoring & Evaluation Elementary Opening and Introduction to the Course](https://reader031.vdocuments.mx/reader031/viewer/2022013012/56649e9c5503460f94b9d30e/html5/thumbnails/28.jpg)
VAMESP II
Shannon & Weaver communication model
![Page 29: VAMESP II Monitoring & Evaluation Elementary Opening and Introduction to the Course](https://reader031.vdocuments.mx/reader031/viewer/2022013012/56649e9c5503460f94b9d30e/html5/thumbnails/29.jpg)
VAMESP II
Monitoring & Evaluation
Elementary - Planning for M & E
Section 4
The Logframe, Monitoring Planning and Logistics
![Page 30: VAMESP II Monitoring & Evaluation Elementary Opening and Introduction to the Course](https://reader031.vdocuments.mx/reader031/viewer/2022013012/56649e9c5503460f94b9d30e/html5/thumbnails/30.jpg)
VAMESP II
Logframe and M&E
The Logical Framework (Logframe) is a project planning matrix and a tool of analysis used at all stages of the investment cycle.
![Page 31: VAMESP II Monitoring & Evaluation Elementary Opening and Introduction to the Course](https://reader031.vdocuments.mx/reader031/viewer/2022013012/56649e9c5503460f94b9d30e/html5/thumbnails/31.jpg)
VAMESP II
Logical Framework Logical Framework – Logframe
A basic tool for investment design, implementation and monitoring and evaluation
Analyses existing situation Creates a hierarchy of actions to achieve outcomes Identifies potential risks to achieve outcomes Establishes how outputs and outcomes will be
monitored and evaluated Monitors investments during implementation
A number of names: PDM, LFM, LFA, ZOPP (but basic elements are the
same)
![Page 32: VAMESP II Monitoring & Evaluation Elementary Opening and Introduction to the Course](https://reader031.vdocuments.mx/reader031/viewer/2022013012/56649e9c5503460f94b9d30e/html5/thumbnails/32.jpg)
VAMESP II
Some Logframe Terminology Investment description Narrative summary Goal Purpose Component objective Objective Expected outcomes Objective hierarchy Hierarchy of results Intervention logic
Results Outcomes Outputs Activities Inputs Assumptions Indicators Performance
indicators Means of verification Frequency
![Page 33: VAMESP II Monitoring & Evaluation Elementary Opening and Introduction to the Course](https://reader031.vdocuments.mx/reader031/viewer/2022013012/56649e9c5503460f94b9d30e/html5/thumbnails/33.jpg)
VAMESP II
Logframe Matrix
Description Verifiable Indicators Means of
Verification
Assumptions
Goal
Purpose
Outcomes(Results)
Outputs
Activities
Inputs
Logical Framework Matrix
![Page 34: VAMESP II Monitoring & Evaluation Elementary Opening and Introduction to the Course](https://reader031.vdocuments.mx/reader031/viewer/2022013012/56649e9c5503460f94b9d30e/html5/thumbnails/34.jpg)
VAMESP II
Logical Framework Matrix
Description Verifiable Indicators Means of
Verification
Assumptions
Goal
Purpose
Outcomes(Results)
Outputs
Activities
Inputs
Logical Framework Matrix
![Page 35: VAMESP II Monitoring & Evaluation Elementary Opening and Introduction to the Course](https://reader031.vdocuments.mx/reader031/viewer/2022013012/56649e9c5503460f94b9d30e/html5/thumbnails/35.jpg)
VAMESP II
Developing a Logframe Matrix Is an iterative “step –by-step” process
The description (narrative summary) The assumptions The indicators The means of verification
Look back and test the logic Modify and test again
![Page 36: VAMESP II Monitoring & Evaluation Elementary Opening and Introduction to the Course](https://reader031.vdocuments.mx/reader031/viewer/2022013012/56649e9c5503460f94b9d30e/html5/thumbnails/36.jpg)
VAMESP II
Matrix Logic
![Page 37: VAMESP II Monitoring & Evaluation Elementary Opening and Introduction to the Course](https://reader031.vdocuments.mx/reader031/viewer/2022013012/56649e9c5503460f94b9d30e/html5/thumbnails/37.jpg)
VAMESP II
Linking LFA and Evaluation Criteria
![Page 38: VAMESP II Monitoring & Evaluation Elementary Opening and Introduction to the Course](https://reader031.vdocuments.mx/reader031/viewer/2022013012/56649e9c5503460f94b9d30e/html5/thumbnails/38.jpg)
VAMESP II
Developing a Monitoring Framework
Basic pre-conditions
Guiding the investment strategy for results
Create a learning environment
Ensure effective operations
Develop and use an M&E system
![Page 39: VAMESP II Monitoring & Evaluation Elementary Opening and Introduction to the Course](https://reader031.vdocuments.mx/reader031/viewer/2022013012/56649e9c5503460f94b9d30e/html5/thumbnails/39.jpg)
VAMESP II
The key steps in monitoring planning
Five steps Establish the purpose and the scope Identify performance questions, information
needs and indicators Planning information gathering and organization Planning critical reflection processes and events Planning the necessary conditions and capacities
![Page 40: VAMESP II Monitoring & Evaluation Elementary Opening and Introduction to the Course](https://reader031.vdocuments.mx/reader031/viewer/2022013012/56649e9c5503460f94b9d30e/html5/thumbnails/40.jpg)
VAMESP II
Contents of a monitoring plan Purpose and scope Approach Revised Logical framework and indicators Management information system and
reporting Monitoring work plan and timing of activities Establishing conditions and capacities Monitoring budget Annexes
![Page 41: VAMESP II Monitoring & Evaluation Elementary Opening and Introduction to the Course](https://reader031.vdocuments.mx/reader031/viewer/2022013012/56649e9c5503460f94b9d30e/html5/thumbnails/41.jpg)
VAMESP II
Summary monitoring framework
Logframe Reference
Indicator
What We will measure
How We will measure
Who will measue
Frequency of measurement
How results will be reported
Goal
Purpose
Outcomes
Outputs
Activities
Inputs
![Page 42: VAMESP II Monitoring & Evaluation Elementary Opening and Introduction to the Course](https://reader031.vdocuments.mx/reader031/viewer/2022013012/56649e9c5503460f94b9d30e/html5/thumbnails/42.jpg)
VAMESP II
Maintaining monitoring quality
Update the system to meet changing needs
Usefulness – M&E System provides practical information
Feasibility – methods and procedures are realistic and cost effective
Propriety – M&E activities are conducted legally, ethically and regard for welfare of those affected by the results
Accuracy – M&E outputs provide accurate information to support effective and efficient management
![Page 43: VAMESP II Monitoring & Evaluation Elementary Opening and Introduction to the Course](https://reader031.vdocuments.mx/reader031/viewer/2022013012/56649e9c5503460f94b9d30e/html5/thumbnails/43.jpg)
VAMESP II
Monitoring & Evaluation
Elementary - Indicators
Section 5
Types of Indicators
SMART characteristics
![Page 44: VAMESP II Monitoring & Evaluation Elementary Opening and Introduction to the Course](https://reader031.vdocuments.mx/reader031/viewer/2022013012/56649e9c5503460f94b9d30e/html5/thumbnails/44.jpg)
VAMESP II
Selecting indicators for monitoring
Indicators are “signposts” of change Indicators establish the measures on which
the monitoring system may be built. To select indicators it is necessary to identify
what information is needed to answer the performance questions.
What are performance questions??
![Page 45: VAMESP II Monitoring & Evaluation Elementary Opening and Introduction to the Course](https://reader031.vdocuments.mx/reader031/viewer/2022013012/56649e9c5503460f94b9d30e/html5/thumbnails/45.jpg)
VAMESP II
Performance Questions A performance question helps focus
monitoring activities on what is necessary to know if the investment is performing as planned and why that is so. Ask this question
“What question would we need to answer to know the extent to which an investment is achieving the objective and to explain the success or failure”
![Page 46: VAMESP II Monitoring & Evaluation Elementary Opening and Introduction to the Course](https://reader031.vdocuments.mx/reader031/viewer/2022013012/56649e9c5503460f94b9d30e/html5/thumbnails/46.jpg)
VAMESP II
Performance questions are not complex!
Logical Framework
Basic Performance Questions
Inputs What resources has the ODA investment actually procured and used?
Activities
What has the ODA investment actually done?
Outputs What has the ODA investment delivered as a result of its activities?
Outcomes
What has been achieved as a result of the outputs?
Purpose What has been learned from implementation of the ODA investment than can contribute to improved implementation or to building knowledge.
![Page 47: VAMESP II Monitoring & Evaluation Elementary Opening and Introduction to the Course](https://reader031.vdocuments.mx/reader031/viewer/2022013012/56649e9c5503460f94b9d30e/html5/thumbnails/47.jpg)
VAMESP II
Criteria and Indicators
provide the objective and measurable basis on which the assessment of performance and achievement can be analysed and allow a rigorous assessment on which judgements can be based in drawing conclusions and recommendations from the monitoring exercise.
they should be developed in conjunction with the stakeholders involved (including participation from the community) and may incorporate both quantitative and qualitative assessments.
![Page 48: VAMESP II Monitoring & Evaluation Elementary Opening and Introduction to the Course](https://reader031.vdocuments.mx/reader031/viewer/2022013012/56649e9c5503460f94b9d30e/html5/thumbnails/48.jpg)
VAMESP II
Principles in defining and selecting indicators
Do not remain static
Should complement existing systems
Developed in partnership with stakeholders
Simple and Cost effective to measure
Linked across scale Practical and objectively verifiable
![Page 49: VAMESP II Monitoring & Evaluation Elementary Opening and Introduction to the Course](https://reader031.vdocuments.mx/reader031/viewer/2022013012/56649e9c5503460f94b9d30e/html5/thumbnails/49.jpg)
VAMESP II
Characteristics of indicatorsTo be fully effective and usable
indicators need to be SMART:
Simple Measurable Attributable Relevant Timely
![Page 50: VAMESP II Monitoring & Evaluation Elementary Opening and Introduction to the Course](https://reader031.vdocuments.mx/reader031/viewer/2022013012/56649e9c5503460f94b9d30e/html5/thumbnails/50.jpg)
VAMESP II
Characteristics of Indicators
To be strategically meaningful indicators need to be clearly specified in terms of:
Inputs
Outputs
Outcomes
Impact
![Page 51: VAMESP II Monitoring & Evaluation Elementary Opening and Introduction to the Course](https://reader031.vdocuments.mx/reader031/viewer/2022013012/56649e9c5503460f94b9d30e/html5/thumbnails/51.jpg)
VAMESP II
Types (Signposts) of Change
Type of Change Example of ChangeThe presence of something
Number of restored district health centres
The type of access to a new innovation or service
The level of use
The extent of an activity or coverage
The relevance of the new innovation
The quality of an innovation
The effort required to achieve change
![Page 52: VAMESP II Monitoring & Evaluation Elementary Opening and Introduction to the Course](https://reader031.vdocuments.mx/reader031/viewer/2022013012/56649e9c5503460f94b9d30e/html5/thumbnails/52.jpg)
VAMESP II
Key types of indicators Quantitative
Easily quantified – measured by defined numerical values
kilometers of road constructed Number of commune primary schools provided
Qualitative Not as easy to measure with numerical values.
Narrative descriptions may be necessary Degree of participation of women Minutes of farmers meetings Perceptions of well being
![Page 53: VAMESP II Monitoring & Evaluation Elementary Opening and Introduction to the Course](https://reader031.vdocuments.mx/reader031/viewer/2022013012/56649e9c5503460f94b9d30e/html5/thumbnails/53.jpg)
VAMESP II
Categories of indicators
Leading indicators Measuring outputs/outcomes and performance drivers
Process indicators Routine data and measures used for attributing results
Lagging indicators Measuring final outcomes / impacts
![Page 54: VAMESP II Monitoring & Evaluation Elementary Opening and Introduction to the Course](https://reader031.vdocuments.mx/reader031/viewer/2022013012/56649e9c5503460f94b9d30e/html5/thumbnails/54.jpg)
VAMESP II
How to define indicators Decide on what to measure and why ?
Specify for each activity, intended result, or goal
Decide what is to be measured The quantity……..how much? The quality ………what? The target group…….who? The time/period……..starting when and for how long? The place………where?
Check whether the indicator(s) describe the overall purpose, results and activities accurately, If not, other indicators will have to be added, new ones found, or proxy measures used.
![Page 55: VAMESP II Monitoring & Evaluation Elementary Opening and Introduction to the Course](https://reader031.vdocuments.mx/reader031/viewer/2022013012/56649e9c5503460f94b9d30e/html5/thumbnails/55.jpg)
VAMESP II
What, How, Who, When
Logframe Reference
Indicator
What We will measure
How We will measure
Who will measue
Frequency of measurement
How results will be reported
Goal
Purpose
Outcomes
Outputs
Activities
Inputs
![Page 56: VAMESP II Monitoring & Evaluation Elementary Opening and Introduction to the Course](https://reader031.vdocuments.mx/reader031/viewer/2022013012/56649e9c5503460f94b9d30e/html5/thumbnails/56.jpg)
VAMESP II
The essential questions?? What to monitor and why?
How to obtain relevant and worthwhile information?
Who may be expected to use it and for what purpose?
![Page 57: VAMESP II Monitoring & Evaluation Elementary Opening and Introduction to the Course](https://reader031.vdocuments.mx/reader031/viewer/2022013012/56649e9c5503460f94b9d30e/html5/thumbnails/57.jpg)
VAMESP II
Types of Change
Type of Change Example of indicatorThe presence of something Number of restored district health centres
The type of access to a new innovation or service
Number of primary-aged school children with access to commune primary schools
The level of use The frequency of traffic use of new commune road
The extent of an activity or coverage
The proportion of mothers in the poorest 20% of households with access to village health workers
The relevance of the new innovation
Do seed banks resolve a production constraint or not
The quality of an innovation The quality of health services from restored District health Centres
The effort required to achieve change
The labour required for a new soil management technique
![Page 58: VAMESP II Monitoring & Evaluation Elementary Opening and Introduction to the Course](https://reader031.vdocuments.mx/reader031/viewer/2022013012/56649e9c5503460f94b9d30e/html5/thumbnails/58.jpg)
VAMESP II
Monitoring & Evaluation
Elementary – Data for Monitoring
Section 6
Baseline Data and Collection Methods
![Page 59: VAMESP II Monitoring & Evaluation Elementary Opening and Introduction to the Course](https://reader031.vdocuments.mx/reader031/viewer/2022013012/56649e9c5503460f94b9d30e/html5/thumbnails/59.jpg)
VAMESP II
The Data Journey
Source: IFAD 2002
![Page 60: VAMESP II Monitoring & Evaluation Elementary Opening and Introduction to the Course](https://reader031.vdocuments.mx/reader031/viewer/2022013012/56649e9c5503460f94b9d30e/html5/thumbnails/60.jpg)
VAMESP II
Data and Information
Data is raw material and has no meaning on its own
Synthesis and analysis of data produces information
Information explains and provides lessons which emerge as knowledge
![Page 61: VAMESP II Monitoring & Evaluation Elementary Opening and Introduction to the Course](https://reader031.vdocuments.mx/reader031/viewer/2022013012/56649e9c5503460f94b9d30e/html5/thumbnails/61.jpg)
VAMESP II
Types of Data and analysis Quantitative data
Analysis involves calculations – totals, average numbers, percentages compared to plan or targets. Can be more complex frequency, regression and cost-benefit analysis.
Qualitative data Analysis is very different and more difficult
than quantitative data. Can involve opinions, photo-points, stakeholder perceptions and non-standard answers
![Page 62: VAMESP II Monitoring & Evaluation Elementary Opening and Introduction to the Course](https://reader031.vdocuments.mx/reader031/viewer/2022013012/56649e9c5503460f94b9d30e/html5/thumbnails/62.jpg)
VAMESP II
Methods of comparison
“Before” and “after” Data gathered at the beginning, mid-term and
at completion
“With” and “without” (control) Compare data from locations with investment
activities and from locations without investment activities
“Variance from plan” Compare the actual performance or results with
the planned performance or results
![Page 63: VAMESP II Monitoring & Evaluation Elementary Opening and Introduction to the Course](https://reader031.vdocuments.mx/reader031/viewer/2022013012/56649e9c5503460f94b9d30e/html5/thumbnails/63.jpg)
VAMESP II
Sample analytical tool to transform data
GoV Participants Meeting Competency Criteria
05
1015
2025
3035
4045
H1-03
H2-03
H1-04
H2-04
H1-05
H2-05
H1-06
H2-06
H1-07
H2-07
H1-08
H2-08
Program half year period
GoV
Sta
ff P
artic
ipan
ts
(No.
)
-10.0%
10.0%
30.0%
50.0%
70.0%
90.0%
110.0%
GoV
Sta
ff P
artic
ipan
ts
(%)
Total participating 1/
No. meetingcompetency criteria 2/
% competent
Target
![Page 64: VAMESP II Monitoring & Evaluation Elementary Opening and Introduction to the Course](https://reader031.vdocuments.mx/reader031/viewer/2022013012/56649e9c5503460f94b9d30e/html5/thumbnails/64.jpg)
VAMESP II
Example – Comparing Activities
Activities Conducted Jan 2004 - Dec 2006
0
10
20
30
40
50
60
H1-2004 H2-2004 H1-2005 H2-2005 H1-2006 H2-2006
VAMESP Half-Year Period
Num
ber o
f Act
ivitie
s
Formal Training
OTJ Training Events
PCC Meetings
PMC Meetings
S H Consultations
TWG Meetings
Workshops
Study tours
Field Visits
![Page 65: VAMESP II Monitoring & Evaluation Elementary Opening and Introduction to the Course](https://reader031.vdocuments.mx/reader031/viewer/2022013012/56649e9c5503460f94b9d30e/html5/thumbnails/65.jpg)
VAMESP II
Monitoring & Evaluation
Elementary - Reporting
Section 10
Audience, Communication, Presentation & Interpretation
![Page 66: VAMESP II Monitoring & Evaluation Elementary Opening and Introduction to the Course](https://reader031.vdocuments.mx/reader031/viewer/2022013012/56649e9c5503460f94b9d30e/html5/thumbnails/66.jpg)
VAMESP II
Communication of findings
Why communicate M&E results Many potential audiences:
Government Funding agencies Steering Committees Cooperating institutions Implementing partners Primary stakeholders
Funders and managers need information on impact Partners and stakeholders want to understand problems
in order to find solutions
![Page 67: VAMESP II Monitoring & Evaluation Elementary Opening and Introduction to the Course](https://reader031.vdocuments.mx/reader031/viewer/2022013012/56649e9c5503460f94b9d30e/html5/thumbnails/67.jpg)
VAMESP II
Sharing Results of M&E
Build value throughout the M&E process to arrive at an informative, useful ‘story’ that makes an effective contribution to development cooperation
![Page 68: VAMESP II Monitoring & Evaluation Elementary Opening and Introduction to the Course](https://reader031.vdocuments.mx/reader031/viewer/2022013012/56649e9c5503460f94b9d30e/html5/thumbnails/68.jpg)
VAMESP II
Sharing results of findings
Develop a strategy for sharing results that optimizes the benefits to be derived from the monitoring & evaluation
Meet the information needs of key audiences while respecting the organisation’s agenda and context for development
![Page 69: VAMESP II Monitoring & Evaluation Elementary Opening and Introduction to the Course](https://reader031.vdocuments.mx/reader031/viewer/2022013012/56649e9c5503460f94b9d30e/html5/thumbnails/69.jpg)
VAMESP II
Communication opportunities
Carefully consider all opportunities for communicating results and be creative in devising a strategy that enhances the potential for making a ‘real’ contribution both within the organisation and externally.
![Page 70: VAMESP II Monitoring & Evaluation Elementary Opening and Introduction to the Course](https://reader031.vdocuments.mx/reader031/viewer/2022013012/56649e9c5503460f94b9d30e/html5/thumbnails/70.jpg)
VAMESP II
Communication principles
Identify the audience (use an audience matrix)
Integrate communication into M&E plan
Consider communication an investment
Communication requires feedback
![Page 71: VAMESP II Monitoring & Evaluation Elementary Opening and Introduction to the Course](https://reader031.vdocuments.mx/reader031/viewer/2022013012/56649e9c5503460f94b9d30e/html5/thumbnails/71.jpg)
VAMESP II
Communication and M&E Planning
Source: IFAD 2002
![Page 72: VAMESP II Monitoring & Evaluation Elementary Opening and Introduction to the Course](https://reader031.vdocuments.mx/reader031/viewer/2022013012/56649e9c5503460f94b9d30e/html5/thumbnails/72.jpg)
VAMESP II
Audience Matrix
Types of Monitoring Information
Types of
Audiences
Progress
toward
goal
Achieve-
ments
Economic
Impacts
Follow-up
Decisions
Actions
Community Members H M M L M
Project Management Unit Staff
M H L H H
Provincial Agency M H L H M
Ministry H L H H M
Funding Agencies H M H H M
Other groups L M L L L
Priority Code H = High M = Medium L = Low
Derived from: IFAD (2002)
![Page 73: VAMESP II Monitoring & Evaluation Elementary Opening and Introduction to the Course](https://reader031.vdocuments.mx/reader031/viewer/2022013012/56649e9c5503460f94b9d30e/html5/thumbnails/73.jpg)
VAMESP II
Practical Presentation of Information
Clarity of message for specific finding Frequency of communication Timeliness of feedback Consider location of the audience Effective use of graphics for analysis Keep feedback sessions focused
![Page 74: VAMESP II Monitoring & Evaluation Elementary Opening and Introduction to the Course](https://reader031.vdocuments.mx/reader031/viewer/2022013012/56649e9c5503460f94b9d30e/html5/thumbnails/74.jpg)
VAMESP II
Communication Media
Written reporting Qtrly, 6 mthly, annual & mid term reports,
newsletters Oral reporting
Direct discussion, radio programs Visual display
Graphs, charts, maps, photographs, video, drama
Electronic communication Email, electronic newsletters, websites
![Page 75: VAMESP II Monitoring & Evaluation Elementary Opening and Introduction to the Course](https://reader031.vdocuments.mx/reader031/viewer/2022013012/56649e9c5503460f94b9d30e/html5/thumbnails/75.jpg)
VAMESP II
Example – Interpreting data – Baseline in Time Series
GoV Participants Meeting Competency Criteria
05
1015
2025
3035
4045
H1-03
H2-03
H1-04
H2-04
H1-05
H2-05
H1-06
H2-06
H1-07
H2-07
H1-08
H2-08
Program half year period
GoV
Sta
ff P
artic
ipan
ts
(No.
)
-10.0%
10.0%
30.0%
50.0%
70.0%
90.0%
110.0%
GoV
Sta
ff P
artic
ipan
ts
(%)
Total participating 1/
No. meetingcompetency criteria 2/
% competent
Target
![Page 76: VAMESP II Monitoring & Evaluation Elementary Opening and Introduction to the Course](https://reader031.vdocuments.mx/reader031/viewer/2022013012/56649e9c5503460f94b9d30e/html5/thumbnails/76.jpg)
VAMESP II
Example – Interpreting data – Variance from plan
Plan VS Actual Disbursement - VAMESP II (A$)
$0
$200,000
$400,000
$600,000
$800,000
$1,000,000
$1,200,000
$1,400,000
$1,600,000
$1,800,000
$2,000,000
J an Feb Mar Apr May J un J ul Aug Sep Oct Nov Dec J an Feb
Plan
Actual
![Page 77: VAMESP II Monitoring & Evaluation Elementary Opening and Introduction to the Course](https://reader031.vdocuments.mx/reader031/viewer/2022013012/56649e9c5503460f94b9d30e/html5/thumbnails/77.jpg)
VAMESP II
Example – Interpreting data – Comparing types of activities
Activities Conducted Jan 2004 - Dec 2006
0
10
20
30
40
50
60
H1-2004 H2-2004 H1-2005 H2-2005 H1-2006 H2-2006
VAMESP Half-Year Period
Num
ber o
f Act
ivitie
s
Formal Training
OTJ Training Events
PCC Meetings
PMC Meetings
S H Consultations
TWG Meetings
Workshops
Study tours
Field Visits
![Page 78: VAMESP II Monitoring & Evaluation Elementary Opening and Introduction to the Course](https://reader031.vdocuments.mx/reader031/viewer/2022013012/56649e9c5503460f94b9d30e/html5/thumbnails/78.jpg)
VAMESP II
Example – Interpreting data – Comparing progress against parameters
00.5
1
1.52
2.5
33.5
4Efficiency
Effectiveness
ImpactRelevance
Sustainability
![Page 79: VAMESP II Monitoring & Evaluation Elementary Opening and Introduction to the Course](https://reader031.vdocuments.mx/reader031/viewer/2022013012/56649e9c5503460f94b9d30e/html5/thumbnails/79.jpg)
VAMESP II
Monitoring & Evaluation
Elementary – Results Based Management
Section 12
Concepts and Definitions
![Page 80: VAMESP II Monitoring & Evaluation Elementary Opening and Introduction to the Course](https://reader031.vdocuments.mx/reader031/viewer/2022013012/56649e9c5503460f94b9d30e/html5/thumbnails/80.jpg)
VAMESP II
What is RBM
Results Based Management (RBM) is a management philosophy and approach that emphasizes development results in planning, implementation, learning and reporting.
![Page 81: VAMESP II Monitoring & Evaluation Elementary Opening and Introduction to the Course](https://reader031.vdocuments.mx/reader031/viewer/2022013012/56649e9c5503460f94b9d30e/html5/thumbnails/81.jpg)
VAMESP II
RBM needs participation Participation is an important ingredient
of RBM. Experience shows that participation improves the quality, effectiveness and sustainability of development actions.
RBM involves the participation of stakeholders in the: planning of projects & programmes; implementation of projects; monitoring and reporting of results.
![Page 82: VAMESP II Monitoring & Evaluation Elementary Opening and Introduction to the Course](https://reader031.vdocuments.mx/reader031/viewer/2022013012/56649e9c5503460f94b9d30e/html5/thumbnails/82.jpg)
VAMESP II
Results Based Management – Definition
Results Based Management is defined as: “A management strategyy focussing on
perfromance and achievement of outputs, outcomes and impacts “ (OECD-DAC 2002)
A “Result” is also defined by OECD-DAC as: “The output, outcome or impact [intended or
unintended, positive and/or negative] of a development intervention”
![Page 83: VAMESP II Monitoring & Evaluation Elementary Opening and Introduction to the Course](https://reader031.vdocuments.mx/reader031/viewer/2022013012/56649e9c5503460f94b9d30e/html5/thumbnails/83.jpg)
VAMESP II
Managing “Outputs” vs “Outcomes”
“Administrative” management Focuses on managing inputs,
activities and production of “outputs”
“Results based” management Focuses on management towards the
achievement of outcomes
![Page 84: VAMESP II Monitoring & Evaluation Elementary Opening and Introduction to the Course](https://reader031.vdocuments.mx/reader031/viewer/2022013012/56649e9c5503460f94b9d30e/html5/thumbnails/84.jpg)
VAMESP II
Comparing “administrative” and “results based” management
Administrative Results BasedDescription of situation before project start
Baseline data to define the situation before project start
Benchmarks for activities and outputs
Indicators prepared for outcomes
Collect data on imputs, activities and immediate outputs
Collect data on outputs – are they contributing to outcomes
Routine reporting on provision of inputs and outputs
More focus on perceptions of change among stakeholders
Directly linked to a discrete project
More qualitative & quantitiative analysis of progress to outcomes
Provides information on administrative, routine implementation and management issues
Project and strategic partners look at broader development effectiveness
![Page 85: VAMESP II Monitoring & Evaluation Elementary Opening and Introduction to the Course](https://reader031.vdocuments.mx/reader031/viewer/2022013012/56649e9c5503460f94b9d30e/html5/thumbnails/85.jpg)
VAMESP II
Good management Good management still requires:
The recording and monitoring of inputs and outputs; but
Recognises: That outcomes also must be monitored to
measure the success of a development activity. Outcome monitoring is a continual and systematic
process of collecting and analysing data to measure performance of ODA investments
![Page 86: VAMESP II Monitoring & Evaluation Elementary Opening and Introduction to the Course](https://reader031.vdocuments.mx/reader031/viewer/2022013012/56649e9c5503460f94b9d30e/html5/thumbnails/86.jpg)
VAMESP II
Key RBM Concepts RBM seeks to capture the process of change in the short,
medium and long-term. Development results are commonly linked together in a
results chain. Short-term results or outputs will contribute to medium-term results or outcomes which in turn will contribute to the long-term results or impact.
Completed activities are not short-term results or outputs. Outputs are the actual effects of completed activities.
RBM should identify the links between gender equality issues and the results the project seeks to achieve.
Beneficiary Reach refers to individual, groups or organizations affected by or benefiting from the results.
Results are influenced by the socio-economic context, gender analysis, the local capacity of stakeholders and organizations, the participation of stakeholders, available resources and the programme/project duration.