Three Days TOT on Results-Based Management
(RBM)for M&E Staff – ITA
By
Fida Karim Head of Monitoring & Evaluation (M&E)
Dated: 10th to 12th August, 2015
Planning, Monitoring, Evaluation & Reporting together for developmental results: Results-based Management-RBM (RBM)?
Logical Framework Approach (LFA) Planning for results Monitoring for results Evaluating for results Enhancing the use of knowledge from monitoring
and evaluation
Three Days TOT On Result Based Management for M&E Staff; Conducted by : Mr. Fida Karim – Head of M&E Department ITA 2
Day 1: Major Topics To Be Covered
Tuesday, May 2, 2023
RBM is a management approach and project Planning, Monitoring, Evaluation & Reporting tool aimed at to improve management effectiveness and accountability in achieving developmental results.
RBM is focused on chain results: output, outcomes, and impact.
RBM objectively measure how well results are being achieved and report on measures taken to improve them.
It enable senior management and Project Management team to judge if work is going in the right direction, whether progress and success can be claimed, and how future efforts might be improved on the basis of ground realities and data.
RBM is concerned with learning, risk management and accountability.
Three Days TOT On Result Based Management for M&E Staff; Conducted by : Mr. Fida Karim – Head of M&E Department ITA 3
What is Results-Based Management (RBM)?
Tuesday, May 2, 2023
Tuesday, May 2, 2023Three Days TOT On Result Based Management for M&E Staff; Conducted by : Mr. Fida Karim – Head of M&E Department ITA 4
• Without proper planning and clear articulation of intended results, it is not clear what should be monitored and how; hence monitoring cannot be done well.• Without effective planning (clear results frameworks), the basis for evaluation is weak; hence evaluation cannot be done well.• Without careful monitoring, the necessary data is not collected; hence evaluation cannot be done well.• Monitoring is necessary, but not sufficient, for evaluation.• Monitoring facilitates evaluation, but evaluation uses additional new data collection and different frameworks for analysis.• Monitoring and evaluation of a programme will often lead to changes in programme plans. This may mean further changing or modifying data collection for monitoring purposes.
Understanding inter-linkages and dependencies between Planning, Monitoring and Evaluation
Tuesday, May 2, 2023Three Days TOT On Result Based Management for M&E Staff; Conducted by : Mr. Fida Karim – Head of M&E Department ITA 5
• Traditional M&E mainly focuses on inputs & Activities and success is measured by expenditure (e.g. profit maximization not social welfare maximization)
• Result Based Management (RMB) focuses on internal results and performance of organization in producing developmental results and positive changes in society. •Management for Developmental Results (MfDR) seeks to keep the focus on development assistance demonstrating real and meaningful results at targeted community level.
• RBM & MfDR focuses on good planning, monitoring, evaluation, learning and feeding back into planning to bring continuous improvement in quality of service delivery.
Traditional M&E Vs RBM vs MfDR
Three Days TOT On Result Based Management for M&E Staff; Conducted by : Mr. Fida Karim – Head of M&E Department ITA 6Tuesday, May 2, 2023
Problem analysis to understand causes
Stakeholder analysis
Structuring of programmes around a chain of desired results - addressing causes
Causality in the chain of results (if… then logic)
Use of ‘change language’ (future conditional)
Reliance on indicators to measure performance
Costing of results rather than isolated activity budgeting
Three Days TOT On Result Based Management for M&E Staff 7
Common elements of RBM
Three Days TOT On Result Based Management for M&E Staff 8
A Typology for RBM
HIV incidence reduced
Leadership empowered
Skills of Nat’l Aids Comm. strengthened
Train 250 district AIDS officers
Results Like… Focus @
Timeframe
<1 yr
<3 yrs
5 yrs
5-10 yrs
more
less
Collective
Accountability Institutional/
Behavioural
Operational/ skills, abilities,
products & services
Human!
UNOutcome
Impact
Output
Activity
if
if
if
then
then
then
Assumptio
nsUNOutcome
Impact
Output
Activity
Outcome
Impact
Output
Activity
Three Days TOT On Result Based Management for M&E Staff; Conducted by : Mr. Fida Karim – Head of M&E Department ITA 9
Key Features of RBMFocus on:Analyzing problems and determining their causes;Identifying measurable changes (results) to be achieved based on problem analysis;Designing strategies and activities that
will lead to these changes (results);Balancing expected results with the resources available;
Tuesday, May 2, 2023
Three Days TOT On Result Based Management for M&E Staff; Conducted by : Mr. Fida Karim – Head of M&E Department ITA 10
Key Features of RBM
Monitoring progress regularly and adjusting activities to ensure
results are achievedEvaluating, documenting and incorporating lessons learned into next planning phase;Reporting on the results achieved and their contribution to achieving goals;
Tuesday, May 2, 2023
Three Days TOT On Result Based Management for M&E Staff; Conducted by : Mr. Fida Karim – Head of M&E Department ITA 11
Why Results-Based Management?Why Results-Based Management? Improved focus on results instead of activities Improved transparency, quality & accountability Improved measurement of programme
achievements (performance rather than utilization)
Enhanced strategic focus Industry standard To get more funds!! It is a global trend: using results-based
management to improve the efficiency of development program
Tuesday, May 2, 2023
RESULTSare measurable changes in knowledge, skills, motivation, behavior, decision making, practices, policies, social action, social, economic and environmental conditions that derive from a cause-and-effect relationship. There are three types of such changes:1. Intended or unintended, 2. Positive or negative3. Direct or indirect.Change can be set in motion by a development intervention – outputs, outcomes and impacts.
.
What is Result?*C
ON
TRO
L & R
ESPOSIB
ILITY
MA
KIN
G A
DIFFER
ENC
E
Three Days TOT On Result Based Management for M&E Staff; Conducted by : Mr. Fida Karim – Head of M&E Department ITA 13
Key RBM Terms (iii)Key RBM Terms (iii)
Inputs
Activities
Outputs Outcomes Impact
How?
What we want? Why?
Tuesday, May 2, 2023
Results
Three Days TOT On Result Based Management for M&E Staff; Conducted by : Mr. Fida Karim – Head of M&E Department ITA 14
Definitions Definitions (Implementation) (Implementation)
InputInput: these are human, material, financial : these are human, material, financial and other resources that are required to and other resources that are required to undertake activities. undertake activities.
ActivitiesActivities: Actions taken or work : Actions taken or work performed to produce specific outputs performed to produce specific outputs through mobilizing inputs.through mobilizing inputs.
Tuesday, May 2, 2023
Three Days TOT On Result Based Management for M&E Staff; Conducted by : Mr. Fida Karim – Head of M&E Department ITA 15
Definitions (Results)Definitions (Results) Output: Output: these are immediate results as a these are immediate results as a
consequences of completed activitiesconsequences of completed activities OutcomeOutcome: these are likely or achieved : these are likely or achieved
short-term and medium-term effects of short-term and medium-term effects of outputs.outputs.
Impact:Impact: long-term effects that are the long-term effects that are the logical consequence of the achievement logical consequence of the achievement of the outcomes.of the outcomes.
Tuesday, May 2, 2023
Three Days TOT On Result Based Management for M&E Staff; Conducted by : Mr. Fida Karim – Head of M&E Department ITA 16
Input? Activity? Output? Outcome? Impact?
Splash (cause)
Ripple(effect)
Tuesday, May 2, 2023
Three Days TOT On Result Based Management for M&E Staff; Conducted by : Mr. Fida Karim – Head of M&E Department ITA 17
Key principles of RBMKey principles of RBM Define expected results first and Define expected results first and
activities lateractivities later Foster the active participation of Foster the active participation of
stakeholdersstakeholders Ensure that all stakeholders work Ensure that all stakeholders work
towards achieving expected towards achieving expected resultsresults
Apprise your work critically and Apprise your work critically and learn the lessonslearn the lessons
Tuesday, May 2, 2023
Three Days TOT On Result Based Management for M&E Staff; Conducted by : Mr. Fida Karim – Head of M&E Department ITA 18
Hierarchy of RBM TermsHierarchy of RBM TermsIMPACTIMPACT
Sustainable improvements in society or well-Sustainable improvements in society or well-being of peoplebeing of people
OUTCOMEOUTCOMEChanges in behavior or improvements in access Changes in behavior or improvements in access
or quality of resourcesor quality of resourcesOUTPUTOUTPUT
Product of project/ program activitiesProduct of project/ program activitiesACTIVITIESACTIVITIES
Activities done by project/programActivities done by project/programINPUTINPUT
Resources needed to undertake activitiesResources needed to undertake activities
Tuesday, May 2, 2023
The LFA is an RBM tool used for systematic planning, implementing, monitoring, and evaluating projects/ programmes.
Three Days TOT On Result Based Management for M&E Staff; Conducted by : Mr. Fida Karim – Head of M&E Department ITA 19
RBM and Logical Framework Approach (LFA)
Tuesday, May 2, 2023
The Logical Framework Approach (LFA)
Three Days TOT On Result Based Management for M&E Staff; Conducted by : Mr. Fida Karim – Head of M&E Department ITA 20
Features of LFAFeatures of LFA : : stakeholder involvementstakeholder involvement needs-based approachneeds-based approach logical intervention approachlogical intervention approach framework for assessing framework for assessing
relevance, feasibility and relevance, feasibility and sustainabilitysustainability
Tuesday, May 2, 2023
Three Days TOT On Result Based Management for M&E Staff; Conducted by : Mr. Fida Karim – Head of M&E Department ITA 21
The Logical Framework The Logical Framework Approach Approach
Features of LFAFeatures of LFA : :results-oriented – not activity
drivenlogically sets objectives and their
causal relationshipsshows whether objectives have
been achieved: Indicators (for M&E)
describes external factors that influence the project’s success: assumptions and risks Tuesday, May 2, 2023
Three Days TOT On Result Based Management for M&E Staff; Conducted by : Mr. Fida Karim – Head of M&E Department ITA 22
The Result/Logical frame The Result/Logical frame MatrixMatrix
The Logical Framework Matrix is used to The Logical Framework Matrix is used to present information about project present information about project objectives, outputs and activities in a objectives, outputs and activities in a systematic and logical way. systematic and logical way.
The basic Logframe matrix contains 16 The basic Logframe matrix contains 16 cells organized into 4 columns and 4 rows, cells organized into 4 columns and 4 rows, as indicated in the next slide:as indicated in the next slide:
Tuesday, May 2, 2023
Three Days TOT On Result Based Management for M&E Staff; Conducted by : Mr. Fida Karim – Head of M&E Department ITA 23
The Logical Framework Matrix (LFM)Objectives &
activities
Purpose/ (Outcome)
Goal (Impact)
Outputs
Activities
Indicators Means of verification Assumptions
Tuesday, May 2, 2023
Ways how to depict the Results chain
Logframe
Three Days TOT On Result Based Management for M&E Staff 24
NarrativeIMPACTA. XXXXXXXXOUTCOMEA.1. XXXXXXXXA.2. XXXXXXX XXXOUTPUTSA.1.1.XXX XXX XXXXA.1.2. XXXXXXXA.2.1. XXXXXXXXACTIVITIESA.1.1.XXX XXX XXXXA.1.2. XXXXXXXA.2.1. XXXXXXXX
Results Tree Results Pyramid
Results Chain
Three Days TOT On Result Based Management for M&E Staff 25
Results not logically linked Results not sufficiently specific Results are composites of several results Results don’t express change (e.g. support
provided to strengthen….) Results statements are too wordy Confusion between levels of results Indicators
Not logically linked to the resultNot measurableAre new results
Three Days TOT On Result Based Management for M&E Staff 26
Results chain problems
They can Clarify the scale and scope of a result in the results framework Demonstrate progress when things go right Provide early warning when things go wrong Assist in identifying changes that need to be made in strategy and practice Inform decision making Facilitate effective evaluation
Indicators describe how the intended results will be measured - accountability
Objectively verifiable, repeatable measures of a particular condition
They force clarification of what is meant by the result …….the fine print!
Must be accompanied by baselines and targets
Three Days TOT On Result Based Management for M&E Staff 27
Indicators ‘indicate’ that change is happening or not happening
1. A key test in ensuring whether an indicator is really good is to define “means of verification” for each indicator;
2. Are data sources available?
3. What does it take to obtain the data? Who needs to be involved? How much will it cost to obtain the data?
4. If means of verification are unclear or unrealistic, the indicator and, possibly, the result need to be revisited;
5. The definition of results, indicators and means of verification is an iterative process.
Three Days TOT On Result Based Management for M&E Staff 28
Indicators and Means of Verification
They should be SMART:SPECIFIC The indicator needs to be as specific as possible in terms of quantity, quality, time, location, target groups, baseline and target for the indicatorMEASURABLE Will the indicator show desirable change? Is it a reliable and clear measure of results? Is it sensitive to changes in policies & programmes? Do stakeholders agree on exactly what to measure?ACHIEVABLE Are the result(s) realistic and based on risk assessment, partnership strategy and other factors contributing to the underlying resultRELEVANT Is it relevant to the intended result? Does it reflect the expectations and success criteria for change in the target groups?TRACKABLE/Time-boundResults are never open-ended—there is an expected date of accomplishment. Therefore, indicator should be trackable.
Three Days TOT On Result Based Management for M&E Staff 29
What are good indicators?
Unit of analysis; Existing baseline information; Target for subsequent comparison; Expected perceptions or judgments of
progress by stakeholders; Detailed description of expected conditions
or situations to be observed; 1-4 indicators for each result are adequate
– fewer the better to reduce cost.
Three Days TOT On Result Based Management for M&E Staff30
What to specify for each indicator?
Types of indicatorsFactual indicators Factual (yes/no)
Existence (yes/no) Classes (x/y/z)
Policy recommendation submittedConstitution passed by ParliamentChamber of Commerce established Existence of free electronic media: free, partly free, not free
Numeric indicators Number
Percentage
Ratio
No. of government officials trainedNo. of regional networks on aid effectiveness createdNo. of regional CSOs attending regional conference on human rights
% of government budget devoted to social sectors% of population with access to basic health care
Ratio of female to male school enrolmentRatio of doctors per 1.000 people
Three Days TOT On Result Based Management for M&E Staff 31
Qualitative indicatorsProcess changeBehavioral changesAttitudinal changes
ExamplesInclusive and participatory policy making processes in place in 10 countries in the regionPresence of a functional network of local governance practitioners in the regionPro-poor policies formulated Increased level of awareness on human rights among CSOs and governments in the region Quality of judiciary processes improved Improved perception of public on existence of free electronic mediaExtent of involvement of CSOs representing indigenous groups in national planning processesQuality of public-private partnership in the preparation of national plan of action on SME development
Three Days TOT On Result Based Management for M&E Staff 32
Indirect (Proxy) IndicatorsDirect indicators are preferable. However, they do sometimes not exist, are
too expensive or inefficient to obtain. In such cases, indirect indicator have to be used.
Purpose Direct indicator Proxy indicator as substituteImprovements in the national judicial system
Quality of judicial cases improved -> difficult and expensive to obtain
Number of judicial cases challenged in the higher courtsRatio of cases filed to the cases processed in the judicial courts
Increase in per capita family income in the poorest state
household income -> household income survey needed -> not frequent enough
changes in local retail sales Increase in the total number of cell phone users in the state
To increase per capita income of small farmers
Crop sales and food consumption -> survey needed -> too expensive
purchase of typical consumer items improvement in buildings (e.g. roof) of life style (consumption of meat per week)
Three Days TOT On Result Based Management for M&E Staff 33
“If-Then” causality between levels of results Common results language to describe changes Collective accountability increases as you move
up the chain of results towards outcomes and impacts (Key message M&E can’t do it alone without proper support and commitment of Senior Management !)
A results matrix is a mean not an end A results matrix is contextual
Three Days TOT On Result Based Management for M&E Staff 34
Principles of RBM
Start with the results Determine indicators to measure progress
towards achieving each result Define explicit targets for each indicator to judge
achievement Collect information to verify/monitor the
achievement/progress Review, analysis and report actual results
Three Days TOT On Result Based Management for M&E Staff; Conducted by : Mr. Fida Karim – Head of M&E Department ITA 35
Key RBM Techniques
Tuesday, May 2, 2023
RBM are good and useful techniques But it won’t work itself and not sufficient to achieve
results RBM depends on the organization's ability to create a
management culture that is focused on results. Manage change in your organization.
Three Days TOT On Result Based Management for M&E Staff; Conducted by : Mr. Fida Karim – Head of M&E Department ITA 36
RBM
Tuesday, May 2, 2023
• Getting started• Stakeholder engagement• The planning exercise: Information flow, timeline and method.• Finalizing the results framework • Preparing to operationalize• Putting it together: Planning for change.
Planning for Results
Tuesday, May 2, 2023Three Days TOT On Result Based Management for M&E Staff; Conducted by : Mr. Fida Karim – Head of M&E Department ITA 37
• Preparing to monitor by reinforcing the initial M&E plan• Monitor: Collection of data, analysis and reporting: Information flow, timeline and process.• Use of monitoring data for management action and decision making
Monitoring for results
Tuesday, May 2, 2023Three Days TOT On Result Based Management for M&E Staff; Conducted by : Mr. Fida Karim – Head of M&E Department ITA 38
Based on the logical framework Strengthens accountability and transparency Provides information for effective
management Helps determine what works well and what
requires improvement Builds knowledge on the basis of best
practices, case studies and success stories etc. regarding developmental results.
Three Days TOT On Result Based Management for M&E Staff; Conducted by : Mr. Fida Karim – Head of M&E Department ITA 39
Monitoring for results
Tuesday, May 2, 2023
• What is Evaluation?-It’s a management tool to periodically assess an intervention.• Why evaluate? Uses of evaluationIt is a rigorous process to independently assess an intervention in terms of Relevance, Efficiency, Effectiveness, Impact & Sustainability. • Evaluation policy: Principles, norms and standards for evaluation.• Types of evaluationProgramme units
Evaluating for results
Tuesday, May 2, 2023Three Days TOT On Result Based Management for M&E Staff; Conducted by : Mr. Fida Karim – Head of M&E Department ITA 40
• Why use monitoring and evaluation?• Learning and generating knowledge from monitoring and evaluation• Knowledge products and dissemination-Case Study-Success Story-Best Practices-Lesson Learnt
Enhancing the use of knowledge from Monitoring and Evaluation
Tuesday, May 2, 2023Three Days TOT On Result Based Management for M&E Staff; Conducted by : Mr. Fida Karim – Head of M&E Department ITA 41
In some situations, the problem may have been previously identified and presented with an analysis and proposal for the government, UNDP or other funding partner to consider. A common problem in these situations is that many project proposals are presented to the funding agency with a fixed solution. Quite often, the solution presented only relates to part of a bigger problem. This is often because the agency presenting the proposal tends to be concerned with obtaining financing for the component(s) for which it has a strong interest. For example, an NGO may submit a proposal for assistance to strengthen its capacity to participate in monitoring national elections. While this may be an important project, it is likely that it would only address part of a bigger problem.
Good results-oriented programming requires that all project-level proposals be subject to a problem analysis to determine whether the stated problem is part of a bigger problem and whether the proposed solution will be adequate to address the challenges. The answers to these questions can sometimes be found, particularly in situations where the projects proposed are within the context of an already designed national programme (such as a Poverty Reduction Strategy Paper, UNDAF or country programme). However, in many cases, there will need to be deeper discussions of what the larger problem is and what other actions are needed by different partners to solve that problem. The aim in asking these questions is not to slow down the process of project review and approval but to ensure that problems are analysed properly and appropriate solutions are found. These solutions may involve actions beyond the scope of the specific project. This is one of the differences between a project approach and a results focused approach to development.
Difference between a ‘project’ and a ‘results-based’ approach to development
Tuesday, May 2, 2023Three Days TOT On Result Based Management for M&E Staff; Conducted by : Mr. Fida Karim – Head of M&E Department ITA 42