building rbm
TRANSCRIPT
-
7/28/2019 Building RBM
1/62
IPDETModule4:
BuildingaResults-B
MonitoringandEvaluSystem
Results-based-based Monitorin
1 2 3 4 5 61
-
7/28/2019 Building RBM
2/62
IPDET
Introduction
ImportanceofResults-Based
WhatIsResults-BasedM&E?
ReasonstoDoResults-Base
Traditionalvs.Results-Based
TenStepstoBuildingResults
M&E
-
7/28/2019 Building RBM
3/62
IPDET
Importance
Growingpressuresindevelo
countriestoimproveperform
publicsectors Tracksresultsofgovernment
overtime
Isamanagementtool
-
7/28/2019 Building RBM
4/62
IPDET
Planning forImprovement SelectingRealistic Targets51
Conducting aReadinessAssessment3
Selecting KeyIndicators toMonitorOutcomes7
The Role ofEvaluations
2
Agreeingon Outcomesto Monitor andEvaluate
4
Baseline Data onIndicators Where Are WeToday?
6
Monitoringfor Results RepFind
-
7/28/2019 Building RBM
5/62
IPDET
ThePowerofMeasurin
Results
Ifyoudonotmeasureresults,youca
successfromfailure
Ifyoucannotseesuccess,youcann
rewardit Ifyoucannotrewardsuccess,youar
rewardingfailure
Ifyoucannotseesuccess,youcann
Ifyoucannotrecognizefailure,youc Ifyoucandemonstrateresults,youc
publicsupport
-
7/28/2019 Building RBM
6/62
IPDET
Results-BasedMo
Results-basedmonitoring(w
monitoring)isacontinuous
collectingandanalyzinginforcomparehowwellaproject,
policyisbeingimplementeda
expectedresults
-
7/28/2019 Building RBM
7/62
IPDET
Results-BasedEva
Results-basedevaluationisa
assessmentofaplanned,on
completedinterventiontodetrelevance,efficiency,effectiv
impact,andsustainability
-
7/28/2019 Building RBM
8/62
IPDET
DifferencebetweenRe
BasedMonitoringand
Evaluation Monitoringisfocusedontracking
movementtowardstheachievem
specific,predeterminedtargets
Evaluationtakesabroaderview
intervention,consideringnotonly
towardstatedgoals,butthelogic
initiative,aswellasitsconseque Bothareneededtobeabletobe
policies,programs,andprojects
-
7/28/2019 Building RBM
9/62
IPDET
KeyTypesofMoni
Res
Implem(Mean
Outcomes
Impacts
Results
Inputs
Activities
Outputs
Implementation
-
7/28/2019 Building RBM
10/62
IPDET
ComplementaryRo
MonitoringandEvaMonitoring9 Clarifiesprogramobjectives
9 Linksactivitiesandtheir
resourcestoobjectives
9 Translatesobjectivesintoperformanceindicatorsandsets
targets
9 Routinelycollectsdataonthese
indicators,comparesactual
resultswithtargets9 Reportsprogresstomanagers
andalertsthemtoproblems
Eva9 Analyzeswhy
wereorwere
9 Assessesspe
contributions9 Examinesimp
9 Exploresunint
9 Provideslesso
significantac
programpoterecommenda
improvement
-
7/28/2019 Building RBM
11/62
IPDET
Reasons:
Providescrucialinformationabousectorperformance
Providesaviewovertimeontheproject,program,orpolicy
Promotescredibilityandpubliccoreportingontheresultsofprogra
Helpsformulateandjustifybudge
Identifiespotentiallypromisingprorpracticesbystudyingpilots
(continuedonnextslide)
-
7/28/2019 Building RBM
12/62
IPDET
Reasons:(cont.)
Focusesattentiononachievingoimportanttotheorganizationanditsstakeholders
Providestimely,frequentinforma
Helpsestablishkeygoalsandou
Permitsmanagerstoidentifyandtocorrectweaknesses
Supportsadevelopmentagendashiftingtowardsgreateraccountaforaidlending
-
7/28/2019 Building RBM
13/62
IPDET
Comparison
Traditional
inputs
activities outputs
Results-Based
combinestra
assessmento
andimpacts
allowsorgani
modifyandm
adjustmentstchangeimple
processes
-
7/28/2019 Building RBM
14/62
IPDETStep1:Conductin
ReadinessAssessmPlanning forImprovement Selecting RealisticTargets
51
Conducting aReadinessAssessment
3
Selecting KeyIndicators toMonitor Outcomes
7
The Role ofEvaluations
2
Agreeingon Outcomes toMonitor andEvaluate
4
Baseline Data onIndicators Where AreWe Today?
6
Monitoring forResults RF
Ten Steps to BuildinResults-Based M&E S
-
7/28/2019 Building RBM
15/62
-
7/28/2019 Building RBM
16/62
IPDET
Incentives
Sortouttheanswerstothese
WhatisdrivingtheneedforbuM&Esystem?
WhoarethechampionsforbuusinganM&Esystem?
WhatismotivatingthosewhocbuildingandM&Esystem?
Whowillbenefitfromthesyste
Whowillnotbenefit?
-
7/28/2019 Building RBM
17/62
IPDET
Rolesand
Responsibilities Ask:
Whataretherolesofcentralandlin
assessingperformance?
Whatistheroleofparliament?
Whatistheroleofthesupremeaud
Doministriesandagenciessharein
oneanother?
Whointhecountryproducestrustw
Whereinthegovernmentaredatau
-
7/28/2019 Building RBM
18/62
IPDET
OrganizationalCap
Assesscurrentgovernmentcmonitorandevaluate:
Technicalskills Managerialskills
Existingdatasystemsandthe
Technologyavailable
Fiscalresourcesavailable Institutionalexperience
-
7/28/2019 Building RBM
19/62
IPDET
BarrierstoM&E
Doanyofthesebarrierstobuildisystemnowexist? Lackoffiscalresources
Lackofpoliticalwill
Lackofchampion
Lackofexpertise&knowledge
Lackofstrategy
Lackofpriorexperience Howdoweconfrontthesebarrie
-
7/28/2019 Building RBM
20/62
IPDET
KeyQuestionsforPredict
SuccessinBuildinganM&
DoesaclearmandateexistforM&E? PRSP?,Law?CivilSociety?Other?
Istherethepresenceofstrongleadersseniorlevelsofthegovernment?
Isthereadesiretoseeresourceandplinkedtothebudget?
Howreliableisinformationthatmaybepolicyandmanagementdecisionmaki
Howinvolvediscivilsocietyasapartngovernment?
Aretherepocketsofinnovationthatcabeginningpracticesorpilotprograms?
-
7/28/2019 Building RBM
21/62
IPDET Step2:AgreeingPerformanceOutco
MonitorandEvalPlanning forImprovement Selecting RealisticTargets
51
Conducting aReadinessAssessment
3
Selecting KeyIndicators toMonitor Outcomes
7
The Role ofEvaluations
2
Agreeingon Outcomesto Monitor andEvaluate
4
Baseline Data onIndicators Where AreWe Today?
6
Monitoring forResults RF
-
7/28/2019 Building RBM
22/62
IPDET
WhyanEmphasis
Outcomes? Makesexplicittheintendedobjec
ofgovernmentaction (Knowwhereyouaregoingbeforeyouge
Outcomesarewhatproduceben
Clearlysettingoutcomesiskeyto
andbuildingresults-basedM&E
Important!Budgettooutputs,maoutcomes!
-
7/28/2019 Building RBM
23/62
IPDET
DevelopingOutcom
OnePolicyArea:E
2.Primary
schoollearningoutcomesfor
childrenare
improved
1.Nationschildrenhave
improvedaccess
topre-school
programs
Baselines:Indicators:Outcomes:
-
7/28/2019 Building RBM
24/62
IPDET
Outcomes:
Outcomesareusuallynotdir
measured onlyreportedon
Outcomesmustbetranslatedkeyindicators
Whenchoosingoutcomes,D
alone! agreementiscrucia
-
7/28/2019 Building RBM
25/62
IPDET Step3:SelectingIndicatorstoMon
OutcomesPlanning forImprovement Selecting RealisticTargets
51
Conducting aReadinessAssessment
3
Selecting KeyIndicators toMonitorOutcomes7
The Role ofEvaluations
2
Agreeingon Outcomes toMonitor andEvaluate
4
Baseline Data onIndicators Where AreWe Today?
6
Monitoring forResults RF
-
7/28/2019 Building RBM
26/62
IPDET
Indicators
Aspecificmeasure,thatwhe
systematicallyovertimeindic
progress(ornot)towardasptarget
Indicatorsasksthequestion:
Howwillweknowsuccesswh
-
7/28/2019 Building RBM
27/62
IPDET
IndicatorDevelopm
CREAM
Clear
Relevant Economic
Adequate
Monitorable
-
7/28/2019 Building RBM
28/62
IPDET
MatrixforSelecting
Indicators
Indicator Datacollectionmethod
Frequency& cost tocollect
Who willcollectdata?
Difficultyto collect
Datasource
1.
2.
3.
-
7/28/2019 Building RBM
29/62
IPDET
DevelopingSetofO
IndicatorsforOnePolicyAr
1.%ofGrade6students
scoring70%orbetteron
standardizedmathandscience
tests2.%ofGrade6students
scoringhigheronstandardized
mathandsciencetestsin
comparisontobaselinedata
1. %ofeligibleurbanchildren
enrolledinpre-school
education2.%ofeligibleruralchildren
enrolledinpre-school
education
BaselinesIndicators:Outcomes:
2.Primary
schoollearning
outcomesforchildrenare
improved
1.Nations
childrenhave
improvedaccesstopre-school
programs
-
7/28/2019 Building RBM
30/62
IPDET
DevelopingIndicat
Developyourownindicatorsyourneeds
Developinggoodindicatorsutakesmorethanonetry
Arrivingatfinalindicatorswil
Stateallindicatorsneutrallyincreasein ordecreasein
Pilot,Pilot,andPilot!
-
7/28/2019 Building RBM
31/62
IPDETStep4:GatheringBa
DataonIndicatorPlanning forImprovement Selecting RealisticTargets
51
Conducting aReadinessAssessment
3
Selecting KeyIndicators toMonitor Outcomes
7
The Role ofEvaluations
2
Agreeingon Outcomes toMonitor andEvaluate
4
Baseline Data onIndicators WhereAre We Today?
6
Monitoring forResults RF
-
7/28/2019 Building RBM
32/62
IPDET
BaselineDataand
Sources Baselinedata: Measurementstofindoutwhereare
Primarysource: gatheredspecificallyfortheproject
Secondarysource: collectedforanotherpurpose,butc
from:withinorganization,governme
internationaldatasources,etc.
cansavemoneybutbecarefultoetrulytheinformationyouneed
-
7/28/2019 Building RBM
33/62
IPDET
PossibleSources
Writtenrecords
Individualsinvolvedwiththe
Thegeneralpublic
Trainedobservers
Mechanicalmeasurements
Geographicalinformationsys
-
7/28/2019 Building RBM
34/62
IPDET
DesignDataCollec
Methods1. Decide how to obtain the da
need from each source
2. Prepare data collection inst
3. Develop procedures for usedata collection instruments
-
7/28/2019 Building RBM
35/62
IPDET
Conversation
with
concerned
individuals
CommunityInterviews
Field
visits
Reviews of
official
records (MIS
and admin
data)
Participant
Observation
Key informant
interviews
Focus
Group
Interviews
Questionnaires
Informal/Less Structured Methods More Structu
One-T
Surv
Directobservation
-
7/28/2019 Building RBM
36/62
IPDET
ContinuingExample,Develop
DataforOnePolicyArea: Edu
1.47%in2002sc
70%orbetterinm
50%in2002scor
orbetter
2.Mean%score
forGrad6studen
mathwas68%,a
forscience
1.75%urbanin
2.40%ruralin20
1.%ofGrade6students
scoring70%orbetteron
standardizedmathand
sciencetests
2.%ofGrade6students
scoringhigheronstandardized
mathandsciencetestsin
comparisontobaselinedata
2. Primary
schoollearningoutcomesfor
childrenare
improved
1. %ofeligibleurbanchildren
enrolledinpre-school
education
2.%ofeligibleruralchildren
enrolledinpre-school
education
1. Nations
childrenhave
improvedaccess
topre-school
programs
BaselineIndicators:Outcomes:
-
7/28/2019 Building RBM
37/62
IPDET Step5:PlanningImprovements Se
RealisticTargetPlanning forImprovement SelectingRealistic Targets51
Conducting aReadinessAssessment
3
Selecting KeyIndicators toMonitor Outcomes
7
The Role ofEvaluations
2
Agreeingon Outcomes toMonitor andEvaluate
4
Baseline Data onIndicators Where AreWe Today?
6
Monitoring forResults RF
-
7/28/2019 Building RBM
38/62
IPDET
Targets:
The quantifiable levels of thethat a country or organization
achieve at a given point in timExample:
Agricultural exports will increasthree years by 20% over the b
-
7/28/2019 Building RBM
39/62
IPDET
IdentifyingExpectedor
LevelofImprovementR
Targets
Baseline
Indicator
Level
Desired
Level of
Improvement
Assumes afinite and
expected levelof inputs,
activities, andoutputs
+ =
-
7/28/2019 Building RBM
40/62
IPDET
Caution:
Ittakestimetoobservethee
improvements,therefore:
Berealisticwhensettingtarge Avoidpromisingtoomuchand
programmingyourselftofail
-
7/28/2019 Building RBM
41/62
IPDET
ContinuingExample,Setting
TargetsforOnePolicyArea:
1.47%in2002sc
70%orbetterinm
50%in2002scor
orbetter
2.Mean%score
forGrad6studen
mathwas68%,a
forscience
1.75%urbanin
2.40%ruralin20
1.%ofGrade6students
scoring70%orbetteron
standardizedmathand
sciencetests
2.%ofGrade6students
scoringhigheronstandardized
mathandsciencetestsin
comparisontobaselinedata
2. Primary
schoollearningoutcomesfor
childrenare
improved
1. %ofeligibleurbanchildren
enrolledinpre-school
education
2.%ofeligibleruralchildren
enrolledinpre-school
education
1. Nations
childrenhave
improvedaccess
topre-school
programs
BaselineIndicators:Outcomes:
-
7/28/2019 Building RBM
42/62
IPDETStep6:Monitoringfor
Planning forImprovement Selecting RealisticTargets
51
Conducting aReadinessAssessment
3
Selecting KeyIndicators toMonitor Outcomes
7
The Role ofEvaluations
2
Agreeingon Outcomes toMonitor andEvaluate
4
Baseline Data onIndicators Where AreWe Today?
6
Monitoringfor ResultsRF
-
7/28/2019 Building RBM
43/62
IPDET
KeyTypesofMoni
Res
Implem(Mean
Outcomes
Impacts
Results
Inputs
Activities
Outputs
Implementation
-
7/28/2019 Building RBM
44/62
IPDET
Implementation
MonitoringLinksto
ResultsMonitoring
Means andStrategies(Multi-year andAnnual WorkPlans)
Means andStrategies(Multi-year andAnnual WorkPlans)
Means andStrategies(Multi-year andAnnual WorkPlans)
Means andStrategies(Multi-year andAnnual WorkPlans)
Means andStrategies(Multi-year andAnnual WorkPlans)
Means andStrategies(Multi-year andAnnual WorkPlans)
Target 1 Target 2 Target 3
Outcome
-
7/28/2019 Building RBM
45/62
IPDET
Target 1
Partner 1
Partner 2
Partner 3
Means & Strategy
Partner 1
Partner 2
Partner 3
Means & Strategy
Partn
Partn
Partn
Means &
Target 2
Outcome Outcome
Impact
-
7/28/2019 Building RBM
46/62
IPDET
SuccessfulMonito
Systems Tobesuccessful,everymon
systemneedsthefollowing:
ownership management
maintenance
credibility
-
7/28/2019 Building RBM
47/62
-
7/28/2019 Building RBM
48/62
IPDET
EvaluationWhent
Anytimethereisanunexpectedresult
furtherinvestigation
Whenresourceorbudgetallocationsa
acrossprojects,programs,orpolicies
Whenadecisionisbeingmadewheth
expandapilot
Whenthereisalongperiodwithnoim
andthereasonsforthisarenotclear Whensimilarprogramsorpoliciesare
divergentoutcomes
-
7/28/2019 Building RBM
49/62
IPDET
EvaluationMeansStrategy
Whetherwearedoingtherigh Rationale/justification Cleartheoryofchange
Operation
Whetherwearedoingthingsr Effectivenessinachievingexpe Efficiencyinoptimizingresourc Clientsatisfaction
Learning
Whethertherearebetterway
Alternatives Bestpractices Lessonslearned
-
7/28/2019 Building RBM
50/62
IPDETStep8:ReportingFin
Planning forImprovement Selecting RealisticTargets
51
Conducting aReadinessAssessment
3
Selecting KeyIndicators toMonitor Outcomes
7
The Role ofEvaluations
2
Agreeingon Outcomes toMonitor andEvaluate
4
Baseline Data onIndicators Where AreWe Today?
6
Monitoring forResults ReFi
-
7/28/2019 Building RBM
51/62
IPDET
AnalyzingandRep
Data Givesinformationonthestatusoprograms,andpolicies
Providescluestoproblems
Createsopportunitiestoconsiderimprovementsinthe(projects,prpolicy)implementationstrategies
Providesimportantinformationovtrendsanddirections
Helpsconfirmorchallengetheory
-
7/28/2019 Building RBM
52/62
IPDET
AnalyzingData
Examinechangesovertime
Comparepresenttopastdata
trendsandotherchanges
Themoredatapointsyouhavcertainyouareofyourtrends
?Time
Improving accessto rural markets
Acc
ess
Tim
Improvingto rural m
Acce
ss
-
7/28/2019 Building RBM
53/62
IPDET
PresentingData
Presentmostimportantdata
Useanappendixorasepara
fordetailedtechnicaldata Usevisualpresentations(cha
graphs,maps)toillustratean
keypoints Avoiddatadumps
-
7/28/2019 Building RBM
54/62
IPDETStep9:UsingFindi
Planning forImprovement Selecting RealisticTargets
51
Conducting aReadinessAssessment
3
Selecting KeyIndicators toMonitor Outcomes
7
The Role ofEvaluations
2
Agreeingon Outcomes toMonitor andEvaluate
4
Baseline Data onIndicators Where AreWe Today?
6
Monitoring forResults RF
-
7/28/2019 Building RBM
55/62
IPDET
TenUsesofResul
Findings Respondstoelectedofficials anddemandsforaccountability
Helpsformulateandjustifybudge
Helpsinmakingoperationalresoallocationdecisions
Triggersin-depthexaminationsoperformanceproblemsexistandcorrectionsareneeded
Helpsmotivatepersonneltocontprogramimprovements
(continuedonnextslide)
-
7/28/2019 Building RBM
56/62
IPDET
TenUsesofResul
Findings(cont.) Monitorstheperformanceofcontgranteesagainstperformanceta
Providesdataforspecial,in-deptevaluations
Helpsprovideservicesmoreeffic
Supportsstrategicandotherlongplanningefforts(byprovidingbasinformationandlatertrackingpro
Communicatesbetterwiththepubuildpublictrust
-
7/28/2019 Building RBM
57/62
IPDET
NineStrategiesfor
SharingInformatio Empowerthemedia
EnactFreedomofInformation legisla
InstituteE-government
Addinformationoninternalandextern Publishannualbudgetreports
Engagecivilsocietyandcitizengroups
Strengthenparliamentaryoversight
StrengthentheOfficeoftheAuditorGe Shareandcompareresultsfindingswidevelopmentpartners
-
7/28/2019 Building RBM
58/62
IPDET Step10:SustaininM&ESystemwithi
OrganizationPlanning forImprovement Selecting RealisticTargets
51
Conducting aReadinessAssessment
3
Selecting KeyIndicators toMonitor Outcomes
7
The Role ofEvaluations
2
Agreeingon Outcomes toMonitor andEvaluate
4
Baseline Data onIndicators Where AreWe Today?
6
Monitoring forResults RF
-
7/28/2019 Building RBM
59/62
IPDET
CriticalComponen
CrucialtoSustainin Demand
Clearrolesandresponsibilitie
Trustworthyandcredibleinfo
Accountability
Capacity
Incentives
-
7/28/2019 Building RBM
60/62
IPDET
LastReminders
ThedemandforcapacitybuildingTheonlywayanorganizationcandownhill
Keepyourchampionsonyoursidthem!
EstablishtheunderstandingwithofFinanceandtheParliamentthsystemneedssustainedresource
Lookforeveryopportunitytolinkinformationtobudgetandresourdecisions
(continuedonnextslide)
-
7/28/2019 Building RBM
61/62
IPDET
LastReminders(co
Beginwithpiloteffortstodemonstrresults-basedmonitoringandevalu
Beginwithanenclavestrategy(e.g
innovation)asopposedtoawhole-governmentapproach.
Monitorbothimplementationprogrresultsachievements.
Complementperformancemonitorevaluationstoensurebetterunderspublicsectorresults.
-
7/28/2019 Building RBM
62/62
To continue on to the
Next Module click here
To return to the
Table of Contents click here
http://../Narrative/M_05-na.pdfhttp://../Narrative/M_05-na.pdfhttp://../Narrative/M_05-na.pdfhttp://../Other/index.pdfhttp://../Other/index.pdfhttp://../Other/index.pdfhttp://../Narrative/M_05-na.pdfhttp://../Other/index.pdf