rbm manual 1
TRANSCRIPT
-
8/6/2019 RBM Manual 1
1/35
Managing Environmental Resources EnablingTransition to Sustainable Livelihood
(MERET)
RESULT BASED MANAGEMENTMONITORING AND EVALUATION TRAINING MANUAL
Compiled by: MERET RBM team
-
8/6/2019 RBM Manual 1
2/35
August 2008, Adwa
Table content
1 INTRODUCTION ....................................................................................................................3
2. WHAT IS A RESULT? ......................................................................................................3
3. THE RESULTS BASED MANAGEMENT PROCESS .................................................... 6
3.1. STRATEGIC PLANNING..............................................................................................7
3.2PERFORMANCE MEASUREMENT ..............................................................................7
3.3MANAGEMENT ..............................................................................................................8
AN RBM SNAPSHOT...........................................................................................................8
4. BENEFITS OF RBM .............................................................................................................9
5. RBM IMPACT ON THE ORGANIZATION ...................................................................... 10
6. THE RBM PROCESS ..........................................................................................................11
6.1. FORMULATING OBJECTIVES AND DEFINING A STRATEGY...........................11
6.2. IDENTIFYING INDICATORS.....................................................................................15
6.3. SETTING TARGETS....................................................................................................19
6.4. MONITORING RESULTS ...........................................................................................21
6.4.1. The M&E Plan Matrix.............................................................................................21
6.4.2. Types of Data .........................................................................................................22
6.4.3 Appropriate Uses of Primary and Secondary Data..................................................24
6.5 REVIEWING AND REPORTING RESULTS...............................................................25
6.5.1 Data Collection versus Data Analysis......................................................................26
6.5.2. Guidelines for writing M&E Reports......................................................................30
6.5.3. Guidelines for providing Feedback on Reports.......................................................31
6.6. INTEGRATION EVALUATION .................................................................................32
6.7. USING PERFORMANCE INFORMATION................................................................34
2
-
8/6/2019 RBM Manual 1
3/35
1 INTRODUCTION
The purpose of this guide is to promote a better understanding of results based management
(RBM) concepts and tools and their applicability to the project. While it provides some practical
information that will help the reader apply RBM in a project, it is not intended as a complete, step-
by-step instructional tool. It is designed to accomplish three key objectives:
Introduce RBM concepts and terms.
Summarize how RBM has been introduced to Project and plans for continued development
of RBM.
Describe the benefits of RBM to the project.
This guide is designed to be used by the projects/programs at federal, regional, woreda and
community/site levels.
2. WHAT IS A RESULT?
Results Based Management is a participatory and team-based management approach that seeks to:
_ Focus an organizations efforts and resources on expected results.
_ Improve effectiveness and sustainability of operations.
_ Improve accountability for resources used.
_ Represents a shift away from focusing on inputs and activities towards the
measurement of results.
In operations this means focusing on changes in the behavior and livelihoods of
beneficiaries.
RBM begins by carefully defining what is meant by the term results. A result is a describable or
measurable change in state that is derived from a cause and effect relationship. It provides a
structured logic model for identifying expected results and the inputs and activities needed to
accomplish those results. The Results-chain is the causal sequence for an operation that stipulates
the necessary sequence to achieve desired objectives - beginning with inputs, moving through
activities and outputs, and culminating in outcomes and impacts.
3
-
8/6/2019 RBM Manual 1
4/35
Log frame Result chain
R
E
S
U
LT
S
Impact The positive & negative, intended or unintended long-term results
produced by operation, either directly or indirectly.
Outcome The medium-term results of an operations outputs.
Outputs
The products, capital goods and services which result from operations;
includes changes resulting from the operation which are relevant to the
achievement of outcomes.
Activities Actions taken or work performed through which inputs are mobilized to
produce specific outputs.
Inputs The financial, human & material resources required to implement the
operations.
RBM asks managers to look beyond their immediate work to the end results of their operation. It
pushes managers to periodically step back and ask, "so what ?what type of changes or impacts are
may activities contributing to?''
As this model indicates, work begins with a set of inputs and activities that result in outputs,
outcomes, and impacts.
Outcome Outcome
Impact
Output Output Output Output
Activities
Inputs
4
-
8/6/2019 RBM Manual 1
5/35
Inputs. these are the human and physical ingredients of word _ the raw materials needed to bring
about the results being sought. They include expertise, equipment, supplies.
Activities. These are what you do with the inputs the actions taken, using the inputs, to produce
specific outputs.
Outputs. These are the most immediate results of your work activities, the results over which you
have the most control. Outputs include products, services, or deliverables.
Outcomes. These are the medium term changes that can be expected as a result of delivering the
outputs. They may take place in families, organization and communities, typically during the life
of the project or work activity. You have less control over outcomes because they are at least one
step removed from the activity. Yet it is important to manage towards outcomes because they
represent the concrete changes you are trying to bring about in your work.
Impacts These are the big picture changes you are working toward but that your work activities
alone may not bring about. Impacts represent the underlying goal of your work; they explain why
the work is important. An impact statement inspires people to work towards a certain future to
which their work activities contribute.
FORMAL DEFINTIONS
Output. The product, capital goods and services which result from a MERET operation:
includes changes resulting from the operation which are relevant to the achievement of
outcomes.
Outcome. The medium-term results of an operations output.
Impact. Positive and negative, intended or unintended long-terms results produced by a
MERET project operation, either directly or indirectly.
According to this definition of results, an output is of value to the extent it contributes to an
outcome or impact. This concept helps an organization focus not simply on results but on critical
results.
5
-
8/6/2019 RBM Manual 1
6/35
3. GUIDING PRINCIPLES FOR M&E STRATEGIES
Fitting into the overall conceptual framework of the monitoring and evaluation is guided by four
general guiding principles:
1. All operations should be regularly and systematically monitored and evaluated, including
processes, performance, intended and unintended consequences and context.
2. M&E must be built into the design of every operation in projects or Programs.
3. Both monitoring and evaluation need to be responsive and appropriate to the situation and the
operation undertaken.
4. M&E strategies must reflect the information needs and approaches established by policies.
3. THE RESULTS BASED MANAGEMENT PROCESS
RBM introduces a structured management approach designed to keep an organization clearly
focused on its expected results throughout the management process. It is a common sense idea
6
-
8/6/2019 RBM Manual 1
7/35
:plan, measure, and manage what you do with a clear eye on the results you want to achieve.
3.1. STRATEGIC PLANNING
Define the results expected and a strategy for achieving them, through a participatory process that
includes all stakeholders. Define the data needed to monitor performance against expected results
and develop plans for collection and reporting performance data.
3.2PERFORMANCE MEASUREMENT
Collect data required to monitor performance and conduct evaluations as necessary to understand
the causes of performance that falls above or below expectations. Report performance
measurement information to internal and external and external stakeholders to support decision-
making and future planning.
Managing
for Results
Strategic
Planning
ManagementPerformance
Measurement
7
-
8/6/2019 RBM Manual 1
8/35
3.3MANAGEMENT
Provide relevant performance management information to managers and teams so they can review
and adjust, as necessary, their plans and strategies and continuously improve their work activities
in order to maximize results.
AN RBM SNAPSHOT
So what does RBM look like? What does it mean to practice results-based management? A
very simple picture of RBM in practice at the project level might look like the following:
Project stakeholders at federal and regional levels, first review the overall land degradation
and potential that the project can fill the gap. After discussion and debate, the workinggroup develops a project strategy that identifies higher level and intermediate outcomes,
as well as the outputs and activities that support these outcomes. A small set of
performance indicators is identified for each of the outcomes included in the project
strategy.
Implementation proceeds with an eye on the strategy. As implementation continues, data
are collected for the performance indicators initially identified by the project workinggroup.
At regular intervals (e.g., every or six months), the same project working group convenes
to review progress towards project results, that is, outcomes (and perhaps outputs) that
have been identified as part of the project strategy. The review includes and assessment of
data collected for each of the performance indicators. The most recently collected data are
compared to baseline and target values for each indicator, and performance trends and
patterns are highlighted. If or when the assessment indicates that progress exceeds or is
substantially bellows expectations, the working group takes a closer look at the full set or
data to determine the possible causes and implications of the unexpected performance.
Project managers make decisions based on this analysis and discussion. For example, they
may decide to shift resources to specific activities, or modify or shut down other activities,
or perhaps to adjust long term project focus or strategic approach. When necessary,
evaluations are conducted to supplement the performance data collected through regular
monitoring systems. Implementation, data collection and performance review and
adjustment continue throughout the life of the project.
8
-
8/6/2019 RBM Manual 1
9/35
This periodic-and on going review of performance towards project outcomes pushers managers
to continually step back and focus on the achievement of medium and long term outcomes. It asks
managers to see project activities as tasks that contribute to broader changes, and not as ends in
themselves.
4. BENEFITS OF RBM
As implied by the RBM snapshot in the proceeding section, RBM offers discernable benefits in
a number of areas, including: People think RBM is just reporting system, while RBM does
facilitate reporting, it is first and foremost a management system.
Planning. Perhaps the biggest benefit of RBM at the planning stage is that it focuses
planning on outcomes and higher level change. At the same time, RBM recognizes the
fundamental importance of inputs and activities. The planning process integrates
implementation and day-to-day work with the anticipated changes such work supports.
The (strategic) framework developed during the planning process provides a structure for
keeping new and on-going activities focused on key outcomes, thereby bringing discipline
to activity level planning. In short, planning in the RBM context delivers two key benefits:
an emphasis on outcome level change and a more focused and strategic process for
identifying and conducting day-to-day activities.
Building consensus and ownership. RBM requires broad participation by stakeholders,
partners and internal staff. Comprehensive discussions between are a critical element of
the process of developing objectives, strategies, and performance indicators. Partners who
have participated in defining expected results have a greater sense of ownership and
commitment to those results and to ensuring the resources, activities, and output needed to
accomplish those results. Though participation is absolutely critical in most aspects of
project (and non-project) work, a comparatively higher and broader level of understanding
and commitment is created when discussing and agreeing to overarching strategies and
outcomes. And even when consensus is not reached, participants have a much better
understanding of each other perspectives, greately facilitating future collaboration and
cooperation.
Management. By systematically collecting, analyzing and assessing data on results
achieved, poorly performing components of an operation can be quickly identified and
adjusted mid-stream. Resources can be reallocated to those activities with the highest
payoff in terms of results, or moved to activities which appear to need more support to
begin to deliver results. The term has the information it needs to understand whether and
9
-
8/6/2019 RBM Manual 1
10/35
to what extent progress is being made towards its objectives, and can take appropriate
action to continuously improve performance.
Communication. A result-based approach encourages and facilitates improved
communication with internal and external stakeholders. Several of the steps in the RBM
process lead to management tools/products that clearly and efficiently communicate
important information about a given program. For example, a strategic framework/logic
model, usually prepared during the planning phase, can quickly and easily communicate
the intent and content of a given project or operation to stakeholders and partners.
Communication is also fostered by collecting and sharing data and information on the
success of the project at every level in the results chain.
Reporting. RBM provides a disciplined framework for reporting on results. Because
RBM requires the collection of comparable data for all performance indicators, and
because it also requires the development of a strategic or logical framework, managers can
more accurately present observed changes and more confidently discuss the progress of a
given project or operation. Managers can thus better the reporting process to demonstrate
effectiveness and make the case to stakeholders and sponsors for continuing support and
additional resources.
5. RBM IMPACT ON THE ORGANIZATION
RBM leads to a number of changes to key elements of an organization. These changes are at the
same time both a cause and an effect of RBM. They are required in order for RBM to be effective,
but they also become possible and are encouraged by the structured planning, monitoring, and
management process associated with RBM
Accountability. Managers and staff are more accountable for managing for results rather
than simply moving inputs and managing activities.
Empowerment. Managers and staff are increasingly empowered to make corrective
adjustments and shift resources in order to improve results.
Participation and partnership. Partners, donors, and beneficiaries participate more fully
in planning, monitoring, and management activities.
Policy and procedures. Managers and staff need support mechanisms, including training,
technical assistance, database, guides, and shared best practices information.
Organizational culture. Values, attitudes and behaviors change to support RBM,
including instilling a commitment to open and honest performance reporting, reorientation
10
-
8/6/2019 RBM Manual 1
11/35
away from inputs and activities towards results, and encouraging learning based on
evaluation.
6. THE RBM PROCESS
The RBM process provides procedures and mechanisms that redefine strategic planning,
performance measurement, and management so that the entire cycle remains constantly linked to
the desired end results. The table below shows the seven key steps in the RBM process.
Table
This Step Accomplish this
Formulating objectives and defining strategies Defines the results we are trying to achieve and our strategy
for achieving them
Identifying indicators Identifies what we need to measure in order to understand
whether we are accomplishing the results we want to achieve
Setting targets Defines how much progress we need to make on what
timeframe
Monitoring results Collects the data needed to measure our progress
Reviewing and reporting results Compiles, analyzes, and reports the data in a way that meets
the needs of different levels of the organization
Integrating evaluation Uses evaluations to understand why performance exceeds or
falls short of expectations
Using performance information Uses the performance information we have developed to
continuously improve our performance
6.1. FORMULATING OBJECTIVES AND DEFINING A STRATEGY
This entails identifying in clear, measurable terms the results being sought and developing a
conceptual framework for how the results will be achieved. As part of the planning process,
objectives should be clarified by defining precise and measurable statements of the results to be
achieved and then identifying the strategies or means for meeting those objectives. The Logical
Framework or Log frame, currently used by MERET project, is an example of this process. It is afive-level hierarchical model of the cause and effect relationships (sometimes referred to as the
results chain) that lead to a desired end result. The five level structures are as follows: inputs are
used to undertake activities that lead to the delivery ofoutputs, which lead to the attainment of
outcomes that contribute to an impact. The MERET logic model (2005-06) is indicated on annex1.
Setting objectives begins with a clear definition of results (impacts, outcomes, and outputs). To
identify objectives:
11
-
8/6/2019 RBM Manual 1
12/35
Engage staff working in MERET project in the process. Good objective-setting depends on
broad participation, but additionally participation helps communicate objectives to all
involved, clearly roles, and solidly commitment and buy-in by articulating the impact of
everyones efforts.
Begin by identifying the impact or end result desired. This is the center of any resultsframework. Defining the desired impact is critical because it becomes the starting point for
all subsequent planning, monitoring, and management activities, and it forms the standard
by which the project will be judged.
Next identify the outcomes necessary to achieve the desired impact. Clarify the causal
linkages between results.
Identify critical assumptions about conditions necessary in order for the results model to
hold true. After working top-down to identify impacts, outcomes, and outputs, work
bottom up to identify and critically examine assumptions behind the cause and effect
relationship in the model.
The main Components of an M&E Strategy are:
_ A logical framework.
_ An M&E plan for data collection and analysis, covering baseline,
_ Reporting flows and formats.
_ A feedback and review plan.
_ A capacity building design.
_ An implementation schedule.
_ A budget.
The logical framework matrix is the foundation document for both operation design and M&E.
Additional elements of the M&E strategy are extensions of the logical framework that describe
how indicators will be used in practice to measure implementation performance
and results achievement.
The Logical Framework outlines:
12
-
8/6/2019 RBM Manual 1
13/35
Clearly defined and realistic objectives, assumptions and risks that describe how the
operation is designed to work.
A minimum set of results indicators for each objective and assumption that are feasible
to collect and analyse. Indicators measure performance on implementation and
achievement of results. The means of verification provided in the logical frameworkfor each indicator outlines the source of data needed to answer each indicator.
The main Contents of the Logical Framework Matrix
Each of the four columns in the Logical Framework is described in the following paragraphs. The
first and fourth columns articulate operation design and assumptions, while the second and third
columns outline the M&E performance measurement indicators and means in order to test whether
or not the hypothesis articulated in the operation design holds true.
Column 1: This column outlines the design or internal logic of the operation. It incorporates a
hierarchy of what the operation will do (inputs, activities and outputs) and what it will seek to
achieve (purpose and goal).
Column 2: This column outlines how the design will be monitored and evaluated by providing the
indicators used to measure whether or not various elements of the operation design have occurred
as planned.
Column 3: This column specifies the source(s) of information or the means of verification for
assessing the indicators.
Column 4: This column outlines the external assumptions and risks related to each level of the
internal design logic that is necessary for the next level up to occur.
How to check the Design Logic in a Logical Framework
To check the design logic of the logical framework, review and test the internal and external logic
(columns 1 and 4, respectively) and the feasibility of the operation s logical framework. Test the
logic beginning with inputs and move upwards towards the impact using an if internal logic) and
(external logic) then (internal logic at the next level) logic test. Where necessary, just the logical
framework to overcome logic flaws or unfeasible/unlikely relationships among various levels of
13
-
8/6/2019 RBM Manual 1
14/35
the logical framework hierarchy. If no logical framework exists for the operation, consult the
Logical Framework Guidelines.
Specifically check that the following conditions hold:
_ Inputs are necessary and sufficient for activities to take place
_ Activities are necessary and sufficient for outputs that are of the quality and quantity
specified and that will be delivered on time.
_ All outputs are necessary, and all outputs plus assumptions at the output level are necessary
and sufficient to achieve the outcome.
_ The outcome plus assumptions at the outcome level are necessary and sufficient to achieve
the impact.
_ The impact, outcome, and output statements are not simply restatements, summaries or
aggregations of each other, but rather reflect the resulting joint outcome of one level plus
the assumptions at that same level.
_ Each results hierarchy level represents a distinct and separate level, and each logical
framework element within a results hierarchy level represents a distinct and separate
element.
_ The impact, outcome, activities, inputs and assumptions are clearly stated, unambiguous and
measurable. Impacts and outcomes are stated positively as the results that WFP wishes to
see.
_ The assumptions are stated positively as assumptions, rather than risks, and they have a very
high probability of coming true.
How to check the M&E Elements in a Logical Framework
_ Indicators for measuring inputs, activities, outputs, outcome and impact are specific,
measurable, accurate, realistic and timely (SMART) (column 2).
_ Beneficiary contact monitoring (BCM) indicators are identified for the purpose of tracking
progress between outputs and are noted at the outcome level.
_ Two levels within one logical framework do not share the same indicator (if they do, the
indicator at one level is not specific enough to that level or the design logic between levels
is flawed).
_ The unit of study (e.g. individuals, children, households, organizations) in the numerator
and, where applicable, the denominator of each indicator are clearly defined such that there
is no ambiguity in calculating the indicator.
14
-
8/6/2019 RBM Manual 1
15/35
_ The means of verification for each indicator (column 3) are sufficiently documented, stating
the source of the data needed to assess the indicator (be sure that sources of secondary data
are in a useable form).
6.2. IDENTIFYING INDICATORS
After objective have been formulated, in partnership with stakeholders, the next step is to select
indicators for measuring progress towards the achievement of expected results. Indicators need to
be developed at each level in the results chain during the design of the project. Indicators specify
what to measure along a scale or dimension (eg. Percent of farmers adopting new technology,
ratio of female to male students, etc). Indicators are empirical conditions which will signalachievement of the desired end and gauge progress.
Below the sample performance indicators that build upon the illustrations included in the
preceding section:
Objectives Illustrative performance indicators
Increase the use of improved agricultural
technologies
1. No and % of farmers in target
communities who use improved seed
2. no and % of farmers in target
communities who apply fertilizer in an
appropriate manner
Rehabilitate and construct agricultural
infrastructure through FFW/asset creation
1. No of kilometers of irrigation canal
restored
2. No of earthen dams constructed or
rehabilitated
3. No f kilometers of drainage canals
restored
15
-
8/6/2019 RBM Manual 1
16/35
As with defining objectives, indicators should be developed using a collaborative approach with
stakeholders. Broad participation not only helps build support and buy-in for the project, but
stakeholders often bring valuable knowledge of data sources and practical data collection
consideration. An example of MERET project(2005-06) indictors are indicated in annex1
Below is a useful approach to take in identifying indicators
Clarify the objectives: review the precise intent of the objectives and make sure you are
clear on the exact changes being sought. Good indicators start with the formulation of
good objectives that everyone agrees on.
Develop a list of possible indicators. Usually, many possible indicators can be readily
identified. Often, it helps to first develop a long list first through brainstorming or drawing
on experiences of similar projects. At this point, encourage creativity and free flow of
ideas.
Assess possible indicators and select the best. In selecting final indicators, you should
set a high standard. Data collection is expensive, so select only those indicators that
represent the most important and basic dimensions of the results sought.
Good indicators should be SMART (Characteristics of good indicators)
Specific-gear to the direct action of the project
Measurable-capable of verification at reasonable cost
Attainable-an indicator should refer to a characteristics which can be attained with
manageable period
Relevant-directly linked to the management and the project objectives.
Traceable Sensitive to change
How to select Indicators
Indicator selection usually takes place during the design process and is reflected in the operations
logical framework matrix (column 2). Indicators should be specific, measurable, accurate, realistic
and timely (SMART). This acronym provides a detailed set of criteria for assessing the
appropriateness of potential indicators. Each of the indicators identified must satisfy the following
conditions as indicated above.
Specific
16
-
8/6/2019 RBM Manual 1
17/35
An appropriate indicator measures only the design element (output, outcome or impact) that it is
intended to measure and none of the other elements in the design. Many indicators are related to
every design element (since all the elements within a design are related), but few are specific
measures of performance for each and every element .Time spent in water collection is a related,
but not specific, measure. Similarly, the number of hours
spent in activities is a related, but not specific, indicator. Because the design must treat each level
in the results hierarchy, and each design element in the level, as a separate and distinct element,
the appropriate indicator at one level (or for one design element) cannot be the appropriate
indicator for another. If an indicator is shared at two levels or between two design elements, either
one of the indicators is not specific enough or the design logic is flawed.
Measurable
An appropriate indicator is measurable and clearly defines the measurement such that two people
would measure it in the same way. For quantitative proportions or percentages this means that
both the numerator and the denominator must be clearly defined. For quantitative whole numbers
and qualitative data it means defining each term within the indicator such that there can be no
misunderstanding as to the meaning of that indicator. This is critical for ensuring that the data
collected by different people at different times are consistent and comparable.
Examples of indicators that are not measurable include the percentage of households that are food-
secure ("food-secure" is not defined precisely) and the percentage of women with increased access
to health services ("access" is not defined precisely). The critical means of ensuring that indicators
are measurable is to define all the terms within the indicator, even those for which a general
agreement about meaning may be shared among staff members.
Accurate
Some indicators are more accurate measures than others. For example, measuring the weight-for-
height of children under 5 years of age will yield a more accurate figure for the percentage of
acutely malnourished (wasted) children than will measuring the mid-upper arm circumference
(MUAC). Again, note the need to define clearly what is meant by "acutely malnourished" in terms
of measurement (previous criteria). Similarly, a seven-day dietary recall will yield a more accurate
measure of food consumption than will asking the average number of meals that were consumed
over the last month. However, the accuracy criteria must be balanced with the other criteria,
taking into consideration the resources available for M&E.
Realistic
17
-
8/6/2019 RBM Manual 1
18/35
The indicators selected must be realistic in terms of their ability to collect the data with the
available resources. Some indicators present major problems for data collection owing to the cost
or skills required (e.g. anthropometric surveys, large-scale sample surveys). Being realistic in
planning what information can be collected ensures that it will, in fact, be collected. This is an
important factor to consider and may lead to compromises on other criteria.
Timely
Indicators must be timely in several aspects. First, they must be timely in terms of the time spent
in data collection. This relates to the resources that are available - staff and partner time being
critical. If it takes two days to collect dietary recall data from one household, this indicator is
probably inappropriate. An appropriate indicator may disaggregate by dry and wet season. Finally,
the time-lag between output delivery and the expected change in outcome and impact indicators
must also be reflected in the indicators that are chosen. This time-lag can be significant, especially
for Country Programmes (CPs) aimed at poverty reduction. Some more general guidelines for
indicator selection, based on commonly found mistakes, include the following:
Do not state the target achievement in the indicator itself:
The indicator is simply a measurement and, as such, should be non-directional (e.g. neither
positive nor negative). Targets should be listed either in the first column of the logical framework
- as part of the operations internal logic - or as a separate column
.
Do not select too many indicators: Managers have a tendency to ask for too much information,
assuming that the more they know the better prepared they will be. The result is often information
overload. Instead, information needs must be related directly to decision-making roles and levels
of management - field managers require more detailed information, while aggregated and
summarized data are used at higher levels. The selection of indicators should reflect this through
the specification of a minimum set of information. There is a tendency for staff and partners to
want to capture every nuance and to identify all the possible indicators during the design of an
operation. A brief reminder about the cost and time needed to collect and analyze the data usually
brings the focus back to the minimum set of information needed.
Do not select indicators that are unnecessarily complex:
Some indicators present major problems for data collection in terms of the skills or resources
required. For example, household income data can be complex and expensive to collect.
Alternative indicators to consider are patterns of expenditure or household characteristics such as
18
-
8/6/2019 RBM Manual 1
19/35
the materials used to construct the house. Qualitative indicators (e.g. wealth ranking) can also
convey complex information, perhaps less accurately but accurately enough for most data needs.
Do not over concentrate on physical progress indicators:
Information about food stocks and distribution is vitally important within a WFP operation, but it
does not provide sufficient information on the performance of the operation. Identifying these
indicators is relatively straightforward. However, information about the results of an operation is
also needed, and the selection of indicators at these levels is slightly more complex. To some
extent, the logical framework mandates the identification of indicators at the outcome and impact
levels, making it an ideal shared framework for operation design and M&E.
6.3. SETTING TARGETS
Once indicators have been identified for project objectives, the next step is to devise targets. A
target is a specific indicator value to be accomplished by a particular data in the future. Final
targets are values to be achieved by the end of the project, whereas interim targets are expected
values at various points-in time over the life of the project. Baseline values-which measure
conditions at the beginning of the project-are useful both in terms of helping to set future targets
and as a means for understanding performance (i.e., actual performance can be usefully compared
to targets and to the relevant baseline values.)
Targets represent commitments signifying what the project intends to achieve in concrete terms,
and become the standards against which a projects performance or degree of success will later be
judged. Targets may be useful as a way to bring the objectives of the project into sharp focus.
They can also help to justify a project by describing in concrete terms what the investment will
produce. Finally, they can help establish a system of accountability for managers and others
involved in the project.
It should be noted as well that sometimes it may be impossible or ill-advised to set targets. (e.g.. if
no baseline or historical data exists to understand trends). In such cases, setting targets becomes
almost purely an exercise in conjecture and can possibly confuse or demoralize these involved in
the project. In the absence of specific targets, simply identifying the desired trend and a general
expectation of the degree of change is a useful substitute.
Sometimes it is also useful to set targets in terms of a range of expected performance. This canprovide some realistic flexibility when considering whether performance is or is not at expected
levels.
19
-
8/6/2019 RBM Manual 1
20/35
Below is a useful approach to take establishing targets:
Define the performance baseline. It is difficult if not impossible to establish a reasonable
performance target without some idea of the starting point. The performance baseline is
the value of the performance indicator at the beginning of the planning period, ideally just
prior to project implementation. Baseline data for performance indicators will be derived
from one of three sources: existing project data, existing data from a secondary source, or
primary data collecting efforts.
Understand historical trends. Perhaps even more important than establishing a single
baseline value understands the underlying historical trend in the indicator value over time.
Is there a trend upward or downward? What can be drawn from existing reports, records or
statistics?
Understand stakeholder expectations. While targets should be set on an objective basis,
it is useful to also get input from donors and other stakeholders regarding what they expect
or need from MERET project activities.
Seek outside expertise. Another source of information for target setting is expert opinion
about what is possible or feasible with respect to a particular indictor and country setting
or situation. Similarly, reviewing research literature may help in setting realistic targets.
Look at related projects. Understand the rate of progress that has been registered in other
projects in similar situations and use this past practice to set targets.
We can use the illustrative indicators provided in the earlier section (Identifying Indicators) to
provide a simplified snapshot of the target setting process. Let us say that we have collected
baseline information that indicates that only 10% of local farmers use improved seeds at the
beginning of our project. We have also spoken with local farmers and former extension agents and
know that most farmers in the area are quite conservative. We have very limited historical data,
but what we have confirms this, i.e, new seed varieties become available four years ago and the
acceptance rate is still only 10%. On the other hand, experience in neighboring woredas or sites
and in other projects shows that farmers will move very dramatically towards new seed varieties
once yield gains are clearly demonstrated on pilot farms, With all of this information in hand, we
20
-
8/6/2019 RBM Manual 1
21/35
decide to set fairly aggressive targets: we expect that 20% of farmers will use new seeds after year
one of the project; 40% after year two; and 50% after year three. These targets, which anticipate a
doubling of farmer acceptance for years one and two, followed by somewhat slower expansion,
reflect our assessment of historical trends, of related project and country experience, and of expert
input from local extension agents and framers.
6.4. MONITORING RESULTS
Once a project strategy and plan are in place, monitoring begins. Data are collected at regular
intervals to measure progress towards project outputs, outcome and impact. A distinction is often
made between implementation monitoring - maintaining records and accounts of project inputs
and activities, and results monitoring measuring results at the output, intermediate outcome and
long-term impact levels. RBM, as might be expected, is primarily focused on results monitoring.
Implementation monitoring data typically comes from on-going project financial accounting and
field records. This information is generally needed frequently to assess compliance with design
budgets, schedules, and work plans. It used to guide day-to-day operations.
Results monitoring measures whether the project is moving towards its objectives-that is, what
results have been accomplished relative to what was planned. Information from results monitoring
is important not only for influencing medium-term project management decisions aimed at
imprinting the projects performance but also for reporting to donors, partners and internal
stakeholders.
Effectively mentoring project performance at the different levels in the results chin involves
different data sources and methods, different frequencies of collection, and varying collection
responsibilities, It is good practice to prepare a performance monitoring plan at the projects outset
that spells out exactly who will collect what data when and how. A performance monitoring plan
serves three principle purposes:
Providing detailed information on indicators definitions, data sources and methods of
collection to ensure the comparability of data over time;
Facilitating the data collection process by defining responsibilities and schedules for data
collection and use; and
Informing data analysis when performance data begin to be collected.
6.4.1. The M&E Plan Matrix
This matrix is a summary of M&E related information, setting out detailed responsibilities for data
collection The table or matrix below is useful for clearly identifying what data is needed, the
21
-
8/6/2019 RBM Manual 1
22/35
source of the data, how often it will be collected, by whom it will be collected, what methods will
be used in collection, and finally in which reports and forums the data will presented. The matrix
is critical for establishing clear roles and responsibilities of WFP and partners. It builds upon the
information already contained in the logical framework and develops assumptions by identifying
relevant indicators and ensuring that the related data is collected, analyzed and used.
Log frame
Element
Indicators
(including
targets)
Means of Verification Use of Information Use of Information
Data
requir
ed
Data
Source
Freque
ncy &
Cost of
Collection
methods
Responsi
b-ility
Reportin
g
Presenta
tion
Impact
Assumptions
Outcome
Assumptions
OutputsAssumptions
Activities
Assumptions
Inputs
6.4.2. Types of Data
Quantitative versus Qualitative
Characteristics of Quantitative and Qualitative Data
Two general types of data exist - quantitative and qualitative -although the distinction between the
two is often blurred. While quantitative data have long been cited as being more objective, and
qualitative data as more subjective, more recent debates have concluded that both types of data
have subjective and objective
characteristics. As qualitative and quantitative data complement each other, both should be used.
Characteristics of Quantitative Data
Quantitative data:
o Seek to quantify the experiences or conditions among beneficiaries in numeric terms.
o Use closed-ended questions with limited potential responses.
22
-
8/6/2019 RBM Manual 1
23/35
o Normally ask women, men, boys and girls to respond to questions on the basis of their
individual experiences, or the experiences of their households.
o Use measurement techniques (e.g. measuring land area; maize yield, by weighing bags of
maize; food consumption, through weighing food quantities to be consumed by type;anthropometric indicators of children).
Characteristics of Qualitative Data
Qualitative data seek to uncover the context, perceptions and quality of, as well as opinions about,
a particular experience or condition as its beneficiaries view it. Data collection methods are more
likely to employ a more participatory approach through the use of open-ended questions that allow
respondents to expand on their initial answers and lead the discussion towards issues that they find
important. These more participatory methods will
commonly be used in the M&E of WFP operations.
Sampling techniques for these methods are often purposive. Even when samples are selected
randomly, these methods rarely require the rigorous determination of sample size, and respondents
are often asked to generalize about the condition or experience in the larger population, rather than
talk about themselves.
Examples ofQuantitative and Qualitative
Quantitative Qualitative
The mean amounts of food commodities
remaining in sampled houses one week after
distribution was 45 kg of maize and 2 kg of
vegetable oil
Most households have used up the majority of their
monthly ration in the first week after delivery
because they are expected to share the ration with
neighbors who are not eligible
38% of households have an income of less
than 300 Kenyan shillings per month
According to women in the focus group discussion,
the majority of households do not have enough
income to meet all of their food purchasing needs
40% of children under 5 years of age are
wasted (< -2 standard deviation weight-for-
height), 90% of wasted children have had
diarrhoea in the last two weeks
Women suggest that every child is malnourished at
some time during the year and they attribute this
to chronic diarrhoea
23
-
8/6/2019 RBM Manual 1
24/35
The mean amount of time women take to
reach the primary dry-season water source in
D.D district is 2.3 hours
Women spend most of the daylight hours collecting
wood, water and fodder for animals. They view
this as the main obstacle preventing
them from participating in other economic
endeavors.
Eight out of ten women in the focus group
discussion have more than one child under 5
years of age
In the village, all the women between 20 and 45
years of age have at least one child under 5, and
most have two. The time spent in child care is the
second largest obstacle to womens participation
in economic endeavors.
58% of new arrivals indicated travelling three
or more days to reach the refugee camp
New arrivals in the refugee camp arrived exhausted
having travelled for long distances, which theysuggested resulted in many deaths along the way
6.4.3 Appropriate Uses of Primary and Secondary Data
The collection of M&E data, both primary and secondary, must focus almost exclusively on the
indicators and assumptions identified at each level in the logical framework for the operation.
Secondary Data
The use of secondary data represents tremendous cost and time savings to the country office, and
every effort should be made to establish what secondary data exist and to assess whether or not
they may be used for the M&E of operations. Primary data are often collected unnecessarily and at
great expense simply because
monitors or evaluators had not been aware that the data were already available. It is critical to
invest the initial time and resources to investigate what data exist, what data collection exercises
are planned for the future, and how relevant the existing data are for the M&E of operations.
Primary Data
However, primary data collection is sometimes warranted. Although a review of secondary datasources should precede any primary data collection, existing data do not always provide the
appropriate indicators or the appropriate disaggregating of indicators needed to monitor and
24
-
8/6/2019 RBM Manual 1
25/35
evaluate operations effectively. Even secondary data that provide the appropriate indicators and
disaggregating of indicators may not be useful if the data are out of date and the situation is likely
to have changed since they were collected. This varies greatly according to the indicator for which
the data are being collected and their volatility. For example, school enrolment data that are one
year old may suffice for establishing baseline conditions prior to a school feeding programme, but
acute nutritional data (wasting) that are only a month old may no longer represent an accurate
estimate of current conditions for that
Importance of Documenting Data Collection Methods
Clear documentation of the methods to be used to collect primary and secondary data must be
developed during the planning stage of an operation. As data are collected, any variations from the
planned data collection methods must also be documented. This ensures that data are collected in
the same way at different points in time and by different people. This is critical for ensuring that
the data are comparable, and improves the accuracy of assessing the changes over time associated
with operations.
6.5 REVIEWING AND REPORTING RESULTS
To be useful, the data collected for monitoring purposes needs to be analyzed and turned in to
meaningful information for the people involved do they can understand performance and make
adjustments that are necessary.
Data + Analysis = Information
Data analysis and reporting serve many different levels of the organization, each with its own
special needs.
Periodic assessments of performance monitoring data helps alert the project team to
performance problems. For example, extending the illustration from the target setting
discussion in the previous section, we may find that the project has performed as excepted
in year one (20%) of farmers using the improved seeds), but substantially under-performed
in year two (23%) of farmers were using the new seeds, far short of the 40% target.
Analysis of performance is on-track, but may not adequately explain why or how
performance falls short of or exceeds expectations. Where causes are fairly
straightforward, the project team can identify immediate corrective action. When causes
25
-
8/6/2019 RBM Manual 1
26/35
are more complex, analysis of performance data may signal the need for more in-depth
study or evaluation to understand shortfalls and identify corrective actions.
Country and region. Performance monitoring data from individual projects can be
analyzed across a portfolio of projects to better understand results at the country or
regional level and to identify strategies for better integrating project activities.
Because information needs are different at different levels in the organization, considerable
thought and care is needed to define processes that provide the necessary information without
imposing overly burdensome collection, analysis, and reporting requirements. Often, it is not
meaningful to simply aggregate data at each level. Data assessment/analysis and reporting need to
flow from a considered definition of the objectives the organization is trying to meet at each level.
6.5.1 Data Collection versus Data Analysis
Taking notes during an interview or discussion, regardless of the methodology being used, is
critical for ensuring that what the respondents say is accurately captured. A common error is for
data collectors to interpret or analyse what respondents have said prior to writing it down. It is
crucial to separate data collection from data analysis and to avoid assuming that you know what
the respondent meant. Data collectors should be encouraged to note any analytic insight that they
might gain from their field experience, but this should not be confused with documenting what the
respondents have actually said.
Key Steps to follow in Data Collection
1. Be sure to separate description and raw data collection from your own analysis, judgement,
interpretation or insight.
2. Do not attempt to recall what was said in an interview or discussion at a later time (e.g. in the
car or back at the office). Inevitably, such recalled data will be biased by your own insights and
analysis.
3. Be disciplined and conscientious in taking detailed field notes at all stages of the fieldwork,
including notes on how the fieldwork that was carried out differed from the fieldwork that was
planned. Notes about how the respondents were selected (in relation to the planned sampling
strategy) are important for assessing comparability among data collected from different sites and
at different points in time.
5. Make notes that refer to the interview or discussion guide, checklist or questionnaire that youare using. It is often helpful to create the checklist with space for adding field notes, ensuring that
each note is correctly situated under the relevant checklist point. Another option is to number the
26
-
8/6/2019 RBM Manual 1
27/35
discussion guide or checklist points and refer to these numbers in your notes. For questionnaires,
the usual practice is to leave space
6. Quote directly from interviews or discussions. This allows people to be represented in their own
words and terms. It also provides powerful anecdotal evidence for reports, proposals, etc.
7. Use the notes that you have taken to confirm important points that are made in order to ensure
that you have understood their intended meaning fully. Notes also facilitate cross-checking with
other sources.
8. Even if you think that a point is not important, document it. This serves two purposes: the point
may prove to be important either later in the interview/discussion or during analysis; and your
noting of every point assures respondents that you are being unbiased in what you document and
giving each persons ideas equal value.
9. Do not let note taking disrupt the flow of the conversation, interview or discussion. In one-on-
one interviews, this is not usually a problem. In group settings, however, where your role as
facilitator is paramount, the use of a facilitator and a separate note taker is the best approach.
Steps to follow for consolidating and processing Qualitative Data
The following 5 steps provide general guidance on how to consolidate and process the majority of
qualitative data. Depending on the methods used in data collection the 5 steps may need to be
modified to suit the data processing needs.
Step 1: Summarize Key Points and Identify Quotations
Review data collection notes for each interview or discussion session. It is likely that the notes are
in very rough form. Circle and note key discussion points and responses and consolidate long
narratives into summary points. Also highlight key quotes that you may want to use in your
presentation of the results and keep a list of quotations that might be used to illustrate important
points made by discussion or interview participants.
Step 2: Organize Key Points in Topic Areas
For each group or individual interview or discussion session organize the key discussion points,
responses, and summary points by topic. Topics discussed by more than one group or respondent
can be compared between groups or individuals. These commonly occurring topics are identified
and systematically listed. It is often useful to arrange the common topics in a simple spreadsheet
having each discussion group serve as a row and each topic listed as a column. This will facilitate
easy comparison between groups or respondents during analysis.
Step 3: Develop Codes describing Separate Categories of Similar Responses
27
-
8/6/2019 RBM Manual 1
28/35
For brevity, you will need to code common topics for each group or individual into categories,
giving like responses or discussion points the same code. Codes can be figures or a system of
words or symbols used to describe each separate category. Determine the number of categories for
each topic by looking at the varying
responses or discussion points from each group discussion or individual interview. Be careful not
to dilute nuances and differences in responses or discussions. If you are in doubt give responses
independent codes. Sub-codes can be used to capture nuances for responses or discussion points
that are similar, but not exactly the same. The coding will assist greatly in making comparisons
between groups and individuals during analysis. Use the code category other only for responses
or discussion points that are very infrequent and where these outlying or rare responses or
discussion points are not important for subsequent analysis. Use the codes in your spreadsheet and
be sure to provide a description of what each code means in a key or legend that accompanies the
table.
Step 4: Labelling Products from Participatory Exercises
Products from participatory exercises used to stimulate discussion such as maps, diagrams, or
rankings will not fit nicely into a spreadsheet. Each of these should be separated out from other
data collection notes so that they may be compared for differences and similarities between
groups. The use of note cards, clearly indicating in a label the group or individual from which the
product came, can be helpful.
Step 5: Listing of Discussion Points on Unique Topics
Due to the open-ended nature of qualitative inquiry, topics brought up during the discussion or
interview (e.g. those not pre-planned and turned into topics and coded categories in steps 2 and 3),
should be listed as bullet summary points. Many of these may not be comparable between groups
due to the fact that the issue may
have been raised in one group, but not in another. However, it is critical to separate out these
points prior to analysis as they may provide valuable insights into what makes one group or one
individual different from others (e.g. issues of importance to them, unique context or
circumstances).
Steps to follow for consolidating and processing Quantitative Data
The following 6 steps outline the main tasks related to consolidating and processing quantitative
data, prior to analysis.
28
-
8/6/2019 RBM Manual 1
29/35
Step 1: Nominate a Person and set a Procedure to ensure the Quality of Data Entry
When entering quantitative data into the database or spreadsheet, set up a quality check procedure
such as having someone who is not entering data check every 10th case to make sure it was
entered correctly.
Step 2: Entering Numeric Variables on Spreadsheets
Numeric variables should be entered into the spreadsheet or database with each variable on the
questionnaire making up a column and each case or questionnaire making up a row. The type of
case will depend on the unit of study (e.g. individual, households, school, or other).
Step 3: Entering Continuous Variable Data on Spreadsheets
Enter raw numeric values for continuous variables (e.g. age, weight, height, anthropometric Z-
scores, income). A new categorical variable can be created from the continuous variable later to
assist in analysis. For two or more variables that will be combined to make a third variable, be
sure and enter each separately. (For example, the number of children born and the number of
children died should be entered as separate variables
and the proportion of children who have died could be created as a third variable). The intent is to
ensure that the detail is not lost during data entry so that categories and variable calculations can
be adjusted later if need be.
Step 4: Coding and Labelling Variables
Code categorical nominal variables numerically (e.g. give each option in the variable a number).
Where the variable is ordinal (e.g. defining a things position in a series), be sure to order the
codes in a logical sequence (e.g. 1 equals lowest and 5 equals the highest). In SPSS and some
other software applications it is possible to give each numeric variable a value label (e.g. the
nominal label that corresponds with the numeric code). For excel and other software that do not
have this function, create a key for each nominal variable that lists the numeric codes and the
corresponding nominal label.
Step 5: Dealing with a Missing Value
Be sure to enter 0 for cases in which the answer given is 0, do not leave the cell blank. A blank
cell indicates a missing value (e.g. the respondent did not answer the question, the interviewer
skipped the question by mistake, the question was not applicable to the respondent, or the answer
was illegible). It is best practice to code missing values as 99, 999, or 9999. Make sure the number
29
-
8/6/2019 RBM Manual 1
30/35
of 9s make the value an impossible value for the variable (e.g. for a variable that is number of
cattle, use 9999 since 99 cattle may be a plausible number in some areas). It is important to code
missing values so that they can be excluded during analysis on a case by case basis (e.g. by setting
the missing value outside the range of plausible values you can selectively exclude it from
analysis in any of the computer software packages described above).
Step 6: Data Cleaning Methods
Even with quality controls it will be necessary to clean the data, especially for large data sets
with many variables and cases. This allows for obvious errors in data entry to be corrected as well
as for excluding responses that simply do not make sense. (Note that the majority of these should
be caught in data collection, but even the best quality control procedures miss some mistakes.) To
clean the data run simple tests on each variable in the dataset. For example a variable denoting the
sex or gender of the respondent (1 = male, 2 = female) should only take values 1 or 2. If a value
such as 3 exists, then you know a data entry mistake has occurred. Also look for impossible values
(outside the range of plausibility) such as a child weighing 100 kg, a mother being 10 years old, a
mother being a male, etc
6.5.2. Guidelines for writing M&E Reports
1. Be as concise as possible given the information that needs to be conveyed. Be consistent with
the amount of information to be presented.
2. Focus on resultsbeing achieved compared with the expected results as defined in the logframe
or defined in the objectives, and link the use of resources allocated to their delivery and use.
Check that the expected results were realistic. All too often expected results are heroic and
unattainable!
3. Be sure to include a section describing why the data was collected and the report produced (e.g.
Introduction).
4. Be sure to include a section describing the data sources and collection methods used so that
your findings are objectively verifiable.
5. Be clear on youraudience (e.g. Country Directors, Governments, donors, technical persons)
and ensure that the information is meaningful and useful to the intended reader. You will need to
adjust the content of the report to the user of the information.
6. Write in plain language that can be understood by the target audience. Avoid technical jargon
and detail when submitted reports to management.
30
-
8/6/2019 RBM Manual 1
31/35
7. Ensure timely submission of progress reports. Even if incomplete in certain aspects or
component coverage, it is better to circulate key results in other areas rather than wait for the
complete picture.
8. Provide a brief summary (1 page) - sometimes called an executive summary - at the beginning
and ensure it accurately captures the content and recommendations in the report. This is often the
only part of the report that the majority of people who receive it will read.
9. Be consistent in your use of terminology, definitions and descriptions of partners, activities
and places. Define any technical terms or acronyms.
10. Present complex data with the help of figures, summary tables, maps, photographs, and
graphs.
11. Only highlight the most significant key points or words (using bold, italics or other stresses).
12. Include references for sources and authorities.
13. Include a table of contents for reports over 5 pages in length.
6.5.3. Guidelines for providing Feedback on Reports
The M&E Plan identifies the report in which the M&E information is included and sets out at
which forums or meetings the information or the reports themselves will be presented and
discussed. The M&E Plan, therefore, sets out the major formal feedback opportunities and ensures
that M&E reports are disseminated to all stakeholders and appropriate formal and informal
discussions are held concerning key finding. This aims to permit timely and informed decision-
making by the various stakeholder groups. This is especially crucial for information relating to
results.
Those units and individuals receiving M&E reports need to provide both formal and informal
feedback to the authors of reports. To the extent possible, they should acknowledge receipt of
progress report and provide comments regarding report conclusions, recommendations and
timeliness. Informal feedback to authors of M&E reports provides valuable lessons for them and
ensures them that the information is being used and reviewed. This in turn provides motivation to
maintain high data collection and reporting standards. Individualized feedback is especially
important when the author and the receiver are not working in the same organization or are in
different locations.
Examples of Formal Feedback Opportunities to be stated in the M&E Plan
31
-
8/6/2019 RBM Manual 1
32/35
The following are examples of meetings or workshops where M&E information or reports could
be shared. The appropriate content and purpose of sharing the information is briefly explained.
Government/donor/UN briefing sessions - To update key stakeholders on operation
progress, performance, partnerships and critical assumptions as well as results.
Quarterly progress review meetings - To review output progress (planned versus actual),findings and early evidence of outcome and to act on improvement proposals.
Semi-annual or annual meetings/ workshops - To review output progress (planned versus
actual), findings and early evidence of outcome and to formally agree to/decide on concrete action
to be taken.
Self-evaluation workshop - To include Implementing Partners (relevant Government
agencies and NGOs) in the finalization and review of the self-evaluation section report. They may
take part in the assessment of the operations performance.
Evaluation debriefing workshop - To present and discuss initial evaluation findings at the
end of the field mission stage of the evaluation to stakeholders to obtain their feedback ensuring
that it is incorporated into the final report and appropriately addressed in follow-up action.
6.6. INTEGRATION EVALUATION
Performance monitoring alone is often not sufficient to fully understand performance issues and
must be complemented by more in-depth evaluations. Integration evaluation with performance
monitoring allows managers to better understand causes and effects and to consider a boarder and
more fundamental set of interventions to improve project performance.
Performances Monitoring ---- Evaluation-------
Tracks and alerts management as to
whether actual results are being
achieved as planned.
Based on a results framework and
defined performance indicators
An on-going, routine effort to gather
data, analyze it, and report on results.
Systematic effort designed to answer
specific questions about
performance.
Focuses on why results are or are not
being achieved and other
performance issues.
Conducted as needed, also to address
issues raised during performance
monitoring.
32
-
8/6/2019 RBM Manual 1
33/35
It is important to emphasize that evaluation need not be a large expensive, pro forma undertaking.
Instead, evaluation or evaluation activities should be a management tool, driven by managers to
answer the critical questions they have about the performance of their projects.
Below is a summary of the kinds of performance issues and criteria evaluations are well-sited to
addressing:
Implementation performance. Assessing specific implementation/process problems or
the extent to which a project/ program is operating as intended.
Adequacy and Timeliness. Assessing the adequacy of inputs to carry out activities, and
the timeliness of inputs to bring about outputs and outcomes.
Outcomes and impact: Identifying the factors that explain the differences between
planned and actual results and the positive and negative-intended or unintended long-term
results produced by an operation, either directly or indirectly.
Effectiveness. Understanding the extent to which the operations objectives were
achieved, or are expected to be achieved, taking into account their relative importance.
Efficiency. Comparing project outputs or outcomes to the costs to produce them and
identifying alternatives to meet a given result.
Relevance. Reviewing the continued relevance of the project results in light of changing
beneficiary needs, partner country priorities, or donor goals,
Sustainability. Assessing the continuation of results after completion of a project (i.e.,
after donor support terminates).
Coverage and Targeting. Determining the extent to which targets and planned coverage
have been met and the right people have benefited at the right time.
33
-
8/6/2019 RBM Manual 1
34/35
6.7. USING PERFORMANCE INFORMATION
In results-based management systems, performance information (from both performance
measurement and evaluation sources) serves two primary aims or uses. One use is an internal
management tool for making project improvements; the second is accountability reporting.
Management improvement. The first major use of performance information is to provide
continuous feedback to managers about the results they are achieving, so they can then use
the information to improve their performance even more. This use is often referred to as
managing-for results. Sometimes discussions of this internal management use are further
subdivided into related aspects or processes-promotion of learning, facilitating decision-
making, and team building.
Learning. Performance information promotes continuous learning about what results are
being achieved and why. It makes the project team smarter about causes and effects,
risks, and other aspects of project management.
Decision-making. Performance information also provides the basis for good decision
making. It turns the decision-making process into a fact-based process, with heightened
understanding of the implementation strategy, the results achieved, and the relationship
between the two.
Team-building. The open reporting of performance of results makes the management
process transparent to all stakeholders, creates a more unified consensus to take the
necessary actions to improve performance, and leads to broader ownership andbuy-in to
project success.
Accountability. The second key use of performance information is for performance
reporting and accountability. Accountability-for results has several dimensions. One is
external accountability of the organization to the Executive Board and donor countries.
Another is internal accountability of individual employees or work units to higher levels in
the organizational hierarchy. Another is accountability to partners and to beneficiaries.
A mistake organizations often make is to assume that performance information will be used
simply because it is collected. A more likely scenario is that useful performance data gets
collected, but later, in the crush of project activities, no time is found to review, analyze, and use
34
-
8/6/2019 RBM Manual 1
35/35
the data to understand the results being achieved and improvements that are needed. Use of
performance information needs to be scheduled and planned. Managers should put a real data on
their work plan to take a step back and review performance, and they should provide adequate
time for staff to conduct the analysis for the performance review.