report on training on results-based ... on training on results-based management, monitoring and...
TRANSCRIPT
REPORT ON TRAINING ON RESULTS-BASED
MANAGEMENT, MONITORING AND EVALUATION
ISTANBUL, TURKEY
9 - 13 JANUARY 2017
ELD TRAINING www.eldtraining.com
RESULTS-BASED MANAGEMENT, MONITORING AND EVALUATION ISTANBUL, TURKEY - 9 - 13 JANUARY 2017 ELD TRAINING
www.eldtraining.com Page 1
CONTENTS INTRODUCTION ....................................................................................................................................... 3 SUMMARY AND EVALUATION ................................................................................................................ 4 DAY ONE .................................................................................................................................................. 6
INTRODUCTION ................................................................................................................................... 6 PRINCIPLES OF RBM ............................................................................................................................ 8 THE LOGICAL FRAMEWORK APPROACH: INTRODUCTION .................................................................. 9 THE LOGICAL FRAMEWORK APPROACH: ANALYSIS ............................................................................ 9 DAY-END FEEDBACK .......................................................................................................................... 10
DAY TWO ............................................................................................................................................... 11 PROBLEM ANALYSIS ...................................................................................................................... 11 STAKEHOLDER ANALYSIS ............................................................................................................... 11 SETTING OBJECTIVES ..................................................................................................................... 11 SELECTING A STRATEGY ................................................................................................................ 11
THE LOGICAL FRAMEWORK APPROACH: PLANNING ........................................................................ 12 DAY-END FEEDBACK .......................................................................................................................... 12
DAY THREE ............................................................................................................................................ 13 PROJECT MONITORING AND EVALUATION ....................................................................................... 13 KADIKÖY TOUR .................................................................................................................................. 13 DAY FOUR .......................................................................................................................................... 13 ESTABLISHING AN M&E SYSTEM ....................................................................................................... 14 DAY-END FEEDBACK .......................................................................................................................... 15
DAY FIVE ................................................................................................................................................ 16 DATA ANALYSIS ................................................................................................................................. 16 RESULTS-BASED REPORTING ............................................................................................................. 16 THEORY OF CHANGE ......................................................................................................................... 17 REVIEW, REFLECTION AND EVALUATION.......................................................................................... 17
PARTICIPANT LIST .................................................................................................................................. 18 TRAINING SCHEDULE ............................................................................................................................ 19 WORK PRODUCED BY PARTICIPANTS ................................................................................................... 21
RESULTS-BASED MANAGEMENT, MONITORING AND EVALUATION ISTANBUL, TURKEY - 9 - 13 JANUARY 2017 ELD TRAINING
www.eldtraining.com Page 2
INTRODUCTION This report documents the process and evaluates the outcomes of a five-day training on Results-Based Management, Monitoring and Evaluation held at Istanbul, Turkey, 9 - 13 January 2017. The objectives of the training were that participants would:
• Understand the philosophy and principles of RBM • Have a thorough knowledge and understanding of the Logical Framework Approach (LFA) • Be able to use the Logical Framework Approach in their own fields of work for the planning,
implementation and monitoring of projects • Understand how to apply Results-Based Management throughout the project cycle
(identification – formulation – implementation – monitoring/evaluation and reporting) • Have developed a Monitoring and Evaluation plan for their selected projects
The training was designed and delivered by ELD Director Neil Kendrick, and there were five participants in total, representing:
• Global Communities • The International Organisation for Migration
Other participants were, unfortunately, unable to attend - some due to the recent terror attack in Istanbul were not allowed to travel by their respective organisations, while severe weather caused major disruptions in air travel which affected others. The training was held at the Double Tree by Hilton Hotel, Moda, Kadikoy, and ran from 09.00 to approximately 17.00 daily. This report is based on daily feedback from participants, trainer's observations and a final evaluation from participants.
RESULTS-BASED MANAGEMENT, MONITORING AND EVALUATION ISTANBUL, TURKEY - 9 - 13 JANUARY 2017 ELD TRAINING
www.eldtraining.com Page 3
SUMMARY AND EVALUATION The training was monitored through regular feedback from participants which is included at the end of each day's narrative in this report. A final evaluation was conducted on the last day which indicated a high level of satisfaction. Despite its ambitious range of objectives, the training was successful in delivering what was expected. All participants felt their expectations were very much met and indicated that the design and methodology of the course contributed to this. All participants said they had, to a great extent, learned what was needed and felt confident about applying the tools and approaches in their working areas. (Although a couple of participants did suggest they needed to review what was covered prior to implementing.) One of the highest rated questions was whether the training would have an impact on their projects, which scored a solid '9' out of a possible 10. Several participants suggested that one or more additional training days would have been useful for further practising LFA and M&E planning. The final evaluation results - where participants rated the course out of a possible 10 on a range of criteria - are below. Scores presented are the average score from participants. EVALUATION QUESTION AVERAGE
RATING / 10 (based on 5 participant responses)
COMMENTS
OVERALL, WERE YOUR EXPECTATIONS AND OBJECTIVES MET?
8.8 Almost all my expectations were met; everything I needed to know was covered, although I would have like more time to study reporting
WERE THE MATERIALS, METHODOLOGY AND TOPICS COVERED SUPPORTIVE IN YOUR LEARNING?
9.2 The materials covered everything and answered all my questions Yes, but a wider range of examples from different contexts would be useful
DID YOU LEARN WHAT YOU NEEDED?
9.6 We learned something about every aspect of RBM: but to deeply understand this subject it will need months! Before this training I was facing real problems with designing a Log Frame and establishing indicators, so it was very useful Not 100%, but I learned many new / useful skills I think sessions on report writing would have been useful
RESULTS-BASED MANAGEMENT, MONITORING AND EVALUATION ISTANBUL, TURKEY - 9 - 13 JANUARY 2017 ELD TRAINING
www.eldtraining.com Page 4
DO YOU FEEL CONFIDENT / ABLE TO APPLY WHAT WAS LEARNED?
8.2 Yes - after this training I now have a clear picture in my mind about why we follow these steps of M&E I am confident, but it isn't my decision! It will require management support and time to implement what was learned
DO YOU THINK THIS TRAINING WILL HAVE A POSITIVE IMPACT ON FUTURE PROJECTS?
8.4 Sure - as we try to become more expert in our (M&E) unit we will try to apply everything we have learned in our upcoming plans Every single thing I have learned will be applied to various programmes This depends on support / cooperation from management!
RESULTS-BASED MANAGEMENT, MONITORING AND EVALUATION ISTANBUL, TURKEY - 9 - 13 JANUARY 2017 ELD TRAINING
www.eldtraining.com Page 5
DAY ONE The first day established norms and expectations for the training, explored the philosophy and concepts of Results-Based Management (RBM) and introduced the Logical Framework Approach (LFA). Participants also began the extended project work of the course, developing a Context Analysis and Problem Tree for their selected projects. INTRODUCTION The first session set the parameters for the training, allowing participants to introduce themselves and share their expectations. Participants also became familiar with the overall plan for the training. During the first session participants introduced themselves and shared their expectations of the training (collected in the Mind Map below). These expectations would serve both to help adapt the course to participants' specific needs and interests, as well as to provide a baseline against which progress could be measured on the final day.
RESULTS-BASED MANAGEMENT, MONITORING AND EVALUATION ISTANBUL, TURKEY - 9 - 13 JANUARY 2017 ELD TRAINING
www.eldtraining.com Page 6
After discussing these expectations the training plan was shared.
After a discussion on logistics, the introduction session was concluded.
RESULTS-BASED MANAGEMENT, MONITORING AND EVALUATION ISTANBUL, TURKEY - 9 - 13 JANUARY 2017 ELD TRAINING
www.eldtraining.com Page 7
PRINCIPLES OF RBM The second session responded to the participants' expectation of learning the theory and principles of RBM. After a brief explanation of the history of RBM and the reason it was brought into mainstream development thinking, we walked through a series of steps to clarify our conceptual understanding the approach. As well as slides, participants shared their own experiences and understanding freely both with the whole group as well as in small brainstorming sessions. First we examined the concept of results, seeing the results are changes in a state or condition that derive from a cause-and-effect relationship and that these changes can be intended, unintended, positive or negative and can occur at output, outcome and impact level. • The UNDG definition of RBM shared as a management strategy by which all actors ensure that
their • Processes • Products • Services
• Contribute to the achievement of desired results: • Outputs • Outcomes • Higher level goals / impact
• A key component of RBM is performance monitoring to objectively measure how well results are being achieved and report on measures taken to improve them
Next, participants shared their own ideas around the question 'Why RBM?' before looking at key features of the approach which included: • Problem analysis to understand causes • Stakeholder analysis • Structuring of programmes around a chain of desired results • Causality in the chain of results (if… then logic) • Use of ‘change language’ (future conditional) • Feedback loops to apply learning
Strategic selection of projects was also covered, considering the importance of projects selected reflecting value (contribution to a national or regional goal), support / partnership and capacity. Participants then discussed, in teams, both the benefits of applying RBM as well as the potential risks / limitations, before the trainer wrapped up the session with a summary of the key points.
RESULTS-BASED MANAGEMENT, MONITORING AND EVALUATION ISTANBUL, TURKEY - 9 - 13 JANUARY 2017 ELD TRAINING
www.eldtraining.com Page 8
THE LOGICAL FRAMEWORK APPROACH: INTRODUCTION The first afternoon session examined the Logical Framework Approach (LFA) - its history and purpose, the Log Frame (LF) itself, as well as overviewing the steps of the LFA process. After a discussion of the LFA's purpose and origins, we examined the Log Frame itself - the different sections and columns, and what each was used for. This was illustrated with a case study from a camp for IDPs (Internally Displaced People). Participants learned the two types of logic - the vertical logic of the Results Chain and the diagonal where each future hypothesis rests on assumptions at the level below. The session wrapped up with an overview of the 10 steps of the LFA process. THE LOGICAL FRAMEWORK APPROACH: ANALYSIS The rest of the day looked at the first two steps of the LFA - Context Analysis and Problem Analysis. First, various tools were introduced for Context Analysis. These included:
• Focus Group Discussion (including roundtables, forums and workshops) • PESTEL • SWOT • PRA and PLA – Social Mapping, Resource Mapping, Transect Walks, Problem Tree,
Ranking Activities • Desk Study • Survey • Observation • Interview • Gender Analysis
Particular attention was given to the Problem Tree as a tool for Problem Analysis. Participants then formed two teams to agree on a project to develop throughout the training. In one team, the three staff of Global Communities chose to address improving economic security among female Syrian refugees in Adana Turkey; while IOM selected a cross-border project to repair damaged homes in Idleb City, Syria. Teams then developed a Context Analysis for their selected issue and started to develop their Problem Trees.
RESULTS-BASED MANAGEMENT, MONITORING AND EVALUATION ISTANBUL, TURKEY - 9 - 13 JANUARY 2017 ELD TRAINING
www.eldtraining.com Page 9
DAY-END FEEDBACK At the end of each day participants noted down what they were happy about / worried about from the course so far. Feedback from Day One was overall, extremely positive.
Happy about ... Questions and Concerns ...
We achieved our first day's objectives successfully and covered all the points It was an interesting day with lots of practical information and learning The training is well planned and the trainer fully prepared I like your relaxed style of teaching The hotel environment is very good Nice first day
The meeting room wasn't well signposted Will we go deep into M&E I hope to learn more tools and techniques
RESULTS-BASED MANAGEMENT, MONITORING AND EVALUATION ISTANBUL, TURKEY - 9 - 13 JANUARY 2017 ELD TRAINING
www.eldtraining.com Page 10
DAY TWO On the second day teams were fully involved in group work on their selected projects. Problem Trees were reviewed, finalised and tested, stakeholder analysis carried out and project objectives set. Teams also selected their project strategies, developed their results chains and started to analyse the external environment / develop assumptions for their Log Frames. PROBLEM ANALYSIS The Problem Tree had been introduced on the first day, and in the first session teams finalised the Problem Trees for their selected projects. They used Mind Manager (a Mind Mapping software) for this, and the results are annexed. Problems were brainstormed, a core problem identified and the causes and effects organised into a logical sequence. Problem Trees were then tested for validity. Considerable time was devoted to this step due to its fundamental role in the LFA process and, consequently, its being a foundation for any results-based intervention. STAKEHOLDER ANALYSIS The day continued with an in-depth look at Stakeholder Analysis. As well as the standard approach - identifying various stakeholders' levels of interest in and influence on the project - we also explored Stakeholder Communication and Stakeholder Engagement. Teams analysed the various stakeholders' roles in the project and developed a communication strategy for each stakeholder to ensure clear communication as well as to take account for any resistance. Stakeholder Management Plans are included in participants' annexed M&E plans. SETTING OBJECTIVES The day continued with objectives setting. Participants learned how, my rephrasing the negative statements from the Problem Tree as positives, and testing the new tree (Objectives Tree) as a means-ends hypothesis. Those statements that could not be changed were set aside - later to emerge as either Assumptions (things beyond the project's ability to directly control) or Context (the environment in which the project will operate). SELECTING A STRATEGY In the next session, participants attempted to find common areas in the 'roots' of the objectives tree that represented various possible approaches to addressing the original core problem / achieving the project objective. The various approaches were then compared using a ranking system to assess the viability of each in order to combine those approaches into a comprehensive and prioritised project strategy.
RESULTS-BASED MANAGEMENT, MONITORING AND EVALUATION ISTANBUL, TURKEY - 9 - 13 JANUARY 2017 ELD TRAINING
www.eldtraining.com Page 11
THE LOGICAL FRAMEWORK APPROACH: PLANNING After sharing the results so far in the plenary, teams moved onto the planning steps of the Logical Framework Approach. Groups worked step-by-step through the rest of the day, turning their analysis into a clear plan. Using a simple Log Frame template they set outcome and goal, followed by outputs, activities and inputs. Teams then started to examine the project context, at precondition level as well as at output and outcome levels, before testing the Log Frame hypothesis. DAY-END FEEDBACK
Happy about ... Questions and Concerns ...
Thank you! Very satisfied with all the new information I got today Food was great Practical and full of details: thanks a million! I am happy we could complete a Log Frame in a systematic way Great planning by Neil! Happy with the environment of the training
Will the weather improve? Feel ready now to learn about M&E / reports
RESULTS-BASED MANAGEMENT, MONITORING AND EVALUATION ISTANBUL, TURKEY - 9 - 13 JANUARY 2017 ELD TRAINING
www.eldtraining.com Page 12
DAY THREE Day Three again saw a lot of practical, hands-on work. A lot of M&E theory was covered, and teams developed indicators for each level of result. The weather had also improved, so we were able, as planned, to give participants a free afternoon to join a guided tour of the district. PROJECT MONITORING AND EVALUATION The day began with a plenary discussion, sharing and presentation on Monitoring and Evaluation. Both terms were defined, and the differences between Monitoring and Evaluation clarified. After discussing what we monitor and why, we examined the differences between traditional and results-based monitoring - and how the purpose in results-based monitoring is to inform learning / decision making rather than to track, for the ultimate purpose of measuring change / progress towards results. The session concluded by examining a key component of M&E planning - defining Key Evaluation Questions. Evaluation questions define the key issues to be explored by an evaluation. They are developed and prioritised by program staff, evaluation personnel, funders, and other stakeholders. Participants then, in teams, developed the indicators for their projects. KADIKÖY TOUR We have started to recognise that taking a break during the training allows participants to have more successful learning outcomes. To this end, we hired Ms Nihan Vural (http://www.istanbultravelogue.com), a very unique guide with great knowledge of hidden Istanbul, to take participants on a walking tour of the district. This was followed by dinner at the world famous Çiya Restaurant, known for its wealth of Anatolian dishes.
RESULTS-BASED MANAGEMENT, MONITORING AND EVALUATION ISTANBUL, TURKEY - 9 - 13 JANUARY 2017 ELD TRAINING
www.eldtraining.com Page 13
DAY FOUR ESTABLISHING AN M&E SYSTEM The next session explored the steps of establishing an M&E system / framework. After defining what an M&E system is and its purpose, steps discussed were: 1. Defining the results 2. Establishing indicators 3. Determining how data will be collected, from where this data will come 4. Articulating risks and assumptions 5. Grouping indicators 6. Designing data collection tools 7. Data collection and management 8. Analysing the data and examine the trends and meaning behind the data 9. Interpreting, compiling and presenting the analysed data 10. Using the data to inform learning Results and indicators had already been developed as part of the LFA process; and many of the methods had already been covered by the Log Frames' MOV and during the first day's sessions on Context Analysis. However, we also explored other qualitative methods for data collection - Case Study and MSC (Most Significant Change) - and discussed methodological consideration when selecting / combining data collection methods. The importance of clarifying frequency and responsibility - responsibility for collection, verification, analysis and reporting - were also covered. On step 4, we spent time examining how to identify, measure and evaluate risk. Participants were introduced to the concept of logging risk through a Risk Register, and the importance of including periodic risk monitoring as a component of their M&E planning. Grouping indicators were discussed as a way to ensure more efficient collection of data. This means identifying indicators which can be verified using the same method or tool, ideally from the same source and at the same time / with the same frequency. The importance of having clear tools - named and explained - was also covered, to ensure that data is collected consistently by all staff. Throughout the afternoon teams developed their own M&E plans, which are annexed. The process forced them to critically examine each of their indicators and identify efficient, economical and verifiable ways to gather the information. As their plans developed a very comprehensive framework emerged, which, if applied to their selected projects, will lead to better monitoring and, consequently, a greater likelihood of achieving results. The IOM team presented their M&E plan at the end of the day for sharing / learning and feedback, while the Global Communities team presented back the next morning.
RESULTS-BASED MANAGEMENT, MONITORING AND EVALUATION ISTANBUL, TURKEY - 9 - 13 JANUARY 2017 ELD TRAINING
www.eldtraining.com Page 14
DAY-END FEEDBACK
Happy about ... Questions and Concerns ...
Thank you! Very nice group-centred day with very useful information I think the group work is the core interaction of this training and all group work was well-prepared, well-guided and the process easy to follow Presentations and group work were both great Well-planned day Great lunch As a programme person I am now much more motivated to learn more about M&E
Will we finish early on Friday?
RESULTS-BASED MANAGEMENT, MONITORING AND EVALUATION ISTANBUL, TURKEY - 9 - 13 JANUARY 2017 ELD TRAINING
www.eldtraining.com Page 15
DAY FIVE On the final day we reviewed the M&E plan from the Global Communities team, and looked at the remaining steps of M&E planning: Data Analysis and Reporting. We also introduced Theory of Change and examined how it differs from LFA, as well as discussing challenges we may face when operationalising M&E. DATA ANALYSIS Two exercises were conducted to practice Data Analysis. In the first exercise, participants examined and discussed data gathered from a countrywide survey of peoples' opinions in Nepal at the height of that country's civil war. The exercise was interesting in that there was often more than on interpretation for each graph; and through study and discussion it was possible to draw out underlying issues not explicitly obvious from the question. A useful learning from the task was that data is never neutral - everything has some significance. Also, it is the reporter's job to extract meaning from the data and guide readers / decision-makers to the main message the data offers. The second exercise was more qualitative and gave participants further practice in using Mind Mapping. From a list of 'facts and figures' on the status of children, teams first categorised the information and then compiled into a comprehensive map. From the map they were able to find underlying issues of cause and effect - such as early marriage and its connections to girl education, labour and maternal mortality - that the data, in isolation, are not immediately apparent. A key learning was that, through a three-step process, the meaning / conclusions of the data can become apparent in an organic way. RESULTS-BASED REPORTING The next session looked at reporting. We discussed the responsibilities of the reporter and the purpose of reporting, before examining specific questions to be considered when setting report objectives. We also examined the concept of the four levels of Evaluation - Reactions and Feeling, Learning and Knowledge, Change in Behaviour and Impact. We saw how, over time, we can move 'up' the levels, and with each periodic collection of data we can gather information on more levels as we move towards evaluation. We looked at the different types of reports produced during an intervention - progress reports, periodic reports and evaluations. Each was presented as a Mind Map. We also noted how these maps, due to their flexible nature, can be designed to also include Stakeholder Monitoring and Risk Monitoring. The session wrapped up with a discussion of key issues to consider during the reporting process.
RESULTS-BASED MANAGEMENT, MONITORING AND EVALUATION ISTANBUL, TURKEY - 9 - 13 JANUARY 2017 ELD TRAINING
www.eldtraining.com Page 16
THEORY OF CHANGE The next session introduced Theory of Change. While we did not go into great depth, we learned enough about the principles behind it to see how it can be used to mitigate some of the inherent weaknesses of LFA. Through various examples, participants clearly developed their understanding of this model. REVIEW, REFLECTION AND EVALUATION The final part of the day was spent reflecting on what had been covered so far. We reviewed what RBM is and its common features. Next, participants brainstormed the strengths and limitations of RBM, and also challenges they might face operationalising the approach in their organisations / projects. After a brief review of the key points of the course, and a look back at the first day's expectations, participants completed a short exit survey (see Summary and Evaluation) before certificates were presented.
RESULTS-BASED MANAGEMENT, MONITORING AND EVALUATION ISTANBUL, TURKEY - 9 - 13 JANUARY 2017 ELD TRAINING
www.eldtraining.com Page 17
PARTICIPANT LIST Hosam Alddeen Fael
Global Communities - Turkey Program
Ethem Can Ipek
Global Communities - Turkey Program
Ahmed Salkini Global Communities - Turkey Program
Boshra Biri IOM [email protected]
Muhammad Almasri
RESULTS-BASED MANAGEMENT, MONITORING AND EVALUATION ISTANBUL, TURKEY - 9 - 13 JANUARY 2017 ELD TRAINING
www.eldtraining.com Page 18
TRAINING SCHEDULE
1. INTRODUCTION
OBJECTIVES EXPECTATIONS TRAINING PLAN LOGISTICS
2. PRINCIPLES OF RBM
WHAT ARE RESULTS? WHAT IS RBM? WHY RBM? COMMON FEATURES OF RBM RBM AND THE PROJECT CYCLE STRATEGIC SELECTION OF PROJECTS APPLYING RBM
3. THE LOGICAL FRAMEWORK APPROACH: INTRODUCTION
WHAT IS THE LFA? WHAT IS A LOGFRAME? VERIFYING THE PROJECT DESIGN OVERVIEW OF THE LFA STEPS
4. THE LOGICAL FRAMEWORK APPROACH: ANALYSIS
CONTEXT ANALYSIS PROBLEM ANALYSIS STAKEHOLDER ANALYSIS SETTING OBJECTIVES SELECTING A STRATEGY
5. THE LOGICAL FRAMEWORK APPROACH: PLANNING
OUTCOME AND GOAL OUTPUTS ACTIVITIES INPUTS ASSUMPTIONS / PROJECT CONTEXT INDICATORS
RESULTS-BASED MANAGEMENT, MONITORING AND EVALUATION ISTANBUL, TURKEY - 9 - 13 JANUARY 2017 ELD TRAINING
www.eldtraining.com Page 19
6. PROJECT MONITORING AND EVALUATION
MONITORING EVALUATION DIFFERENCES MONITORING: WHAT AND WHY? TRADITIONAL VS. RESULTS-BASED M&E M&E PLANNING
7. ESTABLISHING AN M&E SYSTEM
DEFINING THE RESULTS ESTABLISHING INDICATORS DATA SOURCES AND METHODS RISKS AND ASSUMPTIONS GROUPING INDICATORS DESIGNING DATA COLLECTION TOOLS
8. DATA ANALYSIS
SURVEY ANALYSIS MIND MAPPING ANALYSIS
9. RESULTS-BASED REPORTING
PURPOSE OF REPORTING RESPONSIBILITIES OF THE REPORTER SETTING REPORTING OBJECTIVES FOUR LEVELS OF EVALUATION PROGRESS REPORTS PERIODIC REPORTS EVALUATIONS
10. THEORY OF CHANGE
11. REVIEW, REFLECTION AND EVALUATION
WHAT IS RBM? COMMON FEATURES OF RBM WEAKNESSES, LIMITATIONS AND CHALLENGES OF RBM OPERATIONALISING RBM PERSONAL ACTION PLANNING OPEN SPACE DISCUSSION COURSE EVALUATION CERTIFICATION
RESULTS-BASED MANAGEMENT, MONITORING AND EVALUATION ISTANBUL, TURKEY - 9 - 13 JANUARY 2017 ELD TRAINING
www.eldtraining.com Page 20
WORK PRODUCED BY PARTICIPANTS
IOM
Shelter rehabilitation- Idleb
Monitoring & Evaluation Plan 15 January 2017
Contents
Acronyms .............................................................................................................................................. 22
1 Introduction ................................................................................................................................... 22
1.1 Purpose of this plan .............................................................................................................. 22
1.2 Project summary ................................................................................................................... 23
2 Logical Framework ......................................................................................................................... 24
3 Indicators ....................................................................................................................................... 26
4 Roles & Responsibilities ................................................................................................................. 29
5 Data Flow ....................................................................................................................................... 31
6 Data Management ......................................................................................................................... 31
6.1 Storage .................................................................................................................................. 31
6.2 Analysis ................................................................................................................................. 31
6.3 Privacy ................................................................................................................................... 31
7 Appendices .................................................................................................................................... 32
7.1 Tools ...................................................................................................................................... 41
PDM Survey ......................................................................................... Error! Bookmark not defined.
FGD ....................................................................................................... Error! Bookmark not defined.
Observation .......................................................................................... Error! Bookmark not defined.
7.2 STAKEHOLDER MANAGEMENT ............................................................................................. 32
7.3 RISK MANAGEMENT .............................................................................................................. 33
RESULTS-BASED MANAGEMENT, MONITORING AND EVALUATION ISTANBUL, TURKEY - 9 - 13 JANUARY 2017 ELD TRAINING
www.eldtraining.com Page 21
Acronyms FGD Focus group discussion IPs implementing partner IOM international organization for migration IDPs internally displaced people PDM post distribution monitoring HH house hold Rehab. Rehabilitation CfW Cash for work TBD To be defined M&E Monitoring and Evaluation MoH Ministry of Health NGO Non-Governmental Organization TOT Training of Trainers
Introduction Into its 6th year, the conflict inside Syria remain unresolved and the heavy fighting between the government and opposition forces continue throughout the country causing both destruction and division resulting in massive humanitarian needs across Syria as well as massive displacement of the population. The number of people in need of assistance has grown to over 13.5 million, including 6.3 million internally displaced persons.
Purpose of this plan To define the tools processes, frequencies, responsibilities and methodologies of the M&E unit approach for the shelter rehab. Project.
RESULTS-BASED MANAGEMENT, MONITORING AND EVALUATION ISTANBUL, TURKEY - 9 - 13 JANUARY 2017 ELD TRAINING
www.eldtraining.com Page 22
Project summary IOM will rehabilitate 2000 damaged HH in Idleb with cash for work and providing in-kind construction material kit.
Title Shelter rehabilitation
Starting Date Jun. 2017
Duration 4 month
Partners -
Target Area Idleb city
Beneficiaries Partially damaged HH owners
Cost 836.000 USD
Funding Source TBD
Goal Improved living conditions for most vulnerable Idlep citizens by repairing 2000 damaged HH
RESULTS-BASED MANAGEMENT, MONITORING AND EVALUATION ISTANBUL, TURKEY - 9 - 13 JANUARY 2017 ELD TRAINING
www.eldtraining.com Page 23
Logical Framework
Narrative summary
Indicators MoV Assumptions
Goal Improved living conditions for most vulnerable
Idlep citizens
-Reduced mortality because of cold weather by 10%
-75% of HHH reported that their houses are in good condition and can protect them from bad weather
Makeshift hospitals in Idlep city statistics
-PDM data
Outcome 2000 citizens living in damaged HH in Idleb have proper shelter by Oct 17
-75% (1500HH) have repaired their houses by mid of November
PDM, FGDs and Observation datasets
Houses are not damaged and shelling doesn’t escalate
-no unusually severe weather
RESULTS-BASED MANAGEMENT, MONITORING AND EVALUATION ISTANBUL, TURKEY - 9 - 13 JANUARY 2017 ELD TRAINING
www.eldtraining.com Page 24
Outputs 2000 HH are able to repair and maintain
By the beginning of Sep. 90% of HHs have received their first CFW payment and kit
Progressive reports.
-Beneficiaries didn’t Sell the kits.
-Beneficiaries have enough Experience to repair their Houses
Activities 1-Importing building materials
2-Distributing cash for work
2000 construction kits -400.000$
-cash for work budget 200.000$
-project manager, M&E officer, 2 logistics and 4 operations assistants and 1 team leader 48.000$
-operational costs 48.000$
-Armed groups guarantee full access to the targeted area
-extensive shelling not existing.
Money transfer agency fully functional and guaranteed.
RESULTS-BASED MANAGEMENT, MONITORING AND EVALUATION ISTANBUL, TURKEY - 9 - 13 JANUARY 2017 ELD TRAINING
www.eldtraining.com Page 25
Indicators
Indicator By the beginning of Sep. 90% of HHs have received their first CFW payment and kit
Definition 1800 HH owners have received the kits and the money to start the repairing process.
Purpose To assess that timely assistance have reach most of Ben.
Baseline 0 HH received kits and money
Target 1800 HH received kits and money
Data Collection Field staff will collect beneficiaries' Signature to confirm receiving 1st CFW payment and the kit.
Tool Progressive report
Frequency Milestones
Responsible Field staff
Reporting Daily reports to be submitted by field staff to team leader, team leader to submit fortnightly reports to project manager, project manager to submit milestones reports to management.
Quality Control Onsite visits by M&E officer
RESULTS-BASED MANAGEMENT, MONITORING AND EVALUATION ISTANBUL, TURKEY - 9 - 13 JANUARY 2017 ELD TRAINING
www.eldtraining.com Page 26
Indicator 75% (1500HH) have repaired their houses by mid-November
Definition 1500 HH out of the targeted 2000 have completed repairing their houses
Purpose to insure project meets its purpose
Baseline 0 HH
Target 1500 HH
Data Collection Qualitative and quantitative methodology
Tool PDM , gender-disaggregated FGDs
Frequency One time
Responsible Enumerators and M&E officer
Reporting PDM report
Quality Control Data leaning at field level to be done by the M&E officer
RESULTS-BASED MANAGEMENT, MONITORING AND EVALUATION ISTANBUL, TURKEY - 9 - 13 JANUARY 2017 ELD TRAINING
www.eldtraining.com Page 27
Indicator 75% of HHH reported that their houses are in good condition and can protect them from bad weather
Definition HH out of the targeted 2000 have completed repairing their houses
Purpose Targeted community is fully aware of the intervention goal
Baseline 0 of selected HH
Target 1500 HH
Data Collection Qualitative and quantitative methodology
Tool PDM , gender-disaggregated FGDs
Frequency One time
Responsible Enumerators and M&E officer
Reporting PDM report
Quality Control Data leaning at field level to be done by the M&E officer
RESULTS-BASED MANAGEMENT, MONITORING AND EVALUATION ISTANBUL, TURKEY - 9 - 13 JANUARY 2017 ELD TRAINING
www.eldtraining.com Page 28
Indicator Reduced mortality because of cold weather by 10%
Definition By the makeshift hospitals cold-caused mortalities drop by 10%
Purpose Targeted community is fully aware of the intervention goal
Baseline last year 100 people died from the cold weather
Target After the project finish mortality percentage should be less than 9%
Data Collection Makeshift hospitals records
Tool
Frequency One time after the project Feb. 2018
Responsible M&E officer
Reporting Statistics report
Quality Control
RESULTS-BASED MANAGEMENT, MONITORING AND EVALUATION ISTANBUL, TURKEY - 9 - 13 JANUARY 2017 ELD TRAINING
www.eldtraining.com Page 29
Roles & Responsibilities
Role Responsibilities
Project design Project manager in consultation with M&E officer
M&E plan M&E officer in consultation with project manager
Onsite distribution monitoring
Operation assistant and M&E officer
FGD M&E officer to train the enumerators( FGD facilitator)
PDM data collection Enumerators under supervision of M&E officer
Data analysis and PDM report
M&E officer
RESULTS-BASED MANAGEMENT, MONITORING AND EVALUATION ISTANBUL, TURKEY - 9 - 13 JANUARY 2017 ELD TRAINING
www.eldtraining.com Page 30
Data Flow
Daily reports to be submitted by field staff to team leader, team leader to submit fortnightly reports to project manager, project manager to submit milestones reports to management
Data Management
Storage The data will be saved in internal organization server
Analysis Excel 2013 and SPSS 22 version will be used for data analysis
Privacy Data to be anonymised before any sharing unless in case of references for other projects
RESULTS-BASED MANAGEMENT, MONITORING AND EVALUATION ISTANBUL, TURKEY - 9 - 13 JANUARY 2017 ELD TRAINING
www.eldtraining.com Page 31
Appendices
STAKEHOLDER MANAGEMENT STAKEHOLDER Donor Beneficiaries LEVEL OF INTEREST High High LEVEL OF INFLUENCE High High WHAT SUPPORT DO WE NEED FROM THEM?
Money, sources and guidance
Interest
WHAT IS THE STAKEHOLDER’S ROLE?
Funding the project
Participation and consultation
WHAT ARE THE STAKEHOLDER’S INTERESTS AND CONCERNS?
Improving living conditions in conflict zone/ misuse of fund
Improved living conditions/ Financial concerns, time, quality of material provided
WHAT IS OUR STRATEGY TO GAIN SUPPORT OR MINIMISE OPPOSITION?
Planning and advocacy
Consultation and advocacy
HOW WILL WE COMMUNICATE WITH THIS GROUP?
Meetings FGDs and needs assessment
WHAT INFORMATION DO THEY NEED?
Detailed push information
Pull information about selection criteria , entitlements and assistance
HOW OFTEN? Periodically/ progressive reports
When needed/
PERSON RESPONSIBLE Project manager Field staff of IPs CHANNEL Direct
meetings/email Direct/ through community leadership
RESULTS-BASED MANAGEMENT, MONITORING AND EVALUATION ISTANBUL, TURKEY - 9 - 13 JANUARY 2017 ELD TRAINING
www.eldtraining.com Page 32
RISK MANAGEMENT RISK (DESCRIPTION) RISK
RATING TYPE RESPONSE REVIEW
DATE MONITORING AND CONTROL
Extensive shelling
High security Suspend activities in the extensive shelling days
1st of July
Beneficiaries might Sell the kits
mid Economic CFW to payments to be divided into 4 stages based on the progress to motivate repairing process
15th of Sep
Beneficiaries might not have enough Experience to repair their Houses
low Human resource
To promote community cooperation
15th of Sep
Armed groups guarantee full access to the targeted area
mid Security Rely on civil community to put some pressure on the armed groups
1st July 2017
Money transfer agency fully functional and guaranteed
low Security Agency to take full responsibility in moving money and distributing
15thjune 2017
RESULTS-BASED MANAGEMENT, MONITORING AND EVALUATION ISTANBUL, TURKEY - 9 - 13 JANUARY 2017 ELD TRAINING
www.eldtraining.com Page 33
<Logo>
EAH Team
Women Empowerment
Monitoring & Evaluation Plan
15 January 2017
Contents
Acronyms .............................................................................................................................................. 22
1 Introduction ................................................................................................................................... 22
1.1 Purpose of this plan .............................................................................................................. 22
1.2 Project summary ................................................................................................................... 23
2 Logical Framework ......................................................................................................................... 24
3 Indicators ....................................................................................................................................... 26
4 Roles & Responsibilities ................................................................................................................. 29
5 Data Flow ....................................................................................................................................... 31
6 Data Management ......................................................................................................................... 31
6.1 Storage .................................................................................................................................. 31
6.2 Analysis ................................................................................................................................. 31
6.3 Privacy ................................................................................................................................... 31
7 Appendices .................................................................................................................................... 32
7.1 Tools ...................................................................................................................................... 41
<Tool Title> .......................................................................................... Error! Bookmark not defined.
RESULTS-BASED MANAGEMENT, MONITORING AND EVALUATION ISTANBUL, TURKEY - 9 - 13 JANUARY 2017 ELD TRAINING
www.eldtraining.com Page 34
<Tool Title> .......................................................................................... Error! Bookmark not defined.
<Tool Title> .......................................................................................... Error! Bookmark not defined.
7.2 STAKEHOLDER MANAGEMENT ............................................................................................. 32
7.3 RISK MANAGEMENT .............................................................................................................. 33
Acronyms
M&E Monitoring and Evaluation NGO Non-Governmental Organization TOT Training of Trainers
Introduction From 2012 to 2016, 80K Syrian refugees flee to Adana City, Turkey; 10% of them are work-age female. They could not adapt the Turkish life/customs. In addition to that, they could not speak Turkish so in result they did not become accustomed with cultural differences. Moreover, having lack of marketable skills is another challenge for the women refugees, which resulted poor living conditions, dependency, and limited access to social rights.
RESULTS-BASED MANAGEMENT, MONITORING AND EVALUATION ISTANBUL, TURKEY - 9 - 13 JANUARY 2017 ELD TRAINING
www.eldtraining.com Page 35
Purpose of this plan Protection Programme with co-operation with M&E team worked to structure project’s framework systematically plan, we will collect data about the target beneficiaries
Project summary The main goal of this project is to improve the economic security of female refugee in Adana.
Title Women Empowerment
Starting Date 15 Jan 2017
Duration 4 months
Partners Direct implementation
Target Area Adana, Turkey
Beneficiaries 500 Syrian female refugees
Cost 500K USD
Funding Source International donors
Goal Providing economical security to female refugees in Adana
RESULTS-BASED MANAGEMENT, MONITORING AND EVALUATION ISTANBUL, TURKEY - 9 - 13 JANUARY 2017 ELD TRAINING
www.eldtraining.com Page 36
Logical Framework NARRATIVE SUMMARY INDICATORS MOV ASSUMPTIONS
GOAL FEMALE IDPS IN ADANA ARE more ECONOMICALLY SECURED
NUMBER OF SYRIAN WOMEN WORKERS IN ADANA INCREASED BY 50%
Participants Self reporting surveys
OUTCOME AT LEAST 75% OUT OF 500 SYRIAN WOMEN ARE gainfully EMPLOYED IN ADANA BY APRIL 2017
AT LEAST 75% OUT OF 500 SYRIAN WOMEN JOINED THE NEW JOB BY JUNE 2017
SURVEYS NO MAJOR FLUCTUATION IN THE WORK MARKET
OUTPUTS 1-500 SYRIAN WOMEN ARE more ABLE TO ACCESS LABOR MARKET IN ADANA
2-500 SYRIAN WOMEN ARE ABLE TO APPLY TO WORK LEGALLY IN ADANA
1-ALL TRAINED SYRIAN WOMEN HAVE ACQUIRED THE SKILLS OF HAIR DRESSING, MAKEUP, CHILDCARE, COOKING, HOSPITAL ENTRANCE, AND BASIC TURKISH LANGUAGE.
2-ALL TRAINED WOMEN UNDERSTAND THE REQUIREMENTS OF WORKING LEGALLY IN ADANA
1-BASIC LANGUAGE TEST
2-PRACTICAL
CERTIFICATION
1-CULTURALLY ACCEPT WORKS \ WILLNESS TO WORK
2- TURKISH GOVERNMENT WILL continue to PROVIDE WORK PERMIT FOR SYRIAN PARTICIPANTS
ACTIVITIES 1- COORDINATE WITH LOCAL MUNICIPALITY TO USE THEIR TRAINING CENTERS
2/ CONDUCT 6 VOCATIONAL TRAINING FOR THE NEXT 3 MONTHS
2-CULTURAL EDUCATION PROGRAMS
3-PROVIDING INFORMATION TO 500 SYRIAN WOMEN ABOUT WORKING LEGALLY IN TURKEY
INPUTS
STIPENDS FOR 500 BENEFICIARIES X 800 USD FOR 4 MONTHS
16 TRAINERS X 800 USD PER TRAINER\MONTH, 4 CENTER MANAGER X 1000 USD PER MANAGER\MONTH, 1 ADMIN ASSISTANT 900 USD\MONTH, AND 1 FIELD M&E ASSISTANT 900 USD\MONTH FOR 4 MONTHS
200 EDUCATION KITS
1-LOCAL AUTHORITIES APPROVES THE ACTIVITIES
RESULTS-BASED MANAGEMENT, MONITORING AND EVALUATION ISTANBUL, TURKEY - 9 - 13 JANUARY 2017 ELD TRAINING
www.eldtraining.com Page 37
Indicators
Indicator NUMBER OF SYRIAN WOMEN WORKERS IN ADANA INCREASED BY 50%
Definition Measuring the increase in the percentages of Syrian female workers in Adana
Purpose Providing economical security to female refugees in Adana
Baseline A current percentage of female Syrian workers in Adana is 7%.
Target Current percentages of female Syrian workers in Adana is 57%.
Data Collection Around the sample of 10 participants will be tracked
Tool Case study records
Frequency Once a month
Responsible Field M&E assistant
Reporting M&E assistant will collect the data and report it to the M&E and project managers
Quality Control Field M&E assistant will attend the trainings about case study and reporting
RESULTS-BASED MANAGEMENT, MONITORING AND EVALUATION ISTANBUL, TURKEY - 9 - 13 JANUARY 2017 ELD TRAINING
www.eldtraining.com Page 38
Indicator AT LEAST 75% OUT OF 500 SYRIAN WOMEN JOINED THE NEW JOB BY JUNE 2017
Definition
Purpose To know how many OF 500 SYRIAN WOMEN ARE LEGALLY EMPLOYED IN ADANA BY APRIL 2017
Baseline 0% of targeted female Syrian women are employed
Target 75% of targeted Syrian women refugees will be employed
Data Collection Conduct baseline survey on the start of the project and second survey will be conducted at the end of the project on June 15, 2017.
Tool Surveys
Frequency Twice (mid of the training then end of the training)
Responsible Administrative assistant
Reporting Administrative assistant will conduct and collect the survey results and report it and share the survey data with the program and M&E managers
Quality Control Program officer will triangle the information for the audit process
Roles & Responsibilities
Role Responsibilities
Field M&E assistant Field M&E assistant will collect the data and report it to the M&E and project managers
Admin assistant Administrative assistant will conduct and collect the survey results and report it and share the survey data with the program and M&E managers
Program officer Program officer will triangle the information for the audit process
M&E manager Compile the reports and submit it to the donor
Program manager Review the technical aspect of the report and share it with M&E manager
RESULTS-BASED MANAGEMENT, MONITORING AND EVALUATION ISTANBUL, TURKEY - 9 - 13 JANUARY 2017 ELD TRAINING
www.eldtraining.com Page 39
Data Flow
RESULTS-BASED MANAGEMENT, MONITORING AND EVALUATION ISTANBUL, TURKEY - 9 - 13 JANUARY 2017 ELD TRAINING
www.eldtraining.com Page 40
Data Management
Storage and Analysis Collected data will be uploaded to organization`s database by mobile data application. Program officer has access to data and responsible to collect it and create charts on excel to share it with Program Manager. Electronic copy of data and reports will be stored in cloud for 5 years.
Privacy Field M&E assistant and Program officer have direct access to personal data. Date will be stored for audit process and will be deleted after 5 years. Reports will not refer to people names.
Appendices
Tools Fulcrum
Fulcrum is a mobile application to collect and share data.
Microsoft Excel
We are using data sheets of Microsoft Excel to track the attendance of the participants. Also
Microsoft Word
For biweekly and sector reports,
STAKEHOLDER MANAGEMENT By analysing the stakeholders before we start, we should identify and prioritize them professionally. We also determine their needs and document the results. If we analyse their needs, we can manage it based on win-win strategy. Our main two stakeholders are Syrian women refugees and local authorities in Adana. Their level of interest and influence is high.
RESULTS-BASED MANAGEMENT, MONITORING AND EVALUATION ISTANBUL, TURKEY - 9 - 13 JANUARY 2017 ELD TRAINING
www.eldtraining.com Page 41