This report is commissioned by Action Against Hunger | ACF International. The comments contained herein reflect the opinions of the Evaluator only.

External Evaluation

Humanitarian Support to Conflict and Flood-affected Populations in Pakistan

Funded by ECHO Ideas for Life Trust , September 2012

Photo © Luqman Ahmed

2

1 Contents

1 Contents ...................................................................................................................................................... 2

2 Acknowledgements ..................................................................................................................................... 3

3 List of Abbreviations ................................................................................................................................... 4

4 Basic Information of the Project .................................................................................................................. 5

5 Executive Summary .................................................................................................................................... 6

6 Background and the Context ...................................................................................................................... 9

6.1 ACF’s Intervention .............................................................................................................................. 9

6.2 Purpose and Objectives of the Evaluation ........................................................................................ 10

6.3 The Evaluation Methodology ............................................................................................................ 10

7 Evaluation Findings & Analysis................................................................................................................. 14

7.1 Project Design................................................................................................................................... 14

7.2 Beneficiary Targeting ........................................................................................................................ 16

7.3 Project Implementation ..................................................................................................................... 17

7.4 Support Systems .............................................................................................................................. 27

7.5 Project Sustainability ........................................................................................................................ 28

7.6 FSL Surveillance ............................................................................................................................... 29

7.7 Project Monitoring ............................................................................................................................. 29

7.8 Livelihoods Recovery Committees (LRC) ......................................................................................... 31

8 Lessons Learnt ......................................................................................................................................... 33

9 Recommendations .................................................................................................................................... 34

10 Annexes ................................................................................................................................................ 39

10.1 Rating of the Project based on DAC Criteria ........................................................................................ 39

10.2 Best Practice ......................................................................................................................................... 41

10.3 Evaluation TORs ................................................................................................................................... 42

10.4 The Evaluation Team ............................................................................................................................ 54

10.5 Evaluation Schedule ............................................................................................................................. 55

10.6 List of Persons Interviewed and Sites Visited ....................................................................................... 56

10.7 Documents Consulted .......................................................................................................................... 65

10.8 Data Collection Tools ............................................................................................................................ 66

3

2 Acknowledgements The evaluation team would like to thank Action Against Hunger (ACF) staff at Head Quarters (Ben Allen, Catriona Statham and Charmaine Brett), Pakistan Country Office team (Onno van Manen, Paola Maria Valdettaro (throughout the entire evaluation process), Eric Rheinstein (during the initial stages of evaluation) and Field teams (Mushtaq in Dir and Waqas Halim in Peshawar for coordination and support during the External Evaluation of ACF’s Humanitarian Support to Conflict and Flood-affected Populations in Pakistan. This report is the result of the efforts, contributions and support of many people from ACF, district line department, district management, UN clusters in Khyber Pakhtunkhwa (KP) and the communities in Charsadda, Nowshera and Dir. We thankfully acknowledge their support and contributions. We would like to thank the members of Committees, male and female beneficiaries of several key project interventions for providing us valuable feedback on them. The team would also like to thank the stakeholders at the district level, district line department, for taking the time to meet us and share their valuable views and expertise. We would also like to thank our field team, Muhammad Jamal Shah, Mehreen Anis, Laila Khan, Kiran Khan, Zakaullah Khalil, Kamran Khan for their efforts and hard work in data collection in Charsadda, Nowshera and Dir. The dedication and hard work of the team consisting of field staff and technical experts enabled us to compile this report. We would also like to thank our colleagues at Ideas for Life Trust, SadafRaza and Sara Saeed, for backhand help and support during the evaluation. We are also thankful to the statisticians, Mohammad Kashif and Mohammad Nadeem for processing the data. Luqman Ahmed - Zulqarnain Malik – Dr Bahadar Nawab Ideas for Life Trust Islamabad, Pakistan July 2012

4

3 List of Abbreviations ACF Action Against Hunger ACO Assistant Coordination Officer CFW Cash for Work DAC Disaster Assistance Committee DAP Di-Ammonium Phosphate DCO District Coordination Officer DDMO District Disaster Management Officer DRM Disaster Risk Management ECHO European Commission's Humanitarian Aid Office EDO Executive District Officer FATA Federally Administered Tribal Area FDMA FATA Disaster Management Authority FGDs Focus Group Discussions FSL Food Security and Livelihood HAP Humanitarian Accountability Partnership HH Households I/NGOs International /Non-Governmental Organizations IDPs Internally Displaced Persons KG Kilogram KPK Khyber Pakhtunkhwa LRC Livelihoods Recovery Committee NIC National Identity Card NOC No Objection Certificate PDMA Provincial Disaster Management Authority SSI Semi structured Interviews TORs Terms of Reference UCG Unconditional Cash Grant UCs Union Councils UN United Nations WASH Water, Sanitation and Hygiene WUCs Water User Committee

5

4 Basic Information of the Project Project Title: Humanitarian support to Conflict and Flood-affected populations in Pakistan Name of organization

ACF International

Target Areas: District Nowshera: UCs Mughalki, Chowky Town, Kabul River, Pir Sabaq Navi Kali District Charsadda: UCs Agra, Mirza Dherai, Doshera, Tarnab, Dheri Zardad District Lower Dir: UCs Lal Qilla, Khonge, Kotki, Khall, Chakdara

Total Beneficiaries

146,825 flood / conflict affected persons

Project Results

WASH Beneficiaries –98,325 persons Key Activities:Provision of Safe and Sufficient water, Provision of Household Latrines, Hygiene Promotion and Hygiene Kits, Waste Management Systems, Rehabilitation of Irrigation Channels, Capacity Building FSL Beneficiaries – 45,000 persons Key Activities:CFW, UCG, Livestock Support, Agriculture Support, Horticulture Support, Surveillance, Capacity Building DRM Beneficiaries – 3,500 persons Key Activities: Community Risk & Vulnerability Mapping, Sensitization of local authorities on DRR, Reinforcement of coping mechanisms

Duration of project

12 months (May 2011 to April 2012) + 1 month extension

6

5 Executive Summary In the spring of 2009 military operations against militants in Khyber Pakhtunkhwa province, lead to an exodus of about 2.3 million people. Large-scale displacement occurred again in Pakistan in August and September 2010 after the worst flooding that hit the country, forced over 7 million people to leave their homes.With support from ECHO, ACF provided safe drinking water, proper hygiene and sanitation facilities to 10,925 flood and conflict-affected households and food security and livelihoods support to 5,000 households. Upon completion of the project, an external evaluation was carried out to evaluate the impact and approach of ACF’s project activities.This study focused on the following areas: Project Design; Beneficiary Targeting; Project Implementation; Support Systems; Project Sustainability; FSL Surveillance; Project Monitoring and Livelihoods Recovery Committees (LRC). The Evaluation was done based on DAC criteria: Impact, Sustainability, Coherence, Coverage, Relevance / Appropriateness, Effectiveness and Efficiency to gauge success of the project or otherwise. The evaluation team used a mixed-method approach for this evaluation. Findings of this evaluation study and their analysis are presented as under: Project Design: All the areas targeted under this intervention by ACF were affected either by the conflict, as District Lower Dir, or by 2010 floods, as Districts Charsadda and Nowshera. ACF targeted the most urgent needs of the affected population in resettling and restoring their lives and livelihoods. The overall assistance was found relevant to the needs of the affected population. This was possible through the initial assessments carried out by ACF teams.ACF coordinated with the WASH and Food Security Clusters, line departments as well as with other agencies working in the project areas for information sharing, networking, and enhanced local level coordination. The line departments in the project districts were pleased with project interventions, as they were engaged in the implementation of the activities. For instance the Agriculture department had been involved in the Agriculture trainings; distribution of seeds. Similarly, the Livestock department was also engaged in the selection of goat breeds, vaccination of goats and trainings on livestock management. The Agriculture and Livestock Departments were also invited to visit the communities to monitor the progress of project activities.ACF FSL technical experts developed the Working Manual for Village Livelihood Recovery Committees (LRCs) that were formed within the KPHA project under study. ACF teams were then trained over all necessary steps to adhere to the guidance provided within the manual and ensure standardization. Cash Transfer Guidelines were also developed by ACF Technical Experts to ensure transparency, accountability and uniformity but the implementation team faced challenges in terms of maintaining impartiality within the LRCs / WUCs during beneficiary selection resulting in 40% of the CFW beneficiaries receiving less number of man days than the standard food basket. Problems related to transferring cash to beneficiaries remained a challenge that resulted in ACF teams adopting several approaches rather than ensuring standardization. This indicates ACF team’s flexible approach to adapt to the situation and facilitate the beneficiaries to the maximum in transferring cash in a timely fashion. It is important to negotiate with the banks for providing cash to the disaster affected communities even in the remote areas of KP in a swift, safe and dignified manner. Beneficiary Targeting: The evaluation team learnt during the course of evaluation that except for a few odd cases, those who received the inputs were generally selected as per the agreed criteria. Most importantly, unconditional cash grants were provided to the disabled and widows who needed them the most. Some complaints were raised regarding nepotism by the LRCs / WUCs but no concrete evidence could be provided. Project Implementation:92% of the sample beneficiaries of agriculture support confirmed that the inputs were provided timely. Wheat seeds were very effective and of very good quality. Fairs organized to facilitate provision of goats to the affected communities were a highly appropriate modality which allowed the beneficiaries to choose preferred animals as per their need. The

7

trainings related to agriculture and livestock support were also appropriate and relevant to the needs and the context. CFW was highly appropriate to the needs of the community.Water provision through dug wells or a repaired scheme was demand driven and highly useful. Most of the respondents were satisfied with the taste of the water. The hygiene kits contained useful items for water storage and transportation. Hygiene promotion was very informative and useful. Less flexibility was shown to accommodate the needs of the disabled, pregnant women and elders with regards to changes in latrine design.Risk assessments and trainings on DRR were also highly relevant. If ACF had complete twelve months to implement this project, the impact upon the lives of the disaster and flood-affected people would have been manifolds. Some concerns still need to be addressed such as instances of impartiality noticed within the LRCs; more effective planning for cash transfers; regular and process oriented monitoring to ensure quality and accountability which play a crucial role in achieving the desired impact. It is commendable that ACF started this project with an at least 6 months’ time handicap and yet managed to reach vulnerable communities.77% of the sample beneficiaries of agriculture support indicated that they were satisfied with the quantity of crop inputs.82% of the sample beneficiaries of livestock support were satisfied with the assistance. 97% of the sample beneficiaries confirmed receiving trainings on effective crop management and livestock management. Most of the money earned by the CFW labourers was utilised to meet their immediate food and health needs.100% sample of beneficiaries indicated their satisfaction over the quality and quantity of water. 94% of the sample beneficiaries were satisfied with the latrines.100% of the sample beneficiaries confirmed that they were satisfied with the hygiene kits received.The communities were very happy with the rehabilitation of irrigation channels for agriculture which had a direct positive impact over their crop yields. Support System: ACF has dedicated technical experts on board. Support systems have played a significant role in providing maximum assistance to the program teams and in setting up protocols to ensure transparency and accountability. However, support systems should provide additional facilitation and support to program teams in reaching disaster affected communities. They should be actively engaged in supporting program teams in Cash distributions among the beneficiaries, particularly in cases where large beneficiaries have to be served in a short period of time. Project Sustainability: Many households in the target areas had stopped open defecation and preferred to use the household latrines. Many other households inspired from the project initiated the construction of latrines within their compounds themselves. The rehabilitation of irrigation channels for agriculture purposes improved crop yields. The engagement of the affected communities with local authorities in the form of LRCs / WUCs may continue to a limited extent in taking ownership of the collective problems. FSL Surveillance: ACF planned regular surveillance of local market prices for essential food items but realized that the field team lacked skills and capacity to design and implement surveillance activity and was overwhelmed with the execution of other components of the project. Project Monitoring: Upon delivery of various inputs ACF teams conducted post distribution monitoring surveys to track the outputs and outcomes. Apart from this, ACF team also planned and carried out monitoring visits with officials from Agriculture and Livestock Departments. At the local level LRCs and WUCs were also engaged in monitoring. Monitoring and Evaluation as a key program function within ACF program in Pakistan still needs a lot of improvements. Livelihoods Recovery Committees (LRC): ACF teams arranged broad based community meetings in each target village and invited the communities to form village committees. LRCs / WUCs were in several cases represented by individuals or dominant groups within the villages. Female committees were not aware of their role and details of project inputs and were not even active. The

8

ACF team was committed to the accountability and complaint mechanisms and the committee office bearers and beneficiaries were aware of the complaint system. Lessons Learnt: Participatory planning is extremely important right from inception to completion of the project; more investment in community mobilization, greater is the impartiality within community structure resulting in greater impact; regular output tracking coupled with process monitoring ensures reaching desired impact. Recommendations: ACF is recommended to negotiate with banks in future and improve Cash Transfer mechanisms; integrate DRR more actively in ACF emergency programs such as KPHA; ACF must invest more time and resources in building its capacity in developing and running FSL Surveillance system; ACF needs to set up robust monitoring systems that ensure successful delivery of results as well as encourage adherence to processes and promote standardization; ACF is strongly recommended to build the capacity of its teams in community organization and mobilization; there is a clear need for ACF to invest more on integration of Humanitarian Accountability principles across its programs and initiate capacity building of its teams in HAP.

9

6 Background and the Context Over the last decade, Pakistan has experienced large-scale involuntary internal displacement caused by a range of factors. The main cause for internal displacement in the spring of 2009 was military operations against militants in Malakand region of the Khyber Pakhtunkhwa province, leading to an exodus of about 2.3 million people in a little over a fortnight and creating one of the largest displacement crises in recent times. Military offensives against Taliban militants in the Federally Administered Tribal Area (FATA) bordering Afghanistan also contributed considerably to involuntary displacement, pushing the number of internally displaced persons to 2.7 million between April and July 2009, the largest internal displacement of population in the country’s history until then. Large-scale displacement occurred in Pakistan in August and September 2010 again after the worst flooding to hit the country in living memory affected 20 million people, forcing over 7 million people from their homes. 2010 was the second consecutive year since the Afghan refugee crisis began in 1979 that the number of internally displaced persons in Pakistan exceeded that of registered refugees. Although most of the flood-affected IDPs returned to their home areas soon after floodwater receded, they were mainly living out in the open as over 1.9 million houses had been damaged or destroyed across the country. The large-scale displacement crises in 2009 and 2010 were only the latest human exodus in Pakistan. They had been preceded by dislocation of population following clashes between rival militant and sectarian groups in the tribal areas; military operations against extremist militants in parts of FATA and against insurgents/dissidents in Balochistan; generalized violence and violations of human rights; and other natural and human-caused disasters, including a devastating earthquake in Khyber Pakhtunkhwa and Azad Kashmir; the annual havoc caused by floods across the country; sea intrusion; and displacement induced by development projects such as enhancement of water storage capacity at Mangla Dam, the construction of Mirani Dam and the sale of two Islands off Karachi to an international real estate developer. All indicators suggest that internal displacement will remain a key issue of concern in Pakistan at least in the medium term. Pakistan’s government failed to respond adequately to the conflict-induced displacement crisis between 2004 and 2008. But since then, in tandem with international agencies, it has provided support to a huge number of IDPs. Its achievements include the registration of more than half a million internally displaced households; the provision of food and non-food items to the majority of those households; of national identity cards to almost 80,000 displaced women; of health care services to many of those residing in displacement-affected communities, and shelter for many of the most vulnerable IDPs in camps. Cash cards distributed to at least 320,000 households have been an efficient means of reaching IDPs outside camps (UNHCR, January 2010).

6.1 ACF’s Intervention ACF was part of the Flash Appeal and actively contributed/supported the rolling out of the Cluster system in KPK and in Sindh, where it began supporting flood affected populations. With support from ECHO, ACF provided safe drinking water, proper hygiene and sanitation facilities to 10,925 flood and conflict affected households and food security and livelihoods support to 5,000 households.

10

6.2 Purpose and Objectives of the Evaluation As per TORs, the objective of this evaluation is to assess the impact and approach of ACF’s project activities in Khyber Pakhtunkhwa and to provide recommendations for future programming. This study focused on the following areas: Project Design Beneficiary Targeting Project Implementation Support Systems Project Sustainability FSL Surveillance Project Monitoring Livelihoods Recovery Committees (LRC)

The Evaluation was done based on DAC criteria: Impact, Sustainability, Coherence, Coverage, Relevance / Appropriateness, Effectiveness and Efficiency to gauge success of the project or otherwise.

6.3 The Evaluation Methodology The evaluation team used a mixed-method approach for this evaluation. Participatory tools and data collection techniques were employed.

6.3.1 Quantitative Data Collection Household Survey: In order to gather specific feedback from direct beneficiaries and to draw a statistical analysis of objectively verifiable indicators and variables within the project framework, a Household Survey was conducted in the target areas sampled under this evaluation.

6.3.1.1 Sample Size The evaluation team conducted a household survey over a sample of 400 households spread across sampled target areas covered under the project. This representative sample was selected based on the Guide developed by Krejcie and Morgan 1970, presented here.

11

6.3.1.2 Sampling Technique – Multi Stage Sampling Villages were selected based on purposive and proportionate sampling techniques in a multi-stage sampling manner. Sampling was based on the lists of beneficiaries and target villages with varying population. A list of sample villages was developed which was shared with the project team for verification and finalization. At the first stage, Proportionate Sampling technique was used to select villages. The sampling was done on the basis of list of target villages provided by ACF. The list carried the population of each target village and the total number of beneficiaries in each one; in some villages due to a bigger proportion of beneficiaries than those in others, more villages with a higher concentration of beneficiaries of the project were selected to facilitate effective interaction with them considering time constraints. Some of those villages were also selected where only WASH interventions were carried out to understand the differential needs and appropriateness of the response. At the second stage Purposive Sampling technique was used to prioritize villages further for this evaluation study. This was done based on the list of villages shortlisted at the first stage. Each target village would have received assistance with varying population and varying needs. Considering the possibility where all the villages might not have a standard set of interventions or had all types of beneficiaries in each village, the purposive sampling technique allowed choosing villages from those selected in the first phase to ensure presence of all types of beneficiaries for various interventions in order to grasp a comprehensive picture of the project, while staying within the limited resources. Selection of households was done randomly using the beneficiary lists, made available by ACF. Through quota sampling techniques, households were selected from each sample village based on the proportion of beneficiary households in various villages. Higher household interviews were conducted in villages with a higher number of beneficiary households. While selecting the sample villages due consideration was also given to accessibility in terms of remoteness of each sample village from central locations in each district; safety and security within these areas were also considered.

6.3.1.3 Sample Areas Please refer to Annexure 10.6.

6.3.1.4 Tool – Household Survey Questionnaire The HH Survey Questionnaire was designed to help the evaluation team to gather statistic information on multiple project indicators such as HH income, HH expenditure on food, HH assets, food security, recovery of livelihoods, etc. These variables and indicators were drawn during the secondary data collection phase, particularly from the review of project documents and through discussions with ACF teams. Efforts were made by the technical experts on the team to develop a comprehensive questionnaire that would cover all components and thematic areas within the project framework.

12

6.3.1.5 Household Survey Team The household survey was administered by locally recruited enumerators who had undergone extensive training process to enable them to administer the household survey questionnaire as per clear understanding of various thematic questions and international norms and standards. The field survey teams (one male and one female supervisors, two male and two female enumerators) had reasonable gender balance to facilitate access to both men and women beneficiaries of the project to capture gender differential impact.

6.3.2 Qualitative Data Collection

6.3.2.1 Focus Group Discussion Focus group discussions were organized with different groups of beneficiaries based on different components of the project. The evaluation team organized two FGDs, one with male beneficiaries and LRC members and one with female beneficiaries and LRC members in each selected village in order to capture maximum information on each thematic area of the project for cross validation and comprehensive qualitative analysis. Each FGD was conducted with up to 14 persons and the respondents were selected randomly from the beneficiary lists. In consultation with ACF, a FGDs guide was developed to facilitate the evaluation team in guiding all the discussions. Participants of the FGDs were selected on the basis of the following criteria: 1. Up to 14 male respondents (beneficiaries of Food Security and Livelihoods and WASH support)

selected through simple random sampling technique from each sample village using beneficiary lists provided by ACF.

2. Up to 14 female respondents (beneficiaries of Food Security and Livelihoods and WASH support )selected through simple random sampling technique from each sample village using beneficiary lists provided by ACF.

3. Representation of members of LRCs from each sample village, where available. 4. A portion of participants of the FGDs to be selected from the most vulnerable groups of the

village. A total of 35 FGDs were conducted with male and female beneficiaries (18 with male and 17 with female), in three districts of Nowshera, Charsadda and Dir.

6.3.2.2 Semi Structured Interviews The data collection team also administered face to face semi/structured interviews with non-beneficiaries and with key informants in the selected villages (identified during the initial stages of the evaluation, in consultation with the field teams). The SSIs were the tools through which opinions and perceptions of non-beneficiaries and key stakeholders were gathered regarding project design, implementation, accountability and impact from an external point of view. SSI guides were developed in order to extract the necessary information required for a comprehensive qualitative analysis. SSIs were conducted with the following people (the list of external stakeholders and non-beneficiaries is provided at the end of the report, as annexure): 1. Male and female non beneficiaries in selected villages 2. ACF Project Staff 3. ACF Management 4. Local authorities

a. DCO / ACO / DDMO

13

b. Agri & Livestock officers 5. UN Clusters

A total of 10 SSIs were conducted in the project areas and nine interviews were conducted with the external stakeholders, while number of consultations was made with the staff and management of ACF at Country office Islamabad, Peshawar and Dir.

6.3.2.3 Case Studies Case Studies were collected from the field for both demonstration and evidence purposes. A total of two human interest stories were collected during the course of the evaluation.

6.3.2.4 Direct Observations The field team remained observant during field visits. The field team also took pictures both for evidence and reporting purposes.

6.3.3 Primary Data Collection – Tools

S# Type of Tool Type of Respondent Number Total 1 HH

Interviews Direct Male & Female Beneficiaries 400 HH

Questionnaire 2 FGDs Direct Male Beneficiaries & LRC

Members 1 / Sample Village 18

3 FGDs Direct Female Beneficiaries & LRC Members

1 / Sample Village 17

4 Case Studies Direct Beneficiaries (Male or Female)

WASH FSL

4

5 SSIs ACF Project Staff 2 2 6 SSIs Non Beneficiaries (Male & Female) 10 7 SSIs ACF Management 8 8 8 SSIs The District Coordination Officer /

ACO / DDMA 1 1

9 SSIs Agricultures & Livestock officers 3 3 10 SSIs UN Clusters 2 2

6.3.4 Data Analysis Analysis of the entire quantitative data was done using FoxPro and SPSS packages for statistical analysis. For this purpose, the household questionnaires were coded and data entry was done simultaneously as per the coding plan which was developed by the data management team. The enumerators were trained in coding and data entry by the core consultants. Qualitative data was also analyzed using comparative analysis approach for expressing opinions and perceptions. It was ensured that reported findings and analysis would be substantiated with numeric data. Where required, for ease of comprehension, information was collated and presented in tabulated and graphic forms. The recommendations were framed in a way that they could assist in follow up action planning.

14

7 Evaluation Findings& Analysis

7.1 Project Design Targeting of Affected Areas Both primary and secondary data collected during the course of evaluation revealed that all the areas targeted under this intervention by ACF were affected either by the conflict, as District Lower Dir,or by 2010 floods, as Districts Charsadda and Nowshera. Review of ‘Integrated Rapid Assessment Report – FSL and WASH conducted by ACF in January 2011 and discussions with the communities during the course of the evaluation suggested that though there were several organizations that flooded the devastated area and provided one time relief to the affected population, very few considered supporting these communities in their recovery. ACF targeted the most urgent needs of the affected population in resettling and restoring their lives and livelihoods. Prioritization of Needs A general consensus reached by all the respondents in various villages targeted by ACF suggests that the overall assistance was highly relevant to the needs of the affected population that needed immediate support to rebuild their lives. From the discussions with the affected population in sample villages, the evaluation team learnt that the beneficiaries believed that it was a highly effective design as the project supported communities in restoring their livelihoods and addressing serious health and hygiene concerns, through engaging men, women and children. This was possible through the initial assessments carried out by ACF teams to understand the context and identify priority needs of the affected population. This participatory process at the design stage allowed them to engage affected communities and ascertain their urgent needs. Review of the ‘KPHA Baseline Report’, assessment conducted by ACF FSL team in KP in November 2011 reveals that 58% of the respondent households were farming households, dependant on agriculture for their livelihoods. The report also suggested that out of the farming households 33.8% of the households didn’t have the resources to cultivate their lands for the 2011 Rabi crops. The components such as support to small farmers with agriculture inputs, assistance through cash labour, livestock support and water and sanitation needs included in the design thus indicates realistic planning rooted in the community consultations at the initial stages. The evaluation team was able to see the needs based approach brought in use in several villages where the communities were provided either with WASH or FSL assistance only. In other target villages, the evaluation team successfully verified evidence of both WASH and FSL needs and ACF’s subsequent assistance delivered based on the integrated approach. Yet the evaluation team visited a few target villages where the communities appeared to have both the needs but ACF focussed only on one component which is also indicative of the fact that with meagre resources it becomes highly challenging to support everyone with everything. Baseline conducted by ACF at the initial stages of the program design allowed them in prioritizing the most vulnerable communities and their most pressing needs, which appears to have been the right approach to intervene in this case. Key Lesson Learnt:Participatory planning is extremely important right from inception to completion of the project.

The evaluation team during the course of evaluation learnt that a large number of organizations were present in the flood / conflict affected areas providing emergency and early recovery assistance to the affected populations in various sectors such as WASH, Shelter and On Farm / Off Farm Livelihoods but ACF was quite successful in identifying and plugging the gaps in this overall effort by the humanitarian community. The evaluation team also did not come across any major

15

duplication in the target areas sampled for this study as ACF at the designing phase effectively coordinated with PDMA and key actors through Clusters coordination mechanism at the Provincial level. The evaluation team met with the external stakeholders during the evaluation of Humanitarian Support to Conflict and Flood-affected populations in Pakistan project. These external stakeholders included Cluster Coordinators, Agriculture and Livestock Department Officials and District Disaster Management Authority officials. ACF coordinated within the Clusters for WASH and Food Security for information sharing, networking, and enhanced local level coordination, with the line departments as well as with other agencies working in the target areas. The WASH Cluster is led by UNICEF in KPK, which was initiated to enhance the coordination to avoid duplication and to fill in the gap. Similarly, WFP is the co-lead in Agriculture and Food Security working group, while FAO is co-lead in Agriculture. ACF participated in the Cluster meetings, wherein geographic coordination as well as technical coordination was organized at district level on cash for work fees, etc. under the Cluster system and in consultation with the district management and line departments. The coordinators of the lead agencies in the Clusters, interviewed by the evaluation team were positive about the coordination of ACF. Similarly the line departments met during the evaluation, were also positive about contribution of international agencies to the overall response and their intention to coordinate. Officials interviewed from the line departments such as Agriculture and Livestock offices at the district level in Charsadda were not fully aware of coordination over technical matters between ACF and some of the officials from these offices engaged in the project. One of the reasons could be lack of internal coordination between Governmental departments, district management and the PDMA and therefore lack of a structured way for the agencies, NGOs to approach these departments. Though it was brought to the notice of the concerned governmental authorities at the provincial level by the agencies in coordination and Cluster meetings, the agencies found it extremely challenging in some instances to get the No Objection Certificate (NOC) in time. ACF’s own instance, where the project activities could not start in the initial five months due to delay in the issuance of NOC is an example of the work of the agencies being seriously hampered due to the delay. The process is simplified now and the PDMA has developed simple format, whereas the INGO and/or the partners have to submit the proposal to work in a certain area to the Cluster. Once the Cluster vets it, it goes to the PDMA, which issues the NOC. The line departments in the project districts were pleased with project interventions, as they were engaged in the design and implementation of the activities. For example the Agriculture Director in Dir mentioned that his department was involved by the ACF in the Agriculture trainings, and also in the monitoring of these trainings. Similarly, the department was also consulted for the selection of the seeds to be distributed, as Frontier Seed Industry, a recognized seed producing body, was engaged to provide high yielding variety, which thus also provides opportunity to the farmers to replace their local seed variety with the high yielding variety. Though it had been initially planned to monitor the impact of the new variety of seeds, the post-harvest monitoring could not be conducted later on due to the delay in the harvest, caused by weather conditions in KP. Similarly, the department of Agriculture in Nowshera was also engaged in farmers’ training in sowing, post-harvest storage, seeds selection, seeds cleaning, weed removal and pesticides. There were instances where the coordination was extremely vital for the success of the project interventions and ACF’s field teams did manage quite well. ACF conducted livestock management trainings as well as distribution of goats, and toward this effect, the District Livestock Officers at the

16

project districts shared with the evaluation team that their department had been engaged in vaccination of goats. Local doctors and veterinary officers at UC level were also engaged by the line department to impart trainings. ACF teams consulted Livestock Departments for the selection of goat breed, for the organization of the goat fairs, and the check-up and vaccination of goats before handing them over to the beneficiaries. Officials from Agriculture and Livestock Departments were also invited to visit the project areas, together with ACF teams, to see the progress of the activities. The DDMO in Charsadda shared with the evaluation team about the impact of ACF’s training to the line departments on DRR, mentioning the need to focus on a contingency plan, information sharing at district level, and the distribution of kits, as part of disaster preparedness plan. It was mentioned by the officials of district line departments, that they were not consulted during the inception phase, when the implementation modalities were structured and finalized (quantity and quality of inputs transferred, baseline assessments, community mobilization, beneficiary selection, etc). On this account, the Agriculture department in Dir shared with the consulting team that with ACF, the department’s involvement was limited to a great extent to information sharing, and in the implementation and monitoring of the project activities, they had no role. Moreover, the role assigned to them did not have sufficient space. The Director Agriculture, giving the example of other International agencies, said that this space need to be allotted at the planning level, so the department should be fully engaged in the process, right from the beginning. This might be due to ACF not giving the opportunity to the line departments in the selection of beneficiaries, which was obviously not appreciated by these departments. Standardization ACF FSL technical experts developed Working Manual for Village Livelihood Recovery Committees (LRCs) that were formed within the KPHA project under study. The manual highlighted key parameters for community participation within the project execution. It also presented the LRC’s structure, its role and responsibilities, the process of beneficiary targeting for all the key components of the project and LRC’s role in setting up and execution of Accountability and Complaint mechanism. ACF teams were then trained over all necessary steps to adhere to the guidance provided within the manual and ensure standardization across project locations.

7.2 Beneficiary Targeting Beneficiary Selection Criteria Review of the project documents including the LRC Manual and PDMs suggests that beneficiary selection was done on the basis of set of selection criteria designed for each input in order to include maximum number of deserving affected persons in this assistance and avoid duplication. ACF’s senior mission personnel from Islamabad and Peshawar offices shared that the beneficiary selection criteria allowed ACF to deliver an appropriate intervention addressing pressing needs with relevant inputs. This mechanism was established by the ACF management responsible for implementing this intervention and was well supported by the technical leads present in Islamabad and in Peshawar office. The Technical Coordinators also made their efforts to align the selection criteria and input packages with the recommendations provided by the UN Clusters and Government Line Departments. The evaluation team during all the FGDs in the sample villages found these criteria appropriate to the local context and were clearly set keeping in view the vulnerable and affected populations within the three target Districts. This was possible due to the baseline done at the design stage.

17

The evaluation team learnt that except for a few odd cases, one, where a beneficiary was selected from District Charsadda for agriculture inputs and was not a farmer by profession, those who received the inputs were selected as per the criteria agreed. Small farmers received the agriculture support, vulnerable women and men were provided with livestock support, those engaged in daily wage labour were involved in CFW, vulnerable households were provided with latrines and communities who had no access to safe and sufficient water were assisted through water supply schemes, wells or hand pumps. Most importantly unconditional cash grants were provided to disabled and widows who needed them the most. The discussions with the beneficiaries and LRCs / WUCs revealed that the committees selected beneficiaries based on the number of each input made available by ACF. Since the LRCs / WUCs were in several cases lead by dominant individuals / groups hence the communities in some cases didn’t question the decisions and accepted whatever was being offered. Some complaints were raised regarding nepotism by the LRCs / WUCs but no concrete evidence could be provided. The evaluation team noted dissatisfaction among 100% non-beneficiaries or those who couldn’t get selected for heavy inputs such as latrines or who couldn’t get agriculture and livestock support but received just CFW or only hygiene kits. The evaluation team was able to establish the fact that the criteria were applied during the selection of beneficiaries and most of those who were not satisfied didn’t have convincing arguments for their dissatisfaction. Complaints raised with ACF in such cases resulted into verifications by ACF teams and were found not justified as the complainants didn’t fulfil the beneficiary selection criteria.

7.3 Project Implementation Provision of wheat seeds and fertilizers was very effective as it was provided in a timely fashion thus allowing the beneficiaries to sow wheat on time, a crucial step in the revival of conflict and flood-affected communities in the target areas. The household survey conducted during the evaluation also confirms this as 92% of the sample beneficiaries of agriculture support confirmed that the inputs were provided timely. This was a daunting task given the huge delay in the project start up, owing to the delayed approval of the project from ECHO and painfully slow process of seeking NOC from the PDMA. It will be fair to say that ACF teams both in the country office and in the field have done a tremendous job in delivering the project inputs, particularly those that were time / season sensitive, well in time. Feedback from the beneficiaries strongly suggests that wheat seeds were effective and of very good quality. 84% of the respondent beneficiaries confirmed during the household survey, conducted as part of the evaluation that they were satisfied with the crop yields after utilizing the agriculture inputs provided under KPHA. Those who were unable to prepare their lands on time made sure that they stored the seed provided by ACF to be sown next year as they knew that the seed provided was of very good quality and if sown in the right conditions could give high yields. Fairs organized to facilitate provision of goats to the affected communities was also very effective modality which allowed the beneficiaries to choose appropriate animals as per their need. The feedback from the beneficiaries clearly indicates that the breed was as per their needs and the local context and they were happy with the assistance.

18

‘Report on ACF Goat Fairs Program’ suggests that 750 households received goats, allowing them the opportunity to rebuild their assets. The trainings related to agriculture and livestock support were also directly beneficial and clearly assisted the beneficiaries in improving their production. The water, sanitation and hygiene (WASH) interventions provided to affected population were also relevant, appropriate and on time. Water provision through dug wells or repaired schemes was demand-driven and highly useful. Availability and access to clean and safe water were a serious need in the flood affected areas. The shallow water wells of approximately 30 to 40 feet depth were contaminated. Therefore, development of new water supply schemes or repair of the old dug well or tube well was very well received by the community. Most of the respondents were satisfied with the taste of the water and they also believed that there would be fewer incidences of related diseases after the provision of safe drinking water. The hygiene kits contained useful items for water storage and transportation and were well received by the beneficiaries. Both women and men also appreciated and acknowledged the benefits of the hygiene promotion and found it very informative and useful. The mode of communication for creating awareness and motivating the target community was appropriate and as per the understanding of the local population. During field work it was found that local community liked the design and structure of the latrines, materials used in the construction of latrines and items in the hygiene kits, newly constructed and repaired hand pumps for water supply and the delivery of hygiene promotion sessions. It was, however, also observed that some community preferred one WASH intervention (e.g. water) but they got other less prioritized items such as hygiene kits or latrine as the priority and needs of population vary in different locations in the affected areas. For example villagers ShahiKulaliin UC Tarnabin District Charsadda insisted on water supply schemes but they got hygiene training, similarly in village Namsair in UC Khal in Lower Dir some villagers really needs water for drinking but they got latrines. The design of the latrines (i.e. superstructure, commode) was as per the local norms but less flexibility was shown to accommodate the needs of the disabled, pregnant women and elders. For example the father of a polio infected boys wanted change in the interior design of the latrines but was not given sitting commode. The household data suggests that 98% sample beneficiaries of household latrines confirmed that there were 20 or fewer people in a household using ACF provided latrines. It was found though in some cases latrines were provided to the community that didn’t have sufficient water to use the latrines properly and were fetching water home only for drinking purpose and therefore preferred open defecation. An old lady in the union council Hall in the Lower Dir mentioned that it took around 30 minutes to fetch one bucket of water from the nearby spring and we cannot afford to use it for flush latrine provided by ACF. We cannot afford to use latrines unless it is raining or during night time. Defecation of children around the house was common although ACF latrine was built there. A septic tank of 4x6 feet was constructed and sealed from inner sides and bottom where the black water from the latrine was to be collected. Each septic tank had an outlet which was supposed to drain either in the open street or into the nearby irrigation channels. Since grey water from washrooms and kitchen and black water from septic tanks were not taken care of by the project, the untreated wastewater inside the houses as well as in the streets are continuous source of spreading pathogens. Therefore, instead of breaking the pathogen cycle and improving health of the people, the opposite is feared to be true risking the health of larger numbers living downstream. Risk assessments and trainings on DRR were highly relevant to the context but the time constraints mounted due to start-up delays couldn’t allow the ACF teams and the communities in carrying out effective risk reduction planning.

19

Analysis of information presented above suggests that the overall assistance has been highly effective in providing a kick start to the recovery process of disaster affected populations in the target areas. Approaches and modalities adopted have been reasonably successful in ensuring that the communities receive inputs that have triggered early recovery of their livelihood activities. Farmers were able to cultivate their crops and initiate recovery of losses faced due to the floods and conflict. Provision of wheat seed and fertilizer along with cash for land preparation was highly effective for the farmers who were able to cultivate their fields. CFW for landless labourers and unconditional cash grant support to most vulnerable was also the most effective way of ensuring the inclusion of landless and most vulnerable and marginalized sections of the community who deserved this assistance the most giving them the opportunity to survive and in some cases sustain themselves with dignity. Provision of inputs to women allowed them to own and rebuild assets and be acknowledged within the society. Provision of safe and sufficient water and household latrines along with hygiene promotion most certainly is an effective approach towards reducing morbidity among the disaster affected communities as evident from the established scientific literature. The assistance has ensured that the targeted households avoid open defecation, particularly women and elderly. Women are no more worried to wait for the dark as the latrines facilities are available inside their homes. The trend of construction of latrines on self-help basis was also observed in some villages after this demonstration of ACF interventions. The approaches adopted by ACF to deliver this intervention were also highly effective, such as voucher system, goat fairs, joint capacity building and planning initiatives. If ACF had complete twelve months to implement this project, the impact upon the lives of the disaster affected people would have been manifolds. Some concerns still need to be addressed such as instances of impartiality noticed within the LRCs; more effective planning for cash transfers; regular and process oriented monitoring to ensure quality and accountability which play a crucial role in achieving the desired impact. The evaluation team is of the view that such processes can only be developed and ensured once the communities have been effectively engaged through the process of mobilization which needs fair bit of time. Short term relief and recovery projects like the one under study do not allow enough time for setting up such mechanisms. It is commendable that ACF started this project with an at least six months’ time handicap and yet managed to reach vulnerable communities. Nevertheless strong and committed efforts are required to build the capacity and improve quality. Attention to Disaster Risk Reduction provides a crucial plug-in in this scenario, which can allow ACF to engage with such communities in peace times. The evaluation team was able to verify and validate the provision of inputs to flood and conflict-affected population as under: FSL Cash for Work 22501 households were engaged in Cash for Work schemes aimed at repairing small scale community infrastructure and construction of household and community WASH infrastructure. Each beneficiary was provided with labour opportunity with a daily rate of PKR 350. Irrigation channels and WASH infrastructure highly benefitted the communities. The household survey carried out during the evaluation study indicates that 60% of the sample CFW beneficiaries confirmed that they were engaged in CFW for 24 or more days. The rest of the 40% beneficiaries confirmed their 1 Post Distribution Monitoring (PDM) Report, Cash for Work and Unconditional Grant Programs of KPHA, ACF FSL

KPHA, April 2012

20

engagement in CFW for less than 24 days and hence they earned less than PKR 7,200 which was calculated based on the WFP’s Food Basket valuated at PKR 5,000 per household to cover access to basic food basket. FGDs confirmed that there was a variation in the CFW benefit for different households in a village, especially in villages where WASH component was also implemented as it provided greater number of man days under CFW schemes. Cash Transfer Standard Operation Procedures for the project were also developed by ACF Technical Experts to ensure transparency, accountability and uniformity. For the purpose the program team had extended internal discussions with the operations teams to streamline the cash transfer mechanisms and ensure standardization across all target locations. Discussions and agreements with banks were also carried out to facilitate the entire process.

Beneficiary selection criteria and guidelines for input delivery, particularly Cash Transfer were put in place and efforts were made to adhere to these but the implementation team faced challenges in terms of maintaining impartiality within the LRCs / WUCs during beneficiary selection and implementing activities uniformly. CFW was also used to build WASH infrastructure hence some beneficiaries engaged for CFW in WASH received more man days than the others but the minimum man days to match the value of food

basket for each household was compromised for the above mentioned 40% of CFW beneficiaries. Below table provides details of beneficiary response with regards to the number of days they were engaged in CFW. % of Respondents

1.5% 1.5% 1.5% 1.5% 20% 5.5% 4% 1.5% 1.5% 1.5% 39% 21%

CFW Days 7 8 9 10 12 15 16 17 18 20 24 30 Discussions with beneficiaries during the FGDs clearly indicated that the decision making within the LRCs / WUCs was not participatory which resulted in some beneficiaries receiving more man days than the others. Cash Transfer Mechanism Problems related to transferring cash to beneficiaries remained a challenge that resulted in ACF teams adopting several approaches rather than ensuring standardization. At the planning stage ACF explored several options for smooth, transparent and dignified cash transfer mechanisms and initiated transfers through a local bank but eventually large sums were transferred to beneficiaries directly and indirectly as liquid cash. This indicates ACF team’s flexible approach to adapt to the situation and facilitate the beneficiaries to the maximum in transferring cash in a timely fashion. The household survey conducted in the villages selected for the evaluation study reveals that 54% of the beneficiaries of Cash Transfer component received cheques which carried names of the beneficiaries and their NIC numbers. The banks would hand over the cash to the beneficiaries upon verifying the beneficiary name from the NIC. This required smooth and effective planning in order to allow a convenient system for the beneficiaries to access cash. Since the rush to meet the targets couldn’t allow ACF teams to spend more time in planning considering limited capacity of bank branches in the three target Districts to deal with such scale of visitors to the banks, beneficiaries had difficulties in en-cashing their cheques and had to wait in long queues outside the banks. They had to pay repeated visits to the banks, in cases of expired National Identity

21

Cards(NICs) and due to confusions related to next of kin in some cases, where women and disabled couldn’t go to banks and authorized their next of kin to receive on their behalf. Visiting a bank is a significant exposure for populations living in remote rural areas and possession of a National Identity Card is a necessary national requirement allowing an individual to take part in electoral process. Cash transfers through banks would have definitely contributed in providing such an exposure to the beneficiaries and encouraged them to get their CNICs, but in the humanitarian perspective they have a right to access assistance at their door step, with dignity, which should not be compromised. The ACF FSL team in the field shared that as a strategy to speed up the cash transfer and minimize difficulties to beneficiaries in accessing cash from banks, ACF teams identified group leaders from the LRCs and issued cheques in their names who withdrew cash for the whole group working on a specific CFW scheme in a village. These leaders then distributed cash to the CFW beneficiaries. ACF field teams played a crucial role in distributing cash and worked diligently to reach the beneficiaries but were overwhelmed with targets towards the end during the No Cost Extension period, particularly in Lower Dir District and didn’t have the capacity to closely monitor every such cash distribution. Hence they had to rely, more than anticipated, over the LRC members in monitoring and distribution of a fairly large sum of cash to beneficiaries. Aid agencies have tried several cash transfer mechanisms within their livelihoods recovery programming across Pakistan in the past, using Pakistan Postal Services, Transfers through Mobile Phones, Banks, Vouchers, etc. Disaster affected populations in most of the cases have difficulty in accessing cash from banks, hence mobile banking though a bit costly, but perhaps would be the safest, transparent and most dignified way of transferring cash to the beneficiaries. A crucial lesson that ACF has already learnt in the process and is now utilising mobile banking system in some of its cash transfer programs in Sindh Province. Keeping in view the enormous business being provided to the banks by aid agencies in Pakistan under CTP, it is important to remind them of their social responsibility and negotiate with them for providing cash to the disaster-affected communities even in the remote areas of KP in a swift and safe manner. Recommendation: ACF is recommended to negotiate with banks in future and improve Cash Transfer mechanisms.

Unconditional Cash Grants Unconditional cash grants were provided to 2502most vulnerable persons in the affected villages including disabled and widows. Size of the unconditional cash grant was calculated again on the basis of WFP’s food basket for a household. FGDs in the sample villages revealed that the community highly appreciated this intervention. Cash for work and unconditional cash grants turned out to be very successful activities which provided labourers with labour at their door step and a decent

daily wage rate and provided source of income to the most vulnerable populations. In return critical community infrastructure was repaired in most of the cases which will have long term impact over

2 Ibid.

22

the communities if properly maintained. The impact over all the beneficiary households might have been more or less similar if uniformity was ensured. Income generated from this activity allowed them to buy food and meet other necessities of life especially health care. Household Dietary Diversity Survey conducted by the evaluation team suggests that most of the money earned by the beneficiaries was utilised to meet their immediate household needs, including food, utilities and health care. Only 8% of the sampled CFW/UCG beneficiary respondents suggested that they utilized cash for repayment of loan. ‘PDM Report on Cash For Work and Unconditional Cash Grants Program of KPHA’ developed by FSL KPHA Team in April 2012 also suggested similar trends as highlighted by the evaluation exercise, with the highest amount of cash each household utilizing on basic food needs. The overall FSL assistance particularly CFW and UCG had some contribution in improving household access to food which is evident from the Household Dietary Diversity Table. The impact is little due to the exceptional inflation that rocketed in the early quarter of the year. Market surveillance in such scenarios becomes crucial and can help in adjusting the modality to cover the food basket.

Agriculture & Horticulture Support Farmers who needed assistance in cultivating their crops in the flood / conflict-affected area were provided with wheat seed and fertilizers. It was a crucial assistance which allowed them to grow their crops, their economic mainstay. Each beneficiary farmer was assisted with the provision of crop inputs for one acre of land through vouchers. The project made provisions to provide each farmer with 50KG wheat seed, one bag of DAP (50KG), two bags of Urea (50KG each) and a cash sum of PKR 6,000 to prepare their lands. The household survey conducted with a sample of beneficiaries in target villages indicates that 47% of the sample beneficiaries of agriculture support purchased the recommended package from the specified local markets using the vouchers. Rest of the 53% of the sample beneficiaries purchased crop inputs as per their own needs. 78% of those who didn’t purchase the recommended package preferred not to buy DAP and 66% chose to buy only one bag of Urea. Discussions during FGDs revealed that they preferred farm yard manure over synthetic fertilizers as they could benefit more than one crop production. 77% of the sample beneficiaries of agriculture support, during the household survey indicated that they were satisfied with the quantity of crop inputs. Vegetable seeds were provided to women in each target village. The inputs were aimed at allowing women to grow their own vegetables and be able to meet household food requirements. Livestock Support Through this assistance men and women of the flood and conflict-affected areas were provided with goats to rebuild their assets and contribute to their diet thereby improving household food security. Each goat-beneficiary woman was provided with a pregnant goat, supplement feed and fodder seed through organizing goat fairs in the local areas. The household survey carried out in the target villages selected for the purpose of this evaluation confirmed that 80% of the sample goat beneficiaries were women. ‘Report on ACF Goat Fairs Program in Lower Dir, Nowshera and Charsadda Districts’ developed in April 2012 confirmed that 855 goats were purchased by 750 targeted households through goat fairs using cash vouchers. The report also indicated that 81% of the total beneficiaries were women, in accordance with the findings of the evaluation exercise. The household survey conducted during the evaluation also indicated that 98% of the sample beneficiaries were satisfied with the breed. The data also suggests that 93% of the sample goat beneficiaries confirmed receiving grants for supplement feed and fodder seed. Analysis of the household data reveals that 82% of the sample beneficiaries of livestock support were satisfied with the assistance provided and the goat population was steadily growing in the area. Capacity Building and Training The household data confirms that 97% of the sample beneficiaries of agriculture and livestock support confirmed receiving trainings on effective crop management and livestock management respectively. They were satisfied and believed that the trainings effectively helped them in improving their practices and knowledge. The overall impact of this assistance is positive as it has built a good image of the humanitarian community in the target areas. Agriculture support had a huge immediate impact on the beneficiary households who were able to revive their agriculture through the assistance provided under this project. They were not forced to seek loans or sell remaining assets and were able to cultivate their lands once again. 86% of the sample beneficiaries indicated that the training on crop management was very useful to them as they learnt tremendously about pests and fungal attacks on crops and various

24

FerozaNaseer, a 70-year-old woman, living with two of her daughters and a disabled son in ShahiKulalay Village in Charsada was affected by floods in 2010 as her house was damaged. She had no source of income. ACF after conducting an initial survey included her as a beneficiary for livestock package. She purchased a goat in the fair which delivered a kid and is also giving some milk. She said “By the grace of ALLAH my source of income has increased and now I have two goats, I am thankful to Allah and ACF for providing assistance and support in the hour of need.

remedies to get rid of them. They also learnt about safe and successful sowing and irrigation practices which had a direct impact on their crops. Livestock also had a significant immediate impact due to relevant and appropriate targeting. There were several instances in all the three districts where women shared that their household assets and had a clear improvement after receiving goats. Most significant immediate impact was of the unconditional cash grants provided to most vulnerable households within each village, represented by physically challenged persons, widows, elderly etc. They were able to lead their lives with dignity as the assistance allowed them to meet some of their urgent basic needs. It also allowed the community to recognize their challenges and their existence and importance within the villages.

DRR PADR Assessments and Village DRR Plans The evaluation team validated the activities for carrying out risk assessment and organizing DRR sessions for communities to have basic understanding on disasters and disaster management at the community level. These were one off sessions which were delivered to the beneficiaries in the target areas, often conducted as joint sessions for several target villages. The evaluation team learnt that the beneficiaries had retained some basic knowledge from these sessions but required further sessions to establish thorough understanding of various risk reduction measures that could be set up at the local level. Discussions with the beneficiaries also indicated that the village DRR plans were discussed in such sessions but there were no clear evidences of practical demonstration of the same in any of the villages selected for the evaluation study. Workshops with Local Authorities on DRR The local authorities were aware of the trainings conducted by ACF during the course of the project implementation. DDMO in Charsadda shared that DRR trainings conducted by ACF was a very critical activity for the line departments that participated in these trainings. This activity had triggered internal discussions within the line departments at the district level and the district authorities intended to initiate contingency planning in the wake of approaching monsoon season. DRR is an essential component of the KPHA intervention which targeted the disaster affected communities. Including basic risk analysis and risk reduction measures allowed ACF to understand the present and future vulnerabilities of these communities. Though at a very basic level and at a preliminary stage, but DRR inclusion in KPHA has presented an opportunity to the communities to some extent to prepare themselves for the future disasters thereby reducing their vulnerability. Investment in DRR will also significantly improve ACF’s preparedness for an

25

effective and timely response to future disasters and will reduce disaster risks to its planned and ongoing investment in disaster prone communities. Recommendation: Integrate DRR more actively in ACF emergency Programs such as KPHA WASH Access to improved sanitation, quality water and hygienic practices were poor even before the 2010 flood in the rural areas of Nowshera, Charsadda and Lower Dir like in many other parts of rural KPK. The flood, however, damaged the already dilapidated structure and services of water supply, latrines and drainage systems. The affected people, therefore, could not maintain minimum SPHERE standards of hygiene, drinking water and sanitation due to lack of resources and services. SPHERE minimum standards may not be relevant here since the affected population are living in their homes. Most of them have some access to water quantity which may not have safe quality. Similarly, in the selected villages the concept of community latrine is not applicable and people do not use neighbours latrines so the distance to latrines and number of people per latrines are not relevant as proposed by SPHERE standards. Provision of Safe and Sufficient Water 100% sample of beneficiaries of water provision reached through household survey during the evaluation indicated their satisfaction over the quality and quantity of water made available through different water sources. In the rural areas of District Nowshera, Charsadda and Lower Dir, shallow wells, dug wells and spring (particularly in Lower Dir) were the main source of drinking water. All these sources of water were either totally or partially damaged and/or contaminated and the affected population initially survived on using water from remote sources and later on they were forced to drink the available but contaminated water. Since the flood water went directly into the open wells, therefore, the shallow water in the areas was entirely contaminated and some of the wells filled with mud were not in operation even at the time of evaluation. The beneficiaries expressed their gratitude towards ACF as none of the organizations providing assistance earlier in the target areas focussed on such crucial needs on a longer term basis. Provision of Household Latrines to EVI / PwD 60% sample of beneficiaries who received household latrines reached through household survey during the evaluation confirmed that they were provided with hand washing facility along with the latrine while 40% indicated that they didn’t. The household survey also reveals that 40% of the sample beneficiaries did not receive shower facility. FGDs with beneficiaries and WUCs confirmed that the beneficiaries did not opt for shower facility in most of the cases. In general 94% of the sample beneficiaries interviewed during the household survey confirmed that they were satisfied with the latrines.Showers could not be provided at some places due to following reasons: for shower option inside the latrine, water tank on the top floor of latrine was needed but many poor household do not have water pump and/or electricity or even water well inside their homes so was not possible to pump water to the tank, most of the household prefer water drum or bucket which they traditionally used for shower and flushing. ACF also realized that installing water tank for shower is not cost effective and sustainable since continuous pumping to tanks is not possible for the poor families. Therefore, shower was removed from the original design and water buckets were given to the beneficiaries’ households.

26

Said Wali Khan, 50, living in village Gulshan Abad of UC Agra, District Charsadda, with his twelve family members, owns four Jarebland that used to produce eight to ten mounds of wheat per Jareb. His land and the water channel were badly affected by 2010 floods. Rehabilitation of the irrigation channel supported by ACF reduced the water wastage and increased its flow greatly allowing sufficient water for his crops in a short time. This has resulted in the increase in yield to twelve mounds per Jareb. He said “Now I am earning well from my land and will be able to afford my children’s education”.

Hygiene Promotion & Provision of Hygiene Kits Analysis of the data collected through the household survey reveals that 98% of the sample beneficiaries of hygiene promotion confirmed that they attended ACF led sessions on health risks and safe hygiene and sanitation practices. The data also suggests that ACF has been quite successful in transferring essential hygiene promotion knowledge to the flood and conflict affected people of the target areas as 96% beneficiaries were aware of the essential / critical times when to WASH hands, 94% beneficiaries were aware of the use of latrines as key element of safe sanitation and 93% beneficiaries were aware of critical ways to store water. The data analysis suggests that 95% of the sample beneficiaries of the hygiene promotion sessions were satisfied with the benefit they received from hygiene promotion sessions. Household interviews of the beneficiaries of hygiene kits suggest that 99% sample beneficiaries received ACF provided containers to store water safely. As per the data 98% beneficiaries of the hygiene kits confirmed that ACF provided them with container to transport water safely and 100% of the sample beneficiaries confirmed that they were satisfied with hygiene kits. Environmental Sanitation Discussions with the beneficiaries also suggest that the project assisted them in solid waste management in their villages through identifying and setting up places for dumping and managing solid waste but the communities couldn’t make effective use of it and appeared careless in ensuring cleanliness of their environment. This was mainly due WUC’s passive role in leading communal tasks after project completion. Rehabilitation of Small Scale Irrigation Canals for Agriculture During FGDs with the beneficiaries in the target villages selected for the evaluation study the evaluation team learnt that the communities were very happy with the rehabilitation of irrigation channels for agriculture. It had a direct impact over their crop yield as their fields were receiving better water quantities in shorter periods of time.

27

7.4 Support Systems The evaluation team learnt during the course of evaluation that ACF in Pakistan is building on the lessons learnt in a very challenging environment during the course of implementing this project and has already started streamlining processes and improving approaches to achieve the results and the immediate impact in its current programs. ACF has dedicated technical experts on board both at the Country Office in Islamabad and in the field bases in KP to provide technical assistance and backstopping to field staff in emergency relief and recovery operations. During the execution of this project under evaluation, these teams also acted as a bridge between national and district level sector Clusters and coordination mechanisms, between local authorities and ACF field operations in seeking technical advice on specifications and modalities from multiple sources within key sectors; and based on their own technical expertise seeking consensus and ensuring standardization, addressing local challenges and enhancing quality across ACF program. These experts within ACF ensured that the inputs provided by ACF were as per the agreed standards determined by the Clusters and the Line Departments, validated by the evaluation team in the sampled villages. These experts were able to provide guidance through the guidelines that they developed in consultation with local teams. There were some hindrances faced by ACF in ensuring regular hands on support and on ground monitoring of the project delivery by these experts as most of them were expatriates and couldn’t access field areas frequently, due to delays in NOCs, or spend ample time in the field due to insecure volatile situation in KP. Apart from this, hiring qualified expatriate Field Coordinator for KP, a crucial program role that was expected to lead the implementation of this project in KP ensuring integration between FSL and WASH, also remained a serious challenge. These challenges were to some extent managed by keeping a robust team in the field lead by Deputy Program Managers for WASH and FSL managing their teams supported by Head of Base leading program support functions on ground, security permitting field visits by the Program Managers and Technical Coordinators and remote assistance from the Country Office in Islamabad. Keeping in view the security risks and challenges agencies face in engaging expatriates in Pakistan, particularly in the areas such as KP, and given the availability of qualified management and sector specific technical human resource in the country, ACF, after its own experiences in KPHA, is already on the move to nationalize key program positions. This would further increase ACF’s experts’ presence on ground, definitely reducing its cost per beneficiary and eventually leading to clear improvements in program quality. The evaluation team didn’t have adequate data to do an exact calculation of the cost per beneficiary under KPHA. Support systems have played a significant role in providing maximum assistance to the program teams in setting up protocols to ensure transparency and accountability. These systems must trickle down to the field level activities, particularly the cash transfer programming wherein the support systems should provide additional facilitation and support to program teams in reaching disaster affected communities in the far flung areas. Support systems could carry out negotiations with service providers such as banks in providing quality and timely services as per plans. They could also join program teams in monitoring cash withdrawals at banks and cash disbursements in the field and assist field teams in completing necessary documentation, especially in cases when program teams are overwhelmed with targets and short of human resources.

28

The overall facilitation provided by the Support Systems was generally efficient in terms of timely procurements and making payments to vendors and suppliers. The program team was able to deliver project inputs with varying modalities based on this assistance. There is a need to encourage regular discussions among the program and program support teams to identify mutually agreeable solutions in order to enhance efficiency.

7.5 Project Sustainability KPHA project has been successful in providing a crucial start to the recovery of the flood and conflict affected communities of the target areas, which is linked to sustainability in many ways. Construction / rehabilitation of WASH related infrastructure and irrigation channels is clearly an essential intervention covering immediate needs and addressing longer term vulnerabilities. Provision of latrines has already had a demonstrated effect within the communities and the evaluation team learnt during the evaluation that many households in the target areas had stopped open defecation and preferred to use the household latrines while many others initiated the construction of latrines within their compounds on self-help basis. Hygiene promotion contributed in initiating changes in behaviours and practices related to hygiene and sanitation which is expected to reduce morbidity. Rehabilitation of drinking water sources and supply was also an essential support to the disaster affected communities which is expected to continue for long term and provide safe and sufficient water to these communities for years to come. Rehabilitation of irrigation channels for agriculture purposes has improved crop yields and will continue to support farmers in taking high yields from their lands in the years to come, thereby improving their economic conditions and reducing their vulnerabilities. The evidence showed that communities will continue benefiting from these new and rehabilitated infrastructures in the target villages. Capacity building on agriculture and livestock support and DRM reflect sustainable efforts that will continue to transfer knowledge and skills among communities which is expected to contribute to change in the longer term. The project has also paved the way for a long term engagement of the affected communities with local authorities who are eventually responsible for supporting them. Trainings conducted by ACF had triggered internal discussions within the line departments at the district level and the district authorities intended to initiate contingency planning in the wake of approaching monsoon season. Further efforts are required to establish thorough understanding of the flood and conflict-affected communities in various risk reduction measures that could be set up at the local level. LRCs / WUCs may continue, to a limited extent in taking ownership of the collective problems and facilitating communities in solving them in an organized fashion. The degree of their active existence ,as a direct result of KPHA, in the target areas may vary but their linkages with the stakeholders are expected to grow and it is anticipated that they will prove vital in case disasters strike these areas in future. They have the potential to re-organize quickly and lead future relief efforts in these areas. Further efforts are most certainly required to transfer basic skills and knowledge to them and reform them, to some extent, to reflect equal and impartial representation of the communities of their origin.

29

7.6 FSL Surveillance ACF planned regular surveillance of local market prices for essential food items during the course of the project implementation. For the purpose a template was designed and project staff regularly collected prices from the local markets. Information from the market surveillance was presented and analysed on monthly basis in regular progress updates generated by the Program Manager. Review of the sample reports carrying analysis of food prices in target areas and discussion with the FSL Coordinator in Islamabad was very useful in understanding ACF’s FSL Surveillance System. Generally the objective of such a surveillance system is to continuously assess the food security of the population and be able to forecast and raise alert in case of negative trends. In order to have an effective surveillance, it is vital to identify indicators that are appropriate to the local context, in KPHA Project’s case food items and their prices that could impact on household income and expenditure in the target districts. In the case of KPHA, the surveillance system was a simple follow up of prices. The FSL team in the field couldn’t focus more on analysis of the price fluctuation impact on a household. Also it is not very clear how prices were selected to follow up. ACF realizes that the field team lacked skills and capacity to design and implement surveillance activity and was overwhelmed with the execution of other components of the project and couldn’t carry on with setting up a proper surveillance system that could help in decision making. It can be concluded that ACF Surveillance system remained a brief activity during the project life and couldn’t build on the information further to establish a system that required much greater resources than those envisaged. Recommendation: ACF must invest more time and resources in building its capacity in developing and running FSL Surveillance system

7.7 Project Monitoring The evaluation team learnt that ACF had put in place systems to monitor the project deliverables under this intervention. Key features of this system included regular verification of beneficiary selection process, monitoring the delivery / distribution of inputs through various modalities including cheque disbursements at the banks to the beneficiaries of CFW and UCG; voucher redemption in the local markets to access appropriate agriculture and horticulture inputs; voucher redemption to purchase healthy and quality goats during goat fairs; monitoring of CFW and WASH infrastructure schemes implementation on ground. For the purpose initial benchmarks were identified based on the baselines carried out by ACF teams. Feedback from this level of project monitoring was documented and shared through monthly progress reports. Upon delivery of various inputs ACF teams also conducted post distribution monitoring surveys to track the outputs and outcomes of the assistance provided. ACF technical leads at the initial stages of project implementation devised methodology for PDMs based on the indicators from the log frame. The methodology advised the field teams over complete mechanism of selecting sample size, data collection and analysis, frequency of data collection and roles and responsibilities. PDMs allowed them to understand the consequences, identify further gaps and improve planning and counter challenges. Findings of the PDMs were documented in the PDM reports, shared with the ACF management team in Islamabad and shared with HQ and ECHO. PDMs within FSL were conducted to gauge communities’ satisfaction for engagement in cash

30

for work and understand the trends in cash utilization; verify targeting and assess trends in cash utilization in the case of UCG; gauge satisfaction of beneficiary with goat fairs modality and the quality of animals provided under the livestock component and assess the benefits received by the communities; and verify the timely purchase and utilization of crop inputs. PDMs conducted by the WASH teams focussed on gauging beneficiary satisfaction over the WASH inputs and beneficiary engagement in hygiene promotion and communal sanitation activities. Apart from this ACF team also planned and carried out joint monitoring visits with officials from agriculture and livestock departments to target areas and monitor key project activities such as goat fairs to ensure their inputs into the process in order to improve the effectiveness of the project deliverables. At the local / on ground implementation level LRCs and WUCs were also engaged and tasked to monitoring project delivery in the target villages. After the review of Baseline report, PDM reports, activity / technical and monthly progress reports and detailed discussions with the ACF teams in Lower Dir, Peshawar and Islamabad, the evaluation team can conclusively state that serious efforts were made by ACF team to monitor the delivery of project inputs and track project outputs. These efforts indicate high quality of technical and sector specific expertise made available by ACF to manage and monitor project deliverables. Hence significant technical sector-specific project data was collected against the indicators and preliminary analysis done to assess project delivery however the frequency remained on the lower side and ACF teams couldn’t monitor the project activities at a regular frequency during the life of the KPHA project. The evaluation team believes that there are several reasons that lead to this situation. Monitoring and Evaluation as a key program function within ACF program in Pakistan still needs a lot of improvements. It needs to be strengthened at the level of programming rather than taking it up only as a project based requirement. ACF needs to set up a monitoring system across programs both at the National and at the Provincial level. Regular program reviews must continue at both the levels to allow program teams, both Expatriates and National staff to discuss project specific delivery, challenges, experiences and explore ways in which all the projects can contribute and converge to achieve ACF program related strategic objectives. This regular feedback system should also allow senior and mid-level staff from program and program support functions to sit together and highlight areas of improvement and identify best practices and share them for learning and standardization across ACF programs. Apart from this ACF also needs to build the capacity of its M&E team to develop a system that allows regular consolidation of its program related data to enhance integration and provide an external view to identify gaps and suggest improvements to enhance program quality. Such a system must facilitate regular internal and external mid-term and end of project reviews and evaluations and ensure dissemination of best practices and key lessons learnt. Within such a system Lessons learnt workshops should be organized regularly at the end of every project to register feedback from project teams and Learning papers should be developed for sharing with the rest of the ACF teams in the country and in the region. Another crucial aspect that can enhance program quality is process monitoring. The initial delay in the kick-off of KPHA pressed the teams to deliver the project in a very short timeframe hence the project team was overwhelmed by the targets and less focus was given to monitoring the processes that lead to project delivery. This eventually resulted in teams pushing for achieving targets and the monitoring was limited to the level of outputs. Developing specific protocols / standard operating procedures for each key component and monitoring the adherence / compliance to every key step within these protocols can greatly improve the project delivery and

31

ensuring standardization across project areas based on these protocols can direct the outputs towards achieving a uniform intended impact. Key Lesson Learnt: Regular output tracking coupled with process monitoring ensures reaching desired impact

Recommendation: ACF needs to set up robust monitoring systems that ensure successful delivery of results as well as encourage adherence to processes and promote standardization

7.8 Livelihoods Recovery Committees (LRC) The discussions with the beneficiaries, LRCs / WUCs and project staff within ACF suggest that at the initiation of the project tACF teams arranged broad based community meetings in each target village and invited the communities to form village committees in order to roll out this intervention. Efforts were made to form male and female committees responsible for monitoring and execution of project implementation in every village. During visits to the sampled villages and discussions with beneficiaries and LRCs / WUCs, the evaluation team learnt that all the assistance was provided through the LRCs / WUCs who took lead in identification and selection of beneficiaries and monitoring project activities in each village. It was observed that the LRCs / WUCs were in several cases represented by individuals or dominant groups within the villages who belonged to these villages and were responsible mainly for decisions regarding selection of beneficiaries. As per the local norm, all decision making regarding collective village matters was done by these individuals or dominant groups in these villages. Other committee members and office bearers were not that active and mostly remained at the receiving end as the entire village had benefited from at least one of the project inputs, particularly in the villages where ACF had intervened with an integrated approach. Female committees were not aware of their role and details of project inputs and were not even active. They were engaged only in distribution of goods to female beneficiaries. They had almost negligible role in decision making. Nevertheless, some of the LRCs’ male members, in particular the key individuals played a significant role in executing project implementation on ground. All project activities, particularly rehabilitation of infrastructure and distribution of inputs directly or through vouchers and fairs was carried out on time which allowed ACF to achieve the outputs. The quality of such interventions can definitely improve if more time and resources are invested in ensuring a process oriented approach in the formation of representative and impartial community structures. Short term relief and recovery projects like the one under study do not allow enough time for setting up such mechanisms, hence there is a need to devise approaches and steps to fast track community sensitization and mobilization. Also ACF teams need to be thoroughly trained in community mobilization and innovative communication skills to be able to sensitize communities and organize them rapidly within short term programs and lead the process ensuring community structures remain accountable and impartial as much as possible. Key Lesson Learnt: More investment in community mobilization, greater is the impartiality within community structures resulting in greater impact

Recommendation: ACF is strongly recommended to build the capacity of its teams in community organization and mobilization

32

7.9 Accountability and Complaint Mechanism ACF team was committed to the accountability and complaint mechanisms and the committee office bearers and beneficiaries were aware of the complaint system. The evaluation team learnt from the beneficiaries that ACF team had displayed contact numbers for ACF officials during distributions and meetings in each village and the beneficiaries were briefed about how the complaint mechanism worked. ACF teams received complaints mostly with regards to dissatisfaction from beneficiaries who couldn’t get selected for one input but received another and blamed LRCs of nepotism. Complaints raised with ACF in such cases resulted into verifications lead by ACF teams and were found not justified as the complainants didn’t fulfil the beneficiary selection criteria. The district management and the line departments also handled complaints, as people approached them with complaints. This means an accountability mechanism also existed within the district authorities. The evaluation team found through interviews with the officials at the district level, that the complaints were registered with the departments, and sometime Jirgas of local elders and district management were called in to resolve the issues. Mechanism for complaints was structured in Charsadda in a manner, whereas the complaints come to the DCO/ACO, in writing, which were then marked to DDMO. In a complaint shared with the DCO Charsadda, a local Khan raised an issue that inputs distributed by ACF in his village had not reached the deserving persons. ACF’s field team attended the Jirga, where it was clarified that the complaint was based on miscommunication. A similar instance was also shared by Agriculture Director in Nowshera. The evaluation team learnt during the course of evaluation that the communities in most of the cases were not aware of the overall assistance provided in each village, particularly at the initial stages, which appeared to be part of a strategy adopted by ACF team to avoid conflicts and favouritism at the outset of the project implementation. Female committees were also unaware of the project details and beneficiary selection criteria, which indicates their limited involvement in decision making due to the local norms. Most of the beneficiaries were not clear why they were not selected for cash grants or agriculture or livestock package as they were not aware of the criteria and of the overall assistance being provided in the village by ACF. Humanitarian Accountability systems including complaint mechanism are vital to ensure impartiality within LRCs resulting in transparency with regards to beneficiary selection and hence enhancing effectiveness of the overall programme. HAP systems ensure planned benefits reaching the most deserving among the communities and achievement of intended impact. Recommendation: There is a clear need for ACF to invest more on integration of Humanitarian Accountability principles across its programs and initiate capacity building of its teams in HAP.

33

8 Lessons Learnt Some of the key lessons learnt are as under: a. Participatory planning is extremely important right from inception to completion of the project Baseline conducted by ACF team at the outset of project designing allowed them to grasp the local context and identify community priorities. KPHA hence targeted the most urgent needs of the affected population with relevant and appropriate solutions. This formed the basis of participatory planning at the initial stages. All challenges faced during execution of KPHA and similar interventions can be easily handled if participatory planning remains central to the entire project cycle. Goat fairs within KPHA are also an excellent example of participatory planning where all relevant stakeholders were fully involved and delivered their responsibilities in a timely and appropriate manner, a modality that other humanitarian actors need to learn from this experience. b. More investment in community mobilization, greater is the impartiality within community

structures resulting in greater impact Formation of representative and impartial community structures is extremely important in achieving uniform impact within interventions where communities are expected to play a central role. Emergency interventions like KPHA need innovative ideas and strong approaches for community sensitization and mobilization. ACF teams need to be thoroughly trained in such communication skills to be able to organize the communities within short term programs and lead the process ensuring community structures remain accountable and impartial as much as possible. c. Regular output tracking coupled with process monitoring ensures reaching desired impact Program quality can be enhanced if there is a regular system of feedback and input from various levels of the program participants. Easy to understand and measureable indicators and simplified processes along with clear protocols can be a very useful recipe for quality programming and achieving intended impact. ACF needs to invest more on M&E section in order to achieve this.

34

9 Recommendations Specific recommendations presented below have been drawn from the findings and analysis presented above in Section 7 of the report. FSL a. ACF is recommended to negotiate with banks in future and improve Cash Transfer

mechanisms. When engaging banks for cash transfer programming, ACF needs to negotiate terms and conditions to enable swift, safe and dignified cash transfer to the beneficiaries, particularly the most vulnerable. A large majority of beneficiaries of KPHA project reside in rural areas, away from urban centres; in such cases banks must ensure delivery of cash close to their homes. In cases where mobile banking is not a possibility at all then banks need to ensure that they are able to transfer cash by more traditional means. b. Joint planning (between ACF and LRC) is strongly recommended as an effective way for

improving Cash transfer programming

FSL technical teams are suggested to develop protocols for cash transfer through banks. These protocols should emerge out of a discussion with LRCs and with bank, both at the bank head office level and at the local branches level. Beneficiary case load per branch is to be determined and working days are to be blocked by each branch for providing services to the beneficiaries on a priority basis. Proper facilitation and comfort for the beneficiaries, particularly for women, elderly and physically challenged, at each branch is to be ensured by both ACF teams and bank authorities. Discussions and planning with the LRCs must result into determining appropriate days for each beneficiary group to visit their respective branches and receive cash. ACF teams on ground with the help of LRCs will have to ensure that each beneficiary is well aware of the specified days and protocols.

c. Keeping in view the inflation trends based on market surveillance, ACF teams should allow

adjustments in the value of food basket and overall cash transfer modalities

There is a huge time gap between project development of KPHA and execution of cash transfer programming within it. The value of the food basket calculated at the design stage may not be as relevant later due to exceptional inflation trends and fluctuation in prices of commodities and fuels on a weekly basis. Regular Market surveillance should be able to address such challenges. Market surveillance data from project baseline stage is to be compared with the prices collected at various intervals till the execution stage. If the analysis suggests that there is a clear variation in trends then the value of food basket should be adjusted and necessary changes should be made to cash transfer modalities. Such changes may be proposed to donors in a timely manner and formal approval should be sought. These adjustments based on regular market surveillance will allow the project to effectively cover the food basket thereby achieving intended impact of the CTP component. WASH a. Special needs of the most vulnerable target groups should be catered for at the design

stage

35

The project interventions should be flexible in design and delivery in accommodating those with special needs. For example the demand of sitting commode instead of squatting commode for the polio infected disabled persons should be considered by the ACF team at the planning stage. b. Household latrines should be designed keeping in view the availability of water. In remote hilly and water scarce areas where women fetch water from distances taking hours, the construction of water-based latrine should be discouraged. The focus in such areas should be on water supply followed by latrines. Constructing a latrine in a house which hardly manages water for drinking and other domestic purpose would be less appropriate. Raised water tanks require pumping facilities therefore were phased out as they are too expensive. The traditional water drums inside the latrines should be encouraged which can be used for both flushing, shower and hand washing. c. Wastewater from latrines should not be drained into the open streets or irrigation channels

as it poses serious health risk.

The discharge of untreated wastewater from the septic tank of latrines to open drains or irrigation channels has serious health and environmental consequences. This waste should either be passed through cheap and affordable local treatment e.g. subsurface constructed wetlands or let leach down to the soil if the groundwater is deep and soil profile contains clay layers to prevent contamination of the groundwater. Emptying of septic tank when it is filled and discharging it in the nearby treatment plant could be another option but availability of such treatment facilities and cost and labour involved in the transportation may pose a problem. Another theoretically sound but culturally challenging option could be a dry toilet (waterless) thereby separating urine and faeces. In this case urine can be directly applied to crops after one month storage and faeces can be composted before its application as manure in agriculture production.

d. Capacity building of communities in solid waste management is recommended. Communities should be trained in solid waste segregation, collection and dumping in appropriate sites. The collected solid waste should be composted and local community should be trained on how to segregate and compost their household organic waste and use it as manure in the kitchen gardens. Preparedness Planning a. Integrate DRR and preparedness planning more actively in ACF emergency Programs such

as KPHA

Integrating DRR would include a thorough risk analysis at the time of project inception in order to establish a baseline. It is recommended that mapping of such hazard prone communities is carried out. Based on their risk profile, the most risk prone communities should be engaged in basic preparedness and awareness raising activities. This will enable communities to identify hazard prone areas within each village, safe havens or shelters along with evacuation routes. These plans should be displayed within each village and the whole community must be aware of their preparedness plan.

36

Support Systems a. Support teams should be actively engaged in facilitating cash transfer programming Cash transfer programming may include various modalities. Support teams must be part of the process that should result into setting up protocols for each modality. They should support the program team in negotiations with the banks both at the bank head office level and with the local branches in target areas. Support teams should monitor the quality of facilitation and services provided by the banks during specified days for cash transfers to beneficiaries. They should also accompany program teams to the field in case of direct cash distributions to beneficiaries. Support from the finance/admin department would increase transparency and accountability and hence quality within the cash transfer programming. b. Qualified and experienced national staff on key program positions will improve the quality of

ACF programs in Pakistan. Given the security situation in the country, nationalization of key program positions is essential to ensure qualified and experienced managers are present on ground and available to monitor and support field teams in delivering quality programs. As far as technical sectors are concerned, ACF will have to build the capacity of its senior program team with the help of its expatriate technical leads who will provide technical backstopping and support in setting up quality assurance systems within each sector. FSL Surveillance a. ACF must invest more time and resources in building its capacity in developing and running

FSL Surveillance system.

ACF technical teams are suggested to develop FSL Surveillance Strategy for ACF programming across Pakistan. This strategy should describe the purpose and objectives of FSL Surveillance, provide rationale for selecting specific market systems for selected commodities, methodology for carrying out data collection and analysis and operational protocols for guiding the technical teams in setting up a regular market surveillance system. The protocols should inform on the process of 1) determining essential commodities, whose prices affect routine consumption of a vulnerable household, 2) developing a system in which market prices from target districts are regularly recorded, 3) diverse Analysis to be generated that can help in decisions related to program appropriateness, thus improving program quality and its impact and 4) capacity building of FSL teams in FSL Surveillance system. Project Monitoring a. ACF needs to set up robust monitoring systems that ensure successful delivery of results as

well as encourage adherence to processes and promote standardization.

Following measures are suggested in order to set up monitoring and evaluation system within ACF program in Pakistan: M&E Initial Assessment ACF is required to carry out a thorough initial assessment to understand the M&E needs within ACF Pakistan. This exercise should assess information needs of the organization based on the

37

diversity of thematic areas within ACF programs in Pakistan, ACF Head quarter’s needs and donor requirements. It should also provide a detailed analysis of existing staff capacity and identify expertise required to develop and run M&E functions. Setting up an M&E System Based on the analysis of the initial assessment a customized M&E function is to be set up within ACF Pakistan. This function should be external to the programs and provide necessary analysed programmatic information to agreed recipients and should provide technical support to all programs across the country. It may have the following features: M&E Team – A core team is suggested that remains external to the programs to provide

external independent view and out of the box analysis to improve the quality of programs.

M&E Framework – A framework is to be developed that identifies all program related M&E questions, key indicators, means of verification, data collection methods, specific responsibilities and data collection frequencies.

Capacity Building – The core team is suggested to develop SoPs and protocols in order to

mainstream M&E in ACF programs. It should also plan and carry out capacity building of program teams in monitoring and evaluation. The core team should establish reporting mechanisms and train the teams on developing effective M&E reports.

Database – Based on the program needs a suitable database is to be developed which

allows consolidation of information from various programs and provides analysis with reference to strategic program objectives and indicators.

Evaluation and Learning– M&E function should also facilitate all mid-term reviews and end

of project evaluations. It should also facilitate lessons learnt workshops to identify and document best practices, key lessons learnt and disseminate internally and externally.

Accountability a. ACF is strongly recommended to build the capacity of its teams in community organization

and mobilization.

ACF needs to develop a Community Organization and Mobilization Manual for its programs in Pakistan. This manual should serve as a guide for transforming passive recipients of humanitarian assistance into local community based structures that represent all sections of the society, are impartial and are mobilized to take positive collective decisions in the best interest of the entire community. The manual should provide clarity on the essence of community mobilization with reference to development and detailed description of its purpose and objectives. It should also provide a thorough understanding of essential steps required to carry out community organization and community mobilization. These steps must also contain detailed processes and clear indicators and benchmarks that can help the teams in monitoring the process. This strategy should be able to guide ACF programs on how and what to communicate to programs related external audience that include all primary and secondary stakeholders. b. There is a clear need for ACF to invest more on integration of Humanitarian Accountability

Principles across its programs and initiate capacity building of its teams in HAP.

38

It is suggested that ACF develops its humanitarian accountability framework in light of its humanitarian programming in Pakistan. The framework should inform programs on key indicators for ensuring transparent and accountable program delivery in ACF Pakistan’s programs context.

10 Annexes

10.1 Rating of the Project based on DAC Criteria Criteria

Rating Rationale

(1 low, 5 high) 1 2 3 4 5

Impact The project has had a positive impact on the lives of the flood / conflict affected people targeted under this project. Rehabilitation of irrigation channels for agriculture improved crop yields; cash for work, unconditional cash grants agriculture and livestock support clearly improved beneficiaries’ access to income and enabled them to cover pressing household food, health and other priority needs. WASH inputs contributed in initiating changes in behaviours and practices related to hygiene and sanitation which is expected to reduce morbidity. Trainings conducted by ACF had triggered internal discussions within the line departments at the District level and the District authorities intended to initiate contingency planning in the wake of approaching monsoon season.

Sustainability Construction / rehabilitation of WASH related infrastructure and irrigation channels is clearly an essential intervention covering immediate needs and addressing longer term vulnerabilities. If maintained the communities will continue benefiting from them. Hygiene promotion and capacity building on agriculture and livestock support and DRM reflect sustainable efforts that will continue to transfer knowledge and skills among communities which will contribute to change in the longer term. The project has also paved the way for a long term engagement of the affected communities with local authorities who are eventually responsible for supporting them. Further efforts are required to establish thorough understanding of the flood and conflict affected communities in various risk reduction measures that could be initiated at the local level. LRCs / WUCs may continue, to a limited extent in taking ownership of the collective problems and facilitating communities in solving them in an organized fashion.

Coherence Major duplications were successfully avoided as ACF, at the designing phase, coordinated with PDMA and key actors through clusters coordination mechanism at the Provincial level which allowed the project to plug the gaps effectively. Existing vulnerabilities need continued efforts to ensure that communities are able to fully recover from the losses faced. Community Based Disaster Risk Reduction is an essential approach that must be rolled out in these communities to help them face disasters in future with lower risk of loss to lives and livelihoods.

Coverage This intervention wasn’t designed as blanket coverage. Hence 100% households in a village and 100% villages in affected UCs were not covered. There were communities that had both FSL &WASH needs but with meagre resources it was highly challenging for ACF to support everyone with everything. Nevertheless ACF successfully covered some of the most crucial needs of various categories of flood / conflict affected men, women, children and the most vulnerable people in most of the villages.

Relevance / Appropriateness

ACF targeted the most urgent needs of the affected population in resettling and restoring their lives and livelihoods which were the most relevant and appropriate interventions, suitable to the context. Baseline conducted by ACF at the initial stages of the program design allowed them in prioritizing the most vulnerable communities and their most pressing needs, which appears to have been the right approach to intervene in this case. Strategies and approaches adopted for the execution of various components were highly appropriate to the context and contributed to the success of the overall assistance. Integration of FSL with WASH and adoption of Cash Transfer modalities that included Cash Vouchers and Livestock fairs allowed communities choice that enhanced relevance.

40

Effectiveness The interventions remained effective for the beneficiaries, in both FSL and WASH components. 100% beneficiaries of water provision had improved access to safe and sufficient water for drinking and other household usage. Rehabilitation of irrigation channels had directly improved crop yields. Latrines, hygiene kits and hygiene promotion sessions had clearly increased communities’ practices related to safe hygiene and sanitation. Narrowed timeframe didn’t allow ACF to accommodate flexibility in latrine designs keeping in view the special needs of the physically challenged persons. In some cases provision of latrine was not linked with availability of water which resulted in very limited use and open defecation by most of the household members continued. Cash transfer through Vouchers and organizing livestock fairs remained successful in allowing beneficiaries' choice with regards to quantity and quality of inputs. Community mobilization could not be effectively carried out due to the initial time loss hence LRCs / WUCs were in several cases represented by individuals or dominant groups within the villages who were responsible mainly for decisions regarding selection of beneficiaries. Female committees were also not aware of their role and details of project inputs and had almost negligible role in decision making.

Efficiency Most of the project inputs were provided in a timely manner. This was a daunting task given the huge delay in the project start up, owing to the delayed approval of the project from ECHO and painfully slow process of seeking NOC from the PDMA. It will be fair to say that ACF teams both in the country office and in the field have done a tremendous job in delivering the project inputs, particularly those that were time / season sensitive well in time, thus allowing the beneficiaries to sow wheat on time, a crucial step in the revival of conflict and flood affected communities in the target areas. ACF explored several options for smooth, transparent and dignified cash transfer mechanisms and initiated transfers through a local bank but eventually large sums were transferred to beneficiaries directly and indirectly as liquid cash. This was adopted as a strategy to speed up the cash transfer and minimize difficulties to beneficiaries in accessing cash from banks. ACF teams identified group leaders from among LRCs and issued cheques in their names who withdrew cash for the whole group working on a specific CFW scheme in a village. These leaders then distributed cash to the CFW beneficiaries. ACF field team could not effectively design and implement regular surveillance of the markets as they were overwhelmed with the execution of other components of the project. The overall assistance was efficiently delivered keeping in view high per beneficiary costs due to remoteness, challenging and risk prone environment, technical expertise engaged to deliver this intervention and late start-up.

41

10.2 Best Practice

Cash Vouchers and Livestock Fairs Innovative Features & Key Characteristics

Livestock restocking among the disaster affected communities through organizing goat fairs and using Cash Vouchers for transferring cash to vendors that supplied goats in the fair and providing vouchers to targeted beneficiaries and allowing them to select goats of their choice is a highly innovative way of restocking livestock. Key characteristics are as under: Place for goat fair was selected in consultation with authorities Vendors were selected and registered Criteria for goat breed and age was developed in consultation with livestock

department Awareness about the entire mechanism was given to both beneficiaries and

vendors Beneficiaries were issued with vouchers A specific day was fixed for the goat fair and all stakeholders informed Goats that didn’t meet the criteria were rejected before staring the fair Beneficiaries purchased goats and the vendors received the vouchers from

them Before exiting the fair, all goats were examined and vaccinated by officials

from livestock department Upon submitting their vouchers to ACF, vendors were paid through ACF’s

regular payment mechanism Advantages of the goat fairs and cash vouchers were as under: There was no direct cash handling involved Communities had choice to select goats as per their need. It enhanced

relevance of the intervention to the realistic needs. 80% of the beneficiaries were women who visited the fairs and took active part

in a controlled economic activity where there was close monitoring and no middle men or market taxes

There was freedom to bargain and utilize saving per the household needs This also supported in reviving / strengthening the local markets and small

scale vendors who also brought their local goats for sale and benefited from the process

Cash Vouchers and Livestock Fairs reduced the risk of large vendors getting benefited in case of centralized procurement

It also reduced the risk of importing non local breeds / varieties from other Provinces which would have been unsuitable for the local context resulting in high morbidity and mortality among the goats thereby reducing the impact

Involving local Livestock Department for vaccination of animals was another successful aspect within the livestock fair where local communities received awareness on livestock health and measures to prevent livestock mortality and morbidity

This increased communities’ interaction with the local authorities and highlighted the responsibility of these departments in service delivery

Practical/Specific Recommendations for Roll Out

Ensure that the beneficiaries are selected as per the criteria, those who don’t deserve or are not interested should not be selected

Awareness and sensitization of beneficiaries should be focussed properly before the actual fair so that the beneficiaries purchase best goats rather than trying to save money

A mechanism should be devised so that cash in shape of savings in case of a better bargain should not be the attraction

Efforts should be made to ensure that there is a uniform impact of the activity over all the beneficiary households

Documented success or lessons learnt should be widely disseminated for others' benefit

Small farmers and women from villages who own goats may also be encouraged to get registered as vendors

42

10.3 Evaluation TORs

TERMS OF REFERENCE

For the External Evaluation of ACF’s

Humanitarian support to Conflict and Flood-affected populations in Pakistan

Programme Funded by ECHO

Contract Reference ECHO/PAK/BUD/2011/91014

3rd April 2012

43

1. CONTRACTUAL DETAILS OF THE EVALUATION 1.1. Key Evaluation Dates

Expected Start Date: 7th May 2012 End Date: 19th June 2012 Submission of Draft Report 7th June 2012 Submission of Final Report 19th June 2012

1.2. Language of the Evaluation Language Requirements for the Evaluation: English Language of the Report: English

1.3. Workplan& Timetable Activities Working

Days Briefing HQ 1 Travel to the mission 1 Briefing Mission, review of documents, and preparation of field work 3 Collection of secondary information in provincial capital 1 Field Work - Nowshera/Charsadda 9 Field Work - Timergara 5 Collection of secondary information in capital 1 Data Analysis and preparation of the first draft report 7 Workshop and presentation of finding in Peshawar Office 1 In country debriefing in Islamabad Office 1 Finalization of the report on the basis of Field, HQ, and ACF-UK comments 2 Total 32

44

1.4. Budget for the Evaluation EURO Total Amount Available 10,000 Deadline for invoicing (Contract End):

30 July 2012

USD Units Total ACF-UK ACF-Field Comments Consultant's Fee 400 32 12800 √

International Travel 1500 1 1500 √ Airport Transfers 50 1 50 √

Visa 100 1 100 √ - Domestic flights 600 1 600 √

Per Diem/Food 20 20 400 √ for Field work days only Accommodation 0 0 0 - - consultant to stay in ACF GH

Translator/Assistant 0 0 0 - - Local transportation 0 0 0 - - consultant will use ACF cars

Bank charges 16 3 48 √

Total 15,498 USD

(12,000 EUR) The consultant is responsible for personal insurance during the evaluation. The consultant will also provide any necessary materials (including laptops) required for the evaluation. 1.5. Nationality of Consultant:

Due to ACF security Guidelines and the current Pakistani Government’s treatment of certain nationalities the consultant should not be a US citizen. 2. DETAILS OF THE PROGRAMME Name of the Programme: KPHA Location: Khyber Patunkhwa Starting Date: 1 May 2011 End Date: 30 April 2012

2.1. Map of Programme Area

45

2.2. Programme Overview Over the last decade, Pakistan has experienced large-scale involuntary internal displacement caused by a range of factors. 2010 was the second consecutive year since the Afghan refugee crisis began in 1979 that the number of internally displaced persons in Pakistan exceeded that of registered refugees. The main cause for internal displacement in the spring of 2009 was military operations against militants in Malakand region of the Khyber Pakhtunkhwa province, leading to an exodus of about 2.3 million people in a little over a fortnight and creating one of the largest displacement crises in recent times. Military offensives against Taliban militants in the Federally Administered Tribal Area (FATA) bordering Afghanistan also contributed considerably to involuntary displacement, pushing the number of internally displaced persons to 2.7 million between April and July 2009, the largest internal displacement of population in the country’s history until then. Large-scale displacement occurred in Pakistan in August and September 2010 again after the worst flooding to hit the country in living memory affected 20 million people, forcing over 7 million people from their homes. Although most of the flood IDPs returned to their home areas soon after floodwaters receded, they were mainly living out in the open as over 1.9 million houses had been damaged or destroyed across the country. The large-scale displacement crises in 2009 and 2010 were only the latest human exodus in Pakistan. They had been preceded by dislocation of population following clashes between rival militant and sectarian groups in the tribal areas; military operations against extremist militants in parts of FATA and against insurgents/dissidents in Balochistan; generalised violence and violations of human rights; and other natural and human-caused disasters, including a devastating earthquake in Khyber Pakhtunkhwa and Azad Kashmir; the annual havoc caused by floods across the country; sea intrusion; and displacement induced by development projects such as enhancement of water storage capacity at Mangla Dam, construction of Mirani Dam and sale of two Islands off Karachi to

46

an international real estate developer. All indicators suggest that internal displacement will remain a key issue of concern in Pakistan at least in the medium term. Response to different crisis by various stakeholders Pakistan’s government failed to respond adequately to the conflict-induced displacement crisis between 2004 and 2008. But since then, in tandem with international agencies, it has provided support to a huge number of IDPs. Its achievements include the registration of more than half a million internally displaced households; the provision of food and non-food items to the majority of those households; of national identity cards to almost 80,000 displaced women; of health care services to many of those residing in displacement-affected communities, and shelter for many of the most vulnerable IDPs in camps. Cash cards distributed to at least 320,000 households have been an efficient means of reaching IDPs outside camps (UNHCR, January 2010). Although it receives foreign financial support and the advice of UNHCR, the government has actively managed the response to internal displacement (Dawn, January 2010; SSG 2010). While the UN Humanitarian Coordinator’s office, Special Envoy’s Office and OCHA have coordinated international agencies, other national authorities as well as international agencies have assumed sector-specific responsibilities (PHRP, 2010).

ACH was part of the Flash Appeal and actively contributed/supported the rolling out of the cluster system in KPK and in Sindh, where it began supporting flood affected populations. As of 31 January of 2011, the government declared an end to relief, and at end of March, official end to the emergency, with some districts allowed to continue with emergency actions, mostly in the south. Most importantly, the national and provincial governments allowed agencies to operate for a limited time without NOCs, thus facilitating field activity timing. This has now lapsed, and proposed activities like this must be approved by the KPK PDMA and district authorities.

Deaths 1,985

Injured 2,946

Households damaged 1,744,471

Population affected 20,184,550

Cropped areas (Hectares) 2,244,644

Districts Affected 78 2.3. General Objective To reduce mortality and suffering and increase resilience of conflict and flood-affected populations including the most vulnerable persons in area of intervention.

2.4. Specific Objectives/Results

To prevent and mitigate the spread of water borne disease and food insecurity

R 1: At least 10,925 flood and conflict affected households (98,325 individuals) have improve access to safe drinking water, proper hygiene and sanitation facilities. R2: 5,000 flood and conflict affected households have reinforced their food security and increased or preserved their asset base.

R 3: Improved awareness and capacity within high risk communities and among stakeholders to identify and manage disaster risk including capacity building at the UC and Community level.

2.5. Programme Activities R 1: At least 10,925 flood and conflict affected households (98,325 individuals) have improve access to safe drinking water, proper hygiene and sanitation facilities:

47

Rehabilitation/Improvement of 33 disaster resilient major water supply schemes, 35 smaller

community water sources 6 water resources irrigation canals for production, 10 waste management systems and enforcement/refreshment of 78 water committees for management of the systems.

Support 300 households sanitation for EVI/PwD and Rehabilitation of WASH services in 15 Institutional structures

Conduct behaviour change communication through 200 male and 200 female general hygiene promotion sessions on personal, domestic and environmental sanitation related sessions including 500 female door to door sessions, 3000 hygiene kits during or after sessions, training/mobilization of frontline hygiene workers including Lady Health Workers, teachers and health centre staff.

R2: 5,000 flood and conflict affected households have reinforced their food security and increased or preserved their asset base. Beneficiary selection Mobilization of 2,500 households to engage in works project Provision of small animal livestock assets to 750 households Provision of agricultural and horticultural support to 1750 households Capacity building and training activities Surveillance activities R 3: Improved awareness and capacity within high risk communities and among stakeholders to identify and manage disaster risk including capacity building at the UC and Community level: Conducting of PADR assessments at community level Stakeholder/community key people training in DRR Reinforcement and development of disaster preparedness coping mechanisms Development of emergency response plan at community level with a contingency for at base

level

Specific Activities WaSH Rehabilitation/ Improvement of 25 major piped water supply scheme Rehabilitation of 35 smaller (hand pumps, wells) community water sources Rehabilitation of 15 water supply systems in schools or health structures Rehabilitation of 6 secondary small scale irrigation canals for agriculture water quality testing (Bacteriological and chemical testing) Rehabilitation of 15 sanitary latrines equipped with hand washing facilities in schools or health structures Implementation of 10 waste management systems Support to EVI/PwD family construction and/or rehabilitation of 300 accessible household latrines with hand washing facilities and showers. Distribution of 3000 hygiene kits Conduct BCC through 200 male general hygiene promotion sessions Conduct BCC through 200 female general hygiene promotion sessions Conduct general hygiene promotion sessions with 500 female door to door sessions Mobilization/training of frontline hygiene workers including Lady Health workers, teachers and Health centre staff. Establishment/reactivation and training of 78 water points/schemes committees Training and sensitization of local authorities and local institutions on Community Managed Disaster Risk Reduction (CMDRR) and support to community exchange visits FSL Mobilization of 25 Livelihoods Recovery Committees at village level to serve as partners in the project.

48

Recruitment of 100 workers in 25 villages Selection of beneficiaries of unconditional cash grants Determination of wage rate and working days Determination of CfW schemas Monitoring of activities Disbursement of CfW payroll and UCG Conduct PDM Selection of beneficiaries of livestock support Signature of MoU with Livestock and Agriculture Extension Departments Selection of vendors of goats Training of beneficiaries of livestock support Distribution of vouchers Organization of goat fair Selection of beneficiaries of agricultural support Selection of providers of agricultural inputs Training of beneficiaries of agricultural support Distribution of vouchers for agricultural inputs Conduct PDM and PHM Selection of beneficiaries of horticulture support Identification of providers of inputs Training of beneficiaries of horticulture support Distribution of inputs to beneficiaries Monitoring of activities DRM Reinforcement and development of disaster preparedness coping mechanisms Development of emergency response plans and contingency stocks

3. AIM OF THE EVALUATION 3.1. Target User(s) of the Evaluation ACF WaSH Coordinator, FSL Coordinator, Deputy Country, Director for Programs Implementing HQ New York and Madrid HQ Field Level WaSH and FSL Field Team Other ECHO

3.2. Objective(s) of the Evaluation - To evaluate the impact and approach of ACF’s KPHA WaSHand FSL project activities in Khyber Pakhtunkhwa and to provide recommendations for future programming. - Assess the current approach to community mobilising using Livelihood Recovery Committees and provide recommendation for future programming

- Analyze implementation strategy and efficiency of intervention and identify best practices (activities within the context)

- Assess relevance and methodology used for selection of beneficiaries in the field

3.3. Scope of the Evaluation

49

1. Project Design Were the objectives, results and activities selected in the implementation of this program

relevant for the identified population needs? Evaluate the level of collaboration with stakeholders in the area during the design and

implementation phases of the project. To what degree was this project in line with their projects? Did the project consider and integrate in its strategy Clusters recommendations and guidelines? Were the different project components of the WaSHand FSL interventions effectively integrated

in each sector and between them? Was the integration of the Livestock and Agriculture Extension Department to the FSL

component of the project successful? What were the key successes or failures? How well has ACF worked with the line ministries and PDMA in the DRM component of the

project? What systems were in place to ensure that outputs provided were of the highest quality possible

and were acceptable to beneficiaries? Were beneficiaries' preferences been taken into consideration in the selection of the distribution

methods? 2. Beneficiary Targeting To what extent were the most vulnerable members of the target population effectively covered

by the project? Was the criteria and indicators defined in the project accurate to identify the vulnerable

population? Did the beneficiaries selected for the project meet the criteria established in the project design? Evaluate the targeting of beneficiaries (effectiveness/appropriateness) for the 300 household

latrines – did we succeed in targeting EVI/PWD in the WaSH component? What efforts were made to ensure that certain vulnerable groups and areas were not

overlooked? Were the Livelihoods Recovery Committees (LRC) an efficient way of identifying and selecting

beneficiaries? Or, on the contrary, did they bias the selection? What was the level of participation of the communities during the targeting process?

3. Project Implementation Did the program reach the intended results and objectives? If not, what were the main constrains

faced by the organization? If not, is it a consequence of poor quality of the initial assessment, poor project design or poor implementation?

To what extent was the project implemented according to plan? Evaluate the factors limiting the timing of the implementation of the project.

Was the assistance appropriate in relation to the customs and practices of the target populations?

Evaluate the degree of satisfaction with the program (design, implementation and outcomes) in the target communities.

How effective were the different components of the intervention in ensuring relevant and timely project delivery?

Evaluate the negative or unexpected impacts from the interventions. What are the main problems that the population identified concerning the implementation of the

activities? Were those problems properly identified by the project team and solutions proposed during the

duration of the program? What was the level of beneficiary participation in selection, design and implementation and

monitoring of the project? Are the activities matching with what people wanted? What measures were taken to ensure a gender balance at all stages of the project (assessment,

design and implementation)? Did chosen activities allow for this?

FSL Have assessments been properly done for the selection of the distribution methods in FSL

program? Have the distribution methods been appropriately chosen given the results of those analyses? Is there any noticeable impact of these distribution methods on the markets? How did the local population perceive the short-term benefits of the Cowl against the long-term

50

benefits of the WaSH infrastructure? Have wages earned for CfW activities effectively helped beneficiaries to preserve and recover

assets that support future income generation? How and for what purposes cash earned was mostly used?

Evaluate the direct and indirect impacts of the different modalities of the distribution of agricultural inputs and small animals.

Evaluate the impact of the different modalities of FSL intervention on food access of the beneficiary population in the short and medium term

Evaluate the impact of the FSL intervention on the recovery capacity of the beneficiary population

Evaluate the impact of their training and disaster plans e.g. impact on capacity of trained population to be better prepared through the training and preparedness plans.

WaSH How appropriate was the Hygiene kit distribution (in terms of content, timing and inclusion)? How relevant were the technical water and sanitation solutions provided by ACF given the

identified problems? Evaluate the impact of implementation on the availability and quality of water available in the

communities Did the water point/system construction/rehabilitations provide a significant decrease in water

contamination? Evaluate the impact on agricultural production (increment in irrigated lands, expected increment

in production) of the restored and rehabilitated irrigation channels. Evaluate the environmental impact of the restored and rehabilitated drainage channels. What effect did the hygiene promotion training have on beneficiaries' behaviour? What economic, social and behavioural changes have the water supply and sanitation facilities

brought in the project area? How well has ACF reinforced existing hygiene promotion channels by training teachers and lady

health workers?

4. Support Systems How realistic and appropriate was the budget of the project? Was it respected? Were admin/log procedures in place and understood, accepted and respected by all? Was the support team efficient in supporting the implementation of the programme? What were

the main issues/weaknesses? Can bottlenecks be identified in the supply chain of the goods and services required by the

project? Provide recommendations on improving support processes. How could subcontractors have improved efficiency in the implementation of this project? Were there alternatives to reduce the cost per beneficiary without affecting the quality of the

outputs of the project? Were the human resources systems in place appropriate? Was there added-value in having

expatriate programme managers? How successful was the semi remote management (due to NOCs) implementation period? 5. Project Sustainability Evaluate ACF's exit strategy for the water point construction and rehabilitation component using

the committee training/capacity building and links with WaSH institutions. Were the construction material and masonry techniques applied appropriate (were they flood

resistant?) What alternatives would have been better? Evaluate whether the maintenance requirements of the structures were within the capacity of the

beneficiary users and if ACF did enough to reinforce these local capacities. Assess how the support (particularly training) provided to beneficiaries in the framework of the

agriculture FSL intervention (goats, agricultural inputs and horticulture) increased their production capacity in the medium term.

Did the FSL intervention in all its modalities appropriately consider improving resilience of beneficiaries to future shocks?

Evaluate the level of ownership of project activities by the communities; consider local structures (village committees) and (local) ACF personnel.

6. FSL Surveillance

51

Was the surveillance system effective in identifying products and markets to be considered in the program?

Was the information gathered through the surveillance system systematically shared with main stakeholders, including beneficiaries of the project?

Was the surveillance system appropriately handed over to local authorities or communities to ensure continuity at project end?

7. Project Monitoring How suitable and effective were the M&E systems in place through the duration of the project? Were indicators appropriately selected to measure the intended outputs and impacts? Are the tools used to gather information appropriate to provide the information necessary to the

follow up of the project? Is information appropriately analysed and results taken into account to improve the

implementation of the project along the cycle? How can monitoring techniques be improved?

8. Livelihoods Recovery Committees (LRC) Are LRCs representative of the population of beneficiaries? Has the LRCs setup improved

participation of the beneficiaries? What systems were put in place to ensure the transparency of the LRC to ACF and to the

beneficiaries? Have those systems been effective? Has the system put in place for the project effectively captured complaints and in general the

concerns of the population around the project? Has the system been efficient in fairly addressing those concerns?

Has the system been effective in providing ACF with contextual information about the way the LRCs have been dealing with complaints presented by the population?

What was the level of awareness of stakeholders (especially beneficiaries / non-beneficiaries) of project objectives and targeting criteria and mechanisms?

What is the impact of the method of beneficiary selection, especially the creation or strengthening of existent LRCs, on the power dynamics of the targeted villages?

3.4. Evaluation Criteria ACF subscribes to the Development Assistance Committee (DAC) criteria for evaluation: Impact, Sustainability, Coherence, Coverage, Relevance / Appropriateness, Effectiveness and Efficiency. ACF also promotes systematic analysis of the monitoring system and cross cutting issues (gender, HIV/AIDS etc). All external evaluations are expected to use DAC criteria in data analysis and reporting. In particular, the evaluation must complete the following table and include it as part of the final report. The evaluator will be expected to use the following table to rank the performance of the overall intervention using the DAC criteria. The table should be included either in the Executive Summary and/or the Main Body of the report.

Criteria Rating

(1 low, 5 high) Rationale

1 2 3 4 5 Impact Sustainability Coherence Coverage Relevance/Appropriateness Effectiveness Efficiency

3.5. Best Practices The evaluation is expected to provide at least one key example of Best Practice from the project/programme. This example should relate to the technical area of intervention, either in terms of processes or systems, and should be potentially applicable to other contexts where ACF operates.

52

This example of Best Practice should be presented as an Annex. 3.6. Evaluation Outputs The result of this evaluation should be presented in a written report and through several oral presentations:

One on the mission (to Head of Mission and relevant technical staff)

One at HQ (through teleconference to ACF HQ). 3.7. Methodology 3.7.1. Briefing Prior to the evaluation taking place, the evaluator is expected to attend a briefing at HQ level, and at field level with the Head of Mission and/or the relevant technical focal point. Briefings by telephone must be agreed in advance. 3.7.2. Field activities Consultants are expected to collect an appropriate range of data. This includes (but not limited to):

Direct information: Interviews with beneficiaries - Visit to project sites and to the facilities provided to the beneficiaries

Indirect information: Interviews with local representatives; interviews with project staff expatriate and national staff; meeting with local authorities, groups of beneficiaries, humanitarian agencies, donor representatives and other stakeholders. For indirect data collection, standard and participatory evaluation methods are expected to be used (HH interviews and FGDs with beneficiaries, non-beneficiaries, key informants – health workers, teachers and leaders)

Secondary information analysis: including analysis of project monitoring data or of any other relevant statistical data.

3.7.3. Report The report shall follow the following format.

Cover Page Table of Contents Executive Summary: must be a standalone summary, describing the programme, main findings of the

evaluation, and conclusions and recommendations. This will be no more than 2 pages in length. Main Body: The main body of the report shall elaborate the points listed in the Executive Summary. It

will include references to the methodology used for the evaluation and the context of the action. In particular, for each key conclusion there should be a corresponding recommendation. Recommendations should be as realistic, operational and pragmatic as possible; that is, they should take careful account of the circumstances currently prevailing in the context of the action, and of the resources available to implement it both locally and in the Commission. Annexes: Listed and correctly numbered. Format for the main body of the report is:

o Background Information o Methodology o Findings & Discussions o Conclusions Recommendations o Annex I (Best Practice) o Annex II (DAC-based Rating Table)

The report should be submitted in the language specified in the ToR. The report should not be longer than 30 pages including annexes. The draft report should be submitted no later than 10 calendar days after departure from the field. The final report will be submitted no later than the end date of the consultancy contract. Annexes to the report will be accepted in the working language of the country and programme subject to the evaluation. 3.7.4. Debriefing & Learning Workshop The evaluator should facilitate a learning workshop: To present the draft report and the findings of the evaluation to the Mission and other stakeholders.

53

To gather feedback on the findings and build consensus on recommendations. To develop action-oriented workshop statements on lessons learned and proposed improvements for the

future. 3.7.5. Debriefing with ACF HQ The evaluator should provide a debriefing with the relevant ACF HQ on her/his draft report, and on the main findings, conclusions and recommendations of the evaluation. Relevant comments should be incorporated in the final report. 4. PROFILE OF THE EVALUATOR Extensive experience in the implementation of FSL/WaSH programmes Strong experience in the evaluation of humanitarian / development projects Good Knowledge of KPK context Strong technical knowledge on wide range of emergency and post emergency WaSH activities Significant experience in coordination, design, implementation, monitoring and evaluation of programmes Good communications skills and experience of workshop facilitation Ability to write clear and useful reports (may be required to produce examples of previous work) Fluent in English Understanding of donor (ECHO) requirements Ability to manage the available time and resources and to work to tight deadlines Independence from the parties involved.

5. RIGHTS The ownership of the draft and final documentation belong to the agency and the funding donor exclusively. The document, or publication related to it, will not be shared with anybody except ACF before the delivery by ACF of the final document to the donor. ACF is to be the main addressee of the evaluation and its results might impact on both operational and technical strategies. This being said, ACF is likely to share the results of the evaluation with the following groups: Donor(s) Governmental partners Various co-ordination bodies Intellectual Property Rights All documentation related to the Assignment (whether or not in the course of your duties) shall remain the sole and exclusive property of the Charity

54

10.4 The Evaluation Team Luqman Ahmed Zulqarnain Malik Dr BahadarNawab

55

10.5 Evaluation Schedule

EVALUATION SCHEDULE BY IDEAS FOR LIFE TRUST

S# Date Task Responsibility

1 13 / 06 / 2012

HQ Briefing Luqman Ahmed Zulqarnain

2 14 / 06 / 2012 16 / 06 / 2012

Briefing Mission & Review of Documents Luqman Ahmed Zulqarnain

3 15 / 06 / 2012 17 / 06 / 2012

Refining of Methodology, Data Collection Tools & Techniques Dr Bahadar Luqman Ahmed Zulqarnain

4 18 / 06 / 2012 Secondary Data collection and Planning meeting in Peshawar Luqman Ahmed Zulqarnain

5 19 / 06 / 2012 Finalizing methodology, tools and Training of the data collection team in Peshawar

Luqman Ahmed Zulqarnain

6 20 / 06 / 2012

Field Work in Charsadda / UC Agra Village Gulshan Abad & Village Sheikh Kalay

Evaluation Team

7 21 / 06 / 2012 Field Work in Charsadda / UC Tarnab Village Shahi Kulali& Village ShenayKalay

Evaluation Team Dr.Bahadur Joins the team

8 21 / 06 / 2012 Meeting with Food Security and WASH Clusters in Peshawar Luqman Ahmed 9 22 / 06 / 2012 Field Work in Charsadda / UC Mirzadher

Village DageMukarram& Village GhazoDherai

Evaluation Team

10 22 / 06 / 2012 Meeting with ACF’s Management in Peshawar to discuss ACF’s Systems

Zulqarnain

11 23 / 06 / 2012

Field Work in Nowshera / UC PirSabaq Village Zara Maina& Village MiangulKalay

Evaluation Team

12 24 / 06 / 2012 Data entry, cleaning, editing, verification Evaluation Team 13 25 / 06 / 2012 Field Work in Nowshera / UC Kabul River

Village Gujar Kas & Village WisalAbad

Evaluation Team

14 26 / 06 / 2012 Meeting with Line department & DDMO in Charsadda&Nowshera Luqman Ahmed 15 26 / 06 / 2012 Field Work in Nowshera / UC Chowki Town

Village ManaKhel& Village Shaheed Abad

Evaluation Team

16 26 / 06 / 2012 Travel to Lower Dir from Peshawar in the afternoon Evaluation Team 17 27 / 06 / 2012 Travel from Islamabad to Dir Luqman Ahmed

Zulqarnain Dr.Bahadur

18 27 / 06 / 2012 Meeting with Line department in Lower Dir Luqman Ahmed 19 27 / 06 / 2012 Meeting with ACF’s Management in Dir to discuss ACF’s Systems Zulqarnain 20 27 / 06 / 2012

28 / 06 / 2012 Field Work in Lower Dir / UC Chakdara Village Sessadha Village Ali Mast Village Chakdara Khas

Evaluation Team

21 28 / 06 / 2012 Return to Islamabad from Dir Luqman Ahmed Zulqarnain Dr.Bahadur

22 29 / 06 / 2012 30 / 06 / 2012

Field Work in Lower Dir / UC Chakdara Village Namseer Village KozKallay Village Banrdgay

Evaluation Team

23 30 / 06 / 2012 Return to Peshawar from Dir Evaluation Team 24 03 / 07 / 2012 Meeting with ACF’s Management to discuss ACF’s Systems in

Islamabad Zulqarnain

25 03 / 07 / 2012 Meeting with ECHO Luqman Ahmed 26 05 / 07 / 2012

09 / 07 / 2012 Preparation for Debrief Dr.Bahadar

Luqman Ahmed Zulqarnain

27 10 / 07 / 2012 Debriefing with ACF Islamabad Dr.Bahadar Luqman Ahmed Zulqarnain

28 18 / 07 / 2012 Evaluation Report First Draft to be shared with ACF Luqman Ahmed 29 27/ 07 / 2012 Evaluation Report Final Draft to be shared with ACF Mr.Luqman Ahmed

56

10.6 List of Persons Interviewed and Sites Visited List of persons contacted during external evaluation List of participants for FGDs (female beneficiaries) No Name Type of Beneficiary Village UC District 1 Saira Livestock Gulshan Abad Agra Charsadda 2 Nadia Hygiene kit 3 Muslehat Hygiene kit 4 Shazia Hygiene kit-livestock 5 Parizadgai Hygiene kit 6 Basmina Livestock 7 Saidulwara Livestock-Hygeine kit 8 Mashalla Hygiene kit 9 Napeda Hygiene kit 10 Shamia Hygiene Kit 11 Bakhria Hygiene kit 12 Shenai Hygiene kit-Livestock 13 Waheeda Hygiene kit 14 Hamida Hygiene kit 15 Salma Hygiene kit 16 Saira Livestock Sheikh Kalay Agra Charsadda 17 Nadia Hygiene kit 18 Muslehat Hygiene kit 19 Shazia Hygiene kit-livestock 20 Parizadgai Hygiene kit 21 Basmina Livestock 22 Saidulwara Livestook-Hygeine kit 23 Mashalla Hygiene kit 24 Napeda Hygiene kit 25 Shamia Hygiene Kit 26 Bakhria Hygiene kit 27 Shenai Hygiene kit-Livestock 28 Waheeda Hygiene kit 29 Hamida Hygiene kit 30 Salma Hygiene kit 31 Nelofar/ Zareen

khan Hygiene kit ShahiKulali Tarnab Charsadda

32 Reshama / Misal khan

Hygiene kit

33 Zahra / Jameel khan

Hygiene kit-Livestock

34 Farida / Rasheed khan

Hygiene kit

35 Feroza / Nasir khan

Hygiene kit- Livestock

36 Jamila / Javeed Hygiene kit 37 Pashmina /

Wahab gull Latrine –Hygiene kit

38 Saleem / Orangzeb

Hygiene kit

57

39 Jehan / FazalRehman

Hygiene kit

40 Khalid/Sahar gull Hygiene kit-hygiene promotion

ShenayKalay Tarnab Charsadda

41 Saidulwara/Inayatullah

Hygiene kit-hygiene promotion

42 Shaheen/Wasil Hygiene kit-hygiene promotion

43 Nazuka/Naseer Hygiene kit-hygiene promotion

44 Khoshnood/ Farman

Hygiene kit-hygiene promotion

45 Shagufta/ Hidayat

Hygiene kit-hygiene promotion

46 Fozia/Asad Hygiene kit-hygiene promotion

47 Sharafat/Sahib zada

Hygiene kit-hygiene promotion

48 Yasmeen/Shamsulqamar

Hygiene kit-hygiene promotion

49 Huma/Naseem khan

Hygiene kit-hygiene promotion

50 Bas bibi/Tila Muhammad

Latrine DageMukarram Mirzadher Charsadda

51 Fahmida/Akbar shah

Latrine-Hygiene kit

52 Tayyaba/Fazalshair

Latrine –Hygiene kit

53 Bibiamna/ Hukkam

Latrine-Hygiene kit

54 Sahiba/Diyyan Latrine-Hygiene kit 55 Jamshaida/

Ramzad Latrine- Hygiene kit

56 Sanafroza/ Shair Muhammad

Latrine- Hygiene kit

57 Khushnudia/ Muhammad khan

Latrine- Hygiene kit

58 Musarat/ Ziaurehman

Hygiene kit GhazoDherai Mirzadher Charsadda

59 Anwar begam/ Subhanudeen

Latrine

60 Mohsina/Bahar Ali Latrine 61 Shaheen/

Imtehan UCG

62 Mohtarama/ ShairAlam

UCG

63 Fahmida/ M Ilyas

Hygiene kit

64 Mohsina/Bahan Hygiene kit-promotion 65 Sabahat/Japan Hygiene kit-promotion 66 Jamila/Nasir Hygiene kit-promotion 67 Gulabzari/

Nazar Hygiene kit-promotion Zara Maina PirSabaq Nowshera

68 Guljahan/ Wajidali

Hygiene kit-promotion

58

69 Guldasta/Jahanzeb

Hygiene kit-promotion

70 Musarat/Noor nabi khan

Hygiene kit-promotion

71 Shamasahar/ Bazo

Livestock

72 Gulroja/Gul shah Seeds 73 Hajra/Shairdil Livestock 74 Hussanbibi/

Fazalewali Seeds

75 Jahanzeba/ Misal khan

Hygiene kit MiangulKalay PirSabaq Nowshera

76 Marwar/Abdul khaliq

Hygiene kit

77 Shamim/Sahib Hygiene kit 78 Azra/Afsar Hygiene kit 79 Jamila/

Noorhabib Livestock

80 HusanBano/Amir Akbar

Livestock

81 Maoba/Taqeer Hygiene kit 82 Basra/Mir hamza Hygiene kit 83 Shairbano/Nazeer Hygeine kit 84 Sangina

/ShairAlam Hygiene kit-Promotion-Latrine

GujarKas Kabul River Nowshera

85 Gulmina /Mirozal Hygiene kit-Promotion

86 Johara /Rasool khan

Hygiene kit-Promotion

87 Shaheen /Gulzareen khan

Hygiene kit-Promotion

88 Shahtareen /Jamodin

Hygiene kit-Promotion

89 Ishrat /Zareef khan

Hygeine-Promotion-Latrine

90 Akhtaray /Sapaid Ali khan

Hygeine kit-Promotion

91 Asmat/Niaz gull Livestock Wisalabad Kabul River Nowshera 92 Tasleem/Farid gull Livestock 93 Fayyaz/Qadir Unconditional grant 94 Rafsan/

Ahsanullah Livestock

95 Tajmina/Wali khan Hygeine kit 96 Tahira/Arshad Livestock 97 Merajbibi/Ali khan Seeds

98 Gulzaro/ Yaqoob khan

Hygeine kit

99 Fozia/Zarshad Hygiene kit Manakhel Chowki Town

Nowshera 100 Nohat/

Khursheed Hygiene kit

101 Wahida/ Haqnawaz

Hygiene kit

102 Sartaj/ Seeds-Hygiene kit Shaheedabad Chowki Nowshera

59

Shahjahan Town 103 Saima/Waqar Seeds 104 Sardara/

Ahsanullah Hygiene kit

105 Shahida/ Banaras

Agriculture

106 Riasat/Hayat khan Livestock 107 Saima/Amir jahan Livestock 108 Nargis/

Jamrood khan Seeds

109 Amina/Sabir khan Seeds 110 Tasleem/Din

Muhammad Agriculture

111 Munawar sultan Livestock 112 Fahmida Hygeine kit-promotion Sessadah Chakdara Dir 113 Hifsa Hygeine kit-promotion 114 Benazeer Hygeine kit-promotion

115 Yasmeen Hygeine kit-promotion

116 Raseeda Hygeine kit-promotion 117 Shazia Livestock 118 Wahida Livestock 119 Hamida Livestock 120 Bakhatjahan Livestock 121 Mampari Latrine 122 Syed haroonjahan Livestock-Hygeine kit

123 Saheen Livestock 124 Amina Livestock 125 Khatoon Livestock 126 Sakeena Livestock 127 Abida Hygiene kit Ali Mast Chakdara Dir 128 Shakira Hygiene kit 129 Tajbibi Latrine 130 Tasleem Hygiene kit 131 Saiqa Latrine 132 Rohafza Latrine 133 Noorjahan Latrine 134 Shamim Hygiene kit 135 Jamila Hygiene kit 136 Shahilal/

Gulraziq Unconditional grant ChakdaraKhas

(Dholay Shah) Chakdara Dir

137 Sardara/ Dilawar khan

Unconditional grant

138 Amtulsama/Ghulam

Latrine

139 Ayesha/ Habibullah

Latrine

140 Robina/ Jnashair

Latrine

141 Anwarsultan/ Mukamil

Latrine

142 Wahida/ Latrine

60

Salombar 143 Fazilat/Mir Ahmad

khan Unconditional grant

144 Uzma Livestock Namsair Khal Dir 145 Zeenat Seeds 146 Zahida Hygeine kits 147 Noor khaista Latrine –hygeine kit 148 Bastia Agriculture 149 Khatima Hygeine kit 150 Sherinanwar Latrine 151 Shaigull Latrine 152 Nahida Livestock KuzKalay Khal Dir 153 Khais Seeds 154 Bachahida Latrine 155 Adiabibi Unconditional grant 156 Tajdara Unconditional grant 157 Parveen Seeds 158 Musarat Livestock 159 Farida Seeds - unconditional

grant

List of persons contacted during external evaluation List of participants for FGDs (Male beneficiaries) No Name Type of Beneficiary Village UC District 1 Salahuddin/ Rahim

Ullah Latrine (Chairman) Gulshan Abad Agra Charsadda

2 Said Wali/Ashraf Uddin Khan

Hygiene Kit

3 MohibGul/ Fazli Wahid

Hygiene Kit

4 FazliRehman/ GulRehman

Hygiene Kit and CFW

5 SardarWali/ HabibGul Hygiene Kit 6 GhaniRehman/

FazliRehman Hygiene Kit and CFW

7 Akhtar Ali/ Yaqob Khan

Hygiene Kit and CFW

8 Sohbat Khan/ AnarGulam

Hygiene Kit and CFW

9 SajidUllah/ HazratSher

Hygiene Kit and CFW

10 Gul Muhammad/ Ashraf Uddin

Hygiene Kit, Latrine and CFW

11 Khan Muhammad/Fazal Muhammad

Hygiene Kit Sheikh Kalay Agra Charsadda

12 Naseer Ahmed/ GhulamNazir

Sulfate and Seed

13 Raz Muhammad/ Fazal Muhammad

Hygiene Kit

14 Jan Muhammad/ Wali Muhammad

Sulfate and Seed

15 Awal Khan/ Ali Khan Sulfate and Seed

61

16 Insha Muhammad/ Fida Muhammad

Sulfate and Seed

17 Muhammad Jalal/ Nisar Muhammad

Sulfate and Seed

18 Muhammad Saleem/ Noor Haleem

Sulfate and Seed

19 Mumtaz/ Turab Khan Sulfate and Seed 20 Fazal Muhammad/

Johar Ali Hygiene Kit

21 Jamuddin CFW, H-Kit ShahiKulali Tarnab Charsadda 22 Noor Ullah Jan CFW 23 GulZada Latrine, H-Kit, UCG 24 Payo Din Latrine, Goat, H-Kit 25 Abrar Latrine, H-Kit 26 Zakaullah Latrine, H-Kit 27 Zia Ullah CFW 28 Shahid Shah CFW 29 Muhammad Shahzad Latrine, CFW, H-Kit 30 Tila Muhammad Goat (To my

daughter) 31 Muhammad Aslam Agriculture Support 32 Muradullah Agriculture Support 33 Asfandiyar Latrine 34 Niaz Ali CFW 35 Bakhtiar Agriculture Support 36 Baswar Latrine, CFW 37 Momin Khan H-Kit ShenayKalay Tarnab Charsadda 38 Sabir H-Kit 39 Salih Shah H-Kit 40 Nouroz H-Kit 41 Abdullah H-Kit 42 Muhammad Musa

Khan CFW DageMukarram Mirzadher Charsadda

43 MuharamGul Latrine 44 Akber Shah Latrine 45 Ali Khan CFW, Latrine 46 Muqadar Khan Latrine 47 Sahib Zada Latrine 48 Arif Member No Assistance 49 Liaqat Ali Chairman No Assistance GhazoDherai Mirzadher Charsadda 50 Nadeem CFW 51 Jehangir member No Assistance 52 Tariq Volunteer No Assistance 53 Naseer Ahmed

(member) No Assistance

54 Muhammad Hussain Volunteer

No Assistance

55 Bahar Ali Latrine, CFW 56 Wakil Latrine, UCG 57 ZarifGul Latrine, CFW 58 GhaniRehman Goat Zara Maina PirSabaq Nowshera 59 Atlas Khan CFW 60 Jan Mohammad CFW 61 FazalGhani Goat 62 Mir Ghani Agri. Support 63 GulZarif Goat 64 Sardar Akbar Agri. Support 65 Mukaram Shah Agri. Support 66 Noor Faraz Agri. Support 67 Rehan Said Agri. Support

62

68 Sabit Khan Horticulture support, H-Kit

69 Umer H-Kit, CFW 70 ObaidUllah H-kit 71 Taj Muhammad H-Kit 72 Shamshad Agri. Support 73 Naeem Agri. Support 74 Noor Wali Khan Agri. Support 75 Masit CFW MiangulKalay PirSabaq Nowshera 76 Esaf Khan CFW, H-Kit 77 Muhammad Khan H-Kit 78 Itbar Khan Agriculture Support,

CFW, H-Kit 79 Nasir Khan Goat , H-Kit 80 Ajab Khan Agriculture Support 81 Yasir UCG, H-Kit 82 Atta Muhammad Agriculture Support,

CFW, H-Kit 83 Noor Rehman Agriculture Support,

H-Kit 84 Jehangir Goat 85 Amir Taj CFW 86 Amir Afzal CFW 87 Manzar Goat 88 Sarwar Khan CFW, H-Kit 89 Mashaor Agriculture Support,

H-Kit 90 SiadAfzal CFW, H-Kit 91 Noor Muhammad Agriculture Support, 92 Saiful CFW 93 Rasol Khan, member H-Kit, Latrine GujarKas Kabul

River Nowshera

94 Lawang, member Latrine, H-Kit 95 Riyal Khan H-Kit 96 Misal Khan Latrine, H-Kit 97 AlamZeb Latrine, H-Kit 98 Akbar Jan Latrine, H-Kit 99 Amir Afzal H-Kit 100 Zahir Shah CFW, H-Kit 101 GulZamir H-Kit 102 FaqirGul CFW, H-Kit 103 Sufaid Khan,

Chairman Agriculture Support, Wisalabad Kabul

River Nowshera

104 Rafiq CFW 105 Dolat Khan Livestock Support 106 Wali Khan CFW, H-Kit 107 ZahirGul CFW, H-Kit 108 Hamid Gul CFW, H-Kit 109 Noor CFW 110 Tehseen CFW 111 GhulamNabi

(Secretary) H. Kit, H, Promotion Manakhel Chowki

Town Nowshera

112 Dilawar (member) H.Kit, H, Promotion 113 SherAfzal Non 114 Mir Alam Khan

(Chairman) H. Kit

115 Din Muammad CFW, H-Kit, H-Pro Shaheedabad Chowki Town

Nowshera 116 Islamir H-Kit, H-Pro, CFW 117 Wisal CFW 118 Daryafat H-Kit, H-Pro, CFW 119 Dilawar H-Kit, H-Pro

63

120 Falak Niaz H-Kit, H-Pro, CFW, Agriculture Support

121 Gulwaz Shah CFW 122 Shujat Ali H-Kit, H-Pro 123 Said Ali Shah Agriculture Support 124 Amir Sultan (member) Ariculture, H-Kit Sessadah Chakdara Dir 125 Shah Wazir Agri 126 Samiullah member CFW 127 BadshahRehman

member Agriculture

128 Amin Ur Rehman Agriculture 129 Sajad Khan (member) CFW, 130 Muhammad Yar Khan

member Agriculture

131 HazratAyoub Chairman

CFE, H-Kit

132 NasirIqbal 133 Ghafar Khan H-Kit Ali Mast Chakdara Dir 134 Zia UlHaq H-Kit 135 Ali Khan (member

Son of chairman) Latrine and H-Kit

136 HajiburRehman (member Son of chairman)

H-Kit

137 Sardar Ali I was not in the village at that time

138 HawaldarChairman H-Kit 139 Hamid Ullah H-Kit 140 Khlil (member Son of

chairman) H-Kit

141 Sanober Latrine, CFW ChakdaraKhas (Dholay Shah) Chakdara Dir 142 Muhammad Naeem CFW 143 FazleRehman Latrine, CFW 144 Abdul Raziq UCG 145 HabibulNabi Agriculture Support 146 Khalid Agriculture, CFW 147 Abdul Hameed Agriculture 148 Ghulam Agriculture, Latrine,

CFW 149 Diyar Khan CFW, Agriculture 150 Said Amin Khan Agriculture, CFW 151 Rasheed Agriculture support 152 ZahoorZada

Chairman Agriculture Support, H-Kit

Namsair Khal Dir

153 Attau r Rehman Agriculture Support, H-Kit

154 Abdul Habib Agriculture Support, H-Kit

155 KifayatUllah Agriculture Support, H-Kit

156 JalilZada Agriculture Support 157 Amir Haleem Agriculture Support,

H-Kit 158 Qasim Jan Agriculture Support,

H-Kit 159 Adnan Said Agriculture Support,

H-Kit

64

160 Habib Ur Rehman Agriculture Support, H-Kit

161 SarBeland Khan Chairman

Agriculture Support KuzKalay Khal Dir

162 BadshahRawan H-Kit 163 MuhammdDawd Khan Agriculture Support,

H-Kit 164 Alamgir Khan,

member H-Kit and goat.

165 Paris Khan Water Supply Bandrgay Khal Dir 166 BahaderZeb Latrine and H-Kit

List of persons contacted during external evaluation

List of participants for SSIs (Non beneficiaries) No Name Gender Village UC District

1 JehanUllah Male Zara Maina PirSabaq Nowshera

2 Abida Female Zara Maina PirSabaq Nowshera

3 BakhtZari Female Wisalabad Kabul River Nowshera

4 Nihar Female MiangulKalay PirSabaq Nowshera

5 Afsana Female KuzKalay Khal Dir

6 Abida Female Manakhel Chowki Town Nowshera

7 Shamim Female Sessada Chakdara Dir

8 Chamana Female Shaheedabad Chowki Town Nowshera

9 Sabir Khan Male Barndgay Khal Dir

10 Shah Jehan Male Sheikh Kalay Agra Charsadda List of persons contacted during external evaluation List of External Stakeholders No Name Designation Organization Place 1 FidaHussain Cluster Coordinator - WASH

Cluster UNICEF Peshawar

2 Naseem Amir

Deputy Cluster Coordinator - FSL Cluster

FAO Peshawar

3 Khalid Khan

Provincial Cluster Coordinator - Emergency Rehab &Coord. Unit - FSL Cluster

FAO Peshawar

4 Sultan Hussain Shah

Agriculture Director Distt Line Department

Nowshera

5 Syed Aqeel Shah

Agriculture Officer Distt Line Department

Nowshera

6 BaseerMohd Agriculture Officer Distt Line Department

Nowshera

7 Mohammad Zahoor

DDMO Distt Management

Charsadda

8 Dr. Mohammad Khalil

Distt Livestock Officer Distt Line Department

Dir

9 Ubaidullah Khan

Agriculture Director Distt Line Department

Dir

65

10.7 Documents Consulted Project Proposal Interim Report PDM Reports Baseline Reports Beneficiary Database Report on Goat Fairs Success Stories Technical Reports on Livestock LRC Manual ACF Guidelines

66

10.8 Data Collection Tools FGD Guide – Direct Male / Female Beneficiaries and LRCs / WUCs Name of District / UC / Village: Total No. of HHs in the village: Total No. of Beneficiary HHs in the village: Participants:

S# Name Type of Beneficiaries Age

1

2 3 4 5 6 7 8 9 10 11 12 13

14 15

Geographic Targeting Which UCs were severely affected by the conflict / flood in 2010? How and when was your UC and village

affected? Which agencies provided assistance in your village and what kind? Was it relevant? Relevance, Appropriateness & Timeliness What were the immediate unmet needs of the communities in the post conflict / flood situation prior to this

project’s implementation? What assistance was provided to your village by ACF under this Project? CFW What % of the village households took part in CFW in your village? Was cash received by CFW beneficiaries enough to meet immediate basic needs or recover lost assets or

repay existing debts? What type of community infrastructure schemes were rehabilitated / implemented through CFW? Did engagement in cash for work affect availability of labor during the sowing season? What was the mode of payment to CFW laborers? Was it appropriate and timely? Were there any challenges faced by CFW beneficiaries while accessing cash from the banks? Unconditional Cash Grants How many persons received unconditional cash grants in your village? Was cash enough to meet immediate basic food and non food needs of the beneficiary household? What was the mode of payment? Was it appropriate and timely? Were there any challenges faced by UCG beneficiaries while accessing cash from the banks? Agriculture Horticulture Support What % of village farmers / women received agriculture / horticulture inputs? Were these inputs appropriate to the farmer’s / kitchen garden’s requirements? Were these enough for the land cultivated? Were these as per the local agricultural norms and practices? Was the quality of these inputs satisfactory? Was this assistance timely to allow the farmers to sow their crops on time and women to grow vegetables

on time? Was any training provided to the farmers / owners of kitchen gardens? Was it relevant?

67

Livestock Support What % of village households received livestock package? Was it appropriate to the household’s requirement and the local context? Was the supplementary feed ample and appropriate? Was this assistance timely to allow the beneficiaries to sow fodder on time? Were goat fairs and voucher system appropriate? Was any training provided to livestock beneficiaries? Was it relevant? Hygiene Kits & Hygiene Promotion What % of the village households received hygiene kits? Were these relevant to the immediate needs? Were these kits and their contents appropriate to the requirements of a household? Were there any hygiene promotion sessions conducted? Were they relevant? Safe & Sufficient Water What % of the village households have increased access to water through ACF’s assisted water supply

schemes, hand pumps or wells? Is this assistance appropriate to the immediate needs of the village? If not then why? Is the quantity and quality of water made available satisfactory? If not then why? Who provided technical assistance in designing? What process was followed to implement these small scale schemes in your village? What was community’s contribution? Household Latrines What % of the village households have been provided with household latrines? Are these relevant to the immediate needs of the beneficiary household? If not then why? Are these latrines and hand WASHing facilities appropriate to the local context? If not then why? Waste Management Systems What % of the village households have access to waste management system set up in this village? Is it relevant to the immediate needs of the beneficiary household? If not then why? Rehabilitation of small scale irrigation canals What % of the village households have access to water for irrigating their crops through the rehabilitation of

canals under this project? Is it relevant to the immediate needs of the village farmers? If not then why? Who provided technical assistance in designing? What process was followed to implement these small scale schemes in your village? What was community’s contribution? DRM Was participatory risk assessment, response planning and capacity building on DRR relevant to the

community needs? If not then why? Was this part of the project appropriate to the local context? If not then why? Are you aware of the village DRR plan? What is the role of the community in this plan? Was this overall assistance according to the needs? Was this a timely assistance? Beneficiary Targeting, Community Participation, Quality & Accountability Are you aware of any selection criteria developed for beneficiary selection? If yes, can you provide some of

the basic details? Was it applied? If not then how was the selection done? Were the most vulnerable (Disabled, Poorest, Widows, Orphans, elderly people) targeted appropriately? If

not then why? Who selected the beneficiaries in your village? Was the community involved in beneficiary identification and selection? Were women involved in identification and selection of the female beneficiaries? Are you satisfied with their participation? If not then why? Are you satisfied with the process adopted with beneficiary targeting? If not why and what would you

recommend for future? How was Livelihood Recovery Committee formed? What was the role of LRC / WUC in implementation of this project? Did LRC / WUC represent the entire village? Who decided that the assistance provided was community’s priority need?

68

What was the process for community input into decision making regarding need and assistance prioritization? What was women’s involvement in prioritization? Were any considerations made to benefit vulnerable groups?

Were any vulnerable individuals in the village involved in prioritizing community needs and type of assistance?

Are you / other beneficiaries satisfied with the performance of LRCs / WUCs? Was there any manipulation in selection of beneficiaries or favoritism noticed?

Were any operation & maintenance trainings provided to the community? Is maintenance of community infrastructure schemes (restored under this project) being done? If yes, who is doing the maintenance?

Do you think LRC / WUC will take ownership and responsibility for regular maintenance of these schemes in future?

How was your community informed about this ACF implemented project? Did the project team provide details of assistance to the community, prior to its implementation? Were there any conflicts raised during project implementation? Do you know of any complaints by the community regarding project implementation? If yes, were they

raised? If yes, with whom? Was any action taken? Was it satisfactory? Effectiveness, Immediate Impact & Sustainability What in your opinion worked very well within this project and what should change about it and why? Are you/other beneficiaries satisfied with the assistance provided? If not why and what would you

recommend for future? Has there been any immediate impact from this project? income through CFW / UCG income through cultivation using agriculture inputs improvement in diet through livestock and kitchen gardening inputs improved crops in case of restoration of irrigation channels reduction in water borne diseases through safe water and improved sanitation Did this project create any difference in reducing challenges to the affected populations in the post conflict /

flood context and allow return to normality? Has involvement in the project empowered women in the village? Did women utilize the money from unconditional cash grants, themselves or was it eventually in control of

males of the household? Do you think the LRC / WUC will continue working in the collective interests of the community? Did the project facilitate establishment of linkages between LRCs / WUCs and Government Line

Departments such as Agriculture Department, Civil Works, irrigation department, livestock department, while implementing this project? Will these continue to strengthen and benefit the community?

69

HOUSEHOLD SURVEY QUESTIONNAIRE

External Evaluation of ACF’s Humanitarian Support to Conflict and Flood-affected Populations in Pakistan

Write Code Please

Edited by

Checked by Verified By

Coded by

Entered by

A Identification

Codes

1 District

2 UC

3 Village

4 Reporter

5 Interview No.

1 No of HHs in This Village?

2 No of Ben HHs in This Village?

3 Date _______________

4 Name of Respondent

5 Age of the Respondent (years)

6 Sex of the Respondent Male 1 Female 2

7 Type of Beneficiary Safe Drinking Water..... .................................1 Household Latrines.......................................2 Hygiene Promotion........................................3 Hygiene Kits ..................................................4 Cash For Work..............................................5 Livestock Support..........................................6 Agriculture and Horticulture Support.............7 Capacity Building..........................................8

70

B Result 2 Beneficiary Households

B1 Income from Cash For Work / Unconditional Cash Grant

B1.1 No. of individuals from HH engaged in CFW

B1.2 No. of Days each HH received employment in CFW (Monthly)

B1.3 Daily Income Codes 1. 300 2. 350 3. Other (Please Specify)

B1.4 Monthly Income Codes 1. 7200 2. Other (Please Specify)

B1.5 How was payment received? Codes 1. Weekly Instalment in Cash 2. Full Cash at the end 3. Other (Please Specify)

B1.6 Are you satisfied with the benefit you received from CFW? Codes 1. Yes 2. No (If No, Please Specify)

B2 Cash Utilization

B2.1 % of the money earned from CFW utilized for purchasing basic food for the HH

B2.2 % of the money earned from CFW utilized for covering basic utilities of the HH

B2.3 % of the money earned from CFW utilized for Covering housing / shelter needs of the HH

B2.4 % of the money earned from CFW utilized for Covering basic health needs of the HH

B2.5 % of the money earned from CFW utilized for Covering Loan Repayment

B2.6 % of the money earned from CFW utilized for investment in productive assets for the HH

B2.7 % of the money earned from CFW utilized for Other Purpose (Please Specify)

71

B3A Household Dietary Diversity Score – What type of food the HH used to eat before receiving cash under CFW / UCG

Food Type B3A.1 Cereals (Corn, Rice, Roti, Biscuit, bread)

B3A.2 Roots & Tubers (Potato)

E3A.3 Vegetables

B3A.4 Fruits

B3A.5 Meat & Poultry

B3A.6 Eggs

B3A.7 Fish

B3A.8 (Lentils, beans)

B3A.9 Milk & Milk Products

B3A.10 Oil / fats (Oil, Butter, Ghee)

B3A.10 Sugar

B3A.10 Tea

Codes Yes=1 No=2

Food Source Codes Own Produce = 1 Purchase = 2 Food Aid = 3 Other = 4 (Specify)

Notes

72

B3A Household Dietary Diversity Score – What type of food the HH used to eat upon receiving cash under CFW / UCG

Food Type B3A.1 Cereals (Corn, Rice, Roti, Biscuit, bread)

B3A.2 Roots & Tubers (Potato)

E3A.3 Vegetables

B3A.4 Fruits

B3A.5 Meat & Poultry

B3A.6 Eggs

B3A.7 Fish

B3A.8 (Lentils, beans)

B3A.9 Milk & Milk Products

B3A.10 Oil / fats (Oil, Butter, Ghee)

B3A.10 Sugar

B3A.10 Tea

Codes Yes=1 No=2

Food Source Codes Own Produce = 1 Purchase = 2 Food Aid = 3 Other = 4 (Specify)

Notes

B4 Livestock Support

B4.1 Who received the livestock package within the household? Codes 1. Female 2. Male

B4.2 Are you satisfied with the breed? Codes 1. Yes 2. No (If No, Please Specify)

B4.3 Are you satisfied with its age and condition? Codes 1. Yes 2. No (If No, Please Specify)

B4.4 Did you receive livestock feed? Codes 1. Yes 2. No (If No, Please Specify)

B4.5 Did you receive fodder seed? Codes 1. Yes 2. No (If No, Please Specify)

B4.6 Were you able to grow fodder and was it enough to cover the needs? Codes 1. Yes 2. No (If No, Please Specify)

B4.7 Are you satisfied with the benefit you received from Livestock Support? Codes 1. Yes 2. No (Please Specify how)

Notes

73

B4 Livestock Support

B4.8 Did you receive any training on livestock management? Codes 1. Yes 2. No

B4.9 Has it resulted in any improvement in better yields from last year? Codes 1. Yes 2. No (Please Specify how)

Notes

B5 Agriculture / Horticulture Support

B5.1 Who received the Agriculture / Horticulture package within the household? Codes 1. Female 2. Male

B5.2 What is the size of your landholding? Codes 1. 5 Acres or more 2. Less than 5 Acres

B5.3 What crop inputs did you receive from ACF? Codes 1. 50 Kg DAP (1 bag) + 50 Kg Urea (2 bags) + 40-50 Kg Wheat Seed 2. Other (Please Specify)

B5.4 What vegetable inputs did you receive from ACF? Codes 1. Vegetable seeds, fertilizers and tools 2. Other (Please Specify)

B5.5 How much land were you able to sow with these inputs? Codes 1. 1 Acre (2 Jarib) 2. Other (Please Specify how)

B5.6 Were the crop / vegetable inputs enough in quantity? Codes 1. Yes 2. No (Please Specify how)

Notes

74

B5 Agriculture / Horticulture Support

B5.7 Were the crop / vegetable inputs provided timely? Codes 1. Yes 2. No (Please Specify how)

B5.8 Was your yield satisfactory? Codes 1. Yes 2. No (Please Specify how)

B5.9 Are you satisfied with the benefit you received from Agriculture Support? Codes 1. Yes 2. No (Please Specify how)

B5.10 Did you receive any training on crop management? Codes 1. Yes 2. No (If No, Please Specify)

B5.11 Has it resulted in any improvement in better yields from last year? Codes 1. Yes 2. No (If No, Please Specify)

Notes C Result 1 Beneficiary Households

C1 Provision of Water

C1.1 Has the project improved your access to safe and sufficient water? Codes 1. Yes 2. No (Please Specify how)

C1.2 Is the water colourless? Codes 1. Yes 2. No (If No, Please Specify)

C1.3 Is the water odourless? Codes 1. Yes 2. No (If No, Please Specify)

C1.4 Does water have bad taste? Codes 1. Yes 2. No (If No, Please Specify)

C1.5 Are you satisfied with the benefit you received from provision of clean water? Codes 1. Yes 2. No (Please Specify How)

Notes

75

C2 Provision of Household Latrines

C2.1 Has ACF provided you with a household latrine? Codes 1. Yes 2. No (If No, Please Specify)

C2.2 Is there a hand WASHing facility and a shower provided along with the latrine? Codes 1. Yes 2. No (If No, Please Specify)

C2.3 How many individuals are there in the HH who use the latrine? Codes 1. 20 or less 2. More than 20

C2.4 Are you satisfied with the benefit you received from provision of household latrine? Codes 1. Yes 2. No (Please Specify how)

Notes

C3 Hygiene Promotion

C3.1 Have you attended ACF led sessions on health risks and safe hygiene and sanitation practices? Codes 1. Yes 2. No (If No, Please Specify)

C3.2 Are you aware of the essential / critical times when to WASH hands? Codes 1. After Defecation, before having meals, after working in the fields, before cooking food 2. Don’t Know 3. Other (Please Specify)

C3.3 Are you aware of key elements of safe sanitation? Codes 1. Use latrines 2. Don’t Know 3. Other (Please Specify)

C3.4 Are you aware of critical ways to store water? Codes 1. Store water in a clean and covered container placed above the ground 2. Don’t Know 3. Other (Please Specify)

C3.5 Are you satisfied with the benefit you received from hygiene promotion sessions? Codes 1. Yes 2. No (If No, Please Specify)

Notes

76

C4 Hygiene Kits

C4.1 Has ACF provided you with container to store water safely? Codes 1. Yes 2. No (If No, Please Specify)

C4.2 Has ACF provided you with container to transport water safely? Codes 1. Yes 2. No (If No, Please Specify)

C4.3 Were you provided with a practical demonstration on how to WASH hands correctly? Codes 1. Yes 2. No (If No, Please Specify)

C4.4 Are you satisfied with the benefit you received from hygiene Kits? Codes 1. Yes 2. No (Please Specify How)

SSI Guide – ACF Staff

Performance Do you think the project has been able to meet the intended results/outputs? Why, if not? What facilitated / hindered (staffing, logistics, environment, govt. partners, local practices) ACF

Project performance (results / outputs)? Please share example/s? Did the external context evolve/change? What changes happened and how project management

systems/structures were adapted to external changes to ensure timely and effective delivery? What were the gaps, challenges and risks (security, logistics, govt. policy etc) for you and how were

those addressed? Share any evidence of innovative solutions? How does the normal PCM cycle (Planning and M&E) work for ACF? What M&E systems were put in place to ensure quality, integration and standardization across

program? What remote monitoring mechanism was established in case where technical experts or senior

management was unable to reach the target areas due to security or other reasons? What were the coordination mechanisms (between ACF field office and CO)? How much did

coordination facilitate/hinder the program/project delivery? What will you do different in future to avoid those gaps/challenges? What are ACF’s key lessons learnt with reference to achieving intended results? Any documentation /

experiences that you may share? Targeting How were target UCs and villages affected by the conflict / floods in 2010? Which agencies provided assistance in these target areas and of what kind? Was that adequate? How did ACF identify priority areas for this intervention? What criterion was established for geographic targeting, was that effective? What were the gaps, challenges and risk with geographic targeting (access, security, weather etc),

how were those addressed, please provide evidence? What were the key lessons learned with geographic targeting for future use? What was the population coverage for your intervention, what was the beneficiary make up in terms

of gender? What criterion was established to identify beneficiaries? What process was adopted for beneficiaries

identification/targeting, How effective was that? How did you integrate assistance to most vulnerable in beneficiary targeting, how effective was that

(women, elderly, people with disabilities etc)? What were the key challenges, gaps and risks with beneficiary targeting, how were those addressed

and how would you do it differently in future? Relevance, Appropriateness & Timeliness What were the priority needs of the conflict / flood 2010 affected people and how did ACF assess

those needs? How did ACF engage communities / beneficiaries in need identification, prioritization and tailoring of

assistance to conflict / flood affected populations? Was the response timely, appropriate and cost effective? If no then why? How special needs of most vulnerable groups were addressed in identification, prioritization, and

tailoring / contextualization (children, women, landless men, elderly, and persons with disabilities)? Support Systems How ACF CO supported FO in implementing this project? What operational systems were put in place by ACF to support its teams in ensuring a timely,

appropriate and cost effective intervention? How procurements were done and what mechanism was adopted to ensure delivery of quality goods

to the beneficiaries? Were there any gaps, challenges and risks with your systems and structures? What would you like to change/do differently in future?

Quality & Accountability, Connectedness & Coherence

78

In your opinion do you think ACF followed internationally agreed quality/compliance standards in Humanitarian Assistance (SPHERE, HAP, Code of Conduct, Aid Coordination etc)?

Are you aware of any guidelines / protocols developed by the Government to guide early recovery assistance in the target Districts? If there are any, has ACF in your opinion complied with them?

Were there any gaps in complying with above mentioned standards / protocols, how were those addressed? Are there any lessons learned for future compliance/adherence to quality standards?

How did ACF / IP ensure effective coordination with key stakeholders at various levels, from national to local?

What measures/mechanisms were in place for geographic coordination & collaboration to avoid duplication and resource maximization? How effective were those?

What in your opinion needs to be improved in coordination at various levels? To what extent did ACF have the capacity, systems and procedures, sufficient human resources and

appropriate level of preparedness to facilitate a rapid and appropriate response? To what extent was the ‘accountability to the beneficiaries’ promoted? To what extent did ACF follow up with complaints? Did the responses reduce future vulnerabilities? Did the short-term emergency activities take longer-

term issues into account? Immediate Impact & Sustainability How did ACF create a difference in peoples’ lives (direct & indirect beneficiaries) in terms of

immediate impact? Did ACF intervention (design and delivery) contribute to sustainability of results, how effective was it,

please provide evidence? If no then why? Have ACF been successful in fostering an effective working relationship with KP Government and

linking various Provincial & local Government actors / departments with the target communities? If no then why?

SSI Guide – External Stakeholders Performance Are you aware of KPHA Project implemented by ACF in your Province / District starting 2011? Were you involved in any way with this project? Do you think the project has been able to meet the intended results/outputs? Are you satisfied with ACF’s performance in this project? What has been your role in the designing and implementation of this project? Geographic Targeting Which UCs were severely affected by the conflict / flood 2010? How were target UCs and villages

affected? Which agencies provided assistance in these target areas and of what kind? Was that adequate,

relevant and timely? Are you aware of any mechanisms in place for geographic coordination & collaboration to avoid

duplication and resource maximization? How effective were those? How did ACF identify priority areas for this intervention? Did you have any role in this process? Do you have any recommendations for ACF related to geographic targeting in future? Relevance, Appropriateness & Timeliness What were the immediate needs of the communities in the post conflict / floods situation prior to this

project’s implementation? Was ACF’s assistance under this project according to community needs? Was this a timely

assistance? How did the KPHA project approach differ from other assistance delivered in target community? What were the advantages and disadvantages of such an approach? What in your opinion should change about the process / approach and why? Are you satisfied with the assistance provided? If not why and what would you recommend for future? Quality & Accountability

79

In your opinion do you think ACF followed internationally agreed quality/compliance standards in Humanitarian Assistance (SPHERE, HAP, Code of Conduct, Aid Coordination etc)?

Are you aware of any guidelines / protocols developed by the Government to guide humanitarian assistance in the target Districts / Agencies? If there are any, has ACF in your opinion complied with them?

Do you have any recommendations for ACF related to this? Did ACF coordinate with you during the course of this project? Did this coordination have any impact on the quality of the project? How effective this coordination was in terms of beneficiary targeting, programme design, avoiding

duplication, facilitation, problem solving etc.? What in your opinion needs to be improved by ACF in terms of coordination at various levels? Immediate Impact & Sustainability Did this assistance create any difference in reducing challenges to meet basic needs at household

and community level? SSI Guide – Male / Female Non-Beneficiaries Name of District / UC / Village: Total No. of HHs in the village: Total No. of Beneficiary HHs in the village: Name of Participant: Status of Participant – Non Beneficiary Geographic Targeting Which UCs were severely affected by the conflict / flood 2010? How and when was your UC and

village affected? Which agencies provided assistance in your village and what kind? Was it relevant? Relevance, Appropriateness & Timeliness What were the immediate unmet needs of the communities in the post conflict / flood situation prior to

this project’s implementation? What assistance was provided to your village by ACF under this Project? Was this overall assistance according to the needs? Was this a timely assistance? If not then why? Beneficiary Targeting, Community Participation, Quality & Accountability Who selected the beneficiaries in your village? Was the community involved in beneficiary

identification? Are you aware of any selection criteria developed for beneficiary selection? If yes, can you provide

some of the basic details? Was it applied? Why you were not selected? Did you get assistance from other agencies? What and how much? Were any vulnerable households (With Disabled, Poorest, Widows, Orphans, elderly people etc.)

selected as beneficiaries? Were any vulnerable individuals in the village involved in prioritizing community needs and type of assistance?

How was LRC / WUCs formed? Who selected the members of the LRC / WUCs? Are all members of LRC / WUCs direct beneficiaries of the project? What are the various groups of vulnerable people in your village? Does LRC / WUCs represent all these groups and other sections of the village? What was the role of LRC / WUCs in implementation of this project? Who decided that the assistance provided was community’s priority need? Was LRC / WUCs proper forums of community involvement? Any suggestion for improvement? Are you / other beneficiaries satisfied with the performance of LRC / WUCs? Was there any

manipulation in selection of beneficiaries or favoritism noticed? Were there any conflicts raised during project implementation? Do you know of any complaints by the community regarding project implementation? If yes, were they

raised? If yes, with whom? Was any action taken? Was it satisfactory? Immediate Impact & Sustainability Has there been any immediate impact from this project? income through CFW / UCG income through cultivation using agriculture inputs

80

improvement in diet through livestock and kitchen gardening inputs improved crops in case of restoration of irrigation channels reduction in water borne diseases through safe water and improved sanitation Did this project create any difference in reducing challenges to the affected populations in the post

conflict / flood context and allow return to normality? Do you think the LRC / WUC will continue working in the collective interests of the community?