2015

Patrick N. Kpanyen (Ph.D.)Germue M. Gbawoquiya (MSc)Joseph J. Gartor (BSc Candidate)

EXTERNAL EVALUATION OF CARITAS LIBERIA EBOLA

EMERGENCY Response PROJECT EA34/2014

ContentsLIST OF ABBREVIATIONS......................................................................................................1

1. ACKNOWLEDGEMENT....................................................................................................2

2. EXECUTIVE SUMMARY..................................................................................................3

Key findings of the Evaluation.................................................................................................4

3. INTRODUCTION................................................................................................................5

4. PROJECT CONTEXT.........................................................................................................6

5. METHODOLOGY...............................................................................................................8

6. ETHICAL CONSIDERATIONS.........................................................................................9

7. EVALUATION LIMITATIONS:........................................................................................9

8. PROJECT DESCRIPTION AND OVERVIEW..............................................................10

9. FINDINGS...........................................................................................................................13

9.1 Relevance.....................................................................................................................15

9.2 Effectiveness......................................................................................................................27

9.3 Efficiency...........................................................................................................................30

9.4 Impact................................................................................................................................32

9.5 Sustainability.....................................................................................................................33

10. CONCLUSIONS, LESSONS LEARNT AND RECOMMENDATIONS...................34

INFORMED CONSENT FORM...............................................................................................39

1

LIST OF ABBREVIATIONS

CG Caritas GbarngaJPC Justice and Peace CommissionCORDAID Catholic Organization for Relief and Development Aid (Caritas Netherlands)CI Caritas InternationalisCL Caritas LiberiaCABICOL Catholic Bishops Conference of LiberiaEVD Ebola Virus DiseaseWHO World Health OrganizationEA Emergency AppealCRS Catholic Relief ServicesCAFOD Catholic Agency for Overseas Development (Caritas England and Wales)ETU Ebola Treatment UnitSS Sample SizePA Proportional AllocationSPSS Statistical Package for Social SciencesNFI Non-Food ItemsSit-Rep Situational ReportTOT Training of TrainersTOV Training of Volunteers

2

1. ACKNOWLEDGEMENT

This project demanded huge efforts, work and dedication. Still, implementation would not have been possible if Caritas Liberia (CL) did not have the support of many individuals and organizations. Therefore we would like to extend our sincere gratitude to all of them.

First, we are thankful to all of the International Caritas through Caritas Internationals (CI) as a body for their financial and logistical support, and for providing necessary guidance concerning the project implementation. We cannot forget the effort of Mr. Andre Stelder, seconded by Cordaid to CL, who, from the beginning of the project exerted energy to ensure that most tasks were undertaken.

We are also grateful to our local Caritas Gbarnga (CB) and Monrovia and also the Catholic Justice and Peace Commission (JPC), the Mother Pattern College of Health Sciences for successful implementation of the project activities. We acknowledge the work of the Financial Auditors and External Evaluation Team for provision of expertise, and professionalism in the work done. Without their superior knowledge and experience, the Project would lack in quality of outcomes, and thus their support was essential. We would like to express our sincere thanks to all project volunteers in both dioceses who devoted their time, knowledge and skills in the implementation of this project. We cannot forget to express our deepest appreciation to all those who provided us the possibility to complete this report.  A special gratitude to all of our project staff, Caritas Liberia (CL), whose contribution in stimulating suggestions and encouragement, helped us to coordinate our project especially writing this report.

Furthermore, we would also like to acknowledge with much appreciation the crucial role of CABICOL (Catholic Bishops Conference of Liberia), who are our host and overall guardian and also Caritas Cape Palmas who were not implementers, but opened their doors for coordination and support.

Last but not least, many thanks go to the head of the project, Rev. Fr. Dr. Patrick M. Kaba, who invested so much effort in guiding the team to achieve the project goal. Nevertheless, we express our gratitude to our families and colleagues for their kind co-operation and encouragement which helped us with the implementation and completion of this project.

Kindest regards,

Willet Lady Salue, (MPH)Project Manager

3

2. EXECUTIVE SUMMARY

The Emergency Appeal project (EA-34) was drafted in September 2014 and launched in October 2014 when the Ebola Virus Disease (EVD) outbreak was at its peak in Liberia with Caritas Liberia as national implementers. The program was later revised as the outbreak plummets in subsequent months and focus shifted from case management to concentrating on support to survivors, orphans and livelihood recovery to people made vulnerable by the disease. As the disease exploded through the country with a very high death toll in the country, the psychosocial component of the project became obvious and very essential. The program placed emphasis on the monitoring and advocacy approach to deliverables from national Government and other interested actors. It was implemented in all ten counties within the Archdiocese of Monrovia and the Diocese of Gbarnga with a cumulative population of 2.2 million according to the last Liberia National Population and Housing Census 2008, more than half (63 percent) of Liberia’s entire population. The Catholic Justice and Peace Commission (JPC) provided training in various communities in human rights monitoring along with other forms of violations. The Mother Pattern College of Health Sciences provided the psychosocial Counseling and trauma healing component using Training of Volunteers approach to impact the communities. One of the major components of the project was to ensure a final external evaluation to measure the gains made by the project to assess its relevance, efficiency, effectiveness, impact and sustainability. The external evaluation of the EA34/2014 project took all together three weeks commencing mid-October to early November with equal time extension at no cost and a consultant hired. The project itself was implemented for six months commencing January with two months no cost extension. The major objectives for the project were to:

Assess the extent to which the project achieved its purpose and delivered on intended results or outputs.

Draw lessons learnt, good practice and make appropriate recommendations to inform future program designs and implementation practices.

Key findings of the Evaluation

The data gathered from the field exercise indicate that nearly all beneficiaries who received aid from the program declared its relevance and value added to their livelihood. The items were distributed based on case densities and vulnerability criteria. The Incidence Management System (IMS) of the Ministry of Health provided daily situation reports (Sit reps) and had regular information sharing at its central office 9:00Am every morning for which Caritas Liberia was a part. The briefing from MOH helped CL staff to locate the most vulnerable populations. Responses from the Hygiene Kits distribution showed that 94.1% of the beneficiaries said that they were satisfied and that the hygiene kits were useful to them during the outbreak. In a series of focus group discussions, all beneficiaries declared that the provision of the assorted food items was relevant and help improved their livelihood and to comply with

4

the restrictions from the Ministry of Health during at the time. Nearly all beneficiaries who received cash transfer benefits declared that the cash transfer helped them with livelihood recovery and appreciated CL and its partners for the assistance. All farming beneficiaries who received farming inputs from CL admitted that they were very pleased and happy.All Households hosting orphans and Survivals of EVD declared that the school fees and school kits assistance to the orphans was awesome and the kits was able to help the children to cope with their loss and look forward to live in a positive direction.Human Rights and Psychosocial responses were not very sharp during the field interviews. Discussants could barely recall these events/interventions.

Several key recommendations were made including the following themes: Human and Infrastructure Capacity building for CL staff to be better prepared for

future programs and to serve the people of Liberia in a trusted and reliable way. Develop and/ or improve Policies that nurtures transparency and use resource

wisely. Acquisition of an office for CL with a warehouse and packing lot for vehicles to

safe guard the organization’s property from criminals and for work efficiently. Support decentralization and decision making at local levels to promote

ownership and motivate local staff. CL through its EA 34 project built vulnerable women business capacities by

providing cash transfer that helped them engaged in the establishment of small businesses at village levels to provide access to immediate needs of the locals. This helped them mitigate poverty and further improve the livelihood of their families and the communities.

CL and its development partners prioritize and continue with the provision of agriculture inputs such as tools and seeds to vulnerable households that will take off beyond the EA-34 Project to enhance national food security but with expert advice on procurement for quality assurance.

The Government of Liberia, the church through CL, donors and development partners promote social protection and welfare programs aimed at supporting orphans and others made vulnerable by Ebola given the level of stigma attached to the disease. The activities should aim at sustaining and expanding the interventions based on the results and lessons learnt from the EA-34 project.

CL has gotten its first experience in emergency program implementation and is well respected for that in the country. It received support and funding from various partners to respond to the Ebola Virus Disease after careful needs assessment and partnering with the Ministry of Health and the local communities amongst to help vulnerable people. The expert skills and knowledge acquired from implementing the EA-34 project under

5

difficult circumstances is a testimony to the staff’s devotion and passion to serve humanity.

3. INTRODUCTION

The external Evaluation of EA-34 project implemented by CL used the five most prominent and widely adopted evaluation criteria developed by the Development Assistance Committee of the Economic Cooperation and Development (OECD/DAC) standards to analyze major outputs from the project at all levels of interventions. The five criteria used to conduct the external evaluation looked at the relevance, effectiveness, efficiency, impact and sustainability based on OECD/DAC guidelines.

The external evaluation took all together three weeks of field activities which commenced in Mid-October to early November, 2015. The exercise utilized mixed combination of qualitative and quantitative methods for data analysis to meet the objectives of the evaluation. A structured questionnaire was developed and administered to different target groups at each level of intervention based on sample size proportionate to size of the population. In addition to that, the team of evaluators conducted four key informant interview (KII) with Caritas Monrovia and Gbarnga offices involving five key informants from each diocese that involve five males and one females. A total of sixteen focus group discussions were held with local church groups (parishioners) and at the community level with thirty three males and twenty five females in attendance from both dioceses. This also involve field visit to schools where Caritas sponsor Ebola orphans are currently enrolled and supported.

The evaluation report was prepared using various instruments to measure output from each intervention and integrated SPHERE and relevant best practice for humanitarian assistance to analyze the data, present findings and draw from lessons learnt to prepare make recommendations for future programs designs. With the vision of the organization and the core values of the church which are to respect the sanctity of human life from conception to death and to provide preferential options for the poor and vulnerable, the project interventions were guided by Humanitarian codes of conduct and best practices.

Caritas Liberia, the helping hand of the church reaching out to the poor and vulnerable people in Liberia realized that the awful Ebola situation was a serious challenge to the Country and its citizens. They swiftly join in the battle against Ebola to meet the physical, mental and Spiritual needs for the affected families, communities and vulnerable poor.The project implementation initially faced numerous challenges ranging from human capacity to logistics. The internal evaluation also experience some of these

6

difficulties when schedules were changed due to some administrative and logistics problems.

4. PROJECT CONTEXT

The Ebola Virus Disease (EVD) outbreak which hit the West African Mano River Union Basin countries of Guinea, Sierra Leone, and Liberia in December, 2013 has been the most devastating outbreak in the history of the viral hemorrhagic disease. The weird manner in which the disease entered and surfaced in Guinea(a neighboring country to Liberia) for the first time in the history of West Africa, and with little known about it, the in experience of health authorities, poor health care facilities and services in the three countries provided an opportunity for the disease to thrive, snatching away precious lives and placing grave financial burden on the already staggering economy of three of the world’s least developed and poorest countries.

The onset of the outbreak was deceivingly slow. The Ebola Virus Disease entered Liberia (index case) from Guinea to Lofa County, a local and border trade community with Guinea occupied predominantly by the Liberia minority Muslim tribal group, the Mandingoes, on March 30, 2014. By early April, five more cases were confirmed but all died and the situation stabilized with no further cases during the latter part of April and most part of May. The Government of Liberia’s governing authorities including the Ministry of Health did not really understand what it meant to report a case of Ebola in the country and to pay keen attention to the situation. There was a complete political misalignment and counter accusations between the Ministry of Health Authorities and other important state actors and decision makers across government. On one hand, the virus played its usual viral time role; while on the other hand, the officials of the government were disenchanted over the situation and politicized an outbreak that could have been well managed from the onset thereby plunging the entire country into a humanitarian emergency situation in subsequent months. The disease was so virulent that took with it many lives in its wake. Liberia reported the highest number of death in what is known to be the longest and complex outbreak since the emergence of Ebola Virus Disease (EVD) in 1976 according to Public Health experts. In August and September, 2014, the country was reporting from 300 to 400 new cases everyday according to the Ministry of Health situation report (Sit rep, 2015). These months known by locals as the doom days for Liberia witnessed tragic scenes, the gates to treatment centers overflowed and were shot down, makeshift structures erected with no more capacity, bodies seeing lying on hospital grounds or sometimes at home with families and not collected for several days ensuring the infection. Flights were suspended, Fuel and food ran record low, and Schools, businesses, borders, markets closed. Though the capital city was hardest hit, every one of Liberia’s 15 counties eventually reported cases. At one point,

7

virtually no treatment beds for Ebola patients available anywhere in the country. Local volunteers worked in treatment centers, on burial teams, or as ambulance drivers driven by a sense of community responsibility and patriotic duty to end Ebola and bring hope back to the people. As the number of cases grew exponentially, international assistance began to pour in. All these efforts helped push the number of cases down to zero. The impression of a calm situation turned out to be an illusion. The first additional cases in Monrovia were reported and the city was ill-prepared to cope with the onslaught of infections that rapidly followed as the virus raged havoc everywhere. On 6 August, President Ellen Johnson Sirleaf declared a three-month state of emergency and announced several strict measures aimed at getting cases down. In September, WHO began construction of a new treatment centers, using teams of 100 construction workers laboring in round-the-clock shifts. On 21 September, the Island Clinic was formally handed over by WHO to Liberia’s Ministry of Health. The clinic added 150 beds to Monrovia’s limited treatment capacity. However, within 24 hours after opening, the clinic was overflowing with patients, demonstrating the desperate need for more treatment beds. WHO supported the construction of 2 additional Ebola treatment centers, augmenting Monrovia’s treatment capacity by another 400 beds. The remaining need was eventually met by multiple partners. The rapid increase in treatment capacity, especially in Monrovia, likely did much to turn the outbreak around. The outbreak began to subside in late October, when more new cases were detected early and rapidly treated in isolation, and more safe and dignified burials were performed. Case-fatality rates dropped as the number of survivors grew; public perceptions changed from viewing treatments centers as “death traps” to seeing them as places of hope. That altered perception, in turn, encouraged more patients to seek early treatment.The Ebola Virus Disease outbreak had severe health, socio-economic encounters on every fabric of the Liberian society in an undesirable way, thereby, disrupting the development progress achieved since the restoration of peace and democracy in 2003 to a post war country with the help of the International Community.

5. METHODOLOGY

The external evaluation for the Emergency Appeal (EA-34) Project fully utilized a mixed-method tactic, combining both quantitative and qualitative research techniques in order to meet the objectives of the evaluation. Secondary data were collected through a review of related literature (distribution logs, school registries, project mid-term evaluation report amongst others) to give better context to the primary data gathered from the field exercise. The field team headed by the lead consultant also employed direct observation methods to get first-hand information and to develop some clues. The sample size determination was done using the Taro Yamane formula for sample size calculation based on proportion assuming a sampling error level of 5% and 95% confidence. The

8

number of subjects selected at each level of the intervention is large to generate enough statistical powerFurthermore, the review used a structured survey tool, Focus Group Discussions, Key Informant Interview, statistical analysis of the CL database and desk review of key documentation to collect and triangulate the information. Data were entered into SPSS and analyzed, graphs were generated and edited in Microsoft excel.

The external evaluation was carried out in Monrovia and Gbarnga Dioceses of the Catholic Church of Liberia. The Diocese of Monrovia, the oldest of the three Catholic dioceses in Liberia has seven counties (Montserrado, Bomi, Cape Mount, Grand Bassa, Rivercess, Margibi and Gbarpolu) and is the most populated. Gbarnga Diocese, the last and newest of the three dioceses has only three counties (Bong, Nimba and Lofa).Of the ten counties that benefited from the Emergency Appeal Project in both dioceses; two counties (Gbarpolu and Lofa) were randomly excluded from the evaluation leaving eight through a sample random sampling technique without bias and replacement. This was done to ensure that each county as a unit had equal and independent chance of being selected to form part of the review since the time factor could not allow the evaluators evaluate all the counties. A combined cluster and stratified sampling techniques were deployed to ensure that sub-groups were proportionately represented to account for the difference in group dynamics and that the sample was also representative of the parent population. The desire sample for each of the intervention in each county differs since the quantity of items distributed wasn’t unique and the items were also different(Hygiene kits, school fees and school kits, food supply, provision of cash and seed rice, NFI support to hardship survivors and human right training by the Catholic Justice and Peace Commission, psycho-social counseling).

6. ETHICAL CONSIDERATIONS

The research protocol was reviewed and approved by the appropriate research authorities in country (UL-PIR). The evaluation posed no known risk to participants since it was nothing scientific requiring introducing interferences to research subjects. Yet, all participants were duly informed of the process and the objective for the evaluation explained. The issues with confidentiality were adhered to straightly. The junior evaluators were trained to introduce themselves and to divulge their identities and the agency they represent, and to speak in clear tone that the respondents would understand without hindrance. The respondents were assured that enrollment in the external evaluation was voluntary and that they had the right to withdraw their consent at any phase of the research without suffering any penalty. They were encouraged to sign or thumb print the inform consent form as confirmation of their voluntary enrollment in the study.

7. EVALUATION LIMITATIONS:

9

While the evaluation process was planned in great details, there were also some process-related challenges which may have in some respects affected the outcome and quality of the findings. The major limitations were as follows:

The time allotted for the evaluation was not enough to cover as many respondents as possible due to the bad road conditions and logistical constraints.

Given that different communities have different market days and farming activities, it was not possible to meet some of the respondents at home because they might have either gone on their farms or left for the market to either buy or sell their commodities

The evaluation did not take place as scheduled because there was so much pressure on the field vehicles used by Caritas in between activities

Some of project beneficiaries especially those affected by the Ebola Virus diseases either died or move away to unknown location due to Ebola related stigma

8. PROJECT DESCRIPTION AND OVERVIEW

The Emergency Appeal (EA) was drafted in September 2014, with the intention to start implementation on October 1, 2014 but adjusted due to delays in partners funding and capacity challenges. The project also had a two month extension without cost. The total project funding was 1,174,054.21 USD. However, with Caritas Liberia operating on a skeleton staff due to lack of core funding and ongoing projects, there was serious concern about capacity to run such a large programme. More to that, except for a small soon to be closed down office of the Catholic Relief Services (CRS), there were no Caritas members established in Liberia which could assist Caritas Liberia implement the project. Caritas Internationalis (CI) suggested that an Emergency Accompanier would be necessary to build Caritas Liberia’s capacity to deal with the situation. Caritas Netherlands (Cordaid) was found willing to support Caritas Liberia and to recruit and send one Emergency Accompanier and possibly one Finance Manager to Liberia. However, processes took their course and time went by, the Emergency Accompanier was recruited and appointed on 15 December, and after a short induction, arrived in Liberia on 25 December 2014. Caritas Liberia had not yet started implementation awaiting the arrival of the Emergency Accompanier and the first transfer of funds. The Emergency Accompanier’s arrival during the Christmas break when all Caritas offices were closed until 5 January, added to the delay.

Meanwhile, Caritas Cape Palmas received direct support from Caritas Germany. Although the five counties of the Diocese (Maryland, Grand Kru, Sinoe, River Gee and Grand Gedeh) had relatively few Ebola cases to report, community mobilization has taken place and has hopefully contributed to more than twenty one days of no new cases

10

reported at the close of December, 2014. The Diocese of Cape Palmas was therefore dropped from the Emergency Appeal. The Diocese of Gbarnga, with three counties (Bong, Nimba and Lofa) has been supported by Caritas England and Wales (CAFOD) to provide food support to quarantined households until 31 December 2014. CAFOD later joined and pledged its support to the Emergency Appeal Project.

The most important change in the original concept document was the reduction of number of households in quarantine which needed food support. As there were only few households and communities left under quarantine, the quantities of food supplies drastically reduce in the budget from 3000 households in the original plan, down to 400 households in the new one over six months. What comes in the place of the food support would be recovery of livelihoods. Families in affected communities have not been able to plant or to harvest the last farming season. They were scared to get together in kuu- groups (traditional agriculture in Liberia works with kuu-groups which work the land together, from clearing, scratching, sowing, weeding, and harvesting). Because of Ebola Outbreak, people were afraid to come together. Crops remained on the field unharvested as farmers have not been able to get a kuu-group to harvest despite many people being out of work. Families have eaten their seed rice reserves and impossible to resume the agricultural cycle.Non-farming households received cash transfer to resume or scale up their income generating activity. Survivors often return to burnt out houses with no assets left, due to communities fire-cleansing of Ebola infected areas. For hardship cases, cash transfer was provided to buy clothes, cooking utensils, mattresses amongst others to start up again.Community mobilisers identified orphans and tried to accommodate them in their own communities with relatives or neighbors as this was preferable to institutionalization. In order to assist host families of orphans to deal with the extra expenses in this harsh economic situation, a package of support was offered:

A system was put in place to ensure funds were used for livelihood recovery and not for other urgent expenses (accounting, visual inspection). The Emergency appeal Project was aimed at assisting the needy and vulnerable households during the Ebola Virus Disease Outbreak to cope with life and avoid being affected. The original logical frame work is therefore mimicked in the below matrix for ease of reference.

Goal To contribute to improving health and wellbeing of vulnerable

11

populations in LiberiaEA To stop transmission of Ebola virus in affected and at risk counties by

scaling up effective community-based outbreak control measures

Activities - Training of Trainers (TOT) for community mobilisers- Parishes , participants and Diocese trained on Ebola

prevention - Focus Group Discussion held for community leaders & health

workers- Parishes/outstations & Dioceses conducted sensitization

during Sunday masses- Hygiene kits distributed to households

Results - Training of Trainers (TOT) were conducted for 101 (36 females and 65 males) community mobilizers. Target 100%

- 12 parishes x 100 participants x 2 Dioceses trained on Ebola prevention (585 females and 854 males). Target 100% achieved.

- 30 focus discussions x 2 dioceses for community leaders and health workers (1,244 females and 1051 males). Target 100% achieved

- 12 parishes/Outstations x 2 Dioceses conducted sensitization during Sunday Masses over a period 10 weeks achieved

- Hygiene Kits distributed to 3,165 households (1,814 females and 1351 males). Target achieved 98%

Data collection Instrument

Document and desk-based reviewStructured questionnaire for client stratificationFocus discussion and home visitDirector observation techniqueMeetings with key informantsMeeting and interview with CL staff in Monrovia and Gbarnga Dioceses

Output 1.2 Ebola survivors, affected families and orphans in priority communities are better able to deal with their situation after psychosocial and trauma counseling per month.

Activities Sessions were held to identify survivors, affected families and orphans and their communities for psychosocial and trauma counseling per month.

Results 60 sessions to identified survivors, affected families and orphans and

12

their communities for psycho social and trauma counseling per month.

Data collection Instrument

Structured questionnaire for client satisfactionFocus discussion and home visitsDirect observation techniques to see evidenceMeetings with key informantsMeetings and interviews with CL staff in Monrovia and Gbarnga Dioceses

Output 1.3 Vulnerable affected communities, quarantined families and families hosting orphans have mitigated the quarantine period and its aftermath and are on their way to recover their livelihoods

Activities - Distributed basic Food package to affected vulnerable households, survivors or orphan hosting communities

- Provision of farming packages (seeds & tools) or cash transfer to families hosting orphans

Results Distributed food package to 480 affected vulnerable households, survivors or orphan hosting families (293 female households & 187 male households). Target achieved 120%Provided farming packages (seeds & tools) and cash transfer to 400 families hosting orphans. Target achieved 133%Provided school fees & school kits to 546 orphans. Target 121% achieved Provided farming packages (seeds & tools) to 595 affected vulnerable households to resume farming. Target 85% achieved Provided cash transfer to 495 non-farming vulnerable households to resume income generation activities. Provided NFI support to 92 hardship cases of survivors. Target achieved 123%

Data Collection Instrument

Document/ desk-based reviewStructured questionnaire for client stratificationFocus discussion and home visitDirector observation techniqueMeetings with key informantsMeeting and interview with CL staff in Monrovia and Gbarnga Dioceses

9. FINDINGS

13

The evaluation is considered a learning opportunity for Caritas Liberia and its partners involved in the implementation of the (EA-34/2014) project so as to attribute to improving performance, quality, accountability, sustainability, empowerment, capacity building and emergency preparedness to handle re-emergence of the Ebola Virus Disease or related disaster.The objectives of the Evaluation were in two folds;

To assess the extent to which the project achieve its purpose and delivered on intended results/outputs

To drew from the result lessons learnt, best practice and to make future recommendations to Caritas and partners to inform future program design and implementation.

The objectives set in the evaluation employed the ideologies of the OECD/DAC evaluation Criteria considering the relevance, the adequacy of the project versus the need of the people at the time, its efficiency, effectiveness, sustainability, and impact, and to verify to what extent humanitarian best practice us (SPHERE) standards were met. SPHERE standards envision a world in which all people affected by disaster or conflict are able to re-establish their lives and recover their livelihood in ways that respects and promote their dignity as humans. The objectives of the EA-34 Project were put into set of activities and out puts to achieve them. Due to several adjustments made during the roll out of the activities, some results exceeded the outputs while others struggle to meet the set bench marks or did not meet them due to changing circumstances at the time.

Several needy persons, who were not envisioned by CL showed up during the distribution and pleaded with CL to assist them. CL responded by making some adjustment in the plan to respond to their request.

Significant pressure was put on CL staff working with volunteers due to increasing demand from affected populations as situation depicted. Logistics and personnel were not adequate to reach intended targets as roads that had not been used over a period or during the rainy season presented bottlenecks.

Activities were coordinated with other agencies and organizations under the Ministry of Health incidence management system providing similar assistance mostly in urban and accessible areas thereby making it difficult for respondents to recall activities related to each organization.

The EA-34 Project was designed to respond to the Humanitarian crisis with focus on victims and vulnerable populations. Activities and assistance were elaborated in accordance to SPHERE standards in a manner that is appropriate, fair and transparent based on the core values of the church. The evaluation should help CL as an organization to make improvement and upscale knowledge and skills for emergency preparedness and response. Since this project was the first of its kind

14

for CL and as needs and operating context changed overtime, CL should build upon what they have done to develop and improve their internal capacities. The findings from the evaluation should therefor help to provide the needed evidence as a basis for transforming CL.The findings presented below were generated from the structured interview with direct beneficiaries, key informants, Focus Group Discussions, documents review and Analysis, and direct observation methods from the field exercise.

9.1 Relevance

The relevance of the project looks at its suitability to the priorities and policies of the recipients. The evaluation looked into knowing if the Project was in line with the local needs and condition of the affected people at the time. To determine the germaneness of the interventions, the following questions were asked to guide the evaluation.

How important were the project activities and services to the beneficiaries? What do the beneficiaries think of the project? Its relevance, appropriateness and outcomes? Are the outcomes sustainable?

What do other primary and secondary stakeholders (e.g. staff, parish officials, local government officials, others) think of the project? Were the most vulnerable reached? Was the targeting appropriate?

Were appropriate systems of downwards accountability (participation, information sharing and feedback), put in place and used by project participants? Did the feedback received shape project design and implementation?

Were the initial assessments of a good quality and based on strong beneficiary participation?

How was gender taken into account throughout the programme?

This project was the first of its kind to be implemented by CL with limited staff capacity and logistical constraints. The basic premise for the project was the distribution of several items (Hygiene kits, food items agriculture inputs, NFI support, cash, school fees and school kits) to Ebola Victims and vulnerable population in the Archdiocese of Monrovia and Gbarnga with the goal to improve their health and well-being. During the civil conflict especially in the nineties, Liberia’s unemployment rate was 85% according to the Labour Force survey results. In 2010, The Liberia Institute of Statistics and Geo-Information Services (LISGIS) conducted the Labour force survey and put the unemployment rate at 3.7% but with most people about (68%) falling into vulnerable employment category. This means that a huge proportion of Liberians were own-account workers from petty businesses, subsistence agriculture workers, palm wine tappers, charcoal burners, wheel barrow riders amongst others. Most of these activities were

15

slightly tilted to the rural areas were a bulk of those in vulnerable employment work on their farms without any significant gender difference. Many of these activities that were the livelihood support to these individuals, households and their families were halted during the Ebola Virus Disease outbreak for fear and safety, and at some point government restriction. The movement of basis life commodities from urban to rural and vice versa was interrupted or put to an abrupt end thereby intensifying the hardship on the affected communities in the two dioceses. The provision of safe drinking water and basic sanitation is worse in the rural areas. The evaluation took to a local affected community in the Diocese of Monrovia were the inhabitants live on muddy creek water without any form of water treatment or sanitary measure seen below. They admitted drinking from that creek water during the outbreak and still getting water from there on a daily basis.

16

SOURCE OF DRINKING WATER FOR A LOCAL COMMUNITY OBSERVED IN THE DIOCESE OF MONROVIA DURING THE EVALUATION.

The Project intervention took more than half of the country as seen in the map below. Only the five counties of the Diocese of Cape Palmas (Maryland, Grand Kru, Sinoe,

River Gee, and Grand Gedeh) where the outbreak was not intense were not covered. However, Caritas Germany along with the Government of Liberia and other partners intervene and massive community mobilization took place hopefully contributing to the success of preventing Ebola in that part of the country. This based EA-34 project intervention map and corresponding population is therefore placed in this work to show the Geo-graphic areas covered by the project.Findings from the Hygiene kits which contains (4/5 liter of bleach, 3-4 bars of soap, one budget with faucets, and one small bucket for collecting waste water) and other materials revealed the relevance of the project without doubt. A representative sample was selected containing 337 household heads that received the hygiene kits and were interviewed using a structure questionnaire and triangulated with focus Group Discussions and key informant interviews. A graphic presentation of the hygiene kits by counties is shown below:

17

Figure 1: Hygiene Kits distribution by County

Grand Bassa Bomi Bong Cape Mount Margibi Montserrado Nimba Rivercess

11%

14.8%

19.3%

14.8% 14.5%

8%

14.2%

3.3%

The rise and fall in the graph indicate the case densities for the distribution signifying that counties that reported numerous cases had enough victims and vulnerable population. However, according to client satisfaction using the structured questionnaire, nearly all 98.2% of the household heads (respondents) admitted that they received and used the hygiene kits contents as they were taught by the project implementers and 94% said that it was very useful in helping them to prevent the virus disease. However, for the gender dimension, 55% of household heads who received the hygiene kits were females compared with 45% of males who were house hold heads. All the respondents also declared that all contents listed in the hygiene kits were received and they were satisfied with the quality. This assertion was further crossed checked during the focal group discussion were a local male respondent refer to the distribution as “very timely and God’s divine Intervention. He continues,” my son, we were not doing anything. Ebola kept us home and we are not doing anything. These things these people brought, we have to walk distance to the cities before we can get them, only God can repay them”. In addition to that another female elderly woman in the Catholic diocese of Gbanga said that “We have given up hope of receiving anything from the Government and its partners because long distance and bad road conditions separated them from us, I am very surprised and even the food we eat doesn’t reach in our stomach due to fear of the unknown, I give praises to God and the field teams from CL, they are our national heroes”. Altogether, 16 focal group discussions were held with local government groups, parish leadership, health workers, school groups and nearly all of them said that the project interventions including the hygiene kits distribution address their needs and priorities at the time. As presented in the picture that showed the source of drinking water for a rural community in the Diocese of Monrovia, it was observed by the evaluation team that most of the villages were without a source of safe water for drinking and other

18

usage, but Hygiene buckets were still seen in use without soap and detergent but plain unsafe stream water.According to the focus group discussions conducted across both Dioceses, an overwhelming majority (>85%) said that they benefited from important health messages from their parishes through their pastors, community leaders and other community volunteers on Ebola prevention and control measures. However, more than half could remember some of what they were taught during the evaluation. Such as wash hands with soap and water as often as possible, watch out for signs and symptoms of Ebola (headache, fever, red eyes, body pains, bleeding, sore throat and others), do not share things with sick persons and early treatment at the hospital. All of them declared that the information provided them by people who provided it helped them out to avoid contracting the virus and the teachings were simple and practical.

Assorted material (Food support)

The provision of assorted food items (package) containing (one 50kg bag of rice, one 50kg bag of beans, five gallons of oil, one pack of salt, one pack vita cubes and 10 cans of sardine fish) in the frame of the EA-34 came at a crucial time when most working families, though in vulnerable or own account employment, were out of jobs due to the outbreak and without any income. Most families who could work out their farms and get food for their households were unable to do so.Merger savings or what is known in Liberia as hand to mouth savings were rapidly used up and proved insufficient to provide food and other basic household commodities for the length of time stayed home.All beneficiaries interviewed during the focus group discussion were united in their voices that the food assistance was much appreciated and helped them a lot during their trouble times to adhere to the safety regulations provided by government and CL and other partners. The below graph showed the geographic distribution of basic food items. In both Dioceses, counties that were hard hit received more items than the other counties. However, in the structured questionnaire, only 57.8% of the beneficiaries said that the food items were enough for the time.

19

Figure 2: Distribution of Assorted Materials (Food support) by County

Grand Bassa

Bomi

Bong

Cape Mount

Margibi

Montserrado

Nimba

Rivercess

11.4%

14.5%

17.5%

10.8%

13.9%

7.8%

17.5

6.6%

Provision of NFI support

Initially, it was planned that the project was going to provide NFI support to help most vulnerable survivors in both Gbarnga and Monrovia Dioceses. However, after community consultation, the majority of beneficiaries indicated that they wanted to go back to do business for sustainability. Most of these hardship cases were dependent on relatives who died and their income generation and livelihood support stopped abruptly. Therefore, the idea of NFI support was replaced with cash transfer of USD 100.00 per survivor. The distribution of the cash transfer to the hardship survivors is presented in the below graph by county of residence. It follows that counties with the highest numbers of survivals and vulnerable people based on case densities received the highest amount in the below table.

Table 1: Support to Hardship Survivals and Vulnerable by County Distribution Counties Frequency Percent

Grand Bassa 7 13.0Bomi 10 18.5Cape Mount 13 24.1Margibi 13 24.1Montserrado 8 14.8Nimba 3 5.6Total 54 100.0

20

The evaluation further asked the hardship Ebola survivors as for the usage of the USD 100.00 received as cash transfer to begin some income generating activities. The data collected showed that only 53.7% of the hardship survivors invested the money into business and nearly 30% used the money to buy food. According to the focus group discussions held in the communities, the views were divided with others claiming that one first of all needed to put something into the stomach before starting any business. Overall, they were all satisfied with the cash transfer and passionately agreed that the amount provided helped improved their livelihood and enhanced their adherence to the restriction imposed by government.

Cash Transfer (None farming families)The project also provided cash transfer to Non-farming families to jump start some business activities to assist their household. The distribution of the cash transfer is presented in the graph below by Geographic demarcation. It is glaring in this graph those counties that reported the highest number of cases had more affected and vulnerable people than those with fewer cases. To that respect, counties that reported more cases received huge portion of the cash transfer distribution.

Figure 3: Cash Transfer Distribution by County

Grand Bassa

Bomi

Bong

Cape Mount

Margibi

Montserrado

Nimba

Rivercess

11.3%

19.4%

2.2%

19.9%

14.5%

21%

3.8%

8.1%

Taking gender into consideration amongst the affected non-farming families, females received huge chunk of the money 69.4% compare with the males. When the usage of the cash transfer was assessed using the structure questionnaire, it showed that three quarters of the recipients invested the cash received in business and nearly 20% bought food for their household. The evaluator further probed if the cash provided to the non-farming families had any value added to their livelihood, 91.9% of the respondents affirmed that the money had some value added and improved their livelihood. During home visits,

21

most respondents were proud to show the team of evaluators some of the businesses they invested the money in ranging from petty table markets, hair dressing, carpentry, and blocks making amongst others. The focus group discussions held in the communities demonstrate that all the respondents rated the intervention as excellent and that it met their needs at the time and lauded CL and its partners.

Provision of farming support

A total of 400 families hosting Ebola orphans (201 females and 199 males) received a farming package in a one-off distribution in May and June 2015. The farming package included 50kg bags of local seed rice, 25kg bag of varieties of vegetables, and other farming tools such as cutlasses, shovels, diggers, files, wheelbarrows, rakes, axes amongst others. Due to the demand from locals, the distribution exceeded the plan target in the log frame by 100 beneficiaries. A representative sample was selected and a structured questionnaire used to evaluate the distribution. The distribution of the support to the farming families hosting orphans is presented in the graph below by geographic demarcation. You will also note that counties with most cases had more orphans compare with others. An assessment of the quantity of the agriculture input for the duration of one farming season was carried out. A little of half of the project beneficiaries 58.4% said that the agriculture inputs were not sufficient for the whole farming season. However, during the community based focal group discussions, all the respondents hail the endeavor and declared that it was relevant and had value added to their livelihood. A local woman said, “We did not harvest our crops after doing all the hard work when Ebola forced us to leave our farms with the wild life. So what CL and its partners did for us by providing seeds and tools is remarkable. Only God can repay them”.

Figure 4: Distribution of Farmers Support by County

Grand B

assa

Bomi

Bong

Cape M

ount

Margibi

Montserra

do

Nimba

Rivercess

9.6%

14.7% 14.7% 14.7%

11.2%

6.6%

18.8%

9.6%

22

Psycho-social Activities

In the original project document, training activity was planned and conducted. One hundred TOV sessions for community mobilizers in psycho-social and trauma counseling regarding Ebola at county levels were replicated at communities’ level to assist victims. A team of training experts from the Mother Pattern College of Health Sciences, HIV and Infections Division, a catholic run training health institution delivered the training. Having thoroughly reviewed the training manual, the team of experts in consultation with Caritas revised and replaced TOT with TOV for community mobilizers in order to directly be implemented at the community level to have greater impact on the target population. The Ministry of Health of the Republic of Liberia, the lead agency of government fighting the deadly Ebola Virus Disease standards were followed during the crisis intervention and integrated EVD protocol, psycho-social and trauma counseling was delivered in one training to 100 community mobilizers across the 10 project intervention counties. As the disease raged through the country in its wake taking precious lives, including those of health workers who should be caring for the sick, the situation became hopeless and overcast with national mourning. Even though not put blatantly by the beneficiaries, the issue became obvious and glaring that all those residing within the borders of Liberia needed some form of psycho-social and trauma counseling, though the affected individuals, families, and communities who lost their relatives and love ones in such a manner that they were not accorded the opportunity to bid them farewell needed psycho-social and trauma counseling most. In accordance to the Liberian tradition, families are required by custom to organize befitting funeral services for their deceased relatives ranging from given the deceased the first and final bath before they are taken to the mortuary for further preservation stretching from couple of days to weeks before committal. It is during this time period that other relatives and friends would commute between the deceased residence and sometimes stayed with the family until couple of weeks after burial. This group/community therapy worked best, dignifies the dead and is a pride to the deceased family in the society.

During a focus group discussion in the diocese of Gbarnga, a local woman explained her tribulation to the team of evaluators in tears with head bowed. “I sold pepper and bitter balls for some years to support my daughter in nursing school until she finally graduated. Following her graduation, she was immediately employed at the Phebe Lutheran Hospital. After working just a couple of months, the news came that she was amongst the five nurses that died in that Hospital from Ebola. The news hit us hard and it was hard to believe and accept because I had just talked with her and she was not sick. Up to date my children, I have not seen my daughter coming home neither have they brought her corpse. I am still hopeful that she is somewhere and will come home one day. Her father had a massive stroke and has not recovered, the grief is so hard to bear” she narrated. According to the different groups that took part in the discussions, nearly all of them

23

confirmed that a lot of trauma counseling activities took place, people went in and out from the churches, government, community dwellers, and a host of other people that they could not remembered to help them cope with their grief and lost. In this context, the evaluation of the psychosocial and trauma counseling assessment was complex and did not present anything concrete that one can aligned to a group of therapist but yet not questionable because some kind of activities occur in that line showing its relevance. They also mentioned the stigma attached to Ebola death in the family and the long term effect of constant depression, anxiety, sleepless activities especially at night, isolation and rejection. The different psycho-social and trauma healing activities were not given a sharp feedback like the others, the respondents were observed careworn to recall exactly what transpired with the counseling aspect. Though they admitted to changes observed in their state of grief, but much was not achieved in the area of psycho-social and trauma counseling activities. On the other hand, some of the psychosocial activities that actually occurred among these beneficiaries included; organization of games both board games (ludo, checkers and cards) and sports (football and kickball matches) among survivors, affected individuals and their communities in the Monrovia diocese. Gbarnga diocese organized cultural events, drama and special visits with affected individual and their communities

Provision of School fees and school kits

A total of 546 orphans, 248 girls and 298 boys benefited from this exercise.In the month of February 2015, 546 orphans mostly consisting of boys (55%) benefited from school fees and school kits. The contents of the school kits were ( one book bag, one dozen of copy books, one math set, one pack of pens, pencils each and a ruler). In addition, cash was given to guardians or community members taking care of the orphan(s) to locally procure uniforms, shoes, text books and other needed items. A little adjustment was made to serve additional orphans due to the huge burden of Ebola created orphans in the community and more of the previously enlisted orphans attending government school. One donor also contributed later specifically to serve additional orphans. . As many orphans who attended public schools and lower grades, the USD 260.00 per student cost was spread to cover other needy orphans. According to the documents reviewed, the data from the schools registries and Caritas distribution logs matched simultaneously. During a focal group discussion in both dioceses (Gbarnga and Monrovia), it is indicative that all the community members who are care takers of orphan(s) said that the provision of those items and the cash payment of school fees addressed their needs in a timely manner. A local man was spotted in the group of discussants pointing out that “We were thinking that these kids would never go back to school again following the death of their parents or care taker. God will bless the people who thought on supporting these orphans in such a timely way and much appreciated. Let God blessed their hands to get more.” The discussions were further loom with fear as to

24

what will happen to keep these kids in school the next academic year. The team of evaluators then decided to visit some of the schools to see evidence after reviewing the distribution logs and having the focus group discussions. The kids were seeing in school and the administration of the schools visited appreciated the visit of the evaluation team and CL for the assistance. They admitted that they were very pleased with the project and that the right orphans were actually enrolled in school the right time. Yet they express similar fear of what happens next to the kids when the fees paid are exhausted for the academic year. Visual inspections of some of the schools visited by the evaluation were

mare with good and bad scenes as the orphans’ mustered courage to be in school. The good news was that some of the hygiene kits distributed from the project were observed at the entry of some of the schools with plain water but no soap or detergent. However, some of the school infrastructure was in terrible shape like the ones shown below with see through walls in a local community within the Diocese of Monrovia.

Human Rights Training

25

Two persons per county were trained by the Catholic Justice and Peace Commission (JPC) to monitor all forms of violation (distribution fraud, domestic violence and illegal arrest) on the side of government and other actors. In addition, 120 representatives of the 60 communities were trained to monitor, advocate, and carry out facts finding on basic human rights violations and to document them. In the structured questionnaire, 4 of the counties confirmed receiving human rights training from (JPC) of the 8 counties selected for the evaluation. The graph below depicts their level of satisfaction with the training provided by JPC.

As for those who were trained (120) representatives from the 60 communities, the series of focus group discussion at the community and Parish level did not receive a sharp response. Most of the respondents were observed looking in the air for long in an attempt to recall the intervention. Except for two of the counties, Rivercess and Bomi where few of the respondents (<50%) declared that few case of fraud and domestic violence were reported but the culprits were never trial but seen moving around freely. In another development an elder woman said “my son, we talking about people dying and what time do we have to monitor human rights violations, hearts were broken, fear overwhelm us, so I think this should be at a later time”.However, the views were mixed as to the misuse of relief funds with others saying the implementers did very well while others were of the opinion that something went wrong. There were no reports for orphans abused reported neither any report on orphans being out of school. However, the national government had given institutions the right to suspend procurement regulations amidst the outbreak to make procedures simple and deliver emergency services in time. The president has since apologized to the nation for that and measures are now in place the recovery phase.

9.2 EffectivenessEffectiveness measures the extent to which project objectives have been achieved. The following questions were used to access the effectiveness of the EA-34 project.

To what extend were planned outputs achieved? How would you assess the livelihood support to individuals/beneficiaries? Were outputs/deliverables of an appropriate technical quality?

Though monitoring and evaluation framework were developed and used by Caritas Liberia project management, the project it was so demanding in terms of personnel and other capacity. To some extent, the M&E framework helped to tracked and bench marked the out puts against the objectives. The distribution went as planned and feedbacks from the beneficiaries indicate that there were significant gains from the project.

Hygiene Kits

26

The distributions of the hygiene kits were highly effective because they reached the target beneficiaries nearly 100%. They proved effective in helping the vulnerable population as Ebola raged through homes and took advantage of poor and precarious sanitary conditions. In many community shopping centers, some of the items found in the hygiene kits are not usually placed there for sale like bleach for example. In local shops, people concentrate on commodities that can be bought easily likely imported goods such as magic cubes, salt and others. Bleach is not used mostly in local communities in Liberia for hand washing but rather used to wash clothes and remove stains. Most local people wash their clothes in rivers and bash them on the surface of rocks along river banks to get clean and would not bother to buy bleach. This prevents local petty traders not to concentrate on buying commodities of such. Now in the context of Ebola, bleach has become a house hood name for Ebola prevention. According to the structured questionnaire and the Focus Group Discussions held, majority of the respondents embraced the hygiene kits distribution and said it was very useful in helping them to prevent the deadly Ebola Virus Disease. However, in most homes visited during the evaluation, buckets from the hygiene kits were observed sitting in front of most homes with plain water without soap neither bleach. Most of the respondents said the idea of hand washing helped them out a lot during the outbreak and they wish to continue that practice even without bleach or soap.

Assorted Materials (Food Support)

The project provided emergency support in the form of food to affected and vulnerable house hoods. The team of evaluators met scores of traumatized families to get feedback on the effectiveness of the items received. An overwhelming majority of them said they were happy and pleased with the items and that the distribution came just in time when they were without money to purchase them. According to them in the focal group discussions, the food items were of good quality and no complain of any food poisoning emerged after distribution. They lauded Caritas and said the food assistance was essential and effectives in helping them to meet their most essential needs during their period of bereavement, fear, sorrow and had value added to their live hood.

Provision of NFI Support

As stated earlier under the relevance, it was decided and agreed through a consultative meeting with the community that Caritas provide cash transfer of (US100) in place of the NFI support to jump start livelihood activities for sustainability. According to information gathered from the structured questionnaire and focus group discussion, all the vulnerable survivors were with one voice that the cash provided them was done effectively and had helped them to begin something that they call their own account. Most of them said this was the most important scheme then getting money from the bank

27

as loan and later begin to worry how to pay back. They said with this amount, they can always recuperate from their petty businesses even with the fluctuation of exchange rates and difficult economic situation. They admitted that the support given them by Caritas during has given them confidence to improve their livelihood. They were all grateful to Caritas and its funding partners and promise to use the money wisely for the intended purpose. The cash transfer to none farming families also received similar feedback in terms of effectiveness.

Provision of Farming Support

Four hundred and eighty vulnerable households hosting survivors or orphans were provided agriculture inputs from the EA-34 project. The distributions of these items were highly effective in that the planned outputs exceeded 100% benchmarked. According to the structured questionnaire and the focus group discussion held, nearly all the beneficiaries declared that the agriculture inputs actually helped them to regain their livelihood support. They confirm that the outbreak prevented them from going back to their farms, Accordingly, some of them said that certain seeds placed under ground did not came up while others said theirs came up well. In terms of the quality of the inputs provided them, the views were mixed. On visual inspection of some of the tools, some were in good shape while majority of them needed repairs or total replacement.

Psychosocial and Trauma Counseling

The psychosocial and trauma counseling aspect of the project did not receive a sharp response from the respondents even with a follow-up visits by the team of evaluators. During a focus group discussion with the respondents in both dioceses, most of respondents could not remember specifically which group of people performed psycho-social and trauma counseling during their period of bereavement. A few them admitted that people came from the churches, hospitals, , Ministry Health, parishes and other works of live to counsel them but it was not continued. So, the aspect of psycho-social counseling and trauma healing was not so effectives according the information from the focus group discussions. Respondents were observed struggling over what to say compare with the other intervention. In Liberia, not much capacity/expertise is within country to carry out effective psycho-social and trauma counseling.

The human right aspect was also not effective due to responses from the beneficiaries as stated in the relevance. In the midst of fear for a disease that takes away several lives in a short time, most people were not keen on rights issues but some of the respondents were able to recall few cases of malpractice to donor funding and distribution problems. More is needed to be done in the areas of psycho-social and trauma counseling as well as human rights since they are difficult to measure.

28

Provision of School fees and School Kits

The Ebola Virus Disease placed graved financial burden on families who were directly affected. The virus took away precious lives that were source/support to the education of their children and other dependents. In Liberia, school support to children is a serious challenge to most parents while others have the capacity to their children’s education. This situation changed with most school-aged children losing their parents or guardians during the Ebola onslaught. According to the evaluation results, using the structured questionnaire and the series of focus group discussions, all families hosting orphans and spoken to say that the school support to orphans was awesome and very effective. A local woman during the discussion said “if these people had not come to help us put these children back in school following the death of their parents, most of them would be sitting home by now”. So the help from Caritas and its partners has backed us in a positive direction. According to them, the help was so effective and came the right time.

Coordination

Though the project implementation was the first of its kind by Caritas Liberia; significant gains were made despite capacity challenges. Coordination is important were multiple projects targeting the same problem are coordinated to avoid duplication of activities and enhanced standard services delivery and avoid wastage. The project management staff proved very effective and created partnership and built relationship with several stakeholders (WFP, CRS, and UNDP, WHO, UNICEF, CDC, Red Cross Societies and others). The lead for the Ebola fight in Liberia was the Incident Management System (IMS) under the auspices of the Ministry of Health. Caritas Liberia regularly attended the IMS meetings, the psycho-social working group, at times and food security cluster meetings and UN Security Council meetings. At the county level, the local Caritas county coordinators attended the county IMS meetings and certain working groups.

9.3 EfficiencyEfficiency looks at the organization capacity and the way the project was implemented given the allocated resources. The following questions when answered should give clue as to how efficient the project was.

Was the project cost effective? Was there enough staff of appropriate competency for the achievement of the

intended objectives?

The assistance and services provided under the EA-34 project were based on needs assessment conducted by CL as the Ebola Virus Disease raged in the country. As stated earlier in the project narrative document, at the time the project proposal was drafted, Caritas Liberia was operating on a minimal scale and with capacity challenges. At the

29

start of the project, Caritas needed to hire staff but many national and international organizations including the Government of Liberia had done massive recruitment of most competent staff in the country. To this end, Caritas had no option but to hire and do on job coaching of its internal staff. The success achieved in implementing the project activities is largely credited to the devotion and strong coordination mechanisms on the part of Caritas staff, the emergency accompanier seconded by Cordaid, management team, county coordinators, community structures and the host of volunteers who made so much sacrifice to serve their country. The emergency accompanier continually advised the National Coordinator, project manager on budget issues and provided other technical support. Caritas staffs, who were directly responsible for the implementation of the project were required to provide biweekly plan, monthly funds request and report to the project manager at Caritas National office. However, field visits, direct observation methods, and institutional review using key informant Interview techniques squarely indicate that Caritas Liberia is seriously understaff and with low capacity in term of staff level of educational training and attainment. A head account of Caritas staff suggests that they are less than 10 in both dioceses with scores of volunteers. Nearly all of the staffs are without any formal training for disaster management and emergency preparedness and response. A training needs assessment is appropriate for Caritas Liberia in order to develop best fit curriculum that will address the needs of the institution and promote its vision.Caritas Liberia has only two vehicles in good working condition. Cordaid provided a donation in kind which included a pickup truck, motorbikes, buckets and other sanitary materials. Caritas Senegal also donated a vehicle during the Ebola fight making a total of two vehicles. The local office of Caritas in the diocese of Gbarnga is without any vehicle. The donated vehicles and motorbikes were seen during the evaluation but much is required to serve a population of approximately 4.2 million people sitting of a land surface area of 42,000 square miles. According to the project documents reviewed, provisions were not made to purchase or hired vehicles thereby making it difficult at times to move goods and services to targeted areas. The office space provided Caritas is the size of a dwelling home in Liberia which should host an average household size of 5.1. It is without a warehouse neither a parking lot. The National Coordinator, Project Management and other essential staff were seen sitting in tiny rooms without much ventilation. During the evaluation exercise, there we no meeting halls beside tiny offices which housed the National Coordinator, the project Manager and sometimes consultants. The need exist if Caritas Liberia is to expand and continue the good work it started to find a better office space with meeting halls, ware houses(s) and a vehicle packing lost to safe guard the organizations’ property to better serve humanity.

9.4 Impact

30

Impact is assessed by considering the positive and negative long term effects of implemented project activities. The evaluation intends to answer the following questions in this light.

What has happened as a result of the project implementation? What are the wider socio-economic, technical, and environmental, of the project

on individuals’ disaggregated by sex, age groups, communities and institution?

Project impacts are generally identified based on quantitative data analysis using a baseline and spread over a long period in time. Interviews and focus group discussion with direct beneficiaries and staff members can give an indication of outcomes of projects. All beneficiaries and a cross section of community leaders, inhabitants, church leaders, and parishioners met during the evaluation were thankful to Caritas for the EA-34 for providing them emergency assistance when they needed it most. Nearly all, ranging from (90-100 percent) admitted through the structured questionnaire and the focus group discussions that the assistance provided by Caritas and its partners contributed immensely to their livelihood and aided them cope with their grief during the EVD outbreak.

However, the massive Ebola prevention messages provided through the project and the provision of hygiene kits has enhanced the level of sanitation across the affected communities. The evaluators observed to their dismay that hand washing now has become a routine as community dwellers were seen committed to practice during their daily activities. The provision of the school support has boost enrollment in most of the schools visited. Pupils who were not in school were seen wearing uniforms with book bags as they see peers who are orphans courageous enough to go back to school despite their loss. Caregivers and guardians became more affectionate to orphans with the support of the community. Farmers were again seen meeting in groups and discussing their farming activities including challenges and way forward. The farming activities has enhanced food security to some extent and most communities that used to struggle for food can now afford to feed their households without attempting to borrow from friends or do barter system with some of their essential commodities. The farmers were seen rejoicing and saying that Caritas has put them back on their feet post Ebola. The help for them is well appreciated.The cash assistance provided to none farming families and hardship survivors proved well-intentioned and have enhanced economic activities in most of the communities visited. The majority of households visited were seen into some form of business. Business activities diversified with people selling little items that were not usually sold at the community level before that outbreak.

31

9.5 SustainabilitySustainability looked at the likelihood of long-term benefits resulting from the implementation of the project. Key questions considered to assess sustainability were:

. Was the project design and implementation informed by learning from previous experience from the dioceses?

. Was new learning being captured and acted upon during the project implementation?

This project was the first of its kind design and implemented by Caritas Liberia. It serves as learning opportunity for the staff of Caritas to build on and to improve the design of futures projects of such that nature. It is without doubt that emergency interventions are just for time being to address certain unusual situations. When those situations changed, sustainability then becomes a matter of scaling down to more important matters arising from the interventions. Emergency funds are provided without an idea that it would have being the case, but circumstance depicts that. In this light, sustainability is hard to measure but a long term plan and possible proposal generation and support, shall point to specifics and direct which aspect of programme in intervention should be carried out in a sustained manner for longer term impact.

10. CONCLUSIONS, LESSONS LEARNT AND RECOMMENDATIONS

32

The 8-month CARITAS EA-34 project provides an opportunity to build on lessons learnt and promote a broader roll-out approach in future emergencies programs. On steadiness, despite the relatively short implementation period, it is clear that the basic premises of the EA project is valid in practice and that broader stakeholder engagement in participatory process will promote further an improved understanding, buy in to future project objectives and enhance collaboration. Although pre-determined, the focus areas of the project were largely considered to be relevant at all levels and progress was made in all dioceses covered towards the achievement of the project’s major objectives. Overall some of the projects main areas of achievements were felt by the beneficiaries (provision of hygiene kits to help prevent transmission, provision of agriculture inputs, cash transfer at various levels, school support to vulnerable people, human rights training, raising awareness, monitoring violations of all kinds and providing psycho-social support to victims).

In addition, hearsay evidence suggests that community based clubs and different groups contributed to increasing confidence levels of affected people during the outbreak and encouraged their engagement in complying with all interventions and keeping safe. To consolidate gains and promote wider impact and sustainability, more remains to be done in areas of advocacy and collaboration, psycho-social support, human rights primarily by building on initial successes achieved and linking work being undertaken in the implementation areas to the national level.

The lessons learnt, presented below, are from the results of the focus group discussions, structured and key informant interviews held with direct and indirect beneficiaries as well as CL staff members involved in the implementation of the project. Another value added to the lessons learnt came from direct observation techniques used by the evaluators during the field visits. While this project is the first of its kind to be implemented by CL, it provided substantial values which enhanced the knowledge and skills of all staff involved in the project roll-out. CL has now gotten first experience in disaster response and had the opportunity to develop partnership and good working relationship with the Central Ministry of Health and other partners through experience sharing. This gain has put Caritas Liberia amongst the list of humanitarian relief providers in the country and should be used to obtain needed support for the organization to forge ahead in future programs designs and implementation. The project was lunched just in time when the Ebola Virus Diseased had passed its uttermost (August & September, 2014) leaving behind several victims and massive socio-economic problems for families and the country at large.This report therefore provides opportunity to emphasize a few points in other to contribute to CL and partners capacity to better assist the neediest:

Organization Capacity

33

Capacity building is an ongoing process that demands, significant time and investment. The project implementation revealed to a greater extent that Caritas Liberia lacked both human and infrastructure capacity needed to adequately serve a population of approximately 4.2 million people with the current level of staffing. Focus group discussion held indicated that majority of the respondents declared the interventions reached them but field staffs were sometimes not on schedule as planned. The key informant interviews also corroborated the immersed hardship by project team to deliver items to villages that were far with just one working vehicle. Rental vehicles in Liberia are often not in good condition and road commendable to go into localities that are inaccessible, much more under such precarious conditions. At some point in time according to contact tracing information obtained from the Ministry of Health, commercial vehicle drivers were carries of infected passengers thereby infecting themselves and families at home, In terms of human capacity, the institutional review process also saw low staff capacity and without adequate training to carry out cortical task. The needs of victims and the vulnerable population, and the complex environment put CL under considerable strain in the area of human capital. In such a context, assistance can only be provided to large number of victims /beneficences relying on adequate and competent staffing capacity with the needed enabling environment. Internal processes and policies are needed to serve as frame for programs and to guarantee effectiveness and efficiency. There was a report of theft to Caritas office and perpetrators went with several valuable items belonging to the organization. The institutional assessment showed that CL current office is not appropriate for the kind of work the organization is involved with. This should be a lesson learnt for CL and it partners to think in the direction of getting a suitable office space that will be spacious and secure enough to protect CL’s property.

Psychosocial and Trauma Counseling

The evaluation has shown that respondent’s knowledge was not very sharp in providing information on the Psychosocial and Trauma Counseling aspect. They could barely remember which group of people provided trauma counseling. It was observed that the psychosocial and trauma counseling did not grind as it should because the respondents were still heart broken and experiencing signs and symptoms of post-traumatic stress disorders as observed. The Liberian educational system does offer trauma counseling related courses because the required experts are not available. This is suggestive that more competencies are required in terms of building capacity of CL in those critical areas. The duplications of activities could also be a factor for respondents not being able to squarely recall psychosocial and trauma counseling at a point in time. Data management and M&E system were not so effectives as should be. The M&E office at CL had relied on coaching methods during the project roll-out. Nearly all documents reviewed were manually kept. Given the experience accrued by the M&E staffs the

34

project implementation, a cutting-edge M&E training is required to upgrade their levels. The appointment of additional staff to the M&E unit shall boast future emergency response and enhance subsequent program activities.

Recommendations

The final evaluation of the EA-34 project is an opportunity to reflect on the past experience from the implementation and the findings to consider future program designs and focus. It is now a unique time to reconsider decisions made in terms of the organization and interventions, and draw recommendations based on results and achievements that measures the projects goals and objectives as well as highlighting its relevance, effectiveness, efficiency, impact, and sustainability based on OECD/DAC standards and best practice. The EA-34 project intervention took place at a very crucial time for CL when the Ebola virus disease raged through the country leaving behind huge social-economic problems. CL had human capacity and resource challenges. The list of recommendations below are intended to contribute to current organization structure based on the findings and lesson learnt to better prepare for upcoming phase of the project.

Human Capacity- based on the findings of the study, it is recommend for CL and it partners to scale up the capacity of the organization through employment of well-trained individuals in all program areas to get value for money and better serve the country and its people. In addition, there is a need to develop the capacity of current staff that implemented the EA-34 project and gleaned so much experience for their first time. The capacity building initiatives should focus in the areas of project management, emergency preparedness and response, Data management, Finance, Report writing, Procurement, and psycho-social and trauma counseling. A transparent recruitment process based on skills and experience, good salary structure, enabling environment and employment benefits reinforce motivation and retention of talented staff. In addition, staff capacity needs assessment for training is key. Supervision and appraisals should be established in a systematic way to enhance dialogue on performance feedback, identify gaps for training and discuss career opportunities.

Infrastructure Capacity - The findings from the study indicate that Caritas has no good office space. There is therefore a dare need, for CL and its partners to rent or build an office with a warehouse, offices, meetings rooms and car parking lots for effective running of future programs.Logistics- The need for logistical provisions was also a major challenge faced by CL during the project implementation. It is therefore recommended that CL and it partners seek funding to procure vehicles, office furniture, printers, air conditioners, stationeries and related office materials.

35

Develop or build upon the internal process and policies documents from the diocese of Brangan to support activities and improve transparency. It is essential that project managers and polices support the implementation of vision and facilitate the execution of future projects. Reliable internal polices, compliance management, robust M&E and quality control will reinforce transparency and trust.

Support and Strengthen Decentralization at local levels

To contribute to relevance of activities and efficiency in the use of resources, staff members and volunteers at local level have sound expertise from the project and spent a great amount of their time meeting with project beneficiaries periodically. Based on their skills and relationship with the local communities, they should be granted more autonomy to make decisions at local level but using approved standards from the central level.

Appendix 1: Key informant interview and focus group discussions inventory

36

Key Informant Interview (KII)

Stakeholder Name No. of Interview Male Female

Caritas Monrovia Office 2 2 1

Caritas Gbarnga Office 2 3 0

Focus Group Discussions (FGDs)

Local Church Groups (Parish level) No. of Interview Male Female

Monrovia Diocese 5 5 6

Gbarnga Diocese 3 3 5

Community Level

Monrovia 5 7 5

Gbarnga 3 8 9

Total 16 23 25

Appendix 2: Key Informant/Focus Group Discussion Interview Sheet

37

Introduction

Good day. My name is Patrick N. Kpanyen. I am working for CARITAS LIBERIA as a consultant on the external evaluation of the Emergency Appeal EA-34 Project during the Ebola Virus Disease outbreak in the country. The Project provided assistance to people affected directly or indirectly by the Ebola Virus Disease. The purpose of the evaluation is to find out to what extent the project achieved its objectives and lessons learned. The findings from our discussion with all of you who were involved or benefited from the project will be written up into a report to be used by Caritas Liberia (CL) and its partners. The discussions will focus on five main areas: Relevance, Effectiveness, Efficiency, Sustainability, and Impact of the project. You are invited to volunteer to participate in the evaluation as respondent. The questions usually take at most an hour to complete. If you do not understand the information or have any other questions, please feel free to ask.

Do you have any question? If No, can we begin the interview/discussions now?Respondent Agrees to be Interviewed …………….1 (circle number 1) Respondent Does not agree to be Interviewed ……2 (End the interview)

Confidentiality

All the information provided in this study will be kept strictly confidential. There is no cost to you for participating in this study. If you have any question(s) about the evaluation, you may please contact Caritas Liberia.

INFORMED CONSENT FORM

I ________________________________, hereby confirm my voluntary participation in this evaluation after having being provided sufficient information by the interviewer.

Name: _______________________________________________Full Job Title: _________________________________________Sex: ___________________Age: ___________________Length of time involved with the EA-34 Project: _______________________________

1. Relevance

38

Given your knowledge of the Emergency Appeal Project which provided several key items to vulnerable people during the Ebola Virus Disease Outbreak in the country, what would you say was the value added of the project and for whom?

Were appropriate systems of downwards accountability (participation, information sharing and feedback), put in place and used by project participants?

Do you think the project manage to improve the livelihood of the target population? If Yes or No, please explain and give examples.

Do you think there were any activities that were not particularly effective? Yes/No. Which ones and why?

2. Effectiveness

To what extent do you think activities planned were achieved during the project rollout?

Was the project effectively implemented? Support your answers by explaining why or why not.

3. Efficiency Was the project implemented in a cost effective way/manner? What

measures were taken (e.g. in procurement, recruitment, implementation etc.) to ensure cost effectiveness during the implementation of this project?

How efficient was the flow of funds from the supporting partners? Were there any delays to implementation? If so, what effect did this have?

Was there enough staff, of appropriate competency, for the achievement of the intended objectives for the project?

4. Sustainability

Was the project design and implementation informed by learning from previous experience. Were new learning being captured and acted upon during the project implementation?

Are there any components of the project that could potentially be scaled up? How? To what level? Can you foresee any challenges?

What did the project do to specifically encourage local participation and ownership? To what extent was this successful?

5. Impact

39

What do you think were the wider social, economic, technical, environmental, effects of the project on individuals by sex, age, communities and institutions?

6. Coherence/ComplementaritiesHow has the project been coordinated with the activities and priorities of other agencies and organizations (including local and national government)?

7. Organization Capacity Perspectives

Was there an appropriate system of management and communication in place to support programme staff?

Was there an appropriate logistics system in place? (E.g. did the procurement process work in a timely, transparent manner?)

40