“moving goal posts” - unicef€¦ · “moving goal posts” final evaluation report of support...

“Moving Goal Posts”

Final evaluation report of

Support to Health Sector Development in Somalia

November 2009

This report is financed by the European Commission and is presented by Egbert Sondorp for UNICEF and the European

Commission. It does not necessarily reflect the opinion of UNICEF or the European Commission.

Evaluation of Support to Health Sector Development programme Somalia

ii

Acknowledgements

This is the final report of a final, external evaluation of the ‘Support to Health Sector Development in Somalia – Capacity Building, Strengthening System Coordination and Performance’ programme. The project was implemented by UNICEF Somalia office and financed by the European Commission.

Earlier draft reports were scrutinised by the members of the evaluation’s reference group and feedback has been incorporated into this final version.

Field work for the evaluation took place in November 2009, over a period of three weeks, divided over a period in Nairobi and Somaliland.

The evaluator wishes to thank all people who have contributed to the evaluation process, in particular:

• All those willing to give time to be interviewed, often at short notice • UNICEF project management for all arrangements, access to documentation, and

participation in the workshop • The evaluation’s reference group for valuable guidance (Renato Corregia, UNOPS; Edda

Costarelli, EC; Gemma Sanmartin, COOPI, Sabrina Pestilli, COSV; Suraya Dalil, UNICEF) • UNOPS staff Nairobi, for all travel and other arrangements • UNICEF staff in Hargeisa, in particular Mohammed Sheigh for miraculously organising a host

of interviews during a short period of time and local travel arrangements, and Maryan, MCH officer, who accompanied the evaluator on trips to Berbera and Borama.

• COOPI staff (Miresi Busana, Dr Ahmed Askar) for company during the field trips and arranging meetings with health staffs in various health facilities

It was good to meet so many people who continue to be highly motivated, sometimes seemingly against all odds, to work for the betterment of Somali’s health sector.


iii

Table of Contents

ACKNOWLEDGEMENTS .................................................................................................................................. II

TABLE OF CONTENTS .................................................................................................................................... III

LIST OF ABBREVIATIONS ............................................................................................................................... V

KEY FINDINGS AND RECOMMENDATIONS ...................................................................................................... 1

EXECUTIVE SUMMARY ................................................................................................................................... 3

1 PREAMBLE ............................................................................................................................................ 7

1.1 PRINCIPAL FEATURES OF THE PROJECT .............................................................................................................. 7

1.2 OBJECTIVES AND PLAN OF WORK OF THE EVALUATION .......................................................................................... 9

2 PROJECT PREPARATION AND DESIGN .................................................................................................. 11

2.1 INITIAL PROJECT DESIGN .............................................................................................................................. 11

2.2 THE PROPOSAL .......................................................................................................................................... 14

3 RELEVANCE OF THE PROJECT .............................................................................................................. 19

4 ACHIEVEMENT OF PURPOSE (EFFECTIVENESS) ..................................................................................... 24

4.1 SUPPORT TO THE MINISTRIES OF HEALTH ........................................................................................................ 26

4.2 HEALTH SECTOR ANALYSIS AND TOOL DEVELOPMENT ......................................................................................... 28

4.3 HEALTH SECTOR COORDINATION ................................................................................................................... 31

4.4 SUPPORT TO HEALTH SERVICE PROVIDERS. ....................................................................................................... 32

5 EFFICIENCY ......................................................................................................................................... 35

6 IMPACT .............................................................................................................................................. 36

7 CONCLUSIONS AND RECOMMENDATIONS .......................................................................................... 38

7.1 OVERALL OUTCOME ................................................................................................................................... 38

7.2 THEMES .................................................................................................................................................. 38

7.2.1 Project design .............................................................................................................................. 38

7.2.2 Implementation ........................................................................................................................... 39

7.2.3 Role of UNICEF ............................................................................................................................ 41

7.2.4 Lot 3 vis‐à‐vis Lot 1 and 2 ............................................................................................................ 42

7.2.5 The EPHS ..................................................................................................................................... 42

7.3 SUSTAINABILITY ........................................................................................................................................ 43

7.4 SPECIFIC RECOMMENDATIONS ...................................................................................................................... 43


iv

8 APPROPRIATENESS OF THE INTERVENTION ......................................................................................... 45

ANNEX 1 – MAP OF SOMALIA (OVERVIEW & ADMINISTRATIVE UNITS) ......................................................... 47

ANNEX 2 – STUDY OBJECTIVES AND RESULTS AS EXPECTED IN THE TOR ....................................................... 48

ANNEX 3 – ORIGINAL LOGICAL FRAMEWORK ............................................................................................... 49

ANNEX 4 – LOGICAL FRAMEWORK (AMENDMENT 2, 28/02/09) ................................................................... 54

ANNEX 5 ‐ PUBLICATIONS BY UNICEF UNDER LOT 3 PROGRAMME ............................................................... 62

ANNEX 6 – LOT 3 CONSULTANTS .................................................................................................................. 63

ANNEX 7 ‐ PEOPLE / ORGANISATIONS CONSULTED FOR FINAL EVALUATION ................................................ 65


v

List of abbreviations

CfP Call for Proposals CHD Child Health Days CISS Coordination of International Support to Somalia (Nairobi based) COSV Committee of the Organization for Voluntary Service COOPI Cooperazione Internationale CSS Central and Southern Somalia (CSS) DFID Department for International Development EC European Commission ECHO European Commission Humanitarian Office EPHS Essential Package of Health Services GFATM Global Fund for the fight against Aids, Tuberculosis and Malaria HCBU Health Capacity Building Units (initially proposed units within Somali MoHs) HSC Health Sector Committee (part of CISS) HIV/AIDS Human Immuno‐Deficiency Virus/Acquired Immune Deficiency Syndrome HMIS Health Management Information System HOAP Harmonisation and Operational Action Plan HR(D) Human Resources (Development) HSAT Health Systems Analysis Team HSS Health System Strengthening (also a working group on HSS, under HSC) HSSP Health system strengthening programme – DFID supported consortium HSSP2 Foreseen follow up programme to HSSP MCH Mother and Child Health MDG Millennium Development Goal HMIS Health Management Information System JNA Joint Needs Assessment M&E Monitoring and Evaluation MDG Millennium Development Goals NGO Non‐Governmental Organization PHC Primary Health Care TB Tuberculosis THET Tropical Health and Education Trust TOR Terms of Reference UN United Nations UNICEF United Nations Children’s Fund WHO World Health Organization


1

Key Findings and Recommendations

Project design phase • The notion to pay more attention to broader health system issues was excellent and very

much valid up to today. The willingness of both EC and UNICEF to take this on, should be commended. However, the final stage of the formulation phase was flawed.

• Due to time pressure and lack of technical expertise in both EC and UNICEF, a project was set up with over‐ambitious, unrealistic objectives, to be achieved in a short, unrealistic timeframe, with insufficient budget, against a background of deteriorating political and security circumstances in Somalia.

• The EC should not have approved this project because of its technical flaws as well as the substantial deviation from what the EC initially had intended with the Lot3 programme; in particular since the EC knew it was withdrawing from the health sector and the project could not be extended

• UNICEF should not have submitted this proposal, as was developed by its staff, to the EC for approval because of its technical flaws and the lacking prospect of continued funding beyond the life of Lot3, a necessary condition to embark on a project of this nature.

Background • The “Health Sector Development programme Somalia”, better known as the Lot3

programme, was implemented by UNICEF at the request of and funded by the EC, from January 2007 to July 2009.

• The programme was meant to steer away from the long term costs of a perpetual short term, project focused approach to health system intervention in Somalia and pay more programmatic attention to broader health system analysis and capacity building

• The programme was overall largely successful in achieving this The way it got there has been rather miraculously


2

Implementation phase • Late arrival of the project manager, continous changes in the political and security

situation, and the poor project design led to the need for repeated changes in the project, adapting to circumstances, but also with an eye to grab opportunities

• Changes were formally laid down in amendments and new logframes. However, these documents retained the unrealistic tone and objectives of the initial proposal, only partly reflected actual activities and were not consistently reported against.

• A core element of the proposal to set up Health Capacty Building Units in the three zonal MoHs did not materialize, primarily due to deteriorating security in all parts of Somalia.

• Main emphasis therefore turned to what became the major achievements of the project: o The production of a series of analytical reports and tools o Significant contribution to strategic leadership for the health sector o Capacity building activities in MoH, mainly the Somaliland.

• Expectations, from the EC and Lot 1 and 2 projects, that Lots would support Lot 1 and 2, were not met.

Key Recommendation • The project has shown that even under the extremely difficult circumstances as Somalia

poses it is possibly to develop broader, health system oriented activities • There is no blueprint for these activities and hardly any lessons are available from other

fragile states, in part to the uniqueness of the Somali context, in part due to virtually total lack of research in this field

• These health system strengthening activities should ideally not be ‘projectised’, but be placed in an ongoing programme, well linked to the health coordination platforms, and be instrumental in strengthening the coordination and strategic leadership

• Such HSS programme will need access to best available data, or should produce its own data; documentation and analysis of relevant health system issues should be ongoing

• While being informed by available data and analysis and aware of available resource capacities (technical and financial) with the invariably manifold stakeholders, the strategic leadership should provide sound guidance as to:

o Desirability and feasibility of longer term health system investment o Strengthen health governane through capacity building of not only the health

authorities, but all stakeholders in health like the communities, civil society and health providers, including private providers.

o Strengthen service delivery, with a view to maximise health gains as would be possible in the given circumstances

o Desirability of small scale piloting of novel approaches, ensuring learning o The ‘early recovery’ and HSS potential of humanitarian project activities


3

Executive Summary

This document reports on the final evaluation of the project “Support to Health Sector Development in Somalia – Capacity Building, Strengthening System Coordination and Performance”, better known as the Lot 3 project. The Lot3 project was commissioned by the EC, implemented by UNICEF from January 2007 – June 2009. From the budget of 2.15 Million Euro in the end 1.31 million Euros was disbursed. EC procedures prevented further no cost extension.

Project design

Prior to 2006, both a report commissioned by the EC and a comprehensive Joint Needs Assessment recommended to steer away from the long term costs of a perpetual short term, project focused approach to health system intervention in Somalia and pay more programmatic attention to broader health system analysis and capacity building. Some of these notions echo in a Call for Proposal the EC launched by March 2006. This CfP was issued, when the EC had already decided to withdraw from the health sector in Somalia in 2 years time.

The CfP called for three proposals, labelled Lot 1, 2 and 3, to support basic quality health services within the framework of a structured and harmonized health system while addressing gender issues. Lot 3 was seen as an umbrella project to the more traditional NGO run projects (Lot 1 and 2) that were to support a number of hospitals and some other health facilities in various parts of Somalia. Lot 3 was then to provide support across the sector and work on institutional development of the three Ministries of Health in Somalia.

Since no one submitted a bid for the Lot 3 CfP, the EC started negotiations with UNICEF to take it on. UNICEF agreed, provided it could change the ToR. It so did through a ‘concept note’. The concept note took on a broad health systems approach and proposed implementation primarily by setting up Health Capacity Building Units in the three MoHs. Lot 1 and 2 could then be supported from these HCBUs.

This concept note was then transformed by UNICEF into a fully fledged technical proposal by November 2006 and approved by the EC. In retrospect, this technical proposal comes across as highly over‐ambitious, with a set of objectives that touches on too many facets of a very poorly developed health sector in an environment with very poor state functions. The proposal was not realistic and probably could never be achieved, and definitely not in the given time frame of 2 years and with a limited budget. In addition, unlike the optimism that had prevailed in early 2006, the situation in Somalia had started to deteriorate again and continued to do so until today.

One of the major conclusions of this evaluation is that this specific proposal should not have been approved. In itself the notion of wanting to do more broad health system strengthening was laudable and should definitely be attempted in these kinds of environments. But the design in the end did not fit the realities, and was subjected to an extremely short time horizon and


4

supported by a donor who wanted to exit from the health sector. However, programming for broader health system support should be encouraged, for all protracted crises, including Somalia. Formulation of such programmes requires broader expertise than was available in the context of Lot 3 formulation and approval.

Project implementation

The project suffered from a delay due to late arrival of the programme manager. By then it had become clear that “given the current situation, with no political solution in sight and no major increase of funds foreseen, the Lot 3 project as originally designed has become less relevant”. Through a workshop with representatives of all stakeholder groups in the Somali health sector in September 2007, a new plan of action was set. The new action plan was more pragmatic, realistic and prioritised – and much of the new plan was achieved.

Paradoxically, amendments and new logframes as formally approved by the EC, were apparently constrained by the initial design and budget structure and continued to be over‐ambitious and unrealistic. Logframes were also not consistently used to report against.

In the mean time, primarily due to deteriorating security conditions next to some operational reasons (around budget, recruitment and restrictions to UN operations) the HCBUs never materialised. Only in Somaliland some support could be given by posting consultants inside the MoHL for a number of months.

Notwithstanding the inherent difficulty of the project and its design, and the dire and deteriorating circumstances in which it had to operate, the project has been successful. In particular by showing that it is possible to work on broader health system issues in a meaningful and operationally relevant way. Usefulness and feasibility of the project should encourage stakeholders, in Somalia, but also elsewhere, to create appropriate platforms from where a broader health system approach can be stimulated. This should not be in the form of a short, time‐bound project like Lot 3, but rather ongoing as long as extreme fragility continues.

The major achievements of the project can be summarised as:

• Strategic leadership for the sector. The project, more specifically through its project manager, provided a strategic leadership role for the Somali health sector which was clearly missing in earlier years. By an active engagement with the CISS’s Health Sector Committee in Nairobi and its various working groups and task forces, the project could act as a ‘catalyst for coordination’. Also input could be given into some of the few new funding initiatives for health in Somalia.

• Series of analytical reports and tools The project produced an impressive series of studies and tools. Many of these will have value for years to come, even if they can’t be fully implemented at the moment due to the difficult situation on the ground. In particular the work on the Essential Package of Health Services was received very positively by all stakeholders.


5

• MoH capacity building Although MoH capacity building has been much more limited than foreseen, in particular for CSZ and Puntland, some capacity building efforts for the Somaliland MoH have been successful. In particular in the field of

Human Resources, where concrete input from consultants hired by the project, with a presence inside the MoH, contributed to a new appreciation of the challenge of HR in reforming the health system as well as a new HR policy;

Planning – where support to HMIS units, coordination meetings in the zone (central and regional) and a series of planning workshops focused around implementing the EPHS and regional management established the central themes and challenges for development of the PHC system.

Investment in HMIS units (with GF Malaria money but managed by the Lot3 project manager) enhanced availability and use of HMIS data for establishing decisions – and enlivened zonal coordination..

Lastly, the EPHS and HR salary documents as well as the facility blueprints are widely seen across all 3 administrations as being core to establishing the agenda for health sector reform

There will never be a quick fix for Somalia’s health sector. The state, or another body on its behalf, in which a variety of other stakeholders may participate, will be needed to ensure stewardship and finance for the provision of equitable health services to the population, at affordable cost and reasonable coverage. This longer term goal of improved health governance may be pursued through a variety of ways, not only through strengthening MoH capacities. As Lot 3 showed, a broader look at health governance may also take account of civil society and the private health providers and investments there may have a positive impact on health governance and state building.

Specific recommendations:

• Addressing broader health system issues in a protracted crisis environment like Somalia proves possible and useful and is to be continued.

• Somehow a ‘HSS Unit’ will have to be created that continues supporting strategic leadership in the health sector. This will have to be based on context specific data collection and analysis, conduct or commissioning of specific studies, policy formulation, and ability to disseminate HSS advice throughout the wider national and international health community involved with the Somali health sector.

• Such a HSS Unit should not be as time bound as the Lot 3 project, but work against a longer time horizon. Activities will vary over time and cannot all be planned in advance. Flexibility will have to guaranteed, which will require a governance structure that is able to allow this flexibility, but nevertheless keeps the Unit accountable, working along prioritised timelines, and responsive to the needs in the environment


6

• It is beyond the scope of this evaluation to recommend the exact place of such a Unit. This will require broader consultation among stakeholders, including donors. It should be closely linked to the existing coordination platforms. It may or may not be hosted by an operational organisation. The example of Lot 3 having been inside UNICEF shows some of the pros and cons. For instance, direct exposure to operational challenges will keep a ‘HSS Unit’ close to the realities in the field, and not become a too distant ‘policy unit’. A downside may be the risk of the Unit getting too much into operations or that others outside the hosting agency feel less ownership.

• There are hardly any examples of similar ‘Units’. If at all more health system oriented activities were systematically pursued in similar circumstances (extremely limited state involvement for prolonged periods; variety of extended ‘humanitarian’ projects), this was usually linked to a particular donor funding operational partners (NGOs). In addition, the Somali context is very specific, incomparable to any other. External expertise may be able to play a useful advisory role for such a Unit, maybe in the form of a distance based panel that could advise on quarterly/annual plans, specific studies, and experiences elsewhere.

• It is recommended to properly document the HSS approach, including process and issues around coordination, as emerged over the lifetime of Lot 3, beyond existing project documentation that has been written for other purposes. Documentation will be of use locally, for any successor programme, but also as case study for an external audience. The international community looks for ways to promote HSS in fragile states, and this case study should become part of the evidence.

• The EPHS is to be piloted at small scale in smaller geographical entities where all levels (community, health post, health centre (MCH Clinic), and basic hospital level with at least comprehensive emergency obstetric care. To phase this in, it may be wise not to start with the full package as described in the EPHS but start with more basic packages as are in use in for instance Afghanistan, Liberia or Southern Sudan.

• Health system development should not be an act of its own, but should seek to pursue sustainable health gains, primarily through service delivery. In the context of Somalia this may take alternative approaches, different from the universal ‘district‐based primary health care approach’. In particular community based approaches, less dependent on fixed health facilities may be one of those alternatives. Also, the Child Health Days as implemented by UNICEF, with an input from Lot 3, should be carefully evaluated in terms of effect (health gains), cost‐effectiveness, and possible positive and negative side‐effects.

• UNICEF took on Lot3 and embarked on broader health system support than it usually does in similar circumstances and proved not fully equipped to do so, in terms of supporting expertise and recruitment. However, the stability UNICEF offered to the Lot3 project made it a good host and, in return, UNICEF health activities have benefited from having Lot 3 in house. It seems UNICEF’s ‘own health activities’, in particular in a context like Somalia, where it has a long track record of extensive support to a variety of health activities and health facilities, can benefit from an improved ‘health system perspective’. UNICEF may therefore want to increase its capacity in this field.


7

1 Preamble

1.1 Principal features of the project It is tempting to give a title to this final evaluation: ‘moving goal posts’. Such a title reflects some key characteristics of the project. Right from the beginning, the project was subjected to a range of changes in its design. During the implementation, continuous changes in the project’s context, primarily the deterioration in the political and security situation in large parts of Somalia, made further changes in approach and activities mandatory. For observers of the project it was therefore not always easy to keep track of the project’s direction. But above all, this title is meant as a tribute to the project management for the constant search for ways to keep the project meaningful and, despite the extremely difficult circumstances in Somalia, to keep aiming for improved strategic leadership for Somalia’s health sector with the health of all Somali people in mind.

The full name of the programme that will be evaluated in this document is “Support to Health Sector Development in Somalia – Capacity Building, Strengthening System Coordination and Performance”. However, among stakeholders in Nairobi and Somalia, it is much better known as the Lot3 project. While Chapter 3 will have more explanation on the origin of this name, this report will refer to the Support to Health Sector Development project as ‘Lot3 project’.

The Lot3 project has been co‐financed by the European Commission for 2 Million Euro and UNICEF for 114,000 Euro and implemented by UNICEF. Part of the agreement was to have this final evaluation done. The Lot 3 programme was planned to last 2 years, from January 2007 till December 2008, but de facto ran from July 2007 – July 2009. The project was managed from the UNICEF Somalia office in Nairobi, from where the various parts of Somalia could be reached.

Following independence in 1960 and the unification of Somaliland with the rest of Somalia, a military coup in 1969 initiated what became known as the Siad Barre Regime that would last till 1991. The latter years of the regime were characterised by increased chaos and civil war acts, like the bombardment of Hargeisa in Somaliland. By 1991, the Somali government had virtually collapsed, interclan fighting continued and Somaliland declared –internationally never recognised‐ independence. Currently, Somalia is divided into 3 zones Somaliland, Puntland and Central and Southern Somalia (CSS), with de facto separate administrative regimes, including three Ministries of Health (MoH).

Somalia has a rudimentary health system, largely consisting of more or less functioning, town‐based hospitals. A network of PHC facilities were developed with donor financing during the Barre regime and are still visible today. This network consists of district hospitals (almost totally non functional), health centres, called MCH clinics, and health posts. Very poor health indicators, poor stewardship of the public health sector, serious lack of skilled health workers and a very poor financial base complete this bleak picture. On top of this, throughout the


8

country and in particular in CSS, pockets of populations facing emergency conditions can be identified every year, usually linked to insecurity, displacement and food shortages.

With extremely limited public health expenditure from domestic revenue and an estimated per capita health aid financing in 2006 of US$7, Somalia’s health sector is grossly under‐funded1. These external funds for the health sector come from EU Member States (26% in 2006), the UN (25%), Global Fund (22%), EC/ECHO (10%) and some other, smaller sources. The bulk is humanitarian assistance, short time bound and localized service delivery projects via NGOs and some larger support and supply programmes through the UN agencies. Aid financing to the health sector, especially from 2004 onwards, favours vertical programs: Polio, TB, HIV and Malaria accounted for 50 percent of total aid in 2006.

Lot3 project activities were to take place in all three zones of Somalia, with additional activities in Nairobi, where most of the coordination for the health sector takes places through the Somali Support Secretariat that supports 5 pillars of sector coordination including the Health Sector Committee. In the Lot3 technical proposal2 the overall objective was formulated as ‘to contribute to the rebuilding of the national health care system in Somalia’, through the development and implementation of sector policies and strategies and system building blocks. Components to achieve this were:

1. Comprehensive institutional assessment for central, regional and local administrations in place, covering: Organisation analysis, Human Resources, health care facilities and applied technology, financing systems and accountability

2. Phased multi‐annual capacity development plan jointly prepared and implemented (first phase), including sectoral planning, budgeting, monitoring, evaluating and coordinating health service delivery

3. Key instruments for Health sector management developed and applied, in particular: Integrated information system (with GFATM, WHO) and sector planning, coordination, M&E systems

4. Implementing Partners (in particular those in Lots 1 and 2) advised and minimum standards achieved

As will be discussed in later chapters these objectives were not achievable and were not realistic in the given context and even less so in the given timeframe.

Key to a better understanding of the dynamics of this project over time are the major changes in the context that occurred during the lifetime of the project. The project development phase occurred under assumptions that the situation in Somalia would improve towards stability and options for development. However, from 2007 onwards, conditions deteriorated substantially. Ever increasing insecurity made access difficult leaving no space for development in most of the areas.

1 ‘A Review Of Health Sector Aid Financing To Somalia (2000‐2006)’, World Bank report, August 2007

2 The formal technical proposal (The Action) as submitted by UNICEF Somalia to the Delegation of the EC in Kenya, Somalia Operations, dated November 2006.


9

1.2 Objectives and plan of work of the evaluation The Terms of Reference for the final evaluation have as objectives for the study ‘to provide UNICEF and the EC with sufficient information as to the:

• appropriateness of the proposal,

• feasibility of the action,

• suitability of the programme implementation strategy and

• impact of the programme

– with lessons learned for similar programmatic approaches both in Somalia and elsewhere in comparable contexts.

A particular focus of the study was to be an evaluation of the application and suitability of the approach to institutional development of the MoH, which was applied through the programme, and recommendations on how this could be strengthened and followed through in the future.

Annex 2 has a full list of the expected study results.

The ToR furthermore acknowledged that while the programme has been implemented Somalia wide, the current security context in Somalia will prevent the evaluator to visit all project sites and all implementation zones. In this regard, the evaluator will be expected to provide an overview of the implementation of the entire programme, based on a thorough in depth desk study, interviews/meeting with partners and stakeholders in Nairobi and accessible zones, and support this with more in‐depth insight into/assessment of implementation in zones and sites to which field visits can be undertaken (Somaliland).

The evaluation was conducted by Dr Egbert Sondorp, with a visit to Nairobi and Somaliland between 20th October and 11th November 2009. For the purpose of the evaluation a reference group was set up i) to ensure that the evaluator has access to and has consulted all relevant information sources and documents related to the project/programme; ii) to validate the evaluation approach; iii) to discuss and comment on reports delivered by the evaluator; and iv) to assist with feedback on findings, conclusions, lessons and recommendations. This reference group consisted of representatives of the European Commission, UNOPS, and UNICEF as well as the NGOs COSV and COOPI. The reference group met three times with the evaluator, an initial briefing, a second meeting to primarily discuss the proposed field visit and a final session with a debrief from the evaluator presenting the preliminary results.

The evaluator conducted a review of an extensive list of project documents and other documents around Somalia and its health sector. Throughout the visit the evaluator conducted 31 interviews with a range of individuals, 4 focus group discussions with a total of about 20 people, and several meetings with hospital and clinic staff during field trips in Somaliland. See annex 7 for a list of people met.


10

During the early phase of the evaluation the evaluator attended a 3‐day workshop which was organised by the project and included representatives from virtually all the stakeholder groups in Somalia’s health sector, both from the three Ministries of Health as from the international community. Even more than the interviews this gave a good impression of the dynamics of the Somali health sector strengthening efforts and its key players.

With Central South Somalia (CSS) and Puntland being out of bounds, the evaluator travelled to Somaliland, conducted more interviews with stakeholders in Hargeisa, and paid visits to Berbera and Boroma. In these three cities as well as in a number of villages along the way a range of health facilities were visited for observation and discussions with staff.

The evaluation was administered and facilitated on behalf of UNICEF by UNOPS. Field trips and interviews inside Somaliland were organised by the UNICEF field office in Hargeisa. COOPI organised the site visits outside Hargeisa.

The report will follow the EC standard reporting format for evaluation studies, as provided in the ToR as far as is relevant for this programme. Spread over chapters 2 (project preparation and design), 3 (relevance of project) and 4 (achievements) the design, changes in the design, implementation and concrete results of the projects are being described and analysed. After brief discussions on the efficiency (Chapter 5) and impact (Ch 6), Chapter 7 will have the final analysis underpinning the conclusions and recommendations of this chapter. With the overview of the project as given in Chapters 1‐7, a separate chapter (Ch 8) can then be devoted to the notion of appropriateness of the intervention and recommendations for future such interventions. This final chapter can be seen as a distinct section as requested for in the ToR.


11

2 Project preparation and design

2.1 Initial project design The European Commission (EC) has been providing support to the Somali health sector for over a decade. A ground breaking report commissioned by the EC in 2004 on “ Identification of Capacity Building Interventions for Local Health Authorities in Somalia”, prepared by Monica Burns, highlighted the long term costs of a perpetual short term, project focused approach to health system intervention in Somalia.

In March 2006, the EC launched a call for three proposals3, called Lot 1, 2 and 3 for support to basic quality health services within the framework of a structured and harmonized health system while addressing gender issues. The call envisaged improved access to sustainable basic quality health services; strengthened capacity of local communities and public administrations to plan, manage and monitor health services; improved management capacity in health services facilities; and development and application of minimum health sector standards, including health strategies, policies and guidelines. It is important to note here, that the EC, at the time of this call had decided to stop financing health sector activities in Somalia, once these 2‐year projects would have been finished. This decision was taken in light of ‘aid effectiveness’ principles that recommends reducing the number of sectors donors should focus on. For Somalia, the EC therefore decided to focus on governance, education and rural development/economic growth, and no longer provide support to the health sector.

Lots 1 and 2 were more traditional support to NGOs and a follow up of projects financed by the Italian government. These project were to ensure enhanced access to basic services by assisting specific public facilities/institutions (hospitals and MCHs) in a number of areas. Lot 1 was granted to the Italian NGO COOPI, with a focus on the hospitals in Boroma and Burao in Somaliland. This 2.1 M Euro project started on 1 August 2007 and, including a no‐cost extension, finished on 31 January 2009. Lot 2 was granted to another Italian NGO, COSV, and its partners (CISP and AAH) and was targeted at a number of health facilities in Puntland and CSS. This €3,277,811 project was to run from 1 November 2007 to 31 December 2009.

Lot 3 was an additional grant aimed at support across the sector and institutional development of the three Ministries of Health of CSS, Puntland and Somaliland, in order to try and assist traditional health projects (in particular Lots 1 and 2) improve on overall performance and to assist such projects to be part of longer term efforts to develop locally owned, planned and

3 ‘Support to Health Sector Development in Somalia’, Guidelines for grant applicants responding to the Call for Proposals 2006. Open Call for Proposal. IT Co‐ financing, 9th European Development Fund. Call for proposals number: Europe AID /122 798/ C / ACT/ SO


12

managed health care. More specifically, ‘Lot 3 will therefore aim to achieve synergy effects and improve networking between all stakeholders involved. Good, proven practices, verified at the point of delivery will be up‐scaled and introduced into policy development. It is expected that Lot 3 will capitalize on efforts to build on training facilities in Somaliland, Puntland and Central / South Somalia (incl. Mogadishu). Lot 3 will guide the other two Lots which will actively adapt to the guidelines developed in close consultation with all stakeholders, most notably the local / regional / central administrations ( Ministries of Health). Ultimately, interventions funded under Lots 1 and 2 will comply with standards and guidelines developed and will increase coverage in terms of modus operandi and practices’.

There were no responses to the call for Lot 3. The EC then entered into negotiations with UNICEF as a proposed implementing partner for Lot 3. UNICEF agreed to consider the call but on the basis of input by UNICEF into the purpose and aims of Lot 3. To that end a senior, head office based UNICEF staff member, with previous experience in Somalia, did an assessment leading to a set of recommendations laid down in a concept note4. The note refers to the above mentioned report by Monica Burns and a similar perspective in the 2005 Joint Needs Assessment, coordinated by the UN and the World Bank, and states that authorities in Somaliland and Puntland have managed to establish rudimentary de facto governance institutions that maintain relative peace and security in the areas under their control for over 5 years. It concludes that “No coordinated and strategic investment from the international community has been forthcoming to increase the capabilities of these nascent authorities across the governance system or sectorally”. The note is less positive about CSS, where still political and military manoeuvrings take place. The concept note then proposes an intervention by setting up “Health Capacity Building Units” in Hargeisa and Puntland. The note argues, that following the principles of early recovery in the transition from emergency to developmental activities, the availability of an important grant from the EC, earmarked for health sector capacity building activities, provides the opportunity to test the ability and the will of the international community to engage in a serious effort of building Somali capabilities to design and manage their national health system. The note then proposes to utilise the EC grant to establish and support a small multi‐disciplinary team comprised of public health experts, strategic planners, managers of human resources in both areas. The purpose of this capacity building unit would be to primarily act as facilitators and mentors of their appointed Somali counterparts by assisting in the following areas:

• collecting data to allow evidence based decisions

• establishing systems and standardised tools

• defining and implementing priorities

4 ‘Strategic Capacity Building for Health Sector in Somalia’; document not dated or signed.


13

The attainment of these objectives will in turn significantly contribute to the goal of the emergence of a national health system. Proposed outputs include:

• Consolidation of the multiple and fragmented Health Information Systems into one user friendly system for managers

• Inventory of available human resources that can be linked to training needs

• Establishment and implementation of Essential Health Package for first and second levels of care

• Establishment of an independent non‐profit Essential Drug procurement and distribution system

• Design a balanced and fair remuneration system for health personnel that links to performance

• Review the present cost recovery system and link it to the design and test of an improved financially sustainable strategy

• Strengthen and expand accountability mechanisms based on user co‐managing and co‐financing of the health system.

The note also sees a role for these units to be tasked with the responsibility to improve, standardise and support the congruency of health activities, particularly those funded by the EC, conducted by NGOs (Lot 1 and 2) and international agencies as outlined in the ECs call for proposals. Finally, the note realises that the proposed EC grant only covers 2 years of operations of the proposed Health Capacity Building unit, yet the scope and duration of such a unit requires a long term commitment to the process by the entire international community for a period lasting over a decade.

This concept note became the core of the full proposal which was developed afterwards, although other elements got added in during and after the proposal phase.

Looking back at this concept note, with the advantage of hindsight, a few comments may be made. Firstly, it takes a health systems approach by wanting to address a range of health system ‘building blocks’ in a coordinated fashion by strengthening the stewardship role of the Ministries of Health responsible for the health sector in their respective areas through a model of capacity building. To take that broader systems approach seems highly warranted at that time, against the background of cautious optimism regarding the conflict in Somalia and the problems created by the short term projectised and verticalised attempts to improve the health sector in the years before, as spelled out in the report quoted earlier (Burns report and JNA). It also reflects lessons learned from other transitional countries where often years of precious time get lost because broader health systems issues do not get addressed. Secondly, while the note takes a much broader health systems perspective than the initial Lot3 call for proposal, it still clearly sees a link between the Lot3 programme and Lot 1 and 2 projects as had been called for by the EC. In particular, the foreseen Health Capacity Building Units would have the


14

task to support and standardise the NGO activities. These Units never materialised and the final project proposal based on the concept note had an even broader and upstream remit. This was amongst the reasons why there was inherently little articulation between Lot3 and Lot1/2, once the Lot3 final proposal had been formulated. It is therefore not surprising that during implementation of Lot3 little support was or could be given to Lot 1+2. Thirdly, the note focuses primarily on the nascent health authorities in the different parts of Somalia. However, it is not clear if sufficient account was taken of the realities that already existed those days, in particular that health coordination was and still largely is taking place at Nairobi level, that (public) health services all over Somalia are in an extremely poor state with little uptake by the population, and that the existing capacities at the various MoHs were very limited. In this kind of environment, health system strengthening alone may not be very productive if it is not accompanied by the means to implement health services. Fourthly, the note does mention the short implementation period of the foreseen Lot3 project of two years, while it states that long term commitment from the international community will be needed for at least a decade. It does not consider the impact on this project in case this long term commitment would not come forward5.

2.2 The proposal Apparently this concept note received endorsement from the EC to form the basis for the development of a full Lot 3 technical proposal. The concept note was then developed into a full proposal mainly by staff in the UNICEF Health section. This was completed by the end of November 2006, with a foreseen starting date of January 2007, an inception phase of 3‐6 months, a first phase implementation of 3‐6 months, a mid‐term review, followed by another 12 months of implementation.

The overall objective was formulated as ‘to contribute to the rebuilding of the national health care system in Somalia’, and the purpose as ‘sector policies, strategies, system building blocks developed and implemented’. As results and main activities per result were listed:

Result 1: Comprehensive institutional assessment for central, regional and local administrations in place, covering: Organisation analysis, HR, health care facilities and applied technology, financing systems and accountability

• Create Health Capacity Building Units (HCBU) to support Ministries of Health across Somalia via contracting international institution

• Establish Steering Committees to support deployment and functionality of HCBU • Assess & recruit staff of the Health Ministries, through Capacity Building Units • Establish Links to local administrations to ensure allow bottom up participative cooperation

5 In fact the consultant recommended UNICEF to turn down the grant if longer term financing was not forthcoming and this grant be usd as a first step in a longer term continuous strategic and implementation effort. UNICEF management evidently believed there was sufficiently high chances of securing longer term investment in this area of work to enable them to go forward with the EC grant. This expectation proved wrong.


15

• Development of monitoring framework to gauge the implementation of the mentoring process • Compile information, conduct interviews and undertake analysis to develop assessment report

Result 2: Phased multi‐ annual capacity development plan jointly prepared and implemented (first phase), including sectoral policy and planning, budgeting, monitoring, evaluating and coordinating health service delivery

• Conduct Review to assist Health Ministries to develop and strengthen the following components of the health systems with full integration of other initiatives (in particular GFATM):

• Health Management Information System, • Essential Health Package, • Essential Drug Procurement, • Human Resource Development, • Health Infrastructure Development, • Health Management Technology, • Remuneration Policy for Health Personnel, • Health Sector Financing system, • Health Sector Regulatory Mechanisms Development, • Public Accountability mechanisms, • Monitoring and Quality Control systems

Result 3: Key instruments for Health sector6 management developed for progressive application where possible during first phase, in particular: integrated information system (with GFATM, WHO) and sector planning, coordination, M&E systems

• In collaboration with vertical programmes (GFATM), undertake development of Health Management Information System (HMIS) provide reliable data for decision making;

• Ensure planning for adequate and skilled manpower available for health service delivery; • Ensure information on essential drugs available in the facilities all the time.

Result 4: Support to the achievement of minimum standards by implementing agencies/partners (in particular those in Lots 1 and 2) provided in the following areas:

• Revise Essential Health Package • Develop training plan for medical staff • Develop Standard job descriptions • Develop and implement essential drug management system • Develop and implement quality control mechanism of laboratory services • Develop and apply health financing procedures • Develop minimum standards for health infrastructure and equipment

The full original logical framework for the proposal is reproduced in the annex 3.

The proposed results and activities obviously entail quite a bit more than initially foreseen in the concept note. Furthermore the extension of ambition was not matched by an extended period for development nor an extended and enlarged programme capacity and budget. However, the following statement still has the more narrow key objective of the concept note in

6 Including private health services provision


16

mind: “the approach taken in this proposal is a long term commitment to capacity building to be based on the mentoring and coaching of Somali health authorities by highly qualified professionals, in an intensified effort to address the development of critical competence, knowledge, skills and attitudes. The primary outcome of this process will be a pool of talented and empowered individuals with the capacity to assume leadership roles, plan strategically and to carry out key managerial, planning and developmental activities that will provide the basis for the strengthening of health systems”. A key proposed action to achieve this, is the establishment and support of an international team of highly‐qualified public health experts, to be based in‐situ in Somalia. Based on the situation at the time of writing of this proposal, Somaliland is suggested as a suitable location to start, followed by a similar unit in Puntland and Central and Southern Somalia, security situation permitting7.

Under various headings the proposal addresses felt uncertainties and risks around this project, in addition to the security issue. For instance, at its front page, it states the intention ‘to facilitate the full integration of Member States into the proposed Action’, which would then also contribute to a ‘joint critical review, after one year of operation, in order to facilitate necessary adaptations and allow for a longer‐ term sustained and structured development with full participation of all stakeholders on the ground’. The document also states that “the implications of the proposal outlined above are considerable and the dangers of failing substantive” but feels this is justified since “both the Somali authorities as well as the international community agree that the availability of capable Somali nationals is key to peace and reconstruction”. It also quotes the JNA Health Sub‐Cluster Report which contains a clear warning that ‘a meaningful health policy formulation process should be launched only later, when a proper political settlement has taken roots’ but, at the same time, this report also favours work to ‘carry out the proposed studies and discuss with all stakeholders their findings and the various policy options emerging from this essential analytical work’. Finally, the proposal lists a number of specific risk factors:

• The approach may be rejected by the Somali authorities • Qualified Somali professionals may not accept the positions set up in the Units • Adequate donor support might not materialize to continue and expand the approach,

thus diminishing the gains by the project in time. • Coordination mechanisms proposed in the proposal are not accepted by donors and

stakeholders A more detailed analysis of the original proposal indicates it was never fully completed or substantially negotiated between UNICEF and the EC. The budget structure does not represent the ambitions or modalities of the proposal and hence the budget is ineffectively constructed to allow completion of the proposal. The proposal process clearly “ran out of time”.

7 Which by the time of submitting this proposal is deteriorating, in particular in CSS.


17

Summing up the process that led to the design of the project as was in the end agreed upon between the EC and UNICEF. The initial Lot3 call for proposal aimed to address lessons learned from the past period with a ‘perpetual short term, project focused approach’, by wanting to set up an umbrella project to the Lot1 and 2 projects which would address some of the system issues that hampered implementation of those projects. UNICEF developed a proposal that gradually gave way to a much broader health systems approach. Through the concept note an emphasis on capacity building of the Ministries of Health was introduced, which then got expanded, partly influenced by the JNA report, with the intention to do a range of preliminary studies and the development of health sector management instruments. All this was formulated against a background of a limited funding period of two years, great uncertainty around buy in from other EC member states donors, which would be needed to guarantee viability of this longer term approach, and the general uncertainty around the political situation in most parts of Somalia, with its impact on security and governance. Looking back, the Lot3 proposal appears to be over‐ambitious for almost any context, in particular within the given time frame, let alone for the very fragile conditions as prevailed in Somalia. As will be discussed later, the Lot3 programme did manage to fulfil some of these ambitions, but virtually inevitably failed in others. The paradox is that the initial intention of wanting to get away from the ‘perpetual short term, project focused approach’ ought to be commended, but that the transformation of this wish in realistic programming proved difficult. The lesson to be learned is not to abolish attempts to do better in transitional contexts, with longer term, more system oriented programmes, but to learn from this attempt and accept it as a case study that will help the international community to improve its transitional, ‘early recovery’ programming. Globally, the latter is still insufficiently addressed, let alone operationalised8 and it is hoped that lessons learned from the Lot3 programme will significantly contribute to future work on these issues. There is hardly any published research on HSS strengthening in contexts like Somalia. The limited work that there is around HSS in fragile states, comes from ‘real’ post‐conflict countries, with a political settlement and substantial re‐engagement and investment by the international community. This is definitely not the case for Somalia9. Nevertheless, there is scope to put in place some building blocks prior to such a settlement and one of the key recommended10 actions is to timely collect and analyse health sector data, as became very much part of the Lot 3 project.

8 Bailey and Pavanello, Untangling early recovery, HPG Policy Brief 38, ODI October 2009

9 Obviously most outspoken for CSS, but the lack of international recognition as independent state and/or absence of clear governmental commitment to health also proves to make it hard for Somaliland to make any substantial progress on strengthening the health sector.

10 Pavignani, E, and Colombo, A, Analysing Disrupted Health Sectors. www.who.int/hac/techguidance/tools/disrupted_sectors/en/index.html


18

This also leads to the question if the proposal should have been approved at all. This may indeed be queried. From the interviews some factors emerged that may help explain why ‘this train was no longer stopped’. One factor was that the EC office in Nairobi no longer had a health adviser at the time this proposal was submitted. The other, and probably more pertinent factor, was the pressure to go ahead with Lot3, since otherwise allocated monies may no longer have been available. Also, the option to use the same amount of money over a longer period, which probably would have been a more valuable approach, was not feasible due to internal EC procedures which did not allow spending on Lot3, being part of a larger envelope, beyond 30 June 2009. Finally, the decision of UNICEF to put this proposal, as was developed by its own staff, forward to the EC, should be queried. The proposal was no longer technically sound and the project could realistically never achieve its objectives. In addition, the proposal had been developed under the assumption that ‘long term commitment from the international community will be needed for at least a decade’, which seemed highly unlikely at the time, with the EC withdrawing from the health sector.


19

3 Relevance of the project

The context in which the project operated constantly changed, which had a range of consequences for the project and led to quite a few changes over time, which will be described in this chapter

Partly foreseen from the start, partly due to implementation bottlenecks and partly due to the changes in the Somali context, the design of the Lot3 programme was subjected to regular review and adaptations, almost to the very end of the contract period. The programme suffered from a number of delays, initially in recruitment of programme staff and later on in getting the most visible parts of the programme, the various studies that were done, disseminated in hard copy format. Many of the other outputs, like input in improved coordination and ‘getting people around the table’ were by nature much less visible and therefore much harder to communicate to the wider group of stakeholders. This also makes it harder to evaluate the links between results and objectives. Another difficulty for the evaluation was the fact that formal project documentation, like the various amendments, new logframes and interim reports were not very consistent, with reports reporting against different objectives as stated in an earlier documents. The full extent of the ‘real project’ only emerged later through the various interviews11. At the time of the evaluation there was no final report of the programme yet. So, much of the less tangible outputs are judged on the various interviews. Most statements as will be used throughout this report could be triangulated and corroborated by other interviews.

For better understanding of later chapters where results and impact will be discussed, some of the changes in design and implementation will be discussed here.

An initial delay in programme implementation was caused by UNICEF’s inability to timely recruit and deploy a project manager. Instead of January 2007, when the programme was due to start, the programme manager only arrived by June 2007. This was one of the reasons why later on, a 6‐months no cost extension was granted for this 24 months programme, to June 200912, to allow as much time as possible for project implementation.

The arrival of the programme manager started the inception phase of the programme, most concretely leading to an inception workshop by early September 2007. The workshop brought together people from the 3 MoHs of Somalia, stakeholders from the international community and a number of external experts that could transfer lessons from elsewhere.

11 This led one of the interviewees come to a very negative conclusion on the project, not having met the stated logframe objectives. A justified observation, but not doing justice to the project that did achieve meaningful results, in the spirit of the project, but not well expressed in the project documentation.

12 While both the project and the EC office in Nairobi would have been in favour of a further no cost extension, this was not possible due to internal EC processes. See chapter 2.


20

The aim was to generate an accepted, prioritized list of actions to support provision of effective and equitable services to the people of Somalia, to be achieved through emergency operations, and more standardised development programmes and/or government‐managed services. Attention was also paid to cross‐cutting issues of financing, information and coordination. A report on available external finance for the health sector13 showed that Somalia is a typical ‘aid orphan’. And while overall funding for health had increased from 3 to 7 US$ per capita, there was a marked shift to vertical programmes, in particular polio, HIV/Aids, malaria and TB, with less money available for health system and general service delivery. The participants agreed on a limited set of actions for the next two years, focusing on (i) specific actions in the development of central institutions, (ii) development of human resources for health, and (iii) delivery of basic services.

Part of the rationale for the workshop was that “given the current situation, with no political solution in sight and no major increase of funds foreseen, the Lot 3 project as originally designed has become less relevant. There is a need to re‐orient Lot 3 towards a more limited and useful set of actions prioritized by current health actors and their relevance”. So, the workshop was

13 ‘A Review Of Health Sector Aid Financing To Somalia (2000‐2006)’, World Bank report, August 2007


21

meant as a participatory way to re‐orient Lot3. And, since then, the project by and large focused on these activities. This is also reflected in later, approved amendments to the project. Annex 4 has the logical framework belonging to the 2nd amendment14. To get easier grasp of the differences between the initially foreseen project and the amended approach the table below shows the ‘intervention logic column’ of both logical frameworks.

Comparison ‘intervention logic’ original logframe vs logframe after amendment 2

Intervention Logic

Original Logframe

Intervention Logic

Logframe Amended 2 1. Overall Objective

To contribute to the development of human and institutional capacity and competence, crucial for the rebuilding of the national health care system in Somalia.

To contribute to the development of the capacity of the health sector (government, private and public providers) in Somalia in order to promote greater efficiency, effectiveness and equity of health service provision and to promote longer term strategic analysis and reform for the sustainable re-modelling of national health system(s)

2. Purpose

Sector policies, strategies, system building blocks developed and implemented

To contribute to improving the capacity, predictability and coherence of health decisions and service delivery options in order to improve current and future health and equity of the Somali people

3. Expected Results

RESULT 1 Comprehensive institutional assessment for central, regional and local administrations in place, covering: Organisation analysis, HR, health care facilities and applied technology, financing systems and accountability

RESULT A Basic functional MoHs

RESULT 2 Phased multi- annual capacity development plan jointly prepared and implemented (first phase), including sectoral planning, budgeting, monitoring, evaluating and coordinating health service delivery

RESULT B Normative standards for key resource and service delivery process defined and endorsed (MoH, HSC, NGO)

RESULT 3 Key instruments for Health sector management developed and applied, in particular: Integrated information system (with GFATM, WHO) and sector planning, coordination, M&E systems

RESULT C Lot 1- & Lot 2 (and other NGO)-health service delivery programmes actively involved in strategic development.

RESULT 4 Implementing Partners (in particular those in Lots 1 and 2) advised and minimum standards achieved

14 This logical framework and the accompanying plan of action for the second amendment are dated 28 February 2009. Possibly the date of formal approval, but must have been in operation over much of 2008.


22

Another major change to the originally foreseen project was its implementation mode. The proposal mentions “to utilise the grant to establish and support a Capacity Building Unit composed of a mixed international team of highly‐qualified public health experts to be initially based in a region where there is relative peace and stability. UNICEF will sub‐contract a well‐established institution with proven experience in capacity building in post‐conflict situations through an international bidding process. This institution will be responsible for the core team of experts, as well as a pool of specialised expertise from which the main team will draw as and when need arises”. This proposed sub‐contracting did not take place. Instead, the project manager implemented the project, assisted by short term external consultants hired for the various tasks at hand. It is not fully clear how this decision to induce a major change in implementation was reached. The evaluator got the impression, based on interviews, that sub‐contracting the bulk of the foreseen project to a third party through a competitive bidding process was not allowed under the EC grant conditions, under which the UNICEF project operated. However, it seems this would have been allowed and that other factors were more important to reach this decision. These factors included

• an expected long delay of several months before a third party agency could have been contracted and put to work,

• a lack of credible belief that after the failure of the open bidding process launched by the EC that credible partners could be found

• and the assessment that the budget would not allow to have both a number of experts and a larger pool of technical assistance these experts could draw on.

This major deviation in project implementation does not seem to have been extensively debated and explicitly decided upon between donor (EC) and implementing partner (UNICEF).

A more subtle change was the geographical focus. While the proposal envisioned an initial set up of the Health Capacity Building Unit in the most stable region, read Somaliland, the project in its first phase of implementation very much focused on all three regions. With a budget that included incentives and operational support for all three zones, and some pressure from the donor to work in all three zones from the start, simultaneous activities in all three zones were planned. And while operations on the ground in CSS never materialized and those in Puntland had to be stopped, the project continues to work on activities potentially relevant for all three regions, and maintains connections with all three MoHs. During the workshop in November 2009, during the evaluation, all three MoHs were present and presented themselves as speaking with one voice.

The idea of setting up Health Capacity Building Units in each MoH, through a team of experts roaming around between locations, was turned into an attempt to hire external technical assistance to be posted in each of the MoHs, while senior staff in all MoHs were to receive financial ‘top ups’. While it proved difficult to find suitable people for these posts, other events in the end prevented these Units to get established.


23

MoH in Mogadishu in CSS, the plan got stalled when both EC and UNICEF no longer considered this to be appropriate due to potential duplication with other multi‐donor funded projects and the limited territorial control and security constraints.

Puntland MoH investment got delayed through a series of kidnappings and lack of access by partners to implementation as well as a series of EC and UN high level political disagreements. This meant delays in project implementation while negotiations for resumed access and terms of engagement were developed.

In Somaliland some salary ‘top ups’ were provided to key senior staff in the centre and regional offices.Lot3 was about to place some experts inside the MoH. But right then, in September 2008, there was a major security incident in Hargeisa, including an attack on the UNDP compound. The aftermath of this incident meant that also Somaliland now became a security Phase IV area, which comes with restrictions of numbers of UN staff that can be on the ground. What remained was sub‐contracting some of the foreseen activities to NGOs, like THET, with a presence in Somaliland and it still proved feasible to have a full time expat in Hargeisa for some months to work on HR issues, followed by a leading HR expert to add strategic inputs.

So, from the initial project design as expressed in the UNICEF concept note prior to the development of a full proposal that put its focus on the development of Health Capacity Building Units for the various MoHs, the project has continually been adapted to the changing context. These changes resulted in an ever more constrained environment, where political and security developments increasingly diminished the space for meaningful programming with partners on the ground. The same developments caused the donor community to become even more hesitant to engage in the health sector, further undermining the notion that Lot3 could be a pre‐cursor to broader health reconstruction in years to come.

Instead, the project, apart from a range of HR, HMIS and Planning support activities with the MoH in Somaliland, started to primarily focus on a range of studies, that have been implemented and published, and a range of activities to strengthen coordination at Nairobi and zonal levels.

Given these dramatic external developments, should the project have been closed earlier, for instance after a mid‐term review? Or should it have been revamped even more drastically than was already done, for instance with more emphasis on direct service delivery? These –retrospective‐ questions are largely rhetorical. In these kind of environments changes do happen gradually, and with ups and downs, and always with glimpses of hope for improvement. This makes it almost impossible for programme staff and closely related other stakeholders to reach a decision to stop activities altogether. And, in relation to the second question, it should be realised that programme funding was almost fully allocated to pay for technical assistance and would not allow purchase of supplies for direct service delivery.


24

4 Achievement of purpose (Effectiveness) The evaluation’s effectiveness criterion looks at the (expected) achievement of the project results and project purpose.

In the previous chapters already a lot has been said about the continuous change to the project design, in an attempt by project management to adapt to the ever changing, and overall deteriorating context in which the project operates and opportunities that developed/closed for networking and learning. While they may not have agreed with all actions taken by the project, the great majority of interviewees agreed that Lot3’s project management probably did get the best possible results from the project, given the inherent difficulty of the project and its design itself, and the dire and deteriorating circumstances in which it had to operate. Many expressed that, where others would likely have given up, it was thanks to the endurance, flexibility and creativity of the project’s management that the project did manage to take important steps in achieving its purpose and overall objective, as expressed in the final logical framework. These results are vulnerable and clearly threatened by Somalia’s current turmoil.

The consultant agrees with the views given above and considers the greatest success of the project that it has shown that a broader health system perspective leads to significantly different activities in a protracted crisis like Somalia, unlike the ever ongoing isolated projects, that may serve a humanitarian purpose but do not contribute much to the build up of a future health system. In other, similar environments there have been calls to take this broader look, but this has hardly ever led to substantial programming. These projects are high‐risk and may not show immediate results. EC and Lot 3, having shown the feasibility of the approach, should be commended for having dared to take this on. Discontinuation because of lack of funding was foreseen, but highly regrettable.

The first year of the project, formally covering the full year 2007, only had activities for the second half of the year, after the arrival of the project manager. Orientation, inception workshop and amending the project were key to this period. As reported in the 1st year report, the activities are summarized in this table, still reported against the original ‘results’.

Results Indicators

1. Comprehensive institutional assessment for central, regional and local administrations in place, covering: Organization analysis, HR, health care facilities and applied technology, financing systems and accountability

• Limited organizational analysis completed.

• Organizational structures for all MoHs collected. Budgets collected and analyzed.

• Key persons to be supported identified.

• Key T.A needs identified.

• Discussions for the drawing up of


25

To take a closer look at the achievement of results and purpose during the full 2 years the programme has been running (June 2007‐09), it seems to make most sense to take the amended logical framework (see annex 4) as starting point. Here the purpose has been formulated as ‘To contribute to improving the capacity, predictability and coherence of health decisions and service delivery options in order to improve current and future health and equity of the Somali people’, to which three ‘results’ should contribute.

• Result A: “Basic functional MoHs” • Result B: “Normative standards for key resource and service delivery process defined

and endorsed (MoH, HSC, NGO)” • Result C: “Lot 1‐ & Lot 2 (and other NGO)‐health service delivery programmes actively

involved in strategic development”.

While this seems to be the last logframe as was submitted to the EC in February 2009 and approved in a letter dated 15th May 2009, it does not seem to reflect the realities of the project and seems to miss out in particular on important achievements of the project in the realm of health sector coordination and the development of a health sector framework. Also, consecutive project reports, like the 2nd year report and final report, do not report against the logframe results, but rather to –changing‐ ‘areas of action’.

To best reflect the elements as were initially formulated for Lot3 and the changing realities of the project environment and subsequent project adaptations, the project achievements will be

performance based contracts with ministries in 2/3 zones.

2. Phased multi- annual capacity development plan jointly prepared and implemented (first phase), including sector planning, budgeting, monitoring, evaluating and coordinating health service delivery

• Key priorities of 3 MoHs defined and recorded

• Restricted priorities of MoH, donors, UN and NGOs defined

• Workshop with donors, UN agencies, 3 ministries and NGO partners convened and defined a way forward.

3. Key instruments for Health sector management developed and applied.

Initial preparatory work launched

4. Implementing Partners (in particular those in Lots 1 and 2) advised and minimum standards achieved.

Various meetings held with COSV and COOPI as well as other key NGOs, GHC, SC-UK, AAH, Merlin,


26

presented here under four headings: support to the MoHs, health sector analysis and tool development, health sector coordination, and support to health service providers.

4.1 Support to the Ministries of Health Support to the CSS MoH was very limited. Salary support proved not possible since that would interfere with a comprehensive Governance programme targeted at the Transitional Federal Government in Mogadishu. In addition, access to Mogadishu further declined to practically zero. However, throughout, there was communication with MoH officials at Nairobi level and they were kept up to date about the analytical work and tools developed in the project.

Support to the Puntland MoH was planned to be similar as for the Somaliland MoH, but the proposed package of support never materialised due to worsening security situation. Access, however, remained possible, and hence involvement of MoH staff in discussions and tool development could take place.

Links with the MoH in Somaliland, formally the Ministry of Health and Labour, were more extensive. Salary top ups could be implemented for 12 senior staff in central and regional key posts – arranged under an innovative and potentially capacity developing performance based contract with the Ministry. Furthermore the programme of salary support was systematized and coordinated with other investments in MoH staff salaries to try and enhance coherence and impact of the overall salary interventions into the health system.

MoH professional staff was systematically engaged in the development of various tools, and formally endorsed some of these, and some technical consultants could work inside the MoH for an extended period. In particular,

• The Human Resource Development (HRD) department could be strengthened, which gave it a new role (as HR reform became central to an appreciation of the challenges of health system reform) and a new policy on HR Management and Development was formulated and endorsed.

• The HMIS unit was supported technically and financially (with GF Malaria money but under management of the Lot 3 project). The project involved considerable support from an NGO (CCM) providing a budget and technical assistance which led to fairly rapid production of valuable health information as well as the birth of a nascent basic standardized health information reporting, analysis and use system.

• The Planning department benefitted from financing for the establishment of quarterly coordination meetings – with considerable technical support. These meetings were invaluable for the launching and testing of tools developed in the programme and gave the planning department a task and an agenda to respond to. Furthermore, specific technical meetings on implementation of the EPHS – facility prioritization, referral management,


27

decentralization, supervision and training all helped establish the core challenges for health sector development.

• A small but critical investment in review of public drug procurement and supply was crucial in re‐setting ground for MoHL understanding of the agenda for development of public procurement and supply.

Among the tools, developed under Lot3 and adopted by the MoH in Somaliland were the Essential Package of Health Services (EHPS), costing of the EPHS, new standardised salary scales, and costed facility blueprints to guide health facility reconstruction. The latter was also linked to a workshop held on health facility prioritisation.

The MoH started to host a series of coordination meetings, supported by the project. In Somaliland 6 out of 8 planned quarterly coordination meetings did materialize and in Puntland 2 out of a planned 4 MOH coordination meetings were held. They were pre‐announced and well attended, primarily by local staff of NGOs and MOH and private sector. Attendance by Nairobi based senior managers of the HSC, NGOs or UN agencies, however, was very low. Meetings were reportedly lively, focused on HMIS and minutes were prepared. The meetings were increasingly led by the MoHL. However, the momentum was not retained beyond the life and support of the Lot3 project and these meetings were no longer held in this format. During the evaluation’s visit to Somaliland, several interviewees felt that health coordination for Somaliland still hardly exists and should be improved.

The project’s initial key emphasis to set up Health Capacity Building Units, or, in the modified plan, to place external technical experts (HR, Planning, Admin and Finance, HMIS) inside the ministries by and large did not materialize, despite initial attempts and energy spent to make this happen, as described in an earlier chapter. Exceptions were the input of two HR consultants who stayed in the MoH for some months and did lead to improvements, while attempts to place consultants to the Department of Finance and Administration and that of Planning did happen, but failed to complete the ToR due to a variety of reasons.

What strikes, in the case of the Somaliland MoH that could be observed, is the extremely limited ‘allocative power’ of the MoH. With an allocation from the ‘national budget’ for health of 2.5%, most of which is tied to payment of (meagre; typically 30‐35 US$ a month) salaries to health staff, there is hardly any space to direct the health sector through budgetary allocations. Health implementation in the region is largely dependent on international and private (including diaspora remittances) initiatives and resources. Capacity, incentive and power for the MoH to really take up a stewardship role of the health sector is very limited and unlikely to improve until broader governance issues and stability have been addressed with new forms of international engagement. Only then public revenue may be directed to health and placed under the stewardship of the MOH. This may imply that a solitary emphasis on capacity building of the MoHs would have been misplaced.


28

4.2 Health sector analysis and tool development A good part of the project’s budget was used to hire a range of external consultants to assist the project and other stakeholders to develop a range of analytical reports and tools. The totality of these reports has become an impressive range of key documents that may prove to be of major value for years to come. Again, current circumstances are not conducive to make full use of them now15 – but they may form the core of a coherent agenda for reform and development of the health sector and provide the foundations for a strategic engagement by donors and implementing agencies.

If the situation improves this background work will be important to kick‐start more extensive health reconstruction. It is this kind of preparatory work that looks at broader health system issues that is not often done during a protracted crisis, but may prove of immense value once the crisis subsides. This leads to the suggestion of the importance of widespread distribution in paper and electronic format to make them last as long as possible. A UNICEF led health sector reform programme from 1997 – 2002 (predominantly in Somaliland) left a series of strategic documents that were very much in evidence until the beginning of Lot 316. Likewise Lot 3 has left a legacy and while the reports were not published during the evaluators visit – it is clear there are efforts to professionally edit and publish these reports as well as widely distribute them and make them available on websites so as to ensure their longevity.

These published reports may also be considered the most visible outputs of the project but this visibility during the evaluation was hampered due to unforeseen long delays in printing hard copies, also for the stakeholders in the final workshop. However, the reports have now been made available and widely distributed as well s being posted on the UNICEF website.

15 The extremely difficult circumstances as they currently prevail in CSS also prevent full engagement with Puntland and Somaliland resulting in very limited external support to the health sector.

16 However, Lot 3 project documentation did not build on these documents and they were not available during the evaluation.


29

Above, two examples of covers of the published series of reports. A full series is listed here17

Reports published by UNICEF under the Lot 3 programme Report 1: Health Systems Strengthening for the Somali People. A workshop report, September 5th –7th 2007 Report 2: An Essential Package of Health Services. Somalia and Puntland 2009 Report 3: An Essential Package of Health Services. Somaliland 2009 Report 4: Intermediate Standardized Salary Support/Incentives: Payment scales for civil

servants and health workers. Somaliland Report 5: Intermediate Standardized Salary Support/Incentives: Payment scales for civil

servants and health workers. Somalia and Puntland Report 6: The Revolving Drug Fund. Bringing the “public” back into the Somaliland public

hospitals. Hargeisa Group Hospital, Somaliland Report 7: Steps Towards Harmonizing External Support for Health Care Provision for the

Somali People Report 8: Exploring Primary Health Care in Somalia. MCH Data ‐Somalia & Somaliland 2007 Report 9: Costing out the EPHS Report 10: Health Care Seeking Behaviour in Somalia: A Literature Review Report 11: The Private Sector and Health: A survey of Somaliland Private Pharmacies Report 12: Construction Guidelines for Basic Health Facilities. Somaliland and Somalia Report 13: Health Systems Strengthening, Somalia. A workshop E‐report, October 2009 Report 14: A report on Human Resource Development is in progress

Two further reports were developed with partnership of UNFPA and WHO: • A situation analysis of reproductive health in Somalia • Reproductive Health: National Strategy and Action Plan 2010 – 2015

17 Also reproduced in annex 5, together with relevant ‘unpublished reports’, for future reference. With, in annex 6, the consultants who have contributed to these reports.


30

A specifically significant achievement has been the work around the Essential Package of Health Services (EPHS) (reports 2&3) and its costing (report 9). Across four levels of services provision, the EPHS lists and provides details on:

• Six core programmes: 1. Maternal, reproductive and neonatal health 2. Child health 3. Communicable disease surveillance and control, including watsan promotion 4. First aid and care of critically ill and injured 5. Treatment of common illness 6. HIV, STIs and TB

• Four additional programmes:

1. Management of chronic disease and other diseases, care of the elderly and palliative care

2. Mental health and mental disability 3. Dental health 4. Eye health

• The 6 management and support components

1. finance 2. human resource management & development 3. EPHS coordination, development and supervision 4. community participation 5. health systems support components 6. health management information system

The purpose of the EPHS is to act as the prime mechanism for strategic service provision of the public sector health service. It is to help clarify health priorities and direct resource allocation. It defines MoH responsibilities and activities at central and regional levels, particularly in coordination, management and supervision of services. It clarifies the role communities play in creating a sustainable and accountable health system. It aims to address current poor access to health and inequalities in health service provision. It provides a road map for action, and is costed to enable detailed budgetary planning, for advocacy purposes and for the government, donors, municipalities, districts and communities to plan how to increase their contribution.

The EPHS helps define health systems standards for the government, UN and NGO agencies and private service providers. It standardises and improves upon existing logistical and supply systems, and adopts essential drugs and equipment lists for each level of provision. The accompanying costing exercise uses a range of scenario’s and may be helpful for future planning and cost‐estimates.

All stakeholders interviewed highly valued the work done on the EPHS. The standardisation of services helps planning a range of activities and apparently a range of agencies are actively using the EHPS in designing their health activities. It indeed provides a ‘road map for action’ as was intended. However, there is not yet much in the way of implementation of the EPHS. A sheer


31

lack of money for health implementation may be the key reason, in addition to the general deterioration in the political and security situation. Another reason may be that the EPHS in its full extent must need to be phased in. To match available funding, while still aiming to attain maximum health gains, packages that are even more basic may have to be formulated as a starting point. UNICEF has begun to do this – as a major operating partner – by identifying which MCHs would best be upgraded first, reviewing drug kits and supervision support as well as standardised curricula for refresher training of MCH staff.

At this stage the EPHS acts as a beacon and hopefully it will be used to have at least some small scale pilot projects where the package can be implemented in a suitable geographical entity. A forthcoming new DFID funding health programme may take this on.

The EPHS costing exercise provides useful trend data and an indication of overall costs. However, since the full EPHS cannot be implemented from scratch, it is more likely to start with a more basic package, to be piloted and scaled up later. The piloting will also need to do costing under real life conditions.

The ‘Intermediate Standardized Salary Support/Incentives: Payment scales for civil servants and health workers’ report (4&5) have also been well received and partly endorsed by the MoHs, and several NGOs use the scales provided. However, proposed payments are considerably higher than MoHs can afford within the current budget constraints. So, also here, broader implementation is awaiting much more encompassing improvements in the health finance situation. At the moment it is even the case that MoHs can not employ staff newly graduating from their nursing school due to lack of budget space and the absence of a pension scheme that would open the possibility to lay off old or redundant workers.

4.3 Health sector coordination In line with the broader aim of the project to strengthen the Somali health sector by looking at broader strategic issues, there was considerable input at the health sector coordination level. The former Somalia Aid Coordination Body (SACB), now CISS (Coordination of International Support to Somalia), includes a health coordination platform, the Health Sector Committee (HSC). Under the HSC a number of working group and taskforces operate, including a working group on Health System Strengthening. Lot 3, specifically through its project manager, provided input that proved to be, according to one of the interviewees, a ‘catalyst for coordination’. For instance, there was attention for due process within the coordination and the agenda setting. Also, the HSS working group was chaired by Lot3 project manager, and there was work on the task force to develop the GAVI HSS proposal and the task force on human resources. Through this position leading to direct inputs into the proposals, a number of new funding initiatives could be influenced using the background knowledge and on the ground verification obtained through the Lot3 activities. For instance, the following funding initiatives received inputs: the Global Fund Round 6 Malaria HMIS project was managed under the EC Lot 3 HSS programme,


32

the Global Fund Round 8 HSS18 section, the HSS proposal submitted to GAVI and the newly proposed DFID HSSP 2 proposal.

One of the studies Lot 3 contributed to (see previous sub‐chapter) was published in the “Steps Towards Harmonizing External Support for Health Care Provision for the Somali People” (HOAP), which recommends a process of development of harmonized funds, information, tools and management of funds. An important first step for this process would be to have joint, harmonized information. The recommendation to set up a Health System Analysis Team may materialize in the coming year.

This work at the coordination level may be less visible than for instance field work or the production of a range of studies, but was seen by a number of interviewees, in particular those that are close to the coordination platforms, as a key role of Lot 3. It provided a strategic leadership role for the sector that no one else was taking on, and its continuation is of great importance. This role was possibly never really foreseen in any of the project documents and was, in fact, pretty informal and partly personality driven. Nevertheless as an investment in evidence, strategy and direction this project was always a “harmonizing” project and to be successful had to influence allocators of finance as well as implementers of services. The contribution to coordination should not be seen as an unintended benefit but should have been more specifically recognized and worked up from the start. At the moment, there does not seem to be a single agency who can take up this role more formally. It can only be hoped that there will be programmes in the health sector that provide staff who will be able to provide the necessary leadership to harmonize external support and improve coordination (Possibly around the HSAT or the new DFID HSSP2, which will collaborate with Sweden to bring a senior health advisor on the scene).

4.4 Support to health service providers. Under this heading a number of different activities are grouped together that have in common that they intended to support or study people or agencies directly involved in the provision of health care, other than the MoHs.

Lot 1 and 2

As described earlier, the Lot 3 project was initially seen as an umbrella programme to Lot 1 and 2, formulated in the initial Call for Proposals as: ‘Lot 3 will guide the other two Lots which will actively adapt to the guidelines developed in close consultation with all stakeholders, most notably the local / regional / central administrations ( Ministries of Health)’. Lot 3, as it evolved,

18 Although an advice to more drastically alter the proposal towards more meaningful HSS activities, as judged by Lot3 and some involved donor agencies, was disregarded.


33

increasingly moved away from Lot 1 and 2. The concerns of Lot 3 with broader strategic issues in the health sector proved removed from what Lot 1 and 2 had expected from Lot 3, i.e. to be an interface between their projects and, primarily the MoHs. While support to or involvement of Lot 1 and 2 remained part of the language in consecutive project documents like the logframes and plans of action, the reality was that there was hardly a specific link. The initially established steering committee, with all three Lots, the EC Somalia Operations Unit and the Italian Cooperation was not productive and soon ceased to function. Of course, like other agencies, Lot 1 and 2 did benefit from work on the EPHS and the Salary Scales, which were both seen as positive and useful developments by Lot 1 and 2 managers. Nevertheless, the interviews showed that among the Lot 1 and 2 agencies there had been higher expectations from Lot 3, and, while it was understood that the project had already changed before it started, there was still some resentment. Understandably, since these projects with their primary focus on support to hospitals also operate in the vacuum that is the Somali health sector. The hospitals supported tend to be (semi‐)autonomous, with only a few having the right mix of staff motivation, financing options and community involvement to make them viable entities that really contribute to the health of the population they serve. Possibly, the follow up project to Lot 1 and 2, financed by the Italian Cooperation and implemented by UNOPS will have inbuilt elements to strengthen the autonomy of the hospitals they will support.

Unicef health programme

During the evaluator’s field visit a range of primary health facilities was visited. And while some were lively, others appeared to be highly under‐utilised despite staff and drugs being present. Obviously there is a lack of trust and demand to use the facilities. That’s seen more often in protracted crisis areas with under resourced health facilities. However, what is surprising that the facilities are still there and in (low level) operation. This ‘resilience’ of MCH Clinics and health posts may very well be a consequence of the continued delivery since many years of drug kits to all facilities by UNICEF. It does not come with much training or supervision of health staff, but this –primarily logistical‐ operation seems to retain a network of primary care facilities that may be revived if ever conditions may become more conducive to improve standards and uptake of quality services.

Lot 3, being based in UNICEF, had some bearing on UNICEF’s ‘own’ health activities and may lead to a more strategic approach of these activities. One of the activities was an overdue revision of the drug kits. The Child Health Days as organised and implemented by UNICEF were formulated and designed by the Lot3 programme manager. Through these Child Health Days all Somali children are targeted to receive a number of health interventions, twice in a year, with the aim of having an immediate health impact are an interesting example of a relatively novel approach for these settings with a set of pros (e.g. direct health gains) and cons (e.g. disruption of other health activities) that are very worthwhile to carefully evaluate and give a place in an overall health strategy, for Somalia and maybe elsewhere.


34

The establishment of a country wide HMIS system is a remarkable achievement for a country like Somalia. Data from all over the country reach the central database that is managed by UNICEF. The project to develop HMIS is funded by the Global Fund for Malaria. While the fund emphasized the need to collect basic PHC data on malaria, the objective was to enhance all PHC facility reporting and build a more effective data set on PHC system utilization and impact. The HMIS project was placed under the management of the Lot 3 PM. This was beneficial for a number of reasons:

(1) It is unlikely that UNICEF would have directed the HMIS project practically and effectively outside of the guiding framework of the overall HSS programming. UNICEF managed to develop a comprehensive MCH data base and report – but the programme also took on the reform and standardization of hospital HMIS (outside of UNICEF programming concerns), which is a key step to reform of the hospital sub‐sector.

(2) The HSS programme very much benefitted from the production of practical decision oriented data. The data started to play a role in coordination and their use by programme managers in UNICEF and some service delivery agencies lead to enhanced management of programmes.

Private sector

With such a weak public sector in a country known for the entrepreneurship of its citizen, it makes sense to consider the private, for profit and non‐profit, sector involvement in health. Among others, this led Lot 3 to commission a study into the role and quality of Somaliland’s ubiquitous Private Pharmacies.

While too high expectations regarding the impact of support to the pharmacies to enhance their quality of prescriptions must be tempered, there seems scope to experiment with forms of social franchising to achieve quality improvements. There does seem to be some cautious investment into experimenting with the private sector by UNICEF and DFID HSSPII in partnership with PSI in Somaliland and guided by this Lot 3 research.

Lot 3 also built up links, among others in collaboration with THET, with a range of people, local health institutes and non‐governmental organisations that work outside the remit of the MoH. These include leaders of private hospitals, doctor and nursing associations, Hargeisa Group hospital, Edna Aden maternity hospital and Manhal Group, as well as the Health Professionals Council. Together with other civil society groups these individuals or institutes may act as agents of change for the health sector and be part of a broader definition of health governance in Somali society.


35

5 Efficiency The Lot 3 project was co‐financed by the EC for 2 Million Euro and UNICEF for 115,000 Euro and implemented by UNICEF. Due to the changes in programme implementation, the budget was extensively amended in due course. In the end, only 1.31 Million was actually spent

Money initially allocated in direct support of the various MoHs, through salary top ups of senior staff and placements of external experts in the ministries, could not be spent, since only support to salaries and placement of staff in the Somaliland MoH materialised.

Since over a quarter of the budget was allocated to these activities, in particular the placement of expatriates, funds had to be re‐allocated. Part went to more extensive use of consultants, either because studies proved more expensive or were done by two consultants instead of one, but also more studies were done than initially foreseen. Another part went to rehabilitation of a drug store at the Hargeisa Group hospital (13,300 Euro) and to rehabilitation and extension of the MoH office in Hargeisa (119,200).

Two issues came up related to the efficient use of monies that were available for the project. The first one was the question of a number of interviewees if UNICEF could not have hired more permanent staff, next to the project manager, to share the burden of the work and/or to follow up one particular portfolio, for instance human resources, instead of the use of short term consultants. According to project management this was not really considered, partly because it was not feasible due to internal UNICEF procedures that would not allow expansion of the UNICEF office and partly since it was felt that specific, but short term expertise was needed for the various studies and tool development. Retrospectively, it would have made sense to have had at least one more permanent team member to share the workload.

The second issue raised was the relatively high pressure to do the work in only two years. A much longer project would have been feasible, with the same amount of money, with more time to work on the development of health coordination and a health strategy framework, and more time to sequence, disseminate and act upon the various studies. Apparently, this was not feasible due to internal EC reasons, as this project was part of a wider envelope that could not be extended. At the rate of disbursement, the project could have been managed for 4‐5 years with significant increases in impact and influence.

This means that a HSS programme like Lot3, spread out over more time, can be effective and is not very costly. Hard to measure, but it is plausible that improved coordination and strategic health investments, based on better evidence, would lead to better (health) outcomes without necessarily increasing costs for the international community at large.


36

6 Impact

The project purpose, in the last available logframe, was formulated as ‘To contribute to improving the capacity, predictability and coherence of health decisions and service delivery options in order to improve current and future health and equity of the Somali people’, which was then to contribute to the overall objective: ‘To contribute to the development of the capacity of the health sector (government, private and public providers) in Somalia in order to promote greater efficiency, effectiveness and equity of health service provision and to promote longer term strategic analysis and reform for the sustainable re‐modelling of national health system(s)

The project had a major real time impact in terms of:

• Developing an HMIS data base for MCHs and hospitals, which has allowed a broadly coherent reform strategy to begin to emerge.

• Developing a reform strategy for the PHC system (drugs, staffing, training, supervision) that UNICEF is now actively trying to address as part of its country programme19.

In other aspects, the project will not yet have achieved many tangible results in terms of the capacity of the government or others for actual improved service provision. This would be too early anyway, after such a short project period, but the conditions were diverse to expect this to happen anyway. In terms of the promotion of longer term strategic analysis and reform, definitely a number of important steps have been set, in particular through the conduct of the various studies and improved health coordination.

If positive change in the overall Somali context will occur over the next few years, the project is likely to still have an impact through the preparatory work now done. In the mean time, smaller scale experimentation can take place to implement a number of the recommended action in the studies. For instance,

• Implementation of the EPHS would yield useful additional lessons and is likely to occur through the DFID HSSP II grant and GF R8 HIV grant.

• More attention to strengthen civil society and possibly the private sector as a way to strengthen health governance may also have beneficial long term impact. This is also likely to happen under the DFID HSSPII grant.

• The development and implementation of the GAVI HSS and GF R8 and R10 HSS components/proposals.

The longevity of such a short project and the necessity to fully role out some of the interventions when the situation is more conducive places special emphasis on the need to

19 Of particular relevance will be the introduction of a new drug kit system in January 2011.


37

“concretize” findings and make them available for future generations. To this extent Lot3 acted to professionally publish a series of reports and has widely distributed them and made them available on the UNICEF web site.


38

7 Conclusions and recommendations

7.1 Overall outcome The overall conclusion of this evaluation of the Lot3 project is that it was a project that was largely successful, while ‐paradoxically – it probably never should have been approved, with over‐ambitious, unrealistic objectives, to be achieved in a short, unrealistic timeframe. This conclusion would still stand if the overall political and security context would have remained the same or would have gradually improved (from the design phase in 2006). Lot 3 management, fortunately, managed to create space to redirect the project, while retaining its underlying philosophy of the pursuit of broader health system analysis and capacity building.

This project was one of the first of its kind and similar projects in future will obviously have to be formulated more realistically. These projects will never be easy to implement and require experienced, senior staff, not always easy to find and recruit. Sufficient support should be built into the project design, from within the implementing agency or as external advisers.

This project was formulated in a period of unsubstantiated political optimism which over influenced the project formulation and made it less pertinent to the realities of the Somali health sector. In the event, the context continued to deteriorate making it even harder to achieve the objectives, while reducing the willingness of donors to invest in the Somali health sector and build on some of the foundations the project was nevertheless able to create.

A few themes that emerged around design and implementation of the Lot 3 project deserve some more in‐depth discussion, followed by a brief not on sustainability of the project and a number of specific recommendations.

7.2 Themes

7.2.1 Project design As had been described earlier in the report, in particular in chapter 2, the project, and its very name, originated from a Call for Proposals by the EC to provide higher level support and guidance to two NGO run service delivery projects, Lot 1 and 2. Apparently, there was no agency available or interested to submit a bid. The EC then approached UNICEF, probably based on its long‐standing track record in the Somali health sector, with a presence on the ground, to take on Lot 3. UNICEF agreed, provided there was room for an own interpretation of the underlying ideas of the ToR for Lot 3, i.e. a broader, more strategic approach to the Somali health sector, away from the ‘perpetual short term, project focused approach’. This resulted in a concept note produced by a senior UNICEF staff member after a short visit.


39

The concept note does take on this broader health systems approach and aims to do so by strengthening the capacity of the three MoHs by setting up Health Capacity Building Units. The idea was that Lot 1 and 2 could be supported from these HCBUs. The suggested approach seems sound enough and could have worked, both in terms of capacity strengthening at the MoH level leading to improved sector leadership as support to Lot 1 and 2. However, it assumes gradual overall political improvement in Somalia and seems to seriously disregard the very poor state the health system and health ministries are in at that moment of time. However, looking back, what surprises most is that the concept note acknowledges that the suggested approach will need concerted efforts for at least a decade, while knowing that the EC would only fund this project for 2 years and would exit from support to the Somali health sector. Assuming that monies would be available for the continuation of the project, necessary to make it a viable initial investment, must even at the time of formulation have been highly questionable.

It made sense that on the basis of the concept note UNICEF started to formulate a full proposal, since the concept note was still close enough to the initial CfP. However, it seems that the final product was insufficiently scrutinised before it was approved by the EC and a grant contract issued to UNICEF. This final technical proposal as more fully described in chapter 3 is not only over‐ambitious, but very unrealistic within the given time and budget frame. In particular in the given, difficult context of Somalia, a situation that was again deteriorating at the time this proposal was completed. Obviously, as can be judged from the draft nature of the proposal as was submitted by November 2006, there was time pressure, in an attempt to get the project contracted off the ground by January 2007. The prospect for the EC Somaliland operations to loose the allocated money if not contracted before 2007 may have contributed to this pressure. Also, by the time this technical proposal was submitted, the EC no longer had a technical adviser in Nairobi to critically look at the feasibility of the project.

This project should not have been approved, given the circumstances in Somalia and the extremely short time frame that was given by the decision of the EC to withdraw from the health sector, in combination with a bureaucratic constraint, internal to the EC, that did not allow for any no‐cost extensions beyond June 2009.

The other side is that UNICEF formulated the project and took it on to implement. It can be assumed that also within UNICEF this proposal was sufficiently scrutinised prior to submission to the donor and acceptance of the contract.

7.2.2 Implementation The two year project started with a half year delay due to late arrival of the project manager. After orientation, the project manager soon concluded that “given the current situation, with no political solution in sight and no major increase of funds foreseen, the Lot 3 project as originally designed has become less relevant”. This led to the initiative to organise an inception workshop in September 2007, where national and international stakeholders in the Somali health sector participated, with a view to re‐orient Lot3. As is described in chapter 3 in more detail,


40

participants agreed on a limited set of actions for the next two years, focusing on (i) specific actions in the development of central institutions, (ii) development of human resources for health, and (iii) delivery of basic services.

This workshop is considered to have contributed to bringing the various stakeholders together, a unique event and a value in itself. Its action list also gave direction to the project implementation for the years to come and did influence formal amendments to the initial proposal as submitted to and approved by the donor. However, the various amendments of proposal and logframe retain much of the over‐ambitious, not very realistic language that characterised the initial proposal. Some interviewees remarked that the influence of the workshop may have been less effective than could have been due to late production of the report, which was only distributed a year after the event.

So, the project kept trying to adapt to circumstances that continued to deteriorate, while still keeping the spirit of the project, to work on larger system building issues, alive. The inception workshop gave direction, but in due time more and more emphasis was put on the production of the studies and tools, since direct activities with and in the MoHs proved less and less possible, essentially due to a serious deterioration of the security situation in all three zones.

While this project could possibly have benefited from a much more open, flexible approach, some of the grant conditions and bureaucratic requirements that come with it made this difficult. As said, the amendments to proposals and logframe seemed to some degree live a life of their own, with only a partial link to the realities of project implementation. Reports do not always report against the objectives that were formulated in changing fashion over time. The available money could only be spent under the initially agreed budget lines20. And while the project implementation had changed considerably over time, this requirement to fit expenditure under these budget lines posed another difficulty.

Another aspect of the implementation was the deviation from the intended sub‐contracting of the project to a third party to a mode with direct implementation by UNICEF. Apparently, fear for more delays was the main reason for this change. The impression is, though, that this was not a very deliberate decision taken after some more in‐depth discussion between UNICEF and EC. Looking back, it is difficult to say what would have been the better option. There would have been a delay. Although a third party might have been less constrained by UN regulations regarding presence on the ground, which was a factor hampering implementation in Somaliland, and would possibly have been more successful in recruiting the right people to staff the HCBUs, it seems unlikely that the third party could have maintained these HCBUs in CSS and Puntland, once security deteriorated so drastically. Project implementation by a third party would probably have given the project greater visibility as an entity on its own. As was, quite a few

20 This is, of course, not specific to the Lot3 project and will apply to most contracted projects. However, most of these projects will not change so much as Lot3 did during its implementation.


41

external observers saw the project as getting mixed with other UNICEF activities including use of time of the project manager. A definitive, ultimate advantage of having UNICEF as implementing partner was UNICEF’s commitment to the project that did not immediate stop at the end of the funding period by June 2009. A range of activities continue work that was started under Lot 3, including the end of project workshop that took place in October 2009.

Implementation on the ground was intended to occur in all three zones, in line with the initial idea of support to both Lot 1 and 2 that also operate in the three zones, the feasibility at the time of proposal formulation and the political imperative to work in all zones, including CSS. In the end, implementation was largely restricted to Somaliland. There was considerable input into the MoHL in Hargeisa, in particular in the HR department, and included support to the rehabilitation and upgrade of the MoHL premises, a very useful way to spend some of the available project monies. A more realistic, focused proposal right from the start with an emphasis on Somaliland as a first phase and with part of the funds allocated to service delivery in a way that would have strengthened leadership and planning capacity in the public sector would possibly have been more productive.

7.2.3 Role of UNICEF Was UNICEF the right organisation to host this project? In similar ‘transitional’ circumstances as Somalia poses, there is scarcity of organisations who can take on the role of providing strategic leadership to the health sector. NGOs usually lack size, capacity, credibility and/or interest. In some countries it was the World Bank or a (group of) donor(s) taking the lead, while WHO is rarely seen to play a role here. With its operational and sustained presence on the ground, less vulnerable to donor volatility, and usual involvement in at least some health aspects, UNICEF then makes a good candidate. On the other hand the organisation may not feel mandated to take this task, and the organisation may not have full interest or capacity to wholeheartedly commit to the task.

UNICEF, knowing its own, sometimes quite tedious and lengthy procedures, could maybe have been more wary, in this case, to take on a project of such short duration that would –as was proven‐ require quite some flexibility.

As said in a previous paragraph, some observers, right or wrong, perceived the Lot 3 project manager getting engaged in other UNICEF (health) activities. UNICEF claims this to have been minimal.

UNICEF, but also others, did point out the synergy between Lot3 and UNICEF’s ‘own’ health activities by hosting Lot3. The excellent, widely praised HMIS system as set up inside UNICEF under GAVI funds, but with considerable input from Lot3’s project management, is one example. As does the current strategies pursued by UNICEF in health and heavily influenced by Lot 3 analysis and recommendations (drug kits, CHDs, HP reform, etc). Lot 3 did create more awareness within UNICEF about its own role in the health sector that for some time had almost


42

been restricted to pure logistics, like the long standing delivery of drug kits to primary care facilities. So, for instance, a welcome revision of the drug kits is one of the spinoffs of this synergetic process.

7.2.4 Lot 3 visàvis Lot 1 and 2 The initial EC CfP where Lot 3 was seen as an umbrella project to Lot 1 and 2 kept hunting the final Lot 3 project, which had considerably changed at the time of its final technical proposal as described above.

The very idea that there would be a supportive Lot 3 project had raised high expectations with Lot1/2 implementing agencies. These expectations were never met, to some disappointment of these agencies, since they only found out after some time, among others through their membership of the Lot 3 Steering Committee that soon ceased to function. However, looking at the technical proposal under which Lot 3 started it was already clear that the link between Lot 1/2 and Lot3 had become very loose, despite repeated statements that Lot 1 and 2 agencies would be supported. Both in terms of field of operation, with the Lot 1/2 NGOs largely operating at hospital level and Lot3 at primary care level, as focus of activities (concrete service delivery versus much more upstream health system strengthening), Lot 3 could not realistically be expected to fulfil the Lot 1 and 2 expectations. After some time, this was realistically accepted by the NGOs. This was less the case with the EC that kept thinking Lot 3 should fulfil the role as foreseen in the original CfP, while it did approve and contract the final technical proposal that would not allow Lot 3 to fulfil that role.

7.2.5 The EPHS The formulation of the Essential Package of Health Services for Somalia is one of the best known outputs from the Lot 3 project, mentioned and usually highly valued by virtually all interviewees. The standardisation of services helps planning and a range of agencies are actively using the EHPS in designing their health activities. While not promoted as a ‘blue print’ but rather as a ‘road map for action’, there may be two reasons why implementation is still limited despite the good reception of the tool. One is definitely the sheer lack of money available in Somalia that can be devoted to horizontal, primary care services. Other post‐conflict countries, like Afghanistan, managed to kick‐start a basic package approach, aimed at universal coverage, for 5‐10 US$ a person a year, for which there was donor commitment for at least a number of years. To deliver a very basic package, much more basic than the package described in the EPHS, is probably more costly than this amount. But, in Somalia that minimum amount is in no way available for this purpose. The other reason may be that the proposed EPHS is too elaborate to use as a starting point. Packages that are much more basic, but for a cost the country can afford (including current and future donor assistance), while still achieving important health gains, may need to be formulated and used in pilot projects in appropriate geographical areas. Feasibility and costs can then be ascertained and scaling up considered. Possibly the DFID funded HSSP2 project can make progress here.


43

7.3 Sustainability The Lot 3 project, as was foreseen right at the beginning can only be the start of a long, sustained effort to reform the Somali health sector, in a way appropriate to the specifics of Somalia, contributing the health of all Somali at a cost the population can afford. It is very difficult to see when major steps in this direction can be taken, given the lack of a political settlement in the area that would lead to improved security and governance. Until such time, some forms of change in the health sector, some capacity building and innovative pilots can be undertaken. While such efforts cannot achieve massive results at the scale that would be needed to really address Somalia’s poor health indicators, experience in other settings has shown that these early developmental, more system oriented activities can pay off, once the situation improves. The lack of such activities is an often repeated observation from areas that are emerging from years of conflict. At that stage it always looks astonishing that no preparatory steps were taken for a post‐conflict period.

This Lot 3 project definitely has taken a number of these steps that may be highly beneficial in future. But it was only a beginning that needs a follow up, and a beginning that met very harsh and deteriorating circumstances.

The various studies done are examples of sustainable outputs that will prove useful in years to come. To be fully utilised the studies and other Lot 3 activities need follow up. Some of this may happen through the new DFID HSSP2 project and the HSAT. But investment in health system activities for Somalia, including Somaliland is extremely limited at the moment.

7.4 Specific recommendations

• Addressing broader health system issues in a protracted crisis environment like Somalia proves possible and useful and is to be continued.

• Somehow a ‘HSS Unit’ will have to be created that continues supporting strategic leadership in the health sector. This will have to be based on context specific data collection and analysis, conduct or commissioning of specific studies, policy formulation, and ability to disseminate HSS advice throughout the wider national and international health community involved with the Somali health sector.

• Such a HSS Unit should not be as time bound as the Lot 3 project, but work against a longer time horizon. Activities will vary over time and cannot all be planned in advance. Flexibility will have to guaranteed, which will require a governance structure that is able to allow this flexibility, but nevertheless keeps the Unit accountable, working along prioritised timelines, and responsive to the needs in the environment

• It is beyond the scope of this evaluation to recommend the exact place of such a Unit. This will require broader consultation among stakeholders, including donors. It should be closely


44

linked to the existing coordination platforms. It may or may not be hosted by an operational organisation. The example of Lot 3 having been inside UNICEF shows some of the pros and cons. For instance, direct exposure to operational challenges will keep a ‘HSS Unit’ close to the realities in the field, and not become a too distant ‘policy unit’. A downside may be the risk of the Unit getting too much into operations or that others outside the hosting agency feel less ownership.

• There are hardly any examples of similar ‘Units’. If at all more health system oriented activities were systematically pursued in similar circumstances (extremely limited state involvement for prolonged periods; variety of extended ‘humanitarian’ projects), this was usually linked to a particular donor funding operational partners (NGOs). In addition, the Somali context is very specific, incomparable to any other. External expertise may be able to play a useful advisory role for such a Unit, maybe in the form of a distance based panel that could advise on quarterly/annual plans, specific studies, and experiences elsewhere.

• It is recommended to properly document the HSS approach, including process and issues around coordination, as emerged over the lifetime of Lot 3, beyond existing project documentation that has been written for other purposes. Documentation will be of use locally, for any successor programme, but also as case study for an external audience. The international community looks for ways to promote HSS in fragile states, and this case study should become part of the evidence.

• The EPHS is to be piloted at small scale in smaller geographical entities where all levels (community, health post, health centre (MCH Clinic), and basic hospital level with at least comprehensive emergency obstetric care. To phase this in, it may be wise not to start with the full package as described in the EPHS but start with more basic packages as are in use in for instance Afghanistan, Liberia or Southern Sudan.

• Health system development should not be an act of its own, but should seek to pursue sustainable health gains, primarily through service delivery. In the context of Somalia this may take alternative approaches, different from the universal ‘district‐based primary health care approach’. In particular community based approaches, less dependent on fixed health facilities may be one of those alternatives. Also, the Child Health Days as implemented by UNICEF, with an input from Lot 3, should be carefully evaluated in terms of effect (health gains), cost‐effectiveness, and possible positive and negative side‐effects.

• UNICEF took on Lot3 and embarked on broader health system support than it usually does in similar circumstances and proved not fully equipped to do so, in terms of supporting expertise and recruitment. However, the stability UNICEF offered to the Lot3 project made it a good host and, in return, UNICEF health activities have benefited from having Lot 3 in house. It seems UNICEF’s ‘own health activities’, in particular in a context like Somalia, where it has a long track record of extensive support to a variety of health activities and health facilities, can benefit from an improved ‘health system perspective’. UNICEF may therefore want to increase its capacity in this field.


45

8 Appropriateness of the intervention

The idea behind the initial Call for Proposals for Lot 3 was to get away from the perpetual short term, project focused, mostly service oriented approach to health sector interventions as had been going on in Somalia for so many years. It was realised that more upstream work on broader health system issues would be necessary, both for the shorter and longer run.

As such this is highly appropriate since time and again it has been shown that such work is neglected in protracted crises and only addressed when there is a more stable post‐conflict situation. The latter often comes with an urgent need to address larger health system issues and a lot could have been prepared while there was still ongoing crisis.

So, the EC should be commended for wanting to take this route. Through the formulation of the CfP for Lot 3 it had wanted to take a small step in this direction, mainly in direct connection with the service oriented Lot 1 and 2 projects. Since the EC had decided to pull out of the health sector of Somalia, only two years remained to implement this project and there would not be any follow up.

Apparently, there was no agency available or interested to submit a bid to this call, which led to the EC negotiating with UNICEF. UNICEF did not want to take on the Lot3 as described in the CfP, which Is understandable, since it probably felt as a too narrow remit and difficult to really implement without taking on a broader health system strengthening approach. The concept note took this broader approach, but still remained more or less close to the remit of the initial CfP. However, it insufficiently took account of the short duration of the project and the high uncertainty of any donor wanting to support the activities afterwards. So, by the time the concept note was conceived and taken as the starting point for the development of a full technical proposal, on the one hand the underlying idea of broader health system approach was really taken up as would be appropriate, but on the other hand there was an increasing gap between this concept and the available time and budget, with the EC wanting to exit from the health sector.

From the concept note to the technical proposal this gap further increased and a set of over ambitious, not very realistic objectives was set in a time bound frame with a limited budget. At that time, while the initial notion to address the health system was still appropriate, the way this notion had turned into a practical, implementable project was no longer appropriate. It probably should not have been approved at the time. The approval was partly fuelled by the desire to spend the available funds for the good of Somalia. This not unfamiliar pressure was apparently not counteracted by an independent judgement by experts who could have warned against the feasibility of the project under these circumstances.

The kind of programming needed to get away from the short term, project focused, service oriented approach towards more broad support to analysis and capacity building for the


46

broader health system is very difficult. There are not blueprints. Any design will need to be highly context specific. Looking back there was insufficient expertise available at the time of formulation and approval of what became the Lot 3 project. More specifically, both UNICEF (during project formulation and internal vetting) and the EC (at the final approval stage) lacked the expertise to really look critically at the proposal and the conditions under which it was to operate.

However, it once more needs to be stated, that against all odds, the project still managed to deliver some high quality work, in terms of improved health sector coordination and the production of the range of studies and tools.

The lesson that should not be drawn from this episode, is that programming towards more broad health system support in these kinds of very fragile conditions is too difficult and should not be attempted. Rather, a key lesson is that it should be realised that such programming is an appropriate direction, but indeed difficult to formulate and implement. Rapid formulation under time pressure without sufficient expertise should be prevented. Rather than full reliance on one or two, somewhat ‘arbitrarily’ chosen consultants to formulate this kind of programme, as is often the case, one could think of setting up a broader panel of experts, who could provide comments. If programmes like this are set in motion, they should not be too time‐bound and have sufficient flexibility, also budget wise, to adapt to changing challenges and opportunities.

Another key lesson is around the promotion of health governance in the kind of very fragile conditions as Somalia poses. The health sector in any country inevitably needs oversight and leadership from public authorities to promote equitable provision of quality health care with some form of risk sharing. This leadership can take a variety of forms and shapes and may not necessarily come with a role in actual service provision. Fragile environments come with weak authorities and poor governance of all sectors, including health. The usual approach to promote improvement is to strengthen MoHs through capacity building, as was also assumed in the Lot 3 proposal. However, as long as there is no overall improvement in governance, the role of the MoH is usually very limited. This is also the case in Somalia, with its three MoHs, each with limited capacity, but also with inherently limited allocative authority. Programming in these environments should probably always take capacity building of MoHs into account to strengthen health governance. However, health governance can be defined much more broadly and also take the role of civil society and the private sector into account. The latter seems in particular pertinent in the case of Somalia, where the private sector and civil society both may provide an entry point to work on overall health system strengthening.


47

Annex 1 – Map of Somalia (overview & administrative units)


48

Annex 2 – Study objectives and results as expected in the ToR The Terms of Reference for the evaluation formulated study objectives and results as follows:

• B. Study objectives

The final evaluation study will provide UNICEF and the EC with sufficient information as to the:

o appropriateness of the proposal, o feasibility of the action, o suitability of the programme implementation strategy and o impact of the programme

– With lessons learned for similar programmatic approaches both in Somalia and elsewhere in comparable contexts.

A particular focus of the study will be an evaluation of the application and suitability of the approach to institutional development of the MoH, which was applied through the programme, and recommendations on how this could be strengthened and followed through in the future.

• C. Study results

The study will deliver the following:

• An overview of the principle features of the overall programme – to include objectives, major components, locations, commitments/disbursements made, any significant changes to approach/activities and reasons why;

• An assessment of the quality of the planning and design of the programme and the relevance to the problems it was intended to address and any recommendations for improvements for future submissions. In particular this should look at the logical framework as the underlying structure, in particular:

o The relationship between the activities and the results of the programme in the logical framework terminology;

o The relationship between the results of the project and the project purpose referred to in the logical framework terminology;

o The relationship between the project purpose and the overall objectives; o The validity of this logic and recommendations for modifications and/or

amendments, with a particular focus on impact indicators • An evaluation of the relevance of the programme concept • An evaluation of the efficiency of the implementation of the programme • An evaluation of the effectiveness of the implementation of the programme • An assessment of impact so far and/or prospects for impact in the longer term • An assessment of the potential sustainability of programme interventions • An evaluation of the institutional capacity development approach to the MoH • An evaluation of the extent the project managed to coordinate and find linkages with

other key strategy platforms and agencies: SSS, WHO/UNFPA, donors, NGOs, other projects such as UNDP Local governance and UNDP civil service support etc.

• An assessment of the achieved and expected impact

In all of these categories the assessment should make an analysis against the physical and policy environment; outline both strengths and weaknesses, present lessons learned and where possible and/or relevant, make recommendations for improvements for future programmes.


49

Annex 3 – Original logical framework

Intervention

Logic

Objectively verifiable

indicators* of achievement

Sources and means of verification

Assumptions

Overall

Objective

To contribute to the development of human and institutional capacity and competence, crucial for the rebuilding of the national health care system in Somalia.

Progress in Health system development: Managerial capacity demonstrated by observations of the ability of mentored staff to continue the further development and ownership of future policy development to support evidence based service delivery

HCBU Progress reports and evaluations

Socio-economic and political stability

The approach accepted by the Somali authorities

Adequate donor support to continue multi year plan capacity development plan.

Purpose

Sector policies, strategies, system building blocks developed and implemented

HCBU Mid term evaluation report

HCBU Progress and Final Reports

Steering Committee Reports

(to include excerpts/feedback from other relevant project reports were applicable i.e. NGOs implementing Lots 1 & 2)

Monitoring and supervision reports, assessment reports,

HMIS analysis

Mentored Somali staff are retained in their positions and allowed to transfer the knowledge gained from the capacity building process

Expected Results

RESULT 1

Comprehensive institutional assessment for central, regional and local administrations in place, covering: Organisation analysis, HR,

Institutional partner contract signed

Inception and Assessment Report

Inception Report

Qualified Somali professionals recruited


50

Intervention

Logic




Assumptions

health care facilities and applied technology, financing systems and accountability

MoUs with Health Authorities HCBU Progress reports

RESULT 2

Phased multi- annual capacity development plan jointly prepared and implemented (first phase), including sectoral planning, budgeting, monitoring, evaluating and coordinating health service delivery

Capacity development plan approved by stakeholders

Steering Committee meeting minutes/reports

Capacity development plan

Steering Committee meeting minutes/reports

Steering Committees validate and endorse plans recommendations.

Donors prepared to support long term commitments outlined in plan

RESULT 3

Key instruments for Health sector21 management developed and applied, in particular: Integrated information system (with GFATM, WHO) and sector planning, coordination, M&E systems

HCBU Mid term evaluation report

HCBU Progress and Final reports

Steering Committee Reports

HCBU Progress reports and Mid-term Evaluation report

Steering Committee Minutes

Publication of key documents related to policy reviews and

Fulfilment of commitments by all parties as per MOUs

Collaboration from the private health sector

21 Including private health services provision


51

Intervention

Logic




Assumptions

guidance by Health Authorities as applicable

RESULT 4

Implementing Partners (in particular those in Lots 1 and 2) advised and minimum standards achieved

Indicators to support the achievement of minimum standards by implementing agencies/partners will include the following areas:

• Revision of Essential Health Package • Development of a training plan for medical staff • Development of Standard job descriptions • Development and implementation of essential drug

management system • Development and implementation of quality control

mechanism of laboratory services • Development and application of health financing

procedures • Development and implementation for health

infrastructure and equipment • Increased utilisation rates of health care services,

increases in common illnesses properly diagnosed, essential drug protocols applied in the treatment of common illnesses and emergency obstetric care improved (to be based upon

Periodic Lot 1 and Lot 2 Progress reports and evaluations

NGO facilities are well managed and operated and not affected by conflict or political interference

HCBU support to capacity development yields measurable and sustained increases in utilisation rates of health care services, increases in common illnesses properly diagnosed, essential drug protocols applied in the treatment of common illnesses and emergency obstetric care improved (to be based upon

Intervention

Logic

Means Sources and means of verification

Assumptions


52

Activities

For RESULT 1

1.1 Creation of a Health Capacity Building Unit (HCBU)

1.1.1 Tendering for Institutional Contract(UNICEF)

1.1.2 Establish the HCBU

1.1.3 Establish Steering Committees with TOR

1.2 Assessment & recruitment of staff

1.3 Identification/assessment of external participants to capacity building process

1.4 Monitoring system for the Implementation of mentoring process

Institution contracted

Core teams established : to include: Expert #1: Team Leader / Project Manager (1x) Expert #2: Health Economist (1x) Expert #3: Human Resources (1x) Expert #4: Public Health (1x)

Units established in Somaliland, Puntland and Central South

Steering committees established in Somaliland, Puntland and Central South. UNICEF Project Officer liaising with existing coordination mechanisms and in country.

Establish incentive packages for Steering Committee and Key Ministry staff

Coordination meetings held with stakeholders

Deployment of HCBU into Ministries

HCBU Inception * Progress reports

Minutes of Steering Committee meetings

Mid-Term Evaluation

Human resource inventory reports

Coordination meeting minutes

NGO partner reports

Financial Reports

Improvements in political and expand the approach stability

The approach is accepted by the Somali authorities

Qualified Somali professionals recruited

Coordination mechanisms in place and functioning


53

Activities

For RESULT 2

Assessment in conjunction with the key stakeholders, including:

• Reviewing existing policies and health sector capacity development initiatives

• Conduct interviews with key personnel • Draft Report for Steering Committee

Coordination activities with key stakeholders (both at Nairobi and regional level)

Mentoring from HCBU and use of specialist consultants to train and support capacity development of Health Ministries to develop and strengthen the following components of the health systems

Activities

For RESULT 3

Health sector management capacity developed for progressive application where possible during first phase, in particular: integrated information system (with GFATM, WHO) and sector planning, coordination, M&E systems

Activities

For RESULT 4

Coordination with Implementing Partners (in Lots 1 and 2) to ensure their projects benefit from result 3


54

Annex 4 – Logical Framework (Amendment 2, 28/02/09)

Intervention

Logic



Sources and means of verification Assumptions

1. Overall

Objective

To contribute to the development of the capacity of the health sector (government, private and public providers) in Somalia in order to promote greater efficiency, effectiveness and equity of health service provision and to promote longer term strategic analysis and reform for the sustainable re-modelling of national health system(s)

1.1 The Health Ministries have improved their governance and leadership capacities to begin to define and create realistic health care systems – Including involvement of regional (decentralised) levels of administration for supervision and support of service delivery.

1.2 The HSC has adopted a set of core normative standards to assist donors and implementing partners to give coherent and strategic advice about investment and action in the health sector (allowing greater coordination and cross programme linkage)

1.3 NGO health providers (especially lots 1 and 2) have a core set of normative guidelines and protocols to assist them to design and implement more effective, efficient and equitable health service

1.1.1 The MoHL SL has established a health sector reform committee that meets at least 4 times per year (minutes available)

1.1.2 The MoHL SL is involved in organising regular health partner coordination meetings and linking coordination platforms to technical platforms in Nairobi (minutes).

1.1.3 The Ministry of Health (SL) participates actively in the process of governance reform (coordination, sectoral integration, analysis of workforce to NCSC).

1.1.4 Each regional health office exists and has a core staff – functioning. 1.1.5 Each regional office provides planning documents and supervisory

reports. 1.1.6 National and regional health information reports are available on a

timely basis in all zones (UNICEF reports). 1.1.7 A national and regional health management information system

managed by the MOHL SL exists and produces credible reports (at least annually).

1.2.1 The HSC has a set of core normative documents on PHC, hospitals, staff (cadres and salaries/incentives), supervision. Donors report using these guidelines in selecting proposals.

1.3.1 NGOs have participated in defining the strategic agenda and their requirements have been detailed and fulfilled (strategic meeting

• Improvements in political context

• Stability expands • Central governance is

realistic in all 3 zones • The approach is

accepted by the Somali authorities

• Ministry of health committed to reform and governance reform

• Adequate donor support to continue.

• Realistic civil service reform process begins and is widely accepted


55

Intervention

Logic




delivery programmes

reports). 1.3.2 NGOs report using normative standards in programming.

2. Purpose

To contribute to improving the capacity, predictability and coherence of health decisions and service delivery options in order to improve current and future health and equity of the Somali people

2.1 The Ministry of Health have core institutional capacities to actively and constructively lead in the management of the health system (core staff, policies and decisions).

2.2 Regulation of medical staff and private providers has been initiated.

2.3 Plans are available for the expansion of access to health care (private and public)

2.1 Existence of MoH institutional capacity – coordination, secretariat for reform etc. (minutes of reports and documented decisions)

2.2 Review of private medical practitioners and pharmacies has been completed and ideas for regulation formulated.

2.3 NGO programmes reflect management of local health systems tied to national models.

2.4 HPC has been created and mandated.


56

Intervention

Logic




3. Expected Results

RESULT A

Basic functional MoHs

A.1 Enhanced internal staff management promoted through conditions of performance based contract with the MoH.

A.2 Health sector reform committee established and active.

A.3 MoH manages regular and effective health coordination meetings.

A.4 Public health experts embedded in Somaliland ministry.

A.5 Budgeting, accounting and reporting systems reviewed.

A.6 Each region in Somaliland has a core regional office involved in delivery of heath services.

A.7 Development of the health Information capacity (in collaboration with the GF malaria)

A.1.1 All supported staff are managed by their functional line manager according to performance expectations on a regular basis (3 monthly performance reports).

A.2 .1 A HSRC is established and meets (minutes)

A.3.1 Coordination platforms for regional offices and partner providers established and meet regularly (minutes).

A.4.1 Public health experts report on structure, policies and process of MoH (organigramme, job descriptions etc.)

A.5.1 Budget and accounting systems documented and approved.

A.6.1 each SL regional office has a core set of staff involved in implementing policy, ensuring PHC systems are supplied and supervised, planning and implementing actions and reporting on HMIS data – and is equipped with a vehicle.

A.8 HMIS systems designed and regular reports issued (UNIOCEF) as well as capacity of MoHL SL developed in order to manage their own data base – and use information for central and regional decision making.

RESULT B

Normative standards for key resource and service delivery

B.1 Health Management Information System functional and producing information to drive allocative decision making

B.2 Essential Health Package: EHP for PHC level; EHP for Secondary (referral)

B.1.1 HMIS Task Force established; HMIS initiatives reviewed; minimum data sets established and key indicators defined.

B.1.2 HMIS policy and guidelines in place.

B.1.3 HMIS regular reports at regional and zonal level produced and discussed -

• Continued donor support

• Political stability • Absence of conflict

and all out


57

Intervention

Logic




process defined and endorsed (MoH, HSC, NGO)

level; and implementation guidelines endorsed

B.3 Essential drugs: Drug needs per facility level defined, drug kits reviewed and altered; public drug management systems reviewed and developed for improved public impact.

B.4 Human resource policies defined and endorsed: professional cadres defined, training plans developed, remuneration policies established, reform of civil service engaged. Recommendations/implementation re. establishment of a certification body

B.5 Health care facilities: development of standard facility blueprints.

B.6 Health Sector financing system: Health financing strategy developed.

B.7 Private health sector assessment: Recommendations toward regulation

B.8 Public accountability mechanisms: Options and mechanisms for user co-management; Statement of Patient’s Rights (in collaboration with local governance

used to promote debate and decision making (coordination minutes)

B.2.1 BPHS endorsed and used by donors, HSC, NGOs and MoH.

B.3.1 PHC drug kits redefined.

B.4.1 Strategic national HR plans defined; training strategies defined, remuneration policies defined.

B.5.1 Blueprints for al standard health care facilities developed.

B.6.1 BPHS costed out with sequencing options

B.7.1 Regulatory bodies formed where possible (SL)

B.8.1 (Interfaces for community/district committees and the health system developed and distributed to district councils and other community representative platforms – detailing core transparency information.

B.9.1 Support to formation of health Professional Council and development of a licensed practitioner registry.

humanitarian crisis.


58

Intervention

Logic




programme)\

B.9 Support to professional associations

RESULT C

Lot 1- & Lot 2 (and other NGO)-health service delivery programmes actively involved in strategic development.

C.1 Lots 1 & 2 have participated in piloting and reviewing different strategic options (see points B.1. – B.8).

C.2 Lots 1 & 2 actively implemented and feedback on policy recommendations.

C.3 Results from lot 3 assist Lots 1 and Lots 2 to develop more coherent and strategic interventions for future service delivery and system development..

C.1 Minutes of steering committee and NGO strategic meetings – working group draft discussion papers.

C.2 review of consultancy reports.

C.3 Interviews with coordinators form Lots 1 and 2 and future proposed actions.

Partners implementing Lot 1 & 2 are able to access all zones comprehensively and no humanitarian crisis..

Intervention

Logic




Activities

For RESULT A

A.1 Draft, Sign and Monitor a performance based contract with each ministry.

A.1.1 Contracts signed

A.1.2 3 monthly reports from ministry available

A.1.3 Funds dispersed.

A.1.1 Contract

A.1.2 Monthly reports

A.1.3 Fund dispersed and received (reports)

• Improvements in political context

• Stability expands • Central governance is

realistic in all 3 zones • The approach is

accepted by the Somali authorities


59

A.2 Establish and support a Health sector reform Committee in each zone.

A.3 Establish and support effective health coordination meetings.

A.4 recruit and embed Public health experts in each ministry.

A.5 Provide a technical consultancy from accounting firm to assist the MoHs to review their planning, budgeting, accounting and reporting systems (financial management) at the central and regional levels.

A.6 Review each region for needed support and support staff (In collaboration with local governance programme)

A.2.1 HSRC formed and functioning (minutes).

A.3.1 Coordination meetings occur regularly with good attendance (minutes)

A.3.2 Coordination linked to Nbo based coordination (HSC receives feedback)

A.3.3 Coordination assists integrated management of aid (governance reform – overall Inter-ministerial coordination improved – minutes)

A.4.1 Experts recruited and placed formally in MoHL SL.

A.4.2 Regular reports of expert to Lot 3 manager.

A.5.1 Guidelines for accounting drafted and endorsed.

A.6.1 Support staff of each region – incentives and transport support as well as training.

A.6.2 Develop a manual of how to support regional health authorities for NGO programmes..

A.2.1 HSRC minutes

A2.2 Policy review documents.

A.2.3 Core policies form each MoH.

A.3.1 Minutes

A.3.2 HSC minutes

A.4.1 Experts contracts.

A.4.2 Reports

A.5.1 Guidelines

A.6.1 3 monthly MoH reports

A.6.2 Report developed.

• Ministry of health committed to reform and governance reform

• Adequate donor support to continue.

• Realistic civil service reform process begins and is widely accepted

Activities

For RESULT B

B.1 Health Management Information System: supported and produces reports.

B.2 Essential Health Package: EHP for PHC level; EHP for Secondary (referral)

B.1 .1 HMIS experts hired (in collaboration with GF Malaria)

B.1.2 HMIS framework and policy

Tools and Guidelines developed and tested

B.1.1 Experts contracted

B.1.2 HMIS system developed and report endorsed.

B.1.3 Reports


60

level; and implementation guidelines endorsed

B.3 Essential drugs: Drug needs per facility level defined, drug kits reviewed and altered; public drug management systems reviewed and developed for improved public impact.

B.4 Human resource policies defined and endorsed: professional cadres defined, training plans developed, remuneration policies established, reform of civil service engaged. Recommendations/implementation re. establishment of a certification body

B.5 Health care facilities: development of standard facility blueprints.

B.6 Health Sector financing system: Health financing strategy developed.

B.7 Private health sector assessment: Recommendations toward regulation

B.8 Public accountability mechanisms: Options and mechanisms for user co-management; Statement of Patient’s Rights (in collaboration with local governance programme)\

B.1.3 HMIS regular reports produced and used.

B.2.1 Consultancy organised

B.2.2 Normative framework developed and endorsed

B.2.3 Options costed to promote rigorous policy/allocative choice process.

B.3.1 Essential drugs for BPHS developed.

B.3.2 UNICEF drug kits and rug management systems re-defined and implemented.

B.3.3 Private RDF systems evaluated and recommendations made for the development of national drug supply and management.

B.4.1 Human resource expert recruited and embedded.

B.4.2 Consultancy contracted

B.4.3 Human Resource frameworks developed for the ministries.

B.4.4 Human resource remuneration policies developed and fed into NCSC reform processes.

B.4.5 Standard incentive packages developed and endorsed if

B.2.1 Consultants contracted

B.2.2 Guidelines produced and endorsed

B.2.3 Options costed – MoH make explicit decisions (minutes HSRC)

B.3.1 Drugs for BPHS listed and reviewed by WHO.

B.3.2 New drug kits developed and implemented.

B.3.3 RDFs explored and options generated.

B.4.1 Recruited and placed

B.4.2 Contracted

B.4.3 Report adopted.

B.4.4 Policies developed and reviewed by MoH (HSRC minutes)

B.4.5 Incentive packages developed – consulted with NGOs and killed or launched.

B.5.1 Blueprints and costs exist.


61

useful.

B.5.1 Facility blueprints developed and costed for all tiers of standard health service delivery.

B.6.1 Health system costing analysis performed.

B.6.2 Health system financing performance reviewed.

B.6.3 Recommendations for development of reliable financing developed.

B.7.1 review of private market and recommendations for progressive regulation developed.

B.8.1 Guidelines for role of local representative platforms (community and district) in transparency and management of health services developed (in collaboration with local governance programme).

B.6.1 Costing of BPHS reported on.

B.6.2 Report on current financing.

B.6.3 Report issued (discussed by HSRC minuted)

B.7.1 Consultancy organised.

B.8.1 guidelines developed and distributed (NGOs and local governance structures)

Activities

For RESULT C

C.1 Lots 1 & 2 have participated in piloting and reviewing different strategic options.

C.2 Lots 1 & 2 actively implemented and feedback on policy recommendations.

C.3 Results from lot 3 assist Lots 1 and Lots 2 to develop more coherent and strategic interventions for future service delivery and system development.

C.1.1 Lots 1 and 2 participated in strategic development and priority setting as well as benefited from Lot 3 outputs.

C.2.1 Reports of consultants (B.1 – B.8) and feedback reports demonstrate engagement and learning in Lots 1 and 2 programmes.

C.3.1 new proposals from NGOs more strategically coherent and ambitious.

C.1.1 Minutes of steering committees and interviews with project managers

C1.2 Lots 1 and 2 participate in NGO strategic meetings (launch, HMIS, BPHS, RH etc.) and in final strategy workshop.

C.2.1 Reports and emails as well as minutes of steering committee meetings.

C.3.1 Proposals and minutes of SC meetings (submission to the HSS WG for endorsement) as well as interviews with key donors


62

Annex 5 Publications by UNICEF under Lot 3 programme

Published:

Report 1: Health Systems Strengthening for the Somali People. A workshop report, September 5th –7th 2007 Report 2: An Essential Package of Health Services. Somalia and Puntland 2009 Report 3: An Essential Package of Health Services. Somaliland 2009 Report 4: Intermediate Standardized Salary Support/Incentives: Payment scales for civil

servants and health workers. Somaliland Report 5: Intermediate Standardized Salary Support/Incentives: Payment scales for civil

servants and health workers. Somalia and Puntland Report 6: The Revolving Drug Fund. Bringing the “public” back into the Somaliland public

hospitals. Hargeisa Group Hospital, Somaliland Report 7: Steps Towards Harmonizing External Support for Health Care Provision for the

Somali People Report 8: Exploring Primary Health Care in Somalia. MCH Data ‐Somalia & Somaliland 2007 Report 9: Costing out the EPHS Report 10: Health Care Seeking Behaviour in Somalia: A Literature Review Report 11: The Private Sector and Health: A survey of Somaliland Private Pharmacies Report 12: Construction Guidelines for Basic Health Facilities. Somaliland and Somalia Report 13: Health Systems Strengthening, Somalia. A workshop E‐report, October 2009 Report 14: A report on Human Resource Development is in progress Additional Documents on Reproductive Health produced in collaboration with WHO/UNFPA • A Situation Analysis of Reproductive Health in Somalia, UNFPA/WHO/UNICEF • Reproductive Health: National Strategy and Action Plan 2010 – 2015 Not published but circulated to limited audiences: (1) Use of Drugs in Somaliland Primary Health Care Facilities, Nomad network 2007 (2) Results of a Preliminary Strategizing Session on Health Facility Prioritization (3) Human Resources for Health Policy and Plan Development Project‐ Final report, THET 10/09 (4) Support for Human Resources for Health, THET/ Ministry of Health & Labour, Somaliland. Narrative Report on HR Specialist Assignment, 11/02/09 – 30/06/09 (5) Report on First Phase of a Consultancy on HRD in Somaliland, LATH Joyce Smith 04/09 (6) Report on Second Phase of a Consultancy on HRD in Somaliland, LATH Joyce Smith 06/09 (7) A suggested outline of an Agreement between Development Partners to Enhance the Performance of Investments in Health for the Somali People (8) Health Sector Development in Somalia – a Discussion Paper for Development of Investment Strategies in Somalia, Austen Davis 12/07 (not for circulation) (9) Set up of the Health Systems Analysis Team and Selection of Host Agency for HSAT, 10/09 (10) Final Evaluation of the EC financed UNICEF HSS programme, Nov 2009 (11) Financial Management Support to Hargeisa Group Hospital, THET/HGH/UNICEF 08/09


63

Annex 6 – Lot 3 consultants Consultant Area of work Reports (see

Annex 5) Contracting agency

Nigel Pearson Development of the Essential Package of Health Services

Published Reports 2 and 3

UNICEF

Jeff Muschell Development of the Essential Package of Health Services

Published Reports 2 and 3

UNICEF

Nigel Pearson

(+ Enrico Pavignani and Renato Correggia)

Harmonization of Donor Support to the Health Sector Somalia

Published Report 7 UNICEF (and EC) contracted Nigel Pearson; DFID contracted Enrico Pavignani; and Italian Cooperation contracted Renato Correggia -> team joint selected by a managing panel.

Jeff Muschell Costing the EPHS Published Report 9 UNICEF

Nigel Pearson Implementing the EPHS – drug kits, supervision and referral

Published Report 2 and 3 (annexes and as separate reports)

UNICEF

Dr Rehana Ahmed and Caitlin Mazilli

Surveying the private pharmacies of Somaliland

Published Report 11 UNICEF

Caitlin Mazzilli Desk review of health seeking behaviour in Somalia.


Mark Beesley Human Resource development – support to MoHL Somaliland HR department

Report unpublished (4)

THET (on behalf of UNICEF)

Joyce Smith Human resource Development – support to MoHL Somaliland HR department in generating a strategic agenda.

Report unpublished (5) and (6)

LATH (on behalf of UNICEF)

Dr Ingvil Sorbye Reproductive Health Situation Analysis

Report published UNFPA contracted – joint managed by UNICEF/WHO and UNFPA

Dr Ingvil Sorbye and Dr Bailah Leigh

Reproductive Health National Strategy Development (and policy recommendations)

Report published WHO contracted Dr Sorbye; DfID contracted Dr Leigh – joint managed by


64

UNICEF/WHO and UNFPA

Dr Paolo Paron Creating an updated and geo-referenced health facilities data base and mapping

Database handed over to UN OCHA

UNICEF

Muhuru Ndungu Reviewing Institutional set up of the Health Systems Analysis Team (HSAT)


UNICEF on behalf of a management panel

James Culverwell Report editor Edited all published reports

UNICEF

Lorenza Rossi HMIS development and data analysis

Development of HMIS database for all MCHs + development of HMIS strengthening programme in Somaliland and Puntland

Many reports including Published Report 8

Global Fund Malaria (UNICEF)

Dr Egbert Sondorp

Evaluation of Lot 3 Report unpublished (11)

UNOPS (on behalf of UNICEF and EC)

Bertrand Chenin Revolving Drug Fund Review and recommendations


Mohammed Khaire

Support to MoHL Somaliland HR department

No report UNICEF on behalf of MoHL

Accountant Support to MoHL finance and administration department

No report UNICEF on behalf of MoHL

Consultant Support to MoHL Department of Planning


THET (on behalf of UNICEF)

Consultant Support to formation and initiation of Health Professionals Council


THET (in partnership with UNICEF)

Sam Mwaura Support to financial management of a Somaliland Public Hospital (HGH)


THET (in partnership with UNICEF)


65

Annex 7 People / organisations consulted for final evaluation Nairobi

• Elio Omobono, Health Advisor, Italian Cooperation • Enrico Pavignani, consultant • Edda Costarelli, Programme Coordinator & Quality Assurance, European Commission

(member reference group) • COOPI: Gemma Sanmartin, project coordinator (member reference group), Alberto

Leone, medical coordinator, Paola Grivel, regional representative • COSV: Sabrina Pestilli, project coordinator (member reference group), and other

members of Lot2 consortium (AAH‐Ben Odera, en Abdi Tari, CISP) • Renato Corregia, health adviser UNOPS (chair of reference group) • Anthony Daly, health and aids advisor Kenya & Somalia, DFID • Rosemary Heenan, Trocaire (for Gedo region) • Kamran Mashhadi, Health Sector Coordinator, Somali Support Secretariate, UNOPS • Caitlin Mazzilli, consultant Pharmacies study • Ingvil Sorbye, consultant Reproductive Health study • Karen Peachey, Director THET • MoH CSS – Mrs Jamila Said, Director General, Dr Abdi Awat, Health Advisor • MoH Puntland – Dr Mohamed Hersi Duale, Vice Minister, Mr Abdikafi, Director of

Planning, Dr Noor M Noor, Gardo Hospital Director • MoHL, Somaliland – HE Minister Mr Abdi Haybe, Mr Khadar Mohamed, Dir Planning,

Dr Shiine, Human Resources • Dr Ahmed Hassan, President Somali Red Crescent • Nigel Pearson, consultant EPHS and HOAP • WHO Somalia‐ Marthe Everard, WHO Representative for Somalia, and Dr Humayun

Rizwan • Paula Vazquez‐Horyaans, European Commission • Austen Davis, Lot 3 Programme Manager, UNICEF • Suraya Dalil, Health Programme Office, Unicef – Somalia (member reference group) • Phil Cooper, Local governance, UNDP • Nadeem Jan, SCF‐UK

Somaliland

• Mohammed Sheigh, Unicef • Maryan, Unicef midwife • MoHL

• HE Minister Mr Abdi Haybe • Anwar Mohammed Egels, DG • Dr Mohamoud Said Ahmed, Director of CDC • Dr Ahmed Suleiman Jama, Director of Training • Mr Abdillahi Abdi, HMIS • Mr Faisal Mohamed, Director of Health Services • Mr Abdi Mohamed Hussein, Director of Admin/Finance


66

• Health Unlimited ‐ Eric Oyod, Field coordinator, Shukri Harir, Liaison Officer, Rohit Odari, Programme Manager

• Deputy Director, Hargeisa Group Hospital • Dept Director, Edna Aden hospital • Rosemary, THET • Dr Abdi, PHC programme officer, WHO • Mr Mohamed, Chair of Hargeisa Regional Health Board • Miresi Busana, Dr Ahmed Askar COOPI • Meeting met hospital staff and RHB in Berbera Hospital

• Dr Hassan, hospital director, Mr Said, chair RHB, Dr Mohammed and other staff • Meeting met hospital staff and RHB in Borama Hospital

• Hospital director, staff, Dean Medical School, Dean Nursing School, RMO and RHB • Roda Ali Ahmed, Head Nurse Tutor, Hargeisa Institute of Health Sciences plus Mr

Ahmed, Director of Nursing School