Document of The World Bank

Report No: 29119

IMPLEMENTATION COMPLETION REPORT(CPL-38290)

ON A

LOAN

IN THE AMOUNT OF US$ 35.7 MILLION

TO THE

LEBANESE REPUBLIC

FOR A

HEALTH SECTOR REHABILITATION PROJECT

June 8, 2004

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

CURRENCY EQUIVALENTS

(Exchange Rate Effective June 4th, 2004)

Currency Unit = Lebanese Pound LL 1.0 = US$ .0007US$ 1.0 = LL 1,500

FISCAL YEARJanuary 1 December 31

ABBREVIATIONS AND ACRONYMS

CAS Country Assistance StrategyCDR Council for Development and Reconstruction DRG Diagnosis Related GroupingEMS Emergency Medical ServicesERRP Emergency Reconstruction and Rehabilitation ProgramGIS Geographic Information SystemGOL Government of LebanonHMO Health Management OrganizationICR Implementation Completion ReportMOH Ministry of HealthMTR Mid-Term ReviewNERP National Emergency Reconstruction ProgramNHA National Health AccountsPCU Project Coordination UnitPSR Project Status ReportQAG Quality Assurance Group

Vice President: Christiaan J. PoortmanCountry Director Joe SabaSector Manager George Schieber

Task Team Leader/Task Manager: Bassam Ramadan

LEBANONHEALTH PROJECT

CONTENTS

Page No.1. Project Data 12. Principal Performance Ratings 13. Assessment of Development Objective and Design, and of Quality at Entry 24. Achievement of Objective and Outputs 75. Major Factors Affecting Implementation and Outcome 106. Sustainability 117. Bank and Borrower Performance 128. Lessons Learned 159. Partner Comments 1610. Additional Information 38Annex 1. Key Performance Indicators/Log Frame Matrix 39Annex 2. Project Costs and Financing 40Annex 3. Economic Costs and Benefits 42Annex 4. Bank Inputs 43Annex 5. Ratings for Achievement of Objectives/Outputs of Components 46Annex 6. Ratings of Bank and Borrower Performance 47Annex 7. List of Supporting Documents 48

Project ID: P034004 Project Name: HEALTH PROJECTTeam Leader: Bassam Ramadan TL Unit: MNSHDICR Type: Core ICR Report Date: June 9, 2004

1. Project DataName: HEALTH PROJECT L/C/TF Number: CPL-38290

Country/Department: LEBANESE REPUBLIC Region: Middle East and North Africa Region

Sector/subsector: Health (91%); Central government administration (9%)Theme: Health system performance (P); Access to urban services for the poor

(S); Civic engagement, participation and community driven development (S)

KEY DATES Original Revised/ActualPCD: 11/19/1993 Effective: 10/04/1995 10/04/1995

Appraisal: 12/01/1994 MTR: 12/31/1997 01/25/1999Approval: 12/20/1994 Closing: 12/31/2000 12/31/2003

Borrower/Implementing Agency: GOVERNMENT OF LEBANON/MINISTRY OF HEALTHOther Partners: WHO

STAFF Current At AppraisalVice President: Christiaan J. Poortman Ciao Koch-WeserCountry Director: Joseph P. Saba Ram K. ChopraSector Director: Michal Rutkowski Jacques F. BaudouyTeam Leader at ICR: Bassam Ramadan Albert SalesICR Primary Author: Firas Raad

2. Principal Performance Ratings

(HS=Highly Satisfactory, S=Satisfactory, U=Unsatisfactory, HL=Highly Likely, L=Likely, UN=Unlikely, HUN=Highly Unlikely, HU=Highly Unsatisfactory, H=High, SU=Substantial, M=Modest, N=Negligible)

Outcome: S

Sustainability: L

Institutional Development Impact: M

Bank Performance: S

Borrower Performance: S

QAG (if available) ICRQuality at Entry: U

Project at Risk at Any Time: Yes

3. Assessment of Development Objective and Design, and of Quality at Entry

3.1 Original Objective:The original development objectives of the project were to:

• improve the country’s health conditions through better allocation and use of resources in the public and private sectors (long-term).

• enable the Ministry of Health (MOH) to effectively perform management and administrative functions that are critical to further development in the sector (short-to-medium term).

• respond to the unmet health needs of poor population groups who rely on public services, particularly selected health centers and general hospitals (short-to-medium term).

The second two development objectives, particularly the former, were deemed critical to the further development of the Lebanese health sector.

During the early 1990s Lebanon emerged from more than fifteen years of civil war and initiated a broad-based National Emergency Reconstruction Program (NERP) as a central element of the post-war recovery process. The program aimed to rehabilitate critical infrastructure damaged by the war and to upgrade the institutional capacity of key governmental sectors significantly weakened by the many years of internal upheaval. In support of the program, the Bank funded an Emergency Reconstruction and Rehabilitation Project (ERRP) in 1993 that focused heavily on repairing several strategic utility sectors and the education and housing sectors. The Health Sector Rehabilitation Project loan followed two years later in 1995 and broadly supported the development objectives of ERRP. The 1998 Country Assistance Strategy (CAS) agreed to by the Bank and the Government of Lebanon (GOL) helped reinforce many of the objectives and activities included in the Health Sector Rehabilitation Project.

The original project development objectives, as broadly defined, were generally responsive to borrower demand and relevant to health sector priorities. The difficult post-conflict environment during 1993-4 (i.e. weakenend public sector, confessional system of government, political instability near Southern border) and the expressed urgency for Bank involvement in Lebanon disallowed any solid analytical work prior to appraisal and the formulation of project objectives. In-depth sector work with institutional counterparts, if circumstances had allowed, could have helped in shaping client demand more towards the strengthening of the primary health care system and public-private contracting mechanisms and less towards the full rehabilitation of public sector hospitals. It could have helped in refining the content of the third development objective by containing the political pressure to follow a bricks and mortar approach (i.e. reducing the emphasis on large-scale public hospital reconstruction), which seemed questionable given the predominance of the private sector and the excess capacity of total hospital beds in the country. Although the project development objectives, as stated, were generally realistic assuming a normal operating environment and reflected many of the Government’s priority needs, project design and implementation arrangements were too complex, demanding and risky given the prevailing weak institutional capacity in Lebanon after the war.

Although two major risks were discussed during appraisal (limited implementation capacity and changing political priorities) and measures were identified to help mitigate those risks, insufficient attention was paid to two other important risk factors: (i) loan effectiveness delays due to schedule misalignments between Bank Board approval and the Lebanese legislative cycle; and (ii) unclear role delineation and ineffective coordination among the main implementing agency, the Council for Development and Reconstruction (CDR), the MOH and the Project Coordination Unit (PCU). A ten-month lag in effectiveness from

- 2 -

December 1994 to October 1995 could have been avoided, among other things, if project delivery had better coincided with the parliamentary calendar in Lebanon.

3.2 Revised Objective:The original objectives of the project were revised following the Mid-Term Review (MTR) in 1999 to make adjustments for significant lags in project implementation (i.e. both the IP and PO ratings were "unsatisfactory" during 1999 -- See Annex 4) and to better reflect the changing policy priorities of the Lebanese government. One sub-component in particular, related to MOH organizational and staffing reforms, progressed very slowly due to broader problems associated with civil service reforms in Lebanon. Another factor which contributed to a revision of the objectives was a change in government preceding the mid-term review. The new government in the second half of 1998 expressed an explicit desire to elevate the health policy reform process and focus on two additional priority areas: (i) developing a health quality accreditation system for hospitals and (ii) strengthening emergency medical services.

These revisions were not deemed significant enough by the Bank’s regional management to require board approval. In retrospect, given the added components and the broader policy scope of the revised project -- as well as changes (albeit minor) in the project development objectives, it would have probably been advisable to seek Board approval for official project restructuring. Perhaps since no project savings were projected, Bank management deemed it unnecessary to take the revisions to the Board.

The long-term objective remained unchanged while the short-to-medium-term objectives were modified and expanded as follows:

The first short-to-medium term objective "to enable the MOH to effectively perform essential lmanagerial and administrative function in the sector" became "to strengthen the planning, monitoring and quality assurance capacity of the MOH and to rationalize health sector financing.” The reasons for moving away from MOH "management" functions and more towards "planning, monitoring and quality assurance" were related to inherent difficulties in restructuring the organization of the MOH and an explicit request by the MOH to incorporate quality assurance dimensions into the project.

The second short-to-medium term objective "to respond to the unmet health needs of poor lpopulation groups who rely on public services particularly select health centers and general hospitals" was reduced to "improve service delivery.” This objective was modified to reflect the desire of the government to improve the quality of health service delivery in both the public and private health care sectors; and for all population groups in the country. This change did not involve a major shift in focus away from the poor but a broadening of scope which included other income groups within Lebanon. Many of the poor in Lebanon still rely on health care from private hospitals via a public MOH referral system -- in addition to their reliance on front-line public sector hospitals. This modification in objectives was sensible in view of the importance of private sector hospitals in servicing all income groups in the country.

A third short-to-medium term objective of providing “support to the development of reform” was ladded following the mid-term review. This objective was appended to the project given the express intention of the government to initiate a broad health sector reform process encompassing the different components of the project. The need for such reforms became increasingly evident when National Health Accounts (NHA) developed through the project indicated that Lebanon was spending about 12 percent of its GDP on health.

- 3 -

The impetus for revising the original project development objectives, again, emanated primarily from slow disbursement rates and a new government intent on initiating a broad health policy reform process with a heightened focus on quality accreditation, emergency medicine systems and hospital payment systems. Greater attention was given, therefore, to health sector planning tools (e.g. strengthening the GIS mapping initiative) and different mechanisms to initiate and sustain a health policy reform process (e.g. creation of a national health reform council). Expanding the policy scope of the project, albeit somewhat risky, reflected the shifting government priorities towards broader health sector reform. The positive end results of the project (i.e. stronger institutional capacity, greater access for the poor, better quality of care), in retrospect, vindicate this degree of risk-taking during the mid-term review.

3.3 Original Components:The five original components of the project were as follows:

PART A: Institutional Strengthening

(1) Essential Administrative and Managerial Functions of MOH (US$5.3 million). The aim of this component was to restore and strengthen the essential administrative functions of the MOH via four particular activities: (i) reviewing and introducing changes to the administrative structure and organizational procedures of the MOH; (ii) developing and establishing a national health information system, (iii) strengthening the management of health centers based on a standard package of public health and clinical services; and (iv) strengthening the management of public hospitals by introducing new management tools (new budgeting and accounting procedures).

(2) Planning and Normative Functions in the Sector (US$1.6 million). The objective of this component was to help rationalize national health care expenditures (public and private) by assisting the MOH staff in: (i) developing a geographic health information system or carte sanitaire designed to spatially present critical health information using modern software and (ii) creating a comprehensive health service master plan to better regulate the rehabilitation and expansion of public health care facilities.

(3) Health Care Financing (US$4.0 million). The component was intended to help the MOH monitor and more effectively contain its expenditure flows to the private sector and to assist in laying the groundwork for various health finance reform initiatives. Some activities intended to help curb MOH spending on patient referrals to the private sector included establishing standardized and computerized patient admission and billing systems in private hospitals under contract with the MOH. The health finance reform component consisted of five studies focusing on the following areas: (i) assessing the impact of MOH allocated expenditures on health care equity and cost-effectiveness; (ii) reviewing cost-recovery mechanisms within public hospitals and health centers; (iii) conducting a national household health expenditure and utilization study; (iv) conducting a study on the feasibility of creating managed care type institutions (Health Maintenance Organizations HMOs); and (v) undertaking a comprehensive review of health financing options in Lebanon.

PART B: Improving Delivery of Public Health and Clinical Services

(4) Rehabilitating Front-Line Referral Hospitals (US$20.0 million). The goal of this component was to rehabilitate six of the main public referral hospitals throughout Lebanon. These hospitals are in Tripoli, Zahle, Baalbeck, Batroun, Dhahr el Basheq and Sour. The activities of the component included major civil works and the procurement of medical equipment, furniture and vehicles. Rehabilitation and not hospital expansion was the clear intention of this component.

(5) Rehabilitating Health Centers (US$6.6 million). This component sought to rehabilitate ten public health centers located in underserved areas with broad population coverage and twenty active nongovernmental health centers. The public centers, as with the public hospitals, were to be physically

- 4 -

rehabilitated and equipped with medical equipment, furniture and vehicles while the private centers were to receive additional medical equipment, training opportunities and operational support for setting up and implementing contractual agreements with the MOH.

The five components, woven together, were designed to support the original project development objectives. Part A was designed to strengthen the institutional capacity of the MOH in a generally unwieldy health sector environment (i.e. weak regulatory control of capital investments and quality standards) and Part B was designed to improve health service delivery -- particularly for the poor. Two general concerns, however, could be raised about the content and the operational assumptions underlying the selected components. The first concern focuses on the US$20.0 million allocated for hospital rehabilitation (around 41 percent of the project loan) and whether more cost-effective and targeted public health initiatives (primary care, prevention and screening programs) could have done more to meet the unmet health needs of the poor especially in light of the already existing excess bed capacity in the hospital sector. As mentioned earlier, some good sector work prior to project delivery, had conditions been different, could have shed greater analytical light on the desirability and efficiency of incorporating a large "bricks and mortar" component within the overall framework of the project. The hospitals targeted for rehabilitation did indeed service the poor. Yet, perhaps, similar outcomes could have been achieved with less investment by assuring the poor greater access to primary health care facilities (whether public or private) and to private sector hospitals. This result could have been facilitated by a greater focus on primary health care initiatives and by more extensive development of effective public-private contracting mechanisms. This point related to the desirability of reconstituting the public sector hospitals (with very sizable capital investments) versus other approaches (i.e. contracting the private sector) was indeed debated within the Bank. The prevailing view leaned more towards the inclusion of a large hospital civil works component given the argued front-line referral function of the targeted hospitals and given the explicitness of client demand. The second concern relates to expectations about implementation capacity particularly in post-conflict conditions and in an environment in which public sector institutions have traditionally been weak. Expecting to build sector capacity, a sometimes three-to-four year process in itself, and simultaneously implement far-reaching reforms (in health financing options) in a six-year period seems to have been too ambitious. Perhaps having scaled down the policy reform dimensions of the project and strengthened the capacity-building aspects would have been a more reasonable option to follow especially in light of the six-year time horizon. Alternatively, a two-phased approach could have been contemplated extending the overall timetable beyond the proposed six-year time period. Since no general project activity in Lebanon preceded this health project, no useful lessons about operating conditions in the country and desired implementation arrangements were readily available to those who prepared its contents.

3.4 Revised Components:The components of the project were revised following extensive consultations between the Bank and the GOL during two mid-term review missions (January and July 1999). Three new priority areas were identified by the GOL: emergency medicine services, health quality accreditation and the strengthening of the health policy reform process. These revisions, particularly the last one, helped in bringing greater government attention and focus to the importance of health sector reforms. In order to fully complete the revised components the project loan was also later extended from the original closing date of December 31, 2000 to the end of 2001 and subsequently twice to December 31, 2003. These later extensions were probably justified given the many delays in civil works and procurement due to unfavorable climate conditions and regional political instability.

Based on these consultations and building on the explicit request of the GOL to focus on the above-mentioned priority areas, the following changes were made to the original project components:

- 5 -

The first component "strengthening essential administrative and managerial functions" was dropped 1.under Part A and its ongoing activities were integrated into the second component which was assigned a new name: "strengthening planning capacity." The new sub-components of the strengthening planning capacity component were renamed and became: (i) developing the carte sanitaire; (ii) strengthening needs-based master planning; (iii) enhancing epidemiological surveillance; and (iv) improving personnel management.

Two other components were added under the revised Part B -- improving service delivery. The first 2.component aimed to develop options for expanding and improving emergency medical services. These services, utilized more by the poor than the non-poor, have traditionally been very weak and under-financed in Lebanon. The second component sought to develop a hospital accreditation system for all hospitals in Lebanon. This measure broadened the service delivery focus of the project to include the private sector and all population groups -- poor and non-poor.

In addition to these new components, an adjustment was made to the number of public sector hospitals targeted for rehabilitation. The Batroun and Zahle hospitals were dropped from the list reducing the number of hospitals from six to four. This adjustment emanated from the ongoing concerns about excess capacity in the hospital sector. Political pressures were inducing greater public hospital expansion elsewhere and a decision was taken to reduce the number of public hospitals slated for rehabilitation.

3. A new "Part C" was added to the list of components -- "support to the development of reform." Under Part C, no clear and self-standing components were laid out yet in the official aide-memoire during the mid-term review but several strategic action steps and high priority areas were later identified. The action steps included the formation of a health reform council and a technical committee; the appointment of a technical committee director and the organization of a national symposium on health care reform. The priority areas, covered as well under Part A, included the development of a comprehensive needs-based master plan; the testing of a case-based hospital payment system and the completion of the health financing studies. The revised components reflected a shift in project development objectives from a "management" focus (i.e. reviewing the administrative structure and procedures of MOH under Part A) and an exclusive focus on the poor to a "planning and monitoring" focus and to a broader set of objectives encompassing all population groups in Lebanon. This shift again to a broader set of policy objectives without first having achieved some of the key building blocks of reform (i.e. introducing positive organizational change within the MOH whether through institutional restructuring or civil service reform) was risky but proved worthwhile in view of the end results of the project.

3.5 Quality at Entry:Project preparation and implementation began prior to the creation of the Quality Assurance Group (QAG) and hence no prior assessment of quality at entry is available. This Implementation Completion Report (ICR), on balance, rates overall quality at entry as “unsatisfactory.” The project was too complex and demanding from the very beginning. Although the stated development objectives were consistent with the Lebanese development priorities embodied in the NERP and later with the 1998 CAS, project design from both a substantive and operational point of view, as well as implementation readiness, exhibited several important shortcomings. These shortcomings, as discussed before, include the following:

An over-emphasis on the rehabilitation of public sector hospitals (about 42 percent of the project lloan amount) given the available hospital bed capacity in the private sector (in 1993). Instead of focusing on large-scale capital investments, more attention could have perhaps been devoted to developing the right contracting and payment mechanisms to assure greater access for the poor to

- 6 -

the privately provided health care services which contribute the bulk of Lebanon’s health delivery system. Some mechanisms were indeed developed over the life of the project (i.e. prospective flat rate payments for specific hospital services) -- yet the development and implementation of these types of mechanisms could have received more focused attention and resources much earlier in the life of the project.

A considerable degree of complexity and some redundancy. The project consisted of five lcomponents, eleven sub-components and covered both civil works and major health policy reform initiatives. Some overlap also existed between identified sub-components (e.g. establishing a national health information system and establishing a GIS-based health map or carte sanitaire) which could have easily been addressed during the preparation stages of the project.

Overly optimistic expectations for the timely implementation of certain project components namely lthe organizational restructuring of the MOH (a process heavily dependent on broader civil service reform efforts) and the introduction of health financing reform initiatives particularly in light of the traditional low capacity of the public sector in Lebanon. A six-year project time frame was unrealistic.

Requirements for implementation readiness were not fully in place following loan effectiveness. lDetailed implementation plans for the rehabilitation of the identified hospitals and health centers (i.e. architectural plans, equipment lists, training plans, bid documents) had not been developed and the exact implementation arrangements between the MOH, CDR and the project coordinating unit had not been fully worked out prior to Board approval and loan effectiveness.

Out of the eight assessment criteria used to evaluate quality at entry, two are deemed to have been satisfactory (environmental analysis and financial management) and the remaining six criteria overall unsatisfactory (project approach, technical and economic analysis, social and stakeholder analysis, institutional capacity analysis, readiness for implementation and risk assessment and sustainability). Greater analytical work and more effective Bank preparation would have clearly helped in enhancing the quality at entry.

4. Achievement of Objective and Outputs

4.1 Outcome/achievement of objective:In line with the learning and accountability functions of an ICR the achievements of the project are rated against the original project development objectives. This ICR finds the overall achievement of project objectives as “satisfactory.” The following paragraphs discuss project achievements vis a vis the original development objectives.

The project’s first short-to-medium term objective to enable the MOH to more effectively perform critical management and administrative functions was largely achieved through a number of reform initiatives (e.g. GIS-based health mapping, NHA, burden of disease analysis, and provider payment reform). Even though institutional strengthening of the MOH progressed slowly during the first phase of the project, considerable progress was made during the second phase. The development of NHA enabled the MOH to better monitor and evaluate public and private health expenditure while the provider payment reforms including the insurance database integration activity (i.e. visa-billing) significantly enhanced the capacity of the MOH to better serve the poor and control its expenditures (e.g. 100 percent progress was achieved in terms of developing the database integration activity -- See Annex 1). The Geographic Informaion System (GIS)-based system has allowed the MOH to spatially analyze and track health care investments throughout the country. These initiatives represent some of the most far-reaching health reform initiatives in the MENA region and, indeed, in the Bank as a whole.

- 7 -

The project’s second short-to-medium term objective “to respond to the unmet health needs of poor population groups who rely on public services particularly at selected health centers and general hospitals” was fulfilled via the rehabilitation of four public hospitals and ten public health centers, the integration of the public insurance databases, the introduction of improved management practices in 30 NGO health centers and the implementation of the hospital accreditation program. These elements of the Bank project made a substantial contribution to the MOH’s capacity to meet the health needs of the poor. Only the emergency services component failed to meet its objectives. More attention could have been paid to intensifying and expanding the activities of this component.

Assessing the achievement of the project’s long-term objective to improve health conditions through better allocating and using resources in the public and private sectors is somewhat problematic given a number of factors: (i) long-term impacts of recently implemented activities cannot be adequately assessed at project completion (ii) general problems with data availability and (iii) questions surrounding attribution. Outcome indicators, like infant mortality and life expectancy, did indeed improve during the life of the project (e.g. the infant mortality rate decreased from 31 deaths per 1,000 live births in 1992 to 26 deaths in 2002) yet attributing those improvements to specific reform activities is almost impossible. Notwithstanding these concerns, progress related to certain “intermediate” indicators can be noted. These indicators point to improvements in efficiency and resource-allocation which in turn affect health outcomes. One indicator relates to the cost incurred by the MOH for referring hospital admissions to the private sector. On account of the provider payment reforms initiated by the project, the cost per hospital admission decreased by more than 15 percent (Source: Ammar, Walid (2003) Health System and Reform in Lebanon, World Health Organization, Geneva). This improvement in efficiency allows for greater indigent coverage provided by the MOH. Another indicator relates to resources allocated to health care quality improvements. By the end of the project, around 68 hospitals had made the necessary investments to elevate their quality standards to a basic quality accreditation level (See performance indicators in Annex 1).

4.2 Outputs by components:The following discussion reviews the outputs of the revised components following the mid-term review:

Component 1: Strengthening the Planning Capacity of the MOHThis component is rated as satisfactory. This component produced several important outputs. A GIS-based health map or carte sanitaire using modern software was developed by the MOH planning directorate to help rationalize national health care investments. Accompanying legislation has also been developed to better ensure the effectiveness of the carte sanitaire as a tool to regulate those investments. Given the predominance of the private sector in Lebanon (90 % of hospital beds) and in the absence of comprehensive needs-based planning such an initiative is of critical importance. A burden of disease study was also implemented under the project. Although the study was unable to produce quantifiable estimates of the burden of specific diseases (e.g. summary measures of population health like DALYs or DALEs), it did bring to light the importance of accurate mortality and morbidity data and focused on ways to improve the current methods of data collection in Lebanon. Using ICD-10 coding procedures for cause of death reporting in all health facilities is one such proposed way to improve the quality of mortality information.

Component 2: Health Financing This component is rated as satisfactory. A series of very innovative outputs were produced under this component. A new provider payment method (i.e. flat-rates for surgical and medical procedures in private hospitals under contract with the Ministry of Health) was developed to help contain the rising costs of private health care services purchased by the public sector. This method of flat rate-based payments is now being followed by the development of more sophisticated diagnosis-related groupings (DRGs). In

- 8 -

conjunction with the flat-rate payment methods the Ministry undertook an activity to electronically integrate the enrollee databases of the various public sector health insurance schemes (e.g. Social Security, Security Services) and streamline the process by which publicly uninsured Lebanese citizens could gain coverage from the Ministry of Health. This latter process by which uninsured patients obtain MOH coverage for private hospital services has been termed by the MOH as the "visa-billing" process (i.e. patients have to first obtain a MOH "visa" or official letter to receive care in private hospitals -- the term does not refer to VISA credit cards). These activities were designed to provide more efficient health care coverage for the poor and prevent the misuse of MOH coverage by already publicly insured individuals. These objectives have largely been achieved. Other important outputs under this component and implemented through a contract with WHO include a national health expenditure and utilization study, a health financing reform study estimating the impact of alternative health insurance options and the development of National Health Accounts (a critical health policy tool for undertaking health financing reforms).

Component 3: Management and Rehabilitation of Health CentersThis component is rated as highly satisfactory. This component witnessed the rehabilitation of ten public health centers including civil works, medical equipment and furniture and the provision of operational support to thirty private health centers under contract with the Ministry of Health. In an effort to support primary health care services provided by these health centers to the poor, a standard package of core services and quality standards were developed jointly by the MOH and various NGOs. A national “empowerment of the periphery” program in line with the trend towards decentralization was also launched by the Ministry of Health to strengthen the role of the eleven district health officials in charge of these centers. This program focused on developing district-level health information systems for better master-planning at the local level and the training of human resources in health resource management and overall computer literacy.

Component 4: Management and Rehabilitation of Front-Line Public HospitalsThis component, overall, is rated as satisfactory. Under this component, four front-line public sector hospitals were rehabilitated and equipped with medical equipment and furniture and have begun servicing poor population groups in line with one of the original project development objectives. The number of hospitals targeted for rehabilitation was reduced from six to four during the mid-term review due to an increased awareness of the "overbeddedness" of the Lebanese health sector. The bed capacity of all four rehabilitated hospitals: Tripoli, Baalbeck, Dhahr Al Bashek, and Shahaar Al Gharbi also increased by almost a hundred percent – a development running contrary to the initial design of the project which emphasized hospital rehabilitation over hospital expansion. In addition to the physical rehabilitation of the facilities, new legislation was developed and implemented to grant greater autonomy to the administrative boards of these hospitals. This effort at hospital “autonomization” is a critical first step to decentralize hospital management procedures yet still faces several bureaucratic obstacles.

Component 5: Improving Emergency Medicine (EMS)This component is rated as unsatisfactory. This component produced modest results compared to initial expectations during the mid-term review. Initially the component aimed to implement plans for comprehensive Emergency Medical Services (EMS) in two or three pilot areas (namely Beirut, Tripoli and Baalbeck) together with a media awareness campaign and the training of human resources in basic and advanced EMS life support procedures. The latter two activities did indeed take place (awareness campaigns and human resource training) but were not systematically and closely coordinated with pilot activities on the ground and did not lead to a comprehensive strategy to improve emergency services in the country.

- 9 -

Component 6: Initiating a Hospital Accreditation ProgramThis component is rated as satisfactory. A significant amount of progress was achieved under this component to initiate a hospital accreditation program in Lebanon (very few middle income countries have attempted such an initiative). The first phase of the program focused on setting basic hospital accreditation standards (implemented in May 2000). These standards focused on basic physical and hospital design-related standards (i.e. sanitation, uninterrupted power supply). The second phase included a national survey covering 134 hospitals to assess conformity with these standards. The results of this survey were then used to determine the awarding of MOH contracts (linking accreditation to MOH payments to hospitals under contract). Following the second phase, 68 hospitals had achieved accreditation status. The third phase concentrated on developing (completed by December 2003) a revised and more demanding set of standards with a shift in focus from monitoring quality related to “structure” and “process” to quality related to management and clinical "outcomes."

4.3 Net Present Value/Economic rate of return:Not Applicable.

4.4 Financial rate of return:Not Applicable.

4.5 Institutional development impact:The impact on institutional development is considerable with the anticipated continuation of the established programs and systems initiated by the project (i.e. hospital quality accreditation, provider payment systems, empowerment of the periphery program, “visa-billing” and the GIS health mapping effort). The new payment system currently in place (i.e. flat-rate hospital reimbursement) has significantly improved the institutional capacity of the MOH to control and regulate its contractual arrangements with the private sector. The modified Diagnosis-Related Groupings (DRGs) system will further strengthen the hospital reimbursement program. These provider payment initiatives have also been linked to the quality accreditation program (i.e. contracts will only be signed with accredited hospitals). This operational link between payment and accreditation has reinforced the introduction of quality systems within Lebanon’s health care institutions. The other systems introduced by the project, like the public insurance database integration system and GIS mapping system have all had a positive impact on the institutional capacity of the Ministry and lay the base for the second phase of major reforms. Using the GIS health mapping system, the MOH can now monitor the geographic distribution of health investments and resources in the country (i.e. facilities, equipment and human resources). A further step, if desired by the GOL, would be to use the GIS system not only as a monitoring devise but as an information tool to help rationalize the flow of public and private investments into the health sector.

5. Major Factors Affecting Implementation and Outcome

5.1 Factors outside the control of government or implementing agency:Public revenues: one constraining factor on the capacity of the Ministry to absorb more skilled human lresources was the varying level of MOH revenues during the life of the project. Due to continuing unstable regional conditions, tax revenues from tourism and other sectors were duly affected and restricted the financial capacity of all sectors to recruit skilled permanent staff members.

5.2 Factors generally subject to government control:Overall government commitment: a long lag in loan effectiveness (10 months), continuous changes in ltaskforce assignments (set up early on in the life of the project to initiate all activities under the different components), periodic suspension of PCU staff contracts (i.e. in 2000), slow civil service reforms, lack of counterpart funds and the absence of a comprehensive master plan to control overall

- 10 -

public sector health care investments indicated varying levels of government commitment toward comprehensive health sector reform.

5.3 Factors generally subject to implementing agency control:Weak project implementation/procurement delays: tensions and problems with coordination among lstaff members of the CDR, the MOH and the PCU led to significant procurement delays in the first few years of the project and for a two-year period between 1998 and 2000. High turnover of PCU staff and changes in project management also negatively affected project implementation.

5.4 Costs and financing:The total completion cost of the project was US$38.4 million, about 80 percent of the projected costs lof US$48.12 at the time of project appraisal. This difference between projected and completed project costs is due to certain over-estimated costs at appraisal (i.e. costs for implementing health care financing reforms and the rehabilitation and management of various health centers) and changes in project components during the mid-term review.

As to financing, the Bank was to fund originally about 75 percent of the project (US$35.7 million of lUS$48.1) and the GOL was to contribute the remaining 25 percent (US$12.4 million). At project completion, these percentages changed to 87 percent (US$33.3 million of US$38.4) financed by the Bank and 13 percent funded by the GOL (US$5.03 of US38.4). This discrepancy in financing percentages resulted from: i) a loan agreement amendment in 2000 following the mid-term review that adjusted upward the Bank disbursement percentages of the "services" and "operational costs" expenditure categories; and ii) a lack of available counterpart funds.

Due to project complexity, a long delay in loan effectiveness, and many procurement difficulties, the lproject closing date was extended three times (closed in December 31, 2003) and the project development objectives were revised once (during the mid-term review).

6. Sustainability

6.1 Rationale for sustainability rating:The sustainability of Part A – strengthening the planning, monitoring and quality assurance capacity of the MOH – including the quality accreditation program under Part B is likely given the already established government programs and systems put in place by the project. The visa-billing and payment systems are fully operational and are currently being managed by 25 ex-PCU database operators. These operators, currently on contract by the MOH, are expected to be fully absorbed (as regular civil service employees) by the MOH over the near term. The GIS mapping system is managed fully by MOH staff and is expected to continue as a government-funded initiative. The hospital accreditation program has already secured government resources for the next two years and has ensured the involvement and financial commitment of the private sector over the long term. The sustainability of these initiatives, again, are very likely given the accrued benefits to the MOH and the Lebanese public at large. The government has a huge incentive to continue implementing these various reform efforts.

One sustainability concern relates to the integration of contracted PCU professional staff into the cadre of the Ministry. This is critical for the continued support to some of the systems developed under the project. However, attracting qualified professional staff into public administration is an issue that goes beyond the project, given the stagnant civil service salary scales. Several ministries, among them MOH, have dealt effectively with this problem through the administration of a system of contracting parallel to that of the civil service. MOH has already contracted with six ex-PCU technical staff and is proposing to extend their contracts upon expiration.

- 11 -

The sustainability of the public hospitals rehabilitated under the program is likely yet not without a few obstacles along the way. Even with the apparent demand for the services of these “autonomized” public hospitals, it is not entirely certain that these hospitals, given the traditionally low fee structure of public sector services will be able to absorb all the recurrent costs in the near future. The new independent managing boards will have to revise their fees structures to ensure the financial soundness and sustainability of their operations, an issue raised by at least two of the hospital directors whose hospitals have been rehabilitated by the project. As to the sustainability of the rehabilitated health centers and the empowerment of the periphery program, it is likely given the limited increase in new recurrent expenditures attributed to the Bank project. Only some minor maintenance and depreciation costs associated with the installed information systems are likely to be incurred.

6.2 Transition arrangement to regular operations:The transitional arrangements set up for the reform initiatives have been partially addressed through the transfer of 25 junior employees and six professional staff from the PCU to the MOH as contracted staff. These employees, as stated before, will be responsible for continuing the various reform initiatives under Part A and the quality accreditation program under Part B. The junior employees are expected to be easily absorbed into the civil service (given similarities in salary scales) while the professional employees are not likely to become public servants. In either event, the professional staff will most likely continue working on the reform initiatives as contracted employees of the government (a widespread practice among certain key ministries in Lebanon).

7. Bank and Borrower Performance

Bank7.1 Lending:Given the absence of an explicit health sector component in the National Emergency Reconstruction Program (NERP), the Bank appropriately identified a need to restore and strengthen the capacity of the Ministry of Health and to assure the delivery of key health care services to the poor. In approaching this issue, the key question it faced was whether to invest in resurrecting public sector provision for the poor (i.e. by rehabilitating several frontline hospitals) or to invest in developing effective methods to contract with and regulate the private sector. It chose the former and gave greater weight to hospital-based public sector provision (in an environment with a predominance of private sector beds) than to primary health care services and to strengthening public sector capacity to make use of the already existing hospital-based capacity in the private sector. Aside from this concern on substance, operationally, the project was too complex and demanding particularly in light of the post-conflict environment and the historically weak capacity of the public sector.

In terms of the right skill mix during appraisal, perhaps two types of specialists should be have been involved in a much closer way right from the beginning: (i) an experienced health planner with expertise in the development of health sector master plans who could have enhanced the design of the health sector rehabilitation component of the project; and a (ii) a seasoned implementation specialist with operational experience in post-conflict environments who could have probably simplified the project design and improved the implementation arrangements on the ground. Overall Bank lending performance is rated as unsatisfactory.

7.2 Supervision:Supervision missions following project preparation and pre-appraisal were relatively short in duration in light of the continuous and critical supervision requirements of initial project implementation in a post-conflict environment. Following the project launch mission in February 1996, three one-week supervision missions had occurred by February 1998. The composition of these supervision teams was also

- 12 -

somewhat inadequate given the wide array of activities under scheduled implementation by the project. The mission in February 1997, for example, consisted of only a health specialist, an operational analyst and a building specialist/architect. Morevoer, individual team members changed frequently from one supervision to another. In addition to the short duration of the supervision missions and questions related to team composition and the continuity of individual membership, early project documentation was not fully up to speed. The first Project Status Report (PSR) to be fully completed was in April 1997 – 14 months after the project launch and 18 months after loan effectiveness. Implementation progress of various activities, therefore, was not carefully monitored and reported during the early stages of the project. Such monitoring and reporting could have signaled the need for project revisions much earlier than the scheduled mid-term review date.

The mid-term review missions (January-August 1999) which led to certain project revisions and the development of a new health policy framework with the MOH represented a significant enhancement in the level and quality of Bank supervision. High-level teams of various specialists worked closely with the MOH over the mid-term review period to redesign the rehabilitation project and to strengthen the then new policy framework for health reform in Lebanon. Following the mid-term review mission, greater commitment to project implementation and the achievement of the revised development objectives was revealed by the MOH, the CDR and other participating stakeholders. In the period between the MTR and the project closing date, Bank performance in supervising the project improved significantly leading to greater implementation progress and loan disbursement. Cumulative loan disbursement increased from about 4.1 million in 1999 to around 14.6 million in 2002. The primary reason behind the improvements in supervision and project disbursement lie with the much closer involvement of the responsible sector director and sector manager; the decision of the Bank to open a country office in Beirut and locate a resident supervision staff responsible for the project and the re-assignment of procurement responsibilities from the CDR to the staff of the MOH and PIU. This in-country presence by the Bank afforded closer and continuous supervision, enhanced the financial management of the project (i.e. financial reporting and record-keeping) and clearly aided in smoothening project implementation and opening up the lines of communication between the various stakeholders involved in the project. The final result has been the development of some of the most-far reaching health sector reforms in the MENA region, particularly the health provider payment initiatives and the hospital accreditation program.

Although early Bank supervision exhibited certain weaknesses, significant improvements were made towards the mid-term and later stages of the project. In light of this important progress, due to in-country Bank presence and greater senior management supervision, Bank performance in terms of supervision is found to be satisfactory.

7.3 Overall Bank performance:Even though Bank performance in terms of lending was earlier rated unsatisfactory, overall Bank performance is rated as satisfactory given the substantial improvements in Bank supervision towards the end of the project cycle and the positive end results achieved on the ground.

Borrower7.4 Preparation:Prior to project appraisal, the borrower, specifically the MOH, displayed significant commitment towards adequately designing and preparing the project for Bank approval. The close involvement of the Minister himself and key MOH officials during the period January-December 1994 assisted greatly in the preparation of all the necessary design and survey work preceding project appraisal. By March 1994 (five months after the first Bank identification mission in October 1993) the MOH had already surveyed 53

- 13 -

health facilities and six general hospitals across the country. Following appraisal, greater preparation work for implementation (i.e. preparing architectural designs and necessary equipment lists) could have perhaps been undertaken during the lag period between Board approval (December 1994) and loan effectiveness (October 1995). On balance, given the strong support of key MOH officials during appraisal and the active involvement of various taskforce to prepare the project, borrower preparation is rated as satisfactory.

7.5 Government implementation performance:Government commitment to project performance varied during the lifetime of the project. During the early years between 1995-1998, the Government was slow on implementing necessary civil service reforms, specific health financing reforms (i.e. provider payment methods) and necessary civil works as proposed by the project. The government also did not, at times, move speedily enough to assure the smooth functioning of the PCU. At the time of the mid-term review, however, a significant amount of commitment was shown by the government to improve project implementation and to situate the project within a broader health policy reform perspective. An inter-ministerial committee was set up to push the reform process forward and very important gains were made in the areas of hospital accreditation, visa-billing, provider payment, NHA, GIS-based health mapping, and burden of disease. The government successfully introduced a national hospital accreditation program (the first country to do so among all MENA countries) to the relevant stakeholders and then began implementing the program in conjunction with the provider payment reforms. The implementation performance of the government is rated “unsatisfactory” for the first four years of the project. However, beyond 1999, particularly after the MOH and PIU assumed greater procurement responsibilities, it is rated as satisfactory. In view of the exceptional results achieved towards the end of the project (especially when compared to the country portfolio and the regional health portfolio), overall, government commitment to creating the right environment for project implementation is rated as satisfactory.

7.6 Implementing Agency:Two major factors affected the early performance of the CDR, the MOH, the PCU and the WHO – the concerned entities with project implementation. These factors were (i) insufficient capacity and expertise in procurement and financial management procedures; and (ii) inadequate communication and unnecessary tension among the different members of the different entities. These two factors seriously impeded implementation and procurement plans during the first few years of the project. The CDR, as the main implementing agency, should be held accountable for many of the implementation delays and operational bottlenecks. Towards the mid-term review and beyond, implementation improved markedly as the PCU and the MOH assumed greater implementation responsibility and as capacity slowly matured within the CDR. The WHO which assumed responsibility for the management of several health financing-related studies, performed relatively well in implementing the various health financing studies. This ICR rates the overall performance of CDR as unsatisfactory and rates the performance of the MOH, the PCU and the WHO as satisfactory.

7.7 Overall Borrower performance:In view of the satisfactory rating for preparation and government implementation and a partial satisfactory rating for implementing agency performance, the overall performance of the borrower is rated as satisfactory.

- 14 -

8. Lessons Learned

Designing comprehensive projects that require elaborate policy frameworks and complex institutional larrangements should be avoided at all costs in post-conflict situations, as it sets the project against unrealistic targets from the start.

The Bank should take a longer perspective and a more incremental approach to its engagement in lpost-conflict situations especially when designing projects that aim at introducing sector reforms in a fragmented policy environment lacking political consensus.

Bank teams preparing projects in post-conflict environments should undertake a thorough analysis of lthe prevailing governance structure in the concerned sector including a close review of the institutional absorptive capacity and the decision-making processes leading to the adoption of major policy reform initiatives.

In the event that one public entity assumes all responsibility for procurement arrangements in lpost-conflict environments (as the case of CDR in Lebanon), the Bank should ensure the transfer of procurement capacity to the relevant line ministries. Such a transfer of standardized knowledge can significantly boost the overall management capacity of the different sector ministries.

The Bank should ensure in-country supervision for most projects, especially if they are complex in lpost-conflict environments. Attempting to build capacity and supervise a project from Bank headquarters or a regional office in these cases does not usually ensure effective implementation.

Revisions in project objectives and components should be invoked at an early stage of the project’s life lwithout unnecessary delay should the conditions for successful implementation prove to be beyond the executing agencies’ institutional capacity. That option saves the Bank and the borrower time and resources that could be deployed in a more cost-effective way to achieve the project’s development objectives.

The Bank should consider very early on the issue of sustainability and the challenge of integrating the lproject staff into the regular staffing structure of the relevant line ministry. This issue is particularly relevant where there exists a significant salary differential (as is the case in most countries) between the hired project staff and the regular line ministry employees.

The Bank should capitalize on unintended lags in loan effectiveness to strengthen the quality of baseline ldata initially collected for overall impact evaluation at the end of the project cycle. Such time lags allow for greater refinement of the peformance indicators chosen for monitoring and evaluation.

- 15 -

9. Partner Comments

(a) Borrower/implementing agency:

REPUBLIC OF LEBANON

MINISTRY OF PUBLIC HEALTH

HEALTH SECTOR REHABILIATION PROJECT

Implementation Completion Report

May 2004

1. Project Data

Name: Health Sector Rehabilitation Project (HSRP)Country: LEBANONRegion: Middle East and North Africa

Basic Data:Loan # 3829Amount (WB) $35.70 million Disbursed (WB) $33.341million Disbursement (WB) 94 %

Board Date 12/20/94 Effectiveness Date 10/04/95 Closing Date (original): 12/31/00 Closing Date (actual): 12/31/03 Months between. Board & Effectiveness Date 9.6

Borrower: Government of The Lebanese Republic (GOL).Beneficiary: Ministry of Public Health (MOH) Implementing Agencies: Ministry of Public Health / Council for Development and Reconstruction (CDR)

- 16 -

2. Assessment of Development Objective and Design, and of Quality at Entry

2.1 Original Objectives:

The long-term objective of the project is to improve the country’s health conditions through better allocation and use of resources in the public and private sectors. The short- to medium–term objectives are: (1) to enable the Ministry of Public Health (MOH) to effectively perform management and administrative functions that are critical to further development in the sector; (2) to respond to unmet health needs of poor population groups who rely on public services, particularly selected health centers and general hospital.

The objectives were ambitious but appropriate at the time, in view of the weakened institutions after more than a decade of social and political instability. The Bank’s rationale was to assist the Government in restoring its role in the health sector, correcting weaknesses in services and programs, and making better use of existing external assistance. It was part of the Bank’s approved Emergency Reconstruction and Rehabilitation Program (March 1993). The HSRP was meant to assist in the institutional development of the MOH, as well as in the rehabilitation of selected public hospitals and health centers.

The Government’s strategy for the development of the health sector focused on rehabilitating the public health system to enable it to deliver the basic services needed to meet the health needs of the poor population.

Most of the planned interventions were proposed along the lines of the traditional functions of the MOH providing services and covering services, though some structural reforms issues (Health Care Financing in particular) were announced in the Project’s reference paper. Systems and processes where developed within the existing organizational objectives.

Over the years of the project’s implementation, the MOH has introduced innovative strategies and objectives, which have changed the initial policy framework in which the project was first conceived.

Examples of such changes are the introduction of the autonomy for public hospitals, changes in the payment mechanisms for hospital services, interest in leading and regulating the overall health system development, development of the health information system, and the development and implementation of a hospital accreditation program.

Even though no clear and formal policy decision has been made on the delivery system run by the MOH, the trend towards increasing autonomy or any other form of reducing the direct involvement in the delivery of care is putting the onset on the contracting role of the MOH.

The major responsibility of the MOH, at this point in time, is the management of large funds allocated to buy hospital services from private and autonomous public hospitals.

In addition, the MOH has increasingly asserted its role as a regulatory body in the health sector.

The HSRP was complex and demanding for the MOH and, therefore, risky. This was mentioned in The Staff Appraisal Report (SAR) that noted “the limited capacity of the MOH to absorb development projects makes it necessary to keep the project as simple as possible”.

The project’s original objectives were clear, but the SAR and the legal documents did not include clear indicators of success. Therefore, if the project had not been restructured, an assessment of the degree to which the original objectives have been achieved would have certainly been difficult.

- 17 -

2.2 Revised Objectives:

A two-phase mid-term review was completed in July/August, 1999. Based on the review, discussions have begun to restructure the design of the project to support the development of a reform framework and to more clearly direct the project impact to improving the quality and efficiency of the public sector.

New activities were introduced in support of the comprehensive health care reform, mainly in areas related to emergency medicine, quality, and hospital payment reform. Ongoing activities that have achieved their objectives have been discontinued or integrated in the MOH. Those that were successfully contributing to the revised project objectives were continued. Those that were not progressing satisfactorily were terminated.

The mid-term review carried out led to proposals for restructuring of the project in the light of the sector reforms initiated by the MOH.

As the project Development Objectives can only be fully achieved in the context of an overall health reform program being implemented, the prospects for achieving project impact seem difficult at best.

Despite these risks, the Government was committed to health reform and to the rationalization of expenditures and specific actions have been discussed to mitigate these risks. Based on the discussions and recommendations from the first half of the mid-term review, the President has issued a decree establishing a Health Reform Council supported by a Technical Committee whose goal is to develop a comprehensive health sector reform program for Lebanon.

It was clear after the mid-term review that not much of these components or their developmental objectives could be altered.

The new components that were introduced related to:Emergency Care.1.Quality Assurance.2.Payment Mechanisms.3.Health Reform.4.

Indeed, the project restructuring involved mainly changes in management, approach and focus, rather than changes in the substance of the development objectives.

2.3 Original Components:

A. Institutional component:

This component had three objectives:

I. Restore Essential Administrative and Managerial Functions of MOH1.1 the project will assist MOH’s staff in:

(a) Reviewing the administrative structure and procedures of MOH.(b) Developing and establishing a national health information system(c) Strengthening the management of health centers(d) Strengthening the management of public hospitals

- 18 -

II. Re-establish Planning and Normative Functions in the Sector1.2 the project will assist MOH’s staff in:

(a) establishing a “Carte Sanitaire” of the country(b) planning for further development of health services

III. Prepare and initiate a Reform Plan for Health Care Financing1.3 the project will assist MOH’s staff in:

(a) containing costs of MOH expenditure for private services(b) preparing reform in health sector financing

B. Improving Health Care Delivery:

I. Health Centers

1.4 The project will provide the necessary civil works, equipment and furniture, vehicles, training and medical supplies to bring the 10 public health centers up to a national standard.

1.5 The project will provide the additional equipment, training and operational support needed to implement MOH/NGO contractual agreements on a pilot basis in 20 health centers run by leading NGOs.

II. Frontline Referral Hospitals

1.6 The project will rehabilitate physical facilities at six of the main public hospitals.

1.7 The project will provide the necessary civil works, equipment and furniture, vehicles, and technical assistance to rehabilitate the buildings and equipment at the six hospitals.

2.4 Revised Components:

Based on the revised project objectives, the restructured project supported the following activities /components:

A. Strengthening the planning, monitoring and quality assurance capacity of MOH and rationalizing health sector financing.1. Strengthening the planning capacity of MOH

a) Carte Sanitaireb) Needs-based master planningc) Epidemiological surveillanced) Personnel Management

2. Rationalizing Health Care Financinga) Cost-containmentb) Health care financing options

B. Improving service deliveryManagement and rehabilitation of health centers1.

- 19 -

Management and rehabilitation of public hospitals2.Improving Emergency Medical Services3.Initiating a quality improvement program4.

C. Supporting the development of a broad health sector strategy

2.5 Quality at Entry:

In retrospect, and with the wisdom of insight, the quality at entry seems to have been partially satisfactory, essentially because of a somewhat deficient project design for such a complex operation being implemented in the context of weakened post-war institutional capacity.

On the positive side, as mentioned above, the project’s objectives were consistent with the government rehabilitation priorities at the time.

At the start of the Project, the agreed arrangements for implementation of the project were that the MOH was responsible for the project, while the CDR would handle procurement on behalf of the MOH. Within the MOH, the Director General (DG) would have the overall responsibility for the project including, in particular, monitoring the work of the appointed Task Forces .As for day-to-day project administration and monitoring and coordination of all activities of the project, the Project Coordination Unit (PCU) of the MOH handled it.

Several task forces were established within MOH to achieve these objectives. Each task force was composed of:

(a) Existing MOH staff; (b) Additional staff who will be recruited specifically for the project and will be gradually incorporated

in MOH’s regular staffing structure;(c) Local and international consultants; and(d) An advisory group consisting of individuals or institutions such as NGOs, universities and

processional associations that have a particular expertise or play a key role in the area concerned.

In addition, an MOH-CDR committee was created and was expected to meet twice a month.

This has ensured that all departments and services of MOH are fully involved in project implementation, with appropriate strengthening, if needed. The project has been set on the output of task-oriented activities. Task forces have been working to improve the performance of different departments. MOH employees had more experience on the current state of affairs, but they lacked skills to prompt changes. These constraints have reduced the staff efforts to set common goals with the MOH head of departments. This practice has been effective in improving the working relationship between task force members and MOH staff that has resulted recently in a smooth transition of the PCU members into the structure of the MOH after the closing of the project.

Better results have been achieved in cases where departments have been able to identify their own priorities and integrated the newly recruited staff in moves started from within the existing MOH services. The absence of well-defined and accepted Department missions has left much of the interventions to deal with the day-to-day routine, response to crisis, contradictory decisions, and low engagement. Institutional building has not gone very far in the absence of clear department objectives.

- 20 -

Even when project members have developed a consistent framework for action, it has been difficult to incorporate such actions in the department mission. The main reasons behind the lack of effective implementation of the proposed actions relate to the overall government bureaucracy and the legislative and regulatory context of the Lebanese administration in the absence of such broad based administrative reform it was unfeasible to incorporate all of the proposed structural changes in the MOH.

During the first five years of implementation, the performance of the project faced a number of problems for reasons related to:

Delays in project start-up;lLack of timely decisions due to the dilution of responsibilities of different parties involved;lUnclear mandate and authority of organizations involved in project implementation; lInadequate supervision and technical support of the World Bank.lThe World Bank's missions skills composition were inadequate, the constant rotation of the consultants lthat joined the missions affected negatively the continuity of the work.

Many of the project risks identified at appraisal have affected the overall implementation of the project. These included the clarity of roles of the MOH, the CDR and the Project Coordination Unit (PCU).

The primary reasons for the excessively high disbursement lag during the first five years of the project were, namely:

(a) Bureaucratic procedures of MOH and CDR and unclear responsibilities and roles of the different parties.

(b) Procurement has been seriously delayed by tensions between the CDR and the PCU;(c) Dilution of responsibilities of different parties involved.

It is clear that the original project had underestimated the risks associated with implementing such a complex operation. The design had paid insufficient attention to project implementation, organization and management issues, especially between the MOH, PCU and the CDR. Most of the deficiencies were related to the implementation arrangements, rather than technical problems.

3. Achievement of Objective and Outputs:

3.1 Outcome/achievement of objective:

At the same time that the project objectives were revised, performance indicators covering project output and outcome/impact were revised. An effort was made to identify indicators that the MOH could collect and monitor. A summary of those indicators is shown in Annex I. It should be noted that, for some of the outcome/impact indicators, the achievements shown cannot be attributed solely to the HSRP project, but are also the results of investments and actions undertaken and/or financed by government and other donors. Some of the targets were too optimistic and unrealistic. With those qualifications, the overall outcome of the project, compared to the revised objectives can be considered satisfactory.

With regard to the strengthening of the MOH administrative capacity, some progress has been made regarding the development of administrative support information systems and capacity building training activities.

Within the current MOH operational functions, there is little doubt that most of the concerns are focused, at

- 21 -

the moment around one major issue: how to improve the contracting position for the purchasing of hospital care, especially in the private sector. The question of providing direct care is not as important as the first one because of the size and resources allocated for the purchased services.

Concerning the objective of Improving Heath Services Delivery, improving the condition of buildings and equipment was valid and a prerequisite to improving quality of service delivery. .

Lebanon is oversupplied with hospital-beds and sophisticated equipment, there exists disparity between mohafazas. Up-to now over-investment has been justified by damaged health facilities. The lack of regulating mechanisms has allowed the private sector to grow in a very chaotic manner. In addition, the MOH sees the current construction and rehabilitation of public hospitals, under the autonomy law, as a means of: (i) improving quality of healthcare both in the public and private sector; (ii) decreasing the number of private hospitals under-contract with MOH; (iii) increasing the coverage of rural areas. At the same time, public and private sectors should be complementary. Competition between them might be unrealistic.In brief, all new construction or equipment projects of public hospital (as well as private) and health centre should be decided according to the Carte sanitaire.

The Support to the development of Health Sector Reform strategy has been tackled through various structural reform initiatives (hospital autonomy, carte sanitaire, hospital payment system, hospital accreditation and information systems). Administrative and organizational changes at the MOH level are still pending. The role of the MOH as a purchaser and provider need to be clarified .The MOH may need to devolve some of the present prerogatives in both areas to more effective and efficient forms of financing and delivery systems. It has however to strengthen its role as a policy and evaluation authority. This would entail several procedural and organizational changes.

(a) The objective of strengthening the planning, monitoring, and quality assurance capacity of the MOH and rationalizing health sector financing was achieved.

(b) The objective of improving health services delivery has been fully achieved. (c) Regarding the objective of supporting the development of a broad health sector strategy, it was

only partially achieved.

3.2 Output by components:

By project components, the assessments are as follows:

A- INSTITUTIONAL STRENGTHENING OF MOH

A.1- Administrative structure and procedures A.1 (a)- Reviewing the administrative structure and procedures of MoH

MOH organizational structure

Status/Achievements: While a draft structure has been elaborated, its implementation is dependent upon the civil service reform under development. At the moment the structure offers no-flexibility to adjust to the evolution of the sector. Sustainability of the project's achievements has been reinforced by the recent integration of six essential task members of the project and 25 database operators into the MOH.

Development impact: A structure well adapted to its role is essential if MOH is to take the leadership in

- 22 -

the efficient management of the sector. The freeze on recruitment and the over-centralized decision process make rapid progress unsightly. Solutions for ensuring the sustainability of the project are therefore sought on a case-by-case basis.

Recommendations: the present structural capacity is very limited, specific expertise and experience is not always available. A facilitating element would be the reduction and streamlining of current administrative bureaucratic duties and the reduction of the weight of centralized decisions (e.g. authorization system). It is very difficult to think how the MOH in the present condition can possible deal with three very different responsibilities as a provider, purchaser and regulator. Even with extensive sub-contracting and parallel agencies, the MOH shall develop additional in-house skills and services.

Personnel Management

Status/ Achievements: A computer base human resource information (HRI) system has been designed and files of approximately 2800 staff registered, it is planned to refine the system developed to include more staff characteristics (such as skills and qualifications), which will increase its relevance and facilitate the staff management (i.e. define training needs of staff). It is also planned to link the system to all departments concerned (i.e. payroll).

Development Impact: A good knowledge of its main resource-staff- is necessary if MOH is to increase its efficiency and upgrade its institutional capacity. This objective has been in great part achieved, and knowledge transferred to the Department of Personnel.

Recommendations: The system has been greatly affected by the recent law of autonomy of hospital, which will lead to the removal staff from the database and their transfer to individual hospitals. For the remaining database, it is planned to link the personnel system to the payroll, and to add complementary information of staff (such as training).As the system is fully implemented and the Personnel Department fully involved, it is important to ensure the sustainability of the system beyond the project life, and in particular secure permanent position for the key staff (operators)

Accounting and budgeting procedures

Status/Achievements: Several development occurred under this component; (i) the transfer of budgets to the autonomous hospitals, (ii) no solution has yet being found to integrate this activity within the structure of the ministry and in particular the Accounting Department, and (iii) it is planned that the reform of the accounting and budgetary procedures be introduced in collaboration with the Ministry of finance which has developed a new automated accounting package for the different Ministries and will be implemented at the MOH as a pilot experience.

Development Impact: An efficient financial management is essential to MOH if it is to improve the use of its resources. A first identifiable is the setting of a system capable of verifying on a regular basis that expenditures meet the financial provisions and strategies of the general budget. The achievement of this objective within the project was not completed.

Recommendations: In case the MOH is asserting its role as a market regulator and incentive –competition manager, it financial functions will have to be extended to assess the effects on health expenditures of regulatory actions (hospital autonomy, DRGs, hospital accreditation, etc..) . The

- 23 -

introduction of new methods for budget allocation across types of services and hospitals, responding to equity and effectiveness criteria need to be reflected in the expenditure performance. In addition, the introduction of new forms of payment mechanisms will affect the hospital’s pricing behavior. Besides strengthening it s auditing capacity, the MOH shall see in the financial management and accounting studies an effective tool in assessing the expenditure pattern, therefore, the institutionalization of the NHA unit in of paramount importance. The introduction of any new financial or operational changes shall be coupled with strengthening of the financial and accounting department, in addition to close collaboration with Information Systems and Health Services Departments at the MOH.

Procurement and maintenance functions:

Status/ achievements: An inventory of all equipment has been carried out. It was used for the rehabilitation and procurement for medical equipment for hospitals under the project. A pilot program to improve maintenance in hospitals has been developed with support of a higher education institution and is about to be tested in the Dahr El Bashek hospital. No further continuation of this activity has been undertaken in light of the implementation of the law of autonomy.

Development Impact: The justification of strengthening the procurement functions at central level is not as strong as it was at project inception, as the Law of Autonomy of hospitals has reduced the volume of activities of the department. Moreover, the department has already achieved most of the improvements within its control and further streamlining is handicapped by civil service regulations that are beyond its control. With regards to maintenance, the approach adopted to work at hospital level remains valid.

Recommendations: a computer based medical equipment registry and maintenance management system has been developed for the four hospitals under rehabilitation by the project, such system could form an initial step towards the full implementation of a computerized inventory and maintenance control and management system in all autonomous hospitals. A central unit should be formed at the MOH to follow up on the implementation of such system.

A.1 (b) Developing and Establishing a National Health Information System

Status/ achievements: This sub-component initially included two activities:

Computerizing and modernizing the internal operation of MOH Public Hospitals and Health lCenters: some work has been accomplished at the MOH central level by computerizing some MOH operations (visa, documents archives, personnel, central pharmacy). In addition, computerization of heath centers data collection is currently underway at 20 health centers to be extended to 100 or more by the end of 2004. For health centers, they have been supplied with standard forms and manuals for data reporting. Computer hardware has been delivered to additional HC bringing the total number of HC currently having computer equipment (PC, UPS and printer) to 29. Setting a national medical monitoring network in association with epidemiologists to collect, lanalyze and publish health data: Major progress has been accomplished in standardizing reporting form format and periodicity. This allowed systematic health data collection, which led to the publishing of EPINews newsletter. The next phase for this second component is to computerize the data collection from all health facilities by the implementation of a web based system.

Main Achievements:

- 24 -

An Information Technology Unit (ITU) at the MOH has been formally established through a ministerial decree.

Visa Billing: Initiate a multi-disciplinary control (medical, financial and administrative) to the lcurrent visa application that (i) will allow to monitor the quality of health services paid for by MOH, (ii) decentralize the process of visa issuing within the regions in accordance to current cost to empower the periphery of MOH structures and finally by introducing and contain the cost of the medical services provided by private health facilities by introducing new payments mechanisms (Flat rate payment, DRG, etc.).

Interconnection of Beneficiaries Database: the establishment of a web based information system lthat registers basic information on all beneficiaries of all public funds. The system interfaces with the visa issuance system in order to verify eligibility of hospitalization automatically. The system shall eliminate dual coverage and save the patients the burden of obtaining non enrollment certificates from public funds.

Hospitals information systems: Promote a demand-driven approach to computerize a number of lpilot hospitals that have been granted autonomy as a way to support their activities. This computerization will focus on the applications for which procedure manuals have been finalized and transmitted to autonomous hospitals. These applications will cover Patients Admission; Budgeting and Billing; Maintenance; Contracts; and Medical Records. To initiate this demand-driven approach, MoH will invite hospitals to submit a proposal to computerize its application support. This activity should be conducted in close coordination with the team working on elaborating and disseminating the procedure manuals.

Computer literacy at the central and peripheral levels : A the Central MOH level, continue capacity building through a comprehensive training program in Information Technology. At HC level, re-launch the stalled training program to support and maximize the impact of current computerization activities at HC level. This included training activities :§ For 7 health units (24 trainees)- year 2001§ For 14 health units (52 trainees)- year 2003.§ Training courses involve windows basic & intermediate, Word basic & intermediate, Excel basic

& intermediate and outlook express & internet explorer. § Training of the qada physicians and Data Entry Teams of the 11 districts on software. § Training of the district teams on the Data collection forms.§ Training of the Data Entry teams on data coding and entry.§ Some financial support for the district staff received through WHO. § For the 6 new districts: an external firm performed the assignment on the data collection/entry,

training of staff and performing some analytical results

Development Impact: A good information system would lay the basis for rational decision-making and better capital planning at modest cost. The activities being developed and implemented will contribute to the building of this information system. However, to achieve their full impact they will need to be included in an integrated system aimed at informing the decision-process.

Recommendations: it is proposed that elaboration of an information system master plan be integrated to the work preparing for a Health reform project, rather than making a new attempt. In the meantime and as a contribution to this process, the project will focus on priority actions, taking into account the

- 25 -

existing situation at central and facilities level in terms of financial, computer and human resources and legislation (hospitals autonomy law) and their potential impact on the quality of health services rendered to patients in private and public health facilities. The access to, and the centralisation of, information could rely on a National Health Information centre, which could host the entire information in the MOH. It will include:

A sophisticated Carte sanitaire, which could monitor the density of the supply. The system has ldeveloped a GIS to improve spatial representation of the results of the monitoring. It can be used as a planning and regulating tool. The various facilities follow a largely standard design, which facilitate the procurement of the related goods and services;A Visa insurance Claim management and billing system, which gives online entry of discharge linformation and medical bills;The inter-connection of beneficiary databases to consolidate the information related to all public lfunds beneficiaries in order to prevent double coverage by MOH and public funds;A hospital accreditation programme, which implement a quality improvement system on lestablished quality standards;A national disease management and control system, a reporting tool for epidemiological lmonitoring and control;A pharmaceutical management system to manage and control pharmaceutical expenditures, drug lregistration, import/export and prices;An information system at district level in the PHC centres.l

A.1 (C) Strengthening the Management of Health Centers

Status/ Achievements: A standard package of PHC services inside and outside the health centers, an essential list in PHC, standard equipment list, health information system and performance indicators to monitor the work were adopted, a set of forms, files and registers was designed; pilot-tested and adopted in contracted health centers. A number of workshops were held for the purpose of strengthening the capacities of the Qada and reinforce their role in the Qada.

Development Impact: The objective of ensuring that the NGO health centers perform as a good quality safety net for the less-well-off is valid.

Recommendations: Evaluate the experience so far with a view to expanding it; continue the training programs and include as much as possible the private, for-profit providers. With respect to the empowerment of the periphery, it was agreed to strengthen the Qada Physician’s role in 5 pilot qadas. It is required at this stage to develop a health centers service impact indicators and quality standards at the health centers operation and catchments' areas serviced in order to evaluate the performance of the health centers and their impact on the health status levels. Financing basic health care activities will remain one of the cornerstones of any health reform whenever it is recognized that providing such kind of care to the MOH beneficiaries is important. The allocation of a budget to ambulatory and PHC care shall be established by an appropriate mechanism of payment. The present approach of setting contracts on an interchange basis (supply of kinds against adoption of PHC services) does not seem to provide strong engagements among providers. Clearer payment system will probably do. At any rate policy dialogue started with the NGOs shall continue on the basis of the MOH prerogatives in terms of authorizing and accrediting NGOs’ services. Such enterprises of public health interests such as health centers can be set up with little approval for a safety or little directions in terms of policy congruence.

A.1 (D) Strengthening the Management of Public Hospitals

- 26 -

Status/ Achievements: Substantial progress has been made in the designing of managerial tools under this sub-component. The scope pf the work has been broadened to include additional, important facets of hospital management, including quality assurance. The main outputs produced are: a cost accounting study of one public hospital, standardized (manual) budget preparation for public hospitals, a plan for piloting new hospital maintenance procedures and training maintenance staff, and development of a set of procedures manuals for use as a managerial tool for public hospitals. The first volume of the latter has been published.

The GOL has sought to address all constraints related to the management of public hospitals through a Law of Autonomy, which has been progressively implemented.

Development impact: The objectives of this component remain relevant, and the original process and output indicators should be achievable by project completion. This component originally proposed also to establish a hospital management training capacity in the MOH. This has not been pursued, and does not appear to be justifiable given that two universities already offer hospital/health management programs.

Recommendations: The activities under this component should be pursued with the aim of assisting hospitals in their transition towards autonomy. Some problems and uncertainties with the Law and its application need to be addressed if it is to provide an effective underpinning for a well-managed, autonomous, public hospital system. In particular, efforts shall concentrate on:

Addressing the problems with the application of the Law of Autonomy. lCompleting the procedural manuals developed as guidelines to hospitals managers, incorporating lthe work done with the newly established accounting and budgeting procedures, personnel management and maintenance;Providing guidelines and support to hospitals in the development/purchase of computerized linformation systems;Implementing a communication and training plan which will be developed to prepare Boards, lmanagers, staff and other guarantors for implementation of the Law, to encourage the use of developed management guidelines and computerized information systems; andDefining clearly the terms of the contractual relationship between MOH and the autnomous lhospitalsThe establishment of a unit to include representatives of all autonomous hospitals; its duty shall lbe to follow up and coordinate the efforts of the autonomous bodies and eliminate redundancy of services, equipment and facilities and to ensure the sharing of resources.

A.1 (E) Empowerment of the Periphery

Status/ Achievements: The ministry reviewed the status and role of the peripheral health authorities and recommends, in addition to many activities, the necessity at this stage to establish an automated Health Information System at the district and Mohafazat levels.

In year 2001, and for many administrative constraints, most activities under the EOP component were phased out. The ongoing programs and activities are limited to the development of health information systems to automate information related to the administration, demographics, epidemiological, sanitation and inspection and health programs at the Qada level.

- 27 -

Major activities under this component include the following: Implementation of the District HIS in 11 districtslImplementation of Mohafazat HIS in 1 MohafazatlProcurement of Equipment, Furniture and Computer systemlTraining on concepts of District Health Management (DHM)lAssessment of Manpower status of all PHUslAssessment of the existing legislation (Laws, regulations, Decrees & Decision)lAssessment of physical status and identification of needs of all PHUs (equipment & Furniture)l

Development Impact: the empowerment of periphery initiative is meant to decentralize some form of MOH authority, besides assigning resources and skills to districts and regional offices of the MOH. It has potentially great impact on devolving and decentralizing the MOH functions not only to the local MOH offices but also to other public institutions (e.g. municipalities). At present the empowerment of the periphery can be understood as having the periphery supporting program implementation, which are centrally developed and controlled.

Recommendations: It is important to define whether the local MOH branches will be entitled to hold and manage funds and services. This would entail important institutional changes. In any case, the center must be strong enough to sustain the process of decentralization and provide direction. It must be able to identify the line of accountability, without interfering in the process.

A.2 Planning and Normative functions in the sector

Carte Sanitaire

Status/ Achievements: This pilot project was conducted with support from a French firm and has resulted in the development of a GIS based health resources assessment. However, it currently provides only an overview about current health situation from a health supply perspective. MOH is following up with the Council of State on enacting an acceptable Carte sanitaire law. Legislative amendments would be needed whereby a hospital’s “Construction Permit” and “Operation Licence” are granted based on the Carte sanitaire and the compliance with the basic standards set within the framework of hospital accreditation.

Development Impact: given the expanding capital investment in the sector, this planning and normative function has become imperative. The liberal framework of medical activities and the profit-orientated private sector lead to high health expenses. Therefore, there is a need of effective control. With this view, the MOH introduced a Carte sanitaire in 02/2000. This is a comprehensive inventory of health infrastructures and medical equipment with the possibility to analyse the needs function of population and existing facilities. This Carte sanitaire is a powerful technical tool to regulate the supply through licensing based on needs assessment.

Recommendations: The future interaction between the CS project and the HIS is very important to build a NHIS. In fact, the CS. initiative taken by MOH from an Information System perspective will provide a ground work for a NHIS since it establishes an objective Information System framework that can be used to provide real and objective view of the current Health Sector situation in Lebanon for both Public and Private Sectors from physical facilities, medical technology and health human resources. The CS should be extended to include the health needs component based on medical information (epidemiological, disease demographic and geographic distribution, etc.) and non-medical information (factors related to water, sewers, social, education, legislation, etc.). The CS should also be

- 28 -

extended to take account of private physician practitioners, facilities under construction and all excluded public facilities. Once these steps are completed, the CS can be used to correlate between the health services supply, the health services needs based on a set of laws and regulation. Since the level of knowledge in GIS at MOH is rather limited, external Technical Assistance to MOH in further developing the CS is needed. This activity should be conducted with the Project and Planning Unit of the MOH to ensure its sustainability.

A.3. Health Care Financing

A.3 (a) Cost containment

Status: Some important work has been completed under this component to develop and implement a system of flat-rate payments for in-patient and same-day surgical procedures. The impact of this on expenditures needs to be carefully evaluated.

Development Impact: The MOH is developing mechanisms for controlling costs in a fee-for-service reimbursement system. It is improving administrative and medical controls and introducing prospective case-based payments for hospital services. These measures would improve value-for-money, but it will take some years to implement them fully. In addition, to achieve maximum benefit from the work of this component, other public guarantors and ideally private insurers (once the system is evaluated) should adopt that system of payments once its cost containment potential has been verified.

Recommendations: high priority should be given to testing, evaluating, and refining the flat-rate payment system. Case mix analysis is of great importance to guide service purchasing, and assess the investments. Utilization Review has to assess whether access to care has been uniform, whether variation in case mix and cost is justified and whether volume increase reflect expected utilization pattern or provider’s cost shifting practices. Case-mix and financial date shall be eventually considered for an assessment of where the money goes. Certain features of the system should be carefully reconsidered:

- The timeframes for review and adjustment of the flat rates should be specified; - Administrative systems for monitoring and auditing should be carefully assessed;- Implementation of new payment mechanisms shall be combined with global budget limits on hospital spending to control growth on a long-term basis.

- Overall monitoring and evaluation of the budgetary utilization and quality impacts needs to be introduced.

- The maintenance, policy planning and continuous evaluation of this new payment system should be institutionalized;

- With regards to the same-day surgeries, in the longer term, the Government should consider, other ambulatory patient classification systems;

In addition coordinating payment approach, at least among public insurance, would avoid also patients' selection, as charges would be equal, whether the patient belongs to one or another public payer. It is remarkable that private hospital providers can negotiate prices and costs with reasonable degree of coherence, while major public fund-holders (MOH and NSSF) do not exert those “privileges” to the same extent.

A.3 (b) Health Financing Reform Study

- 29 -

Status: There are four activities contracted to WHO by GOL: household expenditure and utilization survey, National Health Accounts [NHA], development of health financing options and the burden of disease study.

Development Impact: Some half of Lebanon's population lacks formal health insurance coverage. The risk pooling that takes place through the multiple public 'insurance' programs is inefficient and inequitable. Private insurance, which covers between 15-25 percent of the population, is virtually unregulated and results in risk segmentation as opposed to risk pooling. Using the MOH as the insurer for the poor and the safety net for the rest of the population through in-kind provision of care results in two-tiered quality and access problems for vulnerable -population groups. With health spending at 12 percent of GDP and some $800 per capita (in international PPP-adjusted dollars), if the health system was functioning efficiently, formal universal coverage should be affordable and would improve access, equity, quality, and the Government’s ability to control costs.

Recommendations: The information being developed under this sub-component are critical inputs for the Health Reform and the work needs to be carefully managed by the inter-ministerial committee, particularly the work on defining options for health insurance coverage. In order to improve the system efficiency there is a need to unify the public health sector coverage and fund administration. This would be done through contracting out functions of the MOH and other public funds to a Third Party Administrator. This will facilitate building on the MOH activities in establishing linkages and sharing information with other public insurers.

B. IMPROVING HEALTH CARE DELIVERY

B.1 Health Centers Improvements

Status: The project provided the necessary civil works, equipment and furniture, vehicles, training and medical supplies to bring the 10 public health centers. In addition, the project provided the additional equipment, training and operational support needed to implement MOH/NGO contractual agreements in 30 health centers run by leading NGOs.

Development Impact: Improving the condition of buildings and equipment is a prerequisite to improving quality of service delivery. However, the sustainability and opportunity for MOH to generalize this pilot, as well as the impact of the measures put in place to encourage the delivery of the essential basic services by health centers should be evaluated.

Recommendation: the role of the MOH in such an agreement should focus more on the training and capacity building component rather than on facilities and process. By conduction of structured and well designed workshops and involving local and international experts in the field, an outcome of improved efficiency and productivity will be achieved. More over, the development of impact indicators and quality standards at the health centers and community levels is essential in ensuring that quality service is provided and the right disease prevention efforts are productive.

B.2 Rehabilitation of Front Line Referral Hospitals

Status: it was decided that only four hospitals would be rehabilitated (Tripoli, Dahr El Bashek, Kabreschmoum and Baalbeck).

- 30 -

Development Impact: The health system in Lebanon predominantly relies on MOH contracts with private hospitals to ensure the provision of low-cost essential services to the poor and uninsured. The private hospital system has excess capacity, and appears willing and able to deliver services to MOH patients in all of the five areas where the project was supporting public hospital rehabilitation.

Nonetheless, the political reality is that the Government is committed to successful operation of a network of public hospitals, and has obtained donor finance for a major public hospital building program, notwithstanding the fact that this increase in hospital capacity will exacerbate the serious cost containment problems in the Lebanese health system. In an effort to limit the expansion of hospital beds, the MOH has reassessed the opportunity to rehabilitate the six front-line referral hospitals planned under the project. It was decided that works would be limited to four hospitals (Tripoli, Dahr EI-Bashek, Baalbeck and Kabrechmoun) with the understanding that the rehabilitation will not increase the bed capacity and will serve other valid objectives: creating an impetus for MOH to terminate contracts with sub-standard private hospitals, enabling the Government to cancel plans for higher-cost new hospital construction (Tripoli), and piloting and evaluation of innovation in public hospital management, and leverage for limiting the scale and functionality of these hospitals to an appropriate level.

Recommendations: . The rehabilitation of public hospitals that are working under the law of autonomy is of high priority since it improves the accessibility of the medical services to the underserved and needy areas where the private sector has no incentive to provide services. The rehabilitated hospitals can also compete with the private sector to ensure provision of quality services at affordable cost.

B.3 Emergency Medical Care

Status/ Achievements:

A TV campaign on raising public awareness issues regarding transportation of emergency cases is lshowing since May 2000.

Emergency medical services training for nurses and physicians in the South and Mount Lebanon lwere completed in August 2001.

Development Impact: this component aims at the development of short-term and high impact activities to improve the emergency medical services in Lebanon.

Recommendations: it's recommended to upgrade the EMS services in all hospitals' emergency rooms and to establish EMS centers in all mohafazat. EMS centers will provide on accident site basic services and transport the patient to the specialized hospital regarding his case. The centers shall also provide ongoing training and support to other EMS facilities.

B.4 Improving Quality of Services

Status/ Achievements: The main requirement of the project has been to develop, test, adapt and finalize a Hospital Accreditation Manual, suitable to local circumstances but taking in account international tenets of accreditation as well. The development and testing process involved the consultant working with a group of pilot hospitals, specially chosen to be representative of the acute hospital system in Lebanon. The Manual took a two-tiered approach, delineating Basic Standards and Accreditation Standards. It is proposed that many of the Basic Standards are those that were required for hospital

- 31 -

licensing and will be familiar to hospitals. The Accreditation Standards are higher order Standards and are based on Quality Assurance and Quality Improvement. The consultant has also produced Guidelines for the Standards in the Accreditation Manual. The Guidelines are intended to further explain the Standards and provide hospitals with additional information and examples, for the purpose of achieving the Standards.The Implementation of the new standards has started with the First National Hospital Survey that has been conducted from September 2001 till July 2002. It included the survey of up to 134 hospitals. All hospitals were evaluated against the basic and accreditation Standards and individual Hospital reports were delivered at the end of this phase. Reports included a score, as percentage of the Basic and Accreditation Standards met. The MOH has defined the passing Mark as a combined score of 80% and above for the Basic and 60% for The Accreditation. Contracts were awarded accordingly to hospitals while taking into consideration the geographical distribution of the hospitals to avoid any regional imbalances.

The second round of hospital audits has started in October 2002 and ended in June 2003 .It included the upper-half of hospitals who did not meet the 80-60 requirement as well as the new and pilot hospitals that were not included in the first survey. It also gave the MOH the opportunity to further review and validate the standards.

The MOH , after two years of launching the Accreditation program is revising and refining this tool to bette refelec the policy objectives of improving quality of services and establishing a link between accreditation and the new payment mechanisms and setting up purchsing requirements. Additional steps has been launched during the last year of the project including:

Introducing further refinements to the accreditation manual and introduce changes to problematic lareas where there has been a general consensus on that issue.Modifying the structure of the manual lCovering other areas where standards have not been covered (especially for specialty hospitals and lspecial care units e.g Renal dialysis, , cardiac cat labs. and psychiatry Proposing a new scoring SystemlEstablishing criteria for hospitals being awarded full accreditation (three years) or partial laccreditation (one year). Establishing a Framework for a payment system which will have a direct link to accreditation loutcomes achieved by individual hospitals

Development Impact: the project aims at the development and implementation of a hospital accreditation and quality improvement system for contracting with public and private hospitals based on established quality standards. Completion of this exercise will give the Government of Lebanon a sound basis for ongoing refinement of quality processes, role delineation, resource allocation, building infrastructure development, staff training and health planning. Hospitals, which in the past have provided sub-optimal care to patients, will no longer be able to do so without being exposed and judged by their peers. This level of transparency while threatening to some is welcomed by most who want to see and support the continued success of the accreditation program.

Recommendations: The ongoing implementation of the accreditation process has reached a crossroad now that the World Bank funding has ceased. It is prudent and requested by the key stakeholders to devise a way fir the continuation of this program. The Government will fund the next phase of the accreditation program. It is suggested that an Accreditation management body that would act independently from the MOH shall manage the program, in line with International norms. This body shall have direct reporting lines to the Accreditation committee at the MOH, which would provide for

- 32 -

the strategic direction of the accreditation program. It is envisaged that for the future rounds of hospital international consultants would conduct surveys. International companies would be considered to provide auditors on a rotational basis. A 3-5 years strategic plan for accreditation shall be developed and disseminated to key stakeholders in order to maintain support for the ongoing accreditation program and the implementation of a linked payment system.

C . SUPPORT TO THE DEVELOPMENT OF A BROAD HEALTH SECTOR REFORM STRATEGY

Status/ Achievements: The TORs for the following five basic interrelated components of health systems reform were lprepared: (i) Pharmaceuticals; (ii) Manpower; (iii) Hospital and Inpatient Services; (iv) Public Health and Non-Personal Medical Services; and (v) Ambulatory and Primary Healthcare Services. The RFP was launched in March 2001 to a short list of international and national consultants. Only one short listed firm submitted a proposal. Due to the absence of competition and a change in the scope of the assignment, it was decided to cancel the procurement process. Under the medical human resources management component, a computer-based application has lbeen developed and implemented (WHO funded). It will be integrated with the other related systems within the ministry. Under the management information system (MIS) component, a comprehensive computerization lplan (information technology architecture implementation plan) has been prepared and presented to the Ministry. It is proposed to implement the plan in a phased manner over a period of three years.

A health reform policy note was prepared by the MOH, it was presented to the World Bank for lpossible future funding in January 2003.

Recommendations: Develop, analyze, cost, and assess implementation of reform options based on the institutional and fiscal realities of the country and relevant national and international experience.

3.3 Net Present Value/Economic rate of return:

The staff Appraisal Report (SAR) did not include any economic analysis of the project. There are no data available to calculate a posteriori the economic rates of return or the cost effectiveness of the investments and activities that were actually financed under the project.

3.4 Financial rate of return:

No financial rate of return was calculated for this project.

3.5 Institutional development impact:

Based on the discussion above on the achievements of objectives and the output by component (particularly the institutional development component), the institutional development impact of the project can be assessed as satisfactory.

- 33 -

4. Major Factors Affecting Implementation and Outcome:

4.1 Factors outside the control of government or implementing agency:

The loan took a year to become effective. The project has sought to address key sector issues during its first years of implementation.

4.2 Factors generally subject to government control:

The project outcome was favorably affected by the fact that lessons learned from the problems that affected the performance of the first five years of project implementation were taken into account through the amendment of the loan agreement in October 2000.

In the last three years, the ministry officials were more involved with the project, this has improved the project performance, the cooperation between Government and the Bank has been excellent. The project coordination unit (PCU) was streamlined and performed very well, at a lower cost.

One important feature of this project is the contribution made by international and bilateral agencies, not as co-financiers, but as “contractors” or providers of technical assistance such as WHO.

4.3 Factors generally subject to implementing agency control:

The project was too ambitious and complex to be implemented during the defined timetable and the lack of technical expertise at the MOH.

A more realistic approach would have suggested a much longer implementation time, which was ultimately translated into the project being extended for three times. It is only on the last few years that annual work programs were properly prepared and used as a basic for preparing procurement and disbursement plans; in the end, the system worked well.

4.4 Costs and financing:

At the time of appraisal, the cost of the project was estimated at US$ 48.121 millions IBRD planned to provide financing of US$ 35.7 millions, and the Government contribution was estimated at US$ 12.42 millions. The actual cost or latest estimate of the cost of the project is US$ 38.256 million, which was financed by US$ 33.341 Million from the IBRD credit and only US$ 4.915 Million from the Government. Additional details regarding project costs and financing by procurement arrangement are detailed in annex II.

5. Sustainability:

5.1 Rationale for sustainability rating:

Sustainability is a complex question, particularly in the case of a project that was successful in dealing with many sector issues, and for which there is already a follow on project that continues and expands most of the project activities. Generally, the existence of government commitment and a favorable policy environment as

- 34 -

well as local participation are factors that should contribute to the sustainability of the project.

Regarding the individual components, the epidemiological surveillance system seems to be well established. The question of sustainability for the support to district health services does not arise, while achievements have been satisfactory, the decentralization of the MOH health policy decision making at the district level needs to be enhanced and expanded. Despite a number of uncertainties, the modest results of the institutional development component are by and large likely to be maintained.

The sustainability issue has been lately reinforced by the integration of six key HSRP task members in the MOH newly established units namely; the projects and programs unit and the information technology unit. Additionally, the MOH has integrated 25 HSRP database operators into its structure. The operators will ensure the continuity in the existing information systems operation and productivity.

On balance, and after giving proper weight to the importance of the implemented programs and to their impact, the Sustainability of the project can be rated as "most likely".

5.2 Transition arrangement of regular operations:

As mentioned above, activities carried out under the project are being continued by MOH, as part of its normal responsibilities, with government financing and with the support of donors including a request for a follow-on project financed by the World Bank. The same performance indicators used for this project could easily be monitored, since one of the achievements of the project is the establishment and operation of a satisfactory and well-performing management information system, including health statistics.

6. Bank and Borrower Performance:

Bank6.1 Lending:

As discussed above in the section on quality at entry, the performance of the bank was unsatisfactory. Preparation and appraisal underestimated the risks associated with such a complex operation and paid insufficient attention to project implementation, organization and management issues.

6.2 Supervision:

At the beginning of the project, there may not have been enough support from the Bank supervision team to help the Borrower in carrying out the process or sequence of activities: planning of actions or measures to be taken – procurement plan – disbursement – accounting and financial management, in addition and in light of the tensions and problems that had arisen in implementation arrangements between the CDR and the PCU. During the first few years of the project the World Bank's mission's skills composition were inadequate; the constant rotation of the consultants that joined the missions affected negatively the continuity of the work.

The mid-term review (MTR) for this project, carried out on July/ August 1999, was probably one of the most comprehensive and thorough of all the reviews in which the bank ever participated. The MTR was carried out jointly with MOH. Bank staff, in addition to the 3 international consultants that took part in the mission, were instrumental and deserve credit for ensuring that all the problems were thoroughly analyzed and the remedial measures (including restructuring) were adopted. The Consultants contributed to the development of plans of

- 35 -

actions and reports that were instrumental in the fields of Emergency Care, Quality of Care as well as Payment mechanisms. These reports played a key role in the further development of the project components in a very constructive manner.

In any event, the Bank intervention in the last years of the project has been highly effective, especially through its active local office that has provided support to the project in the technical, financial and procurement areas. The quality of the Bank supervision as well as the support for implementation improved considerably. The Bank's support in amending the loan agreement during the last quarter of the year 2000 has resulted in the transfer of the management of the loan categories 4 and 5 to the MOH; this new management arrangement has facilitated the progress of the project in terms of activities and disbursement.

Borrower

6.3 Preparation:

According to the SAR, the HSRP was prepared by the MOH with the support of consultants. It is likely that at the time the MOH was influenced by and followed the lead provided by the Bank in project preparation.

6.4 Government implementation Performance:

The performance of the Borrower has varied during the eight-year implementation period of the project.

From 1995 until 1998. Initially, the Health Sector Rehabilitation Project (HSRP) faced problems in its performance because of unclear implementation arrangements, bureaucracy of the MOH and CDR as well as the lack of adequate involvement of the World Bank supervisory missions.

Project management was later jeopardized by problems and unacceptable delays in launching the rehabilitation programs for public hospitals due to series of tensions between the CDR and the PCU. The procurement activities were seriously delayed through bureaucratic system of the CDR and the unclear responsibilities and roles of the concerned parties.

From 1999 to 2000. The project was restructured to help the MOH implement its new strategies.The progress of the project during that period was further hindered by the same reasons stated previously as well as the minimal involvement of the MOH main staff in its execution. In the year 2000 the approval on the contracts of the full time consultants by the cabinet was delayed for nine months, this has hindered the project performance to a large extent. A decree was issued by the cabinet in January 2001 which has cancelled the requirement for the cabinet's approval on the contracts.During the second half of the year 2000, the civil works contracts for the rehabilitation of the hospitals were signed; this has given the project a chance to be extended and completed.

From 2001 to 2003. A new Minister of Health was appointed and fully supported the implementation of major project components with a far-reaching national impact. The re-involvement of the MOH officials as well as the active role of the World Bank's country office has given more support to the implementation of the project. This was translated in the cabinet adopting several ministerial decrees that allowed the adoption and implementation of several projects and measures in the areas of: hospital accreditation, establishing a public funds Beneficiaries' database, the review of drug pricing and import, the engagement of a private TPA to manage the contractual relationship between public funds and the private and autonomous hospitals. In addition during this period the procurement of goods and services for the rehabilitation of the four public hospitals was expedited due to the improvement in the working relationship and close coordination between

- 36 -

the MOH and the CDR.

The loan agreement has been amended during the last quarter of the year 2000, this has given the MOH through the PIU more control over the project especially in the management of categories 4 and 5 of the loan that has resulted in a distinguishable improvement in the performance of the project activities and the disbursement of the loan.

The various problems that have affected the project should not overshadow the many achievements of MOH, particularly in recent years, thanks to the dynamism and competence of the highest level of the concerned officials and members of the MOH, PCU, WB and CDR. Here are just a few examples of the very positive initiates that have been taken in a great variety of areas:

The introduction of ‘flat rate’ payments to contracts with private hospitals. lA Visa Insurance Claim Management and Billing System are ongoing. It is an on-line entry of ldischarge information and medical bills.The inter-connection of beneficiary databases to consolidate the information related to all public lfunds beneficiaries. It is to prevent double coverage by MOH and other public funds. It has been accepted by the different stakeholders and lays the foundation for the engagement of a TPA to manage contracts for on behalf of the deferent funds. In addition the MOH addressed the issue of Quality Assurance and Improvement and has linked it lto the award of contracts to hospitals. A hospital accreditation survey has been conducted. The reduction of the number of private hospital-beds contracted by the ministry through this quality control process is underway. (1800 beds were under contract with MOH in 1997, 140 hospitals for 2026 beds were under contract in 2001, 76 hospitals for 1543 beds in 2002.

6.5 Overall Borrower performance:

Overall, the Borrower performance can be rated as satisfactory.

7. Lessons learned:

The lessons learned through the successful implementation of this project in Lebanon were partly applied during the last few years of project implementation and could be used in the preparation of a follow-on health sector reform project. They are as follows:

The government should, at an early stage, develop a comprehensive policy framework and sector lstrategy that forms the basis for the intervention of all donors in the sector. For any new projects, the Bank shall conduct a very careful assessment before supporting any future linfrastructure investments in the health sector.New funds shall target opportunities for profound restructuring and change in policy decisions .New lprojects shall be awarded in light of specific policy adjustments. Future commitments shall reflect more strongly health reform objectives at central and peripheral levels and the changing role of the MOH.Officials of the ministry or relevant government agency should always be completely involved and in lcharge of all stages of the project cycle. They must themselves identify the type of professional input needed to achieve clearer departmental goals. Considerations on the competence and motivations of the civil servants and their interest in new lresponsibilities linked to any new projects shall be carefully assessed.To improve the sense of common purpose between the MOH and any new project, clearer reference l

- 37 -

shall be made as to the capacity of each department Project design should include suitable and clear arrangements for implementation by the Borrower, lincluding a project coordination unit with a clear mandate and provided there shall be a single implementing agency for the project. Project design should include realistic performance indicators, both for project output and project loutcome/impact that the Borrower /implementing agency can collect and monitor, with detailed information on the relevant methodology and the means of measurement. The implementation arrangement of any new project shall be well designed in order to streamline the lwork; the beneficiary shall hold more implementation authority and responsibility in terms of procurement and disbursement as well as contracts' management.

8. Partner comments:

Borrower/implementing agency: (CDR)

The project faced a number of obstacles summarized by:

Change of scope during executionlUnclear procurement procedures due to lack of experience and expertise during the initial phase of lthe project.Unforeseen factors and problems emerged during the execution phase because the project design was lbased originally on rehabilitation of existing facilities that pre-dated the civil war.

Lessons learned

More efforts and time should have been spent on defining clearly the Scope of the project, the lprocurement procedures, technical specifications, and the implementation structure.

Contingency planning should have been put in place to anticipate the problem that arose during lexecution, particularly since the project was designed as a rehabilitation project.

The role of each actor included in the implementation of the project, namely MOH, the PCU within lthe ministry and CDR should have been defined precisely. In addition the procedures adopted and followed by the different actors during execution should have also been clearly identified.

(b) Cofinanciers:

(c) Other partners (NGOs/private sector):

10. Additional Information

- 38 -

Annex 1. Key Performance Indicators/Log Frame Matrix

Outcome / Impact Indicators:

Indicator/Matrix

Projected in last PSR1

Actual/Latest Estimate

1. Percent of hospitals accredited 55 % 55 % 2. Number of districts using improved health information system

5 11

3. Number of health centers using improvement management systems 40 454. Number of primary health care centers adopting new health information systems 25 29

Output Indicators:

Indicator/Matrix

Projected in last PSR1

Actual/Latest Estimate

1. Percentage progress in the upgrading of the geographic health information system (carte sanitaire)

80 % 100 %

2. Percentage progress in the development of the "visa-billing" system and public sector insurance database integration

70 % 100 %

3. Number of hospitals surveyed under the hosptial accreditation program

122 124

4. Number of hospitals resurveyed 33 45

5. Percent progress in hospital civil works 90% 90%

6. Percent progress in hospital equipment procurment

90% 99%

1 End of project

- 39 -

Annex 2. Project Costs and Financing

Project Cost by Component (in US$ million equivalent)AppraisalEstimate

Actual/Latest Estimate

Percentage of Appraisal

Component US$ million US$ million1.Strengthening of Planning Capacity of MOH 2.00 1.74 0.862.Health Care Financing 2.75 2.12 0.773.Management and Rehabilitation of Health Centers 2.00 1.26 0.634.Management and Rehabilitation of Referral Hospitals 25.45 25.83 1.015.Initiating a Quality Improvement Program 2.20 2.00 0.96.Strengthening the Project Coordination Unit 3.60 2.98 0.827.Incremental Operating Cost 2.60 2.31 0.88

Total Baseline Cost 40.60 38.24 Physical Contingencies 7.52

Total Project Costs 48.12 38.24Total Financing Required 48.12 38.24

Note: latest disbursement figures as of May 31st, 2004

Project Costs by Procurement Arrangements (Actual/Latest Estimate) (US$ million equivalent)

Expenditure Category ICBProcurement

NCB Method

1

Other2 N.B.F. Total Cost

1. Works 0.00 14.40 0.00 0.00 14.40(0.00) (11.52) (0.00) (0.00) (11.52)

2. Goods 10.69 1.75 0.00 0.00 12.44(10.12) (1.28) (0.00) (0.00) (11.40)

3. Services 0.00 0.00 9.21 0.00 9.21(0.00) (0.00) (8.71) (0.00) (8.71)

4. Miscellaneous 0.00 0.00 2.32 0.00 2.32(0.00) (0.00) (1.80) (0.00) (1.80)

5. Miscellaneous 0.00(0.00)

0.00(0.00)

9.74(2.26)

0.00(0.00)

9.74(2.26)

6. Miscellaneous 0.00(0.00)

0.00(0.00)

0.00(0.00)

0.00(0.00)

0.00(0.00)

Total 10.69 16.15 21.27 0.00 48.11(10.12) (12.80) (12.77) (0.00) (35.69)

Note: latest disbursement figures as of May 31st, 20041/ Figures in parenthesis are the amounts to be financed by the Bank Loan. All costs include contingencies.2/ Includes civil works and goods to be procured through national shopping, consulting services, services of contracted staff

of the project management office, training, technical assistance services, and incremental operating costs related to (i) managing the project, and (ii) re-lending project funds to local government units.

- 40 -

Project Financing by Component (in US$ million equivalent)

Component Appraisal Estimate Actual/Latest EstimatePercentage of Appraisal

Bank Govt. CoF. Bank Govt. CoF. Bank Govt. CoF.1.Strengthening of Planning Capacity of the MOH

1.50 0.50 1.45 0.29 96.7 58.0

2.Health Care Financing 2.50 0.25 2.12 0.01 84.8 4.03.Management and Rehabilitation of Health Centers

1.50 0.50 1.02 0.25 68.0 50.0

4.Management and Rehabilitation of Referral Hospitals

19.45 6.00 22.28 3.56 114.6 59.3

5.Initiating a Quality Improvement Program

2.20 0.00 1.99 0.00 90.5 0.0

6.Strengthening the Project Coordination Unit

3.00 0.60 2.78 0.20 92.7 33.3

7.Incremental Operating Cost

2.00 0.60 1.80 0.52 90.0 86.7

8.Unallocated 3.55 3.97 2.26 7.61 63.7 191.7

Note: latest disbursement figures as of May 31st, 2004

- 41 -

Annex 3. Economic Costs and Benefits

- 42 -

Annex 4. Bank Inputs

(a) Missions:Stage of Project Cycle Performance Rating No. of Persons and Specialty

(e.g. 2 Economists, 1 FMS, etc.)Month/Year Count Specialty

ImplementationProgress

DevelopmentObjective

Identification/PreparationOctober 1993 4 health specialist, hospital

services specialist, health care financing specialist, public administration specialist

Nov-Dec 1993 6 health specialist, hospital services specialist, health care financing specialist, public administration specialist, building expert, operations assistant

Appraisal/NegotiationMarch 1994 7 health specialist, hospital

services specialist, health care financing specialist, public administration specialist,architect/implementation specialist, public health specialist, staff assistant

July 1994 4 health specialist, economist, architect/implementation specialist, public health specialist

June 1995 3 health specialist, economist, public health specialist, human resource specialist

SupervisionFebruary 1996 4 health specialist, public

health specialist, human resource specialist, building expert/architect

S S

July 1996 3 public health specialist, human resource specialist, public health specialist (second)

S S

March 1997 3 health specialist, operations analyst, architect

U S

September 1997 3 health specialist, public health specialist, architect

S S

May 1998 5 health specialist, public health specialist, health planning specialist, senior portfolio officer and information system specialist

U U

January 1999 7 health sector manager, health specialist, health financing specialist, operations specialist,

U U

- 43 -

implementations specialist, information system specialist, operations analyst

July 1999 8 health sector manager, health specialist, implementation specialist, hospital payment specialist, quality assurance specialist, information systems specialist, emergency services specialist, monitoring and evaluation specialist

U U

August 2000 3 health specialist, financial management specialist, procurement specialist

S U

March 2001 4 health sector managert, task manager, financial management specialist

S S

October 2001 4 task manager, procurement specialist, finacial management specialist

S S

June 2002 6 health sector managert, task manager, procurement specialist, financial management specialist, program assistant, program assistant (second

S S

February 2003 6 task manager, operations specialist, financial management specialist, procurement specialist, program assistant, program assistant (second)

S S

June 2003 8 health sector manager, task manager, procurement specialist, financial management specialist, information officer, operations officer, program assistant (2)

S S

ICRDecember 2003 7 health sector manager, task

manager, health specialist, operations specialist, financial management specialist, procurement specialist, program assistant

S S

- 44 -

(b) Staff:

Stage of Project Cycle Actual/Latest EstimateNo. Staff weeks US$ ('000)

Identification/PreparationAppraisal/NegotiationSupervisionICRTotal

Note: accurate and up-to-date information currently unavailable due to length of project duration.

- 45 -

Annex 5. Ratings for Achievement of Objectives/Outputs of Components(H=High, SU=Substantial, M=Modest, N=Negligible, NA=Not Applicable)

RatingMacro policies H SU M N NASector Policies H SU M N NAPhysical H SU M N NAFinancial H SU M N NAInstitutional Development H SU M N NAEnvironmental H SU M N NA

SocialPoverty Reduction H SU M N NAGender H SU M N NAOther (Please specify) H SU M N NA

Private sector development H SU M N NAPublic sector management H SU M N NAOther (Please specify) H SU M N NA

- 46 -

Annex 6. Ratings of Bank and Borrower Performance

(HS=Highly Satisfactory, S=Satisfactory, U=Unsatisfactory, HU=Highly Unsatisfactory)

6.1 Bank performance Rating

Lending HS S U HUSupervision HS S U HUOverall HS S U HU

6.2 Borrower performance Rating

Preparation HS S U HUGovernment implementation performance HS S U HUImplementation agency performance HS S U HUOverall HS S U HU

- 47 -

Annex 7. List of Supporting Documents

Staff Appraisal Report, Aide Memoires, Project Status Reports, Peer Review Reports1.World Bank-Government of Lebanon correspondence2.Project Status Reports prepared by the Project Coordination Unit3.Ammar, Walid (2003) Health System and Reform in Lebanon, World Health Organization, Geneva4.Study reports prepared by various consultants5.ICR prepared by the Bank for Emergency Reconstruction and Rehabilitation Project (ERRP)6.ICR prepared by the Government of Lebanon for Health Sector Rehabilitation Project7.W. Van Lerberghe et al. (1997, “Reform follows Failure: Pressure for Change in the Lebanese Health 8.Sector," Health Policy and Planning; 12(4): 312-319.W. Van Lerberghe et al. (1997) “Reform follows Failure: Unregulated Private Care in Lebanon,” 9.Health Policy and Planning; 12(4): 296-311.

- 48 -

- 49 -