strategic review of the public-private...

Michael Jordan & Associates

STRATEGIC REVIEW OF

THE PUBLIC-PRIVATE INFRASTRUCTURE ADVISORY FACILITY

November 2004

Prepared by Michael Jordan & Associates


STRATEGIC REVIEW OF THE PUBLIC-PRIVATE INFRASTRUCTURE ADVISORY FACILITY

Table of Contents

Page

EXECUTIVE SUMMARY 1

1. INTRODUCTION 3

2. RELEVANCE 5

3. IMPACT 12

4. EFFICIENCY & EFFECTIVENESS 17

5. LEARNING FROM EXPERIENCE 22

6. RECOMMENDATIONS 24

List of Appendices

A. Terms of Reference B. List of People Interviewed C. Sources of Applications for PPIAF Funding D. Trend in Approved Projects by Region E. Trend in Approved Projects by Activity F. Performance Ratings for Projects Studied G. Distribution of Projects with Recorded Outcomes in the Impact Database H. Trend in Program and Management Expenditures I. Organization Chart of the Program Management Unit J. Processing of Applications K. Donor Contributions to PPIAF L. Preliminary Schematic Approach for country Priorities


Abbreviations and Acronyms

ADB Asian Development Bank CB Capacity Building CFR Country Framework Report CS Consensus Building DFID Department for International Development DGF Development Grant Facility FY Fiscal year (ending 30 June) GBP Global Best Practice IWP Indicative Work Program PATS PPIAF Applications Tracking System PC Program Council PMU Program Management Unit PPI Private Participation in Infrastructure PSI Private Sector Development & Infrastructure Vice-Presidency (World Bank) PT Pioneering Transactions RC Regional Co-ordinator RF Policy, Regulatory and Institutional Reform SIDA Swedish International Development Cooperation Agency ST Infrastructure Development Strategies TAP Technical Advisory Panel TM Task Manager ToR Terms of Reference UNIDO United Nations Industrial Development Organization USAID United States Agency for International Development WBI World Bank Institute

Michael Jordan & Associates 1

EXECUTIVE SUMMARY 1. This study presents the findings and recommendations of an independent strategic review of the

first 5 years of PPIAF’s operations. It was commissioned to assist donors who will be considering renewing their financial commitments to the Program. It addresses four major issues: • Is the PPIAF Program still relevant to help improving infrastructure in developing countries? • Has PPIAF been successful in achieving significant results? • Has PPIAF been operated efficiently and effectively? • Has PPIAF identified and applied the lessons of its experience?

2. Relevance. Against the background of widespread scepticism about private participation in infrastructure, PPIAF still has a potentially important role in assisting governments to address the “unfinished business” of PPI. Its mandate covers all the critical issues from defining strategies to facilitating transactions, from identifying global best practice to consensus building. Its portfolio of 5 years’ advisory work comprises a valuable inventory of case material about alternative approaches to PPI.

3. The buoyant demand for PPIAF services also indicates the Facility remains relevant to developing country governments. The volume of new applications increased in FY 2004 after declining moderately in the previous two years. Applications have been received from 105 countries, but the breadth of coverage has not been matched by sustained demand in all countries. The highest proportions have come from regions where PPIAF has field representation.

4. Overall, PPIAF has demonstrated a strong comparative advantage as an advisory facility. It has a clear and coherent functional focus; a broad range of deliverables and access to WB multi-sectoral expertise; global reach; strong donor support; and a lean, cost-effective management structure.

5. Impact. After a slower than projected start, PPIAF has exceeded its initially expected level of activity by over 25%, reaching cumulative expenditures of US$ 41 million by the end of FY 2003 against a target of US$ 32 million. The outputs of services funded are of high quality across a wide range of interventions. Of the 32 mature projects reviewed in detail in this study, 27 (84%) of the outputs were assessed as being fully or mostly satisfactory. 15 projects or 48% were assessed as having achieved a fully or mostly satisfactory outcome, but in another 12 cases it was too early to make a clear judgment. Estimating the probable outcomes in those cases, 20 (63%) of the projects are assessed to have been fully or mainly successful. Similarly, the Impact Database showed that 45-50% of projects approved in FYs 2000 and 2001 had achieved a positive outcome. However, there are significant variations in the performance between countries, in 42 countries, no positive outcomes have been recorded. PPIAF should apply more stringent country criteria to its project screening processes.

6. Efficiency and Effectiveness. The Program’s distinctive operating model comprising a small Program Management Unit (PMU), and outsourced delivery of PPIAF-funded services has proven to be efficient and cost effective. Management costs, which have declined steady from 28% of total Program expenditures in FY 2000 to 13% in FY 2004.

7. However, there are issues regarding the effectiveness of PPIAF’s governance structure, organization and management processes. The Program Council has an essential role in the governance of the facility, but - as presently structured - is not well suited to provide effective direction and oversight for the PPIAF. Its composition is too large and meetings too short. The Technical Assistance Panel (TAP) has provided useful external insights on the major issues facing PPIAF, but has several limitations as an instrument for evaluating project performance, in particular by not having access to beneficiaries. The Program Management Unit’s role is focused on processing applications and contracting individual assignments: it has no explicit responsibilities for strategy formulation or program evaluation: these are important omissions. In


addition, limited staffing resources and lack of field presence have severely constrained the PMU’s capacity to acquire a stronger management role.

8. PPIAF’s processes for handling contracting work well, but planning and follow up are less effective. PPIAF’s Indicative Work Program is more a forecast than a target and does not serve to set direction or priorities for the Program. PPIAF’s procedure for screening applications is transparent and effective. Project implementation is generally well managed, but takes considerably longer than planned. Extended implementation periods detract from project quality because they create additional problems for effective follow up and closure of projects. The PPIAF Applications Tracking System (PATS) provides a comprehensive framework for recording information, but has several weaknesses as a management tool. PPIAFs monitoring does not include consolidation and analysis of individual projects to evaluate the performance of the overall portfolio.

9. Lessons Learned. PPIAF has a potentially important role in interpreting and communicating the lessons of its experience. However, the TAP highlighted several major areas of concern: in particular, the lack of information for measuring the impact of PPIAF activities, shortcomings in dissemination, the need for broader stakeholder consultation, and the importance of addressing the socio-political as well as economic dimensions of PPI. PPIAF has been generally responsive to the concerns raised by the TAP, mainly at the Program level, but the record of addressing the concerns about country specific activities is mixed. PMU needs to do more to ensure that recommendations regarding reporting, consultation and follow up are applied more consistently.

10. PPIAF has contributed to disseminating knowledge through the promotion of Global Best Practice (GBP). However, dissemination of PPIAF materials is still very uneven: there is no explicit plan for promoting GBP materials, or for comprehensive distance learning programs.

11. Conclusions and Recommendations. PPIAF’s first five years of activity has validated the relevance of its role and operating model. However, there is room for improvement to adapt the model to build on the experience of its pioneering years. In particular, action should be taken in four major areas. • Strengthening Strategic Direction of the Program. Most importantly, PPIAF should balance

the demand-driven approach project selection with a strategic vision of overall program priorities. PPIAF should move towards a more strategic approach to defining priorities based on an evaluation of country performance. In addition, PPIAF should establish a small Steering Group of the Program Council to provide more frequent and substantive strategic inputs. A senior member if the PMU staff should be assigned responsibility for Policy and Planning.

• Increasing Stakeholder Involvement in the Program. More needs to be done to ensure greater stakeholder participation through associating beneficiary governments with the program council, and greater stakeholder involvement in project execution.

• Focusing on Outcomes rather than Outputs. Project management activities need to give greater attention to project outcomes, in particular to implementation, consultation and follow up. PPIAF should introduce Operating Guidelines defining the procedures for carrying out advisory assignments, strengthen poverty reduction analysis and re-design the PATS reporting system so that it acts as a more effective monitoring tool.

• Enhancing PPIAF’s Identity. To ensure that the Program reaches its target beneficiaries and enjoys the full confidence of governments and the private sector, PPIAF needs to raise its profile as an autonomous multi-donor facility. It should expand its field presence. It should also review all communications materials and dissemination programs to ensure these reach key targets audiences and project a clear and strong image of the Program.

12. These recommendations will have implications for PPIAF’s organization, and procedures. Preliminary estimates suggest that the one-time increase in costs can be contained within 20% of total program expenditures: this ratio should improve as the volume of activities increase.


1. INTRODUCTION

1.1 The Public Private Infrastructure Advisory Facility (PPIAF) was established in 1999 as a technical assistance facility with a mandate to help developing countries improve the quality of their infrastructure through involvement of the private sector. During the first five years’ of operation, PPIAF committed US$ 77 million to finance over 300 country-specific and multi-country advisory and related projects. In 2005/06, several donors will be considering renewing their financial commitments to PPIAF. To assist them, DFID proposed that an independent strategic review of the first 5 years of the Program’s operations should be made to:

• Evaluate the success of the Facility against its original objectives; • Reconfirm the validity of the rationale for the Program; and • Identify way for improving the performance of the Program.1

1.2 This report presents the principal findings and recommendations of this strategic review.

1.3 The review was conducted between July and October 2004 by Michael Jordan, Lead Consultant, and David Cook, assisted by Alexia Santallusia.

SCOPE AND METHODOLOGY

Major Issues

1.4 The main body of the report responds to four major issues: • Is the PPIAF Program still relevant to help improving infrastructure in developing countries? • Has PPIAF been successful in achieving significant results? • Has PPIAF been operated efficiently and effectively? • Has PPIAF effectively identified and applied the lessons of its experience?

1.5 Based on the findings and conclusions on these issues, the final chapter of the review sets out recommendations for improvement.

Performance Indicators

1.6 The review was based on the generally accepted logical framework for program evaluation that includes four basic sets of performance indicators: inputs, outputs, outcomes and impact.2 These indicators, which follow a logical sequence of stages toward achieving the overall objectives, are illustrated in the following table.

1 The Terms of Reference for the review are attached at Appendix A. 2 See, for example, World Bank Development Grant Facility, Technical Note, Independent Evaluation: Principles, Guidelines and Good Practice, November 2003.


Major Parameters For Program Evaluation

Inputs

Pre-implementation

Outputs ImpactsOutcomes

Short-term timeframe 6 months-2 years

from approval

Medium-term1-3 years

from delivery

Long-term>5 years

from delivery

•Funding

•Staffing

•Applications received

•Policies & procedures

•Studies conducted

•Laws drafted

•Reforms proposed

•Toolkits prepared

•Policies improved

•Institutions created

•Transactions closed

•Officials trained

•Increased PPI investment

•Greater access

•Improved service provision

Major Sources of Information

1.7 The study has been based on a range of complementary qualitative and quantitative analyses drawing on the following major sources of information.

1.8 Program Documents. To understand PPIAF’s objectives, institutional structure and operations, the team reviewed the Initiating Brief, Program Charter, Program Council (PC) minutes, PPIAF Annual Reports, Annual Work Programs, Technical Advisory Panel (TAP) reports, PPIAF-funded project deliverables, and PPIAF promotional materials.

1.9 Interviews. The obtain a broad perspective of PPIAF’s objectives and operations we interviewed a wide range of donors, present and past PMU staff, Task Managers (TMs), beneficiary officials and other informed observers. A list of the people interviewed is attached as Appendix B.

1.10 PPIAF Databases. To analyse PPIAF operations we drew on detailed PMU records, including the electronic Applications Tracking System used to monitor progress on all proposals for assistance, and Impact Database, which records results associated with completed projects.

1.11 Field Survey. As donors were particularly interested to assess the benefits of the PPIAF program, the review included a detailed study of a sample of specific projects incorporating the views of beneficiaries on the results obtained. The field survey examined the objectives of the project, the process for contracting, execution and follow up, the quality of the services delivered and the results achieved or expected. Given time and budgetary constraints, the survey was limited to a detailed study of 32 projects, including 30 country-specific projects in seven countries in two regions (Ethiopia, Kenya and Zambia in Africa, and Cambodia, Philippines, Thailand and Vietnam in East Asia). The sample was chosen to focus on “mature” projects, i.e. where delivery of the services was completed or at an advanced stage, and to include a representative mix of major types of activity. The 32 projects are equivalent to 18% of the total of estimated 175 completed projects as of June 2004. Given the range of PPIAF’s activities and the variety of factors that influence their results, the findings of the survey should be interpreted with care.

ACKNOWLEDGEMENTS 1.12 We acknowledge with thanks the co-operation and constructive comments we received, in

particular from the Program Management Unit and field staff, World Bank Task Managers, and officials of beneficiary organizations.


2. RELEVANCE

2.1 This section of the report assesses the relevance of PPIAF’s activities over the first five years of operation (FYs 2000-2004) to fulfilling the mission set out in the Program Charter. It addresses three main questions:

• Is the rationale for establishing PPIAF still valid?

• Is there a sustainable demand for PPIAF services?

• Does PPIAF have a significant comparative advantage in supplying donor-funded advisory services?

VALIDITY OF PPIAF RATIONALE 2.2 Several factors have severely tested the PPI paradigm that gained a strong following during the

1990s. Macro-economic crises reduced demand for utility services, and exposed uncovered currency risks associated with the dependence on foreign financing of many infrastructure projects. Problems with individual projects, leading to the cancellation or renegotiation of contracts particularly in the power and water sectors have generated popular scepticism about the viability and benefits of PPI arrangements, and reduced the appetite of international operators and investors to participate in developing country infrastructure. In many cases, expectations of both investors and host country governments were unrealistic, underestimating the scope and difficulty of sustaining reform in the utility sectors to put them on a commercial basis, particularly in breaking past practices of pricing these services at below cost.

Continuing relevance of PPI

2.3 Nevertheless, experience with PPI to date suggests that well designed schemes can deliver better results than monopoly public sector provision, where arrangements are properly structured and adequately regulated. Under these conditions, PPI generally helps increase investment to expand access to service, lower prices and improve productivity and efficiency. The private sector’s technical and managerial competence combined with more sustainable pricing policies and better financial discipline relax the investment constraints that prevailed under public provision and expand opportunities for attracting financing for investment in expansion of services. Improvements in financial performance have allowed companies to invest to improve the quality of service to consumers. The impact of PPI on prices is less clear since this depends on a range of factors: including the extent to which previous price levels covered costs, the extent to which private operators can improve efficiency, and the financing costs of the investments needed to expand or improve service.

Critical Issues to realizing benefits for PPI

2.4 Looking ahead, if PPI is to be sustainable and benefit consumers of infrastructure services, governments will have to address a range of issues that were often overlooked in the initial rush towards private participation.3 First, governments must encourage realistic pricing structures that set and adjust tariffs toward cost covering levels, perhaps combined with explicit subsidies to achieve social objectives such as providing access to services in rural or other high costs areas. Second, the regulatory regime needs to provide opportunities and incentives for improving efficiency, and be viewed as legitimate by consumers and other host country stakeholders. Realizing the benefits of PPI will depend on whether commercial risks are transferred to the private participants, and the legal and regulatory regime provides a stable, transparent and realistic framework to induce them to incur those risks. Thirdly, governments need to build consensus for

3 A fuller discussion of these issues are contained in Ioannis Kessides, Reforming Infrastructure: Privatization, Regulation and Competition, World Bank Policy Research Report, 2004 and Clive Harris, Private Participation in Infrastructure in Developing Countries, World Bank Working Paper No. 5, 2003.


reform through public education and consultative mechanisms. Fourthly, financing for infrastructure investments need to be structured so as to mitigate the impact of currency mismatch. Finally, governments should explore opportunities for expanding services to poorer or under-serviced communities through public-private partnerships or non-traditional small-scale mechanisms.

2.5 These are formidable tasks but the challenges are inescapable. First, the scale of unmet infrastructure needs is vast and pressing in both low-income countries and in under-served or fast growing areas of middle-income countries. Secondly, expanding and improving these services needs to be paid for and efficiently managed, whether the infrastructure assets are in public or private ownership.

2.6 Against this background, PPIAF has a potentially important role in assisting governments to address the “unfinished business” of PPI. Its mandate covers the critical issues facing governments from defining strategies for all forms of private participation in infrastructure to facilitating transactions, from identifying global best practice to consensus building. Its portfolio of 5 years’ advisory work comprises a valuable inventory of case histories about alternative approaches to PPI and the lessons that can be drawn from experience of applying them. Finally, PPIAF’s operating model, combining a small, expert management unit with access to a broad pool of World Bank and other outsourced advisory capacity, allows it to offer and deliver a broad range of specialised assistance on a global scale.

2.7 The evolution of the environment for PPI demonstrates there is a continuing need for PPIAF, and also has important implications for the scope of its activities and mode of operation. Thus, PPIAF role should be seen as contributing to effecting change in consultation with client governments rather than the simply delivering expert technical assistance.

DEMAND FOR PPIAF SERVICES 2.8 The demand for PPIAF services also indicates the Facility remains relevant to promoting greater

private participation in investment and operations of infrastructure in developing countries. In the first five years of operation (FY 2000-2004) the Program supported 330 projects in 90 countries, involving aggregate budgeted PPIAF funding of US$ 77.2 million. 4

Overall Trend in PPIAF Activities

2.9 Demand for PPIAF-financed advisory has remained buoyant despite cooling of enthusiasm for expanding private participation in infrastructure. Over the five years from July 1999, PPIAF received over 600 applications for assistance, requesting aggregate funding of around US$ 160 million5. While there has been an evolution in the type of activities supported (discussed in more detail below) and in the form of private participation involved, the overall demand for PPIAF has been sustained and can be expected to continue. The volume of new applications increased in FY 2004 in both number (109) and value (US$ 22.9 million) after declining moderately in the previous two years, as shown in the following table. Applications increased further in the first quarter of FY 2005.

4 This excludes 6 “legacy” projects approved in FY 1999 and projects approved since 1st July 2004. 5 These figures include only applications that meet PPIAF’s basic criteria and justify detailed consideration.


2.10 The trend in the volume of projects approvals has followed a similar trend, showing a recovery in FY 2004 (63 projects involving US$ 14.2 million) after modest declines in the two previous years. PPIAF grants average around US$ 240,000, but range in size from support for small projects (less than US$ 75,000), which account for about 40% of the number of projects, and grants for larger projects, which average around US$ 360,000.

2.11 The broad geographic coverage of the PPIAF assistance further demonstrates its relevance as a global program: applications for advisory services have been received from 105 countries (in addition to global and regional projects). But, the country distribution of applications (set out in Appendix C) indicates that the breadth of coverage has not been matched by sustained demand in all countries. In 40 countries, PPIAF has funded only a single project indicating a lack of consistent and truly global promotion of PPIAF’s services.

2.12 It is striking that the highest proportions of PPIAF applications have come from the Sub-Saharan Africa (35% by value) and East Asia regions (19%), where PPIAF has field representation, and within Africa from the countries close to its field offices. This concentration of applications reflects the Program’s initial priorities and suggests there is additional untapped demand for PPIAF in other countries, if representation were extended. These two regions also account for around half of the number and value of approved projects.

Sources of Applications for PPIAF Funding FY 2000-04 by value

EE & CA12%

Global8%Latin

America12%

ME & NA4%

South Asia10%

EA & Pacific19%

Africa35%

Trend in Applications for PPIAF Funding

FYs 2000 2001 2002 2003 2004 Total Number 143 142 122 93 106 606

Budget (US$ million) 38.7 46.5 31.2 20.8 22.4 159.7

Note: by year of submission

Trend in Approvals

FYs 2000 2001 2002 2003 2004 Total Number 74 74 59 60 63 330

Budget (US$ m) 15.6 18.7 13.3 14.3 14.2 76.1

Note: by year of approval


2.13 PPIAF has devoted around a minor but still significant proportion of the Program to global projects focusing on the promotion and dissemination of best practice to the international community generally rather to specific countries: these have accounted for averaging 11% of the number of projects and 9% of total grant expenditures. These projects have included multi-sector studies and the development of “tool kits”, for example for highway, ports and water projects designed as training or reference materials for policy-makers. Over time, as opportunities for providing actionable assistance to national governments in policy and strategy development are exhausted, there may be a continuing role for PPIAF in maintaining and disseminating a world-wide database of best practice materials. However, the effectiveness of this type of activity is even more difficult to assess than country-specific assistance since they are directed to broad and diffused audience and cannot be related to country-level policy decisions or transactions. Our field visits indicated that the availability of PPIAF best practice materials on its web-site is not well known in the developing countries we visited, even among government officials directly involved in infrastructure. Gridline products in particular are virtually unknown and are too brief and superficial to be of value in elucidating important issues or illustrating lessons learned: many have been prepared while projects are still in progress. In general, the scope of PPIAF’s global activities appears to be fragmented among a range of disparate products and audiences, arising in part from PPIAF’s primarily reactive stance to applications received. To correct this, the PMU should take a more strategic approach to defining the scope of its global best practice program and priorities for specific activities. The TAP, which has already drawn attention to several major themes, should be closely involved in defining a coherent perspective to this aspect of the Program.6

2.14 The Program Charter mandates PPIAF to offer a broad range of services, extending from “upstream” assistance in designing infrastructure development strategies to “midstream” policy and institutional reforms to “downstream” support for pioneering transactions as well as assistance in overarching consensus or capacity building. The scope and relationship of PPIAF’s activities is illustrated in the following chart. The breadth of this mandate allows PPIAF considerable flexibility to adapt its activities to respond to the varying demands from developing country clients.

Capacity Building

PPIAF Activities

Legal Framework

Regulatory and Political Institutions

FacilitateTransactions

Consensus Building

6 In recent months, the PMU has initiated several projects to support themes identified in the FY 2005 Indicative Work Program.


2.15 Over half of PPIAF projects (number) have been directed to Policy, Regulatory and Institutional Reforms (RF 33%) and Infrastructure Development Strategies (ST 27%). However, the proportion of Capacity Building (CB) and Consensus Building (CS) projects have increased over the five year period from a 19% (combined) in FY 2000 to 36% in FY 2004 (See Appendix E). This shift towards capacity and consensus building, endorsed by the TAP, has been attributed in part to the completion of strategy definition work in many countries and in part as a response to the growing criticism of PPI. Our detailed study of individual projects in the African and East Asian region (discussed in detail in the following section of the report) did not indicate any statistically significant variation in the performance of different types of project, but the strong influence of the host country’s policy environment on project outcomes suggests that support should generally be provided only to governments that are committed to a clear PPI strategy.

2.16 Support for Pioneering Transactions, typically through funding transaction advisors to assist Governments in negotiations with prospective operators or investors in major infrastructure investment projects, has accounted for a very minor (less than 3%) part of the PPIAF program. The limited use of this service appears to have been largely the result of a deliberate policy on the grounds that transaction advice tends to be expensive (experienced transaction advisors command high fee rates) and high risk (the transaction may not be concluded for many reasons), and that Government’s can tap other sources for funding advisory services for viable investment projects. These may be generally valid considerations. On the other hand, Governments of many lower income countries are often reluctant to engage competent (mainly foreign) advisors and, without them, to enter into meaningful dialogue with international operators. Many of the benefits of PPI depend on successful completion of capital investment in new or upgraded facilities, which can have an important demonstration effect for the host country (and for the PPIAF Program). There have been indications that PPIAF has been concerned at avoiding an apparent conflict of interest involved in financing advice to Governments on transactions in which the World Bank and/or IFC are investors (especially where the World Bank is the contracting agent). PPIAF should not be constrained by these considerations from selectively providing transaction support to governments committed to a positive PPI agenda.7

2.17 The Program Charter defines PPIAF’s mission as “to help eliminate poverty and achieve sustainable development by facilitating private participation in infrastructure.” However, PPI does not automatically contribute to reducing poverty. As a recent PPIAF publication concludes, “while the limited available evidence suggests that private provision (of infrastructure) has probably increased access, the extent of any positive effects will depend on several government policies (italics added): namely allowing entry and permitting competition; clarifying property rights and facilitating contract enforcement; allowing prices to cover costs; setting appropriate quality

7 The PMU is currently working on procedures to allow the Program to fund recipient-executed projects.

Distribution of PPIAF Activities FY 2000-04(number approved)

ST27%

BP11%

CB17%

CS10%

RF33%

PT2%


standards; and, targeting subsidies to the poor.”8 Our study suggests that PPIAF’s the Poverty Reduction focus has not been effectively translated into operational terms. In most cases, applications for PPIAF support include only perfunctory reference to the contribution the project is expected to make to poverty reduction: a typical example stated, “The role of economic growth is widely regarded as a major factor that reduces the incidence of poverty”. In addition, several of the project reports reviewed did not deal adequately with the key issues of the affordability of tariffs and expanding access in rural and peri-urban areas. While not all potential projects will have a direct effect on poverty, PPIAF should strengthen the scope and depth of poverty impact of its projects. In, particular, the ToRs for all projects that potentially impact on service and/or pricing levels of utilities essential to the poor should be required to assess their potential poverty implications.

2.18 PPIAF has sought to allocate its funding towards lower income countries and to limit or reduce support to middle-income countries, apparently with the objective assisting poverty reduction. However, national income level is a poor proxy indicator of a project’s potential contribution to poverty reduction PPIAF should proactively seek to support projects that will have a strong positive poverty reduction effect even in middle-income countries because of their direct benefits to alleviating poverty in the country and their contribution to in expanding PPIAF’s knowledge of successful solutions that may be replicable in poorer countries.

2.19 In general, PPIAF is intended to operate as a “demand-driven” facility, critically evaluating applications for assistance proposed by beneficiaries rather than proactively promoting its own proposals. However, this principle is not a completely sufficient guide to allocating resources. First, in many cases, the proponent for a request is a World Bank Task Managers rather than a host government, and in some of these at least formal government endorsement has clearly not proved to represent a substantive commitment to act on the results of the assistance. More importantly, PPIAF has now matured the point where it can independently assess what types of activity are most effective and the conditions for carrying them out successfully. In future, PPIAF should adopt a more strategic approach to allocating resources among competing uses and proactively seeking to promote PPIAF assistance to high priority projects. The organizational implications of this shift in approach are discussed in section 6.

PPIAFS COMPARATIVE ADVANTAGE 2.20 Overall, PPIAF has demonstrated a strong comparative advantage derived from several distinctive

features that justify a further extension of the PPIAF program. A number of other donor-funded TA programs also support policy and institutional reforms relevant to PPI, including, ADB, UNDP, UNIDO and USAID that have occasionally co-financed projects with PPIAF, but none of these have such a comprehensive program as PPIAF. In summary, PPIAF has:

• A clear and coherent functional focus. The Program Charter provides a clear and coherent mandate, which is not always present in other technical assistance programs. The types of activity it supports are clearly connected to the overall mission.

• A broad range of deliverables and access to WB multi-sectoral expertise. A majority of the beneficiaries interviewed during our study said they valued PPIAF assistance because of quality and range of expertise available as well as the grant funding.

• Global reach combined with an ability to fund multi-country activities. Access to in-depth country knowledge and a network of field offices would be difficult to match.

• Strong donor support (especially DFID) allows PPIAF to provide sustained (multi-year) funding for its activities. PPIAF’s projects approval procedure requiring consultation with donors ensures co-ordination with their programs.

8 Timothy Irwin and Penelope Brook, Infrastructure and the Poor: Increasing Access, in Infrastructure for Poor People, page 10.


• A cost-effective and efficient management structure. The combination of a dedicated management unit, grant funding and a broad range of outsourced specialist skills give PPIAF a capability to respond effectively to client requests.


3. IMPACT

3.1 A major purpose of this study was to analyse the extent to which PPIAF has been fulfilling its mandate by examining the impact of its country-specific projects. To address this basic issue, we examined the following questions:

• Has PPIAF reached the targeted level of assistance envisaged in its founding documents?

• Has the quality of the services funded by PPIAF been satisfactory?

• Have PPIAF-funded services made a significant, positive contribution to supporting PPI in the countries served?

ATTAINMENT OF PROGRAM TARGETS 3.2 Although the PPIAF founding documents did not set explicit quantitative targets, the World Bank

submission for funding from the Development Growth Facility (DGF) provides a benchmark to assess the quantitative development of the Program. After a slower than projected start in the first two years’ of operation, PPIAF has exceeded the expected level of activity by over 25%, reaching cumulative expenditures of US$ 41 million by the end of FY 2003 against a target of US$ 32 million as shown in the chart below. Although the volume of expenditure does not reflect the value of delivered services since it includes projects still in progress, it indicates that PPIAF achieved a rapid start up of operations seldom achieved by new programs.

QUALITY OF SERVICES PROVIDED 3.3 To assess the quality of services, we reviewed the project deliverables for the sample of 32

projects, interviewed the beneficiaries of those projects, studied TAP reports and gained general impressions from interviews with donors and others familiar with PPIAF.

3.4 Our detailed study confirmed the view of the TAP and other observers that the services funded by PPIAF are of high quality across a wide range of interventions.9 Of the 32 projects reviewed, 27 (84%) of the project outputs including a cross section of different activities were assessed as being fully or mostly satisfactory. Detailed ratings of the 32 projects, which include a cross section of PPIAF’s activities, is included in Appendix F.

9 See Technical Assistance Panel, Review of Activities Completed During Fiscal 2003, April 2004, page 8.

Projected and Actual Program Expenditures

(cumulative)

17,00021,000

32,000

5,355

14,673

26,611

41,076

28,000

0

10,000

20,000

30,000

40,000

50,000

2000 2001 2002 2003

US$ 000

Projected Costs Actual Costs


3.5 To gain a balanced view of the quality of the Program, we asked project beneficiaries how satisfied they were with the PPIAF-funded services they had received. Beneficiaries of 22 (78%) of the 28 projects reviewed that we were able to meet were highly or mostly satisfied with services received. Although these views represent perceptions rather than concrete outcomes of the projects, they add a dimension to the evaluation of the Program that has been lacking from previous assessments, and confirms the generally positive reputation of the PPIAF program.

CONTRIBUTION TO PROMOTING PPI 3.6 Since PPIAF’s inception, donors have emphasized the importance of focusing attention on

achieving measurable impact. Accordingly, based on recommendations of a study by consultants in 200210, the PMU has been collecting data designed to assess the Program’s performance in achieving its objectives. This effort has been focused on measuring the medium-term outcomes of the Program (e.g. government infrastructure strategies introduced or improved, PPI laws enacted, PPI transactions assisted) rather than the ultimate, longer-term impact of improving the access, quality and delivery of infrastructure services. Because of the extended time-scale needed to achieve these longer-term objectives and the difficulty of isolating the specific contribution of the PPIAF-funded activities among the many factors that influence progress in

10 Assessing the Impact of the Public-Private Infrastructure Advisory Facility: A Review of the First Three Years, Team technologies, inc. May 2002.

Quality of Project Outputs

11

16

5

00

5

10

15

20

Fully Satisfactory MostlySatisfactory

PartlySatisfactory

Not Satisfactory

Number of Projects

Rating of Client Satisfaction

5

17

6

0

4

0

5

10

15

20

FullySatisfactory

MostlySatisfactory

PartlySatisfactory

NotSatisfactory

InsufficientData

Number of Projects


developing infrastructure, the focus on proximate outcomes is justifiable at this stage of the Program’s evolution. It is probably not useful to devote significant effort in attempting to assess PPIAF’s longer-term impact for another 5 years.

3.7 Our evaluation confirms that PPIAF is contributing to improving the enabling environment for PPI. Of 32 mature projects approved prior to FY 2004, 15 or 48% were assessed as having achieved a fully or mostly satisfactory outcome, but in another 12 cases it was either too early to say or there was not sufficient evidence to make a clear judgment. If the latter cases were excluded, 75% of the projects achieved the higher ratings, but this probably gives too positive an impression of general as in several of these cases there had been extended delays in completing or implementing projects, e.g. the proposed restructuring of the State Railways of Thailand. In general, the speed of project execution and progress toward implementing recommended strategies or policies, and completing actual PPI transactions has been slower than expected. As discussed in the next section, PPIAF procedures need to reflect a more realistic timeframe to help ensure that projects attain their objectives.

3.8 We also made a composite rating of the projects studied, incorporating an admittedly subjective assessment of the probable outcomes as well as the appropriateness of the project deliverable to the country situation, the quality of the delivered outputs and the client’s acceptance of those outputs. On this basis, 20 (63%) of the projects studied are assessed to have been fully or mainly successful.

Rating of Project Outcomes

2

13

4

1

12

0

5

10

15

20

FullySatisfactory

MostlySatisfactory

PartlySatisfactory

NotSatisfactory

InsufficientData

Number of Projects

Overall Rating of Projects Reviewed

5

15

12

00

5

10

15

20

Fully successful Mostlysuccessful

Partly successful Not allsuccessful

Number of Projects

Michael Jorda

3.9 An in-depth analysis of the Impact Database11 compiled by the PMU confirms that a significant proportion of PPIAF projects achieved positive outcomes. This analysis correlated the outcomes recorded by PPIAF in the database with all 282 country-specific projects approved, and showed that about 45-50% of all projects approved in FYs 2000 and 2001 had at least one reported outcome. This percentage is probably understated as the PMU has relied mainly on TMs to provide this information and not all have responded. However, the analysis highlights the extended timeframe for delivering PPIAF services and for achieving the proximate results in terms of improved government strategies, policies, institutional arrangements and even longer horizon for achieved increased PPI: so far, only 20% of the projects approved in FY 2003 have recorded specific outcomes although there is no evidence to suggest that the quality of projects has gone down12. However, PPIAF procedures need to accommodate the extended timeframe needed to achieve substantive results from PPIAF activities.

3.10 There ratings(out oflimitedsignificoutcom

3.11 Simila45 couprojecalso vcreditedifferinoperat

11 The title of thdefined as outcoinfrastructure se12 The analysis delivery of outpu13 Based on a ssatisfactory.

Proportion of country specific projects with

n & Associates 15

are significant variations in the performance of projects between countries. The average of projects in the seven countries covered by our detailed study ranged from a high of 2.3 possible 3) in Cambodia and Vietnam to a low of 1.3 in the Philippines.13 Because of the number of cases studied, these figures cannot be regarded as having precise numerical ance, but they confirm that the country environment has a substantial bearing on project es.

rly, all the 91 projects with recorded outcomes in the Impact database are concentrated in ntries (see Appendix G). Among the countries in which PPIAF has executed 5 or more

ts shown in the following table, the proportion of projects with recorded actual outcomes aries widely. In the remaining 42 countries, none of the 65 executed projects have been d with any recorded positive outcome. These variations in project performance reflect g degrees of political stability, acceptance of PPI or commitment to creating an effective ing environment for it. They also suggest that PPIAF should apply more stringent country

is database is a misnomer since it contains information of parameters that would more generally mes (i.e. institutions strengthened, laws passed, etc.) rather than impacts (i.e. improvements in rvices) but it is generally recognizable by this name. of project execution in Chapter 4 indicates that projects take on average 18 months from approval to ts, indicating that actual outcomes can only be expected in a longer timeframe. cale, ranging from 3 for fully satisfactory, 2 for mainly satisfactory, 1 for partly satisfactory to 0 for not

recorded outcomes in Impact Database % of projects approved in year

24% 27%19% 16%

2%

20% 24%

17%

6%

4%0%

10%

20% 30%

40%

50%

60%

2000 2001 2002 2003 2004

Outcome Pending Outcome Achieved


criteria to its project screening processes. In particular, a country’s track record and progress in reform generally should be assessed as important indicators of Government commitment to PPI.

3.12 The analysis of the distribution of outcomes by year of approval, by size or type of activity did not indicate any statistically significant variations in performance among projects.

Recorded Impact of Projects in Most Active PPIAF Countries

Number of Projects

With Actual Outcomes

% of Projects

With Actual & Pending

Outcomes % of

Projects Peru 5 3 60% 3 60% Thailand 7 2 29% 3 43% Laos 5 1 20% 2 40% Brazil 5 1 20% 2 40% Vietnam 7 1 14% 4 57% Cambodia 8 1 13% 3 38% Kenya 10 1 10% 6 60% India 10 1 10% 2 20% Philippines 11 1 9% 2 18% Nigeria 7 0 0% 3 43% Indonesia 5 0 0% 0 0%

Total 80 12 15% 30 38%


4. EFFICIENCY AND EFFECTIVENSS

4.1 To assess the efficiency and effectiveness of the PPIAF Program we examined the following questions:

• Has the Program achieved an efficient use of resources?

• Is the governance and management of the Program transparent and effective?

• Has the funding structure affected the sustainability of the Program?

USE OF RESOURCES 4.2 The PPIAF Charter (paras 31 et seq.) defined the Program’s distinctive operating model

comprising a small Program Management Unit (PMU) focused on administration of the Program, and delivery of PPIAF-funded activities outsourced to World Bank Group staff and external consultants. Total PMU HQs and field office staffing has remained at around 15 for the past three years.

4.3 This operating model has proven to be efficient and cost effective. As noted in Section 2, the Program has exceeded its original operating expectations for assistance delivered. It has also successfully contained the program management costs, which have declined steady from 28% of total Program expenditures in FY 2000 to 13% in 2004. While it is difficult to compare the costs of TA programs since they vary widely in scope and method of operation, PPIAF can be judged a cost-efficient model. The detailed composition of PPIAF expenditures is given in Appendix H.

GOVERNANCE AND MANAGEMENT 4.4 To examine this issue, we also assessed the effectiveness of PPIAF’s organization structure and

major management processes.14

Organization Structure

4.5 The governance and management organization of PPIAF comprises a Program Council (PC), a Technical Advisory Panel (TAP) and a Program Management Unit (PMU).

4.6 The Program Council is the governing body of PPIAF, composed of representatives of the official donors contributing resources to PPIAF, and chaired by the World Bank. The PC is responsible for defining Program policies and priorities, approving the annual work plan and financial plan, reviewing PPIAF performance and overseeing the TAP and PMU. The PC meets once a year.

14 The OED Independent Evaluation of the World Bank’s Global Programs (Annex Table 15) ranks PPIAF’s Governance and Management Arrangements among the highest of 26 Programs examined in detail, including Transparency (High); Clarity of Roles and Responsibilities (High); Fairness to Clients (High), Accountability to Donors (High); Accountability to Developing Countries (Modest); Accountability to Professionals (Substantial). However, OED’s report documenting these findings has not yet been released.

Trend in Program Expenditures

US$'000 2000 2001 2002 2003 2004 Client Activities 3,847 6,865 9,337 12,058 13,534 Program Management 1,508 2,453 2,601 2,407 2,079 Total Program Expenditures 5,355 9,318 11,938 14,465 15,613 Management % of total 28% 26% 22% 17% 13% Source: PPIAF Annual Reports


4.7 Based on a review of the PC meeting documents and discussions with several donors and the PMU, it appears that the PC as presently structured allows donors to participate in the governance of the Program, but is not well suited to provide effective strategic direction for PPIAF. The composition of the PC is large: 20-25 representatives, few of whom are present regularly, as well as additional observers attend meetings. The short annual meeting provides insufficient time to conduct a in-depth review of the Program’s performance, review work plans or address important strategic issues facing it: discussions on these matters have lasted less than 3 hours, although this weakness has been exacerbated by the lack of substantive strategy papers to review. PC meetings have not resulted in any major policy developments. While the PC should remain a representative forum for donors, there is need for smaller, more tightly knit body to provide supervision and guidance between PC meetings.

4.8 The Technical Advisory Panel, comprised of seven international experts, is intended to provide independent advice on strategy and to evaluate the impact of the Program by carrying out ex post evaluations of a sample of PPIAF projects. The Panel has met regularly between FY 2001-2004 and prepared evaluation reports on 45 individual projects.15 However, TAP reviews vary in quality and have several limitations as the main instrument for evaluating the performance of the PPIAF portfolio: • They focus heavily on professional review of deliverables (outputs) rather than assessing

outcomes; • In many cases, they have examined projects too soon after delivery to identify the results

achieved; • They do not include assessments by beneficiaries, and relied heavily on comments from

TMs, who cannot be regarded as disinterested observers; • The concept of “value for money” appears limited to “reasonable cost for equivalent service”,

which is not a meaningful proxy for a cost: benefit analysis. 4.9 The Program Management Unit is responsible for managing the day-to-day operations of the

Facility. The PMU’s role as set out in the Charter (para 32) is focused on reviewing proposals (point a), arranging delivery of PPIAF programs and activities (point b), and proposing and administering the work plan and budget (point e). The charter does not assign the PMU responsibilities for strategy formulation or program evaluation: these are important omissions, and should be corrected.

4.10 The PMU has remained small (with a total staff of 15 handling a total portfolio of over 300 completed or ongoing projects). The PMU is structured around a Program Manager, Deputy Program Manager and four Regional Co-ordinators (RCs), two based in Washington and two in field offices in Africa and East Asia (see Appendix I). As discussed in subsequent sections of this review, the PMU has developed a transparent and efficient process for soliciting proposals for assistance and handling the applications received. The PMU is generally well regarded by donors and TM’s as being efficient, responsive and transparent. However, it should take a broader perspective to the managing the overall direction of the program.

4.11 However, after five years of operation, the small size of the PMU has become a constraint on the effectiveness of the Program in two ways. First, the increasing project portfolio has stretched the staffing resources of the PMU, leaving few resources to take a more strategic perspective on the direction and performance of PPIAF’s overall portfolio. Secondly, the PMU’s lack of field presence (limited to Africa and East Asia) and travel budget for Washington-based RC’s have severely limited the PMU’s capacity to take on a broad promotional role. PMU field staff in the 2 regions has made important contributions to projecting PPIAF as a multi-donor facility and in generating additional demand from potential clients. The staffing constraint was temporarily aggravated during 2004 by the turnover of personnel in 4 of the top 6 positions in the PMU. In addition, without the assurance of multi-year funding, the PMU feels unable to offer multi-year

15 The TAP’s general reviews are discussed in Section 5.


contracts to newly recruited staff, which is clearly a serious limitation in attracting high calibre professional staff for international postings.

4.12 Overall, PPIAF’s the relationship with the World Bank has contributed to the efficiency and effectiveness of the Program. Most front-line management of PPIAF-funded projects is handled by Task Managers engaged by the PMU. Virtually all TMs are World Bank staff members: the Asian Development Bank (ADB) staff are managing 6 PPIAF-funded projects. Most TMs are well-qualified professionals and the access to the World Bank’s pool of expertise is a major advantage to PPIAF. In addition, PPIAF draws on support from World Bank field offices. However, there are some negative aspects to the relationship that need to be mitigated. In some cases World Bank TMs have been less diligent in post-delivery follow up where there is no immediate prospect of a World Bank lending operation or government action. It is desirable that PPIAF is not regarded as an exclusively World Bank facility and that other source of TM skills are developed. In addition, in many beneficiary countries PPIAF is not identified as a distinct entity from the World Bank and its development agenda.

Major Management Processes

4.13 Planning. The main PPIAF planning document is the Indicative Work Program (IWP), which is presented annually to the Program Council for approval. This comprises an estimate of expenditures for the coming year under three general headings: core technical assistance activities, promotion of best practices and program management. Since PPIAF has followed the demand-driven approach to handling proposals, the IWP target for core activities is more a forecast than a target. The main directional significance of the IWP is the level of management expenditures, which in FY 2005 is targeted at 15% of the total. However, the IWP format is not an effective tool for setting direction or priorities for the Program. In particular, the distribution of activities financed by PPIAF is a consequence of the applications approved and is of little value in judging whether the Program is actual addressing client needs.

4.14 Origination. PPIAF’s procedure for soliciting and screening applications for assistance, defined in the Program Charter (Annex III) is transparent and generally timely and effective. This procedure calls for mandatory assessment by independent technical assessors and confirmation of non-objection by Program Council members for all Medium and Large Proposals, i.e. requesting PPIAF-funding for between US$ 75,000 to US$ 250,000, and over US$ 250,000 respectively. For small proposals (up to US$ 75,000), the PMU is authorised to approve proposals based on its own assessment, but may call on independent assessors at its discretion. This process generally works well and expeditiously. The proportion of applications rejected has fallen from 35% in FY 2001 to 20% in FY 200416, but this appears to be because fewer unsuitable applications are proposed rather reflecting than less stringent screening. The average processing time for large and medium proposals has increased, but this mainly reflects delays in meeting conditions for a few difficult projects: if the slowest 10% of proposals are excluded, the average processing time for FY 2004 falls to 2.5 months.

16 See Appendix J

Average Timeframe For Approval By Size Of Proposal (months)

FY 2000 FY 2001 FY 2002 FY 2003 FY 2004 Large 2.16 2.88 4.34 5.06 4.49 Medium 1.83 1.57 3.92 3.34 5.86 Small 1.05 1.44 1.56 2.22 1.80 Overall 1.58 2.04 3.13 3.70 3.61


4.15 Project Execution. Project implementation is generally well managed, but as is common with many TA programs most activities take considerably longer than planned (an average of nearly 10 months for the projects we studied in depth). The major sources of holdup are the procurement of consulting services where international competitive bidding is required and in reaching consensus on final reports or other deliverables. These discrepancies in timing reflect unrealistic planning rather than poor execution.

4.16 PPIAFs approach generally recognizes the importance of thorough consultation with beneficiary governments and other stakeholders in conducting advisory assignments. Developing PPI often calls for examining a range of options and lengthy decision-making processes involving many parties (governments, legislative bodies, regulators, operators and consumer and community groups). Most Project ToRs require PPIAF TMs and advisors to interact with interested parties through steering committees to review work in progress, and hold workshops to disseminate recommended solutions. Others are less specific. More importantly where there are delays in implementation, consultation may be neglected.

4.17 But, extended implementation periods detract from project quality because they create additional problems for effective follow up and closure of projects. TMs and consultants are often programmed to move on to other assignments and may not have sufficient time to devote finalization of reports and dissemination of the deliverables. PPIAF’s procedures need to address how to ensure effective follow up during implementation, and are discussed further in Section 6.

4.18 Monitoring and Evaluation. PPIAF had developed an interactive management information system to monitor program activities. The PPIAF Applications Tracking System (PATS) provides a comprehensive framework for recording information and recording key documents related to PPIAF activities, but has several weaknesses as a management tool. First, the format does not permanently record key milestone dates to be monitored (e.g. the procedures do not distinguishing original target completion date and revised dates). TM’s can change recorded completion dates at will, without endorsement from the PMU, which effectively eliminates their usefulness as benchmarks for program monitoring. Secondly, the input of data and reports is irregular and incomplete. Activity Update Summaries (AUS’s) progress reports were only prepared for about 60% of the projects studied, and completion reports for about 35%: in some cases there was no updating of information at all.17 Thirdly, the Completion/Sign Off report does not provide an adequate basis for evaluating the outputs and most importantly the outcomes of completed projects. In particular, they do not incorporate client responses or independent assessment of project impact. As a result of these weaknesses, the PMU is not well placed to monitor implementation.

4.19 In addition, PPIAFs monitoring system does not provide for consolidation and analysis of individual project reports to provide a basis for evaluating the performance of the overall portfolio. Quarterly reports presently focus on demand data and do not give management or the Program Council the information needed to assess general progress in project implementation, measure the impact of the program, or identify areas of strong and weak performance.

17 The PMU has recently taken steps to correct this.

Delays in Delivering Project Outputs in Sample Studied

By Project Rating

Overall Average 3 2 1

# projects 30 5 13 12

Time to complete (months) 18.60 13.15 19.70 20.37

Delay beyond target (months) 9.91 9.01 10.59 9.45


4.20 TAP was designed to serve as the main instrument for evaluating the performance of the PPIAF Program. However, TAP project reviews have had several weaknesses discussed in para. 4.8 above. The TAP has progressively expanded its work to cover thematic issues, but this does not replace the need to strengthen PPIAF’s project evaluation process.

PROGRAM FUNDING 4.21 PPIAF has consolidated its position as a multi-donor facility by attracting almost US$90 million

(as of 30 June 2004) from an expanded group of 13 multilateral and bilateral donors to support its advisory activities and cover management costs. As shown in the following table, funding has increased progressively to meet the program needs. While funding from donors outside the original nucleus (United Kingdom, Japan and the World Bank) has increased to 22% of the total, PPIAF remains heavily dependent on the continuing support of DFID. A full list of donor contributions is given in Appendix K.

4.22 In common with a number of other development assistance programs, PPIAF has a two-tier funding structure, comprising a Core Fund and Non-Core Funds. Core Funds are available to support all Program activities: the use of Non-Core Funds is restricted to themes, regions or activities designated by the contributing donor. This structure allows donors to draw on PPIAF’s global capabilities to support their individual development assistance priorities. However, it poses a potential risk of diluting the strategic direction of the Program on a divergence of donor interests leads to a fragmentation of its efforts. This does not appear to be a problem for PPIAF. Core funds account for around 70% if the total, as of June 2004.

4.23 As an established program with a strong demand for its services, PPIAF faces a continuing requirement for funding. As of June 2004, PPIAF had available funds of US$ 6.3 million to support new activities, equivalent to about 5 months of new commitments at the current rate. In addition, PPIAF will need to secure additional funding to cover increased management costs arising from ongoing organizational changes, and from recommendations of this review (see Section 6). In addition, to allow PPIAF flexibility to recruit international staff on extended contracts, it is desirable that at least a portion of new donor funding be committed for a multi-year period.

Funding for the PPIAF Program

US$000 FY 2000 FY 2001 FY 2002 FY 2003 FY 2004 Type of Funding

Core 14,555 27,681 33,458 49,305 62,594 Non-core 2,991 5,751 13,501 23,159 28,071 Net investment income 1,026 1,335 2,287 Total funding 17,546 33,432 47,985 73,799 92,952

Donor Contributions UK 9,111 17,584 23,011 35,472 47,510 World Bank 2,625 6,425 8,930 10,930 12,930 Japan 3,400 6,081 8,043 10,043 10,043 Other 2,410 3,342 6,975 16,019 20,182

Total funding 17,546 33,432 46,959 72,464 90,665 UK 52% 53% 49% 49% 52% World Bank 15% 19% 19% 15% 14% Japan 19% 18% 17% 14% 11% Other 14% 10% 15% 22% 22%


5. LESSONS LEARNED

5.1 As a knowledge-based assistance Program with an increasing inventory of case study information on the issues facing the development of PPI in a large number of countries, PPIAF has a potentially important role in interpreting and communicating the lessons of its experience. Over time, PPIAF can make a valuable contribution to PPI by compiling and disseminating global experience of good practice.

5.2 To assess how well PPIAF has been doing in identifying and applying the lessons its experience, we have considered three questions:

• Has PPIAF acted on the recommendations of earlier evaluations?

• Is PPIAF effectively evaluating and applying the lessons of its own experience?

• Is PPIAF effectively disseminating its knowledge to the development community and beneficiary countries?

RESPONSE TO PREVIOUS EVALUATIONS 5.3 As part of its Annual Reviews, the TAP highlighted a number of areas of concern that warranted

action by PPIAF. In addition, in 2002 PPIAF commissioned a special study to assess the impact of PPIAF activities and develop recommendations for improving the design and delivery that was carried out by Team Technologies, Inc.

5.4 The TAP Annual Reviews for FYs 2000, 1, 2, 3 highlighted four major areas of concern, based on their evaluation. PPIAF’s responses to each of these are discussed in turn.

5.5 Measuring Impact. The TAP has drawn attention in all four Annual Reviews to both the importance of measuring the impact of PPIAF activities and the difficulties it faced in fulfilling its evaluating mandate, in particular the lack of follow up information on specific outcomes achieved and of feedback from beneficiaries. PPIAF has made efforts to address this concern, in particular by commissioning the 2002 consultant study. This recommended that PPIAF should focus on measuring the Program’s medium-term results in helping governments improve the enabling environment for PPI. Since then, the PMU regularly collected and published a set of performance indicators of medium-term outcomes: reform strategies adopted, laws drafted, new institutional arrangements set up, PPI transactions facilitated, and officials trained. These indicators of project outcomes are appropriate measures for assessing the proximate performance of the PPIAF Program and the data collected are an important source of information. However, more should be done to analyse and interpret the information, for example by examining the distribution of recorded outcomes by country and activity. In addition, as already noted, the weaknesses in follow up reporting highlighted by the TAP remain to be corrected.

5.6 Dissemination of Experience. The TAP’s 2000 Summary Review concluded that while PPIAF was producing high quality work, it should improve the dissemination of information of reports and other material arising from its activities. This point is discussed below.

5.7 Broader Stakeholder Consultation. Thirdly, several TAP reviews have stressed the importance of involving private sector participants in PPIAF activities, and suggested more should be done. We strongly concur with this view but have seen little evidence that interaction with private sector participants is being actively pursued. Private water operators were invited to participate in reviewing the final draft of the Water Sector Users’ Guide and made valuable constructive suggestions to it. This seems to be more the exception than the rule.

5.8 Recognising the Political Constraints on PPI. Since FY 2002, the TAP has commented that PPIAF-funded activities have traditionally focused on the economic and technical rather than political aspects of PPI and accordingly often did not recognize the political constraints on implementing reform in the infrastructure sectors. This weakness has been evident both in the tendency, at least in earlier years of the Program, for consultants to promote more advanced


forms of PPI, e.g. concessioning of water services, and for inadequate attention to consultation and consensus building activities. PPIAF has moved to address this concern. For example, the TM managing the preparation of the Cambodia Country Framework Report prepared a detailed Action Plan, setting out the specific steps proposed to implement major recommendations of the CFR that served as an agenda for discussions with Government and a program for future work. More generally, PPIAF initiated and funded a study of Emerging Lessons in Consensus Building and Stakeholder Communications in PPI.18 So far, this study has been distributed to “a small group of experts within the World Bank and selected external experts”. No steps have so far been taken to translate the important conclusions and recommendations of the report into PPIAF’s operational practices.

5.9 In summary, PPIAF has been responsive to the concerns raised by the TAP, mainly at the Program level, for example through global or regional initiatives. The record of addressing the concerns that apply to country specific activities is mixed. Some improvements have been made but the PMU needs to do more in developing operational procedures to ensure that recommendations regarding reporting, consultation and follow up are applied consistently by all TMs.

SELF-EVALUATION 5.10 So far, the PMU has only taken limited steps toward in-house evaluation. The main instrument is

the Completion Report/Sign-Off Note. However, as noted in Section 4, this report is poorly designed, in particularly in confusing delivering output with achieving outcomes, does not require input from beneficiaries, and has only been prepared for a minority of completed projects. It is not a sound basis for the PMU to assess the effectiveness of the Program’s activities.

DISSEMINATING KNOWLEDGE 5.11 PPIAF has contributed to disseminating knowledge by supporting best practice development,

promoting regional regulatory fora and distributing Global Best Practice (GBP) materials.

5.12 PPIAF has devoted a significant effort to the promotion of Global Best Practice accounting for 13% of Program expenditures in FY 2004, including the production of toolkits that have been generally well received by officials in beneficiary countries and others, and generic reports on cross-cutting themes. On the other hand, Gridlines (two page summaries of projects) appear to be brief, superficial and prepared before the outcomes of projects have been identified to be useful in communicating significant lessons or promoting the Program.

5.13 PPIAF has also promoted the development of regional regulators’ network (particularly in Africa and South Asia) to serve as fora for exchanging information and experience.

5.14 Thirdly, PPIAF has broadened accessibility to GBP materials via the PPIAF and World Bank Private Sector Departments Rapid Response Unit websites.19 However, attention to the dissemination of PPIAF materials is still very uneven. For example, the Highways toolkit has been actively promoted and disseminated by World Bank Transport Specialists through workshops and distance learning courses, organized in cooperation with the World Bank Institute (WBI) in Africa, Asia and Latin America. But, the ToR for the recently completed project to prepare a Water Sector Users’ Guide did not include any provision for dissemination outside the World Bank. In addition, there is no explicit plan for promoting the GBP materials posted on websites, or for comprehensive distance learning. Our discussions with Government officials in Africa and East Asia indicated that the availability of these materials is not well known at country level, and the PMU should prepare a coherent plan for disseminating and promoting GBP materials.

18 Report prepared by IP3, the Institute for Public-Private Partnerships, Inc., completed in July 2004. 19 Toolkits posted on the RRU website are not identified as PPIAF-funded.


6. RECOMMENDATIONS

6.1 Overall, this review of the PPIAF’s first five years of activity has validated the relevance of its role and operating model. It has also suggests there is room for improvement to adapt the model to the increasingly challenging environment for PPI and to build on the experience of its pioneering years, and to capitalize fully on the high quality of the assistance funded. This section sets out recommendations to achieve this, based on four major themes:

• Strengthening Strategic Direction of the Program. • Increasing Stakeholder Participation in Management and Governance of the Program. • Focusing on Outcomes rather than Outputs. • Enhancing PPIAF’s Identify as an Autonomous Multi-Donor Program.

6.2 Several recommendations build on the reports of the TAP and observations by the PMU and others. Overall, the proposals represent a fine-tuning of the Program rather than a radical new departure. However, to realize the improvements needed, it is critical that the donors, through the Program Council, give clear direction of the steps to be taken and monitor them, perhaps through an implementation audit in 2006.

STRONGER STRATEGIC DIRECTION 6.3 Most importantly, to optimize the impact of the Program, PPIAF needs to balance the demand-

driven approach project selection with a strategic vision of overall program priorities and the key factors for success in its operations. This calls for several changes.

A More Strategic Approach to Setting Country Priorities

6.4 The PMU should move towards a more strategic approach to client selection based on a systematic interpretative evaluation of country performance, incorporating World Bank and donor country assessments and PPIAF program results. • Among countries, PPIAF should distinguish and concentrate promotional efforts to assist

those countries where the Program has the highest potential to make meaningful contribution based on stability, level of FDI, commitment to reform and capacity to implement changes. A schematic approach to differentiating the scale of PPIAF’s effort among different groups of countries is illustrated in Appendix L.

• Within countries, for an initially, limited group reforming countries, PPIAF should define sectoral or thematic priorities based on (a) country diagnostic and (b) a prioritized sequence of potential project interventions, prepared in consultation with host government, WB and donors. PPIAF should adapt its activities in individual countries to the level of progress in defining a meaningful and feasible role for private participation in infrastructure, generally and in specific sub-sectors. In countries where this role is not well defined and accepted, PPIAF should generally limit its support to “upstream” strategies. RC have key role in this process. For other advanced, progressive and turnaround countries, define specific areas where PPIAF will focus support (in general, downstream support will only be considered in sectors where policy reform including PPI strategies are in place). More proactive search for opportunities in under-represented countries (e.g. South Africa, Latin America) and/or sectors (e.g. urban transport) is needed.

• PPIAF should develop a more strategic approach to promoting GBP projects, based on explicit priorities and a multi-year program.

6.5 These proposals do not imply a dilution of the principle that the Program is “demand driven” with client commitment an essential criterion for project selection. PPIAF’s field staff should be proactive in promoting PPIAF’s assistance to potential beneficiaries, but approval of assistance for specific proposals should still depend on demonstrating credible evidence of a supportive country environment and commitment to implement the output. Indeed, there may be a case for applying more stringent cost-sharing conditions for some middle-income countries.


6.6 To implement a more strategic approach to planning, a senior staff member of the PMU staff should be assigned specific responsibility for Policy and Planning, and supported with adequate analytic backup.

Establish a Program Council Steering Group

6.7 Secondly, PPIAF should establish a small Steering Group composed of Program Council members to provide more frequent and substantive strategic inputs. This Steering Group should review strategies proposed by the PMU (as described above), and oversee the performance of the Work Program.

Refocus Technical Advisory Panel on Strategic Issues

6.8 Thirdly, PPIAF should refocus role of TAP on advising on major strategic issues rather than project-specific evaluations. The TAP has already taken the initiative to move in this direction, but this should be formalized and supported by increasing the proportion of members from beneficiary countries and private sector operators. The project evaluation functions of the TAP should be supported by increased responsibilities of the Program Council Steering Group and the PMU, and by providing a budget for the TAP to commission consultants to assist them to get inputs from beneficiaries.

GREATER STAKEHOLDER PARTICIPATION 6.9 The Program Charter noted, “successful design and execution of PPIAF’s mandate requires a

close understanding of the needs, constraints and priorities of recipient governments and the private sector” (para. 22). This review has confirmed that the acceptance and commitment of beneficiary governments is critical to achieving real and lasting impact (in addition to delivering professionally high quality work). More needs to be done to ensure greater stakeholder participation in the Program in the following ways.

Associating Beneficiary Governments with the Program Council

6.10 The World Bank’s OED Phase 2 Independent Evaluation of the World Bank’s Approach to Global Programs (page 98) calls for an “effective voice for the Bank’s client countries on the governing bodies of global programs to achieve better balance between developed and developing countries.” The PPIAF Program Charter (para. 23) foresaw organizing a Public-Private Sector Infrastructure Forum: this should be activated.

Greater Stakeholder Involvement in Project Execution

6.11 Further steps are needed to institutionalize procedures to ensure stakeholder involvement in projects. These should include: • Greater use of client executed projects, in countries with the capacity for transparent and

efficient procurement procedures. • Requiring TMs to prepare follow up Action Plans and agree them with beneficiaries as a

condition for project completion. • More extensive dissemination (ensuring adequate budgets/time for translation and

distribution of documentation). • Ensuring beneficiary and private sector feedback in project evaluation, perhaps through joint

monitoring of major projects.

FOCUS ON OUTCOMES RATHER THAN OUTPUTS 6.12 Project management activities need to be adapted to maximize the potential benefits of PPIAF-

funded projects through greater attention to project outcomes. This should be done.


Define Operating Guidelines

6.13 The PMU should prepare and introduce Operating Guidelines defining the procedures to be followed in carrying out advisory assignments. These should include inter alia: • The key points to be included in ToRs for TMs, with particular attention to procedures for

consultation, implementation and follow up. • The preparation of project budgets, including requirements for post-delivery consultation and

follow up, and policies on co-financing. • The scope of Poverty assessments that should be included in PPIAF-funded assignments.

Redesign PPIAF’s Reporting Systems

6.14 To help ensure more effective monitoring of project completion, the PMU should: • Review design of the PATS reporting system to highlight critical milestone dates and

intended outcomes (performance indicators), so that delays are automatically brought to the PMU’s attention.

• Develop and apply systematic portfolio ratings. • Redesign Completion Report/Sign-Off Note to clarify a definition of project completion that

distinguishes outcomes, i.e. achieving acceptance and implementation of results from outputs, i.e. delivery of advice/assistance.

Undertake Periodic Portfolio Assessments

6.15 To provide an overall perspective on the effectiveness of the program and to identify the instruments and policies that should guide ongoing and future efforts, the PMU should undertake periodic assessments of the overall portfolio, synthesizing the performance of individual projects. This should incorporate and correlate data from project evaluations reports and the Impact database. Specific post-completion studies, involving direct consultations with beneficiaries, should be undertaken.

ENHANCE PPIAF’S IDENTITY 6.16 As we have noted previously, in many countries, PPIAF-funded projects are not clearly

distinguished from other World Bank technical assistance programs. To ensure that the Program reaches its target beneficiaries and enjoys the full confidence of governments and the private sector, PPIAF needs to raise its profile as an autonomous multi-donor facility. This should be done in several ways: • Expand the PMU’s promotional role through strengthening field representation, and

increasing field visits by HQ staff in other regions. • Maximize opportunities for raising awareness of PPIAF in beneficiary countries through

regional fora privatisation agencies, and local representation of contributing donors. • Extend the pool of potential TMs outside the WB, while maintaining quality standards and

consistency of approach • Developing a coherent a plan for the management and dissemination of PPIAF’s Best

Practice Materials (e.g. Tool Kits), including targeted approaches to decision-makers and regulatory professionals. This should include exploring more extensive partnership arrangements for distance learning programs with the World Bank Institute.

• Review PPIAF’s communications materials and dissemination programs to ensure these reach key targets audiences and project a clear and strong image of the Program.

ORGANIZATIONAL AND BUDGETARY IMPLICATIONS 6.17 These recommendations, most particularly in increasing PMU staffing and field representation,

will have implications for PPIAF’s organization, procedures and program costs. Preliminary estimates, discussed with the PMU, suggest that the one-time increase in administration costs can be contained within 20% of total program expenditures in the short-term. The ratio of


management to total Program Expenditures would be expected to fall again as the volume of activity increases.

6.18 In addition, recruiting and retaining high qualify professional staff also depends on PPIAF being able to offer multi-year employment contracts. It is therefore important that donors continuing to support the Program should provide multi-year funding commitments to cover at least part of the Program budget.


PPIAF STRATEGIC REVIEW

APPENDICES


1

Appendix A

TERMS OF REFERENCE Justification The Public Private Infrastructure Advisory Facility (‘PPIAF’) was established in 1999 as a multi-donor technical assistance facility with a mandate to help developing countries improve the quality of their infrastructure through involvement of the private sector. The World Bank manages the facility on behalf of the participating donors and is itself a participating donor. PPIAF complements the Bank’s Infrastructure Action Program, and also aims to complement infrastructure and country programs of participating donors. PPIAF assists governments at all stages of the process of engaging the private sector from the initial development of an infrastructure strategy, through the implementation of a sound enabling environment, to the execution of transactions. PPIAF also disseminates best practice through the medium of toolkits, workshops, a website and help desk. PPIAF assistance is available to countries in DAC lists I and II (column 1). It operates on a demand driven basis, undertaking all assignments which meet threshold eligibility criteria, subject to donor approval which is given on a no objection basis. The PPIAF Program Management Unit (PMU) administers the program, appointing Task Managers typically from within the Bank to manage projects through consultants. To extend its outreach, PPIAF has established small regional offices in Singapore, Nairobi and Pretoria and is considering setting up further offices in Delhi and West Africa. Direction of PPIAF and its programs is provided through a Program Council, comprising all participating donors, which meets annually. The Program Council is chaired by the Bank Vice President for Infrastructure and Private Sector Development. An independent Technical Advisory Panel (‘TAP’), appointed by the Program Council chair, carries out an evaluation each year of the prior year’s activities and presents its findings to the Program Council. After 4 ½ years of operation, PPIAF has achieved some notable successes. Its procedures are well established and efficient. It has built a solid reputation for good work in its chosen area. Over $80 million worth of projects have been conducted in a great many countries across the full range of activities and sectors. PPIAF donors now number 14. At the same time, private sector investment in infrastructure has since 1997 continued a steady decline, especially in Latin America and East Asia, and in SSA it remains at obstinately low levels. Many private sector investments have become unstuck, often over disputes over tariffs. Some sections of society remain opposed to any private sector involvement in the provision of basic public services. The TAP review of 2003/4, just published, considers the reasons for these trends, and points to the need for a better understanding among protagonists (including PPIAF) of the ‘political economy’ environment in which private sector contracts are negotiated. Seen in this context, the work and mission of PPIAF can be envisaged to stretch out many years ahead. For any country, the duration from embracing the concept of private sector participation to success (investment made and track record established) is likely to take several years. Many countries in SSA have not yet embraced the concept and may be unlikely to do so unless success in other countries is first evident. In 2005/6, several donors are due to renew their financial commitments to PPIAF for a further three year period. These donors wish to evaluate the success of the facility against its original objectives, to reconfirm the validity of the original objectives and to seek ways for improving performance and


2

impact. A review is therefore proposed to be carried out by a consultant in order to address these issues. Output The review should lead to a report, which is no longer than 25 pages (excluding appendices) with an executive summary no longer than 2 pages. The report will:

o Consider the aims and objectives of PPIAF as originally conceived and the success, or otherwise, in achieving these. This should be based on quantitative analysis of proposals received and approved, types of proposals approved, reports received, outputs and overall quality; it should also evaluate dissemination outcomes.

o Analyse (for a small sample of selected countries) the combined country level impact of PPIAF activities in that country, the nature of the obstacles for private sector involvement in that country’s infrastructure and the degree of PPIAF’s success in addressing those obstacles.

o Consider the optimum means by which PPIAF could contribute to the long term goals of private sector involvement in infrastructure development. Achievement of long term goals would be indicated by the client government continuing the path of private sector involvement (viz strategy formulation, creation of enabling environment, capacity building, transaction structuring and utility regulation), with such involvement contributing to poverty alleviation.

o Assess the alternative sources of advice available to governments in the fields of PPIAF’s activities and the relative strengths or otherwise of PPIAF, including its brand name and identity, vis-à-vis other sources.

o Assess whether the costs of PPIAF activities represent good value for money. o Consider the governance structure, management organisation, sourcing of work and

activities, control, reporting and funding of PPIAF and their effectiveness in achieving its aims and objectives.

o Comment on the validity of the original objectives of PPIAF and propose changes (if appropriate).

o Recommend (if appropriate) improvements to PPIAF’s ‘products’, focus, management or modus operandi in order better to achieve the goal of poverty alleviation through private sector involvement in infrastructure.

Input Initially, the review would analyze, by desk research and interviews with PPIAF, a large sample of PPIAF activities covering all infrastructure sectors and each type of PPIAF activity in two regions – SSA and East Asia. Thereafter, the consultants would undertake detailed assessments, through field visits and interviews with client governments and other stakeholders, of a suitable number of PPIAF activities in 3-4 countries in each region, the sample of activities being selected by the consultants. A team of two consultants (one being the Lead Consultant) will carry out the review. It is required that each consultant will have: ▫ At least ten years working experience in the developmental field, including the role of the

private sector in developmental activity, preferably in the infrastructure sector. ▫ At least five years working experience in a developing country, preferably in SSA or East

Asia. ▫ Knowledge of issues pertaining to private sector involvement in developing country

infrastructure, including the enabling and regulatory environment, access of the poor to services, financing and risk constraints.

▫ Previous experience of evaluating multilateral donor programs including the use of quantitative techniques.


3

There will be a single contract with the Lead Consultant or company. The Lead Consultant will report to a Steering Group representing the PPIAF donors. The Steering Group will be co-ordinated by DFID.

Timing

June 9 TORs sent to prospective consultants June 18 Receipt of consultants' proposals June 25 Proposals circulated to SG with recommendation July 2 Approval (no objection) of SG Mid July Contract signature Late July Consultant meets PPIAF Sep-Oct Field visits Mid-Oct Consultant review meeting with PPIAF Oct 31 Draft report circulated to SG Mid-Nov SG meet to consider report and follow-up action Nov 30 Final report submitted

June 8, 2004


4

Appendix B

LIST OF PEOPLE INTERVIEWED Organization Name Position World Bank Katherine Sierra Vice President, Infrastructure (Chair of PPIAF Program

Council) Michael Klein Vice President & Chief Economist, IFC Hossein Razavi Director, Infrastructure Economics & Finance Department

(World Bank representative on Program Council) Danny Leipziger Director, Latin America & Caribbean Thomas Davenport Manager, FIAS Chris Gerrard Operations Evaluation Department John Hodges Infrastructure Economics & Finance Department Paul Hubbard Manager, Development Grant Facility Jose Luis Irigoyen Head, Infrastructure Sector, Latin America Timothy Irwin Senior Economist, Private Participation in Infrastructure Zhi Liu Infrastructure Coordinator, Bangkok Anwar Ravat Chief Administrative Officer, PSI Suzanne Smith Rapid Response Unit David de Groot Acting Country Director, Pretoria Loletter Krizenger Senior Economist, Pretoria Gaiv Tata Senior Manager, Private Sector Department Africa Region Makhtar Diop Director, Nairobi PPIAF Jyoti Shukla Program Manager Michael Schur Deputy Program Manager and Coordinator for Global

Programs C.y Ollero Regional Co-ordinator, Europe & Central Asia, Middle East

and North Africa Ibrahima Diong Regional Co-ordinator, Africa Jean-Christophe Ngo Regional Co-ordinator, Latin America Stella Franco Resource Management Assistant Shelly Hahn Analyst Warrick Smith Former Program Manager Ian Alexander Former Regional Coordinator Apurva Sanghi Former Regional Coordinator Tony Gomez-Ibanez Member, Technical Advisory Panel DFID Charles Lutyens Jeff Delmon Private Sector Adviser, Policy Division Simon Lucas Infrastructure Adviser, DFID Vietnam Robert Blakelock Engineering Adviser, DFID Pretoria Simon Bland DFID, Nairobi Asian Development Bank

Rita Nangia Director, Finance & Infrastructure Division

SIDA Mikael Söderbäck Chief Policy Coordinator, Infrastructure Jan Engström Senior Adviser, Economic Co-operation Rolf Eriksson Financing Specialist, Infrastructure Division UNDP Malaye Diop Global Manager, PPPUE Fund Other Tony Clamp Head of Project Finance, Thames Water Ken Caplan Head, Building Partnerships for Development in Water &

Sanitation (BPD)


5

Project Task Managers/

World Bank Beneficiaries Other

AFRICA AFUR Regulators Network

Lorenzo Bertolini (TM) Smundo Mokoena, Chairman Afur

SADC Telecommunications Conference

Peter Smith (TM) and Laurent Besancon

A. Dlamini, SADC Secretariat

ECOWAS Telecomms. Conference

Laurent Besancon for Mavis Ampah (TM)

Ethiopia PSP in Water Ytbarak Tessema for

Catherine Revels (TM) Ato Guililiat Dirham, Head of Planning, Ministry of Water

PPI Road Workshop John Riverson, Lead Highway Engineer

Ato Zaid Woldegabriel, General Manager, Ethiopia Roads Authority

PPP Urban Transport Ajay Kumar (TM) Ato Tesfamichael Nahusenay, City Manager, Addis Ababa

Solid Waste Reform Ytbarak Tessema for Catherine Revels (TM)

Ato Desalegn, Deputy General Manager, Environmental Protection Agency

Kenya Water Options Fook Chuan Eng (TM) George Krhoda, Permanent

Secretary, Ministry of Water Resources

Railways Privatization Anil Bandari (TM) Solomon Kitungu, Director Reforms, Ministry of Finance

Power Sector Restructuring

Joel Maweni (TM) P, Nyoike, Permanent Secretary, Ministry of Power

PSP in Airports Anil Bandari (TM) George Muhuho, Managing Director, Kenya Airports Authority

Road Concession Study

Anil Bandari (TM) Solomon Kitungu Director Reforms Ministry of Finance

Zambia Lusaka Water Options Jane Walker (TM) Henry Mtini

Dr. Engineering LWSC M. Chipwenda, and D. Cruikshank, Zambia Privatisation Agency

NAWASCO Capacity Building

Constantine Chikosi (TM)

Oswrd Chanda, Director NAWASCO

Water Stakeholders Workshop

Jane Walker (TM) Henry Mtini Dr. Engineering LWSC

Nelson Chisenga, Zambia Association of Chambers of Commerce

PSP Solid Waste Management

Jane Walker (TM) Peter Lumambo, Director of Infrastructure, Ministry of Local Government


6

EAST ASIA EAP Regulators Network

Abha Joshi-Ghani (TM) Ha Huy Tuan Deputy Director International Cooperation, Ministry of Finance, Vietnam

Hugh Railton, Deputy Executive Director, Asia-Pacific Telecommunity

Cambodia Country Framework Report

Jordan Schwartz (TM) Nisha Agrawal, Country Manager

Suy Sem Minister of Industry, Mines and Energy

James Brew, Projects Officer, IFC

Private Power Policy Framework

Tun Lean Acting Director General of Energy, Ministry of Industry, Mines & Energy

Framework for PPIs Michael Schur (TM) Output based Aid for Rural Water

Jordan Schwartz (TM) Meng Saktheara, Deputy Director, MIME

Philippines Country Framework Report

Ming Zhang Infrastructure Coordinator, Philippines

Librado Quitoriano Director Infrastructure, National Economic Development Authority

Privatization of Rural Telecommunications

Peter Smith (TM) Agustin Bengzon Undersecretary, Department of Transport and Communications

Epitacio Marquez Chairman, Philippine Telephone Operators Association

Water & Sewerage Legislation

Rey Ancheta Lulu Baclagon Deputy Executive Director, BOT Center

Albert Magalang Executive Director, National SWM Commission

Privatization of Manila SWM

Richard Ondrik Chief Country Officer ADB Chris Hoban, Operations Manager, World Bank Manila office

Thailand Capacity Building for National Telecommunications Commission

Peter Smith (TM) Totsaporn Getuadisorn, Act. Director-General, PTD

Prof. Sutham Yoonaitham, Nominee Member of NTC

Restructuring of State Railways

Nakorn Chanrasorn, Director SRT

Radio Frequency Management

Peter Smith (TM)

Rural Water Trevor Bull State Enterprise Policy Office, Ministry of Finance


7

Vietnam Country Framework Report

Phil Gray Senior Energy Economist

Simon Lucas, Infrastructure Advisor, DFID

Regulatory Framework for Gas Sector

Pham Nguyet Anh Operations Officer

Bui Ngoc Hien Deputy Director Ministry of Planning & Investment

Corporate Rural Electrification

Pham Nguyet Anh Operations Officer

Negotiations for Phu My 2 Power Project

Suman Babbar, Senior Advisor

Nguyen Manh Hung Deputy Director Ministry of Industry


8

Appendix C

SOURCES OF APPLICATIONS FOR PPIAF FUNDING FY 2000-2004

By year of submission

Number 2000 2001 2002 2003 2004 Total

2000-04 % of totalAfrica 35 44 43 26 28 176 29% EA & Pacific 23 15 31 20 18 107 18% EE & CA 15 21 14 14 17 81 13% Global 18 12 6 10 10 56 9% Latin America 23 31 10 11 16 91 15% ME & NA 7 7 2 4 11 31 5% South Asia 22 12 16 8 6 64 11% Grand Total 143 142 122 93 106 606 100%

Budget (US$ million) Africa 9.1 24.5 10.9 6.3 4.8 55.6 35% EA & Pacific 9.0 5.5 8.2 3.8 4.3 30.8 19% EE & CA 3.0 3.9 4.2 3.5 4.0 18.6 12% Global 3.4 3.9 0.8 2.1 2.6 12.8 8% Latin America 5.6 5.1 2.6 2.4 3.6 19.3 12% ME & NA 1.8 1.1 1.4 0.4 1.5 6.3 4% South Asia 6.8 2.5 3.1 2.4 1.6 16.4 10%

Grand Total 38.7 46.5 31.2 20.8 22.4 159.7 100%

Source: PPIAF Applications Data base


9

Appendix D

TREND IN APPROVED PROJECTS BY REGION By year of approval

Number 2000 2001 2002 2003 2004 Total % total

Africa 15 21 22 19 15 92 28% EA & Pacific 11 11 17 8 16 63 19% EE & CA 7 11 6 8 8 40 12% Global 11 9 3 7 6 36 11% Latin America 14 12 3 7 9 45 14% ME & NA 3 3 0 3 6 15 5% South Asia 13 7 8 8 3 39 12%

Total 74 74 59 60 63 330 100% Budgets (US$ million)

Africa 3.54 6.62 5.21 4.33 3.15 22.85 30% EA & Pacific 2.64 3.26 3.58 1.59 3.80 14.87 20% EE & CA 1.21 3.01 2.21 2.76 1.61 10.79 14% Global 2.16 1.89 0.14 1.36 1.57 7.12 9% Latin America 3.20 1.67 0.86 1.60 2.14 9.47 13% ME & NA 0.21 0.70 0.00 0.27 0.81 1.99 3% South Asia 2.65 1.54 1.24 2.40 1.15 8.97 12%

Total 15.61 18.69 13.24 14.31 14.23 76.06 100% Source: PPIAF Applications Data base


10

Appendix E

TREND IN APPROVED PROJECTS BY ACTIVITY Number by FY

2000 2001 2002 2003 2004 Total 00-04

Global Best Practice 11 9 4 5 7 36

Capacity Building 7 11 10 12 14 54

Consensus Building 7 5 4 7 9 32

Pioneering Transactions 4 2 6 Policy, Institutional & Regulatory Reform 24 29 24 16 17 110 Infrastructure Development Strategies 24 20 17 16 14 91

Other 1 1

Total 74 74 59 60 63 330

Source: PPIAF Applications Data base


11

Appendix F

PERFORMANCE RATING FOR PROJECTS STUDIED Project Activity Size Product Satisfaction Outcome Overall

FY Approved Note 1 Note 2 Note 3 Note 4

AFUR 1st Annual Meeting CB M 2003 2 3 2 3 ECOWAS Telecomm. PRI M 2001 2 2 1 1 Ethiopia PPI in Roads Workshop CS S 2002 2 2 2 2 Ethiopia PPP Urban Transport PRI L 2003 2 N N 1 Ethiopia PSP in Water Sector CS S 2002 2 2 1 2 Ethiopia Solid Waste Reform PRI L 2002 1 1 2 2 Kenya Nairobi Water Options IDS L 2000 2 1 1 1 Kenya Power Sector Restructuring IDS L 2001 2 1 2 1 Kenya PSP in Airports IDS L 2003 N N N 1 Kenya Railways Privatization PRI L 2002 2 2 3 3 Kenya Road Concession Study IDS L 2000 2 2 2 2 KH Country Framework Report IDS L 2000 3 3 2 3 KH OBA for Water PRI S 2002 2 3 N 3 KH PPI Framework PRI L 2003 3 2 N 2 KH Private Power Policy PRI L 2002 1 2 N 1 PH Country Framework Report IDS M 1999 2 2 2 2 PH Manila SWM CB S 2002 2 N N 1 PH Private Rural Telecomm IDS S 2003 3 2 0 1 PH Water Legislation PRI M 2000 2 2 N 1 SADC Telecomm Conference CNF M 2002 2 1 1 1 TH Capacity Building for STC PRI L 2000 3 2 N 1 TH Radio Frequency Management CB M 2001 2 3 2 2 TH Restructuring Railways IDS L 2001 2 2 N 1 TH Rural Water IDS M 2002 1 2 N 2 VN Country Framework Report IDS M 1999 2 2 2 2 VN Phu My 2 Negotiations PT S 2001 3 3 3 3 VN Regulation of Downstream Gas PRI L 2001 2 2 N 2 VN Rural Electrification PRI S 2000 3 2 2 2 Zambia Lusaka Water PSP Options IDS L 2001 2 1 2 2 Zambia NAWASCO Capacity Build. CB L 2001 2 1 2 2 Zambia PSP Solid Waste Manage. IDS L 2004 N N N 2 Zambia Water Stakeholders Wkshp CS M 2002 2 2 2 2

Performance was assessed based on following questions:

1. Was product effectively delivered? 2. Was the client satisfied with the results? 3. Were expected outcomes achieved? 4. Have objectives of assignment been achieved?

Ratings were assessed

3 Fully satisfactory 2 Mostly satisfactory 1 Partly satisfactory 0 Unsatisfactory N Insufficient data or too early to assess.


12

Appendix G

DISTRIBUTION OF PROJECTS WITH RECORDED OUTCOMES IN THE IMPACT DATABASE Detail analysis of actual and expected outcomes

# countries

% of PPIAF country clients

# countries with at least 1 recorded outcome 45 52%# countries with Strategies adopted 15 17%# countries with institutions created/strengthened 23 26%# countries with Law/licenses passed 27 31%# countries with transactions assisted (including concession 28 32%and management contracts)# countries that are PPIAF clients 87 100%NB1: regional projects have not been consideredNB2: Some projects have more than 1 impact

Country

Strategies adopted

Institution created/

strengthened

Laws passedTransac-

tions assisted

Total

Total projects with

at least 1 impact

Afghanistan 0 1 1 0 2 1Algeria 1 1 3 2 7 4Argentina 0 0 0 1 1 1Armenia 0 1 1 0 2 1Azerbaijan 1 1 1 2 5 3Bangladesh 1 1 0 1 3 2Bolivia 0 0 1 0 1 1Brazil 0 0 1 1 2 2Bulgaria 3 1 2 1 7 4Cambodia 2 1 1 1 5 3China 2 0 1 0 3 2Colombia 0 0 0 1 1 1Congo 0 0 0 1 1 1Croatia 0 1 1 1 3 1Dominica 0 0 1 0 1 1El Salvador 0 0 0 1 1 1Gambia 0 1 0 0 1 1Ghana 0 0 1 1 2 1Guyana 0 1 1 1 3 1India 2 0 0 0 2 2Kenya 2 1 0 3 6 6Kosovo 0 0 1 1 2 1Laos 1 2 0 0 3 2Latvia 0 1 0 0 1 1Malawi 1 1 1 1 4 3Morocco 0 0 0 1 1 1Mozambique 1 0 0 0 1 1Nepal 0 0 1 0 1 1Nigeria 0 1 1 2 4 3Paraguay 0 1 2 3 6 3Peru 1 1 1 1 4 3Philippinnes 0 1 1 1 3 2Serbia 0 0 0 1 1 1Slovakia 0 1 1 0 2 1South Africa 1 0 0 0 1 1Sri Lanka 0 0 2 0 2 2Tajikistan 0 0 0 1 1 1Tanzania 0 0 1 1 2 2Thailandia 0 1 3 0 4 3Turkey 0 1 1 0 2 1Uganda 1 1 1 1 4 2Ukraine 1 0 0 0 1 1Uzbekistan 0 0 0 1 1 1Vietnam 0 0 1 3 4 4Zambia 0 1 0 1 2 2Total 22 31 34 38 125 91

Detail analysis of expected outcomes

# countries

% of PPIAF country clients

# countries with at least 1 completed outcome 34 39%# countries with completed Strategies adopted 10 11%# countries with comleted institutions created/strengthened 17 20%# countries with completed Law/licenses passed 11 13%# countries with transactions assisted (including concession 11 13%and management contracts)# countries that are PPIAF clients 87 100%

Country

Strategies adopted

Institution created/

strengthened

Laws passedTransac-

tions assisted

Total

Total projects

with at least 1 impact

completed

Afghanistan 0 0 0 0 0 0Algeria 1 1 2 1 5 3Argentina 0 0 0 0 0 0Armenia 0 1 1 0 2 1Azerbaijan 0 0 0 1 1 1Bangladesh 0 1 0 0 1 1Bolivia 0 0 1 0 1 1Brazil 0 0 0 1 1 1Bulgaria 1 1 2 0 4 3Cambodia 1 0 0 0 1 1China 1 0 1 0 2 1Colombia 0 0 0 0 0 0Congo 0 0 0 0 0 0Croatia 0 0 1 0 1 1Dominica 0 0 0 0 0 0El Salvador 0 0 0 0 0 0Gambia 0 1 0 0 1 1Ghana 0 0 1 1 2 1Guyana 0 1 0 1 2 1India 1 0 0 0 1 1Kenya 0 1 0 0 1 1Kosovo 0 0 0 1 1 1Laos 0 1 0 0 1 1Latvia 0 1 0 0 1 1Malawi 1 1 0 0 2 2Morocco 0 0 0 1 1 1Mozambique 1 0 0 0 1 1Nepal 0 0 0 0 0 0Nigeria 0 0 0 0 0 0Paraguay 0 0 1 0 1 1Peru 1 1 1 0 3 3Philippinnes 0 1 0 0 1 1Serbia 0 0 0 0 0 0Slovakia 0 1 0 0 1 1South Africa 0 0 0 0 0 0Sri Lanka 0 0 0 0 0 0Tajikistan 0 0 0 1 1 1Tanzania 0 0 0 1 1 1Thailandia 0 1 1 0 2 2Turkey 0 1 0 0 1 1Uganda 1 1 1 0 3 1Ukraine 1 0 0 0 1 1Uzbekistan 0 0 0 1 1 1Vietnam 0 0 0 1 1 1Zambia 0 1 0 0 1 1Total 11 24 13 11 59 50


13

Appendix H

TREND IN PROGRAM AND MANAGEMENT EXPENDITURES

US$'000 2000 2001 2002 2003 2004 Program Activities

Consultant fees 3,133 5,307 7,619 9,969 11,420 Travel 332 664 576 805 914 Staff costs 221 743 1,066 1,068 1,147 Other 161 151 76 216 53

Sub-total 3,847 6,865 9,337 12,058 13,534 Program Management

PMU core admin 904 847 813 947 966 Technical assessments 25 65 26 27 16 Consultant fees 171 146 253 195 74 Travel 116 229 179 169 120 Other 247 193 204 119 76 Regional co-ordination offices 45 973 1,126 950 827

Sub-total 1,508 2,453 2,601 2,407 2,079 Total 5,355 9,318 11,938 14,465 15,613

Cumulative total 5,355 14,673 26,611 41,076 56,689 Management % total 28% 26% 22% 17% 13%


14

Appendix I

Program Manager

Program Assistant

Information Assistant

Team Assistant

Field Based Field Based

Analyst Operations Analyst

Regional Coordinator Southern Africa Eastern,

rest of SubSaharan Africa

Resource Management Analyst

Deputy Program Manager

ORGANIZATIONAL CHART OF THE PROGRAM MANAGEMENT UNIT

Regional Coordinator East Asia and the Pacific

South Asia

Regional Coordinator Europe and Central Asia

Middle East and Nothern Africa

Regional Coordinator Latin America and the

Caribbean


15

Appendix J

PROCESSING OF APPLICATIONS by year of approval

Number of Proposals 2000 2001 2002 2003 2004 Applications received 143 142 122 93 106 - Less applications withdrawn 26 35 23 19 23 =Applications reviewed 117 107 99 74 83 -Rejections 31 39 41 15 16 Applications from previous fiscal year 12 6 5 4 Applications moved to next fiscal year 12 6 5 4 8 =Applications approved 74 74 59 60 63 -Cancellations 4 3 1 1 0 -Suspensions 0 0 2 0 0 Projects Executed 70 71 56 59 63 Ratios Rejected/Reviewed 26% 36% 41% 20% 19% Executed/Received 49% 50% 46% 63% 59%

Source: Applications Tracking Data


16

Appendix K

DONOR CONTRIBUTIONS TO PPIAF

US$000 as of 30 June 2004

Donor Core Funds

Non-Core Funds

Total

Asian Development Bank 750 750 Canada 1,310 1,310 France 1,016 1,016 Germany 1,058 1,058 Italy 250 250 Japan 7,435 2,608 10,043 Netherlands 1,500 1,500 Norway 1,650 1,650 Sweden 1,149 4,746 5,895 Switzerland 3,173 3,082 6,255 UK-DFID 29,874 17,635 47,509 USAID 500 500 World Bank 12,930 12,930

Total Funds Received 62,595 28,071 90,666


17

Appendix L

PRELIMINARY SCHEMATIC APPROACH FOR COUNTRY PRIORITIES Group Characteristics Priorities Illustrative

Distribution of Effort

1. Advanced countries, e.g Malaysia, Chile

• Political Stability H • PPI Record H • Commitment H • Capacity H

Limit interventions to learning opportunities that can enhance best practice experience and role

5%

2. Progressive countries, e.g. South Africa

• Political Stability H • PPI Record M • Commitment H • Capacity M

Ongoing project-based interventions in high priority sectors

30%

3. Reforming countries, e.g. Vietnam

• Political StabilityH/M • PPI Record M/L • Commitment H • Capacity H

Multi-year, diversified range of interventions

50%

4. Turnaround countries, e.g.Sierra Leone

• Political Stability L • PPI Record L • Commitment H/M • Capacity M/L

Selective opportunistic project support in response to credible client demand

10%

5. Non-performers, e.g. Zimbabwe

• Political Stability L • PPI Record L • Commitment L • Capacity L

Watching brief, plus dissemination of best practice materials

5%

Approval of funding for specific projects should remain conditional on evidence of commitment from designated beneficiaries.