evaluation of the bank's country strategy and...

An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

Zambia: Evaluation of the Bank's Country

Strategy and Program 2002–2015Summary Report

October 2016

IDEV conducts different types of evaluations to achieve its

strategic objectives

Thematic Evaluations Project Cluster Evaluations

Regional Integration Stra

tegy

Evaluations

Project Perfo

rmance Evaluations

(Public Secto

r)Impact Evaluations

Project Performance Evaluations

(Private Sector)

Coun

try S

trate

gy E

valu

atio

n

Evaluation Syntheses

Corporate Evaluations

Sect

or E

valu

atio

ns

Coun

try

Stra

tegy

Eva

luat

ions

An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

Zambia: Evaluation of the Bank's Country

Strategy and Program 2002–2015Summary Report

October 2016

© 2016 African Development Bank Group All rights reserved – Published October 2016

Zambia: Evaluation of the Bank's Country Strategy and Program 2002–2015 - Summary Report IDEV Country Strategy Evaluation, October 2016

Disclaimer

Unless expressly stated otherwise, the findings, interpretations and conclusions expressed in this publication are those of the various authors of the publication and are not necessarily those of the Management of the African Development Bank (the “Bank”) and the African Development Fund (the “Fund”), Boards of Directors, Boards of Governors or the countries they represent.

Use of this publication is at the reader’s sole risk. The content of this publication is provided without warranty of any kind, either express or implied, including without limitation warranties of merchantability, fitness for a particular purpose, and non- infringement of third-party rights. The Bank specifically does not make any warranties or representations as to the accuracy, completeness, reliability or current validity of any information contained in the publication. Under no circumstances including, but not limited to, negligence, shall the Bank be liable for any loss, damage, liability or expense incurred or suffered which is claimed to result directly or indirectly from use of this publication or reliance on its content.

This publication may contain advice, opinions, and statements of various information and content providers. The Bank does not represent or endorse the accuracy, completeness, reliability or current validity of any advice, opinion, statement or other information provided by any information or content provider or other person or entity. Reliance upon any such opinion, advice, statement, or other information shall also be at the reader’s own risk.

About the AfDB

The overarching objective of the African Development Bank Group is to spur sustainable economic development and social progress in its regional member countries (RMCs), thus contributing to poverty reduction. The Bank Group achieves this objective by mobilizing and allocating resources for investment in RMCs and providing policy advice and technical assistance to support development efforts.

About Independent Development Evaluation (IDEV)

The mission of Independent Development Evaluation at the AfDB is to enhance the development effectiveness of the institution in its regional member countries through independent and instrumental evaluations and partnerships for sharing knowledge.

Independent Development Evaluation (IDEV)African Development Bank GroupAfDB HeadquartersAvenue Joseph Anoma, 01 BP 1387, Abidjan 01, Côte d’IvoirePhone: +225 20 26 20 41E-mail: [email protected]

Design & layout: CRÉON – www.creondesign.net Original language: English – Translation: AfDB Language Services Department

ACKNOWLEDGMENTS

Task manager Madhusoodhanan Mampuzhasseril, Principal Evaluation Officer

Team members Erika Maclaughlin, Long Term Consultant, Foday Turay, Chief Evaluation Officer, Latefa Camara, Long Term Consultant, James Sackey, Consultant

Consultant Agrer S.a. - N.v., Belgium. Paolo Liebl Von Schirach, Team Leader. Members: Baptiste Forquy, Habtom Asmelash, Bernd Drechsler, John Murphy, Charles Haanyika, Vikramdityasing Bissoonauthsing

External peer reviewer Bruce Murray, former Director General, Operations Evaluation Department, Asian Development Bank

Knowledge management officers Jayne Musumba, Principal Knowledge Management OfficerJerry Lemogo, Junior Consultant, Communications and Knowledge Management

Special thanks to Norad – The Norwegian Agency for Development Cooperation

Division manager Samer Hachem

Evaluator-General Rakesh Nangia

Acknowledgments iiAbbreviations and Acronyms vExecutive Summary 1Management Response 9

Introduction 22

Background 23Economic Context 23Social Context 24Development Challenges 24

Overview of the Bank's Country Strategies and Portfolio 26Overview of Strategic Priorities 26Project Portfolio 26

Evaluation Approach and Methodology 29Evaluation Issues and Questions 29Methodology and Lines of Evidence 29

Evaluation Findings 30Relevance 30Effectiveness 32Efficiency 45Sustainability 49Crosscutting Themes 52Quality at Entry 55Supervision 58Leveraging and Co-financing 59

Conclusions and Recommendations 61Annexes 67

Contents

Contents

List of FiguresFigure 1 Number of operations and amount approved (2002–2015) 27Figure 2 Number of operations and amount for sectors (2002–2015) 27Figure 3 Zambia Corruption Perceptions Index scores (2007–2015) 37Figure 4 Zambia CPIA Scores - Accountability Transparency and Corruption (2007–2015) 37Figure 5 Ease of Doing Business Indicators (number of days) 38Figure 6 No. of new businesses registered during 2007-2015 42Figure 7 Domestic credit to the private sector (percent GDP) 2006-2014 42Figure 8 Access to improved water and sanitation source 44Figure 9 Disbursement Ratios – Zambia and AfDB (2008-2014) 48

List of TablesTable 1 Relevance rating 30Table 2 Effectiveness rating 32Table 3 PEFA Scores - Procurement and External Audit 36Table 4 Progress on key PEFA indicators 39Table 5 Planned and Delivered Knowledge Work 40Table 6 Progress against selected macroeconomic indicators 43Table 7 Progress on selected health and education indicators (2002-2014) 45Table 8 Relevance rating 46Table 9 Sustainability rating 49Table 10 Regional Tariff Levels 51Table 11 Crosscutting Issue Rating 53Table 12 Operational selectivity in Zambia CSPs 56

vAbbreviations and Acronyms

An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

Abbreviations and Acronyms

ADB African Development Bank

ADF African Development Fund

CEDR Comprehensive Evaluation of the Bank's Development Results

CETZAM Christian Enterprise Trust of Zambia

CPIA Country Policy and Institutional Assessment

CSP Country Strategy Paper

CSPCR CSP Completion Report

CWMIP Community Water Management Improvement Project

DAC Development Assistance Committee

DAPP Development Assistance from Person to Person

DO Development Objective

EPC Engineering, Procurement and Construction

ERB Energy Regulation Board

ESMP Environmental and Social Management Plan

FI Financial Intermediary

FNDP Fifth National Development Plan

GNP Gross National Product

GDP Gross Domestic Product

GRZ Government of Republic of Zambia

HIPC Highly Indebted Poor Countries

HRH Human Resources for Health

ICT Information Communication Technology

IFMIS Integrated Financial Management Information System

IMF International Monetary Fund

ITPC Itezhi-Tezhi Power Corporation

JASZ Joint Assistance Strategy for Zambia

JICA Japanese International Cooperation Agency

MoFNP Ministry of Finance and National Planning (Zambia)

MPSAs Ministry, Provinces and Spending Agencies

MSME Micro Small and Medium Enterprises

NRWSSP National Rural Water Supply and Sanitation Project

OAG Office of the Auditor General

OECD Organization for Economic Cooperation and Development

PAF Performance Assessment Framework

PCGF Partial Credit Guarantee Facility

PEFA Public Expenditure and Financial Accountability

PEMFA Public Expenditure Management and Financial Accountability

PPA Power Purchasing Agreement

PPP Public-Private Partnership

PRBS Poverty Reduction Budget Support

PRODAP Project to Support Lake Tanganyika Integrated Regional Development Program

RDA Roads Development Agency

SIP Small Scale Irrigation Project

SME Small and Medium Enterprises

SNDP Sixth National Development Plan

vi Zambia: Evaluation of the Bank's Country Strategy and Program 2002–2015 - Summary Report

TA Technical Assistance

TSA Treasury Single Account

UA Unit of Account (of African Development Bank)

USD US Dollar

WHO World Health Organization

WSS Water Supply and Sanitation

XSR Extended Supervision Report

ZANACO Zambia National Commercial Bank Plc.

ZCCM Zambia Consolidated Copper Mines

ZESCO Zambia Electricity Supply Corporation

ZMFO Zambia Field Office (of the African Development Bank)

ZMW Zambian Kwacha

ZPPA Zambia Public Procurement Agency

1Executive Summary

An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

This report presents a summary of findings, conclusions and recommendations from an evaluation of the Bank's Country Strategies and Program in Zambia over the period 2002–2015. The evaluation was conducted to support the preparation of the Zambia Country Strategy 2016–2020 and contribute to IDEV's comprehensive evaluation of the Bank's development results (CEDR). The evaluation seeks to assess the relevance and performance of the Bank's strategic interventions in Zambia and identify findings, conclusions and recommendations to inform strategies and operations going forward.

Zambia Country Strategies. The evaluation covers three Country Strategy Papers (CSPs) that is, the 2002–2004 CSP, extended to 2006; the 2007–2010 Joint Assistance Strategy for Zambia (JASZ); and the 2011–2015 CSP. The 2002-2004 strategy emphasized agricultural development, access to water supply and sanitation and the promotion of child welfare. Under the JASZ, strategic emphasis on infrastructure development to support the agricultural sector was retained in addition to the promotion of accountability and transparency in the management of public resources through general budget support. The 2011–2015 CSP signaled a shift in the Bank's priorities, with infrastructure development now placing greater emphasis on regional integration, particularly with regard to transport and power infrastructure. In fact, the Bank's policy-based operations have evolved to target private sector regulatory reforms aimed at increasing access to finance.

Portfolio. During the 2002–2015 period, the Bank approved a total of 43 operations amounting to UA 947.84 million net of cancellations. The portfolio of projects under review comprises 34 projects,

four studies, three emergency operations and two technical assistance grants. In addition to these national operations, there were 19 multinational operations amounting to UA 373.25 million where Zambia was involved in varying degrees. The evaluation focused on national projects and examined regional projects only where there was a tangible national component. Over the years, the portfolio has grown in terms of numbers and the amount committed. The majority of the Bank's support by value has been in the transport, power, WSS and agriculture sectors (31.7 percent, 19.3 percent, 14.3 percent and 10.9 percent respectively).

Evaluation Findings

Relevance

Relevance was assessed as satisfactory. The Bank's CSPs were aligned with Zambia's national development plans. The Bank’s operational priorities reflected national development priorities and the Bank's comparative advantages. This alignment is evident from the general budget support to address financial and budgetary transparency and accountability as identified in the 2005–2006 CSP update. The 2007 JASZ reflected the GRZ's emphasis on good governance as per the Fifth National Development Plan (FNDP). The 2011–2015 CSP's emphasis on the development of economic infrastructure, including domestic and regional transport and power infrastructure, reflected the objectives of the Sixth National Development Plan (SNDP). Interventions targeting improved economic and financial governance reflect the priorities of Zambia's Vision 2030 and the Revised SNDP.

Executive Summary

2 Zambia: Evaluation of the Bank's Country Strategy and Program 2002–2015 - Summary Report

The alignment of projects with Country Strategies was rated satisfactory overall. Alignment of the private sector operations, despite targeting job creation and economic development under the 2002–2004 CSP and the 2007–2010 JASZ, were constrained by the practice of identifying operations ‘opportunistically.’ However, private sector operations were better aligned under the 2011–2015 CSP. Social sector projects, on the other hand, did not always reflect the CSP priorities. Although the 2002–2004 CSP targeted increased access to basic services this objective was expressed only in terms of water supply and sanitation infrastructure rather than health and education. Although support to vocational training and skills development was not explicitly mentioned as an operational priority in the 2011–2015 CSP, the Support to Science and Technology Education Project was identified in the CSP Logframe as a means of addressing skills gaps and reducing barriers to private sector development.

With a few exceptions, the alignment of projects with the needs of beneficiaries was rated satisfactory. While projects in the multi, power, social and transport sectors have addressed alleviation of development constraints at the national or regional level, projects in the agriculture and WSS sectors targeted the needs of specific beneficiary groups. Private sector projects were well-aligned with the needs of targeted companies and financial intermediaries but did not always fully address the needs of SMEs.

Effectiveness

Overall progress is rated as moderately unsatisfactory. The achievement of project outputs across sectors was moderately satisfactory, with 78 percent of outputs delivered successfully. However, achievement of project outcomes was moderately unsatisfactory, with progress demonstrated only toward 67 percent of outcomes.

All the ongoing transport projects have faced considerable implementation delays for the delivery

of outputs. About 50 percent of the physical works were completed for the Nacala Corridor project by the end of September 2015, but 95 percent of the original project timeframe had elapsed. The Kazungula Bridge project was delayed due to procurement issues. At the end of the evaluation period, just 0.98 percent of the project funds had been disbursed, 41 months after the approval of the project. Nonetheless, these projects will most likely achieve their intended outcomes.

Overall, 80 percent of power sector projects achieved planned outputs. The planned Cost of Service Study, which would have informed the power sector reform, has not progressed due to an unsuccessful procurement process. Outcomes achieved include: a) additional electricity generation capacity of 120 MW; b) job creation for members of the local community although this tends to be temporary given the skill requirements required for more permanent jobs; and c) increased access to basic infrastructure, thereby promoting private sector development.

Delivery of outputs of the WSS projects was rated satisfactory at 90 percent across projects. The outcome level achievements include: a) improved access to water sources for over 1,500,000 people; b) increased hours of service from five hours per day to 16 hours per day for the Nkana WSS project and 22 hours for 8 Centers WSS project; and c) reduction in waterborne diseases with the Nkana and 8 Centers project areas reporting a 43 percent and 48 percent decrease in diarrhea and dysentery respectively, while the interventions under the NRWSSP reduced the incidence of diarrhea from 50 percent to 35 percent across the Northern and Luapula provinces.

The majority of the Bank's agricultural operations did not fully deliver the planned outputs and only moderate progress was made toward expected outcomes. However, some progress has been made towards increased incomes for farmers, access to markets and increased use of improved irrigation methods.

3Executive Summary

An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

The PRBS program has a large social component in addition to reforms targeting PFM and private sector regulatory reform. Delivery of outputs for the PRBS was satisfactory in terms of the implementation of identified triggers and reforms. However this was less the case for outputs related to PFM, which included the implementation of IFMIS and the Treasury Single Account (TSA). With respect to outcomes, progress was achieved towards strengthening accountability and transparency functions with the implementation of a National Anti-Corruption Plan and increase in coverage of external audit. The Zambia Public Procurement Agency (ZPPA) was empowered with oversight functions but there are still challenges because of the absence of an impartial complaints redressal system and irregularities resulting from inadequate understanding of guidelines. Private sector regulatory reforms considerably reduced the cost of doing business. There has been improvement in the access to health and education with an increase in supervised childbirths and improvements in teacher student ratios. Nonetheless, limited progress was achieved in terms of strengthening budget credibility and execution with reported irregularities in commitments and poor enforcement of regulations.

Finally, about 87 percent of the planned private sector project outputs was delivered. The achievement of outcomes presents a mixed picture but is largely positive. Positive outcomes were achieved in the area of government revenues, job creation, and access to basic services and infrastructure. Achievements were less than expected in the areas of profitability of companies supported by the Bank (for example Lumwana Mine), and improvement in the terms of finance.

After the establishment of ZMFO, the Bank has been participating effectively in policy dialogue with the GRZ. Since 2006, the Bank has been an active participant in the donor coordination efforts in Zambia, leading the Cooperating Partners Group troika in 2012, as well as leading the agriculture, transport and WSS working groups at various points

over the 2011–2015 period. The Bank is also a prominent dialogue partner for the Performance Assessment Framework (PAF) working group under the PRBS; it possesses a privileged relationship with the GRZ regarding policy dialogue.

Delivery of knowledge work has intensified over the evaluation period, and ZMFO has used studies to inform and support policy dialogue, engaging in a high level policy dialogue on youth employment in 2012 and hosting a seminar on Jobs and Growth in 2014. Knowledge work in the form of studies has also been used to add value to the Bank's operations.

At the broader country strategy level, the Bank's interventions made a tangible contribution towards developing an enabling business environment and increasing access to basic infrastructure and services. However, limited progress was achieved in strengthening public financial management and promoting agricultural productivity and diversity.

The Bank's interventions in the WSS, power and social sectors have made a tangible contribution to increasing access to basic services. National trends for access to improved water and sanitation sources have been positive with the percentage of Zambians having access to an improved water source climbing from 55 percent in 2002 to 65 percent in 2014. In contrast, limited improvement has been seen in access to improved sanitation. The Bank's completed projects in the water sector have benefitted over 1.5 million people. With respect to access to electricity, the Bank has contributed 120 MW to Zambia's installed generation capacity through the implementation of the Itezhi-Tezhi power project. Progress has also been made towards health and education outcomes.

In addition, the Bank has sought to promote economic growth and reduce poverty by increasing agricultural productivity and trade. Although overall agricultural exports have increased, little progress has been made in diversification. Agriculture as a share of total exports has decreased from 11.54 percent in 2002 to 7.49 percent in


2014 though there is an increase in absolute terms. Furthermore, there is evidence of increased land under cultivation and use of improved irrigation techniques. During 2008–2012, the total area cultivated increased by 16 percent, whereas total irrigated land in Zambia increased by seven percent during 2001–2011, suggesting that some gains have been achieved for agricultural productivity. However, there is limited evidence of agricultural diversification away from maize production, which the government has accorded highest priority.

The Bank's interventions have aimed at improving the business environment in terms of ease of doing business, access to finance and business development. The ease of doing business index improved from 92 in 2006 to 83 in 2014. Access to finance as reflected in the domestic credit to the private sector as a percentage to the GDP increased from 10.2 in 2007 to 17.2 in 2013, though the Bank’s contribution in this area was modest. The Bank missed the opportunity to support the corporate social responsibility initiatives adopted by Itezhi-Tezhi Power Corporation and Barrick, which would have enhanced the impact of the Bank’s investments.

Finally, during 2006-2011, the Bank's efforts to strengthen macroeconomic management through general budget support appear to have contributed to the reduced gross government debt and budget deficits. In 2014, however, the gross government debt has increased, reaching 35.1 percent of GDP. Similarly, annual deficits have returned to 2004 levels, reaching 10.6 percent. Furthermore, annual wage expenditure accounted for an increasing share of GDP and, in raw terms, has increased by more than 400 percent, from 2,968 billion ZMW in 2006 to 11,897 billion ZMW in 2014. These changes coincide with a decline in scores on financial governance indices, including CPIA scores for Fiscal and Debt Policy, from 4.5 in 2009 to 3.0 in 2015.

Efficiency

The efficiency of the Bank's portfolio is rated as unsatisfactory owing to considerable timeliness issues and portfolio disbursement ratios which have consistently fallen below the Bank-wide average.

The Bank’s completed projects have incurred on average eight-month delay for every planned year of project implementation. Several factors account for this delay, including project design issues, delays in meeting conditions, complexities with co-financed projects and procurement delays. Project design issues in the Bank’s projects were characterized by an underestimation of costs leading to incomplete implementation of components or additional costs and time due to a re-engineering of the technical design. Indeed, delays in the fulfilment of conditions precedent to first disbursement have been a recurrent problem. The average delay for loan effectiveness was 13 months, while the average delay for first disbursement was five months. However, delays between approval and first disbursement have improved recently, falling from 16 months in 2011 to 12 months in 2014. Such delays have serious implications for the private sector operations where the Bank begins supervision only after the first disbursement. The financial viability of the borrower can deteriorate yet the Bank would be unaware of the risks involved. The complexities of co-financed projects were reflected in the incompatibility between the private and public sector approval processes of the Bank. These were exacerbated by delays in reconciling lending terms among multiple DPs. Finally, procurement delays resulted from weak procurement capacity within line ministries and heavy internal government approval processes. Related to this issue is the requirement that all contracts must be approved by the Office of the Attorney General (OAG), which is not able to clear all contracts in a timely manner due to resource constraints.

5Executive Summary

An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

The overall disbursement ratio has improved during the evaluation period, from 83 percent for the 2002–2004 CSP to 96 percent for the 2007-2010 JASZ and 93 percent for the 2011–2015 CSP. As of 2008, the disbursement ratio for Zambia has fallen slightly below the Bank-wide average, with some recent improvements. As per the calculations available in PCRs, on average, the ratio between ex-post and ex-ante EIRR was 0.96, indicating that the projects came close to meeting their expected rates of return.

Sustainability

Sustainability of the Bank project outcomes was rated moderately unsatisfactory with variability across sectors, as noted below:

❙ Transport sector projects face financial sustainability and institutional capacity risks in ensuring the sustainability of outcomes. Institutional risks arise from management challenges of over-procurement and contract management irregularities.

❙ The primary sustainability concern for the Itezhi-Tezhi power project has been the financial sustainability of the project given that Zambia's energy tariffs continue to fall far below cost recovery levels.

❙ WSS projects have benefitted from the selection of simplified and uniform technical solutions to reduce the cost and complexity of maintenance. But in some instances they have struggled to ensure financial sustainability of operations and reduce unaccounted for water.

❙ The primary risks to the sustainability of agricultural projects stem from institutional sustainability, ownership by the beneficiaries, and exogenous shocks.

❙ Poor sustainability of outcomes from the Bank's PRBS program is attributed to institutional capacity risks and insufficient government ownership.

❙ Finally, sustainability of private sector projects was compromised by regulatory changes as well as variable profitability among the targeted institutions.

Crosscutting Themes

The coverage of gender across the CSP periods was rated moderately unsatisfactory with uneven identification of gender-specific development constraints and few targeted interventions. However, gender has been mainstreamed into project design for the majority of projects by identifying employment and income generating opportunities for women. In some cases, efforts to promote income-generating activities for women were limited by project design omissions whereas other projects missed opportunities in this regard.

The coverage of inclusive growth and environmental sustainability was rated moderately unsatisfactory, and was most explicit under the 2002–2004 CSP but decreased across subsequent CSPs. The themes of inclusive and green growth were integrated across the portfolio through the promotion of socially and environmentally responsible investment and a reduction of disparities between urban and rural areas. Overall, integration of inclusive and green growth within the project portfolio was rated moderately satisfactory.

Leveraging and Co-financing

Approximately 55 percent of projects approved have co-financing from partners. Furthermore, the 2015 CSPCR notes that, over the strategy period, the Bank managed to secure co-financing of 220 percent of the original ADF allocation. With respect to the leveraging of funds, the Bank has served as the lead arranger for three projects, that is, the Itezhi-Tezhi Power Generation and Transmission Project, the Nacala Corridor Road Project and Kazungula Bridge. Through these projects the Bank was able to leverage its own contribution by factors of 16, 3.68 and 1.82, respectively. The Bank has also secured co-financing from emerging donors, including


India Exim Bank, the Development Bank of South Africa and the OPEC Fund for International Development.

Managing for results

All CSPs were rooted in a robust analysis of the political context, macroeconomic context, dimensions of poverty, national development strategies and constraints to growth surrounding the Bank’s projects. Understanding of the country context has been underpinned by consultations to inform CSP design. However the clarity and realism of the intervention logic for the Bank's CSPs remained relatively weak until the 2011–2015 CSP. Despite implementing activities in a broader range of sectors, the Bank's strategy has managed to become more coherent in that projects across a range of sectors now support a more limited number of strategic outcomes.

Project design weaknesses were identified with regard to: a) realism of the intended outcomes given the scope and design of

projects; and b) clarity of the intervention logic, including confusion between project outputs and outcomes issues on the realism of project design arose from the quality of feasibility studies underpinning the project designs. This resulted in an underestimation of project costs, time and quality of engineering designs.

The frequency of supervision has increased over the evaluation period. A portfolio review indicated that, during 2002–2004, projects were supervised, on average, once every two years. The feedback from the government on the Bank’s supervisions shows that the quality of interaction with the Bank has improved over the years with the establishment of the Country Office.

Overall Assessment

In the background of the above findings, the overall performance of the Zambia Country Strategy and Program 2002-2015 has been rated moderately unsatisfactory, as shown below:

Criteria RatingRelevance ❙ Alignment of CSPs with National Development Plans Satisfactory

❙ Alignment of the Project Portfolio with CSPs Satisfactory

❙ Alignment of Projects with the Needs of Beneficiaries Satisfactory

Rating for Relevance SatisfactoryEffectiveness ❙ Delivery of Project Outputs Moderately Satisfactory

❙ Achievement of Outcomes Moderately Unsatisfactory

Rating for Effectiveness Moderately UnsatisfactoryEfficiency ❙ Timeliness of Project Implementation Unsatisfactory

❙ Financial and Economic Performance Moderately Unsatisfactory

Rating for Efficiency Unsatisfactory

7Executive Summary

An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

Conclusions and Recommendations

The foregoing analysis leads to the following main conclusions:

a. The Bank’s Country Strategies and Programs have been well aligned with both national development plans and the Bank's comparative advantage.

b. The Bank's portfolio has become more coherent, adopting an integrated approach to development challenges.

c. The Bank has largely delivered planned outputs but the achievement of outcomes has been limited by project design weaknesses and delayed implementation.

d. The Bank has contributed to increasing access to basic services and improving the business environment. However, opportunities for upscaling have not been leveraged.

e. Political and governance risks are becoming an increasingly important factor in the sustainability of projects.

f. Whereas the Bank's activities have attempted to promote inclusive and green growth, opportunities to mainstream gender have not been fully leveraged.

Based on the above findings and conclusions, the evaluation proposes the following nine recommendations.

1. Continue to support private sector regulatory reform and build capacity among the accountability functions of government, particularly audit and procurement.

2. Strengthen the Bank’s role in donor coordination, analytical work and policy dialogue. The policy dialogue needs to be supported by rigorous analytical work in several areas of public policy including private sector development, public private partnerships, and regulatory systems with policy predictability. Coordinated efforts are required to address deficiencies in the monitoring and evaluation system at the country level and, in this regard, the Bank should contribute to the ongoing initiative. Policy dialogue should also be implemented to

Criteria RatingSustainability ❙ Private Sector Moderately Unsatisfactory

❙ Power Sector Moderately Unsatisfactory

❙ Agriculture Sector Moderately Unsatisfactory

❙ Multi-Sector Unsatisfactory

❙ Social Sector Moderately Unsatisfactory

❙ WSS Sector Moderately Satisfactory

❙ Transport Sector Moderately Satisfactory

Rating for Sustainability Moderately UnsatisfactoryCrosscutting themes ❙ Coverage of Gender within CSPs Moderately Unsatisfactory

❙ Gender mainstreaming across the portfolio Moderately Unsatisfactory

❙ Inclusive and Green Growth within CSPs Moderately Unsatisfactory

❙ Inclusive and Green Growth across the portfolio Moderately Satisfactory

Rating for Crosscutting Themes Moderately UnsatisfactoryOverall Rating Moderately Unsatisfactory


address political and governance risks to the sustainability of projects.

3. Address constraints to private sector involvement in service delivery and address infrastructure bottlenecks. Inadequate tariff structures and macroeconomic pressures create disincentives to private sector participation. The Bank should identify, through analytical work and policy dialogue in particular, opportunities to mitigate the impact of these constraints through a range of instruments, including lending, TA for project selection and guarantees for service delivery and purchasing agreements.

4. Identify opportunities to upscale development outcomes from private sector investments. The corporate social responsibility initiatives of the Itezhi-Tezhi Power Project and the Lumwana Mine provide opportunities for complementary investments which could scale up the development impact of those projects.

5. Deepen the integration of gender in Bank’s operations and engage in policy dialogue on gender with the government in collaboration with cooperating partners. The Bank, with other cooperating partners, should assist in the implementation of the National Gender Policy 2014 with a focus on gender auditing, gender responsive national planning and budgeting, and the establishment of a system for monitoring gender outcomes at country level.

6. Promote synergies within the Bank program and in coordination with other donors between projects which improve

the productivity of small businesses and increase access to finance. Increasing the pool of SMEs that generate sufficient revenue to afford formal financial products remains an obstacle to improving access to finance. Opportunities for creating linkage between interventions which increase production of high value crops, incomes and access to markets among underserved groups and LOC or TA for SME finance should be explored.

7. Identify means of harmonizing project implementation approaches with other co-financing partners as well as within projects that possess both private and public sector components. The Bank needs to work closely with partners to reduce delays by identifying harmonized terms, conditions, engineering and procurement approaches for co-financed projects to avoid implementation delays, cost-overruns and output quality inconsistencies.

8. Develop the capacity of the GRZ for project selection, design and engineering, particularly for infrastructure projects and Public Private Partnerships.

9. Identify and mitigate operational issues surrounding multinational projects aimed at regional integration. Based on the identification of constraints, the Bank should establish linkage between the CSP and the broader regional integration strategy to make sure bottlenecks are addressed at the right level – either through dialogue at country level or coordinated approach at regional level.

9Management Response

An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

Management Response

Management welcomes the Independent Development Evaluation (IDEV) report: Evaluation of the African Development Bank's Country Strategy and Programme for Zambia (2002–2015). The purpose of the Evaluation was to assess, among others, the relevance, effectiveness, efficiency, and sustainability of the strategic interventions and to inform the preparation of the next Zambia Country Strategy. The evaluation revealed that the Bank’s Country Strategies were relevant in terms of strategic focus and alignment in addressing Zambia’s Development challenges. The projects supported by the Bank delivered most of the planned outputs while project outcomes did not always meet the targets. Project efficiency was rated moderately unsatisfactory largely due to the lags in preparing projects and the delays between loan approval and effectiveness for first disbursement. Sustainability was rated moderately unsatisfactory. This was due to low profitability of some private sector projects while issues of cost reflective pricing was lacking in water and energy supported projects.

Introduction

The IDEV evaluation covers the period 2002–2015 and three Country Strategy Papers: the 2002–2004 Country Strategy, which was extended to 2006; the 2007–2010 Joint Assistance Strategy for Zambia formulated as a joint program with support from multilateral and bilateral development agencies operating in Zambia; and the 2011–2015 Country Strategy.

CSP Strategic Focus

The key strategic focus of the three Country Strategies was the support to infrastructure development in support of agriculture, water and sanitation, energy and transport. In the past five years, a higher degree of support to economic and financial governance was pursued to target regulatory reform to facilitate private sector development as well as public financial management reforms.

Evaluation Design

The evaluation team developed an evaluation matrix and theory of change to guide data collection and analysis. This included key indicators and judgment criteria based on a six point scale that was used to create a transparent account of how IDEV examined the achievement of results across each sector.

Although Management received an overall rating for each judgement criteria, management has not seen how each individual project was assessed. This would have increased transparency on the part of the evaluation and allowed management to better understand the findings in terms of outputs and outcomes. As the evaluation period is quite long (more than 10 years), the report could also provide some information on how the performance changed over time, to make the assessment a bit more dynamic and indicate some trends.


The evaluation team notes various valid limitations of the evaluation methodology. Furthermore flaws in the intervention logic and the lack of baseline data made it difficult to evaluate progress on some indicators. Management agrees with the finding. The results based framework will be strengthened in the context of a clear intervention logic as well as indicators will be selected where there is baseline data available when drafting the next Country Strategy Paper.

Program Implementation

The evaluation uses four key criteria: relevance, effectiveness, efficiency, and sustainability to evaluate the country strategies. In addition, cross-cutting issues such as gender, quality-at-entry, and leveraging and co-financing are also assessed. Each of these elements are commented on below.

Relevance

Relevance is measured as the degree of alignment of Country Strategies to National Development plans, and of projects to Country Strategies and beneficiaries.

The alignment of Country Strategies to National Development Plans was rated satisfactory. The Bank has used its comparative advantage in the support of infrastructure development while addressing the specific needs of the Zambian Government as outlined in the National Development Plans (NDPs), consistent with Bank’s Ten Year Strategy (2013– 2022) and the Hig-5s. Given the large investment needs that remain in transport, energy, and water and sanitation, Management will, consistent with the priorities in the NDPs, continue to support this strategic focus in the new Zambia Country Strategy.

Project alignment to the needs of beneficiaries was rated overall satisfactory. Management agrees with the finding that the needs of Small and Medium Sized Enterprises have not always been sufficiently catered for. In the new Country Strategy it is proposed that the Bank will provide more targeted support to Small and Medium Sized Enterprises through collaboration with local financial institutions. Lines of Credit will be better targeted for the needs of SME’s such as providing business development services which are often in high demand. Projects designed to support and expand SME’s will also be developed. In fact, the Bank is currently financing the Cashew Project that links farmers with local processors and helps expand the market for Cashew. Entrepreneurs in the cassava value chain are also being supported through improved product development, produc¬tivity increase, and commercialising the crop. These support is being provided through capacity building and business development services.

Effectiveness

Project effectiveness is measured based on achievements of outputs and outcomes targets. Achievements of outputs are rated moderately satisfactory, while achievements of outcomes are rated moderately unsatisfactory.

Project Output: Most prioritized sectors have demonstrated good progress in achieving planned outputs including private sector, energy, agriculture, water and sanitation, transport, and governance. Agriculture has been a challenging area with limited progress due to a difficult implementation environment and complex procurement processes that contributed to implementation delays.

Project Outcome: Achievement of outcomes is unsatisfactory as a result of start-up delays, implementation and procurement delays, and delayed completion dates for projects. This in turn


An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

affected project impact and achievement of results. The evaluation report states that achievements were less than expected in private sector projects due to profitability and lack of improvement in the terms of finance. It should be noted that for most of the period under evaluation, Zambia was an ADF only country with most of the financing targeting public sector projects. The price of copper between 2011 and 2015 declined by more than 50 percent with prices falling to under USD 4800 per tonne in 2015 affecting Governments revenue generation. The added pressure to Government’s fiscal position led to higher domestic lending pushing up the price of funding thus crowding out the private sector. Average interest rates are high and currently at above 28 percent. Although the Bank has provided Lines of Credit to strengthen SME financing, it has no influence on the cost of treasury bills which are used as the benchmark for commercial lending.

Policy dialogue and knowledge work: Management notes the positive development of the decentralisation process in terms of policy dialogue and donor coordination. The Bank’s physical presence has made it possible to service its key clients, build a close relationship and enhance dialogue with the Government, use the local networks, build the portfolio while leveraging finance from development partners, provide capacity building to project implementation units and respond efficiently to arising needs.

Strategic Outcomes: At the strategic outcome level the Evaluation Report concludes that the Bank has made a positive impact on improving the business environment, and increasing access to basic services and infrastructure. Limited progress has been achieved to strengthen Public Financial Management and promote diversification by promoting the agriculture sector.

The Bank intends to continue supporting agriculture and agribusiness as it is critical for employment and rural incomes and as it is an important source of foreign earnings given its large market potential

within the region. Diversification of the sector has been hindered by the Governments continued support to the Farmer Input Support Program and the purchase programs by the Food Reserve Agency. Both these programs are heavily focussed on supporting the production of maize. From 2015 an e-voucher system provides some promise in diversifying the sector as farmers can now choose inputs of their choice. In the new Country Strategy, Management is proposing to provide a holistic approach to agriculture by supporting value chains and developing markets in line with the Feed Africa-Strategy for Agricultural Transformation in Africa (2016–2025).

Improving public financial management has been a challenge. A Multi-donor Trust Fund, led by the World Bank is currently supporting this area, in which the Bank is not providing direct support. In the new Country Strategy, Management is proposing to strengthen Public Financial Management capacity directly through the implementing agencies. This will help build a critical mass of qualified accountants, internal auditors, and procurement officers and build the needed institutional capacity in the executing agencies that we support. It is felt that institutionalized capacity building of professionals would be more sustainable than project based training or orientation which targets only project staff whose turnover is high as they move on after the projects are completed leaving the executing agencies without the required capacity to undertake other new projects. Agencies will be screened for Public Financial Management skills and needs during project design. Any support to agencies will be based on this assessment. Activities proposed will be coordinated with the activities supported by the Multi-donor Trust Fund to avoid overlaps.

Efficiency

The efficient delivery of projects is evaluated based on the timeliness of projects and on the financial and economic performance that are demonstrated.


Timeliness: The dimension of timeliness is further evaluated using four sub-dimensions: inadequate project design; loan effectiveness and disbursement; co-financing complexities; and, procurement delays. Management notes with concern that despite recent improvements, projects are significantly delayed on the various sub-dimensions mentioned above. Management would like to point out that in the past 18 months the government introduced new approval procedures, including the need to obtain parliamentary ratification of loan agreements with the new constitution, and this caused significant project effectiveness delays.

Management has already taken steps to strengthen project design and preparation to reduce the risk of delays. To reduce delays caused by loan effectiveness and effectiveness to first disbursement any preconditions or other conditions that risk affecting a project will be addressed upfront. This implies that projects will only be elevated to the Board once conditions have been addressed, especially conditions that borders on project readiness and quality at-entry. Key portfolio issues, including weak capacity of project implementation units in procurement, financial management and contract administration are being addressed through joint quarterly portfolio review meetings and annual fiduciary clinics organised with the Ministry of Finance and attendance of project staff.

The Bank has been very successful in leveraging projects through co-financing arrangements. However, in terms of implementation, as pointed out in the Report, significant delays have been experienced, particularly when bilateral donors are involved. Management has strengthened efforts to harmonise internal procedures to avoid similar types of process related delays in the future. Clear agreements on procedures will be made before Board approval and in accordance with PD 02/2015. Lessons learnt from prolonged

negotiations of common terms agreements for co-financing partners would guide the formulation of future projects.

Procurement capacity of Government counterparts is often weak with a high turnover of staff. Despite Bank support to capacity building and training, procurement delays often persist. Management will continue to prioritise and provide training to help build a critical mass of procurement officers in the public sector. As part of new projects public procurement training will be strengthened and institutionalized in the implementing agencies where relevant. In addition longer term support could be provided through the Zambian Public Procurement Authority that is mandated to carry out capacity building of procurement entities.

Financial and economic performance: Between 2002 and 2014, the disbursement ratios for Zambia portfolio had fallen below the Bank wide average. However, in 2015 improvements in disbursement ratios were recorded. The Zambia portfolio is young (2.4 years on average) with limited disbursements in the early period of project implementation. Improvements are therefore expected during 2016 and 2017 as the new projects start to increase their disbursement levels. This is consistent with implementation of most of the high value large infrastructure projects which usually have slow disbursements at the beginning. In addition, by improving quality at entry and project readiness, the disbursement ratio is expected to continue to increase in the coming years.

Management notes that agricultural interventions are rated “moderately satisfactory”, yet the agricultural projects addressed the needs of specific groups of beneficiaries in terms of increasing productivity, improving food security and reducing poverty. Thus this assessment appears to be conflicting, in terms of performance of agricultural projects.


An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

Sustainability

The sustainability rating was evaluated below satisfactory. For utilities, this was related to cost reflective pricing of water and energy. Management agrees that usage of public resources need to be cost reflective. The Bank is currently supporting an industry wide cost of service study that will inform the Zambian energy market on the pricing of electricity at different consumer and firm levels. The water utilities have struggled with improving the none-revenue water in order to reach operational efficiency. The Bank continues to work with utilities to improve infrastructure and metering to reduce leakages while continuing to provide capacity building. Management will take into consideration the social aspect of utilities in terms of providing services and protecting the poor. A carefully designed cost reflective approach and tariff pricing should therefore be pursued when supporting such projects.

The Evaluation Report concludes that some private firms were affected by regulatory changes and variable profitability that adversely affected the sustainability rating, e.g. the support to Lumwana mine and CETZAM financial institution. However, as mentioned earlier, profitability was affected by a significant decline in copper prices between 2011 and 2015 which also affected public finances and raised commercial lending rates. Notwithstanding the above and with a few exceptions, most AfDB-supported firms have managed to adjust to the more difficult business environment and are operating as going concerns.

Cross-Cutting

Two dimensions, gender and inclusive growth, were used to assess the level of integration of cross-cutting issues into Country Strategies and the project portfolio.

Gender: The Bank is implementing the Bank’s Gender Strategy while gender mainstreaming in all its operations. However, according to the Evaluation Report gender received limited attention across the evaluation period. Management acknowledges the need to pay more attention to gender and recognises a strengthened approach to addressing gender as the new Country Strategy is being prepared. Initially, projects lacked effective tools for addressing gender issues in Bank operations. A comprehensive country gender assessment is about to be commissioned to identify gender gaps and challenges, of which the outcome will inform gender specific interventions in Bank operations under the new CSP.

Inclusive Growth: Focus on inclusive and green growth and environmental sustainability was most prominent between 2002 and 2004. Although poverty levels have fallen in urban areas, poverty in rural areas remains stubbornly resistant. Limited access to markets is cited as a key cause. In the new Country Strategy, Management plans to support a stronger linkage between rural production areas and urban markets. This will be done by supporting agriculture value chains and the development of markets. In the current portfolio, climate change is being addressed in several ways. One project, Strengthening Climate Resilience in the Kafue Basin, aims to mitigate against drought and flooding through improved planning and climate resilient infrastructure and production systems. Other projects such as Lake Tanganyika Development and the Agriculture Productivity and Market Enhancement Projects implement climate change as a cross cutting issue being mainstreamed into the main activities.

Some older agriculture projects that were designed and implemented before 2008 may have had some adverse environmental impacts. Currently all projects are classified in terms of their potential level of social and environmental impacts as guided by Bank Policy. Category 1 and 2 projects are subject to Environmental and Social Management Plans. This practice will continue.


Quality at Entry

Two dimensions of quality-at-entry have been analysed. The conclusion is moderately satisfactory for the Quality-at-Entry of Country Strategies while quality and realism of project design was moderately unsatisfactory.

Management notes with satisfaction that the intervention logic improved with the 2011–2015 Country Strategy. The framework still needs to be more practical while higher attention needs to be placed on the realism of indicators, especially at the outcome level. Management will ensure that the results framework in the new Country Strategy demonstrates a clear and coherent theory of change and is more practical.

Management notes that in some instances project design has been inadequate and that this led to under costing of project outputs and a lower number of outputs delivered. Management acknowledges that the use of Project Finance in a Zambian context was challenging due to limited understanding of the approach and weak capacity in the area. In order to build capacity, the Bank provided training to government officials in the region in the use of and preparation of Project Finance.

Leveraging and Co-Financing

Management efforts to promote leveraging and co-financing has been successful in Zambia as indicated in the Evaluation Report. This was particularly relevant following the financial crisis and when Zambia could only access ADF resources. With the reclassification to Blend country, Zambia has increased its access to ADB resources while the need for leveraging may diminish in some types of lower risk projects. However, management will continue to pursue co-financing to forge partnerships with development partners and utilise synergies. A recent example of such a project is the Kariba Dam

rehabilitation project which the Bank is financing with two multilateral partners and a bilateral partner.

Conclusions and Key Lessons

Management appreciates the following conclusions and key lessons emanating from the Evaluation Report:

The Country Strategies and programmes have been well-aligned with the national vision, national development plans and national strategies. This confirms the Banks focus on addressing the large infrastructure gaps that Zambia still faces. Furthermore the Management’s plan to strengthen support to private sector in the new Country Strategy, while addressing regulatory constraints and bottlenecks, is the appropriate strategy going forward.

The integrated approach that the Bank followed in the 2011–2015 Country Strategy by focusing on the strategic outcomes has meant a broader selection of sectors that have received support. Additionally, the Report confirms that

the integrated approach has more coherently addressed Zambia’s development challenges. Management plans to continue this approach in the new Country Strategy.

Although outputs have largely been delivered as planned, the level of achievement of outcomes is below satisfactory largely due to implementation delays. There is need for improved quality at entry to ensure project readiness. For example, for infrastructure projects that require feasibility studies, such studies should be completed before presentation of the project to the Board.

Bank support has contributed to increased basic services and improvements in the business environment. Under private sector, more could be


An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

done in utilising channels for upscaling projects. One example the Report mentions is scaling up activities that the private sector supports through their Corporate Social Responsibility programs.

Political, governance and regulatory risks have increased during the past 5-7 years with regulations that are passed without sufficient consultation with the private sector. This has affected project implementation and profitability of some private sector operators that have reduced their investments. For example, the most profitable mine in the country has not made new investments in the Country since 2008 due to

regulatory changes that are not discussed with the private sector. Enhanced policy dialogue, knowledge work and addressing risk will be important activities in the implementation of the new Country Strategy.

Mainstreaming gender in operations has not been fully leveraged. In the preparation of new projects enhanced gender screening and screening for smart climate approaches, skills needs and employability of direct beneficiaries and a conducive regulatory environment will receive more attention when designing projects in the implementation of the new Country Strategy.

MANAGEMENT ACTION RECORDRecommendation Management response

Recommendation 1: Continue to support private sector regulatory reform and build capacity among the accountability functions of government.

Budget support and TA interventions have been successful in promoting private sector regulatory reform and strengthening of accountability functions, partly due to adequate ownership among implementing bodies. These issues continue to be relevant, particularly the need to strengthen audit and procurement capacity. Furthermore, strengthening external accountability functions can complement the implementation of financial control systems to reduce irregularities and improve public financial management.

AGREED. Support to private sector reforms and capacity building have yielded positive results. These interventions need to be continued and extended to areas identified in the recommendation.

Action:

As proposed in the new CSP, Bank operations will continue to include support to Economic and Governance Reforms. Reforms can be supported through sector budget support or will be included as components in relevant projects and programs, and aligned with the High-5s. Additionally, institutional support program(s) including support to strengthening audit and procurement capacities in the public sector will be considered. Task Managers will be responsible for ensuring that projects are screened for relevant sector reforms during the project preparation stage and included in project design where it makes good sense. Furthermore, task managers will be responsible for assessing needs in relation to accountability functions in the project institutions that we support (ZMFO, 2017)



Recommendation 2: Strengthen the Bank’s role in donor coordination, analytical work and policy dialogue.

The Bank, with its unique positioning in Africa and among the development partners, is expected to play a key role in donor coordination, analytical work and policy dialogue. The policy dialogue needs to be supported by rigorous analytical work, which is also demanded by the government in several areas of public policy including private sector development, public private partnerships, and regulatory systems with policy predictability. The donor coordination efforts are currently subdued due partly to the withdrawal of several cooperating partners from key areas of assistance including budget support. Coordinated efforts are required to address deficiencies in the monitoring and evaluation system at the country level to enable systematic reporting of results with active participation of all line ministries, and the Bank should contribute to the ongoing initiative in this regard. Finally, policy dialogue should also be implemented as a means of addressing increasing political and governance risks to the sustainability of the Bank's projects, particularly with regard to ensuring adequate consultation with the private sector to inform regulatory changes.

AGREED. Management agrees with the key roles that the Bank is already playing in Zambia as well as the need to continue to strengthen them. Although the Budget Support group is no longer active, a Cooperating Partners Group continues to carry out policy dialogue at the highest levels. The Bank remains very active in this group and chaired it in 2012 while also chairing other working groups (transport, water and sanitation, agriculture and monitoring and statistics group) that carry out policy dialogue at the sector level.

Action:

❙ Knowledge work has intensified during the evaluation period as stated in the report (section 5.2.7). The Bank will continue to strengthen analytical work to improve the underpinnings of our policy dialogue. As proposed in the new CSP, non-lending activities will support policy analysis and other knowledge work while policy operations will be pursued at the sector level to facilitate key reforms (e.g. energy reforms). The proposed CSP was based on a diagnostic study prepared by the Bank, relevant Bank studies and policies as well as sector studies from other agencies. Management will continue to identify, coordinate and drive relevant studies. Furthermore, during project preparation and design, relevant studies may be identified and included in the non-lending operations. (ZMFO, Ongoing)

❙ Monitoring and Evaluation at the country level will be strengthened. The Bank is working with the Ministry of Finance to create a simple tool for tracking projects at the formulation stage with the aim of reducing start-up delays. Expanding the tool to also incorporate results monitoring will be investigated and implemented, while taking into account a monitoring system that Government is already developing with the help of the EU, United Kingdom and Germany (Ongoing). The Country Programs Officer will be responsible to coordinate the development of the tracking tool during the first half of 2017. (ZMFO, 2017)

❙ Leveraging additional financial resources, both to finance public operations as well as private ones, would be carefully determined and pursued with various partners, especially for large infrastructure operations in energy, water and sanitation, road transport and agri-business development. (ZMFO, Ongoing).


An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n


Recommendation 3: Address constraints to private sector involvement in service delivery.

Private sector participation in service delivery, particularly in the energy sector, will play an increasing role in addressing Zambia's infrastructure bottlenecks. However, inadequate tariff structures and macroeconomic pressures create disincentives to private sector participation. The Bank should identify, through analytical and policy dialogue in particular, opportunities to mitigate the impact of these constraints through a range of instruments, including lending, TA for project selection and guarantees for service delivery and purchasing agreements.

AGREED. The private sector will continue to be a key sector requiring Bank support. The Bank is currently supporting the undertaking of a cost of service study, whose finding and recommendation should help inform the setting of future tariffs as well as point out inefficiencies in the electricity system as a whole. The study will therefore inform Bank Policy dialogue in discussions of moving toward cost reflective tariffs. In the water sector, tariff and affordability studies are carried out for each utility and presented to the regulator. The tariff progressions are then implemented in parallel with efficiency gains from the utility based on key performance indicators that rewards good progress. However, improving efficiency is a process that can take several years to implement. Thus institutional strengthening and capacity building are important elements during project implementation.

Action:

In the new CSP, as part of the interventions in the energy sector, the Bank will target support to institutional strengthening and reforms as a means of attracting private investors. Furthermore, diversifying the energy mix will be important to reduce risk of relying only on hydro. Support to the sector will also entail providing Technical Assistance in the areas highlighted in the recommendation. The Bank will also support and/or use relevant studies to guide policy dialogue pertaining to the private sector. The Country Office together with the sector departments will be responsible for suggesting which instruments best address inadequate policy or constraints. The Government of Zambia has already requested for energy sector budget support which will provide an opportunity for the Bank to assist in reforming the energy sector and to enhance private sector participation. (ZMFO, 2017)

Recommendation 4: Identify opportunities to upscale development outcomes from private sector investments.

Although the Bank has helped improved access to services and infrastructure through public sector projects, private sector investments have yielded similar results through infrastructure works and corporate social responsibility initiatives. As demonstrated by the experience of the Itezhi-Tezhi Power Project and the Lumwana Mine, additional development impacts and private sector development could be realised through complementary infrastructure investments surrounding these projects. Integrating lessons from past experience into the CSP, the Bank should more systematically identify opportunities to upscale and complement such initiatives, thereby further contributing to service delivery, infrastructure improvements and business development.

AGREED. Private sector operations have in many instances helped improve access to services and infrastructure. The challenge remains bringing on board similar operations to scale up such initiatives.

Action:

The Private Sector and the Agriculture Sector Specialists in the Country Office will be responsible for ensuring the following action points (ZMFO, OPSD, OSAN, 2017):

❙ In the new CSP, the Bank will focus on providing technical assistance such as business development services with the aim of helping identify and build capacity of the private sector.

❙ The Bank will also focus on support to the private sector through financing selected agri-value chains (production, processing and market linkages) and finance infrastructure needs (irrigation, collection, storage). It is expected that through these interventions, additional development impact will be realized.

❙ Management will seek more innovative interface in utilising public and private sector financing windows for agro-processing, irrigation and farm block infrastructure projects.



Recommendation 5: Deepen the integration of gender in Bank’s operations and engage in policy dialogue on gender with the government in collaboration with cooperating partners.

Even though the Bank’s analytical work on gender in Zambia had recommended specific areas for intervention, those were not implemented in earnest. In the increasingly relevant context of feminisation of poverty and HIV/AIDS pandemic, gender violence, women’s unequal access to education, health services, and resources including land and credit, it is imperative that the Bank take concrete steps to integrate gender in its operations with respect to design, implementation and the achievement of results. The Bank, with other cooperating partners, should engage in policy dialogue with the government and assist in the implementation of the National Gender Policy 2014 with focus on gender auditing, gender responsive national planning and budgeting, and establishment of a system for monitoring gender outcomes at country level.

AGREED. The Bank is implementing the Bank’s Gender Strategy while mainstreaming gender in all its operations. The Bank will also continue to assist Government in the implementation of the Zambia National Gender Policy 2014.

Action:

❙ Management will continue to ensure policy dialogue, collaborative analytical work and capacity building with relevant partners including the Ministry of Gender, United National Population Fund, and the Zambian Non-Governmental Organisation Coordinating Committee, as guided by the Zambia National Gender Policy. The Bank has already approved USD 30 million towards the cost of implementing the Zambia Skills Development and Entrepreneurship Project – Supporting Women and Youth (SDEP-SWY). This operation aims to promote job creation, gender equality and poverty reduction. The project seeks to improve the livelihood opportunities of Zambia’s working poor, especially in rural areas and will benefit particularly, women and youth by providing enabling infrastructure and entrepreneurship skills for MSME business development. This operation has commenced implementation, focusing on women and youth groups and is expected to close in 2020. (ZMFO, Ongoing)

❙ The Bank will continue to ensure that all operational staff obtain skills in gender analysis, especially for project preparation and appraisal. The Bank will further mainstream gender in all its operations by developing gender indicators in Results Based Frameworks. (ZMFO, Ongoing)

Recommendation 6: Promote synergies within the Bank program and in coordination with other donors between projects which improve the productivity of small businesses and increase access to finance.

Increasing the pool of SMEs that generate sufficient revenue to afford formal financial products remains an obstacle to improving access to finance. Opportunities for creating linkages between interventions which increase production of high value crops, incomes and access to markets among underserved groups with LOCs or TA for SME finance should be explored. Such linkages may yield synergies in terms of access to finance, business development and economic growth. Outgrower and farmers club schemes provide a proven means of improving agricultural productivity and diversity while linking farmers to markets and increasing both incomes and access to finance. Attention should be paid to ensuring that targeted project mechanisms are included to ensure that the benefits are gender inclusive.

AGREED. Macro-economic instability has an impact on private sector’s ability to borrow as they often compete with Government’s domestic borrowing, impacting the cost of borrowing. Continued support to Government, through policy dialogue will create a conducive environment for private sector operations including lowering interest rates. Access to finance by private sector, especially SMEs forms a critical aspect of proposed pillars of the new CSP. Making resources available to SMEs continues to be a key ingredient in Bank’s operations, especially the development of rural enterprises for job creation and inclusive development. In line with the Feed Africa Strategy, the Risk Sharing Facility will be promoted for agricultural value chain operations with the objective of catalysing the private investments and commercial bank lending.Action:

The Private Sector Specialist in the Country Office will be responsible for ensuring the following action points (ZMFO, OPSD, 2017) ❙ The Bank’s operations and interventions using lines of credit as an instrument will be accompanied by measures to improve capacity for business management and expansion.

❙ The Bank will intensify support to financial institutions to develop financial products for SMEs, especially those involved in agro-processing and value-chain related enterprises. This will include the incorporation of incubation programmes for youth and women SMEs.

❙ The Bank will ensure that financial and technical assistance to SMEs will clearly distinguish and address the peculiarities of enterprises that women usually engage in. Financial products, including insurance and bridge financing, would be applied on exceptional cases to women enterprises.

❙ The Bank will provide support towards agriculture rural enterprise development, promotion of outgrower schemes promotion of agricultural productivity and diversity interventions.


An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n


Recommendation 7: Identify means of harmonising project implementation approaches with other co-financing partners as well as within projects that possess both private and public sector components.

Addressing regional and domestic infrastructure constraints will likely necessitate continued cooperation and co-financing. The experience of the Kazungula Bridge and Itezhi-Tezhi Power Project demonstrates the need for the Bank to work closely with other partners by identifying harmonised terms, conditions, engineering and procurement approaches for co-financed projects in order to avoid implementation delays, cost-overruns and output quality inconsistencies. Furthermore, the design and implementation of co-financed projects with both public and private components should be mindful of differences in the approval processes for these two sources of funds in order to avoid both unnecessary delays and additional costs.

AGREED. Mixed results have been observed with operations that are co-financed with other partners. This is, in part, due to the significant differences in policy, regulatory and fiduciary requirements of each lender. Where project financing is sequenced from one lender to another, delays from one partner significantly affects the operations (procurement and disbursement) of other partners. Harmonization in this case only refers to co-financing arrangements where it is feasible. We have experience that demonstrates that even where rules and procedures are harmonized, for example with the WB, no arrangements are put in place to reduce dual No Objections from the financiers. However, we agree with the observation that it is not possible to apply one set of rules in all circumstances and in this case we have suggested full financing of specific contracts, i.e. parallel financing, to which specific rules would apply as opposed to co-financing a contract and subjecting it to several reviews from financiers.

Action:

The Bank will be responsible for ensuring the following action items (ZMFO, 2017): ❙ The Bank will ensure that project readiness mechanisms have been well articulated and agreed amongst partners. This would include availability of designs, preparatory studies, setting up of project implementation teams and harmonisation of procurement procedures irrespective of source of finance.

❙ The Bank will ensure that common terms agreements for complex transactions, especially PPP and project finance, are well negotiated in advance, ahead of Board approval. This will minimise potential delays and conflicts usually associated with such negotiation after Board approval.



Recommendation 8: Increase the capacity of the GRZ for project selection, design and engineering, particularly for infrastructure projects and Public Private Partnerships.

Inadequate project design and engineering has frustrated the implementation of projects across several sectors, including the transport sector, for which inadequate project design and engineering can result in considerable project delays as well as procurement disputes. Assistance in implementing high quality feasibility studies which identify reasonable project costs would also be beneficial for the Bank's ongoing support to the WSS sector, for which underestimation of the project complexity and costs have limited the successful delivery of improved sanitation facilities for past interventions. As additional PPPs are implemented in Zambia, technical assistance should be provided to improve project selection and ensure that PPP arrangements are providing added value. Finally, greater attention should be devoted to identifying and assessing sensitivity to political and governance risks, including tariff structures, regulatory changes and ownership.

AGREED. As part of project readiness mechanisms that have been put in place, designs for infrastructure-related operations are usually in place ahead of Board approval, with the increased use of MIC TA grants, PPF and other relevant funding facilities. However, there are issues with the quality of some designs that require a review after Board approval to ensure quality during implementation.

Action:

The Country Programs Officer will be responsible for ensuring the following action items are addressed during project preparation (ZMFO, 2017): ❙ The Bank will intensify the use of advance contracting where appropriate for review of engineering designs and associated bills of quantities before Board approval. This will also help project teams to validate project detailed costs before Board consideration.

❙ The Bank will continue to provide capacity building and technical assistance to staff of key implementation units on various aspects of project formulation and implementation including PPP transactions, project formulation, and fiduciary issues such as financial management and procurement.


An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n


Recommendation 9: Identify and mitigate operational issues surrounding multinational projects aimed at regional integration.

The implementation of regional projects has been subject to excessive delays caused by coordination issues at the regional level, as seen in the case of PRODAP, which was discontinued in course of implementation. Delays in the full implementation of one-stop border post constrain the movement goods, services and people, affecting the competitiveness and economic growth of the region in general and the Zambia in particular. Based on the identification of constraints, the Bank should establish linkages between the CSP and the broader regional integration strategy to make sure bottlenecks are addressed at the right level – either through dialogue at country level or coordinated approach at regional level.

AGREED. Regional operations have continued to face significant challenges in coordination and harmonisation of processes. Soft issues of regional integration like the harmonisation of custom procedures, free movement of goods and services, have not received as much attention as infrastructure.

Action:

❙ Task Managers with the support of the Country Programs Officer will ensure that a joint project implementation unit is established for multinational operations with representation from participating countries, as was the case of Kazungula Bridge Project. The use of joint project steering committee would ensure that most of the project issues are resolved at the highest level for speedy implementation of planned activities. (ZMFO, Sector Departments, Ongoing)

❙ In situations where joint financing of projects are required for Multinational operations, the Procurement Officer will ensure that procurement modalities are harmonised during project preparation. This is to avoid multiplication of reviews or approvals by different lenders and participating countries. As much as possible contract packaging will be aligned to total contract financing where adoption of one common procurement rule is not feasible. (ZMFO, ORPF, Ongoing)


This report presents a summary of findings, conclusions and recommendations from an evaluation of the Bank's Country Strategies and Program in Zambia over the period of 2002–2015. This evaluation was conducted for two purposes: (i) to inform the development of the Zambia Country Strategy for 2016–2020; and (ii) to support IDEV's Comprehensive Evaluation of the Bank's

Development Results (CEDR). This evaluation seeks to: (i) provide an evidence-based assessment of the relevance and performance of the Bank's strategic interventions in Zambia; and (ii) identify findings, conclusions and recommendations to inform operations going forward. The evaluation covers all lending and non-lending operations approved during 2002–2015.

Introduction

23Background

An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

Economic Context

During 1964–1973, Zambia's economy was stable – its mineral resources were well developed and the world market was generally favorable. In 1973, oil prices rose sharply followed by a drop in copper prices. At that time, Zambia relied on copper for more than 90 percent of its export revenue.1 The result was continuous budget deficits and numerous external financing arrangements leading to serious economic distress. Throughout the 1980s, macroeconomic management was characterized by increasing state control of all industries, including the mining sector through the creation of Zambia Consolidated Copper Mines (ZCCM). Under state control, copper production decreased considerably because of poor investment in the sector, falling from over 700,000 tons in the early 1970s to just 256,884 tons in 2000.2

Zambia's economic woes were compounded in 1987 when slow progress in implementing an IMF reform agenda led to ineligibility for further IMF support and a discontinuation of finance from bilateral donors. By 1990, as a result of heavy non-concessional borrowing, Zambia had the highest debt to GNP ratio in the world at 225 percent.3 Liquidity problems resulted in very high levels of arrears (USD 2.3 billion), including substantial arrears to multilateral finance institutions. Following a political regime change in 1991, Zambia continued to implement IMF reforms, leading to a resumption of large aid inflows. However, the economy remained undiversified and continued to perform poorly, registering growth as low as 0.3 percent in the 1990s with inflation rising to 70 percent.4 As a result of rapid population growth, Zambia experienced a reduction in real per capita income over the period.

In the 2000s, Zambia's economic situation improved with increased investment in mining, manufacturing, tourism and agriculture. With the sale of ZCCM completed, Zambia became eligible for debt relief under the Highly Indebted Poor Countries (HIPC) initiative. After reaching the HIPC completion point in 2005, Zambia's debt was reduced from USD 7.1 billion to USD 4.5 billion. Under the Multilateral Debt Relief Initiative, the debt stock was further reduced to USD 0.5 billion by the end of 2006.5

Zambia has demonstrated strong economic growth over the evaluation period, with average GDP growth of six percent between 2004 and 2014, reaching a high of 7.6 percent of GDP in 2010.6 The economy has been driven by the service sector, which accounts for 60 percent of GDP, followed by industry at 31.3 percent and agriculture at 8.6 percent.7 Whereas the agriculture sector accounts for 85 percent of total employment, the mining sector accounts for 77 percent of Zambia's export revenue.8 Between 2004 and 2010, Zambia benefitted from both a steady rise in copper prices and a reduction of gross government debt. The HIPC Program and Multilateral Debt Relief Initiative saw Zambia's gross government debt fall from 78 percent of GDP in 2004 to 27 percent in 2006.9

Economic growth has since slowed, with GDP growth just 3.6 percent in 2015 and projected to remain close to three percent in 2016.10 After reaching a record high in 2011, copper prices fell more than 34 percent by 2013. This fall in copper prices coincided with a loosening of fiscal management as the annual deficit returned to 10.6 percent of GDP in 2014.11 External borrowing has filled the gap, with gross government debt returning to 35 percent in 2014. As a result,

Background


the Kwacha depreciated sharply, losing 61 percent of its value in the first 11 months of 2015.12 To add to these difficulties, Zambia has been experiencing a power crisis because of a considerable reduction in water levels in hydro-electric power stations caused by poor rains. The resulting power outages have taken a particular toll on the mining and manufacturing sectors.13 Mining companies are looking to scale back high-cost investments. In Zambia, this has translated into the delay of new investments and job losses (estimated at 7,700 in 2015).14

As such, Zambia's medium-term economic prospects are subject to a degree of uncertainty due to some key domestic risks, that is: (i) the expectation that copper prices will remain weak until 2018; (ii) a potential worsening of the power shortage due to delayed investment; (iii) failure of the government to control the growing public debt and deficit; and (iv) the potential for poor harvests to increase food prices which would place pressure on rural households.15

Social Context

In 2014, the estimated population of Zambia was 15,721,343, with an annual growth rate of 3.1 percent and an average life expectancy of 60 years. The population is evenly split between men and women but is predominantly young with roughly half of the Zambian population under the age of 15.16

Overall, less than half of Zambians have completed primary and secondary education with levels of educational attainment among women lagging behind that of men. Furthermore, whereas 83 percent of Zambian men are literate, just 67 percent of Zambian women are literate.17 Some progress has been made in recent years against key health indicators such as maternal, child and infant mortality but progress has been insufficient to meet the Millennium Development Goal indicators, presented in Annex G. Infant mortality has decreased from 107 to 45 deaths per 1,000 live births during 1992–2014. Similarly, under-five

mortality has decreased from 191 to 75 deaths per 1,000 live births over the same period.18 Although progress has been made in reducing the prevalence of HIV/AIDS, it remains a serious development challenge: an estimated 13.3 percent of Zambians aged 15-49 are HIV positive.19

The majority (67 percent) of Zambians live in rural areas. Geographical inclusion is a serious issue in the country with most social indicators demonstrating an urban/rural divide. Whereas 90 percent of urban households have access to an improved source of water, the same is true for only 47 percent of rural households (national average of 65 percent). Similarly, 56 percent of rural households and 36 percent of urban households lack access to improved sanitation facilities.20

Development Challenges

Poverty Reduction and Inclusive Growth

Despite Zambia's transition to lower-middle income country status in 2011, overall poverty levels have remained high at an estimated 60.5 percent. During 2006–2010, limited progress was made in reducing overall, rural and urban poverty.21 Zambia is a relatively unequal society, with a Gini Coefficient of 57.5, ranked at 145 of 187 countries.22 Furthermore, poverty in Zambia is overwhelmingly a rural phenomenon. Whereas urban poverty levels are estimated at 35 percent, rural poverty levels are estimated to be as high as 74 percent.23

Part of the reason economic growth has not translated into poverty reduction is the fact that job creation has been concentrated among a small segment of highly skilled workers in the industrial and mining sector. Whereas just six percent of the labor force works within the industrial sector, the vast majority – 85 percent – works in the agricultural sector. Rural incomes in agricultural sector have been relatively stagnant since 2000.24

25

An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

Background

Regional Integration

As a land-locked country, promoting regional integration and trade is particularly important to Zambia's development. Although many gains have been achieved, including the introduction of a one-stop border post with Zimbabwe in 2009, the quality of infrastructure and inefficient border administration continue to increase trade costs. The International Growth Centre U.K. found that on average, it requires 44 days and USD 2,678 for Zambian firms to export a standard container of goods by ocean transport.25 These high costs limit global competitiveness and discourage the entry of new firms, particularly in the manufacturing sector.

Access to Finance

The Zambian private sector is divided between large enterprises and MSMEs, with each group facing very different operating challenges. Industry surveys indicate that just a few thousand large businesses produce the majority of Zambia's industrial output and contribute the majority of tax revenues. The vast majority of businesses, however, are tiny, informal owner-operated MSMEs involved in rural agricultural production.26 Of the 4.1 million Zambians who are employed, 88 percent are estimated to work for these small enterprises.27

There is also a gap between larger enterprises and MSMEs in terms of access to formal finance. Only 11 percent of MSMEs use transactional products, such as bank accounts and money transfers, compared to

97 percent of large enterprises. Furthermore, only 2.3 percent of MSMEs were found to use formal credit products compared to 45 percent of large businesses.28 Increasing access to finance for small enterprises faces numerous barriers. First of all, few banks possess branches in rural areas. Furthermore, it is estimated that 95 percent of MSMEs are not registered with any government authority, restricting access to formal finance.29 In addition, most SME clients do not possess adequate collateral or financial documentation to qualify for a loan.30 However, low productivity and the high cost of financial services are perhaps the most serious barriers – it is estimated that only 7-8 percent of MSMEs generate enough revenue to afford a loan from a formal financial institution, which can carry interest rates as high as 20 percent and typically require collateral of up to 200 percent of the loan value.31

Vulnerability to Exogenous Shocks

Zambia's economic dependence on copper poses a serious economic challenge. During 2000–2014, copper exports represented an increasing share of Zambia's export revenue, accounting for 62 percent in 2000 and 78 percent in 2014.32 This increase is attributed to a rise in copper prices and a three-fold increase in copper production. Lack of economic diversification makes Zambia's economy vulnerable to fluctuations in commodity prices, as demonstrated by the most recent economic downturn. As copper prices fall, annual revenue and the value of the Kwacha follow suit, frustrating development planning and macroeconomic management.


Overview of Strategic Priorities

This evaluation covers three Country Strategy Papers (CSPs): (i) the 2002–2004 CSP, extended to 2006; (ii) the 2007–2010 Joint Assistance Strategy for Zambia (JASZ); and (iii) the 2011–2015 CSP. The Bank's strategic priorities for Zambia have evolved considerably over the evaluation period. The 2002–2004 strategy emphasized agricultural development, access to water supply and sanitation and the promotion of child welfare. Under the JASZ, strategic emphasis on infrastructure development to support the agricultural sector was retained in addition to the promotion of accountability and transparency in the management of public resources through the provision of general budget support. The 2011–2015 CSP signals a shift in the Bank's priorities, with infrastructure development now placing greater emphasis on regional integration, particularly with regard to transport and power infrastructure. The Bank's policy-based operations evolved to target private sector regulatory reform. These activities were complemented by private sector interventions aimed at increasing access to finance.

Project Portfolio

During 2002-2015 period, the Bank approved a

total of 43 operations amounting to UA 947.84 million, excluding cancelations. The portfolio of projects under review comprises 34 projects, four studies, three emergency operations and two technical assistance grant.33 In addition to these national operations, there were also 19 multinational operations amounting to UA 373.25 million which involved Zambia to varying degrees. The evaluation focused on national projects, examining regional projects only where there was a tangible national component. Over the years, the portfolio has grown in terms of number and amount committed (see Figure 1).

Nearly 45 percent of the Bank financing over the evaluation period has been done through the ADF window, closely followed by the ADB window at 43 percent. The remaining financing has been provided through NTF (0.9 percent) and other sources (11.6 percent). In total, loans represent 93 percent of net commitments with the remaining 7 percent in the form of grants. The majority of the Bank's support by value has been in the transport, power, WSS and agriculture sectors (31.7 percent, 19.3 percent, 14.3 percent and 10.9 percent respectively). However, agriculture, water supply and sanitation and transport are prominent in terms of numbers, followed by finance, power and multi-sector operations (see Figure 2).

Overview of the Bank's Country Strategies and Portfolio

27Overview of the Bank's Country Strategies and Portfolio

An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

Figure 1: Number of operations and amount approved (2002–2015)

2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 20150

50

100

150

200

250

300

0

1

2

3

4

5

6

7

8

21,30

1 1 1 1

4 4 4 4

7

5

3

2

6

1,3 0,4 66,8 0,6 60,6 66,0 94,0 96,7 264,8165,77,7 102,0

Amount (UA million) Number of operations

Figure 2: Number of operations and amount for sectors (2002–2015)

Amount (UA million)Number of operations

Environment

Ind / Mini / Qua

1

1

6

2

4

11

7

4

7

Finance

Social

Multi-Sector

Agriculture

WSS

Power

Transport

0 50 100 150 200 250 300 350

27,35

30,37

43,76

43,81

81,80

97,82

135,95

184,49

302,49

29Evaluation Approach and Methodology

An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

Evaluation Approach and Methodology

Evaluation Issues and Questions

The evaluation primarily addresses four evaluation issues: relevance, effectiveness, efficiency, and sustainability. However, other strategic issues relevant to the Bank's operations are also examined including: gender, inclusiveness and green growth; quality at entry; supervision; leveraging and innovation. These evaluation issues were used to identify a total of 29 evaluation questions (Annex A).

Methodology and Lines of Evidence

An evaluation matrix and theory of change were developed to guide data collection and analysis, including key indicators and judgment criteria, to provide a transparent account of how IDEV has examined the achievement of results across each sector. See Annex B for the evaluation matrix and intervention logic.

Evaluation issues and questions were addressed through triangulation of evidence from four different sources: (i) a review of project documents; (ii) a literature review; (iii) stakeholder interviews; and (iv) site visits. In total, the evaluation reviewed

over 200 project documents and other literature, collected feedback from over 175 stakeholders and beneficiaries and conducted in-depth site visits of 13 projects. A full listing of supporting documents, interviews and site visits conducted is provided in Annex C. For reporting purposes, each evaluation issue is assessed on a six-point scale ranging from Highly Unsatisfactory to Highly Satisfactory (Annex D).

There are some notable limitations to the evaluation methodology. First, it was often not possible to attribute and quantify the Bank's contribution to sector level outcomes. Furthermore, the evaluation faced challenges regarding the quality and availability of supervision data. Project Logframes were often found to lack baseline data or possess flaws in the intervention logic, such that supervision data did not credibly report on outcomes. Finally, due to time and logistical constraints, it was not possible to conduct detailed interviews with all beneficiary groups. To account for these challenges, the evaluation team sought to triangulate supervision data with evidence from other sources, including interviews, site visits, external literature and evaluative evidence from other DPs.


Evaluation Findings

Relevance

The Bank's CSPs have been well-aligned with the priorities set in GRZ's national development plans over the evaluation period and have responded to changes in these priorities over time. The Bank's project portfolio has generally been aligned with Country Strategies over the period as well as with the needs of targeted beneficiaries.

Alignment with National Development Plans

Over the course of the evaluation period, the Bank's CSPs have been well-aligned with Zambia's national development plans. Operational priorities are reflective of both national development priorities and the Bank's areas of comparative advantage. Furthermore, the Bank's CSPs have been responsive to changes in national development priorities over time. Provision of general budget support to address financial and budgetary transparency and accountability identified in the 2005-2006 CSP Update and the 2007 JASZ reflected the GRZ's increased emphasis on good governance expressed in the Fifth National Development Plan (FNDP).

The 2011-2015 CSP's emphasis on the development of economic infrastructure, including domestic and regional transport and power infrastructure, is reflective of objectives identified in the Sixth National Development Plan (SNDP), particularly with respect to increasing economic growth through removing infrastructure-related bottlenecks. Interventions targeting improved economic and financial governance reflect the priorities of Zambia's Vision 2030 and Revised SNDP with respect to increased

access to affordable finance, streamlined business registration and licensing processes and facilitation of regional trade. Alignment of the Bank's CSPs with National Development Plans is further illustrated in Annex F.

Alignment of Approved Projects with Country Strategies

Barring small exceptions, the alignment of Bank projects to CSPs has been satisfactory across sectors. Despite targeting job creation and economic development, the alignment of the Bank's private sector operations has been limited by the practice of identifying private sector operations opportunistically

Table 1: Relevance rating

Criteria RatingAlignment of CSPs with National Development Plans Satisfactory

Alignment of the Project Portfolio with CSPs Satisfactory

Alignment of Projects with the Needs of Beneficiaries Satisfactory

Overall Rating for Relevance Satisfactory

31Evaluation Findings

An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

under the 2002-2004 CSP and the 2007-2010 JASZ. Under the 2011-2015 CSP private sector operations are better integrated into the Bank's strategic priorities. For example, "supporting access to finance for SMEs through financial intermediation" is meant to contribute to the strategy outcome of improved economic and financial governance.

Social sector projects have not always reflected the strategic priorities of the CSPs. Although the 2002-2004 CSP targeted increased access to basic services, this objective was expressed in terms of water supply and sanitation infrastructure. Similarly, the health and education components of the first Poverty Reduction Budget Support (PRBS) project also aligned indirectly to the 2002-2004 CSP by promoting decentralization of service delivery.

Two interventions in the social sector were approved under the 2011-2015 CSP, although support to vocational training and skills development is not explicitly mentioned as an operational priority in the strategy itself. The Support to Science and Technology Education Project is identified in the CSP Logframe to address skills gaps and reduce barriers to private sector development. The health and education components of the Fourth PRBS project, however, are not clearly aligned with the Bank's CSP over this period.

Alignment of Projects with the Needs of Beneficiaries

Except for a few exceptions, the alignment of the Bank's projects with the needs of beneficiaries has been satisfactory. Projects in the transport, multi-, social and power sectors have addressed the alleviation of development constraints at the national or regional level. In contrast, projects in the agriculture, WSS and private sector targeted the needs of specific beneficiary groups.

Alleviating National and Regional Development Constraints. The poor quality of the transport network is a serious constraint for private sector development with transport challenges adding 40 percent to the cost of production for exports.34 The Bank's projects in the Transport Sector have sought to increase access to markets and economic growth by providing reliable, seamless transport infrastructure along regional trade routes. For example, the Nacala Corridor Road Project will connect Zambia's major productive centers to the Port of Nacala in Mozambique while the Kazungula Bridge and Chinsali-Nakonde Corridor will connect the Copperbelt to the Port of Dar es Salaam.

The Itezhi-Tezhi Power Generation project addresses the ongoing power crisis in Zambia by increasing installed generating capacity. However, the fact that the Bank's investments have targeted hydropower brings up more nuanced issues. Hydropower is the cheapest means of power generation and is better suited to Zambia's current tariff structure. However, the current energy crisis is partly due to an overreliance on hydropower, exacerbated by declining water levels in dam reservoirs due to reduced rainfall. The Bank's complementary Cost of Service Study is particularly relevant to the GRZ's need to achieve cost-reflective tariffs and promote private investment in other sources of clean energy generation, including solar.

For the PRBS Projects, beneficiary needs were expressed in terms of financing requirements for the implementation of the Fifth and Sixth National Development Plans. Each individual project identifies expected budget shortfalls over the period and the extent to which the shortfall would be addressed through general budget support. With respect to skills development, the Support for Science and Technology Education Project (SSTEP), responds to a request for Bank Group support from the GRZ to support a nation-wide program to create one million jobs for youth and enhance skill development in agriculture, mining, tourism and energy.


Targeting Specific Beneficiary Groups. Each of the Bank's agricultural projects have addressed the needs of specific groups of beneficiaries in terms of increasing productivity, improving food security and reducing poverty. For example, the SIP addressed development needs arising from the resettlement of the Tonga Community during the construction of the Kariba Dam in the 1950s. Displaced farmers were engaged in rain-fed subsistence farming on marginal lands. The CWMIP selected beneficiary communities on the basis of agricultural productivity below the national average despite being endowed with favorable climate conditions, fertile soil and water resources.

Similarly, beneficiary districts for Bank's WSS projects were selected based on particularly low levels of access to improved drinking water sources and sanitation facilities. For example, communities covered under the NRWSS Project were found to have average rates of 21percent and 4percent for drinking water sources and sanitation facilities, respectively. In the case of the Nkana WSS project, the intervention was meant to increase the ability of the Nkana Water and Sewerage Company to provide services to fast-growing urban centers within its area of operations.

Private Sector projects were well-aligned to the needs of targeted companies and financial intermediaries (FIs), but did not always fully address the needs of SMEs. The Lumwana Mine addressed the needs of Equinox Minerals, the project sponsor, by addressing the burden of major infrastructure constraints which raise development costs for mining operations. The Bank's participation in the project also secured the participation of other lenders,

allowing the investment to proceed. Meanwhile, Lines of Credit (LOC) Projects were framed in terms of the business goals of the recipient FIs, including business expansion, increased lending to SMEs in particular sectors and introduction of new SME-targeted commercial lending products.

LOC Projects and the Partial Credit Guarantee Facility (PCGF) have been less successful in targeting the needs of beneficiaries due to the absence of means to improve the terms of finance. Cost of finance remains one of the fundamental challenges for small businesses.35 Furthermore, it was never determined whether intended beneficiaries such as microfinance institutions would meet the due diligence requirements of the targeted financial intermediaries (FIs), such that some intended beneficiaries did not access to the project funds.

Effectiveness

Achievement of project outputs for completed projects across each sector has been moderately satisfactory, with 78 percent of outputs delivered successfully. However, achievement of project outcomes has been moderately unsatisfactory overall, with progress demonstrated toward 67 percent of outcomes.36 At the strategy level, the Bank's interventions have made a tangible contribution toward the development of an enabling business environment and increasing access to basic infrastructure and services. Yet limited progress has been achieved in strengthening public financial management and promoting agricultural productivity and diversity. Overall progress is rated as moderately unsatisfactory.

Table 2: Effectiveness rating

Criteria RatingDelivery of Project Outputs Moderately Satisfactory

Achievement of Outcomes Moderately Unsatisfactory

Overall Rating for Effectiveness Moderately Unsatisfactory


An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

Private Sector

Across the Bank's private sector interventions, 87 percent of outputs were delivered as planned. Information from the project XSR, confirmed by site visits, demonstrated that all expected outputs were delivered for the Lumwana Copper Mine, including development of the mine, infrastructure improvements, and environmental and social activities. However, just 84 percent of outputs were delivered for financial sector projects. Meanwhile, for two of the LOCs, funds were not disbursed to the targeted number of SMEs so consequentially some target beneficiaries, such as microfinance institutions, did not access the funds. In the case of CETZAM Financial Services, 46 percent of the funds were used to cover operating costs outside the scope of the loan agreement.37 Delivery of technical assistance (TA) to FIs as part of the LOC projects was more successful in terms of adding new skills and systems. This included training of business development service providers FI staff in credit assessment techniques for SMEs. These successes are worth replicating. However, the installation of LOANCOM and provision of training in environmental impact assessment were less successful due to lack of demand among targeted FIs.

Expected outcomes among private sector projects included: (i) increased government revenues; (ii) job creation; and (iii) profitability of the companies supported. The Lumwana Copper Mine was expected to improve access to basic services and infrastructure in the Lumwana and Solwezi districts whereas outcomes specific to LOC projects include: (i) increased SME lending; and (ii) improved terms of finance.

With respect to Government Revenues, EITI Reconciliation reports indicate that the Lumwana Mine has generated 3.5 billion ZMW in government revenues since 2011,38 whereas subproject data

suggest that the SMEs which benefited from the LOC to ZANACO and Investrust have contributed at least 1.7 billion ZMW.39

In terms of job creation, Private Sector interventions have been linked to the creation of at least 5,700 direct jobs. Supervision documents indicate that LOC projects contributed to job creation among the targeted FIs (425 jobs) and businesses which accessed the LOC (1,007 jobs).40 With respect to indirect job creation, Barrick has concluded local service delivery contracts in excess of USD 500 million with businesses in the Lumwana area as part of its Local SME Development Program – an initiative undertaken independently as part of Barrick's Corporate Social Responsibility activities.41 The International Committee for Mines and Minerals (ICMM) estimates that for every direct mining job created in the Solwezi area, four additional jobs have been created elsewhere in the economy.42

Profitability of the companies supported by the Bank has been uneven. The Lumwana Mine went through a period of heavy losses which threatened continued operations in 2012. The mine returned to modest profitability in 2013 after the implementation of cost-cutting measures.43 For LOC projects, subproject data demonstrated that the majority of enterprises which received loans have been profitable and, in the case of ZANACO, companies registered an average increase in profitability of 197 percent.

Works implemented as part of the Lumwana Copper Mine Project have increased access to basic services and infrastructure, including roads, electricity, ICT, and health services. Since the development of the mine, access to electricity in the Solwezi area has increased fivefold.44 For education, the teacher to student ratio has improved from 1:63 to 1:53 and secondary school attendance has increased to 45 percent, meeting the national average.45


Progress achieved in Increasing SME Lending was limited by the failure of CETZAM Financial Services, for which operations were taken over by the Central Bank of Zambia in May, 2016. However, supervision reports indicate that both ZANACO and Investrust saw increases in SME lending as a proportion of the overall lending portfolio from 25 percent to 30 percent of total loans for ZANACO and from two percent to 18 percent of total loans for Investrust. LOC projects were not successful in improving the terms of finance as average loan tenor and interest rates did not differ significantly from the rest of the FIs' portfolio. Furthermore, average collateral secured under the Partial Credit Guarantee Facility (PCGF) provided to ZANACO remained high at 183 percent. Project stakeholders confirmed that it was not possible to provide financing to SMEs on terms that are different from the rest of their portfolio.

Power Sector

Overall achievement of outputs for the power sector is rated as 80 percent. The Itezhi-Tezhi power plant was fully commissioned in January 2016, with accompanying transmission line packages commissioned in November and December 2015.46 However, the implementation of the Mumbwa and Lusaka West substations were still ongoing at the end of the evaluation. The Mumbwa substation was commissioned in November 2015 through the installation of a decommissioned transformer, enabling ZESCO to fulfil its power supply agreement with First Quantum Minerals. The planned Cost of Service Study, however, has not progressed due to an unsuccessful procurement process.47

With respect to outcomes, the Itezhi-Tezhi power project was expected to contribute to: (i) increased power generation; (ii) job creation; and (iii) increased access to basic infrastructure. With respect to power generation, the Itezhi-Tezhi Power Corporation (ITPC) reports that, subsequent to the commissioning of the plant in January 2016, initial

generation levels were approximately 50 percent of the expected level due to inadequate water level in the Kafue Gorge. However, as of May 2016, favorable weather conditions have since allowed the plant to reach its full generation capacity.

Implementation of the project has contributed to job creation, with stakeholders from ZESCO, ITPC and district governments confirming that members of the local community were hired for initial infrastructure works, brush clearing and erection of transmission towers. However, ensuring permanent local employment has been challenging given the skill requirements for operation and maintenance activities.

Works undertaken by ITPC have contributed to increasing access to basic infrastructure in Itezhi-Tezhi district, thereby promoting private sector development. A water treatment plant and transformers have been installed for use by the community and will eventually serve as a source of revenue for the district. Local government officials in Itezhi-Tezhi noted that the power plant has attracted potential investment from large companies, including Parmalat, but this investment has been delayed so far due to the inadequacy of transport infrastructure linking the district to major commercial centers. Further infrastructure upgrades would be useful in attracting additional investment to the area.

Agriculture Sector

The effectiveness of the Bank's agricultural interventions are rated moderately unsatisfactory. The majority of projects did not fully deliver the planned outputs and only moderate progress was made toward expected outcomes.

Completion of PRODAP outputs was rated at 34 percent, with ten percent of outputs considered to be fully implemented. For the SIP, delivery of outputs was rated to be 70 percent, with just four of six planned irrigation and outgrower schemes


An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

considered to be fully implemented (Nega-Nega (595ha), Sinazongwe (100 ha), Nzenga (95ha) and Buleya Malima (45ha)). To date, 200 ha of sugar at Nega Nega have been harvested in 2016 while planting of crops at Nzenga is on-going. However, improved institutional management entities were installed at the three sites and training in farming techniques, business planning, record keeping and HIV/AIDS awareness exceeded targets. All expected CWMIP outputs were delivered including: (i) value chain analyses; (ii) training of farmers in horticulture, irrigation, soil and water management; (iii) drilling of wells; (iv) establishment of demonstration gardens; and (v) training in business planning and finance.

Achievement of outcomes was uneven across the agriculture projects, with limited progress toward outcomes of PRODAP. The remaining projects, however, have made progress toward expected outcomes, including increased incomes, access to markets and increased use of improved irrigation methods. In the case of PRODAP, no changes were observed for health and social outcomes, including incidence of waterborne diseases or literacy rates among the project beneficiaries. Fish production within the project area has actually decreased by 58 percent since 2004 and the rate of post-harvest losses has remained unchanged. Although a 25 percent increase in the incomes of fisherman in the area was reported, it was not clearly attributable to the project.48 On account of these challenges the project was cancelled prematurely and a new successor project for the development of Lake Tanganyika has been launched.

With respect to access to markets, the Manyonyo irrigation scheme was able to secure a partnership agreement for sugarcane with Zambia Sugar while the Nzenga irrigation scheme was able to secure a similar market arrangement with a South African outgrower company to grow chilies and potatoes. Beneficiaries of CWMIP were able to access finance to grow their businesses with 1,193 farmers obtaining loans through a partnering FI.49

The CWMIP contributed to the increased use of improved irrigation methods among beneficiaries. Cultivated land in the project area increased by 80 percent and fewer beneficiaries relied solely on bucket and drip irrigation. In total, irrigated land in the project area increased by an average of 123 percent per household.50

Beneficiaries of both the SIP and CWMIP demonstrated increased incomes. Annual incomes for farmers within the Buleya Malima irrigation scheme increased from 479 ZMW in 2004 to 11,363 ZMW in 2014 (2,372 percent), whereas in Nzenga, annual incomes increased to an estimated 20,000 ZMW. For the CWMIP, farmers in Mkushi, Masaiti, Kapiri Mposhi and Chingola reported that their income had, on average, increased by a factor of eight.51

Multi-Sector and Social Sector

The Bank’s Poverty Reduction Budget Support (PRBS) program has a large social component in addition to reforms targeting improved Public Financial Management and private sector regulatory reform.

Delivery of outputs for the PRBS was satisfactory in terms of the implementation of identified triggers and reforms.52 Performance was particularly strong for program components which sought to strengthen transparency functions (100 percent), improve access to health and education services (95 percent) and create a business enabling environment (95 percent). However, progress was far less satisfactory for outputs related to PFM (37.5 percent), including the implementation of IFMIS and the Treasury Single Account (TSA).

With respect to governance components of the PRBS Program, key achievements include: (i) new procurement policy and procurement guidelines; (ii) an e-registry of business licenses; and (iii) approval


of a national anti-corruption plan. The GRZ also delivered a number of outputs in the health and education sectors, including: (i) hiring of 5,000 frontline health workers; (ii) the construction of eight hospitals and 231 health posts; and (iii) the construction of 1,000 basic schools and 11,000 classrooms and the hiring of 13,000 teachers. Furthermore, releases from the Ministry of Health to District Boards increased from 13 percent to 16 percent of annual budgets.53

Overall, the achievement of outcomes for the Bank's PRBS operations in the multi- and social sectors is rated moderately satisfactory. Progress was achieved toward strengthening accountability and transparency functions, improving the business and regulatory environment and increasing access to health and education services. However, limited progress was achieved in terms of strengthening budget credibility and execution.

Progress has been achieved in strengthening the GRZ's accountability and transparency functions including anti-corruption, procurement and external audit. Implementation of the National Anti-Corruption Plan has contributed to improved scores on international indices of accountability and corruption, including Transparency International's Corruption Perception Index and the Transparency, Accountability and Corruption index of the CPIA (see Figures 3 and 4).

With respect to external audit, annual audit coverage has increased to 75 percent of public expenditure and 50 percent of public entities.54 Detailed follow-up on audit recommendations has also improved, reaching 99 percent of recommendations in 2010. However, concerns persist regarding the independence of the

OAG, particularly with regard to the inadequacy and unpredictability of financial resources and insufficient staffing.55

Progress in strengthening the national procurement system was more modest. With the implementation of the new Public Procurement Policy, the Zambia Public Procurement Agency (ZPPA) was empowered to seek written justification for the use of less competitive procurement methods and conduct inspections of the procurement activities of MPSAs. However, there continues to be a lack of data on the use of less competitive procurement methods and Zambia continues to lack an impartial complaints procedure. The Auditor General has noted that irregularities continue to be found for the majority of procurement processes due to a lack of understanding of procurement guidelines.56 The extent of the progress achieved is demonstrated by a progression in PEFA scores on indicators PI-19 "Competition and value for money in procurement" and PI-26 "Scope, nature and follow-up of external audit (see Table 3 below).

Promotion of private sector regulatory reform has been more successful. Since 2006, notable reductions have been observed in the number of days to register a business, register a property and import goods, all of which have been targeted under the PRBS Program. The number of days to export, however, has remained relatively stable (see Figure 5). Some reforms have contributed to a genuine decrease in the cost of doing business. For example, the Business Compliance License List was reduced from 517 licenses to just 150, resulting in an annual reduction in the cost of business licensing by 32 percent.57

Table 3: PEFA Scores - Procurement and External Audit

Progress on Key PEFA IndicatorsIndicator 2005 2008 2012PI-19 Competition and value for money in procurement D+ C+ D+

PI-26 Scope, nature and follow-up of external audit B+ B B

Source: PEFA Reports


An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

Figure 3: Zambia Corruption Perceptions Index scores (2007–2015)

0

50

100

150

2007 2008 2009 2010 2011 2012 2013 2014 2015

123115

99 101

9188

83 85

76

Source: Transparency International

Figure 4: Zambia CPIA Scores - Accountability Transparency and Corruption (2007–2015)

0

2

4

6

2007 2008 2009 2010 2011 2012 2013 2014

3,50

4,00 4,00 4,004,17 4,17 4,17

3,83

Source: CPIA, World Bank Group


Progress has also been achieved in terms of increasing access to health and education. Targets were achieved for increased proportion of supervised childbirths, increasing from 46 percent in 2010 to 51.2 percent in 2012.58 Furthermore, the number of districts with pupil to teacher ratios over 80:1 fell from 47 in 2006 to 14 in 2010.59

Finally, limited progress has been made in strengthening Budget Credibility and Internal Controls. Although the implementation of internal control systems, including IFMIS and the Treasury Single Account have recently made some progress, PEFA Reports indicate that there continues to be "widespread" irregularities in commitments with

poor enforcement of regulations.60 PEFA scores for the key indicators (see Table 4) suggest that budget credibility and execution remain problematic.

Water Supply and Sanitation Sector

Delivery of outputs among the Bank's completed WSS projects has been satisfactory at 90 percent across projects. Outputs delivered include the construction and rehabilitation of water points, boreholes, water treatment plants, sewage infrastructure and latrines. Achievement of outcomes has been similarly satisfactory.

Figure 5: Ease of Doing Business Indicators (number of days)

0

10

20

30

40

50

60

70

80

2006 2008 2010 2012 2014

Register a property

Register a business Imports goods

Exports goods

73

4230

3526 25 24 24

18 18 18

7,5

53 5351 51 51

45 45 45

Source: World Development Indicators


An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

The Bank's activities have contributed to improved water infrastructure in over 25 different urban and rural districts, including the construction and rehabilitation of: (i) over 8,000 boreholes and water points; (ii) over 3,000 latrines and sanitation facilities; (iii) 13 treatment plants; and (iv) multiple elements of sewage infrastructure. Furthermore, the Bank has contributed to the establishment of a commercial utility serving multiple districts.61

Over 1,500,000 people benefitted from improved water sources implemented through the Nkana and 8 Centers and RWSSP projects, with over 95 percent of individuals in targeted areas having access to an improved water source. Furthermore, hours of service increased from an average of five hours per day to 16 hours for the Nkana project and 22 hours for 8 Centers project. With respect to sanitation, over 800,000 people gained access to improved sanitation facilities, including 9,924 individuals from unplanned settlements as well as public institutions such as health centers and bus stops. However, achievements in the sanitation components have fallen short of the targets, especially in the construction of on-site sanitation facilities. Progress in increasing collections was uneven with collection ratios declining for the Nkana project and failing to meet targets for the 8 Centers project. All of the completed projects noted a decrease in waterborne diseases with the Nkana and 8 Centers project documents reporting a 43 percent and 48 percent decrease in diarrhea and dysentery among project beneficiaries respectively. As a result

of the interventions implemented under the NRWSSP, incidence of diarrhea fell from 50 percent to 35 percent across the Northern and Luapula provinces.

Transport Sector

All the ongoing transport projects have faced considerable implementation delays for the delivery of outputs. About 50 percent of the physical works were completed for the Nacala Corridor project by the end of September 2015, but 95 percent of the original project timeframe had elapsed. The civil works are now expected to be completed in 2016. However, implementation of the ESMP has been moderately satisfactory with compensation provided to all affected kiosk owners, construction of improved market infrastructure, drilling of nine boreholes and provision of supplies to local health centers to support ongoing HIV/AIDS sensitization. However, three of the nine boreholes provided were no longer operational.62

The Kazungula Bridge project has delayed due to a procurement issue. A dispute over the contract award resulted in the withdrawal of funds by JICA for the bridge works. The Bank is continuing to fund other aspects of the project, excluding the bridge works, due to the issues arising with procurement. The bridge works are now underway and progress has been made in the resettlement of project affected persons. At the end of the evaluation period, however, just 0.98 percent of the project funds were

Table 4: Progress on key PEFA indicators

Progress on Key PEFA IndicatorsIndicator 2005 2008 2012PI-1 (i) Budget Credibility - Aggregate expenditure out-turn compared to original approved budget

C B D

PI-16 Predictability in the availability of funds for the commitment of expenditures

D+ D+ C+

PI-20 Effectiveness of internal controls for non-salary expenditure C C+ C+

Source: PEFA


disbursed, 41 months subsequent to the approval of the projects. As the projects are still ongoing due to the delays a credible assessment of outcomes is not possible. Nevertheless, these project are regarded as likely to achieve their intended outcomes.

Policy Dialogue and Knowledge Work

Prior to the opening of ZMFO, the Bank was not able to participate effectively in regular policy dialogue with the GRZ. However, since 2006, the Bank became an active participant in the donor coordination efforts in Zambia, leading the Cooperating Partners Group troika in 2012, as well as leading the agriculture, transport and WSS working groups at various points over the 2011-2015 CSP period. The Bank is also noted by DPs as a prominent dialogue partner for the PAF working group under the PRBS and regard it as possessing a privileged relationship with the GRZ regarding policy dialogue.

Delivery of knowledge work has intensified over the evaluation period, with a total of nine

works implemented or ongoing over the course of the evaluation period. There is some evidence that ZMFO has used knowledge work to inform and support policy dialogue, hosting a seminar on "Jobs and Growth" in 2014 and engaging in a High Level Policy Dialogue on youth employment in 2012.

Knowledge work has also been used to add value to the Bank's operations. The Nacala Corridor and North-South Corridor Studies are informing the implementation of the Nacala Corridor Road and the Kazungula Bridge projects. The Cost of Service Study for the energy sector will help address risks posed to private sector power investments such as the Itezhi-Tezhi power project by the lack of cost-reflective electricity tariffs. Delivery of knowledge work, however, has often not corresponded with items planned in the CSP (see Table 5 below).

Achievement of Strategy Outcomes

The strategy-level outcomes reflective of the CSPs for Zambia include: a) creation of an enabling business environment; b) transparent and accountable

Table 5: Planned and Delivered Knowledge Work

Proposed 2002-2006 Delivered 2002-2004 ❙ Public Expenditure Review

❙ Country Financial Accountability Assessment

❙ Study on Financial Management of HIPC Resources

None

Proposed 2007-2010 Delivered 2007-2010 ❙ Study for the Creation of a Free Trade Area (COMESA)

❙ Public Expenditure Review of the Agriculture Sector

❙ Nacala Corridor Studies (2010)

❙ SADC North-South Corridor Study (2010)

Proposed 2011-2015 Delivered 2011-2015 ❙ Support for the Preparation of the Zambia Transport Sector Master Plan

❙ Domestic Resource Mobilization for Poverty Reduction

❙ Zambia Private Sector Profile

❙ Constraints Facing Woman Entrepreneurs and Cross-border Traders

❙ Gender-responsive Audit of the Land Resources and Tenure System

❙ 2013 Update of Labor Force Survey (2013)

❙ Study on Zambia's Domestic Resource Mobilization Potential (2014)

❙ Zambia Private Sector Profile (2015)

❙ Zambia Manufacturing Study (2015)

❙ Preparation of a Multi-modal Transport Sector Master Plan (ongoing)

❙ Energy Cost of Service Study (ongoing)

❙ Study of Zambia-DRC Trade (ongoing)

Source: 2005, 2010, 2015 CSP Completion Reports


An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

public financial management; c) increased access to basic services; and d) increased agricultural productivity and diversification. The available evidence suggests that the Bank has made a tangible contribution to improving the business environment and increasing access to basic infrastructure and services. However, limited progress has been achieved with respect to strengthening PFM and promoting agricultural diversification.

Improved Business Environment

Contribution to improving the business environment was assessed in terms of: (i) ease of doing business; (ii) access to finance; and (iii) business development. These outcomes were supported directly by the Bank's interventions in the private sector as well as through the PRBS.

Zambia's Ease of Doing Business ranking has improved from 92 in 2006 to 83 in 2014.63 Similar improvement has been observed for the CPIA Business Regulatory Environment Index, which improved from 3.0 in 2004 to 4.17 in 2015. Improvement in these scores is partially attributable to reductions in the time necessary to register a business, register a property, import and export, all of which were addressed under the PRBS program. As a reflection of the improved business environment, formal business registration has increased and intensified, with 5,318 new businesses having registered in 2007 and 10,199 in 2015 (see Figure 6).

Regarding access to finance, domestic credit to the private sector (percent GDP) has increased during the evaluation period (see Figure 7). Furthermore, the proportion of private firms using banks to finance investment increased from 10.2 in 2007 to 17.2 in 2013.64 However, the extent of the Bank's contribution to these changes was modest, given that fewer than 150 businesses are confirmed to have accessed finance through the Bank's LOCs to Investrust

and ZANACO and total disbursements of just 13.5 million USD. It was unclear that these projects were successful in reaching underserved groups. Although the PCGF was successful in expanding ZANACO's agriculture portfolio, subprojects under the PCGF and the LOC were dominated by Lusaka-based businesses (65 percent and 70 percent respectively). Feedback from ZANACO confirmed that loan recipients were all identified from its existing SME clientele, such that only one PCGF recipient was identified as a ‘first-time borrower’ and only two businesses were identified as ‘women-owned.

Another missed opportunity is the TA provided to Business Development Service Providers and Business Associations. Whereas 15 percent of the initial 105 businesses which received the training reported to have received new financing with 11 having received loans from Investrust,65 there was no long-term monitoring of training recipients which would identify whether the training has indeed enabled them to eventually access finance. Opportunities were also missed in increasing the impact of the Bank's large private sector investments in Zambia, namely the Lumwana mine and the Itezhi-Tezhi power plant. Although these two investments have contributed to local business development by increasing access to services and implementing local supplier development programs, additional large business investments were being prevented by the poor state of transport infrastructure connecting the project sites to major trade centers. Both ITPC and Barrick have implemented comprehensive Corporate Social Responsibility activities which are addressing development constraints in the surrounding communities. Complementary support from the Bank to upscale these activities and further address constraints to private sector development may increase the catalytic potential of the Bank's projects in terms of business development. However, such opportunities are also limited by the small number of private sector investments in


Figure 6: No. of new businesses registered during 2007-2015

2000

4000

6000

8000

10000

12000

2007 2008 2009 2010 2011 2012 2013 2014

3648

5318

6284

5505

6941

8540

9682

10199


Figure 7: Domestic credit to the private sector (percent GDP) 2006-2014

0

5

10

15

20

2006 2007 2008 2009 2010 2011 2012 2013 2014

Domestic Credit to the Private Sector (% GDP)

8,2

9,7

12,2

109,2

13,5

16,2 16,517.1


An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

Zambia which meet the Bank's minimum threshold for project value.

Transparent and Accountable PFM

Contribution to transparent and accountable PFM has been assessed in terms of: (i) gross government debt; (ii) budget deficits; and (iii) public sector wage expenditure. Between 2006 and 2011, the Bank's efforts to strengthen macroeconomic management through general budget support projects appears to have contributed to reductions in gross government debt and reduced annual deficits. However, gross government debt has since increased, reaching 35.1 percent of GDP in 2014. Similarly, annual deficits have returned to 2004 levels, reaching 10.6 percent (see Table 6 below). Furthermore, annual wage expenditure has accounted for an increasing share of GDP and, in raw terms, has increased by more than 400 percent, from 2,968 billion ZMW in 2006 to 11,897 billion ZMW in 2014.66 These changes coincide with a decline in scores on financial governance indices, including CPIA scores for Fiscal and Debt Policy, which each reduced from 4.5 in 2009 to 3.0 in 2015.67

Increased Access to Basic Services

Bank's interventions in the WSS, power and social sectors have made a tangible contribution to increasing access to these basic services.

National trends for access to improved water and sanitation sources have been positive, with the percentage of Zambians with access to an improved water source rising from 55 percent in 2002 to 65 percent in 2014. These gains were largely due to increased access in rural areas, minimizing the disparity between urban and rural access to water. In contrast, limited improvement has been seen in access to improved sanitation (see Figure 8).

The Bank's contribution to these changes has been sizable. Based on urban and rural population data for 2002 and 2014, at the national level, 2.05 million people in rural areas and 2.15 million in urban areas have gained access to an improved water source over this period. Meanwhile, the Bank's completed projects in the water sector have benefitted over 1.5 million people.

With respect to access to electricity, the Bank has contributed 120 MW to Zambia's installed generation capacity through the implementation of the Itezhi-Tezhi power project. This contribution is moderate considering total installed capacity of approx. 2,300 MW.68 Although these levels currently outstrip demand, power is being produced far below capacity due to low water levels in dam reservoirs. The size of the current power deficit is estimated to be 500 MW.69 There is a clear need to expedite investment in the sector. The Ministry

Table 6: Progress against selected macroeconomic indicators

Indicator 2004 2006 2008 2010 2012 2014Real GDP growth (non-oil) 5.4 6.2 5.7 7.6 7.2 5.6

Gross Government Debt (percent GDP) 78.0 27.4 26.7 22.1 25.5 35.1

Overall Budget Balance (percent GDP) -10.3 -8.3 -4.9 -3.0 -3.2 -10.6

Annual Arrears (percent of annual expenditure)

12.0 6.4 1.9 3.7 1.5 2.5

Wages and salaries (percent public expenditures)

29.4 31.6 34.1 41.9 34.0 38.7

Sources: IMF World Economic Outlook, World Development Indicators


of Energy estimates that demand is increasing at the rate of 150-200 MW per year due to growth in economic activities.

Although access to electricity has increased by 11 percent since 2000,70 the timing of the Bank's investments precludes any direct contribution. Based on feedback from ITPC and ZESCO it is expected that the project will contribute to increased access to electricity for districts surrounding the power plant, the Mumbwa substation and the Karumbila mine resulting from the construction of new public infrastructure works. However, the power generated by the plant will primarily supply the Karumbila mine, for which energy requirements are expected to outstrip the maximum generation capacity of the plant at 200 MW.

National health and education outcomes identified as part of the Bank's PRBS activities include: (i) increased childbirths assisted by a skilled attendant; (ii) infant immunization rates; and (iii) primary school completion rates. Progress has been made against all of these indicators. Furthermore, the gap in primary school completion rates between boys and girls has nearly vanished (see Table 7).

However, other indices for access to health services suggest that additional efforts are necessary. Zambia continues to possess less than half of the WHO recommended Human Resources for Health (HRH) at 0.06 physicians per 1,000 people and 0.77 nurses and midwives per 1,000 people.71 Furthermore national pupil to teacher ratios were stable at 48:1 during the evaluation period.72

Figure 8: Access to improved water and sanitation source

0

20

40

60

80

100

2002 2014 2002 2014Water Supply Sanitation

87

37

55

65

4144

51

3236

86

57 56

RuralUrban National average



An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

Agricultural Productivity and Diversification

The Bank has sought to promote economic growth and reduce poverty by increasing agricultural productivity and trade. National statistics suggest that although overall agricultural exports have increased, little progress has been made in agricultural diversification. Agriculture as a share of total exports has decreased from 11.54 percent in 2002 to 7.49 percent in 2014. However, overall exports have increased by a factor of 10, such that the gross value of agricultural exports has actually increased by over USD 600 million.73

There is also evidence of increased land cultivation and use of improved irrigation techniques. Between 2008 and 2012, total hectares cultivated increased by 16 percent, whereas total irrigated land in Zambia increased by seven percent between 2001 and 2011, suggesting that some gains have been achieved for agricultural productivity.

However, there is limited evidence of agricultural diversification away from the production of maize. The Indaba Agricultural Policy Research Institute reports that production of maize among smallholder farmers remained relatively stable between 2008 and 2012 at 60.6 percent and 61 percent of cultivated land respectively.74 Values from the Simpson Diversity Index over this period confirm these data with Zambia assessed at 0.42 in 2004 and 0.4 in 2012, suggesting limited progress over the period.75

The DPs reported that promoting agricultural diversification was stymied by the Farmer Input Support Program (FISP), a national program which provides subsidized inputs of fertilizer and maize seeds to smallholder farmers with crops subsequently

purchased by the Food Reserve Agency. Together, these two programs are estimated to account for nearly 80 percent of expenditures by the Ministry of Agriculture and Livestock. The program has also demonstrated history of inefficiencies, including overspending, poor targeting of beneficiaries, leakages and delays which limit overall productivity and income gains.76 Furthermore, DPs reported that the program serves as a disincentive for diversifying agricultural production away from maize. In 2015, however, the program has shifted to a voucher system, allowing Zambian farmers to select the inputs of their choice.

Efficiency

Overall, the efficiency of the Bank's portfolio is rated as unsatisfactory owing to considerable project implementation delays and portfolio disbursement ratios which have consistently fallen below the Bank-wide average over the evaluation period. Economic Rate of Return for the Bank's projects was not rated due to unavailability of adequate data (available for only seven projects).

Timeliness

The average project implementation ratio for the Bank's completed projects stands at 1.68, suggesting an eight month delay for every planned year of project implementation. Across the portfolio, implementation delays have been caused by four key challenges: (a) inadequate project design; (b) delays in achieving loan effectiveness; (c) procurement delays; and (d) complexities associated with co-financing.

Table 7: Progress on selected health and education indicators (2002-2014)

Indicator 2002 2014Births assisted by a skilled attendant (percent) 42 64

Children 12-23 months fully immunised 60 69

Primary school completion - male (percent) 57 81

Primary school completion - female (percent) 52 80



Inadequate Project Design

Inadequate project design has been particularly problematic in the agricultural and transport sector. Project design deficiencies have resulted in underestimates of project cost and required implementation time as well as time spent on "re-engineering." The SIP was designed without detailed feasibility studies. As such, the project was considerably underfunded. Original cost estimates has identified implementation costs of USD 4,000/ha but the actual cost was USD 21,000/ha.77 Initial funds were sufficient to implement just one of six planned irrigation schemes. The project was eventually implemented over ten years with an initial extension of Bank funds to 2009 as well as additional trust funds. Board approval of agricultural projects should be preceded by detailed feasibility and design studies if project implementation delays are to be avoided. In the case of the Nacala Corridor Road Project, the original project design had considered construction with limited diversions, resulting in both delays and additional costs as the project had to be re-engineered to include necessary diversion works. At the time of the evaluation, just 50 percent of project funds had been disbursed, whereas 95 percent of the original project timeframe had elapsed.

Loan Effectiveness and Disbursement

Delays in achieving loan effectiveness and first disbursement were noted across the portfolio. The average delay between project approval and loan effectiveness over the evaluation period was found to be 415 days (13 months), whereas the average

time from loan effectiveness to first disbursement was 175 days (five months). Overall, this timeframe exceeds standards set in the Bank's Operations Manual, which identifies a timeframe of 14 months between approval and first disbursement. However, the 2015 CSP Completion Report (CSPCR) suggests that delays between project approval and first disbursement have improved for projects approved over the 2011-2015 strategy period with delays falling from 16 months in 2011 to 12 months in 2014.78 The 2013 Development Effectiveness Review for Zambia confirmed that time between approval and first disbursement had fallen to 12 months. This timeframe is in line with the average across the ADF countries but falls short of the target of 11 months identified in the report.79 Two sectors of particular concern are the agriculture sector and the private sector. In the case of the agriculture sector, PRODAP did not reach effectiveness until a full 4.4 years subsequent to approval. Among LOC Projects, average time between project appraisal and loan signing was nearly 20 months, and 25 months between appraisal and first disbursement. For the LOC to ZANACO, it took 20 months from the time of appraisal to loan signature and 26 more months for disbursement.

These delays affected the achievement of project objectives. For example, the financial position of CETZAM deteriorated considerably between the time of appraisal in 2012 and the time of loan signature in September 2014. The deterioration was not monitored over this period based on the Bank's practice of supervision only after disbursement. Had the Bank been aware of this deterioration, it may

Table 8: Relevance rating

Criteria RatingTimeliness of Project Implementation Unsatisfactory

Financial and Economic Performance Moderately Unsatisfactory

Overall Rating for Relevance Unsatisfactory


An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

have been possible to mitigate the risk. Despite the fact that CETZAM was already in default with the Development Bank of Zambia, Bank staff indicated that financial records were not sufficiently scrutinized, and CETZAM was cleared for first disbursement. The Bank became aware of the extent of CETZAM's difficulties only through feedback from the firm hired to provide TA. In the case of Investrust, disbursement delays for TA resulted in the delay in improving credit assessment systems which meant that the majority of the LOCs had already been disbursed. As such, it is unlikely that sub-projects had been assessed using improved systems.

Complexities of Co-financed Projects

Closely related to the issue of delayed loan effectiveness and disbursement is the issue of harmonizing practices with other donors for co-financed projects. This issue was of particular concern for two of the Bank's projects: (i) the Itezhi-Tezhi Power Generation and Transmission Project; and (ii) the Kazungula Bridge Project.

Itezhi-Tezhi involves innovative financial design in which the AfDB cooperates with ZESCO and six other donors to create a special purpose vehicle for power generation. The project involved public and private sector loans from the ADF and ADB windows and the Nigeria Trust Fund. The difficulties experienced in this case arose partly from the incompatibility between the Bank's public and private sector project approval processes. Since the Itezhi-Tezhi project involved both private and public sector loans, effectiveness of the public sector loan was delayed until reconciliation of terms could be completed for the private sector loan, which occurs after approval. Project stakeholders reported that an unwillingness of all parties to compromise on their terms and conditions for the private sector loan resulted in a 29-month delay for loan effectiveness and a 36-month delay to first disbursement.

Through the goodwill demonstrated by ZESCO and TATA Construction, works were expedited, limiting the overall project delay to just five months. However, these delays, partly attributable to the Bank, have come at a cost to the project beneficiaries. Expedited works have resulted in additional costs of over USD 7 million in variation orders as well as USD 1 million in interest for bridging funds.

Similar issues were evident in the implementation of the Kazungula Bridge project, which highlighted potential challenges of working with bilateral partners. As an exception, the Bank chose to adopt JICA's procurement rules and differences arose between the parties with respect to the appropriate technical requirements for the bid. JICA disputed the award of the contract and withdrew their support for the bridgeworks. The resulting delay has been significant, with less than one percent of funds disbursed 39 months after project approval.

Procurement Delays

Procurement delays were noted for multiple projects across each sector. These delays have resulted from: (i) weak procurement capacity within line ministries; and (ii) heavy internal government approval processes.

Procurement capacity within line ministries is weak owing to lack of professionalization of the procurement function, high staff turnover and an unfamiliarity with the Bank requirements. Within sector ministries staff are sent for procurement training at the Institute of Purchasing and Supply. However, this training is better suited to private sector procurement. Although ZMFO has been proactive in providing workshops for GRZ counterparts on the Bank's procurement system, these efforts are limited by high staff turnover. Project delays subsequently result from improperly prepared documents, which must be returned for revision. A second source


of delay is the requirement that all contracts be approved by the Office of the Attorney General (OAG). This requirement, along with the resource constraints within the OAG, were noted to have delayed project procurement by a matter of months.

Financial and Economic Performance

Financial and economic performance of the portfolio was assessed in terms of: (i) the disbursement ratio over the evaluation period relative to the average disbursement ratio across all Bank projects; and (ii) Economic Rates of Return at completion relative to appraisal. The overall disbursement ratio has improved over the evaluation period, from 83.06 percent for the 2002-2004 CSP to 96 percent for

the 2007-2010 JASZ and 93 percent for the 2011-2015 CSP. As of 2008, the annual disbursement ratios for Zambia has fallen slightly below the bank-wide average albeit there has been some recent improvement (See Figure 9 below).

With respect to the economic return, ex-post Economic Internal Rate of Return (EIRR) figures were available for only seven projects across the portfolio. As per the calculations available in PCRs, on average, the ratio between ex-post and ex-ante EIRR was 0.96, indicating that projects came close to meeting their expected rates of return. However, specific outliers can be noted, including the PRODAP Project, for which the ratio fell at just 0.06 percent and the NRWSSP, for which the EIRR far exceeded expectations at appraisal with a ratio of 1.57.

Figure 9: Disbursement Ratios – Zambia and AfDB (2008-2014)

0

10

20

30

40

50

60

2008 2009 2010 2011 2012 2013 2014

Disbursement Ratio ZambiaDisbursement Ratio Bank-wide

27,8

25

46

25

12

23

34

24

55,2

24,4

30,4 31,5

27,5



An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

Sustainability

The sustainability of the Bank's interventions was assessed in terms of the extent to which risks to the sustainability of project outcomes were addressed in project design and implementation, as well as the observed sustainability of outcomes. Sustainability was found to be variable across sectors, but moderately unsatisfactory overall, suggesting that the sustainability of the Bank's operational outcomes in Zambia is not assured.

Private Sector. The sustainability of private sector projects has been compromised by regulatory changes, as well as variable profitability among the targeted institutions, but it was found to be moderately unsatisfactory overall.

In 2012, the Lumwana Mine incurred USD 3.8 billion in impairment charges, but returned to modest profitability in 2013 as a result of cost-cutting measures, earning USD 87 million.80 The mine then temporarily suspended operations in December 2014 as a result of the GRZ's decision to raise royalty fees to 20 percent. It was stated that this would make the mine no longer profitable. Stakeholders at Barrick

noted that this shutdown reflects the continued sensitivity of the mine to changes in the operating context.

Medium-term profitability of FIs targeted through LOC projects was threatened by regulatory changes imposed by the Bank of Zambia. The project design for the three LOCs did not assess the sensitivity of operating profit margins to such regulatory changes. For example, CETZAM experienced a marked decline in profitability subsequent to the Bank of Zambia's

introduction of an interest rate and fee cap for microfinance institutions.

CETZAM previously charged an annual interest rate of 104 percent. With the cap placed at 42 percent, CETZAM could no longer retain sufficient income to sustain its business model. In addition, the quality of CETZAM's loan portfolio deteriorated, with Non-Performing Loans (NPLs) reaching 49 percent of loans. An effort to restructure CETZAM failed and by the end of 2014, CETZAM reported a Return on Assets of (-28 percent), Return on Equity of (-433 percent) and a Cost to Income Ratio of 376 percent.81 CETZAM has been in default with the Bank since July 31, 2015.

Table 9: Sustainability rating

Sector Sustainability Rating Private Sector Moderately Unsatisfactory

Power Sector Moderately Unsatisfactory

Agriculture Sector Moderately Unsatisfactory

Multi-Sector Unsatisfactory

Social Sector Moderately Unsatisfactory

WSS Sector Moderately Satisfactory

Transport Sector Moderately Satisfactory

Overall Rating Moderately Unsatisfactory


ZANACO and Investrust faced similar challenges when the Bank of Zambia decided to increase statutory reserve requirements to 18 percent.82 ZANACO has remained profitable with NPLs below eight percent, absorbing the increase by liquidating a portion of its bond portfolio and incurring a slight decrease in profits. Investrust, however, has faced difficulties. Since disbursement, Investrust's financial position gradually deteriorated, with NPLs reaching 40 percent in 2013. During 2009-2012, Investrust's operating income grew at 84 percent, but operating expenses more than doubled as a result of an aggressive expansion strategy. In this context, the raised capital requirement added pressure to an already difficult operating context. Furthermore, the project XSR notes that the Bank failed to identify weaknesses in Investrust's credit underwriting processes, which could have been addressed through TA. Although the LOC facility has already been repaid to the Bank, reported losses in 2013 and 2014 call into question whether Investrust's SME financing activities will be sustainable.

Agriculture. With respect to agriculture sector projects, the primary risks to sustainability stem from: (i) institutional sustainability; (ii) ownership by the beneficiaries; and (iii) exogenous shocks. Overall, sustainability was found to be moderately unsatisfactory.

The sustainability of the PRODAP project, notwithstanding poor performance with respect to outcomes, was limited by a failure at the outset of the project to: (i) assess the capacity of stakeholders in procurement, financial management and project management; (ii) determine the availability and capacity of local bidders and contractors; and (iii) ensure buy-in through engagement with local suppliers.

For SIP, the Ministry of Agriculture and Livestock has promoted institutional sustainability by establishing companies responsible for all aspects of growing and marketing of crops as well as operation and

maintenance. The African Management Services Company has been hired to provide organizational capacity development to further ensure institutional sustainability. Similarly, as part of the CWMIP, all farmers clubs were trained in group management, crop management, record keeping, water management and loan management. The project also facilitated the registration of participating farmers clubs as legal entities, expanding the range of business transactions these clubs can participate in to grow their businesses.

In the case of the SIP, ownership has been promoted through consultation with beneficiary farmers throughout project design and implementation. For example, the beneficiaries were involved in selecting the sites for new irrigation schemes. However, ownership of the schemes is threatened by the uneven distribution of revenues inherent in the irrigation scheme management structure. Furthermore, potential water issues resulting from climate change have not been satisfactorily addressed for SIP. Two schemes were found to be facing operational challenges due to low water levels in Lake Kariba and, as a result, one of the schemes has gone through periods of non-functionality.

Power Sector. The primary sustainability concern for the Itezhi-Tezhi power project has been the financial sustainability issues given that Zambia's energy tariffs continue to fall far below cost recovery levels. Although the operation of the Itezhi-Tezhi Power Plant is covered by a Power Purchasing Agreement (PPA) with ZESCO and supported by a guarantee from the GRZ, the lack of cost-reflective tariffs pose a serious risk to ZESCO's financial sustainability.

The Energy Regulation Board (ERB) notes that current PPAs range from USD seven cents/kWh to 13.23 cents/kWh whereas the average tariff is USD six cents/kWh.83 These costs do not take into account the ongoing maintenance and investment in the transmission network, nor do they address


An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

Table 10: Regional Tariff Levels

Tariffs Zambia Tanzania Swaziland ZimbabweUS cents/kWh 6.02 16.8 10.9 9.8

Lifeline kWh/ month 0-100 0-50 n/a 0-50

Source: Zambia Energy Regulation Board

system losses, theft and unpaid tariffs. Zambia's electricity tariffs fall far below those of other countries in the region (see Table 10 below).

Political will to move toward cost-reflective tariffs is lacking. In February 2016, the GRZ reversed a previous tariff increase to an average of USD 10.35 cents/kWh.84 As part of this agreement, the lifeline tariff would have been extended from 100 kWh to 300 kWh, while other commercial and residential consumers would pay higher fees. Although the Bank is currently funding a Cost of Service Study to support ZESCO in proposing a tariff increase, its potential impact will be subject to changes in the political environment.

Although the PPA is supported by a guarantee from the GRZ, the current macroeconomic context calls into question whether such agreements could be honored across multiple investments. In terms of constraints to the development of the sector, current tariff levels discourage the conclusion of new PPAs with private power producers and limit the viability of projects for other clean sources of energy, including solar.

Multi-Sector. The poor sustainability of outcomes from the Bank's PRBS program is attributed to institutional capacity risks and insufficient ownership. These risks were exacerbated by an overestimation of the ability of MoFNP to influence the activities of other ministries.

Weaknesses in technical capacity were identified as risks to project implementation for each of the PRBS projects. The risk was deemed mitigated through training provided to the MoFNP on various elements of budget planning and execution under the PEMFA

program. However, little was done to address the technical capacity gaps in line ministries responsible for the implementation of reforms.

Some donors suggested that, given Zambia's MIC status, continued high poverty levels are a reflection of poor ownership rather than insufficient resources. In the implementation of PRBS, ownership was found to be lacking in three areas: (i) implementation of IFMIS, ongoing since 2004; (ii) budget execution reforms; and (iii) decentralization.85 All three items featured in a High Level Dialogue in 2009 through which the PRBS group raised concerns about ongoing delays. More recently some progress has been made in implementing IFMIS and the TSA, but development partners note that continued irregularities and slow progress raised doubts as to whether the underlying changes are substantive or merely procedural.

Donors also expressed concern over variable levels of ownership across the Sector Working Groups. Although ownership of the PRBS agenda was strong within MoFNP, it was somewhat weak among the line ministries, some of which regarded the PRBS program and the Sector Working Groups as requiring that they cede autonomy to MoFNP.86

In the latter years of the Program, the GRZ gained access to the Eurobond market on favorable terms. These funds could be obtained quickly and were subject to fewer conditions. Access to the Eurobond market was identified by other donors as an additional factor underpinning insufficient ownership of budget support targets. However, recent loans have been more expensive,87 placing additional strain on the economy. In 2014, the PRBS Group eventually disbanded due to concerns that ownership and macroeconomic management were


insufficient to justify budget support. Furthermore, some donors doubted the appropriateness of budget support given Zambia's MIC status.

Social Sector. Although progress achieved at the outcome level under the fourth PRBS has been sustained, the mitigation of risks to sustainability at the project level was deemed moderately unsatisfactory. First, as noted above, line ministries including the Ministries of Health, Education and Local Government did not benefit from the TA provided to strengthen budget execution, project management and procurement. Political and governance-related risks identified during project appraisal were limited to the potential for a change in government in 2011. The project failed to address the misuse of funds identified in the 2009 OAG Annual Report with procurement irregularities constituting the most salient risk.88 Ongoing initiatives were underway to strengthen the Anti-Corruption Commission, but no actions were taken to mitigate this risk within the Ministry of Health itself.

Water Supply and Sanitation. The sustainability of outcomes for WSS projects has benefitted from the selection of simplified and uniform technical solutions to reduce the cost and complexity of maintenance as well as the establishment of commercial utilities and community maintenance committees to ensure continued operations. However, WSS projects have struggled to ensure financial sustainability of operations and reduce unaccounted for water. In the case of the Nkana WSS project, non-revenue water stood at 60 percent due to leakages, faulty billing and illegal activities. The 8 Centers project faced similar challenges in ensuring adequate collection of revenues and cost-recovery for water services. In the case of the NRWSSP, lack of consideration for the ability and willingness of project beneficiaries to pay for services has resulted in the failure to establish a local utility for operations and maintenance.

Transport Sector. Relevant risks to sustainability of the Bank's transport sector projects include financial sustainability and institutional capacity. Institutional risks have resulted from recent management challenges identified at the Roads Development Agency, for which a 2008 OAG Report identified over-procurement and contract management irregularities, specifically with regard to high unit prices paid to contractors.89 The report resulted in a forensic audit and dissolution of the RDA's Board in 2011while the Nacala Corridor Project was ongoing. The project addressed institutional capacity risks by providing TA for procurement, management of civil works and collection of baseline data. Independent audit and technical firms were appointed to oversee project implementation. For the Kazungula Bridge Project, these risks will be addressed through the creation of the Kazungula Bridge Authority, a separate agency which will be established after project implementation to own, operate and maintain the bridge.

Financial sustainability and maintenance of the Nacala corridor was addressed through EU-funded design studies which were used to identify the most cost-effective implementation option. The selection of an asphalt concrete paved carriageway is estimated to have reduced maintenance costs by two-thirds over other alternatives. In addition, the GRZ has adopted a Road Maintenance and Management Strategy (2014-2024) which provides a methodology for road maintenance planning and encourages the use of Output and Performance Based Contracting. Funds for maintenance of the road are to be ring-fenced under the National Road Fund Agency.

Crosscutting Themes

Coverage of crosscutting themes, including gender, inclusive growth and green growth has been moderately unsatisfactory over the evaluation period.


An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

Coverage of Gender within the Bank's Strategies. Gender has received limited coverage across the CSP periods and has been moderately unsatisfactory with uneven coverage of gender-specific development constraints and few targeted interventions identified. The 2002-2004 CSP and the Bank’ Multi-sector Country Gender Profile for Zambia (2006) identified specific gender-related issues within the development context, including increased poverty among female-headed households and lack of opportunities in education, asset ownership and economic activities. Initiatives to address these issues were to be incorporated into the Bank's projects through complementary grant funds. However, coverage of gender within the Bank's CSPs decreased after this period. No specific interventions pertaining to gender were identified under the 2007-2010 JASZ. Discussion of gender constraints under the 2011-2015 CSP has been limited to a lack of available resources for implementing the National Gender Policy. However, some specific gender-related interventions were identified, including building GRZ capacity to collect sex-disaggregated data, providing support for gender-responsive budget and conducting gender audits of key sectors in which the Bank is active (for example the proposed gender audit of the transport sector and inventory of women entrepreneurs). Significantly, none of the gender-specific knowledge work was eventually implemented.

Gender Mainstreaming across the Project Portfolio. With respect to the mainstreaming of gender into the Bank's portfolio of projects, gender has been mainstreamed into project design for the majority of projects by identifying employment and income generating opportunities for women. However, in some cases, efforts to promote income-generating activities for women were limited by project design omissions, whereas other projects missed opportunities in this regard. Overall, mainstreaming of gender into project design is rated as moderately unsatisfactory.

LOC projects identified female business owners as a key beneficiary group. It was expected that projects would increase lending to women-owned businesses. However, the project design did not allow for the specific targeting of women in that each loan was subject to the specific due diligence requirements of the financial institutions. Through the provision of TA, however, one financial institution developed a credit product which specifically targets female traders.

Integration of gender into the Bank's agriculture projects has been weak. For example, women are noticeably absent in decision-making positions among irrigation committees established under the SIP. Furthermore, for the CWMIP, treatment of gender was limited to an initiative to collect disaggregated data, which was ultimately unsuccessful. Integration

Table 11: Crosscutting Issue Rating

Criteria Rating Coverage of Gender within CSPs Moderately Unsatisfactory

Gender mainstreaming across the portfolio Moderately Unsatisfactory

Inclusive and Green Growth within CSPs Moderately Unsatisfactory

Inclusive and Green Growth across the portfolio Moderately Satisfactory

Overall Rating for Crosscutting Themes Moderately Unsatisfactory


of gender was more successful in the Water Supply and Sanitation sector, for which the Nkana and 8 Centers WSS projects promoted employment opportunities for women through income-generating activities, such as management of water points.

The integration of gender into industrial and infrastructure projects was uneven. The Lumwana Copper Mine created employment opportunities for women through: (i) the contracting of women's groups for specific services; (ii) implementation of income-generating activities; and (iii) direct hiring and training of women for industrial jobs (for example driving trucks). Similarly, initiatives targeting the capacity development of female contractors and business operators have been integrated into the design of the Kazungula Bridge, Nacala Corridor and Chinsali-Nakonde Road projects. However, no such initiatives were considered in the case of the Itezhi-Tezhi power generation and transmission project. The experience of the Lumwana Copper Mine project suggests that more can be done to promote direct employment of women for semi-skilled jobs in the context of infrastructure and industrial projects.

Inclusive and Green Growth within the Bank's CSPs. Coverage of inclusive and growth and environmental sustainability is most explicit under the 2002-2004 CSP but decreased across subsequent CSPs over the evaluation period. Overall, coverage of inclusive and green growth within the Bank's strategies has been moderately unsatisfactory.

The 2002-2004 CSP explicitly addresses the issue of high levels of rural poverty, noting that a lack of investment in rural areas has left rural populations detached from the direct gains of growth and lacking access to inputs, services and infrastructure. These challenges were to be addressed through the Bank's agriculture and WSS interventions, as well as policy based operations targeting decentralization of governance. Environmental concerns were

addressed in terms of industrial pollution in the Copperbelt region and environmental degradation caused by heavy reliance on farming in rural areas but no specific interventions were identified in this regard.

Coverage of inclusive and sustainable growth is less satisfactory in the 2007-2010 JASZ and 2011-2015 CSP. Both strategies promote poverty reduction through the development of economic infrastructure, but the connection to reducing poverty in rural areas is not explicit. The 2011-2015 CSP identifies the Bank's role cooperating with partners to implement the Zambia Chapter of the Global Pilot Project for Climate Change resilience and recognizes the increased pressure that climate change is placing on rural agricultural activities. However, no specific interventions are identified in this regard.

Inclusive and Green Growth across the Project Portfolio. The themes of inclusive and green growth have been integrated across the portfolio through the promotion of socially and environmentally responsible investment and a reduction of disparities between urban and rural areas. Overall, integration of inclusive and green growth within the project portfolio has been moderately satisfactory.

Promotion of socially responsible investment was achieved through: (i) implementation of Environmental and Social Management Plans for category 1 and 2 projects; and (ii) increasing the capacity of FIs to address environmental and social management issues in their portfolio. All Category 1 and 2 projects in the power, transport, private, and WSS sectors are subject to ESMPs, for which implementation has been satisfactory. Beneficiaries have also made additional efforts to improve their environmental performance. For example, the Lumwana Mine has successfully reduced its water usage such that, in 2013, it was the best performer among Barrick mines in terms of water used per ton of ore produced.90 LOC projects, although deemed as


An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

Category 3, were complemented by TA to strengthen the content and implementation of the targeted FIs' Environment and Social Management policies.

Other measures have been taken by private sector partners to promote social sustainability of projects. Both ITPC and Barrick have established trust funds to support the development implementation of local development plans in cooperation with surrounding chiefdoms and build the capacity of local governments. For example, the Lumwana Mine and ZESCO promoted local employment opportunities by working with chiefdoms to establish lists of eligible workers for temporary and permanent employment. As a result, Barrick estimates that between 30-40 percent of its employees at the mine come from the local community.

Furthermore, nearly all of the Bank's projects in the WSS sector have sought to reduce disparities in service delivery among urban and rural communities by targeting areas where access to water and sanitation had previously been very low. The Bank's PRBS projects in the multi- and social sectors have sought to further reduce regional disparities through decentralization of government and improved delivery of health and social services in underperforming districts.

However, multi-sector projects have missed opportunities to promote green growth. Environmental sustainability was part of the PRBS Program under the 2010 PAF for which the approval and submission of an Environmental Mainstreaming Strategy and Action Plan, Climate Change Response Strategy and Environmental Management Bill were identified as targets. This element of the PAF, however, was not identified among the Bank's project objectives. Furthermore, some agricultural projects have inadvertently contributed to environmental degradation.

Finally, although agricultural projects have been successful in terms of addressing inclusive growth within the project design, these projects

have performed poorly in terms of managing environmental risks. As noted previously, the SIP did not adequately address impacts related to climate change such that two outgrower schemes are now facing challenges due to reduced water levels in Lake Kariba. Furthermore, the CWMIP is likely to cause environmental damage. Farmers in the Chingola, Mkushi and Kapiri Mposhi regions are gardening along dambos (a type of marshland) and perennial rivers, causing the dambos to dry up due to increased water extraction.

Quality at Entry

Quality at entry of the Bank's CSPs over the evaluation period was found to be moderately satisfactory. However, quality at entry for project design was found to be moderately unsatisfactory due to weaknesses in the clarity of project intervention logic as well as the realism of project design.

Quality at Entry of CSPs

At the Strategy level, the quality at entry of the Bank's CSPs for Zambia was assessed in terms of: (i) understanding of the evolving development context; (ii) clarity and realism of the intervention logic; (iii) selectivity and coherence of the project portfolio; and (iv) positioning and comparative advantage.

All CSPs over the evaluation period are rooted in a robust analysis of the political context, macroeconomic context, dimensions of poverty, national development strategies and constraints to growth. Understanding of the country context has been underpinned by the use of consultation to inform CSP design. The design of 2002-2004 CSP was not consultative, relying instead on its alignment with the PRSP and an opportunity provided for members of government, civil society and DPs to provide


feedback subsequent to drafting of the strategy. By contrast, development of the JASZ was rooted in a harmonized division of labor among the DPs, in consultation with GRZ, civil society and the private sector. Development of the 2011-2015 CSP adopted a similar consultative approach but consultations were expanded to include emerging donors such as Saudi Arabia, China, Brazil and India. It should be pointed out that the 2011-2015 CSP was finalized prior to the establishment of a new division of labor under the JASZ-II. Under the new division of labor, finalized in 2012, the Bank was retained as "lead" partners for Agriculture and WSS, notably at odds with the priorities that had been formulated under the CSP.91

The clarity and realism of the intervention logic for the Bank's CSPs remained relatively weak until the 2011-2015 CSP. The 2002-2004 CSP did not identify a results matrix but instead provided a policy matrix which linked projects to sector-level policy objectives without identifying project outputs, outcomes or performance indicators. Similarly, the Bank did not develop a separate results framework for its interventions under the JASZ based on the intention to develop a joint monitoring framework with the GRZ which was never, in fact, delivered. By contrast, though certain challenges remain,

the 2011-2015 CSP presents a credible results framework in which specific projects are linked to strategy outcomes through measurable outputs, outcomes and performance indicators. However, an independent review of the quality at entry of the CSP found that the results Logframe continued to be impractical due to its scope and the fact that actual implementation of the framework would require onerous amounts of time and resources. The strategic selectivity and coherence of the Bank's portfolio has improved over time. The 2002-2004 portfolio identified interventions across five different sectors. The 2007–2010 JASZ again identified five different priority sectors under two different pillars: (i) infrastructure development; and (ii) the promotion of good governance. In contrast, the 2011–2015 CSP narrows the Bank's strategic focus to just three priority sectors under two thematic pillars, that is: (i) Supporting Economic Diversification; and (ii) Supporting Economic and Financial Governance. Although the CSP has become more selective in thematic terms, the number of sectors covered under each portfolio has increased over time with interventions approved in just four sectors between 2004 and 2006, five sectors between 2007 and 2010, and seven sectors between 2011 and 2015 (see Table 12 below). Despite implementing activities in a broader range of sectors, the

Table 12: Operational selectivity in Zambia CSPs

2002-2006 2007-2010 2011-2015CSP Priority Sectors(WSS, Agriculture, Social, Multi-Sector, Private Sector)

CSP Priority Sectors(WSS, Agriculture, Transport, Energy, Multi-Sector)

CSP Priority Sectors(Transport, Energy, Multi-Sector)

Sector # Project percent Funds



WSS 2 40.44 Transport 1 42.37 Energy 2 25.73

Industry 1 33.27 Multi-Sector 2 28.59 Agriculture 4 22.49

Multi-Sector 1 22.22 WSS 1 21.38 Transport 1 21.44

Agriculture 2 4.62 Finance 4 6.26 Social 2 15.65

Finance 0 n/a Agriculture 3 0.01 Environment 2 11.32

Transport 0 n/a Industry 0 n/a WSS 2 1.50

Social 0 n/a Social 0 n/a Finance 1 0.45

Energy 0 n/a Energy 0 n/a Multi-Sector 0 n/a

Environment 0 n/a Environment 0 n/a Industry 0 n/a

Source: SAP


An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

Bank's strategy has become more coherent since projects across a range of sectors now support a more limited number of strategic outcomes. For example, under the 2011-2015 CSP, interventions in the finance, energy, transport, social and multi-sectors complement each other in removing barriers to private sector growth. The advantage of this approach is that it allows the Bank to address multiple facets of each targeted outcome.

However, as indicated by an independent review of quality at entry for the 2011-2015 CSP, such an approach risks dispersing resources too thinly among multiple sectors, thereby limiting the achievement of results across the sector. This risk is most apparent with the Bank's LOC projects, which have reached a relatively small number of businesses. In contrast, the SSTEP is expected to have a broader impact, reaching 4,000 youths.

Quality and Realism of Project Design

Overall, the quality and realism of project design was found to be moderately unsatisfactory, with issues identified across all sectors. In particular, problems were identified with regard to: (i) realism of the intervention logic; (ii) quality and realism of project design. Weaknesses identified with regard to the realism of the project intervention logic resulted from weak realism of the intended outcomes, given the scope and design of projects and lack of clarity in the intervention logic due to confusion between outputs and outcomes.

In the case of LOC Projects, some targets, such as "reduced unemployment among youths 15-24" and "one percent GDP growth in targeted countries," were unrealistic given the size and scope of the projects (together, the three LOCs accounted for just USD 15 million in non-revolving funds). Furthermore, no credible project mechanisms were identified to reduce the cost of finance among subprojects. Beneficiary FIs confirmed that subprojects supported

through the LOC were assessed according to their regular due diligence processes. Otherwise, the FIs may not generate sufficient profit or would be exposed to undue risk. Accordingly, there were no means of ensuring that specific underserved groups would benefit from the funds instead of the FIs' existing clientele.

FIs targeted under the LOC projects also questioned why the funds were not provided on a revolving basis given the targeted number of businesses to be reached. Revolving funds would have allowed the FIs to reach a larger pool of SMEs, especially considering the long tenor of the LOCs (10 years in the case of ZANACO), against the relatively short tenor of the loans provided to SMEs (on average, less than two years).

The intervention logic for the Bank's projects under the PRBS program was limited by confusion among outputs, targets, triggers and outcomes. As such, it was not always possible to determine how GRZ activities addressed the PAF targets or to identify lessons learned. Furthermore, in an effort to be selective, the Bank often limited its focus to specific areas of the PAF. However, disbursements were triggered by performance against the PAF as a whole, calling into question whether donors truly pooled resources to achieve a limited number of shared objectives.

Issues with respect to the realism of project design arose from the quality of feasibility studies underpinning the project designs, resulting in an underestimation of project costs. In the case of infrastructure projects, design challenges arose due to the quality of engineering works.

Quality and use of Feasibility Studies. The majority of projects in the agriculture sector were not supported by feasibility studies to assess cost and implementation arrangements. In the case of the SIP, the funds provided were sufficient to fully implement only one of the six planned outgrower schemes. In the


case of PRODAP, the project design and appraisal did not clearly identify specific locations for investments and did not consider the high dispersal of coastal communities. With respect to implementation arrangements, no consideration was given to the availability and capacity of local suppliers and contractors, which frustrated project implementation.

In contrast, the design of WSS projects was informed from lessons learned identified through previous interventions, including the adoption of demand-driven approaches, an assessment of financial sustainability for maintenance, sensitization of communities and simplicity and uniformity in the selection of equipment. Feasibility studies were used to identify water demand, stakeholders' ability to pay and the collection of baseline data. However, the cost of improvements to sanitation infrastructure was underestimated, resulting in a low uptake of improved sanitation technology among beneficiaries and diminishing performance for these components. Furthermore, the RWSSP and Nkana WSS projects did not fully consider the ability or willingness of communities to pay for water services, as well as operations and maintenance fees, resulting in increases in unaccounted for water in the case of Nkana and lack of a sustainable operations and maintenance structure in the case of the RWSSP.

Engineering Works and Project Modalities. The Bank's projects in the transport sector have been supported by detailed studies which identify cost-effective solutions to regional trade issues. Furthermore, projects in the transport sector have incorporated lessons learned from previous initiatives in other countries, including the importance of integrating TA and community sensitization into the project design. However, initial engineering and design work completed for the Nacala Corridor Road project was insufficient, resulting in delays as additional engineering work was undertaken.

The design and implementation of the Itezhi-Tezhi power project was similarly supported by detailed feasibility studies. Other initiatives undertaken to control a range of risks include negotiation of a take-

or-pay PPA and the use of Engineering, Procurement and Construction (EPC) Contracts. However, one issue noted in the design of the project is the decision to erect a 220 KV transmission line from Itezhi-Tezhi to Mumbwa, necessitating the installation of transformers at the Mumbwa substation, thereby increasing the project cost and complexity. Given that additional investment is expected in the Western Region, a 330 KV line would have facilitated future expansion.

The Ministry of Energy expressed dissatisfaction with the progress achieved in the Itezhi-Tezhi power project to date owing to implementation delays relative to the complexity of the project. Given that the required infrastructure was already in place and owned by ZESCO, it was questioned why Itezhi-Tezhi was implemented as a PPP instead of a more technically complex project. Itezhi-Tezhi was regarded as "low-hanging fruit" within the GRZ's Power Sector Development Master Plan. GRZ stakeholders felt that Itezhi-Tezhi could have been implemented more efficiently as a public sector project, diverting private resources to a project for which a PPP could yield greater efficiency. Although MoFNP has established a PPP division, it was noted that its staff lacked the sector expertise necessary to identify projects for which a PPP modality can bring added value.

Supervision

The frequency of supervision has increased over the evaluation period.

A portfolio review indicated that, between 2002 and 2004, projects were supervised, on average, once every two years. However, the 2015 CSPCR indicates that, between 2011 and 2014, the number of projects supervised twice annually grew from five to eight projects out of an ongoing portfolio of 13. Over that period, the percentage of supervision missions conducted by staff at ZMFO increased from 67 percent to 75 percent. Increased supervision has also contributed to an increase in quarterly reports,


An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

which were found to be available for 50 percent of the ongoing projects in 2011, and 85 percent of ongoing projects in 2013. The feedback from the government on the Bank’s supervisions shows that quality of interaction with the Bank has improved over the evaluation period, particularly after the establishment of the Country Office.

There are instances where supervisions have been effective in improving the quality of the portfolio by taking steps to discontinue ageing and problematic projects (for example PRODAP) by restructuring the projects, limiting the components to suit the funds available and identifying alternative sources of funding to complete all components as with the original plan (for example, Small Scale Irrigation projects).

Leveraging and Co-financing

Throughout the evaluation period, leveraging and co-financing have been identified in the Bank's

CSPs as a means of maximizing the relatively small quantum of ADF resources available.

A portfolio review indicated that approximately 55 percent of projects approved throughout the evaluation period have included some degree of co-financing from partners. Furthermore, the 2015 CSPCR notes that, over the strategy period, the Bank managed to secure co-financing of 220 percent of the original ADF allocation.

With respect to the leveraging of funds, the Bank has served as the lead arranger for three projects: (i) the Itezhi-Tezhi Power Generation and Transmission Project; (ii) The Nacala Corridor Road Project; and (iii) the Kazungula Bridge. Through these projects, the Bank was able to leverage its own contribution by factors of 16, 3.68 and 1.82, respectively. The Bank has also secured co-financing from emerging donors, including India Exim Bank, the Development Bank of South Africa and the OPEC Fund for International Development.

61Conclusions and Recommendations

An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

Conclusions and Recommendations

Conclusions

The Bank’s Country Strategies and Programs have been well-aligned with both national development plans and the Bank's comparative advantage. The Bank's CSP priorities have reflected contemporary national development plans, including the FNDP, SNDP and Vision 2030. The Bank's CSPs have also responded to changes in national development priorities over time. Removal of constraints to growth through economic infrastructure development and strengthening of economic governance reflects the objectives of both the SNDP and Vision 2030, but also align with the Bank's comparative advantage in infrastructure development and policy-based operations.

The Bank's portfolio has become more coherent, adopting an integrated approach to development challenges. Although interventions have been approved in a greater range of sectors across each CSP period, the realism of the strategy logic has improved due to greater coherence of the portfolio. Projects across different sectors now address different facets of a limited number of strategy objectives. Development of economic infrastructure is now being addressed in terms of regional transport and energy infrastructure, trade facilitation and skills development. Strengthening of economic governance is being addressed through complementary initiatives in the multi-sector and financial sector to enhance the regulatory environment and increase access to finance.

The Bank has largely delivered planned outputs, but the achievement of outcomes has been

limited by project design weaknesses and delayed implementation. With the exception of the transport and agriculture sectors, the delivery of outputs has been satisfactory. Transport sector projects have faced considerable delays but project outputs are likely to be achieved. The achievement of project outcomes, however, has been less satisfactory, reflecting the weaknesses in project design, including (i) inadequate targeting of beneficiary needs; (ii) lack of realism in intervention logic; and (iii) weaknesses in project assumptions.

LOC projects were unable to influence the due diligence requirements and lending practices of FIs and did not address constraints arising from the low productivity of Zambian SMEs. Agriculture projects faced challenges in implementation as project design was not supported by detailed feasibility studies, including credible project costs and concrete implementation arrangements. On the other hand, policy-based operations failed to address poor ownership of reforms and capacity constraints among line ministries.

Implementation delays have also affected the achievement of outcomes. The timeliness of project implementation was found to be unsatisfactory across all sectors. Whereas progress has already been made in addressing delays to loan effectiveness and first disbursement, the ongoing portfolio has been experiencing challenges arising from multiparty co-financing arrangements. In particular, projects in the power and transport sectors are delayed due to the need to harmonies terms, conditions and procurement arrangements among multilateral, bilateral and emerging partners. As experienced


in the Itezhi-Tezhi power project, these delays can increase costs and prolong the achievement of outcomes.

The Bank has contributed to increasing access to basic services and improving the business environment. However, opportunities for upscaling have not been leveraged. At the country level, the Bank has contributed to improving access to basic services and infrastructure, including water, sanitation, power and health. In contrast, efforts to strengthen PFM have been less successful. Notably, these gains have not resulted only from direct investments in each sector. Increased access to services and business development have also resulted from large private sector investments, including the Lumwana Mine and Itezhi-Tezhi power plant. However, opportunities to complement and upscale the initiatives undertaken by private sector actors (for example, the corporate social responsibility initiatives by Barrick Gold at the Lumwana Mine) have not been addressed.

Political and governance risks are becoming an increasingly important factor in the sustainability of projects. Whereas risks to the sustainability of earlier projects implemented in the agriculture and WSS sectors stemmed from project design issues, institutional capacity and weak ownership among beneficiaries, more recent interventions are vulnerable to political and governance risks. These risks include regulatory changes implemented without consultation, irregularities in procurement, and lack of political will to address inadequate tariff regimes. For example, the sustainability of LOC projects were limited by the introduction of interest rate caps and increased capital reserve requirements, whereas investments in the power sector will face challenges due to the continued lack of cost-reflective tariffs.

Whereas the Bank's activities have attempted to promote inclusive and green growth, opportunities to mainstream gender have not been fully leveraged. The Bank's projects have attempted to promote sustainable investment as well as the reduction of disparities between urban and rural areas. However, opportunities have not always been leveraged to generate business development and employment opportunities for women. LOC and agriculture projects demonstrate that such initiatives should be supported by targeted project mechanisms whereas the Lumwana Mine project demonstrates that more can be done to promote direct employment opportunities for women within infrastructure and industrial projects.

Recommendations

Based on the above findings and conclusions, IDEV proposes the following recommendations.

1. Continue to support private sector regulatory reform and build capacity among the accountability functions of government.

Budget support and TA interventions have been successful in promoting private sector regulatory reform and strengthening of accountability functions, partly due to adequate ownership among implementing bodies. These issues continue to be relevant, particularly the need to strengthen audit and procurement capacity. Furthermore, strengthening external accountability functions can complement the implementation of financial control systems to reduce irregularities and improve public financial management.

2. Strengthen the Bank’s role in donor coordination, analytical work and policy dialogue.


An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

The Bank, with its unique positioning in Africa and among the DPs, is expected to play a key role in donor coordination, analytical work and policy dialogue. The policy dialogue needs to be supported by rigorous analytical work, which is also demanded by government in several areas of public policy including private sector development, public private partnerships, and regulatory systems with policy predictability. The donor coordination efforts are currently subdued due partly to the withdrawal of several cooperating partners from key areas of assistance including budget support. Coordinated efforts are required to address deficiencies in the monitoring and evaluation system at the country level to enable systematic reporting of results with active participation of all line ministries, and the Bank should contribute to the ongoing initiative in this regard. Finally, policy dialogue should also be implemented as a means of addressing increasing political and governance risks to the sustainability of the Bank's projects, particularly with regard to ensuring adequate consultation with the private sector to inform regulatory changes.

3. Address constraints to private sector involvement in service delivery.

Private sector participation in service delivery, particularly in the energy sector, will play an increasing role in addressing Zambia's infrastructure bottlenecks. However, inadequate tariff structures and macroeconomic pressures create disincentives to private sector participation. The Bank should identify, through analytical and policy dialogue in particular, opportunities to mitigate the impact of these constraints through a range of instruments, including lending, TA for project selection and guarantees for service delivery and purchasing agreements.

4. Identify opportunities to upscale development outcomes from private sector investments.

Although the Bank has helped improved access to services and infrastructure through public sector projects, private sector investments have yielded similar results through infrastructure works and corporate social responsibility initiatives. As demonstrated by the experience of the Itezhi-Tezhi Power Project and the Lumwana Mine, additional development impacts and private sector development could be realized through complementary infrastructure investments surrounding these projects. Integrating lessons from past experience into the CSP, the Bank should more systematically identify opportunities to upscale and complement such initiatives, thereby further contributing to service delivery, infrastructure improvements and business development.

5. Deepen the integration of gender in Bank’s operations and engage in policy dialogue on gender with the government in collaboration with cooperating partners.

Even though the Bank’s analytical work on gender in Zambia had recommended specific areas for intervention, those were not implemented in earnest. In the increasingly relevant context of feminization of poverty and HIV/AIDS pandemic, gender violence, women’s unequal access to education, health services, and resources including land and credit, it is imperative that the Bank take concrete steps to integrate gender in its operations with respect to design, implementation and the achievement of results. The Bank, with other cooperating partners, should engage in policy dialogue with the government and assist in the implementation of the National Gender Policy 2014 with a focus on gender


auditing, gender responsive national planning and budgeting, and the establishment of a system for monitoring gender outcomes at country level.

6. Promote synergies within the Bank program and in coordination with other donors between projects which improve the productivity of small businesses and increase access to finance.

Increasing the pool of SMEs that generate sufficient revenue to afford formal financial products remains an obstacle to improving access to finance. Opportunities for creating linkages between interventions which increase production of high value crops, incomes and access to markets among underserved groups with LOCs or TA for SME finance should be explored. Such linkages may yield synergies in terms of access to finance, business development and economic growth. Outgrower and farmers club schemes provide a proven means of improving agricultural productivity and diversity while linking farmers to markets and increasing both incomes and access to finance. Attention should be paid to ensuring that targeted project mechanisms are included to ensure that the benefits are gender inclusive.

7. Identify means of harmonizing project implementation approaches with other co-financing partners as well as within projects that possess both private and public sector components.

Addressing regional and domestic infrastructure constraints will likely necessitate continued cooperation and co-financing. The experience of the Kazungula Bridge and Itezhi-Tezhi Power Project demonstrates the need for the Bank to work closely with other partners by identifying harmonized terms, conditions, engineering and procurement approaches for co-financed projects in order to avoid

implementation delays, cost-overruns and output quality inconsistencies. Furthermore, the design and implementation of co-financed projects with both public and private components should be mindful of differences in the approval processes for these two sources of funds in order to avoid both unnecessary delays and additional costs.

8. Increase the capacity of the GRZ for project selection, design and engineering, particularly for infrastructure projects and Public Private Partnerships.

Inadequate project design and engineering has frustrated the implementation of projects across several sectors, including the transport sector, for which inadequate project design and engineering can result in considerable project delays as well as procurement disputes. Assistance in implementing high quality feasibility studies, which identify reasonable project costs, would also be beneficial for the Bank's ongoing support to the WSS sector, for which underestimation of the project complexity and costs have limited the successful delivery of improved sanitation facilities for past interventions. As additional PPPs are implemented in Zambia, technical assistance should be provided to improve project selection and ensure that PPP arrangements are providing added value. Finally, greater attention should be devoted to identifying and assessing sensitivity to political and governance risks, including tariff structures, regulatory changes and ownership.

9. Identify and mitigate operational issues surrounding multinational projects aimed at regional integration.

The implementation of regional projects has been subject to excessive delays caused by coordination issues at the regional level, as seen in the case of PRODAP, which was discontinued in course of implementation. Delays in the full implementation


An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

of one-stop border post constrain the movement goods, services and people, affecting the competitiveness and economic growth of the region in general and Zambia in particular. Based on the identification of constraints, the Bank should

establish linkages between the CSP and the broader regional integration strategy to ensure bottlenecks are addressed at the right level – either through dialogue at country level or coordinated approach at regional level.

Annexes


Annex A — Evaluation Questions

Evaluation Issue Specific Evaluation Questions1. Relevance 1.1: To what extent are the Bank's Country Strategies and operations for Zambia aligned with

the development needs, challenges and priorities of the country as well as the needs of the expected beneficiaries?

1.2: To what extent are the Bank's Country Strategies for Zambia aligned with the Bank's own strategies, priorities and policies?

1.3: To what extent are the Bank's interventions in Zambia aligned with the Bank's Country Strategies for Zambia?

2. Effectiveness 2.1: To what extent have the Bank's interventions achieved their objectives in terms of the delivery of outputs?

2.2: To what extent have the Bank's interventions contributed to the achievement of development results in terms of expected outcomes?

2.3: To what extent have the Bank's interventions contributed to the achievement of development results for the country, including intended and unintended impacts?

2.4: To what extent have the Bank's interventions benefitted target group members?

3. Efficiency 3.1: To what extent were the Bank's interventions delivered in an efficient and economical manner (were resources economically converted into outputs)?

3.2: To what extent are the Bank's interventions implemented in a timely manner which is consistent with best practices?

3.3: To what extent did the Bank's intervention achieve good value for money (cost-benefit)?

4. Sustainability 4.1: To what extent are the results achieved likely to continue once the Bank's interventions are completed?

4.2: To what extent has the Bank sought to mitigate risks to the sustainability of results as part of the project design process?

5. Crosscutting Themes 5.1: To what extent are the Bank's interventions inclusive in terms of geography, age and gender?

5.2: To what extent are the Bank's interventions environmentally sustainable and support the transition to green growth?

6. Design and Delivery 6.1: To what extent has the quality of CSPs been satisfactory? Are the Bank's interventions coherent and well-coordinated internally?

6.2: To what extent have the Bank's interventions been selective and strategic?

6.3: How has the Bank adapted its interventions to implementation challenges within the country?

6.4: To what extent has the Bank's engagement been informed by policy dialogue with national actors, DPs and other interested stakeholders?

6.5: How well has the Bank leveraged its financial resources within the country through mechanisms such as productive partnerships and co-financing?

6.6: To what extent has the Bank acted as a knowledge broker, advisor and convener? To what extent have the Bank's interventions been informed by the Bank's analytic work as well as that of other DPs?

6.7: To what extent has the Bank cooperated with other DPs to ensure complementarity and reduce overlaps?

69Annexes

An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

Evaluation Issue Specific Evaluation Questions7. Management for Results 7.1: To what extent has the Bank's supervision of its interventions been effective?

7.2: How has the Bank used supervision and results information to guide the management of its interventions?

7.3: To what extent has the Bank successfully implemented a performance measurement strategy which focuses on the achievement of outputs, outcomes and impacts?

7.4: To what extent have lessons learned been identified and incorporated into subsequent CSPs?

7.5: How has the Bank worked with the GRZ to enhance its own systems of Results-Based Measurement?

7.6: To what extent has the Bank identified and monitored potential implementation risks?


Annex B — Zambia Project Portfolio 2002–2015

Project Name Approval Year Net Loan (UA'000)

Status

Domestic ProjectsAgricultureSmall scale irrigation project* 2000 5,788.6 Completed

Project to Support the Lake Tanganyika Integrated Regional Development Program

2004 1,340.4 Completed

Humanitarian Emergency Assistance 2005 359.8 Closed

Zambia Emergency Assist. Flood Vic. 2008 359.8 Completed

Community Water Management Improvement 2009 571.8 Completed

Finish Supported Small Scale Irrigation 2009 6,171.3 Ongoing

Emergency Assistance to 2009 Flood Mitigation 2010 719.6 Completed

Project Preparation Facility - ZM LISP 2013 299.9 Ongoing

Livestock Infrastructure Support Project 2013 12,000.0 Ongoing

Lake Tanganyika Development Project (Zambia) 2014 21,219.5 Approved

GAFSP - Agriculture Productivity & MKT EP 2014 22,394.6 Ongoing

Cashew Infrastructure Development Project (CIDP) 2015 32,383.0 Approved

EnvironmentStrengthening Climate Resilience Kafue 2013 27,345.6 Ongoing

FinanceZanaco Fapa Ta Grant for Zambian Smes 2008 652.3 Closed

Investrust Zambia 2008 2,518.7 Closed

Zanaco Guarantee Fac./Loc Zanaco- Zambia 2008 7,196.2 Closed

PFSL- FAPA TA - Zambia 2009 672.8 Closed

FRB Subsidiary in Zambia 2012 31,640.3 Approved

CETZAM Financial Services PLC 2013 1,079.4 Ongoing

Industry, Mining & QuarryingLumwana Copper Mining Project 2006 30,371.6 Closed

Multi-sectorPoverty Reduction Budget Support 2006 20,000.0 Closed

Poverty Reduction Budget Support II 2008 14,901.8 Closed

Poverty Reduction Budget SupportT III 2010 31,900.0 Closed

Fourth Poverty Reduction Budget Support 2011 15,000.0 Closed

PowerItezhi-Tezhi Power Project 2012 25,186.7 Approved

Itezhi-Tezhi Power Plant and Transmission Line Project 2012 36,400.0 Approved

Maamba Collieries Power Generation Project 2013 97,706.5 Approved

KARIBA DAM REHABILITATION 2014 25,200.0 Ongoing

SocialSupport for Science & Technology Education 2013 22,220.0 Ongoing

71Annexes

An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

Project Name Approval Year Net Loan (UA'000)

Status

Skills Development and Entrepreneurship Project - Supportin 2015 21,588.6 Ongoing

TransportSadc North-South Tranport Corridor Improvement Study 2006 1,392.8 Closed

Botswana/Zambia - Sadc North-South Corridor Improvement Study 2007 600.0 Closed

Nacala Road Corridor Studies 2009 256.0 Ongoing

Multi-Nacala Road Corridor Project - Phase II (ZAMBIA) 2010 69,369.0 Ongoing

Kazungula Bridge Project 2011 51,000.0 Ongoing

Nacala Road Corridor Development Project– Phase IV 2013 5,000.0 APVD

Chinsali - Nakonde Road Rehabilitation Project (North-South) 2015 174,867.9 APVD

Water Supply and SanitationCentral Province Eight Centers Water Supply and Sanitation 2003 21,342.0 Closed

Rural Water Supply & Sanitation Program 2006 15,000.0 Closed

Nkana Water Supply and Sanitation Project 2008 34,998.4 Closed

Multipurpose Small Dams 2012 755.4 Ongoing

Rural Water Supply and Sanitation II - Transforming Rural Livelihoods in Western Zambia - National

2014 13,937.0 Approved

Lusaka Sanitation Program 2015 35,981.1 Approved

Total (Domestic Projects) 947,837.1Regional ProjectsAgricultureComesa Agric. Marketing Promotion and Reg. Integration Project 2004 3,687.0 Completed

SADC- Strengthening of Insitutions 2006 13,710.0 Completed

Africa Food Crisis Response 2008 20,000.0 Completed

FinancePTA Bank 2nd Line of Credit & Ta Support 2008 649.8 Ongoing

Fifth Line of Credit to The Development Bank of Southern Africa Limited 2011 192,068.8 Completed

Business partners international southern africa fund 2012 4,612.7 Completed

PTA Bank Loc-Equity 2013 2013 32,490.9 Approved

Multi-sectorStatistical Capacity Building Under the International Comparison Program for Africa

2004 18,140.0 Completed

Enhancing Procurement Reforms and Capacity Project 2006 5,660.0 Completed

Capacity Strengthening for GS- IOC 2013 616.9 Approved

PIDA Capacity Building 2013 5,600.0 Approved

SocialAfrican Virtual University Project 2004 5,000.0 Completed

SADC Support To The Control Of Communicable Diseases 2006 20,000.0 Ongoing

SADC Capacity Building For Open And Distance Learning 2006 15,000.0 Ongoing

African Virtual University Support Project - Phase II (AVU II) 2011 10,000.0 Ongoing

TransportProgram for Infrastructure Development 2005 1,679.0 Completed

SADC Technical Assistance - Transport 2006 350.0 Completed

COMESA Airspace Integration Program 2010 5,750.0 Ongoing

Water Supply and SanitationSupport to Sadc Wss Program 2009 18,231.3 Ongoing

Total (Regional Projects) 373,246.4


Annex C — Evaluation Matrix and Theory of Change

1. RelevanceEvaluation Question Judgement Criteria Data Sources1.1: To what extent are the Bank's Country Strategies and operations for Zambia aligned with the development needs, challenges and priorities of the country as well as the needs of the expected beneficiaries?

❙ Thematic consistency of the Bank's CSPs with the GRZ's strategic framework for development and priorities.

❙ Consistency between Bank strategies and country development needs among RMC stakeholders.

❙ Degree of consistency between the Bank's strategies and development needs identified in the available literature.

❙ National and Sector-level Development Strategies

❙ Bank CSPs for Zambia

❙ Interviews with RMC stakeholders

❙ Available literature

1.2: To what extent are the Bank's Country Strategies for Zambia aligned with the Bank's own strategies, priorities and policies?

❙ Degree of consistency between the Zambia Country Strategies and the strategic priorities of the Bank.

❙ Degree of consistency between the Bank's interventions and its strategic priorities at the sector level.

❙ Bank Policy and Strategy documents


❙ Interviews with Bank staff

1.3: To what extent are the Bank's interventions in Zambia aligned with the Bank's Country Strategies?

❙ Degree to which project approvals over each CSP period reflect contemporary CSPs.

❙ File Review of project information


❙ Interviews with Bank staff

Rationale and ApproachThe relevance of the Bank's assistance will be assessed at both the project and country levels along three dimensions: (i) alignment of projects and the CSP with demonstrable development needs; (ii) alignment of the CSP with the Bank's corporate and sector-level strategies; and (iii) alignment of projects with the CSP and with Regional Integration Strategies. Evidence from each of the evaluation questions will be triangulated to provide an overall rating on a 6-point scale from Highly Satisfactory to Highly Unsatisfactory, as provided below.6 – Highly Satisfactory: It is demonstrated that the project objectives doesn’t have any shortcoming in their alignment with: i) the Bank’s CSP, ii) applicable Bank sector strategies, iii) the country’s development strategies, and iv) the beneficiary needs.5 –Satisfactory: It is demonstrated that the project objectives have minor shortcomings in the alignment with: i) the Bank’s CSP, ii) applicable Bank sector strategies, iii) the country’s development strategies, and iv) the beneficiary needs.4 – Moderately Satisfactory: It is demonstrated that the project objectives have moderate shortcomings in the alignment with: i) the Bank’s CSP, ii) applicable Bank sector strategies, iii) the country’s development strategies, and iv) the beneficiary needs3 – Moderately Unsatisfactory: It is demonstrated that the project objectives have significant shortcomings in the alignment with one of the following: i) the Bank’s CSP, ii) applicable Bank sector strategies, iii) the country’s development strategies, and iv) the beneficiary needs. 2 – Unsatisfactory: It is demonstrated that the project objectives have major shortcomings in the alignment with two of the following: i) the Bank’s CSP, ii) applicable Bank sector strategies, iii) the country’s development strategies, and iv) the beneficiary needs. 1 – Highly Unsatisfactory: It is demonstrated that the project objectives have severe shortcomings in the alignment with all of the following: i) the Bank’s CSP, ii) applicable Bank sector strategies, iii) the country’s development strategies, and iv) the beneficiary needs.

73Annexes

An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

2. EffectivenessEvaluation Question Data Sources2.1: To what extent have the Bank's interventions achieved their objectives in terms of the delivery of outputs?

❙ Review of supervision reports and project monitoring information

❙ PCRs, XSRs, BTORs and mid-term reviews

❙ Available country statistics for key indicators

❙ Interviews with project stakeholders (Bank staff, implementing partners, GRZ)

❙ Interviews and/or surveys of beneficiaries as possible

❙ Field visits




Judgement Criteria and RationaleEffectiveness will be examined at the project, sector and strategy levels and will center on the achievement of both outputs and immediate outcomes. The achievement of intermediate outcomes will be considered where credible data are available. This examination of the Bank's effectiveness will be based on a triangulation of qualitative and quantitative evidence from both primary and secondary sources to inform a robust and credible assessment.

First, each project will be rated on a six-point scale ranging from Highly Satisfactory to Highly Unsatisfactory. Next, a rating will be provided for each sector based on the extent to which the collection of projects are found to have contributed to sector-level outcomes, identified below. Finally, a global rating will be provided based on the extent to which the projects are found to have contributed to country-level outcomes identified in the underlying Theory of Change for the CSP.

The rating scale for the assessment of outputs is provided below.

6 – Highly Satisfactory: Based on the output execution ratio all the project output targets were reached or are considered on track to be reached by the end of the project in accordance with quality standards.

5 – Satisfactory: Based on the output execution ratio between 90 percent and 99 percent of the project output targets were reached or are considered on track to be reached by the end of the project. Corrective actions for off track indicators were implemented in a timely manner to ensure that the end of project targets could be achieved in accordance with quality standards.

4 – Moderately Satisfactory: Based on the output execution ratio between 75 percent and 89 percent of the project output targets were reached or are considered on track to be reached by the end of the project. Corrective actions for off track indicators were implemented in a timely manner to ensure that the end of project targets could be achieved in accordance with quality standards.

3 – Moderately Unsatisfactory: Based on the output execution ratio between 50 percent and 74 percent of the project output targets were reached or are considered on track to be reached by the end of the project. Corrective actions for off track indicators were not implemented in a timely manner to ensure that the end of project targets could be achieved.

2 – Unsatisfactory: Based on the output execution ratio between 35 percent and 49 percent of the project output targets were reached or are considered on track to be reached by the end of the project. Corrective actions were not implemented and closely monitored for off track indicators. Poor performance jeopardized the achievement of one or more outcomes of the project.

1 – Highly Unsatisfactory: Based on the output execution ratio less than 35 percent of the project output targets were reached or are considered on track to be reached by the end of the project. Poor performance jeopardized the achievement of most expected outcomes and the possibility of stopping or suspending the project was considered.


2. EffectivenessEvaluation Question Data Sources2.1: To what extent have the Bank's interventions achieved their objectives in terms of the delivery of outputs?

❙ Review of supervision reports and project monitoring information

❙ PCRs, XSRs, BTORs and mid-term reviews

❙ Available country statistics for key indicators

❙ Interviews with project stakeholders (Bank staff, implementing partners, GRZ)

❙ Interviews and/or surveys of beneficiaries as possible

❙ Field visits




Judgement Criteria and RationaleThe rating scale for the assessment of outcomes will be applied as follows:

6 – Highly Satisfactory: Taking into account the latest value of the outcome indicators and the analysis of other relevant exogenous risks/factors and assumptions, it is plausible to expect that all intended project outcomes were achieved or are likely to be achieved.

5 – Satisfactory: Taking into account the latest value of the outcome indicators and the analysis of other relevant exogenous risks/factors and assumptions, it is plausible to expect that most (75 percent) intended project outcomes were achieved or are likely to be achieved.

4 – Moderately Satisfactory: Taking into account the latest value of the outcome indicators and the analysis of other relevant exogenous risks/factors and assumptions, it is plausible to expect that a substantial (50 percent-74 percent) intended project outcomes were achieved or are likely to be achieved.

3 – Moderately Unsatisfactory: Taking into account the latest value of the outcome indicators and the analysis of other relevant exogenous risks/factors and assumptions, it is plausible to expect that few (25-49 percent) intended project outcomes were achieved or are likely to be achieved.

2 –Unsatisfactory: Taking into account the latest value of the outcome indicators and the analysis of other relevant exogenous risks/factors and assumptions, it is plausible to expect that few (5-24 percent) intended project outcomes were achieved or are likely to be achieved.

For completed projects, IDEV will conduct an in-depth assessment of the results chain consistent with the approach recommended in the Evaluation Cooperation Group's "Big Book on Evaluation Good Practice Standards," including:

❙ An assessment of the causal chain in relation to the needs of the target population in collaboration with stakeholders and experts;

❙ Examination of the critical assumptions and expectations inherent in the project's design;

❙ Use of available research evidence and practical experience to compare the project with projects based on similar concepts; and

❙ Observation of the project in operation, focusing on interactions that were expected to produce the intended outcomes.

Special note will be taken of challenges and enabling factors within the implementation context for each project in order to test the Bank's Theory of Change for its operations in Zambia.

75Annexes

An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

Sector Results Chain and Potential IndicatorsAgriculture and Environment Ultimate outcomes

❙ Economic growth and diversification

❙ Reduced poverty and enhance food security in rural communities

Intermediate outcomes ❙ Increase income for farmers

❙ Increased agricultural productivity

❙ Increased agricultural diversification

Immediate outcomes ❙ Increased farm land under cultivation

❙ Increased use of irrigation technology and equipment

❙ Increased access to markets

❙ Increased access to credit

Outputs ❙ Development of new farm blocks and out-grower schemes

❙ Provision of credit to farmers

❙ Guaranteed market agreements

❙ Delivery of improved infrastructure technology

❙ Provision of training and capacity building

Sample Indicators ❙ hectares of cultivated land

❙ hectares of irrigated land

❙ average income for farmers in target area

❙ Simpson Diversity Index by crop

❙ agricultural exports (percent GDP) Production of key crops


Sector Results Chain and Potential IndicatorsWater Supply and Sanitation Ultimate outcomes

❙ Increased access to basic services and infrastructure

❙ Poverty reduction

Intermediate outcomes ❙ Increased use and maintenance of WSSS facilities

❙ Reduced incidence of water borne disease

Immediate outcomes ❙ Increased access to improved WSS facilities

❙ Increased volume of treated water

❙ Increased reliability of water service

❙ Reduced unaccounted for water

Outputs ❙ WSS facilities constructed or rehabilitated

❙ Sanitation facilities provided in public institutions (schools, health center and markets)

❙ Delivery of Sanitation and hygiene promotion training

❙ Implementation of sewerage and treatment infrastructure

❙ Establishment of Commercial Water Utilities and Community Maintenance Groups

Indicators ❙ Number of people with access to safe water and sanitation facilities

❙ Number of cubic meters of water supplied annually

❙ Water supply and sanitation facilities constructed

❙ Number of water points rehabilitated

❙ Number of meters installed

❙ Percent unaccounted for water

❙ Proportion of water tests meeting standards

❙ Number of hours of water service per day

❙ Sewage treatment rate by area

77Annexes

An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

Sector Results Chain and Potential IndicatorsMulti-Sector and Social Sector

Ultimate goal ❙ Transparent and Accountable Public Financial Management

❙ Creation of a Business Enabling environment

❙ Increased Access to Pro-Poor Social Services

Intermediate outcomes ❙ Improved access to health and education services

❙ Strengthened accountability functions

❙ Reduced public debt and budget variance

❙ Increased access to finance for the private sector

❙ Increased business registration

❙ Reduced perceived corruption

Immediate outcomes ❙ Increased financial and administrative decentralization

❙ Improved budget credibility and financial controls

❙ Strengthened procurement and audit functions

❙ Reduced time to register property and businesses

❙ Reduced time to import and export

❙ Reduced number of business licenses

Outputs ❙ Implementation of Private Sector Development Plans

❙ Implementation of a One Stop Border Post

❙ Implementation of Land registry Offices

❙ Annual Audits of Public Accounts

❙ Number of Audit Professionals Hired

❙ Establishment of the ZPPA and Standard Procurement Documents

❙ Implementation of IFMIS and Single Treasury Account

❙ Establishment of the DACF and Local Government Service

❙ Construction of Schools and Health Centers

❙ Hiring of health and education professionals

Indicators ❙ Doing Business Scores

❙ CPIA Scores

❙ PEFA Scores

❙ World Development Indicators - businesses registered, number of days to register a business, etc.

❙ Percentage of budget released to district boards

❙ TPI Corruption Perception Index

❙ Percent Implementation of Audit Recommendations

❙ Percent Audit Coverage

❙ Percent use of competitive procurement processes

❙ Number of days to register a business

❙ Number of days to import and export products

❙ Expenditure variance


Sector Results Chain and Potential IndicatorsPrivate Sector / Finance Ultimate Outcomes

❙ Economic growth and diversification

❙ Creation of an enabling business environment

Intermediate Outcomes ❙ Job creation and employment

❙ Increased access to basic infrastructure and social services

❙ Increased profitability of beneficiary companies

❙ Increased access to finance

❙ Increased government tax revenues

Immediate Outcomes ❙ Increased availability of SME financing products and services

❙ Increased SME lending

❙ Improved SME lending terms

❙ Enhanced capacity for compliance with environmental and social standards and monitoring

Outputs ❙ Provision of training and TA

❙ Provision of LOC to financial institutions

❙ Delivery of planned infrastructure works

❙ Development of an open-cast copper mine

❙ Implementation of ESMP

❙ On-lending of funds to SMEs

Indicators ❙ Number of jobs created (local jobs and jobs for women)

❙ Percent increase in SME lending portfolio among target institutions

❙ Percent increase in SME loan approvals (proportion for women)

❙ Number of new SMEs financed

❙ Percent reduction in collateral requirements

❙ Percent of projects in compliance with Environmental and Social standards

❙ Government tax revenues as a share of GDP/Expenditure

❙ Percent supported women owned businesses and first time borrowers

❙ Annual revenues

❙ Percent NPLs

79Annexes

An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

Sector Results Chain and Potential IndicatorsTransport Ultimate Outcomes

❙ Improved access to basic social services and infrastructure

❙ Inclusive growth and poverty reduction

Intermediate Outcomes ❙ Increased access to markets and social services

❙ Increased regional trade

❙ Job creation and increased incomes

❙ Increased trade volumes

❙ Reduced import and export time and costs

Immediate Outcomes ❙ Reduced vehicle operating costs and transport costs

❙ Improved road safety

❙ Reduced travel and border transit time

❙ Increased traffic throughput

❙ Improved project coordination

Outputs ❙ Delivery of project design plans and specifications, estimates and tender documents for planned infrastructure installations

❙ Provision of consultancy services

❙ Provision of HIV/AIDS sensitization training

❙ number and percentage of ESMP projects successfully implemented

❙ Construction of border posts and bridges

❙ Rehabilitation of key roads

❙ Delivery of procurement capacity development training

Indicators ❙ Annual value of regional and international trade

❙ Quantity of traffic and value of goods transported through border posts

❙ Border transit time

❙ Vehicle operating costs

❙ Number of job created during construction and operation

❙ Annual number of road accidents and fatalities along corridor

❙ Number of people trained

❙ Length of roads rehabilitated

❙ Length of roads built

❙ Number of border stations built


Sector Results Chain and Potential IndicatorsEnergy Ultimate Outcomes

❙ Increased access to basic services and infrastructure

❙ Creation of an enabling business environment

Intermediate Outcomes ❙ Increased government revenues

❙ Increased access to and use of electricity

❙ Reduced carbon emissions

❙ Improved sustainability of power supply

❙ Increased private sector development

Immediate Outcomes ❙ Increased electricity generation capacity

❙ Job creation (for women and local population)

❙ Increased access to basic services

Outputs ❙ Implementation of proposed works

❙ Transmission lines built

❙ Sub-stations built and/or extended

❙ Implementation of proposed ESMP/RAP activities

❙ Provision of capacity development assistance

❙ Improved tariff structure

Indicators ❙ Power supply capacity in MW

❙ National electrification rate

❙ Number of jobs created during construction and operation (for women, youths and local population)

❙ Total up-front and annual concession fees accrued

❙ Length of transmission line built

❙ Number of substations built or extended

❙ Number of ESMP/RAP activities implemented

❙ Net tons of carbon emissions equivalents avoided each year

81Annexes

An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

3. EfficiencyEvaluation Question Indicators Data Sources3.1: To what extent were the Bank's interventions delivered in an efficient and economical manner (were resources economically converted into outputs)?

❙ Implementation Progress Ratings (IPRs)

❙ Perceived timeliness and economy of project implementation

❙ Project Completion Reports

❙ Interviews with Task Managers and implementation partners

3.2: To what extent are the Bank's interventions implemented in a timely manner which is consistent with best practices?

❙ Ratio of planned versus actual implementation time at the project level

❙ Project Completion Reports

❙ Interviews with Task Managers and implementation partners

3.3: To what extent did the Bank's intervention achieve good value for money (cost-benefit)?

❙ Extent to which the chosen project approach has yielded results at less cost relative to other potential approaches.

❙ Economic Internal Rate of Return

❙ Financial Internal Rate of Return

❙ File review of project documents

❙ Project completion reports

❙ Interviews with task managers and implementation partners

❙ Review of available literature

Judgement Criteria and RationaleThe assessment of efficiency will involve three components, namely: (i) the effective use of resources for the delivery of planned outputs; (ii) implementation timeliness; and (iii) extent of the results achieved relative to cost and implementation time (cost-benefit). The assessment will be informed by a triangulation of qualitative and quantitative evidence from project supervision and completion reports, as well as a review of the available literature.

A rating will be provided for each dimension using a six-point rating scale from Highly Satisfactory to Highly Unsatisfactory, depending on the availability of adequate data. In addition, a global rating will be provided for efficiency of the Bank's operations in Zambia over the evaluation period.

With respect to timeliness, the following rating scale will be applied:

6 – Highly Satisfactory: The ratio of planned implementation time (as per PAR) and actual implementation time from the date of approval is expected to be <1.

5 – Satisfactory: The ratio of planned implementation time (as per PAR) and actual implementation time from the date of approval is expected to be 1.0-1.1.

4 – Moderately Satisfactory: The ratio of planned implementation time (as per PAR) from the date of approval and actual implementation time from the date of effectiveness is expected to be 1.11-1.2.

3 – Moderately Unsatisfactory: The ratio of planned implementation time (as per PAR) from the date of effectiveness and actual implementation time from the date of effectiveness is expected to be 1.21-1.3.

2 – Unsatisfactory: The ratio of planned implementation time (as per PAR) from the date of effectiveness and actual implementation time from the date of effectiveness is expected to be 1.31-1.4.

1 – Highly Unsatisfactory: The ratio of planned implementation time (as per PAR) from the date of effectiveness and actual project implementation time from the date of effectiveness is expected to be 1.41-1.5

With respect to economic efficiency, the following rating scale will be applied:

6 – Highly Satisfactory: If EIRR is equal or above the opportunity cost of capital.

5 – Satisfactory: If (90 percent of the opportunity cost of capital ≤ EIRR < the opportunity cost of capital)

4 – Moderately Satisfactory: If (80 percent of the opportunity cost of capital ≤ EIRR < 90 percent of the opportunity cost of capital).

3 – Moderately Unsatisfactory: If (60percent of the opportunity cost of capital ≤ EIRR < 80 percent of the opportunity cost of capital).

2 – Unsatisfactory: If (40 percent of the opportunity cost of capital ≤ EIRR < 60 percent of the opportunity cost of capital).

1 – Highly Unsatisfactory: If EIRR is less than 40 percent of the opportunity cost of capital.


Judgement Criteria and RationaleThe rating scale for the assessment of outcomes will be applied as follows:

6 – Highly Satisfactory: Taking into account the latest value of the outcome indicators and the analysis of other relevant exogenous risks/factors and assumptions, it is plausible to expect that all intended project outcomes were achieved or are likely to be achieved.

5 – Satisfactory: Taking into account the latest value of the outcome indicators and the analysis of other relevant exogenous risks/factors and assumptions, it is plausible to expect that most intended project outcomes were achieved or are likely to be achieved.

4 – Moderately Satisfactory: Taking into account the latest value of the outcome indicators and the analysis of other relevant exogenous risks/factors and assumptions, it is plausible to expect that a substantial (50 percent-74 percent) intended project outcomes were achieved or are likely to be achieved.

3 – Moderately Unsatisfactory: Taking into account the latest value of the outcome indicators and the analysis of other relevant exogenous risks/factors and assumptions, it is plausible to expect that few (25-49 percent) intended project outcomes were achieved or are likely to be achieved.

2 –Unsatisfactory: Taking into account the latest value of the outcome indicators and the analysis of other relevant exogenous risks/factors and assumptions, it is plausible to expect that few (5-24 percent) intended project outcomes were achieved or are likely to be achieved.

For completed projects, IDEV will conduct an in-depth assessment of the results chain consistent with the approach recommended in the Evaluation Cooperation Group's "Big Book on Evaluation Good Practice Standards," including:

❙ An assessment of the causal chain in relation to the needs of the target population in collaboration with stakeholders and experts;

❙ Examination of the critical assumptions and expectations inherent in the project's design;

❙ Use of available research evidence and practical experience to compare the project with projects based on similar concepts; and

❙ Observation of the project in operation, focusing on interactions that were expected to produce the intended outcomes.

Special note will be taken of challenges and enabling factors within the implementation context for each project in order to test the Bank's Theory of Change for its operations in Zambia.

83Annexes

An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

4. SustainabilityEvaluation Question Data Sources4.1: To what extent has the Bank sought to mitigate risks to the sustainability of results as part of the project design process?


❙ Readiness Reviews

❙ Interviews with Bank staff, government and local stakeholders and implementation units

❙ Site visits

Criteria IndicatorsTechnical Soundness ❙ Extent to which the technical design of the project was supported

by extensive analytics.

❙ Extent to which maintenance requirements and operational risks have been identified in the project design.

❙ Extent to which the technical design of the project is based on mechanisms and methods which have a demonstrated track record of success.

Financial and Economic Viability ❙ Extent to which adequate public resources or user fees are available to support the continued operation and maintenance of the investment.

❙ Extent to which private sector operations demonstrate long-term profitability

❙ Extent to which long-term financial and technical support has been secured from partners and co-financers.

Institutional Sustainability ❙ Use of country systems for implementation (where feasible).

❙ Identification and implementation of necessary institutional reforms and mechanisms.

❙ Extent of policy dialogue and national ownership in implementing necessary policy and institutional reforms.

Ownership and Partnerships ❙ Extent to which interventions have been implemented in consultation and partnership with local authorities, CSOs, other donors and the private sector.

Resilience ❙ Degree of project resilience to exogenous factors, including climate and economic shocks

4.2: To what extent are the results achieved likely to continue once the Bank's interventions are completed?


❙ Interviews with Bank staff, government and local stakeholders and implementation units

❙ National Statistics (where available)

❙ Site visits

Sector IndicatorsAgriculture and Environment ❙ Evidence of regular maintenance conducted for infrastructure

investments (for example rural roads, irrigation schemes, storage facilities)

❙ Implementation of identified agricultural sector reforms

❙ Village and district development plans funded and implemented

Water Supply and Sanitation ❙ Share of infrastructure investments appropriately maintained and still operational

❙ Share of water-supply schemes which are financially self-sustainable

❙ Technical and financial capacity for maintenance and operation among local officials.


4. SustainabilitySector IndicatorsTransport ❙ Evidence of regular maintenance for infrastructure investments

❙ Long-term trends is travel times and Vehicle Operating Costs

❙ Long-term financial sustainability of maintenance schemes

❙ Implementation of identified policy reforms

Education ❙ Share of targeted institutions and training facilities operational and maintained

❙ Retention of specialized faculty and trained staff

❙ Financial sustainability of targeted training and educational facilities

Finance / Private Sector Development ❙ Economic and financial performance of selected projects (profitability)

❙ Retention rate of trained staff

❙ Repayment rate for financial support to SMEs

❙ Growth of loans to MSMEs

Multi-Sector ❙ Extent of implementation of targeted reforms

❙ Extent of reforms appropriately supported by public funds

❙ Retention of trained staff

❙ Demonstrated use and maintenance of improved systems

Power ❙ Evidence of regular maintenance for infrastructure investments

❙ Share of electrical distribution systems and transmission lines properly functioning

❙ Long-term financial sustainability of maintenance schemes

❙ Technical and financial capacity for maintenance and operation among local officials

❙ Implementation of identified policy reforms

❙ The extent of line losses (technical and non-technical)

❙ Deviation between unite cost and average tariff

85Annexes

An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

5. Crosscutting themes

Evaluation Question Indicators Data Sources5.1: To what extent do the Bank's interventions inclusive in terms of geography, age and gender?

❙ Degree and depth of gender analysis in the design and delivery of projects

❙ Share of interventions with specific components addressing gender inequality

❙ Share of projects utilizing gender checklists and gender-informed design

❙ Extent of institutional capacity development support provided for the identification and management of gender disparities.

❙ Geographical distribution of operations

❙ Share of bank interventions which explicitly consider or address regional disparities

❙ File review of project documentation

❙ Review of available PCRs/XSRs

❙ Interviews with task managers, local government stakeholders and implementing partners

❙ site visits

5.2: To what extent are the Bank's interventions environmentally sustainable and support the transition to green growth?

❙ Share of infrastructure projects which integrate environmental protection measures

❙ Share of new projects designed using climate checklists, environmental experts and climate-informed design

❙ Existence of mechanisms for monitoring and addressing environmental impact

❙ Extent of support provided for institutional capacity development surrounding environmental management and climate change

Judgement Criteria and RationaleThe assessment of crosscutting themes will involve an examination of the extent to which the themes of inclusive and green growth have been incorporated into the design and delivery of each project. A global assessment will be provided for the Bank's portfolio in Zambia against a six-point scale ranging from Highly Satisfactory to Highly Unsatisfactory

Judgement Criteria and RationaleThe assessment of sustainability will be conducted via two streams of analysis: (i) the extent to which the project design has adequately examined and addressed critical elements to promote the long-term sustainability of the project; and (ii) the demonstrated long-term viability and sustainability of project outputs and outcomes. The assessment will primarily focus on the first stream of analysis. The second stream of analysis will be used to inform sector-level and global ratings depending on the availability and credibility of relevant data.

A rating for sustainability will be provided at both the project and program levels against a six-point scale from Highly Satisfactory to Highly Unsatisfactory, as provided below.

6 – Highly Satisfactory: The project is not technically complex or complexities have been mitigated appropriately. The project does not depend on exogenous factors, or risks to the achievement of results are limited. The project design has involved relevant stakeholders who remain engaged in project implementation. There are few political or governance risks which could impact the implementation of the project. Relevant institutional capacity needs have been identified and addressed. Arrangements have been made for the continued financial sustainability of the project. Identification and mitigation of risks to the achievement and sustainability of results has been comprehensive, contributing the achievement of results. 5 – Satisfactory: Identification and mitigation of risks to sustainability have been adequate, but not comprehensive. Omissions have had no impact on the achievement of results. 4 – Moderately Satisfactory: Identification and mitigation of risks to sustainability have been adequate, but not comprehensive. Some limited impacts on the achievement of results have been observed. 3 – Moderately Unsatisfactory: Identification and mitigation of risks to sustainability has overlooked some key risks. Results have been achieved, but achievement has been influenced by uncontrolled risks.2 – Unsatisfactory: Mitigation of identified risks to sustainability has been inadequate with a material impact on the achievement of results.1 – Highly Unsatisfactory: Limited consideration has been given to risks to sustainability, with substantial limitations to the achievement of intended results.


6. Design and Delivery

Evaluation Question Indicators Data Sources

6.1: To what extent has the quality of CSPs been satisfactory? To what extent are the Bank's interventions coherent and well-coordinated internally?

❙ Extent to which CSPs have been identified as Satisfactory through reviews of Quality at Entry

❙ Extent of linkages among operations in different sectors.

❙ CSP documents

❙ Government Development Strategies

❙ Available literature

❙ Reviews of Quality at Entry

❙ Readiness Reviews

❙ Review of ESWs and knowledge products


❙ Interviews with Country Team, GRZ

❙ stakeholders, partners and co-financers

6.2: To what extent have the Bank's interventions been selective and strategic?

❙ Degree of concentration of the Bank's activities in selected focus areas

❙ Evidence of demonstrated expertise and comparative advantage within selected focus areas

❙ Extent of complementarity with the activities of other donors and partners

6.3: How has the Bank adapted its interventions to implementation challenges within the country?

❙ Evidence of ongoing analysis of implementation challenges and limitations

❙ Share of projects utilizing innovative design approaches and instruments

❙ Share of projects addressing emerging development needs

6.4: To what extent has the Bank's engagement been informed by policy dialogue with national actors, DPs and other interested stakeholders?

❙ Evidence of ongoing and strategic policy dialogue with the GRZ

❙ Evidence of appropriate stakeholder mapping and engagement with potential partners in the design of programs and projects

6.5: How well has the Bank leveraged its financial resources within the country through mechanisms such as productive partnerships and co-financing?

❙ Share of projects which utilize co-financing

❙ Evidence of strategic engagement with partners to identify opportunities for co-financing

❙ Evidence of strategic selection of projects as candidates for partnership and co-financing

6.6: To what extent has the Bank acted as a knowledge broker, advisor and convener? To what extent are the Bank's interventions coherent and well-coordinated internally?

❙ Extent and quality of sector and thematic analyses

❙ Evidence of ongoing research and ESW to inform program and project design

❙ Evidence of TA provided to inform necessary policy and institutional reforms

❙ Evidence of ongoing policy dialogue with government stakeholders to identify opportunities for analytical support and partnership.

6.7: To what extent has the Bank cooperated with other DPs to ensure complementarity and reduce overlaps?

❙ Evidence of donor mapping to inform the Bank's strategic positioning

❙ Extent to which the Bank's interventions overlap with those of other development institutions

❙ Share of the Bank's interventions co-financed by other DPs

❙ Degree of collaboration with other DPs in project design, analytical work, monitoring and reviews

❙ Extent of participation and leadership in donor partnership groupings

87Annexes

An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

Judgement Criteria and RationaleThis assessment will involve consideration of the extent to which the Bank has successfully implemented strategic operating mechanisms to promote the achievement of sustainable development results and maximize its value as a DPs. These mechanisms, identified in corporate strategies such as the TYS and the MTS, include:

❙ Implementing high-quality strategies which are informed by dialogue and consultations with RMCs;

❙ Ensuring selectivity in operations;

❙ Providing innovative solutions to development challenges;

❙ Providing knowledge and technical advice;

❙ Leveraging financial resources through convening co-financers, including other DPs and the private sector; and

❙ Maintaining effective partnerships with other DPs.

A global rating will be provided for management of the Bank's portfolio in Zambia against a six point scale ranging from Highly Satisfactory to Highly Unsatisfactory. The rating will be made based on a triangulation of evidence from project documents, interviews with key stakeholders and site visits.

With respect to the Quality of Project Design, the flowing rating scale will be applied:

6 – Highly Satisfactory: The project design was fully conducive to achieving the project results. The original design was solid and remained appropriate throughout implementation; no adjustments to the scope, implementation arrangements or technical solutions were required to ensure the achievement of the intended outcomes and outputs.

5 – Satisfactory: The project design was largely conducive to achieving the project results. The original design was solid and remained appropriate throughout implementation; minor adjustments to the scope, implementation arrangements or technical solutions were required to ensure the achievement of the intended outcomes and outputs.

4 – Moderately Satisfactory: The project design was moderately conducive to achieving the project results. The original design was to some extent, sound and remained appropriate throughout implementation; adjustments to the scope, minor implementation arrangements or technical solutions were required and they were carried out in a timely manner to ensure the achievement of the intended outcomes and outputs.

3 – Moderately Unsatisfactory: The design was somewhat conducive to achieving the project results. The original design was either weak or lost its relevance during implementation; major adjustments to the scope, implementation arrangements or technical solutions were required during implementation, but these were done with substantial delays which negatively affected the achievement of the intended outcomes and outputs.

2 –Unsatisfactory. From approval to closure, the design was marginally conducive to achieving the project results. The original design was weak and remained irrelevant. Major adjustments to the scope, implementation arrangements or technical solutions were required during implementation, but these were not done which negatively affected the achievement of the intended outcomes and outputs.

1 – Highly Unsatisfactory. The project design was fully not conducive to achieving the project results. The original design was weak and remained irrelevant during implementation. Major adjustments to the scope, implementation arrangements or technical solutions were required during implementation, but these were not done which negatively affected the achievement of the intended outcomes and outputs.


Judgement Criteria and RationaleWith respect to the Quality at Entry of Country Strategies, the following rating scale will be applied:

6 – Highly Satisfactory: The Bank’s strategy is based on a compelling intervention logic demonstrating a thorough understanding of the country’s evolving context, proposing solutions fully adapted to this context in all areas of intervention, and showing substantive innovation (analysis and product mix) over time in responding to challenges to achieving results. The Bank’s strategy presents a compelling analysis of the respective positioning of development partners, areas of comparative advantage and matching of this analysis with the evolving context and challenges of the country to define priority areas of assistance for the Bank and the interconnections between those from a programmatic perspective.

5 –Satisfactory: The Bank’s strategy is based on a sound intervention logic demonstrating a thorough understanding of the country’s evolving context, proposing solutions variably adapted to this context depending on the area of intervention, and showing some innovation (analysis and product mix) over time in responding to challenges to achieving results. The Bank’s strategy presents a clear analysis of the respective positioning of development partners, areas of comparative advantage and matching of this analysis with the evolving context and challenges of the country to define priority areas of assistance for the Bank but does not fully articulate interconnections and integration constraints in the program.

4 – Moderately Satisfactory: The Bank’s strategy is based on a sound intervention logic demonstrating a good understanding of the country’s evolving context, proposing solutions variably adapted to this context depending on the area of intervention, and not showing much innovation (analysis and product mix) over time in responding to challenges to achieving results. The Bank’s strategy presents an analysis of the respective positioning of development partners and areas of comparative advantage but the analysis does not fully show how this translates into priority areas of assistance for the Bank matching the evolving context and challenges of the country.

3 – Moderately Unsatisfactory: The Bank’s strategy is based on an intervention logic showing partial understanding of the country’s evolving context, proposing solutions variably adapted to this context depending on the area of intervention, and not showing much innovation (analysis and product mix) over time in responding to challenges to achieving results. The Bank’s strategy proposes an analysis of positioning and comparative advantage. Priority areas of assistance aligned with needs are proposed without relating them to this analysis.

2 – Unsatisfactory: The Bank’s strategy is based on an unclear intervention logic showing low understanding of the country’s evolving context, proposing “business as usual” solutions variably adapted to this context depending on the area of intervention. The Bank’s strategy proposes a basic analysis of positioning and comparative advantage but fails to articulate clearly the result in terms of areas of intervention which in practice overlap with areas of intervention of the ongoing portfolio.

1 – Highly Unsatisfactory: The Bank’s strategy is disconnected from the evolving context of the country and used as justification for an ongoing portfolio of operations designed and implemented as a compilation of projects agreed with direct national counterparts in the area covered. The Bank’s strategy replicates areas of intervention of the ongoing portfolio without any convincing analysis of positioning.

89Annexes

An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

7. Management for Results

Evaluation Question Indicators Data Sources

7.1: To what extent has the Bank's supervision of its interventions been effective?

❙ Extent to which projects have been supervised on an annual basis

❙ Extent to which supervision has been supported by field offices

❙ Share of projects which underwent a mid-term review of progress


❙ Review of PCRs, XSRs, mid-term reviews and BTORs

❙ Interviews with Task Managers, Field Office Staff, implementation units and GRZ stakeholders

❙ Site visits

7.2: How has the Bank used supervision and results information to guide the management of its interventions?

❙ Evidence that supervision reports have been used to inform ongoing implementation

❙ Evidence of implementation and design changes made in response to ongoing supervision and monitoring

7.3: To what extent has the Bank successfully implemented a performance measurement strategy which focuses on the achievement of outputs, outcomes and impacts?

❙ Share of projects with an appropriate results measurement framework, including clear targets

❙ share of projects possessing appropriate baseline data

❙ Evidence of regular data collection against established results indicators

7.4: To what extent have lessons learned been identified and incorporated into subsequent projects and CSPs?

❙ Extent to which evidence from PCRs/XSRs, Mid-term reviews and evaluations have been incorporated into CSPs

❙ Evidence of incorporation of lessons learned into the design of new projects

7.5: How has the Bank worked with the GRZ to enhance its own systems of Results-Based Measurement?

❙ Conduct of a needs assessment to identify capacity challenges for results based management

❙ Implementation of regular dialogue or capacity development support to promote ongoing management for development results.

7.6: To what extent has the Bank identified, monitored and addressed potential implementation risks?

❙ Evidence of identified implementation risks and mitigation measures

❙ Evidence of ongoing risk monitoring

❙ Use of risk monitoring information to inform project design and implementation

Judgement Criteria and Rationale

A final rating will be provided at the country level based on an assessment of the extent to which the Bank has successfully implemented a rigorous performance measurement regime and used this information to manage risks, guide implementation and inform the development of new projects and country strategies. In addition, this assessment will involve an examination of the Bank's efforts to develop capacity for results based management within Zambia.

A global rating will be provided based on a 6-point scale ranging from Highly Satisfactory to Highly Unsatisfactory based on a triangulation of evidence from project documents, interviews and site visits.


Annex D — Theory of Change for the Bank's Assistance to Zambia (2002-2015)

Activities / Outputs Immediate Outcomes

Reduced travel times and VOC

Improved road networks and safety

Increased traffic throughput

Increased energy generation

Increased access to markets

Increased use of improved irrigation systems

Increased land under cultivation

Increased SME lending

Increased profitability and business expansion

Increased availability of financial products for SMEs

Job creation

Improved terms of finance

Implementation of new systems (IFMIS, TSA and IPPD2)

New legislation developed

Development of DACF and Local Government Service

Construction of schools and hospitals

Regulatory Reforms for Private Sector

Improved commitment controls

Reduced wage expenditure

Strengthened accountability functions

Increased financial and administrative decentralization

Reduced time and cost to start a business

Reduced time to import and export

Increased access to improved WSS sources

Increased volumes of treated water

Increased reliability of water services

Reduced unaccounted for water

Enhanced quality of TVET

Increased TVET enrollment

Construction of roads, bridges and border crossings

Construction of Power stations, sub-stations and transmission lines

Implementation of ESMP

Provision of Technical Assistance

Creation of out-grower schemes, market schemes and farmer groups

Provision of training to farmers

Implementation of improved irrigation technologies

Establishment of new farm blocks

Rehabilitation of TVET Facilities

Provision of Training for Educators

Provisions of loans to SMEs

Development of an open cast mine and supporting infrastructure

Technical Assistance to FIs

Provision of LOC and PCGF to FIs

Water points and sanitation facilities constructed

Hygiene promotion training

Establishment of community management groups and public utilities

Rehabilitation of water treatment and sewerage infrastructure

91Annexes

An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

Intermediate Outcomes Strategic Outcomes

INCLUSIVE AND SUSTAINABLE GROWTH

Private Sector and Finance

Social Sector

Power and Transport

Agriculture

Multi-Sector

Water Supply and Sanitation

Reduced debt and expenditure variance

Reduced urban /rural gaps in service delivery

Increased business development

Increased regional trade

Increased access to social services

Increased energy connections

Increased reliability of power supply

Increased incomes

Increased agricultural productivity

Reduction of skills gaps

Increased business development

Increased government revenues

Increased access to finance

Reduced disease burden

Increased use and maintenance of WSS facilities

Improved Agricultural Productivity, Trade and Diversification

Increased Access to Basic Services

Creation of an Enabling Business Environment

Transparent and Accountable Public Financial Management


Annex E — List of Documents Consulted

African Development Bank Group (1999) "Bank Group Policy on Good Governance."African Development Bank Group (2002) "African Development Bank Group Strategic Plan 2003-2007."African Development Bank Group (2003) "Zambia - Central Province Eight Centers Water Supply and Sanitation Project - Appraisal Report.African Development Bank Group (2004a) "Multinational - Project to Support the Lake Tanganyika Integrated Regional Development Program – PRODAP (Project Appraisal Report)." African Development Bank Group (2004b) "Multinational Program: Proposal for Financial Support for Statistical Capacity Building for Regional Member Countries under the International Comparison Program for Africa."African Development Bank Group (2004c) "Multinational Proposal for an ADF Grant of UA 5 million to Fund to African Virtual University Support Project."African Development Bank Group (2005) "Zambia - Humanitarian Emergency Assistance to the 2005 Drought-Affected Communities (Proposal)."African Development Bank Group (2005) "Zambia 2005 Country Portfolio Performance Review."African Development Bank Group (2006a) "Document de stratégie pays 2002-2004 mise à jour 2006."African Development Bank Group (2006b) "Poverty Reduction Budget Support (PRBS) Appraisal Report."African Development Bank Group (2006c) "Investment Proposal of USD 43,000,000 to the Lumwana Copper Project.African Development Bank Group (2006d) "Zambia: The National Rural Water Supply and Sanitation Program - Appraisal Report."African Development Bank Group (2006e) "African Development Bank, Government of the Republic of Zambia, Zambia: Small-Scale Irrigation Project (SIP): Mid-Term Review Mission - 14TH – 25TH May 2007 (Aide Memoire)." African Development Bank Group (2006f) "Southern African Development Community (SADC) Region: The Strengthening Institutions for Risk Management of Transboundary Animal Diseases (TADs)."African Development Bank group (2006g) "Zambia - Lumwana Copper Mine - Investment Proposal of USD 43,000,000 to the Lumwana Copper Project."African Development Bank Group (2006h) "Multinational - Proposal for an ADF Grant of UA 5,660,000 to fund the Enhancing Procurement Reforms and Capacity Project."African Development Bank Group (2007a) "Joint Assistance Strategy for Zambia (2007-2010)."African Development Bank Group (2007b) "Zambia: Small-Scale Irrigation Project (SIP) Mid-Term Review Mission (14TH – 25TH May 2007) (aide memoire)."African Development Bank Group (2008a) "Zambia: The Nkana Water Supply and Sanitation Project - Appraisal Report."African Development Bank Group (2008b) "African Development Bank Group Medium Term Strategy 2008-2012."African Development Bank Group (2008c) "Project Completion Report: Poverty Reduction Budget Support."African Development Bank Group (2008d) "Second Poverty Reduction Budget Support (PRBS II) Appraisal Report."African Development Bank Group (2008e) "Strategy Update for the Bank's Private Sector Operations."

93Annexes

An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

African Development Bank Group (2008f) "Zambia - Proposal for a Line of Credit of USD 10 Million and a Partial credit Guarantee Facility of USD 8 million to Zambia National Commercial Bank."African Development Bank Group (2008g) "Zambia: Proposal for USD 3.5 Million LOC to Investrust PLC."African Development Bank Group (2008h) "Zambia: Emergency Humanitarian Food Assistance to Flood Victims (Appraisal Report)."African Development Bank Group (2009a) "Back to Office Report: Mission to Zambia - Supervision Mission of the Lumwana Mining Project."African Development Bank Group (2009b) "Guarantee Agreement between the African Development Bank Group, USAID and Zambia National Commercial Bank."African Development Bank Group (2009c) "Aide Memoire: Budget Support Dialogue and Identification of PRBS III Mission."African Development Bank Group (2009d) "BTOR of Poverty Reduction Budget Support Review and Identification of PRBS III Mission."African Development Bank Group (2009e) "Enabling Water Management for Increased Productivity and Resilience among Traditional Farmers in Zambia (Project Appraisal Report)."African Development Bank Group (2009f) "BTOR: Mission to Zambia - Supervision of Lumwana Mining Project."African Development Bank Group (2009g) "Letter of Agreement - Fund for African Private Sector Assistance - Grant for Pulse Financial Services Ltd."African Development Bank Group (2010a) "BTOR of Third Poverty Reduction Budget Support Appraisal Mission."African Development Bank Group (2010b) "Second Poverty Reduction Budget Support (PRBS II) Project Completion Report."African Development Bank Group (2010c) "Third Poverty Reduction Budget Support (PRBS III) Appraisal Report."African Development Bank Group (2010d) "Aide Memoire: AfDB Mission on Appraisal of a Poverty Reduction Budget Support program (PRBS III)."African Development Bank Group (2010e) "BTOR of Zambia PRBS 2009 PAF Annual Assessment Review."African Development Bank Group (2010f) "Fund for African Private Sector Assistance - Letter of Agreement between African Development Bank Group and African Training a Management Services Foundation."African Development Bank Group (2010g) "Zambia - Extended Supervision Report on the Lumwana Copper Mining Project."African Development Bank Group (2010h) "Inter-office Memorandum- Line of Credit Agreement between the African Development Bank Group and Zambia National Commercial Bank."African Development Bank Group (2010i) "Multinational - The Nacala Road Corridor Project – Phase II" (Project Appraisal Report).African Development Bank Group (2010j) "Zambia - Nacala Road Corridor-Phase II: Environmental and Social Impact Assessment Summary."African Development Bank Group (2010k) "Zambia - Humanitarian Emergency Assistance to 2009/2010 Flood Mitigation (Request for Grant)."African Development Bank Group (2010l) "Zambia Country Portfolio Performance Review."African Development Bank Group (2010m) "XSR on the Lumwana Copper Mining Project."


African Development Bank Group (2011a) "Back to Office Report - Investrust Bank Line of Credit and Technical Assistance."African Development Bank Group (2011b) "Republic of Zambia 2011-2015 Country Strategy Paper."African Development Bank Group (2011c) "Country Strategy Paper for Zambia 2011-2015."African Development Bank Group (2011d) Multinational: Kazungula Bridge Project (Environmental and Social Impact Assessment Summary). African Development Bank Group (2011e) "Multinational - Republic of Zambia and Republic of Botswana - Kazungula Bridge Project" (SADC North-South Transport Corridor Improvement) (Project Appraisal Report).African Development Bank Group (2011f) "Fourth Poverty Reduction Budget Support - Project Appraisal Report."African Development Bank Group (2012a) "Third Poverty Reduction Budget Support (PRBS III) Project Completion Report."African Development Bank Group (2012b) "Bank Group Policy on Policy Based Operations."African Development Bank Group (2012c) "Back to Office Report - Supervision Mission on the Line of Credit of 3.5 Million Dollars Extended to Investrust Bank."African Development Bank Group (2012d) "Project Concept Note - Africa SME Program."African Development Bank Group (2012e) "FAPA Project Status Report - August 2012."African Development Bank Group (2012f) "FAPA Project Status Report - March 2012."African Development Bank Group (2012g) "Development of Operational Guidelines for Investments in Multipurpose Small Dams (Appraisal Report)."African Development Bank Group (2012h) "Zambia - Community Water Management Improvement Project (Farmers Clubs) for Traditional Farmers in Mkushi, Kapiri Mposhi, Masaiti and Chingola Districts (Aide Memoire Supervision Mission: 6 to 10 March 2012)".African Development Bank Group (2012g) "Working Paper Series - Bank Financing to Small and Medium Enterprises in East Africa - Findings of a Survey in Kenya, Tanzania, Uganda and Zambia."African Development Bank Group (2012h) "Itezhi-Tezhi Power Project - Summary of the Environmental and Social Impact Assessment."African Development Bank Group (2012i) "Itezhi-Tezhi Hydropower and Transmission Line Project - Appraisal Report."African Development Bank Group (2012j) "Zambia - Poverty Reduction Budget Support Program PRBS IV Back to Office Report."African Development Bank Group (2012k) "Nacala Corridor Environmental Management Plan."African Development Bank Group (2013a) "Back to Office Report - Supervision Mission on the Line of Credit of 3.5 Million Dollars Extended to Investrust Bank."African Development Bank Group (2013b) "Extended Supervision Report - Investrust."African Development Bank Group (2013c) "ADOA Rating, Africa SME Program."African Development Bank Group (2013d) "Private Sector Development Policy."African Development Bank Group (2013e) "Ten Year Strategy for the African Development Bank Group."African Development Bank Group (2013f) Multinational (Malawi/ Zambia) "Nacala Road Corridor Development Project- Phase IV" (Project Appraisal Report).African Development Bank Group (2013g) "ZAMBIA Central Province 8 Centers Water Supply and Sanitation - PCR".

95Annexes

An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

African Development Bank Group (2013h) "Zambia - Livestock Infrastructure Support Project - LISP (Project Preparation Facility)." African Development Bank Group (2013i) "Zambia: Strengthening Climate Resilience in the Kafue Sub-Basin."African Development Bank Group (2013j) "Zambia: Livestock Infrastructure Support Project (LISP) - Project Preparation Facility (PPF)." African Development Bank Group (2013k) "Zambia - Combined 2011-2015 Country Strategy Paper Mid-Term Review and Country Portfolio Performance Review."African Development Bank Group (2013l) "Zambia Maamba Collieries Power Generation project - Senior Loan of USD 150 million."African Development Bank Group (2013m) "Disbursement Letter - Support to Science and Technology Education Project."African Development Bank Group (2013n) "Support to Science and Technology Education Project -Appraisal Report."African Development Bank Group (2013o) "Readiness Review - SSTEP."African Development Bank Group (2013p) "Development Effectiveness Review - Zambia."African Development Bank Group (2014a) "Inter-office Memorandum - Line of Credit to CETZAM Financial Services: Fulfilment of Conditions Precedent to First Disbursement."African Development Bank Group (2014b) "Project Status Report - CETZAM Financial Services 03/31/2014."African Development Bank Group (2014c) "Project Status Report - CETZAM Financial Services 11/14/2014."African Development Bank Group (2014d) "Inter-office Memorandum - Satisfaction of Conditions Precedent and Legal Clearance for Disbursement."African Development Bank Group (2014e) "Line of Credit Agreement between African Development Bank Group and CETZAM Financial Services."African Development Bank Group (2014f) "Board Credit Risk Memorandum - CETZAM Financial Services."African Development Bank Group (2014g) "African Development Bank Group Financial Sector Development Policy and Strategy."African Development Bank Group (2014j) "2013-2017 Private Sector Development Strategy."African Development Bank Group (2014i) "Project Completion Report for Technical Assistance Operations - Enhancing Zambian SMEs Competitiveness and Access to Finance."African Development Bank Group (2014j) "Back to Office Report - ZANACO Line of Credit."African Development Bank Group (2014k) "Nacala Road Corridor Project - Phase II" (Implementation Progress and Results Report 11/11/2014).African Development Bank Group (2014l) "Multinational: Kazungula Bridge Project" (Supervision Mission 13-17 October 2014 - Back to Office Report).African Development Bank Group (2014m) "Zambia - Transforming Rural Livelihoods in Western Zambia – (NRWSSP) Phase II - Project Appraisal Report."African Development Bank Group (2014n) Project to Support Lake Tanganyika Integrated Regional Development Program – PRODAP (Implementation Progress and Results Report 17/01/2014).African Development Bank Group (2014o) "Zambia - Agriculture Productivity and Market Enhancement Project (GAFSP Grant Proposal)."


African Development Bank Group (2014p) "Zambia - Lake Tanganyika Development Project (Project Appraisal Report)."African Development Bank Group (2014q) "Zambia: Small-Scale Irrigation Project (SIP): 2014 Updated Baseline Survey, Ministry of Agriculture and Cooperatives, Government of the Republic of Zambia."African Development Bank Group (2014r) "Finnish Financed Small Scale Irrigation Project Supervision Mission 1-5 December 2014 (Back to Office Report)."African Development Bank Group (2014s) "Zambia - Agriculture Productivity and Market Enhancement Project (APMEP) (Project Appraisal Report)."African Development Bank Group (2014t) "Development of Operational Guidelines for Investments in Multipurpose Small Dams (Implementation Progress and Results Report - IPR)."African Development Bank Group (2014u) "Multinational - Zambia and Zimbabwe Kariba Dam Rehabilitation Project - Appraisal Report."African Development Bank Group (2014v) "Zambia - Fourth Poverty Reduction Budget Support PCR.”African Development Bank Group (2014w) "Zambia - PRBS IV Audit Comments."African Development Bank Group (2014x) "Nacala Road Corridor Project Phase II - BTOR."African Development Bank Group (2014y) "Zambia - Supervision of the National Rural Water Supply and Sanitation Program.''African Development Bank Group (2015a) "Back to Office Report - ZANACO Line of Credit."African Development Bank Group (2015b) "Back-to-Office Report - CETZAM Financial Services."African Development Bank Group (2015c) "Credit Risk Committee Secretariat - CETZAM Lessons Learnt.African Development Bank Group (2015d) "Republic of Zambia 2016-2020 CSP - Draft Concept Note."African Development Bank Group (2015e) "African Economic Outlook - Zambia."African Development Bank Group (2015f) "PCR - Nkana Water Supply and Sanitation Project."African Development Bank Group (2015g) "Project to Support Lake Tanganyika Integrated Regional Development Program – PRODAP (Project Completion Report)."African Development Bank Group (2015h) "Zambia - 2011-2015 CSP Completion Report."African Development Bank Group (2015i) "ITT Power Generation and Transmission Project - Supervision Mission 25 May-06 June - Aide Memoire."African Development Bank Group (2015j) "Back to Office Report - ITT Power Generation and Transmission Project."African Development Bank Group (2015k) "Evaluation of the Small-scale Irrigation Project."African Development Bank Group (2016a) "Draft - XSR - Line of Credit to ZANACO PLC."African Development Bank Group (2016b) "Back to Office Report - Zambia Kariba Dam Rehabilitation Project, ITT Power Project - December 2015."African Development Bank Group (2016c) "Zambia 2016-2020 Country Strategy Paper and 2015 Country Portfolio Performance Review - Draft Concept Paper."African Health Workforce Observatory (2010) "HRH Factsheet - Zambia."Africa Oil and Power (2016) "Power Privatization Series - Zambia Case Study."African Training and Management Services (2011) "Enhancing Zambian Small and Medium Enterprises Competitiveness and Access to Finance (2010-2012) - Biannual Progress Report)."Barrick Gold Corporation (2012) "Responsibility Report - 2012."

97Annexes

An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

Barrick Gold Corporation (2013) "Annual Report - 2012."Barrick Gold Corporation (2013) "Responsibility Report - 2013."Barrick Gold Corporation (2014) "Annual Report - 2013."Barrick Gold Corporation (2014) "Responsibility Report - 2014."Barrick Gold Corporation (2015) "Annual Report - 2014."Barrick Gold Corporation (2016) "Annual Report - 2015."CETZAM Financial Services Plc (2014) "CETZAM Framework for Environmental and Social Management."CETZAM Financial Services Plc (2015) "Summary of Utilization of AFDB LOC."Department for International Development (2015) "Project Completion Report - Growth and Poverty Reduction Grant - Zambia."Desjardins Développement International (2013) Impact Assessment of EFC Zambia."Development Assistance from People to People (2011a) "Community Water Management and Improvement for Traditional Farmers Project in Chingola, Kapiri Mposhi, Masaiti and Mkushi Districts - Annual Report 2011."Development Assistance from People to People (2011b) "Value Chain Analysis and an Analysis of Selected Opportunities for Community Water Management Improvement Project for Traditional Farmers in Mkushi, Kapiri Mposhi, Masaiti and Chingola districts."Development Assistance from People to People (2011c) "Report on Baseline Study for Community Water Management and Improvement for Traditional Farmers Project in Chingola, Kapiri Mposhi, Masaiti and Mkushi Districts."Development Assistance from People to People (2011d) "Community Water Management and Improvement for Traditional Farmers Project in Chingola, Kapiri Mposhi, Masaiti and Mkushi Districts. Annual Report 2011."Development Assistance from People to People (2012a) "DAPP Farmers' Club: Northern Success Stories."Development Assistance from People to People (2012b) "Community Water Management and Improvement for Traditional Farmers Project in Chingola, Kapiri Mposhi, Masaiti and Mkushi Districts - Annual Report 2012."Development Assistance from People to People (2013) "Community Water Management and Improvement for Traditional Farmers Project in Chingola, Kapiri Mposhi, Masaiti and Mkushi Districts - Annual Report 2013."Development Assistance from People to People (2014) "The Community Water Management Improvement Project for Traditional Farmers in Mkushi, Kapiri Mposhi, Masaiti and Chingola Districts (Project Completion Report)."Finscope (2015) "Finscope Zambia 2015."FSDP Secretariat, Bank of Zambia (2010) "Finscope Zambia Survey 2009."FSDP Secretariat, Bank of Zambia (2015) "Finscope Zambia Survey 2015."Government of the Republic of Zambia (2005) "Zambia Public Financial Management Performance Report and Performance Indicators - PEMFA Program Evaluation."Government of the Republic of Zambia (2006a) "Fifth National Development Plan 2006-2010: Broad-based Wealth and Job Creation through Citizenry Participation and Technological Advancement."Government of the Republic of Zambia (2006b) "Vision 2030 for Zambia: A Prosperous Middle-Income Country by 2030."Government of the Republic of Zambia Ministry of Finance and Economic Planning (2006) "Fifth National Development Plan 2006-2010."Government of the Republic of Zambia Ministry of Commerce, Trade and Industry (2007) "Zambia Commercial, Trade and Industrial Policy."


Government of the Republic of Zambia Ministry of Commerce, Trade and Industry (2008a) "Micro, Small and Medium Enterprise Development Policy."Government of the Republic of Zambia (2008b) "Letter of Development Policy." Government of the Republic of Zambia (2008c) "Zambia Public Financial Management Performance Report and Performance Indicators - PEMFA Program Evaluation Assessment and Update."Government of the Republic of Zambia (2008d) "Annual Report of the Roads Development Agency 2007."Government of the Republic of Zambia (2009) "Annual Report of the Roads Development Agency 2008."Government of the Republic of Zambia (2010a) "Public Expenditure Management and Financial Accountability Program Evaluation - Overview Report.Government of the Republic of Zambia (2010b) "Final Report on the 2009 Performance Assessment Framework"Government of the Republic of Zambia - Ministry of Finance and National Planning (2011a) "Sixth National Development Plan 2011-2015."Government of the Republic of Zambia - Ministry of Finance and National Planning (2011b) "Letter of Development Policy - Fourth Poverty Reduction Budget Support Loan."Government of the Republic of Zambia (2012a) "Zambia Public Financial Management Performance Report and Performance Indicators - PEMFA Program Evaluation Assessment and Update."Government of the Republic of Zambia - Ministry of Finance and National Planning (2012b) "Joint Assistance Strategy for Zambia (2011-2015)."Government of the Republic of Zambia (2013) "Public Expenditure Management and Financial Accountability Program Exit Report."Government of the Republic of Zambia Ministry of Agriculture and Livestock (2014a) "Small Scale Irrigation Project: Socio-Economic Impact of SIP on Beneficiaries (Case Study of Buleya Malima Irrigation Scheme)."Government of the Republic of Zambia - Ministry of Health (2014b) "2013-2014 Demographic and Health Survey."Government of the Republic of Zambia - Zambia Development Agency (2015a) "Zambian Mining Sector Profile."Government of the Republic of Zambia - Ministry of Finance and National Planning (2015b) "PRBS Performance Assessment Framework for 2012-2014."Government of the Republic of Zambia - Zambia Energy Regulation Board (2015c) "Statement on electricity Tariff Adjustments."Indaba Agricultural Policy Research Institute (2015d) "Status of Smallholder Crop Diversification in Zambia."International Council for Mines and Minerals (2012) "Enhancing Mining's Contribution to the Zambian Economy and Society."International Growth Centre (2012) "Zambia Regional Integration Policy Challenges."International Hydropower Association (2014) "Country Profile - Zambia."International Labor Organization (2007) "Expand Your Business Program - Training Manual for Growth-Oriented Enterprises."International Labor Organization (2012) "Final Evaluation Report - Zambia SME Support Project - Enhancing Zambian SMEs Competitiveness and Access to Finance."International Trade Centre (2010) "ITC Facilitates Access to Finance Program for Zambian SMEs."Investrust Bank PLC (2014) "Annual Report and Financial Statements - FY 2013."

99Annexes

An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

Investrust Bank PLC (2010) "Memorandum - fulfilment of conditions precedent."International Monetary Fund (2015) "Report on Article Four Consultation in Zambia 2015."IOB (2003) "Results of International Debt Relief (1990-1999)."Itezhi-Tezhi Power Corporation (2015) "Status Report - ITT Power Project."Moore Stephens (2014) "Audit of Financial Statements and Ex-Post Procurement Review of the Project Entitled Community Water Management Improvement Project for Traditional Farmers in Mkushi, Kapiri Mposhi, Masaiti and Chingola districts." NEPAD Infrastructure Project Preparation Facility (IPPF) 2009. Multinational – Mozambique / Malawi / Zambia Nacala Road Corridor Studies.Norad (2011) "Joint Evaluation of Support to Anti-Corruption Efforts (2002-2009)."OECD (2002) "DAC JV for Procurement Country Pilot Program Zambia - Assessment of Public Procurement System."OECD (2007) "OECD-DAC JV for Procurement Country Pilot Program - Zambia Assessment of Public Procurement Systems." OECD (2010) "Evaluation of the Joint Assistance Strategy for Zambia."OECD (2012) "Between High Expectations and Reality - An Evaluation of Budget Support in Zambia."Oxford Policy Management (2010) "Agriculture Case Study: Evaluation of Budget Support in Zambia."Transparency International (2012) "UN Convention against Corruption Civil Society Review: Zambia 2012."Transparency International (2015) "Corruption Perceptions Index 2015."United Nations Conference on Trade and Development (2014) "Challenges and Opportunities Facing Landlocked Countries - Zambia."UNDP (2013) "Millennium Development Goals (MDG) Report for Zambia - 2013."World Bank Group (2009) "Doing Business 2008."World Bank Group (2010) "Doing Business 2009."World Bank Group (2010a) "The Profile and Productivity of Zambian Enterprises."World Bank Group (2011) "Doing Business 2010."World Bank Group (2012) "Zambia Poverty Assessment - Stagnant Poverty and Inequality in a Natural Resource-based Economy."World Bank Group (2015a) "Doing Business 2014."World Bank Group (2015b) "Zambia Country Program Evaluation FY04-13."World Bank Group (2015c) "Zambia Economic Brief - Powering the Zambian Economy."Zambia Extractive Industries Transparency Initiative Council (2012) "Reconciliation Report for the Year 2011."Zambia Extractive Industries Transparency Initiative Council (2013) "Reconciliation Report for the Year 2012."Zambia Extractive Industries Transparency Initiative (2014) "Reconciliation Report for the Year 2013."Zambia National Farmers Union (2010) "Agricultural Productivity in Zambia: Has there been Progress?"ZANACO PLC (2013) Line of Credit Development Outcome Template - October 2013.ZANACO PLC (2014) Line of Credit Development Outcome Template - December 2014.ZANACO PLC (2015) Line of Credit Development Outcome Template - March 2015.ZANACO PLC (2012) "Environmental and Social Management Policy - Version 2.0."ZANACO PLC (2014) "AfDB Report - Use of the Partial Credit Guarantee Facility."


Annex F — Organizations Engaged - Stakeholder Interviews

In keeping with evaluation best practices as well as the agreement reached between the evaluation team and individual stakeholders, it is not possible to provide identifying information about interviewees. The table below provides an indication of the number of interviewees engaged from various organizations.

Organization or Stakeholder Group Number of Stakeholders Engaged

African Development Bank Group 18

Agence Française de Développement / Proparco 1

African Management Services 1

CETZAM Financial Services 1

Development Assistance from People to People 2

Development Bank of Southern Africa (DBSA) 2

Department for International Development (DfID) 1

European Investment Bank (EIB) 4

European Union 1

Food and Agriculture Organization of the UN (FAO) 1

Indaba Agricultural Policy Research Institute 1

International Finance Corporation (IFC) 2

International Fund for Agricultural Development (IFAD) 1

International Growth Centre 1

International Labor Organization (ILO) 1

International Monetary Fund (IMF) 1

Investrust Merchant Bank 2

Itezhi-Tezhi Power Corporation (ITPC) 1

Japan International Cooperation Agency (JICA) 2

KfW Benkengruppe 1

Lumwana Mining Company / Barrick Gold Corporation 7

National Water Supply and Sanitation Council (NWASCO) 2

Nkana Water and Sewerage Company 11

United States Agency for International Development (USAID) 1

World Bank 1

ZANACO Plc 3

Zambia Institute for Policy Analysis and Research (ZIPAR) 1

ZESCO 3

Other Civil Society 1

Project Contractors 6

Other Private Sector 4

Project Beneficiaries 31

District Council Officials 7

Government of the Republic of Zambia - Ministry of Agriculture and Livestock 2

Government of the Republic of Zambia - Central Statistics Office 4

101Annexes

An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

Organization or Stakeholder Group Number of Stakeholders Engaged

Government of the Republic of Zambia - Ministry of Education, Science, Vocational Training and Early Education

4

Government of the Republic of Zambia - Ministry of Energy 5

Government of the Republic of Zambia - Energy Regulation Board 6

Government of the Republic of Zambia - Ministry of Finance and National Planning 3

Government of the Republic of Zambia - Ministry of Fisheries 3

Government of the Republic of Zambia - Ministry of Health 1

Government of the Republic of Zambia - Ministry of Mines and Minerals 3

Government of the Republic of Zambia - Office of the Auditor General 3

Government of the Republic of Zambia - Office for the Promotion of Private Power Investment 1

Government of the Republic of Zambia - Roads Development Agency 5

Government of the Republic of Zambia - Zambia Development Agency 2

Government of the Republic of Zambia - Zambia Public Procurement Authority 2

Government of the Republic of Zambia - Zambia Revenue Authority 9

Total 176


BANK STRATEGY NATIONAL DEVELOPMENT PLANSCOUNTRY STRATEGY PAPER (2002-2004) POVERTY REDUCTION STRATEGY PAPER (2002)Strategic Objective:Poverty reduction through economic growth and social service deliveryAreas of Emphasis: ❙ Strengthening and climate-proofing agricultural infrastructure

❙ Increasing access to safe water and sanitation

❙ Increasing access to health and education services, particularly for vulnerable children

❙ Management of public debt

❙ Decentralization of governance

Strategic Objective:Poverty reduction and broad-based economic growthAreas of Emphasis: ❙ Diversification of production and exports through support to the agricultural sector

❙ Improved delivery of basic social services

❙ Crosscutting emphasis on HIV/AIDS and gender

ZAMBIA JOINT ASSISTANCE STRATEGY (2007-2010) FIFTH NATIONAL DEVELOPMENT PLAN (2006-2010)Strategic Objective:Increased wellbeing of the Zambian populationAreas of Emphasis: ❙ Strengthening of agricultural and water and sanitation infrastructure

❙ Development of regional transport corridors

❙ Regional power interconnectivity

❙ Promotion of domestic debt management

❙ Development of public procurement capacity

❙ Strengthening of accountability and transparency mechanisms

Strategic Objective:Broad-based wealth and job creation Areas of Emphasis: ❙ Development of economic infrastructure in the agricultural sector

❙ Human resources development

❙ Improved delivery of basic social services

❙ Accountable and transparent public expenditure management

COUNTRY STRATEGY PAPER (2011-2015) SIXTH NATIONAL DEVELOPMENT PLAN (2011-2015)Strategic Objective:Economic growth, diversification and poverty reductionAreas of Emphasis: ❙ Development and rehabilitation of national and regional transport infrastructure

❙ Strengthening power generation capacity and national and regional power connectivity

❙ Increased capacity for efficiency, transparent and accountable public financial management

❙ Private sector regulatory reform

❙ Increased access to finance

Strategic Objective:Poverty reduction through economic growth and diversificationAreas of Emphasis: ❙ Infrastructure development, including feeder roads, water canals, access roads and power infrastructure

❙ Promotion of agri-business

❙ Basic Service Delivery

Sources: The 2002 CSP and its update in 2006; the first Joint Assistance Strategy of Zambia 2007-10; and the 2011-15 CSP.

Annex G — Alignment of the Bank's CSPs with National Development Strategies

103Annexes

An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

Annex H — Zambia's Progress against MDG Indicators

Progress Against Selected MDG IndicatorsIndicator 2006-2008 2010-2013 Target 2015 LikelihoodGoal 1 - Eradicate Extreme Hunger and PovertyProportion of population below USD 1 (PPP) per day (percent)

51.0 42.3 29.0 Unlikely

Prevalence of underweight children (percent) 14.6 13.3 12.5 Likely

Goal 2 - Achieve Universal Primary Education Net enrolment in primary education (percent) 97.0 93.7 100 Attention Required

Primary Six Completion Rate (percent) N/A 90.9 100 Attention Required

Literacy Rate 15-24 year-olds (percent) 70 89 100 Attention Required

Goal 3 - Promote Gender Equality and Empower WomenRatio of females to males primary enrolment (girls per 100 boys)

0.97 0.99 1 Likely

Ratio of females to males secondary enrolment(girls per 100 boys)

0.73 0.89 1 Attention Required

Proportion of literate females to males (15-24) 0.8 0.87 1 Likely

Goal 4 - Reduce Child MortalityInfant mortality rate (per 1000 live births) 119 138 63.6 Unlikely

Under-five mortality rate (per 1000 live births) 70 76 35.7 Unlikely

Goal 5 - Improve Maternal HealthMaternal Mortality Rate (per 100,000 live births) 449 483 162 Unlikely

Proportion of births attended by skilled health personnel (percent)

46 46.5 100 Unlikely

Goal 6 - Combat HIV/AIDS, Malaria and other DiseasesHIV Prevalence rate (percent) N/A 14.3 15.6 Met

Population with advanced HIV infection with access to ARVs

N/A 79 80 Likely

New malaria cases per 1000 individuals N/A 330 < 225 Attention Required

Malaria fatality rate per 1000 individuals N/A 34 11 Attention Required

Goal 7 - Ensure Environmental SustainabilityProportion of the population using an improved drinking water source (percent)

40 36.9 25.5 Unlikely

Proportion of the population using an improved sanitation facility (percent)

36.1 percent 32.7 percent 87 percent Unlikely

Goal 8 - Develop a Global Partnership for DevelopmentODA (millions US$) 415 480 N/A

1 Latest data available during the specified period.Sources: African Development Bank 2012-2016 CSP and the 2010 UNDP / GRZ Millennium Development Goals Report.


1. OECD (2012) "Between High Expectations and Reality - An Evaluation of Budget Support in Zambia" pp. 50-52

2. Zambia Development Agency (2015) "Zambia Mining Sector Profile," p. 3. http://www.zda.org.zm/?q=content/mining-sector

3. IOB (2003) "Results of International Debt Relief (1990-1999)," at 1-3; OECD (2012) p. 54

4. OECD 2012 p. 55

5. OECD 2012 p. 56

6. World Bank Group (2015c) "Zambia Economic Brief - Powering the Zambian Economy," p. 7

7. CIA World Fact Book

8. World Bank Group (2015) p. 7

9. OECD 2012 p. 56; CIA World Fact Book (2016) https://www.cia.gov/library/publications/the-world-factbook/fields/2048.html

10. World Bank Group (2015) p. 10; World Bank Group (2016) "Macro Poverty Outlook for Zambia"

11. World Bank Group (2015) p. 7; OECD 2012 pp. 50-52.


13. Ibid. p. 17

14. Ibid. p. 7

15. Ibid. p. 13

16. Data from the World Development Indicators.

17. Government of the Republic of Zambia - Ministry of Health (2014b) "2013-2014 Demographic and Health Survey" p. 2

18. Ibid. p. 7

19. Ibid. p. 14

20. Ibid. p. 2

21. World Bank Group (2012) "Zambia Poverty Assessment - Stagnant Poverty and Inequality in a Natural Resource-based Economy" p. 1

22. Data from World Development Indicators


24. Ibid. p. 1; CIA World Fact Book

25. The International Growth Centre (2012) "Zambia Regional Integration Policy Challenges" p. 6

26. World Bank Group (2010b) "The Profile and Productivity of Zambian Businesses," p. 5

27. Ibid. p. 7

28. Ibid. p. 14

Endnotes

105Annexes

An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

29. Ibid. p. 10

30. African Development Bank Group (2012h) "Working Paper Series - Bank Financing to Small and Medium Enterprises in East Africa: Findings of a Survey in Kenya, Tanzania, Uganda and Zambia" p. 7

31. World Bank Group (2010b) p. 27

32. Data from World Integrated Trade Solution (WITS)

33. The evaluation has included one project approved in 2000, as it was not completed during the planned period but continued implementation through additional funding support and is now nearing completion. The portfolio has been updated with the four operations approved during 2015 to acknowledge the trend in priorities.

34. UNCTAD (2014) "Challenges and Opportunities Facing Landlocked Countries - Zambia," p. 12 http://unctad.org/meetings/en/Presentation/aldc2014_16_ppt_Zambia.pdf

35. World Bank Group (2010b) p. 27

36. Output averages were assessed across completed projects with a value assessed for each output. Outcomes values were similarly assessed for each project and averaged for each sector. Each outcome for which targets were achieved was assessed a value of 1.0, whereas outcomes which were partially achieved were assessed a value of 0.5. Items for which limited progress was achieved were assessed a value of 0.

37. CETZAM Financial Services Plc (2015) "Summary of Utilization of AfDB LOC"

38. Zambia EITI Council (2012) "Zambia Extractive Industries Transparency Initiative - Reconciliation Report for the Year 2011" (See also 2012, 2013 and 2014 reports).

39. ZANACO PLC (2014) Line of Credit Development Outcome Template - December 2014; African Development Bank Group; African Development Bank Group (2013b) "Extended Supervision Report - Investrust"; African Development Bank Group (2010m) "XSR on the Lumwana Copper Mining Project"

40. Ibid.

41. Barrick Gold Corporation (2012) "2012 Responsibility Report" p. 73

42. International Council on Mines and Minerals (2012) "Enhancing Mining's Contribution to the Zambian Economy and Society" p. 8

43. Barrick Gold Corporation (2016) "Annual Report 2015"

44. ICMM (2012) p. 86

45. Ibid. p. 85

46. African Development Bank Group (2015j) "Back to Office Report - ITT Power Generation and Transmission Project."

47. Ibid.

48. African Development Bank Group (2015g) "Project to Support Lake Tanganyika Integrated Regional Development Program (PRODAP) - Project Completion Report."

49. DAPP (2014) "Project Completion Report – The Community Water Management Improvement Project for Traditional Farmers in Mkushi, Kapiri Mposhi, Masaiti and Chingola Districts"

50. Ibid.

51. Ibid. p. 15


52. Assessment compiled against project triggers and outputs using information from: African Development Bank Group (2008c), (2009c), (2009d) and (2010b); and Government of the Republic of Zambia (2008d), (2010b) and (2011b).

53. Government of the Republic of Zambia (2010) "Final Report on the 2009 Performance Assessment Framework"

54. PEFA (2005) paragraphs 3.6.4-3.6.8.

55. Based on feedback from GRZ stakeholders.

56. PEFA 2008 at 3.4.33; PEFA (2012) p. 75.

57. PRBS III PCR p. DI

58. PRBS IV PCR p. DI

59. GoZ (2010) 2009-2010 PAF Report

60. PEFA 2008 paragraph 3.4.33; PEFA 2012 p. 75

61. African Development Bank (2015f) "Project Completion Report - Nkana Water Supply and Sanitation Project; African Development Bank (2013g) "Zambia - Central Province Eight Centers Water Supply and Sanitation Project - Project Completion Report; African Development Bank Group (2014) "Zambia - Supervision of the National Rural Water Supply and Sanitation Program - BTOR"

62. African Development Bank (2014k) "Nacala Road Corridor Project - Phase II - Implementation Progress and Results Report."

63. World Bank Group (2007) "Doing Business 2006;" World Bank Group (2015) "Doing Business 2014."


65. International Labor Organization (2012) "Final Evaluation Report - Zambia SME Support Project - Enhancing Zambian SMEs Competitiveness and Access to Finance."

66. IMF Data.

67. CPIA World Bank.

68. International Hydropower Association (2014) "Country Profile - Zambia" https://www.hydropower.org/country-profiles/zambia

69. Africa Oil and Power (2016) " Power Privatization Series - Zambia Case Study" http://africaoilandpower.com/index.php/2016/05/16/power-privatization-series-zambia-case-study/

70. Data from World Development Indicators "Access to electricity (percent population)"

71. African Health Workforce Observatory (2010) "HRH Factsheet - Zambia" http://www.hrh-observatory.afro.who.int/en/country-monitoring/92-zambia.html


73. Data from World Integrated Trade Service

74. Indaba Agricultural Policy Research Institute (2015) "Status of Smallholder Crop Diversification in Zambia" p. 4

75. Ibid p. 10

76. Oxford Policy Management (2010) "Agriculture Case Study: Evaluation of Budget Support in Zambia" at 176; Zambia National Farmers Union (2010) "Agricultural Productivity in Zambia: Has there been Progress?" p. 24

77. Government of Zambia (2015k) "Evaluation of the Small-scale Irrigation Project"

107Annexes

An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

78. 2011-2015 CSP Completion Report, Table 1

79. African Development Bank Group (2013) "Development Effectiveness Review 2013 - Zambia" p.36.

80. Barrick Gold 2012 Annual Report; Barrick Gold 2015 Annual Report.

81. African Development Bank Group (2015b) "Back-to-Office Report - CETZAM Financial Services"

82. IMF (2015) "Zambia - Article IV Consultation" p. 9.

83. Zambia Energy Regulation Board (2015) "Statement on Electricity Tariff Adjustments." http://www.erb.org.zm/press/statements/ElectricityTariffAdjudtment2015.pdf

84. Bloomberg News, February 17, 2016 " Zambia Plans to Re-visit Power Tariff Increase After Reversal." http://www.bloomberg.com/news/articles/2016-02-16/zambia-plans-to-revisit-power-tariff-increase-after-reversal

85. As confirmed by World Bank (2015) "Zambia Country Program Evaluation FY2004-13."

86. OECD 2012 p. 54

87. IMF (2015) p. 12

88. 2010 Aide Memoire of Annual Joint Supervision p. 7.3

89. Ibid.

90. Barrick Gold (2014b) "2013 Responsibility Report," p. 13.

91. Government of the Republic of Zambia (2012b) "Joint Assistance Strategy for Zambia ii (2011-2015).

Des

ign

& la

yout

: CR

ÉON

• w

ww

.cre

onde

sign

.net

About this Evaluation

This evaluation presents the performance of the African Development Bank’s assistance in Zambia during the 2002–2015 period and its contribution to the development of the country in some key areas including agriculture, governance, infrastructure development, transport, private sector development, access to water supply and sanitation, promotion of child welfare, etc. It was conducted for two purposes: (i) to inform the development of the Zambia Country Strategy Paper for 2016–2020; and (ii) to support IDEV's Comprehensive Evaluation of the Bank's Development Results. The assessment reveals, among others, that the different Country Strategy Papers have been well-aligned with both national development plans and the Bank's comparative advantage, also its portfolio has become more coherent, adopting an integrated approach to development challenges. Moreover, the evaluation confirms that political and governance risks are becoming an increasingly important factor in the sustainability of projects. However, the evaluation points out that more efforts are still needed to scale up private sector initiatives and mainstream gender in development projects.

An IDEV Country Strategy Evaluation

African Development Bank GroupAvenue Joseph Anoma, 01 BP 1387, Abidjan 01, Côte d’IvoirePhone: +225 20 26 20 41E-mail: [email protected]

idev.afdb.org