nigeria country strategy and program evaluation...

Inception report – final (redacted)

Nigeria Country Strategy and Program

Evaluation 2004-14

October 2015

Table of contents

3

Inception report

List of abbreviations 5

1 Introduction 6

1.1 Context and background of the evaluation 6

1.2 The socio-economic backdrop for the AfDB’s engagement 6

2 Purpose and objectives 9

3 Evaluation questions and universe 11

3.1 The Evaluation Questions 11

3.2 The evaluation universe 12

4 Methodology 13

4.1 Evaluation design- a theory based evaluation focused on learning 13

4.2 Evaluation components – a multi-layered approach 13

The strategic level 14

Portfolio level 14

Project level 14

4.3 Selecting projects eligible for PRA 14

4.4 Identifying different project types 17

4.5 Data collection 18

4.6 Sampling 19

4.7 Practical issues relating to data collection 20

4.8 Data analysis and ratings- a rigorous approach 21

5 Limitations, risks and mitigation 23

6 The evaluation process – managing for quality, timeliness and impact 25

6.1 Principles 25

6.2 Deliverables and timeline 25

6.3 Roles and responsibilities 26

6.4 Quality assurance 26

6.5 Dissemination – sharing knowledge 27

Annex 1: Project Portfolio Error! Bookmark not defined.

Annex 2: List of persons consulted during September 2015 scoping mission 31

Annex 3: Overview of AfDBs CSPs for Nigeria during 2004-2014 33

Annex 4: Evaluation matrix 35

Annex 5: Country ToC 43

Annex 6: Portfolio review template 46

4

Inception report

Annex 7: Project results assessments templates 49

Public sector project template 49

Private sector projects template 54

Annex 8: Examples of rating guidance: Effectiveness for public and private sector projects 59

Public sector projects 59

a) Achievement of outputs 59

b) Achievement of outcomes 60

Private sector projects 61

a) Achievement of outputs 61

b) Achievement of outcomes 61

Annex 9: Example of project ToC and indicators identified for collection 65

Annex 10: Examining lines of credit 67

Prior to Visit to the bank/FI 72

Visit to the bank/FI 72

Annex 11: Examining the knowledge aspect of the Country Strategy Papers and Programs 73

Annex 12: Interview protocol 75

Annex 13: List of key informants for semi-structured interviews 77

Public sector projects stakeholder mapping 78

Private sector projects stakeholder mapping 79

Annex 14: Document mapping 81

Mapping of available project documents 81

Public sector documents 81

Private sector documents 82

Mapping of available non-project documents 83

AfDB Documents 83

Other sources 85

Annex 15: Possible structure of main technical report and sector reports 86

Annex 16: Timeline 89

5

Inception report

List of abbreviations

ADF Agricultural Development Fund

AFW African Water Facility

CDE Committee on Development Effectiveness

CEDR Comprehensive Evaluation of Development Results

CSP Country Strategy Paper

CSPE Country Strategy and Program Evaluation

DIR Development Impact Report

ECOWAS Economic Community of West African States

EIB European Investment Bank

EQ Evaluation Question

ESW Economic and Sector Work

FI Financial intermediary/institution

GDP Gross Domestic Product

GHS General Household Survey

HNLSS Harmonised Nigeria Living Standards Survey

IDEV Independent Development Evaluation function

IFC International Finance Corporation

LA Loan agreement

LoC Line of Credit

ODA Official Development Assistance

OECD DAC Organisation for Economic Cooperation and Development/Development

Assistance Committee

PAR Project appraisal report

PCRs Project Completion Reports

PRA Project Results Assessment

PSR Project supervision report

RSDT Road Sector Development Team

RISP Regional Integration Strategy Paper

RMC Regional Member Countries

RWSSI Rural Water and Sanitation Initiative

SFR Special Relief Fund

TA Technical assistance

ToC Theory of Change

TYS Ten-Year Strategy

UA Unit of Account

XSR Expanded supervision report

6

Inception report

1 Introduction

1.1 Context and background of the evaluation

At the request of the African Development Bank's (hereafter "the Bank") Committee on

Development Effectiveness (CODE), the Independent Development Evaluation function (IDEV) is

undertaking a comprehensive evaluation of development results (CEDR) for the period of 2004-

2013. As part of this initiative, IDEV identified a purposive sample of 14 countries which represent

approximately 60% of the Bank's approvals over this period. This sample is representative of the

Bank's portfolio with respect to certain characteristics, such as country size, income, language and

fragility.

The Country Strategy and Program Evaluation (CSPE) initially covers the period 2004-14 (adding

an additional year to the CEDR requirements to ensure findings are as up to date as possible). In

order to establish a proper selection of projects to be studied in depth in the Project Result

Assessment (PRA), some projects with an approval date in 2003 have been included as well (cf.

section 4.3). As the project portfolio in Annex 1 illustrates, the Bank’s program during this period is

divided between lending to the private sector (mainly through lines of credit), lending to the public

sector in agriculture and infrastructure (notably power, transport, and water supply & sanitation) and

some programs in the social sector (education and health). In addition, the Bank invested indirectly

via equity funds and directly in financial institutions.

A scoping mission was conducted 7-18 September 2015. This mission focused on validating the

portfolio, collecting documentation, establishing contacts, testing new PRA templates for public and

private sector projects, communicating the evaluation to stakeholders, and seeking feedback on the

data scope and feasibility. A list of people met is available in Annex 2. The results of this scoping

mission have been used to refine and finalize the methodology and work plan for the evaluation.

Herewith the scoping mission paved way for the creation of the final list of projects eligible for PRA.

The stakeholders to be interviewed during the data collection mission will be subsequently identified

on the basis of this final list.

1.2 The socio-economic backdrop for the AfDB’s engagement

Following an April 2014 statistical "rebasing1" exercise – starting from the FY 2012/2013 – Nigeria

has emerged as Africa's largest economy, with 2013 GDP estimated at US$ 502 billion. This

exercise impacted on other key macroeconomic indicators, lowering the Government’s Budget

deficit to 1% a year, the Government’s expenditures to 15% a year and the level of the sovereign

debt to 10.6% a year. This fundamental change in the figures for the Nigerian economy was also

mirrored by the GDP composition with the share of the three main sectors – oil and gas, trade and

agriculture – plummeting from 85% of the GDP to the 54% following the rebasing. In particular, the

contribution of the oil and gas sector shrank from an estimated 37% share of the overall GDP to a

15.8% share, falling behind the contribution of the agriculture sector, now building up as much as

the 22% of the GDP.

1 Re-basing has entailed an updated of the firms assessed to be operating in Nigeria by roughly ten times since 1990, last

year in which the accounts statistics had been rebased (The World Bank).

7

Inception report

Table 1.1 Macroeconomic and sectoral indicators (after rebasing, World Bank calculations)

2010 2011 2012 2013 2014

Economic indicators2

Real GDP growth (%) 7.8 4.9 4.3 5.4 6.3

Inflation (%) 13.8 9.5 9.3 5.9 4.7

Government debt as % of

GDP 9.4 10.2 10.4 10.6

11

(preliminary)

Services as % of GDP 54.1 52.9 54.3 57.0 59.1

Industry as % of GDP 22.0 24.8 23.7 22.0 20.7

Agriculture as % of GDP 23.9 22.3 22.1 21.0 20.2

The internal composition of the non-oil related GDP as also shown fundamental variations, the

weight of agriculture has been reducing with manufacturing and trade gaining bigger shares. Over

the period concerned by the rebasing exercise, the economic performance of the oil and gas sector

has recorded a negative growth rate, falling by 4.9 and 13.1% in 2012 and 2013 respectively. The

overall economy has nevertheless scored positive and sustained growth rates for the same years,

with a non-oil GDP growth of 5.9% and 8.3% that translated in a total GDP growth of 4.3% in 2012

and 5.4% in 2013. Preliminary data for 2014 and projections for 2015 confirm the negative trend for

the oil and gas sector but also a slowing down in the non-oil GDP growth rate as shown in the table

below.

Table 1.2: Impact of the Oil and Gas sector on the Nigerian Economy

2012 2013 2014 (pl.) 2015 (pr.)

Oil and gas impact3

Real GDP growth (%) 4.3 5.4 6.3 4.8

Oil and Gas as % of GDP -4.9 -13.1 -1.3 -1.6

Non-oil as % of GDP 5.9 8.3 7.3 5.5

Total revenues as % of GDP 14.3 11.0 9.9 7.7

Oil and gas revenue as % of GDP 10.4 7.1 5.8 3.4

Total expenditure as % of GDP 14.7 13.4 12.3 10.6

Overall balance -0.3 -2.4 -2.5 -2.8

Non-oil primary balance -11.6 -9.7 -8.1 -5.5

The weight of the oil and gas sector over the economy may have been lowered as a component of

the GDP, but the weight of the sector In terms of government revenues remains heavy, as shown in

the table above. The lowering of the price of crude oil implied lower revenues and a deterioration of

the account balance of the state, that has been falling in negative territory. In this scenario of falling

revenues and increased pressure on the revenues and on the value of the Naira, the Government

has adopted a tightening fiscal policy. It further has allowed the Naira depreciation and unified the

foreign exchange rates.

Compared with the level of economic growth that has been achieved over the past years, the

official poverty rate level remains strikingly high at 46% of the adult population, according to the last

comprehensive household survey Harmonised Nigeria Living Standards Survey ( HNLSS) 2009-

2010, slightly declining from the 2004 level of 48%. However, smaller general household surveys in

panel format (GHS 2010-2011 and 2012-2013) provided more encouraging results as shown in the

table below.

2 Source: Data Portal .The World Bank, 2015 consulted the 29/09/2015. Available at: http://data.worldbank.org/ 3 Source: ‘IMF press release - IMF Executive Board Concludes 2014 Article IV Consultation with Nigeria’ Press Release No.

15/91, March 4, 2015. Available at: https://www.imf.org/external/np/sec/pr/2015/pr1591.htm

http://data.worldbank.org/

https://www.imf.org/external/np/sec/pr/2015/pr1591.htm

8

Inception report

Table 1.3 Impact of the Oil and Gas sector on the Nigerian Economy

Region HNLSS 2009-10 GHS 2010-2011 GHS 2012-2013 GHS Diff

National 62.6 35.2 33.1 -2.1

Rural 69.1 46.3 44.9 -1.4

Urban 51.2 15.8 12.6 -3.2

North Central 65.8 33.4 31.1 -2.3

North East 75.4 47.1 50.2 3.1

North West 74.2 46.9 45.9 -1.0

South East 54.9 31.7 28.8 -2.9

South 53.3 27.7 24.4 -3.3

South West 47.9 21.2 16.0 -5.2

Although the World Bank warns that the abovementioned figures are not totally comparable and the

GHS are not as comprehensive and accurate as the NHLSS would be, they still provide a proxy of

how the poverty headcount ratio has improved overall. The table also highlights two structural

features of the Nigerian socio-economic landscape, the regional divide between a poor north and a

more prosperous south and the persistence of rural poverty compared to a net decrease in the

urban poverty levels. Income inequality overall is high, with the last GINI index calculation available

(2010) showing an increase to 43 from the 40 recorded in year 2000.

Table 1.4 Social and environmental indicators 4

Social and environmental indicators 1990 2000 2013

Unemployment (%) 7.4 7.6 7.5

Maternal mortality ratio (per 100,000 live

births)

1,200 950 560

Total net primary school enrolment (%) - 65 64

Improved sanitation facilities (% of

population)

38 34 29

Improved water source (% of population) 40 52 68

CO2 emissions (kg per PPP $ of GDP) 0.8 1.2 0.5

Other key social and environmental indicators show mixed trends, with improvements in maternal

health and access to water sources but stable or decreasing indicators for the access to sanitation

facilities and school enrolment. A reason for this mixed performance is to be found in the massive

increase of the Nigerian population that has grown from 95 million inhabitants in 1990 to 173.6

million in 2013, with the life expectancy only growing from 46 years to 52 years on the same period.

The scoping mission confirmed the magnitude of challenges in Nigeria. One of the key issue is to

achieve economic growth in sectors not related to oil and gas, in agriculture in particular, to create

jobs that allow for the absorption of millions of young people looking for work, both now and in the

near future. The Bank’s portfolio seems to be in par with this key issue as it operates a diversified

portfolio. While finance sector projects constitute the largest share in the portfolio, the promotion of

agricultural development is pivotal for the Bank. Growth in this sector is directly promoted through

the funding of public projects in the agricultural sector or indirectly through projects in other sectors

like transport or environment. Agri-businesses are also funded by the bank via Lines of Credit

(LoCs) to commercial banks.

4 Source: Data Portal .The World Bank, 2015 consulted the 29/09/2015. Available at: http://data.worldbank.org/

http://data.worldbank.org/

9

Inception report

2 Purpose and objectives

The purpose of the Nigeria CSPE is two-fold. It is an important building block of evidence for the

CEDR but it is also an important evaluation product in its own right, which should be expected to

inform future programming in Nigeria, and indeed in other countries in the region where the Bank is

active with a similar program. The Bank’s program in Nigeria is large. The program is mainly

characterized by lending both to government (one third of the portfolio) and to the private sector

(close to two third of the portfolio). In addition, several grants have been provided by the Bank to

the public and private sector. The multinational operations account for a relatively small share of the

portfolio. The evaluation will explicitly address the issue of added value, and learning from the

Nigeria experience.

Table 2.1 Public Sector Portfolio overview

Nature Sector Number of

projects

Amount allocated % public

portfolio

% total

portfolio

Public

Agriculture 5 124,537,147 14.81% 5.32%

Environment 2 3,123,510.10 0.37% 0.13%

Transport 1 35,270,000 4.19% 1.51%

Water Supply & Sanitation 5 301,304,967.03 35.83% 12.88%

Power 2 132,000,000 15.70% 5.64%

Finance 1 194,848.90 0.02% 0.0083%

Social 4 31,771,353.67 3.78% 1.36%

Multi-Sector 2 212,707,195.22 25.29% 9.09%

Totals 22 840,909,021.92 100% 35.95%

Table 2.2 Private Sector Portfolio overview

Nature Sector Number of

projects

Amount % public

portfolio

% total

portfolio

Private

Power 1 70,854,146.74 2.34% 0.91%

Agriculture 1 21,279,614.13 1.42% 1.50%

Transport 1 35,076,837.02 4.73% 3.03%

Finance 20 1,371,221,037.29 91.51% 58.62%

Totals 23 1,498,431,635.18 100% 64.05%

The specific objectives of the evaluation are as follows:

To provide credible evaluative evidence on the development results of the Bank's engagement

in Nigeria

To provide credible evaluative evidence on how the Bank has managed its engagement with

Nigeria

To identify the factors (both internal and external) that affect good or poor performance.

To identify lessons and recommendations from the performance and management of the Bank's

support to Nigeria to inform the design and implementation of future strategies and operations

(both in Nigeria and potentially other Regional Member Countries, RMCs).

10

Inception report

These objectives serve multiple clients for this evaluation:

1. The Board - The evaluation will provide the Board with an independent and evidence-based

assessment of the development results that the Bank has achieved in Nigeria as well as with

suggestions for potential improvements to better position the Bank in terms of achieving the

objectives outlined in its Ten-Year Strategy (TYS). The volume of the program makes it of

particular interest to the Board.

2. Bank staff and management - The Nigeria CSPE can inform the development of a new Nigeria

CSP as well as new operations there, by providing the Bank’s office in Nigeria and

headquarters based staff working on Nigeria with independent and evidence–based

assessment of what worked and what did not work, and why, as well as suggestions for

improvement. Lessons based on experience in Nigeria should also be useful for the Bank’s

work in other countries with similar features (e.g. other middle income countries, or where the

Bank is focused on private sector operations).

3. IDEV – the evaluation will inform the CEDR by providing independent and evidence-based

assessment of the development results that the Bank has achieved in this country, as well as

lessons of what worked and what did not and the factors associated with good and poor

performance.

11

Inception report

3 Evaluation questions and universe

3.1 The Evaluation Questions

The Evaluation Questions for the Nigeria CSPE will take as their starting point the standard EQs

used for all CSPEs involved in the CEDR. In addition, the evaluation focuses on the extent to which

the Bank is maximising its added value in the Nigerian context. This added value relates to the

Bank’s focus, its instruments it has available in the country and its non-lending work.

Table 3.1. presents an overview of the Evaluation Questions.

Table 3.1 Overview evaluation questions

Criterion Evaluation Question

Achievement of Development Results

Relevance 1. To what extent are the country strategy and Bank operations aligned with:

a. RMC development needs;

b. RMC development strategies and priorities; and

c. The needs of beneficiaries?

2. To what extent are the interventions in the country aligned with the Bank’s

strategy and priorities?

Effectiveness 3. To what extent have the Bank’s interventions achieved their expected results?

4. To what extent have the Bank’s interventions benefited target group members?

5. To what extent have the Bank’s interventions contributed to the achievement of

development objectives and expected development results of the country, including

impacts (both intended and unintended)?

Sustainability 6. To what extent have achieved benefits continued or will be likely to continue

once the Bank’s interventions are completed?

Cross-cutting

issues

7. To what extent are the Bank’s interventions inclusive (i.e., bringing prosperity by

expanding the economic base across the barriers of age, gender and geography) in terms

of gender equality and regional disparity?

8. To what extent are the Bank’s interventions environmentally sustainable and

support the transition to green growth?

Management of the Bank's interventions

Efficiency 9. To what extent are the Bank’s interventions delivered in an efficient manner

(i.e., whether resources and inputs are economically converted to results)?

10. To what extent are the Bank’s interventions implemented in a timely manner

and in compliance with operational standards?

Design 11. To what extent is the quality of the CSP satisfactory?

12. To what extent has the Bank applied selectivity in designing its country portfolio

and focused on areas where it brings added value?

13. To what extent has the Bank been innovative in adapting its approach to the

country’s context and development challenges/needs?

14. To what extent are the Bank’s interventions coherent and well-coordinated

internally?

Knowledge and

policy advice

15. To what extent has the Bank actively engaged in and influenced policy dialogue

through relevant advice?

16. To what extent has the Bank delivered adequate analytical work in support of its

interventions, positioning and policy advice?

12

Inception report

Criterion Evaluation Question

Partnerships and

leverage

17. To what extent are the Bank’s interventions harmonized with those of other

donors (avoiding duplication, simplifying procedures etc.)?

18. To what extent are the Bank’s interventions and resources bringing in other

players and being leveraged for maximizing development effectiveness at country level?

Managing for

results

19. To what extent has the Bank successfully implemented management systems

that focus on results and allow learning from past experience?

20. To what extent has the Bank supported the development of national capacities

and management systems that focus on results?

Maximizing added value

Added value 21. To what extent has the Bank been able identify and maximize its added value in

the Nigeria context, in terms of:

(a) What it finances

(b) The instruments it has available

(c) Its non-lending work

Lessons

Lessons 22. What are the key factors positively and negatively influencing the achievement

of development results?

3.2 The evaluation universe

The scope of the evaluation in terms of timeframe is 2004-2014, i.e. projects approved in this

period. In terms of looking forward, to examine in what direction the portfolio is evolving more

recent approvals may also be considered (up to July 2015, when the portfolio is finalised). In

addition, in terms of contextualisation and since the evaluation cuts across the CSP periods, the

2004-2014 portfolio will be seen within that broader context. For that same reason projects

approved before 2004 but implemented within the period 2004-2014 will be included in this

evaluation when considered of added value for the assessment of development results. However,

2004-2014 will remain the guiding timeframe.

All projects listed under the Nigeria portfolio will be covered by this evaluation. These concern both

public and private sector projects implemented in Nigeria.

The evaluation will examine also multinational projects like the projects on Invasive Aquatic Weeds

and the Road Sector Development Team (RSDT), which were not included as such in the initial

portfolio, once these are approved by the Bank.

The way these projects will be analysed will be further set out in the next chapter on methodology.

13

Inception report

4 Methodology

4.1 Evaluation design- a theory based evaluation focused on learning

This is a theory-based evaluation. A Theory of Change (ToC) will be employed for both the Bank’s

overall programme for Nigeria and for individual projects that are examined in-depth.

A theory-based evaluation is the most suitable approach for undertaking CSPEs since it will not

only address which results were achieved, but also how and why these results were achieved or

not. The ToC for the Bank’s entire programme for Nigeria is set out in Annex 5. This ToC has been

reconstructed on the basis of the insights obtained from desk research as well as from the scoping

mission with the Bank’s staff that took place 7- 18 September. The ToC will be tested and validated

in the course of the evaluation. Of particular interest for learning purposes are the assumptions and

hypotheses listed. These will also be tested and the results as well as the factors affecting

performance will be recorded in the typology section 4.4. This typology facilitates the identification

of patterns that will be further explored.

4.2 Evaluation components – a multi-layered approach

The Bank’s programme for Nigeria for the period under review is composed of composed of 45

national projects, including 23 private and 22 public sector projects, and 7 multi-national projects.

The Bank’s portfolio in Nigeria is one of its most diversified portfolio’s as it covers a wide variety of

sectors and includes various lending and non-lending projects.

A multi-layered strategy will be employed. Hence the evaluation will be composed of three main

components as summarized in Figure 4.1. The components are closely linked to each other.

Figure 4.1 Multi-layered approach to evaluation

Projectportfolio

assessment

Projectresults

assessment

Detailed assessment of

completed projects

Full portfolio assessment,

looking at design, implementation,

results

Broader assessment,

beyond projects, looking at CSP level

Programmaticassessment

14

Inception report

The strategic level

CSPEs will each require an examination of the programme as a whole ad its main pillars. This

involves an analysis of non-project activities – notably policy dialogue as well as knowledge and

analytical work (studies) undertaken and whether the Bank has well identified its niche or added

value, and managed to build constructive relationships in Nigeria. The quality and results achieved

by the Bank’s knowledge work will be achieved through a combination of document review and key

informant interviews. The overall ToC for the country programme will be assessed at this level.

Portfolio level

The full portfolio of projects will be reviewed, based on available documentation. This portfolio will

be broken down into different parts suitably to focus on each of (i) design/quality at entry; (ii)

implementation, supervision/monitoring; (iii) results, including both outputs and outcomes. This

review will be desk based but complemented by information from primary data collection. It will be

crucial to assess most of the evaluation questions. Annex 6 provides an outline of the specific

criteria to be assessed in the portfolio review. This will be piloted with 3-4 projects before being

adjusted and rolled out to the rest of the portfolio.

Project level

Within the national portfolio, all of the completed/closed and near-completed projects (in terms of

disbursement ratios) will be subject to detailed Project Results Assessments (PRAs), including site

visits (where possible) and other additional data collection methods that are focused on results

(both outputs and outcomes). A template will be completed for each of these projects, providing

multiple lines of evidence for assessments along the various evaluation criteria, plus identification

and categorization of lessons. The templates for public and private sector projects slightly differ with

regard to the indicators for each evaluation (sub) criterion. The templates have been designed in

line with international evaluation guidance standards. The two templates are provided in Annex 7.

Given the focus on development results achieved by the Bank in Nigeria, the effectiveness criterion

is of key importance for this assessment. For each PRA project a tailored ToC will be developed.

This work will be especially crucial to assessing results and understanding the factors that affect

performance.

For each project selected for PRA, before the data collection mission a ToC will be developed to

inform the assessment. Indicators will be identified and the gaps in information highlighted.

Assumptions included in the ToC will be tested on the basis of desk research and the data

collection mission. Factors influencing performance will be noted and categorized. An example of a

project ToC is provided at Annex 8. Each project’s ToC is connected to the ToC at country level

through underlying ToC of the respective sector of the project (e.g. transport, agriculture, finance).

4.3 Selecting projects eligible for PRA

In the (revised) ToR, IDEV has already proposed a set of criteria to identify PRA-eligible projects,

which are supposed to be applied to all CSPEs to provide consistent data for the CEDR.

These criteria establish that the project has to be:

Approved in the period 2004-14, and

Closed/completed or near to completion.

To check whether the project is near completion, the disbursement rates stated in the Bank’s

systems have been verified with the project Task Managers. Furthermore, the following

15

Inception report

interventions are excluded from PRA: (i) projects with a value of less than one million UA; (ii)

studies.

On the basis of these criteria, 18 out of 45 interventions have been selected as eligible for PRA in

the CEDR for Nigeria

Comments on the initial selection

The initial selection included four public sector projects and 14 private sector projects. These

projects accounted for 746,9 million UA or 31% of the total portfolio. Overall, the projects selected

for PRA included 3 agricultural projects, 2 transport projects and 13 financial sector projects.

Twelve of these provided lines of credits (LoCs) to banks. This selection reflected the fact that LoCs

constitute an important share of the AfDB’s portfolio for Nigeria. In several cases LoCs are already

in their second and/ or third tranche (Zenith, Access, Guaranty Trust Bank).

The scoping mission revealed that the selection originally proposed requires to be amended as it

insufficiently constitutes a representative selection of the Bank’s interventions in Nigeria. In

particular, the mission contributed to the highlighting of some inconsistencies in the initial selection:

1. A rigid application of the criteria excluded several promising projects in terms of development

outcomes and lessons learned. For example, the Skills Training and Vocational Education

Programme (STVET) is potentially relevant for Nigeria’s education system since secondary

education has long been neglected by (other) donors. STVET was initially not selected as a

PRA project because of the disbursement rate being lower than 75%.5 Other projects were

excluded because of their approval dated back to 2003.

2. The portfolio in Nigeria is one of the Bank’s most diversified portfolios in terms of sectors

covered. The current focus on the financial sector insufficiently mirrors this diversity. More

importantly, it contributes to a bias towards the private sector, diminishing the actual weight of

the public sector in the project selection. The scoping mission moreover showed that the private

sector bias causes insufficient attention to issues like the impact of co-financing on project

ownership and ultimately the results that have been achieved with that project;

3. The agricultural sector plays a pivotal role in the entire project portfolio of the Bank since

projects from other sectors are in many cases mainly intended to improve agricultural

productivity, e.g. water management and transport projects;

4. Some partners seem to be less willing to cooperate with the evaluation team. The Bank’s field

office signalled that most notably Helios Towers would not be willing to provide us with

additional information. The preparation of the data collection mission will clarify the extent to

which partners are willing to cooperate and ultimately which information will be made available

to us.

Rebalancing the selection

While acknowledging that a different application of the selection criteria would set limitations to the

consistency and comparability of data for the CEDR, we propose to improve the selection by simply

mirroring the structure of the portfolio which consists of 40% public projects and 60% private

projects (in number).

In this way the selected sample of projects more adequately represents the theoretical construct

laid down in the country ToC as well as include a wider range of stakeholders involved in designing,

implementing receiving or administering AfDB lending programmes.

5 The final disbursement is planned for the end of September 2015.

16

Inception report

By doing so, we must point out that the project portfolio we have available is still provisional. As a

matter of fact a revised final project portfolio still has to be validated by the AfDB’s field office.

The information made available through the bank

The level of information available to us differs between the projects currently included in the

portfolio. For the majority of projects a lot of information is available. Project Appraisals, Aide

Memoires or Back-to-office Reports, Project Completion or Status Reports are documents that

contain the most valuable information. Sometimes the original Project Proposals and the

Loan/Grant Agreements are available as well. For most of the private sector projects, further

financial documentation like audits, Expanded Supervision Reports (XSRs) and annual financial

reports are available. All these documents together potentially will provide the team with a fairly

complete picture of the project status and life already at the level of desk research. In some cases,

crucial documents are not available and only the financing agreements or a single Back-to-office

Report has been provided.

Proposing a new selection

In order to arrive at a selection that mirrors the portfolio distribution between public and private

sector projects more adequately, more public sector projects need to be included in the selection.

Keeping in mind the caveats and diverging availability of information presented above, IDEV

decided to include public sector projects that have been approved in 2003. This allowed for the

inclusion of three agricultural projects (in addition to the social project STVET). The total number of

PRA projects increased to 21.

Availability of information for selected PRA projects

The level of information currently available for these 21 PRA projects differs. For the vast majority

of projects a high level of information is available. For those projects for which only limited

information is available, we have asked the Bank whether it is possible to provide this information.

Without this information the projects cannot be selected for PRA.

The revised selection of projects eligible for PRA herewith includes:

Table 4.1 List of PRA projects

No. Project name Project

code

Net loan (in

UA)

Previously

included?

Information

availability

Public Sector projects

1 RURAL ACCESS & MOBILITY

PROJECT

P-NG-

DB0-005

35,270,000 Yes High

2 SUPPORT TO AGRIC. & RURAL

INSTITUTIONS

P-NG-

AA0-026

2,636,876.27 Yes Medium

3 COMMUNITY-BASED AGR. &

RURAL DEVELOPMENT

P-NG-

AA0-025

13,000,000 No Medium/High

4 INVASIVE AQUATIC WEEDS –

NIGERIA (multi)

P-Z1-AA0-

084

1,609,000.00 Yes High

5 SKILLS TRAINING AND

VOCATIONAL EDUCATION

P-NG-IA0-

001

30,000,000 No Low

6 FADAMA DEVELOPMENT

PROJECT

P-NG-

A00-005

19,658,135.07 No Low

7 NERICA DISSEMINATION

PROJECT -NIGERIA

P-Z1-

AA0-073

5,570,000 No Insufficient

17

Inception report

No. Project name Project

code

Net loan (in

UA)

Previously

included?

Information

availability

Private Sector projects

8 LOAN LEKKI TOLL ROAD

PROJECT

P-NG-

DB0-008

35,076,837.02 Yes Medium

9 LOAN NIGERIAN EXPORT

IMPORT BANK

P-NG-

HAA-003

35,427,073.37 Yes Medium/High

10 LOC I ZENITH BANK PLC P-NG-

HAB-004

49,597,902.72 Yes Medium

11 LOC I GUARANTEE TRUST

BANK

P-NG-

HAB-005

28,341,658.70 Yes Insufficient

12 LOC II ZENITH BANK P-NG-

HAB-006


13 LOC I ACCESS BANK NIGERIA P-NG-

HAB-007

24,798,951.36 Yes High

14 LOC II GUARANTY TRUST

BANK

P-NG-

HAB-009

63,768,732.07 Yes Low

15 EMERGENCY LIQUIDITY

FACILITY ZENITH

P-NG-

HAB-013

35,427,073.37 Yes High

16 EMERGENCY LIQUIDITY

FACILITY UBA

P-NG-

HAB-014

35,427,073.37 Yes High

17 LOC III ZENITH BANK PLC P-NG-

HAB-017

88,567,683.42 Yes Low

18 LOC II ACCESS BANK NIGERIA P-NG-

HAB-019


19 FRB SUBSIDIARY IN NIGERIA P-NG-

HAB-022

53,363,332.18 No High

20 HELIOS TOWERS P-NG-

GBO-004

23,000,000.00 No High

21 UBA TRADE FINANCE

INITIATIVE

P-NG-

HAB-015

70,854,146.74 No Low

4.4 Identifying different project types

Overall the portfolio of projects to be examined presents one fundamental division between public

sector projects and private sector projects. The two groups are different in both the level of sector

coverage and the typology and variety of financing instruments deployed.

As shown in table 2.1 from chapter 2 the public sector portfolio covers seven different sectors plus

a multi-sector project. The Bank’s instruments for the public sector include grants and loans. The

funds of these grants or loans stem from specific sources like the Rural Water and Sanitation

Initiative (RSSWI) and the African Water Facility (AFW) or to special funds like the Special Relief

Fund (SFR) or the Agricultural Development Fund (ADF).

18

Inception report

Table 4.2 Public sector portfolio per typology of financing instrument :

The situation of the private sector portfolio is more homogeneous, with a predominance of the

financial projects, 20 over 23 and of the line of credit as a financing instrument. Also, the origin of

the funding is not linked to a special purpose fund as is the case with public sector projects.

Table 4.3 Private sector portfolio per typology of financing instrument

4.5 Data collection

A combination of quantitative and qualitative data collection methods will be used in the evaluation.

Multiple data collection methods is needed to strengthen the quality of the analysis and to be able

to come up with valid findings on programme and project results. Multiple methods will also help

capture the information needed to answer the various EQs to triangulate information sources, and

to support findings with multiple sources of evidence.

14%

27%

18%

4%

5%

9%

23%

MIC Grant ADF Loan SFR Grant RWSSI Grant

AWF Grant ADB Loan ADF Grant

31%

61%

4%4%

ADB Loan ADB LoC Equity Grant

19

Inception report

Data collection methods to be used include: (i) document review, portfolio review, literature review,

(ii) semi-structured interviews, and (iii) focus groups. The evaluation matrix at Annex 4 sets out

how each of these methods will be used to answer the EQs.

(i) Document review

For document review the evaluation team depends heavily on information provided by the Bank. In

addition, documents from third parties such as the World Bank will be analysed. Document reviews

will be undertaken for the PRAs, the portfolio assessments and the assessment at strategic level.

(ii) Semi-structured interviews

Semi-structured interviews will be held with a range of stakeholders, including Bank staff at head

quarters (e.g. project task managers, country economist). In addition, Bank staff in the country

office, project implementing partners at Federal, State and local level, private sector partners,

beneficiaries and other donors (International Finance Corporation (IFC), European Investment Bank

(EIB) will be approached via personal interviews during the data collection mission and clarification

round via email. Before each interview, interview questions will be send to the interviewees. The

interview protocol is included in Annex 11 Annex 12 presents a preliminary list of institutions to be

interviewed. Further review of project level documentation as well as potential snowballing on the

ground should give us further names to be added to this list.

(iii) Focus groups

The team might use focal groups to collect further information. Opportunities for focus groups may

be found in the following areas: development partners, Bank country office staff, and project staff of

implementing partners. The team will determine during the evaluation process whether collecting

information through meetings with focus groups or individual interviews will be more appropriate.

4.6 Sampling

A purposive sampling exercise will be undertaken to identify project sites that will be visited during

the data collection mission. Sites will only be visited for those projects that are selected for PRA. In

addition, both public and private sector projects selected for PRA may include multiple sub-projects,

for example sub-borrowers. The sample will be based on the representation of the main banks and

their sectoral focus (private sector) and the main stakeholders in the sample as well as on the

information available for each project. These sampling criteria are further elaborated below. We will

propose a definite sample as soon as the selection of the PRA projects is approved by IDEV.

For several private sector projects the team needs to consider which of the different sub-borrowers

need to be interviewed. The scoping mission revealed that banks provide a project pipeline when

requesting a LoC from the AfDB. Hence most of the sub-borrowers can be readily identified. All

banks interviewed during the scoping mission indicated to be willing to provide us with this list and

to assist in organising visits to sub-borrowers. We consider that this method might give more

valuable insights than the electronic survey would do. If the sample of sub-borrowers willing to be

interviewed is sufficiently representative and varied compared to the overall list of sub-borrowers,

we propose to skip the electronic survey and limit the data collection to interviews.

When undertaking the visits to sub-borrowers, we aim to guarantee the representativeness by

selecting and visiting sub-borrowers for every bank in the sample and sufficient spread across the

different sectors.

20

Inception report

The table below shows the main sectoral focus of the sub-borrowers that the banks have indicated

in the interviews during the scoping mission:

Table 4.4 Sector focus selected banks

Bank Sector focus

Nexim Manufacturing

Agriculture

Transport and services

Access Agriculture

Manufacturing

Tourism

Zenith Tourism

UBA Oil & gas

GT Bank Construction

Maritime

FRB

A final purposive sampling criterion concerns stakeholders. All important stakeholders should be

represented in our sample. A stakeholder map can be drawn by scanning the available documents

per project and identifying all those actors that are mentioned directly or indirectly. Private and

public sector stakeholder will differ from each other. However, public sector stakeholders will be

relevant for the assessment of private projects and vice versa.

Most of the public sector projects have stakeholders dispersed over the three-tier governance

system of Nigeria, from the Federal down to the State and Local level. The Federal Government

(which is in most cases the borrowing entity) includes the line ministries whose sector is concerned,

the specular regulating authorities and a specific department in the ministry. The same structure

can be found at State level whereas at local level mostly the Local Government Councils are

involved in project implementation.

4.7 Practical issues relating to data collection

There are four main challenges with regards data collection in Nigeria:

The Bank’s own monitoring and documentation varies in quality and detail. Document mapping

has been completed to provide clarity on what documentation is available (annex 13).

The problem of institutional memory and limited hand over will limit the information available for

the first half of the evaluation period. In a number of cases Bank task managers may indicate

that they do not have information pertaining to before they took up their post, indicating that

there may be a broader issue of handover/document storage.

21

Inception report

The data collection mission will be challenging in terms of geography and logistics. The size of

Nigeria is considerable and the Bank’s projects are implemented throughout the country. Public

sector projects in particular are dispersed over Nigeria, for example in Osun State, Cross-River

State and in the area of Kaduna. Visiting these projects is likely to be time-consuming given the

condition of roads and flight connections with Abuja or Lagos. The scoping mission revealed

that it would take at least two days for the team to visit projects in more remote areas. Given the

time and budgetary constraints to this evaluation, the number of site visits will be limited to

approximately six public sector projects. Conducting a proper sampling exercise is therefore

important to identify the most promising projects. In addition, we hired a local consultant to

assist with logistics. Private sector projects are located in the area of Lagos. Visiting these

projects will be mainly arranged via the banks.

The banks that were visited during the scoping mission all indicated to help with arranging visits

to the sub borrowers of their Lines of Credit. This likely generates a bias in the selection of

private sector visits since banks will be inclined to select only good practices. Banks will be

therefore asked whether the evaluation team can select sub borrowers from their project

pipelines.

4.8 Data analysis and ratings- a rigorous approach

Triangulation of data sources will be crucial. It is expected that the evaluation findings should be

based on multiple (i.e. two or more) lines of evidence. The qualitative data generated through

interviews and focus groups will be coded to identify common responses and to allow for robust

analysis.

Factors associated with good and poor performance, for the country program in general and for

Bank projects in particular will be identified and categorized against an agreed typology. A typology

which differentiates between design, implementation, contextual and other factors, is set out in

table 4.5 below. This will also help to test the ToC. Since context will be a crucial aspect of this, one

possible approach is to look at context/ mechanism/ outcome configurations6. The typology will be

adjusted if necessary.

Table 4.5 Typology of factors affecting performance

Stage in results

chain

Design factors Implementation factors External/context factors

Delivery of

inputs

Delivery of

outputs

Intermediate

outcomes

Final outcomes

Impacts

6 Context/mechanism/outcome configurations are used in the Realist Evaluation Approach set out by Pawson and Tilley in

2004. It emphasizes the importance of understanding context in explaining performance patterns.

22

Inception report

In order to understand the contribution of the Bank’s program to development results, a general

ToC has been developed (cf. Annex 5), which will be used to qualitatively assess the Bank’s

contribution. A more quantitative assessment is not feasible, since the Bank’s contribution is in fact

a tiny proportion of finance available in the country. Indeed, overall ODA to Nigeria has remained

below 0.5% of the GNI (World Bank, 2014). Nevertheless the Bank is seeking to unblock particular

bottlenecks, as summarised in the country ToC, so its contribution could be identified in those

areas.

In addition to interviews with key informants, secondary data at national, sector and sub-sector level

available for Nigeria will be analysed both for understanding of context, change over time and to

assist in determining the possible contribution made by the Bank in the key areas in which it has

engaged. Data sources will include for example those relating to investments at national and sector

level, other sectoral data (particularly related to energy, transport, water & sanitation and social

sectors) , exports, employment, poverty, access to finance and financial inclusion.

The primary data that support the evaluation findings will be made available as annexes to the

technical report, and all secondary data will be fully referenced and sourced.

Ratings will be provided at two levels: the individual projects included in the PRA and for the overall

program (the CSP). In both cases a six point rating scale (from highly satisfactory (6) to highly

unsatisfactory (1)) will be employed, against the criteria of relevance, efficiency, effectiveness and

sustainability.

For individual projects, the assessment will be based on the PRA templates (see Annex 7).

Guidance for completing the PRAs and for rating the effectiveness criteria has been made

available by the Bank for both public and private sector projects (see examples in Annex 8). The

three main levels of the evaluation – strategic, portfolio and project - will be brought together to

support an overall assessment of the Bank’s interventions in Nigeria over the period 2004-14. A

six point rating scale will be employed, as with the ratings for individual projects. All ratings will

be clearly justified, and based on multiple lines of evidence. Though ratings will be given for the

four criteria only, the evaluation will also conclude on each of the evaluation questions drawing

from all three levels as appropriate.

23

Inception report

5 Limitations, risks and mitigation

This evaluation has the following limitations, which should be understood and agreed at the outset:

How far down the results chain we can go. The evaluation is looking at a program that is large

for the Bank, but in a context where the funds constitute a small contribution. The evaluation will

seek to look at the outcome of these interventions and their contribution to wider development

results. It will not be possible to establish causality or an unquestionable contribution of

development impacts to this support.

Institutional memory and record storage. The scoping phase has already demonstrated the

difficulty in obtaining records for operations more than a few years old. Every channel will be

explored to establish the existence of documentation, such as supervision reports, including

contacting previous task managers. However, where they are not available, the evaluation

cannot give the benefit of the doubt regarding their existence. Similarly, as with any evaluation

covering more than a decade, there will be problems in finding key informants with institutional

memory beyond the last 3-4 years.

The evaluation has four major risks which will need to be actively managed. These risks and

mitigation strategies are set out in table 5.1 below.

Table 5.1 Risks and their mitigation

Risk Likelihood Potential

impact

Mitigation measure(s)

Lack of data Medium High Although there are serious data limitations the

evaluation design takes these into account. For

example, data available on development outcomes is

limited, which is one of the reasons for focusing on

financial sector aspects.

This inception report is based on an initial review of

documentation and scoping phases, which has

sought to explicitly address weaknesses in data

availability and design accordingly.

Falling behind schedule Medium Medium Because of the requirements for the CEDR the

schedule for this evaluation is tight. Any slippage will

have an impact on the schedule. However, such

slippage will not be material for the aspects of the

CSPE not required for the CEDR. Clear project

planning, sticking to deadlines and sound

communication within the team will all be sued to

minimize the likelihood of slippage.

Particular attention will be paid to those phases which

are often subject to delay: missions (because of the

need to coordinate and be flexible with partners); and

consultation (because of the tendency for comments

to arrive late).

In the event of slippage, priority will also be given to

the PRA to ensure data is available to the CEDR in

early December 2015. In fact, data is not required for

the new CSP’s development until mid-2016.

24

Inception report

Human resource

shortages

Low High The IDEV team working on this evaluation has joined

with the team working on Nigeria. This should allow

for some mutual backstopping. IDEV has engaged a

team of consultants, through the firm ECORYS. This

firm is of reasonable size and has back stopping as

well as quality control functions. The team proposed

is comprised of a mix of senior and junior staff.

Evaluation / review /

assessment fatigue

among interviewees

means that there is a

limited willingness to

grant us time for

interviews and meetings

Low Medium We will work hard to minimise the ‘contact time’,

which we demand of stakeholders by good

preparation of efficient interviews and focus groups.

Also the combined review and evaluation mission for

the energy projects will reduce the time investment for

the interviewees. We will ensure that interviewees of

government institutions will be given ample time to

provide their view on the value added of Bank

interventions in the country.

25

Inception report

6 The evaluation process – managing for quality, timeliness and impact

6.1 Principles

The principles which underwrite IDEV evaluations are in line with international standards, they

include:

Impartiality and independence

Credibility

Usefulness

Participation of stakeholders and partners

Transparency of methods and reporting

6.2 Deliverables and timeline

The deliverables, timelines etc. are set out in the ToRs of the contractual agreement. In summary,

the deliverables for this evaluation will be as follows:

1. Inception report

2. Completed PRA templates

3. Technical report(s) providing overall analysis as well as by sector (a potential outline is provided

at Annex 14).

4. Annexes or separate volumes containing all data used to support the technical report(s),

including coded interview notes, portfolio review files, statistical or financial analysis files.

5. Summary of findings from the South Africa and Nigeria CSPEs relating to lines of credit;

including comparative analysis, trends observed and lessons specific to this instrument.

6. A presentation at a stakeholder workshop.

7. A summary report for CODE (to be drafted by IDEV with consultant input).

The timeline for the evaluation is summarised in table 6.1 and presented in a chart in Annex 15.

Table 6.1 Timeline for the evaluation

Phase and period Main activities

Inception

01 August – 30 September

Scoping visit to Abidjan from 5-7 August, 2015;

Desk review and inception report drafting;

Scoping mission to Nigeria from 7 -18 September, 2015;

Revision inception report (amended and expanded methodology and

work plan);

Sharing draft inception report with reference group and reviewers,

including a meeting to discuss this draft report;

Finalization of the inception report.

Data collection and analysis

1 October – 30 November

Phase 1: desk review;

Review of documents, portfolio review, and identification of data

gaps;

Phase 2: Nigeria visit (1 November to 17 November -TBD);

Key informant interviews, site visits, focus groups, and further data

collection;

Additional data analysis before drafting.

26

Inception report

Phase and period Main activities

Technical report drafting and

finalization

30 November – 18 January

Draft report to IDEV for comment (1). Allow 10 days;

Draft report to reference group and reviewers for comment (2). Allow

14 days;

PRA templates also finalized (2) (by 23 December)

Report finalized (3).

Summary report, workshop and

dissemination

January-February 2016

Summary report drafted by IDEV with Ecorys input

Ecorys attendance at workshop

IDEV to lead on dissemination

6.3 Roles and responsibilities

IDEV works collaboratively with consultants. IDEV staff has organised the scoping mission to

Nigeria in the second week of September in which one of the team members has participated. IDEV

staff will also take part in the data collection mission. IDEV will act as a gateway to Bank staff and

clients. It will also act as guardians on quality control. This Inception Report was drafted jointly.

While the consultants will be tasked with drafting the technical reports, IDEV will lead in drafting the

summary (CODE) report.

6.4 Quality assurance

There are three phases of quality assurance for approved IDEV evaluations, as follows:

Quality threshold 1: Inception report (QT1 draft expected late September, finalization expected

early October)

Quality threshold 2: Technical reports (QT2 draft expected late December, finalization expected

late January)

Quality threshold 3: Summary report (QT3 draft expected early February finalization expected

mid February).

At each stage the drafts are circualted for review and comment from the groups listed below. At

each stage, the comments made are collated and addressed in a matrix, which is provided

alongside the updated draft.

IDEV team, peere reviewers, management

Independent expert reviewer (engaged by IDEV, independently of consulting firm)

Reference group. More information is provided about the reference group below.

Reference Group for Nigeria CSPE

The reference group will meet at each of the three QTs explained above. Reference group members have

been agreed to represent the following:

Nigeria Country Economist;

Nigeria Country Portfolio Officer,

Representative from OPSM,

Representative from ONEC,

Representative from OFSD.

Government of Nigeria: representative of the Ministry of Finance and Planning.

The group is expected to meet three times, though a fourth meeting can be called in order to discuss

recommendations if deemed appropriate. Given the Reference group is geographically split by location,

meetings will make use of VC, and reference group members will also be encouraged to submit comments

in writing.

27

Inception report

The reference group will be chaired by the responsible IDEV manager, and administered by the IDEV Task

manager.

6.5 Dissemination – sharing knowledge

The IDEV team will conduct a workshop to disseminate the evaluation findings, and in particular to

inform the next CSP in early 2016. The next CSP is due in 2017, however, CODE has asked for

CSPEs to come early enough to ensure they inform the early decisions on future CSPs, such as the

choice of pillars.

In addition to the published summary report, a short brief will also be made available both on line

and hard copy. The final inception report and final technical report will also be made available on

the IDEV website. The potential for additional briefs picking up on specific issues will be assessed

based on the strength of evidence in the technical report.

Annex 2: List of persons consulted during September 2015 scoping mission

Note that this list has been established on the basis of business cards available to the consultant

and information provided by the Bank.

Table 0.3 AfDB Country Office

Name Function Email

DORE, OUSMANE Director [email protected]

BARUNGI, BARBARA Lead Economist [email protected]

ALEMU, ZERIHUN GUDETA Chief Country Economist [email protected]

AKINWUMI, EMMANUEL

IBITUASE

Principal Private Sector Specialist [email protected]

OSUBOR, GREGORY Social Sector Expert [email protected]

OPOKU-DARKWA,

PATRICK

Principal Transport Engineer [email protected]

AMADOU, IBRAHIM AHMED Chief Agricultural Economist [email protected]

MOHAMMED, USMAN Principal Disbursement Officer [email protected]

OKPE, BERNARD Administrative Clerk/Receptionist [email protected]

OLAOYE, BOLANLE

PATRICIA

Principal Education Officer [email protected]

OZOR, CORNELIUS Information & Communication

Technology Officer

[email protected]

ANSAH, DENNIS Chief Portfolio Management [email protected]

Table 0.4 Public sector

Federal Ministry of Water and

Natural Resources

Haruna Mohammed Director , Federal Ministry of

Finance

Bala Danshehu Director Highways, Federal

Ministry of Works

[email protected]

Ishaq D. Mohammed Unit manager, RSDT, Federal

Ministry of Works

[email protected]

Federal Ministry of Environment

Federal Ministry of Agriculture

Ubandoma Ularamu National Coordinator, RAMP,

Federal Ministry of Agriculture

[email protected]

Ebrima Njie Commission, Infrastructures,

ECOWAS

[email protected]

Ministry of Education

mailto:[email protected]
















Table 0.1 Private sector

Nigerian Export Import

Bank – Nexim

Bashir M.Wali Executive Director [email protected]

Dorothy Ogbutor Head, Specialised Business

Dept

[email protected]

Adaeze Ebegbulem Head, Trade and Finance Dept [email protected]

Ahmed D.Modibbo Secretary to the Board [email protected]

Tayo Omidiji Head, Strategic Planning [email protected]

ZENITH BANK PLC

Peter Amangbo GMD/CEO [email protected]

Olusola R. Ayodele Manager, mcp Group [email protected]

Ebenezer Onyeagwu Executive Director [email protected]

Anyimah Michael General Manager [email protected]

UBA

Lape Seriki Head, Bilateral Funding and

Correspondent Banking

[email protected]

Sola Yomi-Ajayi Division Head, Financial

Inst.&Int. Organizations

[email protected]

Uche Ike Chief Risk [email protected]

Adepeju Ajibola [email protected]

Ugo Nwaghodoh Group Chief Finance Officer [email protected]

ACCESS Bank

Ediale Eremionkhale Head Office [email protected]

GT Bank

Razaq Jinadu [email protected]

Lara Ogunlaja [email protected]

















Annex 3: Overview of AfDBs CSPs for Nigeria during 2004-2014

There has been some evolution of the program under the four different strategies covering the

2004-2014 period, but also notable continuity. Some of the changes are linked to changes in the

way in which CSPs more generally are written by the Bank.

Table 0.5 Overview of Bank strategic priorities in Nigeria over the period

CSP

period

Pillars Components To note

2002-2004 1. Health sector

Improvement of public health facilities;

Improvement of access to health services.

Overall focus on

poverty reduction.

Strategy does not

contain pillars per

se but makes

reference to areas

of focus

2. Agriculture and

rural

development.

Enhance agricultural production;

Raise income levels; and

Support for food security.

3. Regional

Integration

Development of infrastructure for regional

integration;

Resource mobilization infrastructure projects

that contribute towards enhancing regional and

continental integration.

2005-2009 4. Development of

human capital

through improved

service delivery in

education and

health

Provision of unfettered access to education;

Elimination of gender disparity;

Improve the quality of education at all levels;

Strengthen preventive and curative primary

health care services, to increase access;

Provision of quality primary health care.

In 2010, this

strategy was

extended to 2011.

5. Stimulating non-

oil growth through

enhanced

infrastructure and

agricultural/rural

development

Provision enabling environment for

infrastructure investments;

ensure integrated water management and

development;

protection of water resources and environment;

building public-private partnership for a

sustainable development of water resources.

2011-2013 1. Improving

governance;

Transparency and accountability;

Participation;

Sector governance;

Capacity development;

Judicial reform and democratic governance.

This was a joint

strategy

implemented in

partnership with

the World Bank

2.Maintaining non-

oil growth;

Infrastructure to support growth clusters;

Promoting private sector;

Technical and vocational education to address

the skills gap;

Reducing import bans and high tariff barriers.

3. Promoting human

development

Improve access and utilization of services;

Support to education with particular focus on

girls education;

Support to maternal and child health.

CSP

period

Pillars Components To note

2013-2017 1. Supporting the

Development of a

Sound Policy

Environment;

Public financial management reforms;

Resource mobilization and fiscal federalism;

Private public partnerships;

Financial intermediation;

Gender mainstreaming;

Regional integration.

The CSP

mentions areas of

focus as opposed

to components

under each pillar.

2. Investing in

Critical

Infrastructure to

Promote the

Development of

the Real Sector of

the Economy.

Power;

Transport.

RISP

period

Pillar Components To note

2011-15 1. Linking

regional

markets

Regional transport infrastructure;

Transport and trade facilitation;

Regional energy production and

markets integration.

Pillar II refers also to

strengthening the capacity of

existing Regional Economic

Communities (RECs) in West

Africa. Specifically, the capacity

of ECOWAS/WAEMU, as well as

selected regional institutions, and

national entities where necessary.

2. Capacity

building

Capacity building for effective policy

and regional projects

implementation;

Capacity building for financial

sector integration; and

Support to regional research and

training centres relevant to the

integration agenda.

Consistent features throughout the time frame under review include the following:

There has been emphasis on diversification of growth towards non-oil sectors, through the

provision and enhancement of infrastructure, private sector development, as well as

agricultural/rural development;

There has been emphasis on promotion of public private partnerships, improvement of public

management, sector governance, and institutions.

Annex 4: Evaluation matrix

This matrix is a revised version of the matrix presented in the proposal. It is based on a first assessment on the insights obtained during the scoping mission at the Bank in Abidjan, the

documentation available to us as well as IDEV’s standard templates for analysis. For each of the projects covered in the PRA, mini evaluation matrices will be developed which will identify

indicators for the specific projects. The indicators included in the overarching matrix are not project specific, but the project level evaluations will feed in the overall conclusions.

Relevance

Evaluation question Level of analysis Indicators Methods

1. To what extent are the country strategy and

Bank operations aligned with:

RMC development needs;

RMC development strategies and priorities;

and

The needs of beneficiaries

Strategic, portfolio, project

Strategic level: Alignment of projects rationale and objectives with the Bank’s CSP

and the applicable sector strategies, the country’s development strategies and the

beneficiary needs from design/approval to completion (including any adjustments that

were made to the project in view of changes in the applicable policy environment,

such as project restructuring).

Proportion of CSPs which include analysis (or reference to analysis in

separate document) of beneficiary needs.

Feedback from RMC stakeholders.

Portfolio level: a) Sector distribution and trends volume and size; status of

operations; ratio of lending and non-lending/ private sector and public sector; b)

Percentage and ratio of loans and Grants; ratio of co-financed initiatives with DFIs; c)

Focus of programs: Extent of emphasis on; Regional Integration, financial sector

development, capacity development, response to cross-cutting issues(Gender,

Environment, Governance, knowledge work; d) Contribution of Multinational

operations benefitting the country.

Project level: (i) the relevance of project objectives (to a-d); (ii) relevance of project

design, including - the extent to which the project’s objectives are clearly stated and

focused on outcomes as opposed to outputs; the realism of intended outcomes in the

country’s current circumstances also is assessed; the extent to which project design

adopted the appropriate solutions to the identified problems; (iv) the modifications to

project design; (v) the circumstances prevailing at the time of the evaluation.

Proportion of projects that include satisfactory analysis of beneficiary needs.

PRA,

Portfolio review,

Analysis of AfDB CSP’s and other strategies

and non-bank documentation, semi-structured

interviews

Effectiveness


2. To what extent have the Bank’s

interventions achieved their expected

results?

Portfolio, project, strategic Strategic: To what extent has the theory of the Bank’s engagement in

SA held true.

Portfolio level: Sector performance as documented by the AfDB (per

key sector);

Project level:

Achievement of outputs: output execution ratio7 of both public and

private sector projects;

Achievement of outcomes: Achievement of intermediate and final

outcomes covering core sector indicators 8 of both public and

private sector projects.

PRA,

Portfolio review,

Analysis AfDB CSP’s and other strategies and

non-bank documentation, semi-structured

interviews, focus group with AfDB and in-country


interventions benefited target group

members?

Project Project contribution to expected achievements for target groups

included in the CSPs

Analysis AfDB CSP’s, PRA, semi-structured

interviews


interventions contributed to the

achievement of development objectives

and expected development results of the

country, including impacts (both intended

and unintended)?

Portfolio, project,

strategic.

Contribution of the Bank’s projects and engagement to national

outcomes at sector and project level.

Extent to which project results are aligned to CSP level planned

results.

Unintended outcomes encountered during the project cycle.

PRA,

Portfolio review,

Analysis of AfDB CSP’s and other strategies and

non-bank documentation, semi-structured

interviews

7 Supported by example outputs from the retrospective project theory of change. 8 Supported by example outputs from the retrospective project theory of change..

Sustainability


5. To what extent have achieved benefits

continued or will be likely to continue once

the Bank’s interventions are completed?

Portfolio, project Project:

Reliance of project achievements on the procurement of sound

technology;

Extent to which beneficiaries are able to operate funded

equipment autonomously;

Financial sustainability of project’s operations/organisation;

Institutional capacity in executing/operating organisation;

Ownership and partnerships observed;

Likelihood of continuation of benefits after project completion.

The project’s environmental and social performance in meeting the

Bank’s requirements; and ii) the project’s actual environmental and

social impacts.

Portfolio: Extent to which different aspects of sustainability (financial,

technical, institutional, environmental) were considered at appraisal

and monitored during supervision.

PRA, portfolio review, semi-structured interviews

with AfDB

Cross-cutting issues


6. To what extent are the Bank’s

interventions inclusive (i.e., bringing

prosperity by expanding the economic

base across the barriers of age, gender

and geography) in terms of gender

equality and regional disparity?

Portfolio, project Share of positive outcomes project/ portfolio for men/ women, age

groups, deprived areas or groups/ geographical distribution (%).

Extent to which cross cutting issues are considered at appraisal and

monitored.

Any unintended impacts (if any) on particular groups.

PRA, portfolio review, analysis AfDB strategies,

semi-structured interviews with AfDB,

beneficiaries and other stakeholders


interventions environmentally sustainable

and support the transition to green

growth?

Portfolio, project Environmental performance of projects/ portfolios in terms of pollution

loads, wastes, energy and resource efficiency, biodiversity

conservation, etc.

Compliance of borrowers with social and environmental provisions

included in the loan agreements.

Any unintended environmental impacts (if any).

Extent to which transition to green growth is assessed in appraisal and

monitored.

Efficiency



interventions delivered in an efficient

manner (i.e., whether resources and

inputs are economically converted to

results)?

Portfolio, project Costs of implementation (overheads)

Project level: cost benefit analysis/cost effectiveness analysis (i) at appraisal;

(ii) at time of evaluation.

Return on investments( financial products)

Evidence of responsiveness of Bank to country context that impacts on project

success eg. Effect of government policies and regulations on performance of

projects

PRA,

Portfolio review, semi-structured

interviews, focus group with AfDB and

in-country


interventions implemented in a timely

manner and in compliance with

operational standards?

Portfolio, project Portfolio level:

Evolution of portfolio performance:

Timeliness: Start up delays as well as implementation and closure delays;

IPR ratings;

Overview of project performance(PP/PAR/PPP), disbursement ratio/rating;

Evidence of supervision reporting, identifying and addressing issues of;

procurement, financial management, project design review (changes);

Evidence of frequency of monitoring.

Project management:

Compliance to reporting: audit reports, progress reporting from borrower;

Evidence that problematic projects are well monitored;

Evidence of Response to project management issues.

Project level:

Timeliness: Comparison between the planned and the actual period of

implementation from the date of effectiveness (approval stage – signature

loan agreement- first disbursement);

Dates of disbursement

Supervision and administration.

Design


10. To what extent is the quality of the CSP

satisfactory?

Strategic Compliance with AfDB guidelines valid at the time of drafting the CSP and

current (using existing quality at entry criteria in addition to SA specific

analysis).

RBM

Analysis AfDB strategies, semi-structured

interviews

11. To what extent has the Bank applied

selectivity in designing its country portfolio

and focused on areas where it brings added

value?

Portfolio Strategy: The CSPs demonstrate increased focus and selectivity, based on

analysis of where the Bank adds value.

Portfolio: The portfolio is focused on a limited number of sectors and sub-

sectors, as per CSPs and analysis of where Bank adds value.

Project: identification and appraisal includes consideration of added value.

Stakeholder views on Bank success in finding added value/potential for

future comparative advantage.

No of unsigned, almost approved and abandoned operations after appraisal

No of operations approved but cancelled.

Portfolio review, semi-structured interviews with

AfDB

12. To what extent has the Bank been innovative

in adapting its approach to the country’s

context and development challenges/needs?

Project Evidence of changes to project or strategy design following change in

country or borrower context

Evidence of responsiveness of Bank to country context.(eg. Impact of

government policies on success of operations)

PRA, semi-structured interviews with AfDB and

other stakeholders, focus groups with AfDB and in-

country

13. To what extent are the Bank’s interventions

coherent and well-coordinated internally?

Portfolio, project Evidence of country team coordination efforts

Evidence of complementarity and efficient use of instruments in response to

country needs

Evidence of operations aligning to Bank’s emerging initiatives and strategic

direction.

PRA, Portfolio review, semi-structured interviews

with AfDB

Knowledge and policy advice


14. To what extent has the Bank actively

engaged in and influenced policy

dialogue through relevant advice?

Strategic Extent to which relevant guidelines (on policy dialogue) are implemented

by the Bank;

#Participation in dialogue with RMC at various levels of governance

#Advice provided to RMC and beneficiaries

Analysis CSPE’s and strategic documentation,

semi-structured interviews with AfDB,

beneficiaries and other stakeholders

15. To what extent has the Bank delivered

adequate analytical work in support of its

interventions, positioning and policy

advice?

Strategic, portfolio, project #Studies, monitoring and evaluations, analyses undertaken by the AfDB

Proportion of operations preceded by Bank funded studies

Views from stakeholders on usefulness, availability and dissemination of

knowledge products.

PRA, portfolio review, analysis CSPE’s and

strategic documentation, semi-structured

interviews with AfDB, beneficiaries and other

stakeholders, focus groups in country

Partnerships and leverage



harmonized with those of other DPs avoiding

duplication, simplifying procedures etc.)?

Strategic, portfolio #Meetings with other DPs (institutionalised and occasional)

#Agreements on common strategies with other DPs (e.g. CAF)

#Sectoral partnerships with other donors

Views of stakeholders on efforts made for

harmonisation/coordination/complementarity.

Proportion of projects for which procedures are shared (either with GoSa or

other DPs)

Analysis AfDB strategic documentation, portfolio

review, semi-structured interviews with AfDB,

beneficiaries and other stakeholders (donors)


and resources bringing in other players and

being leveraged for maximizing development

effectiveness at country level?

Strategic, portfolio #Common interventions with other DPs

# Interventions for which other DPs were encouraged to come on board

following Bank engagement.


review, semi-structured interviews with AfDB,

beneficiaries and other stakeholders (donors), in-

country focus group

Managing for results


18. To what extent has the Bank successfully

implemented management systems that

focus on results and allow learning from

past experience?

Strategic, portfolio, project Application RBM principles in the AfDB, inclusion of objectives,

expected results and indicators to monitor progress of achievements;

M&E system in place in AfDB;

Proportion of projects for which problems identified through monitoring;

proportion of which projects adjusted to address problem.

Proportion of projects which consider lessons from previous projects in

their design.


review, PRA, semi-structured interviews with

AfDB and beneficiaries

19. To what extent has the Bank supported

the development of national capacities

and management systems that focus on

results?

Portfolio, project Portfolio: #Capacity building projects funded (lending & non-lending

activities) by the AfDB that aim to introduce RBM (and alike) principles

in SA

Project: Extent to which project design and monitoring process

encouraged clients to take on more results based approach.

Lessons learned


20. What are the key factors positively and

negatively influencing the achievement of

development results?

Portfolio, project Evidence of key program implementation issues linked to project

success/failure from stakeholders perspective.

PRA, portfolio reviews, semi-structured

interviews with AfDB

Annex 5: Country ToC

Strengthening the policies and developing a sound policy environment

and an enabling environment for private

sector development

Investment in climate resilient

infrastructure for road transport ,

energy, agriculture and water, health & sanitation, and

education

Policy advice and analytical work

focusing on resource mobilisation, oil

resources and subsidy manage-ment, job

creation, green growth, gender mainstreaming,

and regional integration

Inreased percentage of Federal, State and local road network in

good conditionProvision of concessional credit

to agribusinessIncrease average travel speed and

commuter travel time reduced

Intermediaries have greater financial

capacity to on lend to SMEs, ensuring their

own sustainability and focusing on

development additionality

Context Input Output Intermediate Outcome Outcome Impact

- One of Africa’s fastest growing economies

- Heavy reliance on oil Need for diversification

- Acess to financing of enterprises/ SMEs is a bottleneck, which might even become stronger due to the drop in the oil price

- Lack of adequate infrastructure in all sectors (transport, energy, water and sanitation) also constraint the growth in all sectors, esp. agriculture

- Fast growing young population facing large unemployment and social deprivation

- Complex governance structure with a Federal level, 36 states and local

authorities

- Authorities’capacities generally face constaints and authority is challenged by Boko Haram in the North-East

Increased production of food and cash crops and

better distribution of fertilizer

Capacity building for PPP

Infrastructure

Federal, State and local roads rehabilitated or

upgraded

Power generation increased and

transmission network extended and strengthened

Increased productivity & income

for farmers

Contribution to improved business

and investment environment

Increase of the actually irrigated area

Reduction in energy losses and additional customers connected

Improved health situation of the

population

Better policy formulation and

implementation by the GoN, leading to

enhanced

macroeconomic environment,

improved public financial

management and improved public-private dialogue,

Increased awareness and skills concerning,

PPP, resource mobilisation, jobcreation, green

growth, gender mainstreaming and regional integration

Increased knowledge-sharing,

in PPP, resource mobilisation, job creation , green

growth, gender mainstreaming and regional integration

Investing in critical, safe and efficient

infrastructure to promote the development of the

real sector of the

economy,

More jobs created in the value chain of agricultural crops

Increase in access to electricity and

reduction in electriciy cost

New urban and rural water connections and

sanitation facilities

Construction and rehabilitation of dams and irrigation schemes

Increased access to clean water and

sanitation facilities

Indirect investments to deepen private sector participa-

tion in infrastruc-ture financing

through private equity funds,

lines of credits and partial risk

guarantees

Increased access to finance in the private

sector

Economic diversification

towards non-oil sectors

Leveraging third-party investment in

the form of co-financing Food security ensured

Inclusive growth and transition to green growth

in Nigeria

Network of public educational and health

facillities expanded

Improved access to education and health

Increas ed share of the population with

education

Workforce has skills in demand

by employers to support job

creation.

Nigerian economy is not

negatively affected by low-oil

price and global economy.

Nigerian politics and public

administration engaged in

reforms.

Government regulation and

policies favorable for successful

implementation, and as a result

the regulatory environment

supports business growth.

No major barriers or stumbling

blocks to regional integration,

trade, and the free flow of

people and capital.

Private sector development supports

inclusive and green growth, not

unequal growth or growth that is not

environmentally sustainable.

Political will for transition to green

growth.

Political and social stability.

Appropriate targeting and

level of resources.

Right instruments available.

Complementarity and

coordination with other

development

partners/sources of finance.

Bank (and other partners)

provide inputs on time and

as planned. This includes

Bank's flexibility in re-

aligning to sudden changes

of government regulations

and policies as well as

adaptability in its approach

to changing market

conditions.

Partner meets conditions to

allow disbursement on time.

Bank inputs are well

designed (evidence

based/ownership/ design)

and sequenced.

AfDB procedures are clear to all relevant

stakeholders

AfDB loans are attractive to borrowers

despite reporting requirements and

procedures.

Implementation on schedule.

Design appropriate and scalable.

Sound management in partner

organisations.

Risks monitored and well managed.

Procurement systems efficient and effective.

Sufficient capacity available to implement.

Training/equipment is effective in

enhancing capacity.

Appropriate institutional

support/capacity/policy space to support

use of funds.

Knowledge work of sufficient quality, and

disseminated.

Improvements in infrastructure

causes economic spill-overs to

other sectors (especially growth

in agricultural production).

Growth in agricultural

production brings along

multiplier effects (e.g. supply to

agribusiness).

New and refurbished

infrastructure is suitable for

harsh tropical conditions (e.g.

resistant to heavy rainfall).

Politics and public

administration are willing to

engage in good governance.

Regulatory environment

encourages take up of access to

finance.

Knowledge shared is relevant to

needs and there is absorption

capacity to use it.

Assumptions

Mainly internal ------------------------------------------------------------------------------------------------------------------> Mainly external

Risks

Political stability challenged by disfunctional Federal government and/ or Boko Haram. Social stability challenged by significant ineqaulities, poverty and high unemployment rates, partiularly among young poeople.The number of jobs created is insufficient to absorb the millions of newcomers on the labour market each year.Low oil price threatens solvency of the Nigerian economy.

Adverse global economic environment leading to increased vulnerability of the country to unexpected capital movements and changes in commodity prices.Fiscal policy of the Government will not be credible and irresponsive.

Annex 6: Portfolio review template

Portfolio review template

Assessment

Area

Focus Criteria Main Sources of Information

and Data

Relevance Alignment with

Bank priorities

(CSPs)

Extent of emphasis on increased job

creation, increased entrepreneurial

activity, and increased regional

Integration

Annual Reports (2004 – 2013)

Country Portfolio & Improvement

Reports

Country Strategy

Credit notes where applicable

Identification or preparation

reports if available

Loan Agreement

Papers(Bank)

Policy and strategy documents

(Bank)

Portfolio data (SAP)

Project Appraisal Reports

Project concept notes

Statistical data (Bank)

Country development plans

(govt)

Alignment with

cross-cutting

issues

Extent of emphasis on inclusive

growth, environmental sustainability,

gender equity, good governance and

knowledge solutions

Alignment with

country

priorities

Degree of alignment with national

policies, sector strategies etc.

Alignment with

beneficiary

needs

Evidence of consideration of

beneficiary needs at appraisal

Evidence that project results meets

beneficiary needs

Bank’s added

value

Evidence of the added value which the

Bank’s strategy and focus provides

Effectiveness Achievement of

actual or

expected

outputs

Output execution ratios

What factors served to enable

outputs?

What factors served to hinder outputs?

Policy reviews(Strategy

documents, MTS 2008-2012,

TYS 2013-2022

Annual Reports (2004 – 2013)

IDEV’s evaluations; SME’s, QaE,

Transport, Energy, ESWs

Development Partners sector

review reports

Achievement of

actual or

expected

outcomes

Degree of contribution to Intermediate

Outcomes, and of Outcomes, in the

country-level ToC

What factors served to enable

outcomes?

What factors served to hinder

outcomes?

Unintended

outcomes

Positive and negative contribution of

unintended outcomes to ToC

Portfolio

distribution

Sector distribution, volume and size;

status of operations; ratio of lending

and non-lending/ private sector and

public sector

Percentage and ratio of loans and

Grants; ratio of co-financed initiatives

with DFIs

Contribution of multinational

operations benefitting the country

Efficiency Cost-benefit

analysis

Economic Rates of Return

Sizes of contribution to economic

growth

SAP portfolio data

Flashlight reports

Outliers report

Supervision reports Cost- Costs of alternative ways to achieve

Assessment

Area


and Data

effectiveness

analysis

project objectives Progress Reports

Project monitoring Reports

Audit reports

Borrowers reporting documents

Reference to bank’s

standard/criteria for self-

assessment and quality control

(see PCR guidelines and

portfolio improvement action plan

series

Project Appraisal Report(PAR)

Project concept notes and

technical notes

Country Strategy Papers (Bank)

ADOA Guidelines

Evaluation of QaE (IDEV)

PCR guidelines

Readiness Review Guidelines

Bank

investment

profitability

Net profit contribution

Return on equity

Timeliness Start up delays9 as well as

implementation and closure delays

What factors served to cause delays?

Quality at entry Evidence of Quality of Logic or Results

Framework.

Evidence of Risk Assessment and

existing mitigation measures in design.

Evidence of consideration of lessons

from previous or similar interventions.

Adequacy of context analysis and

alignment to rationale for intervention

Evidence of consideration of Bank’s

added value/niche.

Co-financing – extent to which Bank

led and leveraged funds from others,

versus played a supportive role.

Adequacy of reporting requirements.

Discrepancy between ADOA ratings

and PAR (for private sector projects

only)

Implementation

progress

IPR ratings

Overview of project

performance(PP/PAR/PPP10),

disbursement ratio/rating

Project

management

Evidence of supervision reporting,

identifying and addressing issues of:

procurement, financial management,

project design review (changes)

Evidence of frequency of monitoring

Evidence that problematic projects are

well monitored

Evidence of responses to project

management issues:

Responses to complex issues with

large co-financed projects

Evidence of internal coordination to

respond to project management

issues

Compliance to reporting requirements:

audit reports, progress reporting

Portfolio Extent to which the portfolio is

9 Assessment of time taken from Project Approval (approval date) to project effectiveness and 1st disbursement 10 Project Assessment Criteria: PP - Problematic Projects; PAR - Projects at Risk; PPP - Potential Problematic Project

Source: Portfolio Flashlight Report

Assessment

Area


and Data

coherence and

performance

coherent and well-coordinated in

achieving objectives of CSP;

Effect of decentralisation on portfolio

performance

Sustainability Technical

soundness

Soundness of technology used,

including operations and maintenance

requirements

Supervision reports

Progress Reports

Project monitoring Reports

Project Appraisal Report(PAR)

Project concept notes and

technical notes

Economic and

financial

viability

Extent to which effective mechanisms

and modalities are in place to ensure a

sustained flow of benefits

Institutional

sustainability

Extent to which improved governance

practices, improved skills and

capacities, and improved institutional

mechanisms have come into effect,

and commitment and ownership exists

to ensure a sustained flow of benefits

Annex 7: Project results assessments templates

Public sector project template

Name of project Insert here

Country Insert here

Year approved Insert here

Year Closed (if still open, insert that open and also level of disbursement) Insert here

Sector Insert here

PCR available Yes/No

1.1 RELEVANCE

1.1 Relevance of project objectives Rating

(1 - 6) The relevance of objectives assesses to what extend the project purpose as specified in the RLF was aligned with the Bank’s CSP and the applicable sector

strategies, the country’s development strategies and the beneficiary needs from design/approval to completion (including any adjustments that were made to the

project in view of changes in the applicable policy environment, such as project restructuring).

Insert your evidence here Insert your

rating here

1.2 Relevance of project design to achieve those objective

The relevance of project design should consider: (i) the extent to which the project’s objectives are clearly stated and focused on outcomes as opposed to outputs;

(ii) The realism of intended outcomes in the country’s current circumstances also is assessed; (iii) the extent to which project design adopted the appropriate

solutions to the identified problems; (iv) the modifications to project design; (v) the circumstances prevailing at the time of the evaluation.


rating here

Overall Rating for Relevance Insert your

rating here

2. EFFECTIVENESS

The assessment of Effectiveness tests the validity of the anticipated links between the project’s activities, outputs, and intended outcomes (the results chain). Actual, expected and

unintended results of an operation are included in the assessment of Effectiveness. For PBOs the assessment should not only review the extent to which outputs were delivered (i.e.

agreed-upon policy reforms took place), but also the degree to which complementary measures necessary for their implementation occurred (e.g. public awareness, policy dialogue

and institutional arrangements).

2.1 Achievement of outputs Rating

(1 - 6) The assessment of outputs is based on the output execution ratio. It should consider the planned (targets) and actual output or those who are considered on track

to be reached. In determining the final rating, no formula based on a pre-determined weight applied to individual outputs is undertaken. It is considered good

practice to select no more than 10 output indicators in the RLF and to take into account the relative importance of the various components of the project in their

selection. The overall output rating is based on the percentage of outputs (output execution ratio) that reached or are on track to meet the end of project target.


rating here

2.2 Achievement of outcomes

The assessment of outcome puts is based on the direct and intermediate outcomes stated in the retrospective project logic model. It should cover at least the core

sector indicators.


rating here

2.2 Unintended outcomes (if any)

The assessment will cover all unintended outcome which came out during the project cycle


rating here

Overall Rating for Effectiveness Insert your

rating here

3. EFFICIENCY

The Efficiency assessment attempts to answer two questions: (i) Did the benefits of the project (achieved or expected to be achieved) exceed project costs; and (ii) Were the benefits

of the project achieved at least cost? Cost-benefits analysis helps to address the first question. To address the second question a cost-effectiveness analysis is carried out. Good

practices suggest also the, In addition to the traditional measures of efficiency (cost-benefit analysis and cost-effectiveness analysis), the Efficiency assessment considers aspects of

project design and implementation that either contributed to or reduced efficiency (Timeless and Implementation progress) to the extent they are not already captured in the

evaluation’s cost-benefit or cost-effectiveness analysis.

3.1 Cost-benefits analysis Rating

(1 - 6) Cost-benefit analysis is carried out to the extent that data is available. The validity of the cost-benefit analysis conducted at appraisal/mid-term review is re-

assessed at completion It is a recommended to use the same model that was developed at appraisal. For PBOs a quantitative assessment will be done if an

Economic Rate of Return (ERR) was calculated at appraisal, otherwise an assessment could be done with regards to the contribution of policy reforms to

economic growth (if not applicable, indicate N/A for this criterion).


rating here

3.2 Cost-Effectiveness Rating

(1 - 6) The analysis considers the cost of alternative ways to achieve project objectives, unit costs for comparable activities, sector or industry standards, and/or other

available evidence of the efficient use of project resources.


rating here

3.3 Timeliness Rating

(1 - 6) The timeliness of project implementation is based on a comparison between the planned and the actual period of implementation from the date of effectiveness.

For PBOs, the timely releases of the tranche(s) are assessed through this criterion. The following rating scale applies


rating here

3.4 Implementation progress (IP) Rating

(1 - 6) The IP rating will be derived from the IPR that shall be updated in tandem with the PCR preparation. The IP rating takes into account all applicable IP criteria

assessed under each of the three main categories: i) compliance with covenants (project covenants, environmental and social safeguards and audit compliance),

ii) project systems and procedures (procurement, financial management and monitoring and evaluation), and iii) project execution and financing (disbursement,

budget commitments, counterpart funding and co-financing). The simple arithmetic average of the individual ratings is calculated to derive the final rating. The

overall IP rating is provided as follows.


rating here

Overall Rating for Efficiency Insert your

rating here

4. SUSTAINABILITY

The assessment of sustainability considers the extent to which the project has addressed risks during implementation and put in place mechanisms to ensure the continued flow of

benefits after completion. It should also evaluate risks to the sustainability of development outcomes and/or the project’s benefits, including the resilience to exogenous factors. The

overall rating of the sustainability outcome is the mean of the rating of the following four criteria: i) technical sustainability; ii) financial sustainability, iii) institutional sustainability and

strengthening of capacities, iv) ownership and sustainability of partnerships and v) environmental and social sustainability.

4.1 Technical Soundness Rating

(1 - 6) The criterion assesses the extent to which the project achievements rely on sound technology using inputs efficiently and providing productivity gains. It includes

operation and maintenance (O&M) facilitation, availability of recurrent funding, spare parts, workshop facilities etc.)

Insert your evidence here

Insert your

rating here

4.2 Economic and Financial viability Rating

(1 - 6) This criterion assesses the extent to which funding mechanisms and modalities (e.g. tariffs, user fees, maintenance fees, budgetary allocations, other stakeholder

contributions, aid flows, etc.) have been put in place to ensure the continued flow of benefits after project completion, with particular emphasis on financial

sustainability. For PBOs the assessment should focus on the financial sustainability of the reforms, as well as the Bank’s policy dialogue to promote financial

sustainability of the reforms.


Insert your

rating here

4.3 Institutional sustainability and strengthening of capacities Rating

(1 - 6) The criterion assesses the extent to which the project has contributed to strengthen institutional capacities - including for example through the use of country systems

- that will facilitate the continued flow of benefits associated with the project. An appreciation should be made with regards to whether or not improved governance

practices or improved skills, procedures, incentives, structures, or institutional mechanisms came into effect as a result of the operation. For PBOs this should include

an assessment on the contributions made to building the capacity to lead and manage the policy reform process; as well as the extent to which the political economy

of decision making was conducive to reform, the Government’s commitment to reform and how the design reinforced national ownership


Insert your

rating here

4.4 Political and governance environment

This criteria assesses the extent political and governance developments that could impact the government’s priorities with respect to the project. This includes (but is

not limited to) upcoming elections or an impending change in government; and other factors that could impact the political commitment to the operation or operational

engagement and the political decisions required for sustainability of project results (including laws and the provision of counterpart financing). Special attention should

be paid to fraud, corruption and other unethical practices resulting from governance failures.


4.5 ownership and sustainability of partnerships Rating

(1 - 6) The assessment determines whether the project has effectively involved relevant stakeholders, promoted a sense of ownership amongst the beneficiaries (both men

and women) and put in place effective partnerships with relevant stakeholders (e.g. local authorities, civil society organizations, private sector, donors) as required for

the continued maintenance of the project outputs. For PBOs, the assessment should include the extent to which the Government conducted extensive consultations

during the preparation and implementation of the PRSP and the extent to which the Bank supported the Government in deepening the consultation processes


Insert your

rating here

4.6 environmental and social sustainability Rating

(1 - 6) This criterion would normally only apply to Environmental Category I and II projects. It assesses the extent to which the environmental and social

mitigation/enhancement measures of the project were implemented, the capacity of country institutions and systems and the availabil²ity of funding to ensure the

environmental and social sustainability of the operation. The Environmental and Social Safeguards rating in the IPR should be used as a guidance.


Insert your

rating here

4.7 Resilience to exogenous factors and risk management Rating

(1 - 6) This criteria assesses the extent to which the achievements depend on exogenous factors, such as the terms of trade, the world market prices or the political situation

in neighbouring countries.


Overall Rating for Sustainability Insert your

rating here

Private sector projects template

Name of project Insert here

Country Insert here

Year approved Insert here

Project Reached Early Operating Maturity11 (if not, include level of disbursement) Insert here

Sector Insert here

XSR and/or XSREN available XSR: Yes/No XSREN (Yes/No)

1.1 RELEVANCE

1.1 Relevance of project objectives Rating

(1 - 6) The relevance of objectives assesses to what extend the project purpose as specified in the approval document was aligned with the relevant RMC CSP and the

applicable sector strategies, the country’s own development strategies and the beneficiary needs from design/approval to completion (including any adjustments that

were made to the project in view of changes in the applicable policy environment, such as project restructuring).

Insert your evidence here Insert rating

here

1.2 Relevance of project design to achieve project objective (Quality of front-end work and additionality)

The relevance of project design is evaluated via assessing the following:

A) ‘Screening, Appraisal and Structuring’ stage. This sub-dimension assesses how the Bank carried out its work on the project prior to commitment with

reference to the following specific aspects: i) Relevance of the investment to the Bank’s corporate, country and sector strategies (see above); ii) Identification

of risks that the investment would fail to meet the intended development objectives or generate adequate returns; iii) The sponsors, company, management,

country conditions, market dynamics, project concept, configuration and costs; iv) Financing plan, sources of financing, and assumptions used in financial

and economic projections; v) Political risks and mitigation measures; vi) Environmental and social risk assessment and action plans to mitigate adverse

effects; vii) Investment instrument selection, structure, pricing, exit mechanism, security, covenants and other terms and conditions; and viii) Client

satisfaction with the Bank’s pre-commitment work;

B) Additionality: The Bank’s additionality measures what Bank financing brings to the project over and above commercial financiers. It is based on the

counterfactual assessment of how the project would have proceeded without Bank financing. This dimension is measured through two sub-indicators:

financial additionality and non-financial additionality. The rating for additionality is a synthesis of the rating of its two underlying sub-indicators. i) Financial

Additionality measures the special contribution that the Bank’s funding offers the client that would otherwise not have been offered by other financiers which

includes; would the client have been able to obtain sufficient financing from private sources on appropriate terms? Did the Bank catalyse other funding or did

it merely fill a financing gap? Was the Bank’s financing needed to reduce risk or provide comfort thereby encouraging other financiers to invest in the

11 Refers to a point in time at which an investment operation is ready for self-evaluation. See Annex 1 for detailed description of project typology and criteria for early operating maturity by project type.

undertaking? ii) Non-Financial Additionality measures the Bank’s contribution to reducing the projects risk profile, design or functioning. The rating is

determined by considering answers to questions such as: Was the Bank needed to bring about a fair allocation of risks and responsibilities e.g., between

public and private investors? Did the Bank’s participation lead to improved design, enable the client to adopt new or better standards or contribute to the

client’s capacity building objectives through technical assistance, training, etc.?


here

Overall Rating for Relevance

Insert rating

here

2. EFFECTIVENESS

The assessment of Effectiveness includes accounting for the actual, expected and unintended results on outcomes level for an operation. For Lines of Credit operations, the

outcomes typically accrue on the level of the partner financial intermediary. The changes in the underlying portfolio of the financial intermediary as well as the increased efficiency

and financial deepening as a result of Bank operations. Finally, for equity/investment funds operations, it is preferred to extend the assessment to investee companies.

2.1 Achievement of outputs Rating

(1 - 6) The assessment of outputs is based on the output execution ratio. It should consider the realization of actual physical outputs of the project. Depending on the type of

project, this could be production line in expansion operations, establishment of plant and/or equipment in greenfield operations, etc. In determining the final rating,

output rating is based on the percentage of outputs (output execution ratio) that reached or are on track to meet the end of project target.


here

2.2 Achievement of outcomes Rating

(1 - 6) The assessment of outcome is based on the direct and intermediate outcomes stated in the retrospective project logic model. Typical outcomes of a private sector

operation covers the following areas:

i) Economic benefits; the best indicator of a non-financial market project’s contribution to economic growth is its economic rate of return (ERR) or economic return on

invested capital (EROIC). Ideally, the ERR/EROIC considers and quantifies the projects economic effects on all its economic stakeholders12. For Financial Markets

projects the economic benefits measures the extent to which the sub-projects financed with the proceeds of the line of credit or the investee companies in the case

of equity funds are providing a net economic benefit to stakeholders including and beyond the FI’s owners and financiers. Such benefits include, but are not limited

to: Contribution to government revenues resulting from taxes paid by the intermediary, sub-projects or investee companies; Contribution to poverty alleviation, social

or gender equality and regional development etc.; Delivery of community services such as entrepreneurship training, educational programs and other community

services; employment generated;

ii) Contribution to Private Sector Development; measures the extent to which the project has spread benefits of growth of productive private enterprise beyond the

12 The universe of entities impacted by a project in addition to the financiers and employees include: government, the rest of society, customers, producers of complementary products, competitors,

suppliers and neighbors.

project company, i.e. on issues such as competition, market expansion, private ownership & entrepreneurship, development of financial institutions and markets,

standards of corporate governance, transfer of technology and dispersion of skills, and the development of physical infrastructures used by other private parties.

The project can have positive or negative impacts on private sector development and it is necessary to establish that the impacts are attributable to the project.

Indicators include: Upstream and downstream supply linkages to local private businesses; introduction of new technology and know-how; enhancement of private

ownership and entrepreneurship; contribution to improving the environments for private sector development and an open economy; greater competition and

competitiveness; broad demonstration effects in the local economy and follow-on investments by other investors; domestic capital market development and greater

resource allocation efficiency; improvements in standards for corporate governance and business conduct; and development of physical infrastructures used by

other private parties.


here

2.2 Unintended outcomes (if any) Rating

(1 - 6) The assessment will cover all unintended outcome which came out during the project cycle


here

Overall Rating for Effectiveness Insert rating

here

3. EFFICIENCY

The Efficiency assessment examines the Bank investment profitability, the timeliness of Bank implementation and disbursement as well as its supervision and administration

arrangements.

3.1 Bank Investment Profitability Rating

(1 - 6) For the Bank to continue to be sustainable, the investments it makes, whether in the form of loans or equity have to be profitable. For loans: The best indicator of the

Bank’s investment (profitability) in a project is the net profit contribution (gross income less financing costs, loan loss provisions/ write-offs, transaction costs and

administrative costs measured in discounted cash flow terms. However, because of the difficulty in estimating transaction and administrative costs associated with

individual projects before the Bank implements a viable cost accounting system, a qualitative approach based on gross profit contribution (gross income less financing

costs, loan loss provisions/ write-offs) is recommended. For equity investments, profitability shall be measured by comparing the nominal internal rate of return (also

referred to as return on equity (ROE)), computed using projected dividends and

Insert your evidence here Insert

rating here

3.2 Timeliness Rating

(1 - 6) The timeliness of project implementation is based on a comparison between the planned and the actual period of implementation from the date of effectiveness. For

LOCs, the timely releases of the tranche(s) are assessed through this criterion. The following rating scale applies


rating here

3.3 Supervision and Administration Rating

(1 - 6) Supervision and administration starts after the investment agreement between the Bank and the beneficiary has been signed. The assessment indicates the extent to

which the Bank has professionally executed its responsibilities post commitment. The factors related to the Bank’s administration of the investment that must be taken

into consideration include: The monitoring of the client company’s compliance with investment covenants and conditions; the completeness of supervision reports in

documenting project implementation and risks; the monitoring of the client company’s environmental and social performance; the adequacy and timeliness of the

Bank’s response to emerging problems or opportunities; the contributions made by Bank representatives on investee company Boards; client satisfaction with the

Bank’s service quality; and the continuity of the Bank’s service delivery when monitoring staff changes occur.


rating here

Overall Rating for Efficiency Insert

rating here

4. SUSTAINABILITY

The assessment of sustainability considers the extent to which the performance of the project as a proxy for its long-term sustainability

4.1 Business Success i.e., financial performance and fulfilment of project business objectives Rating

(1 - 6) For Non-Financial Market projects, this sub-dimension measures the project’s actual and projected financial impact on the project’s financiers (lenders and equity

investors) over the economic life of the project, the project’s contribution to other business goals articulated at appraisal. The effect of the project on the company is

assessed on a “with versus without” basis, or on a “before versus after” basis. The principal indicator for business success is the financial rate of return (FRR) based

on real, after tax cash flows for project loans or the return on invested capital (ROIC) in the case of corporate investments. For financial market operations, business

success measures the effect of the LOC/investment on the FI’s profitability and contribution to other business goals.


Insert rating

here

4.6 Environmental and Social Performance Rating

(1 - 6) The rating of environment and social performance is based primarily on the Bank’s specified standards in effect at approval and secondarily on the Bank’s

environmental and social standards prevailing at the time of evaluation. The assessment should cover i) the project’s environmental and social performance in

meeting the Bank’s requirements; and ii) the project’s actual environmental and social impacts, including pollution loads, wastes, energy and resource efficiency,

biodiversity conservation, workers’ and communities’ health and safety, public consultation and participation, land acquisition and cultural heritage. In the case of

financial intermediation projects, the assessment should consider the adequacy of the FI’s or fund manager’s Environmental and social management system and its

implementation.


here

Overall Rating for Sustainability Insert rating

here

Annex 8: Examples of rating guidance: Effectiveness for public and private sector projects

The following is taken from the broader guidance for project results assessment, to provide an example. Full guidance is available

covering four criteria for each of the public and private sector projects.

Public sector projects

EFFECTIVENESS

The assessment of Effectiveness tests the validity of the anticipated links between the project’s activities, outputs, and intended

outcomes (the results chain). Actual, expected and unintended results of an operation are included in the assessment of

Effectiveness. For PBOs the assessment should not only review the extent to which outputs were delivered (i.e. agreed-upon policy

reforms took place), but also the degree to which complementary measures necessary for their implementation occurred (e.g.

public awareness, policy dialogue and institutional arrangements).

a) Achievement of outputs

The assessment of outputs is based on the output execution ratio (see table below) and the quality of outputs. It should consider

the planned (targets) and actual output or those who are considered on track to be reached. In determining the final rating, no

formula based on a pre-determined weight applied to individual outputs is undertaken. If possible, select no more than 10 output

indicators in the RLF and to take into account the relative importance of the various components of the project in their selection.

Any selection need to be clearly justified.

The overall output rating is based on the percentage of outputs (output execution ratio) that reached or are on track to meet the end

of project target.

The following table demonstrates what is expected:

Major Activities Expected Outputs Actual Outputs Outputs execution rate Outputs quality

assessment

The following rating scale applies:

6 – Highly Satisfactory: Based on the output execution ratio all the project output targets were reached or are considered

on track to be reached by the end of the project in accordance with quality standards;

5 – Satisfactory: Based on the output execution ratio between 90% and 99% of the project output targets were reached or

are considered on track to be reached by the end of the project. Corrective actions for off track indicators were implemented

in a timely manner to ensure that the end of project targets could be achieved in accordance with quality standards;

4 – Moderately Satisfactory: Based on the output execution ratio between 75% and 89% of the project output targets were

reached or are considered on track to be reached by the end of the project. Corrective actions for off track indicators were

implemented in a timely manner to ensure that the end of project targets could be achieved in accordance with quality

standards;

3 – Moderately Unsatisfactory: Based on the output execution ratio between 50% and 74% of the project output targets

were reached or are considered on track to be reached by the end of the project. Corrective actions for off track indicators

were not implemented in a timely manner to ensure that the end of project targets could be achieved;

2 – Unsatisfactory: Based on the output execution ratio between 35% and 49% of the project output targets were reached

or are considered on track to be reached by the end of the project. Corrective actions were not implemented and closely

monitored for off track indicators. Poor performance jeopardized the achievement of one or more outcomes of the project;

1 – Highly Unsatisfactory: Based on the output execution ratio less than 35% of the project output targets were reached or

are considered on track to be reached by the end of the project. Poor performance jeopardized the achievement of most

expected outcomes and the possibility of stopping or suspending the project was considered.

b) Achievement of outcomes

Outcomes are assessed against the project’s objectives as contained in the Project Appraisal Report (PAR). The assessment of

outcome is based on the direct and intermediate outcomes stated in the retrospective project logic model. If the statement of project

objectives in the appraisal documents is unclear or is focused on outputs rather than outcomes, the evaluator reconstructs an

outcome-oriented statement of objectives using the project’s results chain, performance indicators and targets, and other

information including country strategies and interviews with government officials and AfDB staff. The anticipated links between the

project’s activities, outputs, and intended outcomes are summarized in the project’s results chain. The results chain is taken from

the PAR. If the results chain is absent or poorly defined, the evaluator constructs a retrospective results chain from the project’s

objectives, components, and key performance indicators.

The evaluator should make sure that data collection remains open to unintended results that have not anticipated by including

some open-ended questions in interviews and questionnaires, and by encouraging reporting of unexpected results. Unexpected

benefits, once known about, can be designed into future interventions.

The ECG's 'Big Book on Evaluation Good Practice Standards'13 recommends the following approach in this area:

1. Assess the causal chain in relation to the needs of the target population, collaborating with stakeholders and experts;

2. Examine the critical assumptions and expectations inherent in the project’s design, reviewing the logic and plausibility of the results chain.

Again, this is done in collaboration with stakeholders;

3. Use available research evidence and practical experience elsewhere, comparing the project with projects based on similar concepts;

4. Observe the project in operation, focusing on interactions that were expected to produce the intended outcomes.

For PBOs the assessment should not only review the extent to which outputs were delivered (i.e. agreed-upon policy reforms took

place), but also the degree to which complementary measures necessary for their implementation occurred (e.g. public awareness,

policy dialogue and institutional arrangements). Since PBOs are typically joint with other donors and RMC governments and are

implemented through country systems, it may be difficult to attribute a direct link between the specific inputs of the Bank Group

(and those of other partners) and the expected results. Therefore, the progress will be measured in terms of the collective efforts of

the RMC and other partners, where applicable, while taking into account other external factors.


6 – Highly Satisfactory: Taking into account the latest value of the outcome indicators and the analysis of other relevant

exogenous risks/factors and assumptions, it is plausible to expect that all intended project outcomes were achieved or are

likely to be achieved;

5 – Satisfactory: Taking into account the latest value of the outcome indicators and the analysis of other relevant

exogenous risks/factors and assumptions, it is plausible to expect that most (75%) intended project outcomes were

achieved or are likely to be achieved;

4 – Moderately Satisfactory: Taking into account the latest value of the outcome indicators and the analysis of other

relevant exogenous risks/factors and assumptions, it is plausible to expect that a substantial (50%-74%) intended project

outcomes were achieved or are likely to be achieved;

3 – Moderately Unsatisfactory: Taking into account the latest value of the outcome indicators and the analysis of other

relevant exogenous risks/factors and assumptions, it is plausible to expect that few (25-49%) intended project outcomes

were achieved or are likely to be achieved;

13 https://www.ecgnet.org/document/ecg-big-book-good-practice-standards.

2 –Unsatisfactory: Taking into account the latest value of the outcome indicators and the analysis of other relevant

exogenous risks/factors and assumptions, it is plausible to expect that few (5-24%) intended project outcomes were

achieved or are likely to be achieved;

1 – Highly Unsatisfactory: Taking into account the latest value of the outcome indicators and the analysis of other relevant

exogenous risks/factors and assumptions, it is plausible to expect that very few (<5%) of the intended project outcomes

were achieved or are likely to be achieved.

Private sector projects

EFFECTIVENESS

The assessment of Effectiveness includes accounting for the actual, expected and unintended results on outcomes level for an

operation. For Lines of Credit operations, the outcomes typically accrue on the level of the partner financial intermediary.

Measurement of changes in the underlying portfolio of the financial intermediary as well as the increased efficiency and financial

deepening as a result of Bank operations would represent desirable outcomes. Finally, for equity/investment funds operations, it is

preferred to extend the assessment to investee companies.

a) Achievement of outputs

The assessment of outputs is based on the output execution ratio. It should consider the realization of actual physical outputs of the

project. Depending on the type of project, this could be production line in expansion operations, establishment of plant and/or

equipment in Greenfield operations, etc. In determining the final rating, output rating is based on the percentage of outputs (output

execution ratio) that reached or are on track to meet the end of project implementation targets.


6 – Highly Satisfactory: Based on the output execution ratio all the project output targets were reached or are considered on track

to be reached by the end of the project in accordance with quality standards.

5 – Satisfactory: Based on the output execution ratio between 90% and 99% of the project output targets were reached or are

considered on track to be reached by the end of the project. Corrective actions for off track indicators were implemented in a timely

manner to ensure that the end of project targets could be achieved in accordance with quality standards.

4 – Moderately Satisfactory: Based on the output execution ratio between 75% and 89% of the project output targets were

reached or are considered on track to be reached by the end of the project. Corrective actions for off track indicators were

implemented in a timely manner to ensure that the end of project targets could be achieved in accordance with quality standards.

3 – Moderately Unsatisfactory: Based on the output execution ratio between 50% and 74% of the project output targets were

reached or are considered on track to be reached by the end of the project. Corrective actions for off track indicators were not

implemented in a timely manner to ensure that the end of project targets could be achieved.

2 – Unsatisfactory: Based on the output execution ratio between 35% and 49% of the project output targets were reached or are

considered on track to be reached by the end of the project. Corrective actions were not implemented and closely monitored for off

track indicators. Poor performance jeopardized the achievement of one or more outcomes of the project.

1 – Highly Unsatisfactory: Based on the output execution ratio less than 35% of the project output targets were reached or are

considered on track to be reached by the end of the project. Poor performance jeopardized the achievement of most expected

outcomes and the possibility of stopping or suspending the project was considered.

b) Achievement of outcomes

The assessment of outcome is based on the direct and intermediate outcomes stated in the retrospective project logic model.

Typical outcomes of a private sector operation cover the following areas:

i) Economic benefits; the best indicator of a non-financial market project’s contribution to economic growth is its economic

rate of return (ERR) or economic return on invested capital (EROIC). Ideally, the ERR/EROIC considers and quantifies the projects

economic effects on all its economic stakeholders . For Financial Markets projects the economic benefits measures the extent to

which the sub-projects financed with the proceeds of the line of credit or the investee companies in the case of equity funds are

providing a net economic benefit to stakeholders including and beyond the FI’s owners and financiers. Such benefits include, but

are not limited to: Contribution to government revenues resulting from taxes paid by the intermediary, sub-projects or investee

companies; contribution to poverty alleviation, social or gender equality and regional development etc.; delivery of community

services such as entrepreneurship training, educational programs and other community services; employment generated.


6 – Highly Satisfactory: Both: (i) the project has succeeded in reaching targeted groups of sub-borrowers; and (ii) there is direct

evidence (from sub-portfolio data) that sub-borrowers have made strong economic contributions, or indirect evidence (from market

data) that market sectors supported by the project and/or more generally by the financial intermediary are major economic

contributors to society.

5 – Satisfactory: Both: (i) the project has succeeded in reaching targeted groups of sub-borrower; and (ii) there is direct evidence

(from sub-portfolio data) that sub-borrowers are economically viable, or indirect evidence (from market data) that market sectors

supported by the project and/or more generally by the financial intermediary are economically viable and do not rely on economic

distortions to maintain their commercial viability.

4 – Moderately Satisfactory: Both: (i) the project has succeeded in reaching targeted groups of sub-borrower; but (ii) there is no

evidence that sub-borrowers do not rely on economic distortions to maintain their commercial viability.

3 – Moderately Unsatisfactory: (i) the project has largely failed to reach targeted groups of sub-borrower; however, there is

evidence that sub-projects are economically viable.

2 –Unsatisfactory: Either: (i) the project has largely failed to reach targeted groups of sub-borrower; or (ii) there is direct evidence

(from sub-portfolio data) that most sub-borrowers are not economically viable, or indirect evidence (from market data) that market

sectors supported by the project and/or more generally by the financial intermediary are weak economic contributors to society.

1 – Highly Unsatisfactory: LoC: Both: (i) the project has largely failed to reach targeted groups of sub-borrower; and (ii) there is

direct evidence (from sub-portfolio data) that most sub-borrowers are not economically viable, or indirect evidence (from market

data) that market sectors supported by the project and/or more generally by the financial intermediary are weak economic

contributors to society.

ii) Contribution to Private Sector Development; measures the extent to which the project has spread benefits of growth of

productive private enterprise beyond the project company, i.e. on issues such as competition, market expansion, private ownership

& entrepreneurship, development of financial institutions and markets, standards of corporate governance, transfer of technology

and dispersion of skills, and the development of physical infrastructures used by other private parties. The project can have positive

or negative impacts on private sector development and it is necessary to establish that the impacts are attributable to the project.

Indicators include: Upstream and downstream supply linkages to local private businesses; introduction of new technology and

know-how; enhancement of private ownership and entrepreneurship; contribution to improving the environments for private sector

development and an open economy; greater competition and competitiveness; broad demonstration effects in the local economy

and follow-on investments by other investors; domestic capital market development and greater resource allocation efficiency;

improvements in standards for corporate governance and business conduct; and development of physical infrastructures used by

other private parties.


6 – Highly Satisfactory: Considering its size, the project had: (a) substantial positive effects consistent with the Bank’s mandate

objectives of promoting economic development and poverty reduction by furthering the country’s private sector development or the

development of efficient financial / capital markets; and (b) no negative impacts in this respect.

5 – Satisfactory: The project had: (a) demonstrable effects consistent with the Bank’s mandate objectives of promoting economic

development and poverty reduction by furthering private sector development or the development of efficient financial / capital

markets; and (b) a clear preponderance of sustainable positive impacts in this respect.

4 – Moderately Satisfactory: The project had: (a) demonstrable positive effects consistent with the Bank’s mandate objectives of

promoting economic development and poverty reduction by furthering private sector development or the development of efficient

financial / capital markets; However, there is lack of evidence on the sustainability of such effects.

3 – Moderately Unsatisfactory: The project made no discernible contribution, either positive or negative as supported by available

evidence.

2 –Unsatisfactory: The project had mainly negative effects in respect of the Bank’s mandate objectives of promoting economic

development and poverty reduction by furthering private sector development or the development of efficient financial / capital

markets, but these negative effects are not expected to be of long duration or broad applicability.

1 – Highly Unsatisfactory: The project had substantial negative effects in respect of the Bank’s mandate objectives of promoting

economic development and poverty reduction by furthering private sector development or the development of efficient financial /

capital markets, and these impacts are likely to be widespread, of long duration, or both.

Annex 9: Example of project ToC and indicators identified for collection

An example of a project LoC has been taken from the South Africa evaluation. The figure below presents a Theory of Change for

the Medupi power project.

Project Theory of Change – Medupi Power Plant Project

67

Annex 10: Examining lines of credit

First, this Annex will explain what LoCs are and how data is collected for their evaluation at the

strategic level. Second, it explains how a theory of Change (ToC of LoC) can be applied at his

evaluation level.

Definition

LoCs are a method of reaching SMEs and other enterprises that AfDB could either not reach

directly or which would be prohibitively expensive to lend to directly. They involve loans by AfDB to

financial intermediaries (FIs, primarily banks14) that are on-lent by the FIs to SMEs and other

defined categories of beneficiaries. AfDB’s credit risk is with the FI. LoCs in Africa have traditionally

been denominated in US dollars but in recent years there has been a growth in local currency LoCs

in countries where AfDB can raise local currency funding, including both Nigeria and South Africa.

Other objectives for LoCs include:

Strengthening financial intermediaries so that they can better serve SMEs and other

beneficiaries that they are either not supporting or only in a limited way. This may involve the

provision of technical assistance to FIs alongside LoCs.

Improving FI’s access to medium and long term funding that enables them to extend the tenor

of their lending.

Financial and private sector development, including increased access to credit.

Preliminary overview of AfDB LoCs in Nigeria

The table below summarizes the key features of the LoCs in the Nigerian portfolio, divided between

PRA projects and non-PRA projects.

Table 0.6: Overview LoCs

Bank/FI Approve

d

Tenor Focus Sub

Project

Amount

($m)

No of Sub-

loans

Max

($m

)

Min

($m

)

Averag

e

Year

s

Defined

?

$m $m $m

PRAs

Access

Bank 1

2007 General (?) unclear 35 9 12.

7

1.1 3.9

Access

Bank 2

2014 10 SMEs Yes 100 -

GTB 1 2006 7 General No 40 6 20.

4

0.3 6.7

GTB 2 2010 7 General No 100 4 32.

5

7.1 20.1

NEXIM

Bank

2011 -

UBA ELF 2009 5 SMEs,

infrastrucur

e

No 50 4 27.

2

2.8 12.5

Zenith 1 2005 5 Exporters No 70 14 10.

6

1.3 6.1

14 Loans can also be made to micro-finance institutions/banks for on-lending to micro enterprises.

Zenith 2 2006 7 General No 100 11 31 0.5 9.1

Zenith ELF 2009 5 General No 50 12 20.

0

3.0 9.9

Zenith 3 2014 7 SMEs Yes 125

Other Nigerian LoCs

Bank of

Industry

2011

Developme

nt Bank of

Nigeria

2014 400

Fidelity

Bank

2012 75 or

150?

Stanbic

IBTC Bank

2014 7 SMEs and

renewable

energy

100

Africa

Finance

Corporatio

n - regional

2012 10 Infrastructu

re and

industry

By loan

size

200 4 30 25 28.8

Assessing development outcomes

LoCs are provided by the development institution (the AfDB) to a financial intermediary (e.g. a

commercial bank) that subsequently finances subprojects (e.g. SMEs). In practice, it is not always

possible to trace back the origin of funded projects by financial intermediaries and hence of the

development outcomes of the LoCs provide by the AfDB. According to the AfDB this is especially

the case for LOCs that are denominated in foreign currency while the actual lending takes place in

local currency. Difficulties to retrieve information on development outcomes in these cases arise

from money flows and reporting on this by the financial intermediaries.

In Nigeria, LoCs are predominantly provided by the AfDB in US Dollars and on-lending takes place

in US Dollars to designated clients with businesses with foreign exchange accounts. In this way a

clear link between AfDB funding and final beneficiaries can be established, especially if the rates

and tenure periods offered by the AfDB are passed on to clients or if long-term maturities offered by

the AfDB enabled the financial intermediary to have a better ability to lend long-term.

Nevertheless, the assessment of development outcomes of LoCs should take into account the

potential difficulties by differentiating between the levels at which analysis is possible. The ToC for

the LoCs (cf. below) shows the focus of these analyses at different levels. Whereas the analysis of

the level of financial intermediaries focuses mainly on outputs, in particular on the lending capacity

and behavior of financial intermediaries, the analysis at the level of sub borrowers focuses on

intermediate outcomes such as job creation and increased access to finance by SMEs.

69

Figure 0.1 Theory of Change for a line of credit

71

The level at which analysis can take place depends to a significant extent on the information

available and its quality. Below, we have investigated which information is potentially available

through the AfDB.

Project life-cycle documentation

To monitor how its funding is used and to measure development outcomes of the ultimate

beneficiaries of LOCs, the AfDB applies various processes and procedures that are included in the

project life-cycle and summarized in the figure below and further elaborated below the figure.

Figure 0.2 LoC project life-cycle

1. The AfDB Project Appraisal Reports (PARs) for LoCs contain sections on the expected/target

development effects together with Results Based Logical Frameworks (Logframes, in older

LoCs referred to as an Economic and Development Impact Matrix) that set out the results

chains.

2. Loan Agreements (LAs) with banks/FIs set out, inter alia, (i) the purpose of the LoC, (ii) a more

detailed schedule “Objectives and Description” and (iii) a list of the reporting requirements

covering both the bank/FI and the sub-projects. It should be noted that in a sample of LAs for

South African and Nigerian banks/FIs seen by Ecorys, the LAs do not include the detailed

development objectives/goals specified in PAR logframes.

3. Sub-Project Reports that should be provided by banks/FIs on a semi-annual basis (lists of

sub-projects) and annually the “development outcomes” achieved for each project according to

templates in the LAs.

4. Supervision Documents are prepared by the AfDB portfolio division. They are based on the

reports provided by banks/FIs and information gathered during supervision visits to banks/FIs

and a sample of sub-projects. The comprise bank to office reports (BTOs) and the more formal

project supervision report (PSRs).

5. Expanded Supervision Reports (XSRs) are in depth reports that are prepared for mature

projects and assess, inter alia, the development outcomes of the portfolio of sub-projects

financed by LoCs.

Proposed Approach to Assessing LoC Sub-Project Development Outcomes

For each LoC that is studied in-depth (PRA) the sub-project evaluation activities will include, but not

necessarily be limited to:

Prior to Visit to the bank/FI

1. Extraction of logframes and related information in PARs.

2. Comparison of logframes with development outcomes and reporting requirements included in

LAs: (i) the purpose of the LoC, (ii) a more detailed schedule “Objectives and Description” and

(iii) a list of the reporting requirements covering both the bank/FI and the sub-projects.

3. Assessment of the appropriateness of the development indicators contained in both logframes

and LAs. It is of note that IFC’s DOTS systems lists the following essential characteristics of an

adequate indicator known as the “RATE” scale:

Relevant - good indicators must capture the essence of the intended result.

Aggregatable – in order to assess, compare and report development results of all AfDB

supported projects, results from individual projects need to be aggregated.

Time-bound and targeted, i.e. by when and by how much a certain result is expected to be

achieved.

4. Assess whether and how the LoCs score on the development indicators is used in the Financial

Sector Development Strategy. Since all of the LoCs were approved before this Strategy, this

step is primarily to see whether LoCs developed in a similar fashion as foreseen in the Strategy.

5. Verify the most up-to-date, definitive list of sub-projects provided by the bank/FI. This is

important because some LoCs seem to be conflicting with information in the supervision

documents (BTOs and PSRs) with different sub-project lists in the files. In addition, for some

projects the aggregate amount of sub-loans exceeds the amount of the LoC. Also, some sub-

loans may have tenures exceeding those of the LoCs.

6. Reviewing/analysing the most up to date development outcome/impact reports provided by the

bank/FI

7. Review of PSRs and where they have been prepared, XSRs and a comparison with information

provided by banks/FIs as well as other thematic evaluations that might have covered these LoC

operations, for example the SME Evaluation, or the Microfinance evaluation.

8. Preparation of sub-project portfolio specific questions for discussions at meetings with bank/FI.

Visit to the bank/FI

9. Discussion of AfDB logframe. The purpose is to understand how involved the bank/FI was in the

logframe preparation.

10. Preparation of sub-project portfolio specific questions for discussions at meetings with bank/FI.

11. On a sub-project by sub-project basis validation of the development outcomes data reported to

AfDB. In Nigeria, visits will be made to a sample of sub-projects for the bank’s LoCs. During

these visits the tangible, on-the-ground development outcomes will be assessed and compared

with those reported by the banks to AfDB

.

73

Annex 11: Examining the knowledge aspect of the Country Strategy Papers and Programs

Knowledge and policy products (non-lending activities)

Evaluation Questions EQ15 and EQ16, labelled under the category “Knowledge and policy advice”

are part of the broader block of evaluation questions aimed at assessing the Management of the

Bank’s interventions”. Even though currently no significant AfDB project on knowledge and policy

advice is implemented in Nigeria, knowledge sharing constitutes an important part of the CSP

2013-17.

These two questions are also relevant in the light of the recent approval of a Knowledge

Management Strategy (KMS) for the period 2015-2020 since the vision set therein is for the AfDB to

become the premier knowledge institution in Africa in the areas of its mandate.

In particular, the two evaluation questions considered are tightly linked to the second pillar of the

KMS, “Strengthening the quality of the institution’s policy dialogue, advisory services, and

involvement in the development debate”. Indeed, as per the Terms of Reference, the CSPEs will

answer have to assess:

EQ15: To what extent has the Bank actively engaged in and influenced policy dialogue through

relevant advice? and

EQ16: To what extent has the Bank delivered adequate analytical work in support of its


Before describing how each question will be addressed, it is important to stress that themes linked

to knowledge management and policy dialogue have been gradually inserted in the Bank’s country

and regional strategies over the period covered by this evaluation and mostly after 2009.

EQ15. To what extent has the Bank actively engaged in and influenced policy dialogue through

relevant advice?

The level of analysis required by this question is programmatic. The examination will require a

distinction between the Bank’s engagement and the Bank’s influence of the policy dialogue.

Active engagement in policy dialogue can be measured through the participation of the Bank to

relevant initiatives occurred in the period 2004-2014 as well as from the production of studies,

papers or forecasts tailored on Nigeria and the region. The presence and the activities of the Bank’s

Country Office can also be examined through semi-structured (telephone) interviews and

documentary reviews. The number of events organized or hosted by the Bank in the country or

focusing on the country will give an overview of how continuous and deep the engagement of the

Bank has been.

Additionally, the overall level of engagement may be examined by contrasting the Bank’s

knowledge products with similar products of comparable development institutions (e.g. World Bank)

to be able to put it in the appropriate relative terms. A brief benchmarking exercise will allow the

evaluators to identify the niches in which the Bank operates or may operate and will feed in the

assessment of the value added.

The capacity of influencing policy dialogue is less straightforward to assess. A review and

synthesis of the reforms and best practices spearheaded by the Bank will provide the base for the

analysis of the policy evolution in the country concerned, but cannot alone provide a direct link

between the activity of the Bank and the reforms carried out in the country. An assessment of the

role of the other donors and (telephone) interviews with relevant stakeholders will prove necessary

to assess the level of influence the Bank has had.

EQ16. To what extent has the Bank delivered adequate analytical work in support of its


This question will require an analysis spread over all the three layers of analysis at program, project

and portfolio-level. Part of the documentary review, analysis and synthesis produced in addressing

EQ15 can feed in EQ16, although once again the mere presence of analytical work does not say

much about its inherent adequacy.

Comparative analysis with similar documents produced by other development institutions for the

same typology of projects or for co-financed projects can provide insights on the quality of the

Bank’s analytical support to its interventions. The same exercise can be done at the program level,

comparing different CSPs, Country Analysis, Country Sector Reviews with similar documents

produced by other development institutions. Such a comparison will highlight eventual lacks but

also strengths of the Bank’s analytical approach.

Also the timelines and regularity in the production of the documentation can be a useful indicator to

assess the capacity and the willingness of the Bank to produce knowledge content relating in

particular to the CSP’s “Pillars”.

These findings have then to be confirmed by interviews with relevant stakeholders during the field

mission and additional telephone interviews with the producers of the analytical work. The question

will therefore be answered through a double-layered analysis with a first step centered on desk

research and a second one based on interviews.

It is noted that the mode of communicating and distributing analytical work of the Bank may affect

the responses of the users. This aspect will, however, not be examined in the scope of this

evaluation.

75

Annex 12: Interview protocol

The interviews will follow as much as possible the key evaluation questions. Given the restricted

time for each interview the questions will be related to data and information gaps which will be

identified on the basis of the document review. Before each field mission the following procedure

will be followed:

Interview protocol

1) Based on the document review, a list of pre-interview questions for each sector will be prepared and

shared with IDEV;

2) A list of interview questions will be sent to the interviewees in advance (together with a short description of

the purposes of the evaluation) – see below for list of general questions. The main purpose of these

questions is merely to ‘refresh’ interviewee’s memory prior to the interview, so the interviewee can prepare

himself/herself and understands the main focus of the interview. It is noted that only certain evaluation

questions will be asked to the potential interviewees, depending on their knowledge of the projects and

their involvement. In a few cases interviewees may not know about the projects of the Bank, but has

profound knowledge on the South African economy and respective sector which may provide valuable

insights for the evaluation;

3) The interviewee will be assured of the confidential nature of the interview;

4) The main focus of the interview will be the interview questions. Nonetheless it will be useful to reiterate

shortly the objectives of the evaluation and provide opportunities for feedback and discussion;

5) If needed, the context/ background of AfDB activities in South Africa will be discussed as well as the

institutional setting of the sector and the role of the organisation of the interviewee therein;

6) The main focus will be on the main challenges and success factors encountered when implementing AfDB

projects (using the evaluation questions);

7) Discuss the availability of documents, timeline for obtaining these documents; additional sources; and if

relevant potential other interviewees for data collection;

8) Interviewees of financial institutions will be asked their support in conducting the survey among

beneficiaries of Lines of Credit, the questions included in the survey and to discuss the approach of

undertaking the survey (i.e. ideally by requesting the banks to send the survey request to the

beneficiaries);

9) The interviewee will be informed about the follow-up steps of the evaluation

10) For each interview minutes will be prepared and shared with the interviewee for check on factual errors

and omissions;

11) The data and information in the interview minutes will be coded to enable structured analysis;

12) If possible some follow-up questions and clarifications by telephone may take place, depending on the

availability and preparedness of the interviewed persons.

77

Annex 13: List of key informants for semi-structured interviews

On the basis of the information available to us we developed an initial stakeholder overview. In the

tables below we provide an overview of the key stakeholders of the Bank’s public and private sector

projects disaggregated by sector.

Public sector projects stakeholder mapping

Sector Federal Level State Level Province/Local body Level Donors Private Sector

Agriculture

Office of the Chief Economic Adviser to the President

Federal Ministry of Agriculture and Rural Development (FMARD):

12 departments and one unit, namely: Department of Agriculture; Department of

Rural Development; Department of Planning, Research and Statistics; Department of

Agricultural Science; Department of Livestock and Pest Control Services;

Department of Cooperatives Department of Storage and Strategic Grain Reserve;

Department of Fertilizer; Department of Land Resources; Department of Fisheries;

Department of Finance; Department of Administration and Supplies; and the Project

Coordinating Unit.

The Federal Department of Planning, Research and Statistics (FDPRS)

Agricultural and Rural Management Training Institute (ARMTI)

The Nigerian Institute of Social and Economic Research (NISER)

State Ministry of Agriculture (SMOA), Agriculture Development, Livestock and Veterinary

Services, Engineering Department, Fisheries Department and Forestry Department. Some

states in addition, have the Cooperative Development Department and the Produce Grading,

Inspection and Pest Control Department.

State Agricultural Transformation Implementation Committee (SATIC)

Department of Integrated Rural Development, which promotes rural infrastructure, and

Community Based Organizations (CBO)

Public institutions supporting agriculture and rural development at state level include:

(i) Technical Service Support, provided by the Ministry of Agriculture (MoA), Agricultural

Development Authorities (ADA), Agricultural Development Programmes (ADP), Ministry of Water

Resources and Rural Water Development, Ministry of Commerce, Industry and Cooperatives,

Ministry of Works, Housing and Transport);

(ii) Socio- Economic Support, provided by Ministry of Health, Ministry of Women Affairs and

Social Development, Ministry of Education;

(iii) Sector Planning, Monitoring and Evaluation, provided by Ministry of Local Government or

Chieftaincy Affairs, State Planning Commission or Ministry of Economic Development or

Planning; and

(iv) Commercial/Financial Services

Local Government Councils

(LGC)

Local Government Area

(LGA)

Environment NDCC (Niger Delta Development Commission)

Shell Petroleum Development

Company

Niger Delta Environment Survey

Finance Federal Ministry of Finance

Multi-SectorFederal Ministry of Finance

Sectoral Ministries WHO-AFRO

PowerFederal Ministry of Finance

Federal Ministry of Power (FMP)

Nigeria Bulk Electircity Trading

SocialFederal Ministry of Finance

Federal Ministry of Education: Science and Technology Department

Safe School Initiative (SSI) Secretariat

WHO-AFRO

Transport Cross River State Ministry of Works

Water Sup SanFederal Ministry of Agricultural and Water Resources (FMAWR)


Yobe Basin Trust Fund of Nigeria

Rural Water Supply and Sanitation Agency (RWASSA) is the executing Agency in Yobe State

Osun Rural Water Supply and Environmental Sanitation Agency (RWESA) is the executing

agency in Osun State

Water Corporation of Oyo State

Taraba State Water Supply Agency

Kaduna State Water Board

State Mnistry of Water Resources

State Ministry of Work

State Water Board

Local Government Councils

(LGC)

Local Government Area

(LGA)

Private sector projects stakeholder mapping

Sector Federal Level State Level Province/Local body Level Donors Private Sector

Agriculture Cross River State Forestry Commission Local Council Group ECOWAS Bank for Investment and Development (USD

20 million), the Islamic Corporation for the

Development of the Private Sector (USD 30 million)

and a set of local commercial banks (USD 40 million). In

addition to the sponsor’s 23% contribution to equity,

additional equity is expected to be provided by DEG,

Finfund, PROPARCO, Emerging Capital Partners and

one individual investor.m

Negris Group Ltd

Accugas Ltd

Finance Central Bank of Nigeria

Federal Ministry of Finance

Federal Government of Nigeria

Asset Managent Company of Nigeria (AMCON)

Nigeria Deposit and Insurance Corporation

National Drug Law Enforcement Agency (NDLEA)

Economic and Financial Crime Commission (EFCC)

NIgerian Financial Intelligence Unit (NFIU)

Small and Medium Enterprises Development Agency of

Nigeria (SMEDAN)

Belgian Investment Company for Developing Countries

(BIO)

European Bank for Reconstruction and Development

(EBRD)

Netherlands Development Finance Company (FMO)

Kreditanstalt für Wiederaufbau (KfW)

International Financial Cooperation (IFC)

Fund for the African Private Sector Assistanct (FAPA)

Development finance institutions (“DFIs”): Bank of

Agriculture (“BOA”), the Bank of Industry (“BOI”), the

Federal Mortgage Bank (“FMB”), the Nigerian Export-

Import Bank (“NEXIM”) and the Infrastructure Bank

(“IB”)

Banks: United Bank of Africa, First Bank of Nigeria,

Access Bank, Zenith Bank, ETI, Union Bank of Nigeria.

Power River State Government Host Communities (NGO) Belgian Investment Company for Developing Countries

(BIO)

Netherlands Development Finance Company (FMO)

International Financial Cooperation (IFC)

Kreditanstalt für Wiederaufbau (KfW)

DFID (through the CDC)

Gas Aggrecgation Company Nigeria Ltd

Transport Federal Government of Nigeria


Security and Exchange Commission of Nigeria

Nigerian Corporate Affairs Commission (through Larue

Ltd)

Lagos State Government Asset and Resource Management Company

Africa Infrastructure Investment Fund (Macquarie

Bank)

HITECH

81

Annex 14: Document mapping

Based on the document overview send by the Bank we developed a simplified overview of available project documents. The overview of non-project documents

has been adopted from the Bank’s overview.

Mapping of available project documents

Public sector documents

Sector Project Name Project Reference Funding Status PREPARATION DOCUMENTS IMPLEMENTATION DOCUMENTS COMPLETION DOCUMENTS

Agriculture MIC GRANT ASSET MAPPING OF ECONOMIC OPPORTUNITIES IN NIGERIA P-NG-A00-007 MIC GRANT APVD P

Agriculture SUPPORT TO AGRIC. & RURAL INSTITUTIONS P-NG-AA0-026 ADF COMP P

Agriculture NATIONAL PROGRAMME FOR FOOD SECURITY P-NG-AA0-027 ADF COMP P

Agriculture AGRICULTURAL TRANSFORMATION AGENDA SUPPO P-NG-AAB-003 ADF APVD P

Agriculture SUPPORT COMBATING AVIAN INFLUENZA 2006 P-NG-AAE-002 SRF FUND COMP P

Agriculture INVASIVE AQUATIC WEEDS - NIGERIA P-Z1-AA0-084 ADF COMP P

Environment NIGER DELTA SOCIAL AND ENVIRONMENT STUDY P-NG-C00-001 ADF CLSD P

Finance TRADE MISPRICING THE HIDDEN DRAINAGE P-NG-HB0-006 MIC GRANT OnGo P

Multi-Sector TRANSPORT SECTOR AND ECONOMIC GOVERNANCE P-NG-K00-005 ADB LOAN OnGo P P

Multi-Sector CAPACITY DEVT PROGRAM FOR NASS ON MICTAF P-NG-KF0-002 MIC GRANT COMP P No report yet No report yet: due by October 2015

Power ECONOMIC AND POWER SECTOR REFORM PROGRAM P-NG-FA0-002 ADF LOAN OnGo P P Aide Memoire OK

Power PRG I N SUPPORT OF POWER SECTOR P-NG-FA0-006 ADF LOAN APVD P

Social SKILLS TRAINING AND VOCATIONAL EDUCATION P-NG-IA0-001 ADF OnGo P P Aide Memoire OK

Social EMERG. ASSIST. TO CH ***** P-NG-IA0-003 SRF FUND APVD P Disborsement effectiveness

Social AIDE MULTINAT URG ERADIC POLIOMYELITE P-NG-IBE-001 SRF FUND COMP P

Social EMERGENCY ASSIST EBOLA P-NG-IBE-002 SRF FUND APVD P P

Transport RURAL ACCESS & MOBILITY PROJECT P-NG-DB0-005 ADF LOAN OnGo P Aide Memoire OK

Water Sup SanKOMADUGU-YOBE BASIN STRATEGIC DEVT PLAN P-NG-EAZ-002 AWF Grant APVD P

Water Sup SanRURAL WATER & SAN SUB-PROG (YOBE & OSUN) P-NG-E00-002 RWSSI Loan OnGo P To be sentMission Aide memoires (PCR

supervision) to be sent

Water Sup SanURBAN WATER & SAN IMPROVEMENT PROJECT P-NG-E00-004 ADF LOAN OnGo P PMission Aide memoires (PCR


Water Sup SanZARIA WATER EXPANSION & SAN PROJ, P-NG-E00-005 ADF LOAN OnGo P PMission Aide memoires (PCR


Water Sup SanURBAN WATER REFORM & PORT HARCOURT WSSP P-NG-E00-007 ADB LOAN APVD P

Private sector documents

Sector Project Name Project Reference Funding Status PREPARATION DOCUMENTSIMPLEMENTATION DOCUMENTSCOMPLETION DOCUMENTS

Agriculture OKIPP P-NG-AAG-002 ADB APVD P P

Finance FIDELITY BANK LOC P-NG-HA0-005 ADB/ LOC OnGo P P

Finance BANK OF INDUSTRY - BO P-NG-HAA-002 ADB LOAN APVD P

Finance NIGERIAN EXPORT IMPORT BANK - NEXIM P-NG-HAA-003 ADB LOAN OnGo P

Finance STANBIC IBTC BANK PLC P-NG-HAB-016 ADB LOAN APVD P P

Finance STANBIC IBTC BANK PLC P-NG-HAB-016 CLEAN TECH FUND APVD P P

Finance ZENITH BANK PLC - LOC III P-NG-HAB-017 ADB/LOC APVD P P

Finance ACCESS BANK NIGERIA LOC II P-NG-HAB-019 ADB/LOC APVD P P

Finance FRB SUBSIDIARY IN NIGERIA P-NG-HAB-022 ADB/LOC OnGo

Finance LAPO MICROFINANCE BANK LTD. P-NG-HB0-005 ADB LOAN APVD P P

Finance ABN MICROFINANCE TA GRANT P-NG-HA0-003 Priv Asst Fund/GRANT OnGo P P

Finance LINE OF CREDIT TO ZENITH BANK PLC P-NG-HAB-004 ADB/LOC CLSD P P P

Finance HELIOS TOWERS ADB/LOC OnGo

Finance LOC TO GUARANTEE TRUST BANK P-NG-HAB-005 ADB/LOC CLSD P P P

Finance ZENITH BANK LOC II P-NG-HAB-006 ADB/LOC OnGo P P P

Finance LINE OF CREDIT TO ACCESS BANK PLC P-NG-HAB-007 ADB/LOC OnGo

Finance LINE OF CREDIT II TO GUARANTY TUST BANK P-NG-HAB-009 ADB/LOC OnGo P P

Finance ZENITH EMERGENCY LIQUIDITY FACILITY P-NG-HAB-013 ADB/LOC OnGo P P P

Finance UBA EMERGENCY LIQUIDITY FACILITY P-NG-HAB-014 ADB/LOC CLSD P P P

Finance UBA TRADE FINANCE INITIATIVE P-NG-HAB-015 ADB/LOC CLSD P

Finance DEVELOPMENT BANK OF NIGERIA(DBN) P-NG-HAA-004 EQUITY APVD P P

Finance AFRICA FINANCE CORPORATION(AFC) ADB/LOC OnGo P P

Power INDORAMA FERTILIZER P-NG-FD0-002 ADB LOAN OnGo P P

Transport LEKKI TOLL ROAD PROJECT P-NG-DB0-008 ADB LOAN(PPP) OnGo P

83

Mapping of available non-project documents

AfDB Documents

DOCUMENT TITLE AVAILABILITY

THE ONEBANK GROUP RESULT MEASUREMENT FRAMEWORK 2013-2016 x

INTEGRATED SAFEGUARD SYSTEM POLICY AND OPERATIONAL SAFEGUARDS x

BANK GROUPGENDER POLICY x

BANK GROUP TRANSPORT SECTOR POLICY x

BANK GROUP TRADE FINANCE PROGRAM BUSINESS PLAN x

BANK GROUP OPERATIONS MANUAL x

BANK GROUP GENDER STRATEGY 2014-2018 x

BANK GROUP FINANCIAL SECTOR DEVELOPMENT & STRATEGY x

BANK GROUP GENDER STRATEGY 2014-2018 x

BANK GROUP FINANCIAL SECTOR DEVELOPMENT & STRATEGY x

BANK GROUP ENERGY POLICY x

BANK GROUP AGRICULTURE SECTOR STRATEGY 2010-2014 x

BANK GROUP AGRICULTURE AND RURAL DEVELOPMENT POLICY x

ADOA ADDITIONALITY AND DEVELOPMENT OUTCOMES FRAMEWORKD 2009 x

BANK GROUP REGIONAL INTEGRATION STRATEGY x

IDEV OPERATION MANUAL x

BUSINESS MANUAL - PRIVATE SECTOR OPERATION x

IDEV EVALUATION - REVIEW OF THE BANK ESW 2005 -2010 x

IDEV EVALUATION - TRANSPORT SECTOR x

IDEV EVALUATION - MICROFINANCE + Database + Portfolio Review x

IDEV EVALUATION - NON SOVEREIGN OPERATIONS+ Database+ Portfolio review x

IDEV EVALUATION - QUALITY AT ENTRY x

IDEV EVALUATION - PROJECT RESULT REPORTING x

IDEV EVALUATION - SME x

DECENTRALIZATION x

85

Other sources

Bertelsmann Stiftung. 2014 ‘BTI 2014 — Nigeria Country Report’. Gütersloh: Bertelsmann Stiftung, 2014 EY. 2014 ‘Africa by numbers – A focus on Nigeria Special report issued for: World Economic Forum on Africa 2014’ London, 2014 EYGM Limited KPMG. 2015 ‘Country Report – Nigeria 2014 Quarter 4’ London, KPMG LLP 2015

KPMG. 2014 ‘Nigeria Country Profile – KPMG Africa Region 2012/2013’ London, KPMG LLP 2014 IFAD. 2013 ‘Federal Republic of Nigeria – COSOP MID-TERM REVIEW 2013’ Report No: 3554-NG, Rome IFAD. 2010 ‘Federal Republic of Nigeria – Country strategic opportunities programme’, EB/2010/99/R.11, Rome IMF. 2015,’ IMF press release - IMF Executive Board Concludes 2014 Article IV Consultation with Nigeria’, International Monetary Fund, Press Release No. 15/91, March 4, 2015, Washington, D.C. IMF. 2013, ‘Nigeria: Financial Sector Stability Assessment’, International Monetary Fund, Country

Report No. 13/142, Washington, D.C. IMF, World Bank. 2013. ‘Financial Sector Assessment Program : Nigeria - Crisis Management and Crisis Preparedness Frameworks. World Bank, Washington, DC’. © World Bank. https://openknowledge.worldbank.org/handle/10986/15964 License: CC BY 3.0 IMF. 2013, ‘Nigeria: Publication of Financial Sector Assessment Program Documentation–– Technical Note of Banking Cross-Border Issues’, International Monetary Fund, Country Report No.

13/142, Washington, D.C. IMF. 2013, ‘Nigeria: Publication of Financial Sector Assessment Program Documentation–– Technical Note on Stress Testing’, International Monetary Fund, Country Report No. 13/142,

Washington, D.C. IMF. 2013, ‘Nigeria: Publication of Financial Sector Assessment Program Documentation–– Technical Note on Crisis Management and Crisis Preparedness Frameworks’, International

Monetary Fund, Country Report No. 13/142, Washington, D.C. IMF. 2013, ‘Publication of Financial Sector Assessment Program Documentation–– Technical Note on Strengthening Monetary and Liquidity Management’, International Monetary Fund, Country

Report No. 13/142, Washington, D.C. IMF. 2013, ‘Publication of Financial Sector Assessment Program Documentation–– Detailed Assessment of Implementation of IOSCO Objectives and Principles of Securities Regulation, International Monetary Fund, Country Report No. 13/142, Washington, D.C. World Bank. 2015. ‘Nigeria Economic Report 2014’, July 2, 2015, Washington, DC: World Bank Group. World Bank. 2014. Doing Business 2015: Going Beyond Efficiency. Washington, DC: World Bank Group. DOI: 10.1596/978-1-4648-0351-2. License: Creative Commons Attribution CC BY 3.0 IGO World Bank. 2014. Doing Business in Nigeria 2014: Understanding Regulations for Small and Medium-Size Enterprises. Washington, DC: World Bank Group World Bank. 2011,’Country Partnership Strategy Progress Report For The Federal Republic Of Nigeria For The Period Fy10 – Fy13’, Washington, DC: World Bank Group

Annex 15: Possible structure of main technical report and sector reports

Besides the main technical report, two separate sector reports will be produced each covering:

Infrastructure projects in the energy sector and the transport sector

Lines of Credit projects

Structure of the main technical report

Executive summary

1. Introduction: Description of purpose and objectives; limitations and data; structure of report

2. Country context: Description of social and economic developments

3. Composition of portfolio: trends, types, size, volume, focus

4. Analysis of Portfolio Data

3.1 Portfolio performance: Implementation performance, Sector performance, monitoring

and reporting;

3.2 Private sector operations: by sector: design issues, implementation issues, strategic

outcomes;

3.3 Public sector operations: by sector: design issues, implementation issues, strategic

outcomes;

3.4 Multinational operations: contribution to CSP outcome;

3.5 Technical Assistance/knowledge sharing: performance and contribution to CSP

outcome;

3.6 Comparison of overall performance of Bank Portfolio to Country Portfolio (trends over

the period).

5. Program Outcome and Performance Summary:

4.1 Strategic overview: contribution of portfolio to CSP results; added value of the Bank,

Borrowers Performance, Partnerships;

4.2 Relevance: summary (refer to template details);

4.3 Efficiency and Implementation (refer to template details);

4.4 Potential for results.

6. Key lessons and going forward (this should include issues such as)

5.1 Strategic positioning of the Bank;

5.2 Engagement with the country;

5.3 Institutional arrangement: decentralization, resources, re-structuring.

87

Structure of the sector reports

Executive summary

1. Introduction: Description of purpose and objectives; limitations and data; structure of report

2. Context: Description of sector, key stakeholders, developments in the sector, sector policy in

South Africa

3. Composition of project portfolio in the respective sector: trends, types, size, volume, focus

4. Assessment

a. Relevance

b. Effectiveness

c. Efficiency

d. Sustainability

e. Added value of Bank interventions in the sector

5. Key lessons

89

Annex 16: Timeline

ID Task Name Start Finishjan 2016aug 2015 sep 2015 dec 2015okt 2015 nov 2015

29-1116-8 15-11 22-11 6-128-114-10 18-10 27-1230-8 13-9 25-1020-9 11-10 1-112-8 13-1227-9 20-1223-89-8 10-13-16-9

1 30-9-20155-8-2015Inception phase

2 7-8-20155-8-2015Scoping mission Abidjan

3 4-9-201510-8-2015Desk research and inception report drafting

4 18-9-20157-9-2015Scoping mission Nigeria

30-9-201521-9-2015Finalisation Inception report

6

7

5

13-11-20151-10-2015Data collection and analysis

26-10-20151-10-2015Phase 1: Desk-based review of documents, portfolio review, identification of data gaps

17-11-20152-11-2015Phase 2: Data collection mission Nigeria8

9

10

11

12

13

13-1-201618-11-2015Technical report drafting and finalization

23-12-201518-11-2015Draft report to IDEV for comment

23-12-201518-11-2015PRA templates finalized

14-1-201623-12-2015Draft report to reference group and reviewers for comment

29-1-201615-1-2016Finalisation report

14 4-3-201615-1-2016Summary report, workshop and dissemination

17-1

feb 2016

24-1 31-1

mrt 2016

7-2 14-2 21-2 28-2 6-3