document of the world bank · 2018. 7. 6. · document of the world bank for official use only...

Document of The World Bank

FOR OFFICIAL USE ONLY

Report No: ICR00002349

IMPLEMENTATION COMPLETION AND RESULTS REPORT

TF Number: 010024

ON A

GRANT

IN THE AMOUNT OF $114.12 MILLION

TO THE

Islamic Republic of Afghanistan

FOR A

PUBLIC FINANCIAL MANAGEMENT REFORM II ( P120427 )

June 28, 2018

Governance Global Practice South Asia Region

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

CURRENCY EQUIVALENTS

(Exchange Rate Effective June 1, 2018)

Currency Unit = Afghani (AFN)

71.20 AFN = US$1

FISCAL YEAR

July 1 ‐ June 30

Regional Vice President: Hartwig Schafer

Country Director: Shubham Chaudhuri

Senior Global Practice Director: Deborah L. Wetzel

Practice Manager: Ismaila B. Ceesay

Task Team Leader(s): Syed Waseem Abbas Kazmi, Adenike Sherifat Oyeyiola

ICR TTL: Bernard James Haven

ICR Main Contributor: Sati Achath

ABBREVIATIONS AND ACRONYMS

ADB Asian Development Bank KPI Key Performance Indicator ACCA Association of Chartered Certified

Accountants LTO Large Taxpayer Office

ACSI Afghanistan Civil Service Institute M&E Monitoring and Evaluation ADB Asian Development Bank IAD Internal Audit Department AF Additional Financing IDA International Development

Association AFMIS Afghanistan Financial Management

Information System ISR Implementation Status

Report ANDS Afghanistan National Development

Strategy MOF Ministry of Finance

ARDS Afghanistan Reconstruction and Development services

PFC Procurement Facilitation Consultant

ARTF Afghanistan Reconstruction Trust Fund PEFA Public Expenditure and Financial Accountability

ATP Audit Training Program PFM Public Financial Management CAS Country Assistance Strategy PMIS Procurement Management

Information System CAT Certified Accounting Technician PPU Procurement Policy Unit DFID Department for International

Development RIMU Reform Implementation and

Management Unit DG Director General SAO Supreme Audit Office EA Environmental Assessment SBD Standard Bidding Document FM Financial Management SIGTAS Standard Integrated

Government Tax Administration System

FMC Financial Management Consultant SDU Special Disbursement Unit GDP Gross Domestic Product STO Small Taxpayer Office HR Human Resources TA Technical Assistance NPA National Procurement Authority TOR Terms of Reference NPC National Procurement Commission USAID United States Agency for

International Development NPI National Procurement Institute UNDP United Nations Development

Program NPV Net Present Value VAT Value Added Tax

TABLE OF CONTENTS

DATA SHEET ............................................................................................................................ 1 ABSTRACT ................................................................................................................................ 5 I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES ........................................................ 6

A. CONTEXT AT APPRAISAL ........................................................................................................... 6 B. SIGNIFICANT CHANGES DURING IMPLEMENTATION ................................................................. 9

II. OUTCOME ...................................................................................................................... 11 A. RELEVANCE OF PDOs .............................................................................................................. 11 B. ACHIEVEMENT OF PDOs (EFFICACY) ........................................................................................ 12 C. EFFICIENCY ............................................................................................................................. 15 D. JUSTIFICATION OF OVERALL OUTCOME RATING ..................................................................... 18 E. OTHER OUTCOMES AND IMPACTS .......................................................................................... 18

III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME ................................ 21 A. KEY FACTORS DURING PREPARATION..................................................................................... 21 B. KEY FACTORS DURING IMPLEMENTATION .............................................................................. 23

IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME .. 24 A. QUALITY OF MONITORING AND EVALUATION (M&E) ............................................................. 24 B. ENVIRONMENTAL, SOCIAL, AND FIDUCIARY COMPLIANCE ..................................................... 24 C. BANK PERFORMANCE ............................................................................................................. 25 D. RISK TO DEVELOPMENT OUTCOME ........................................................................................ 26

V. LESSONS AND RECOMMENDATIONS .............................................................................. 26 ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS ............................................................ 28 ANNEX 2. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION ......................... 44 ANNEX 3. PROJECT COST BY COMPONENT............................................................................. 46 ANNEX 4. EFFICIENCY ANALYSIS ............................................................................................ 47 ANNEX 5. BORROWER’S ICR .................................................................................................. 50 ANNEX 6. BORROWER, CO‐FINANCIER AND OTHER PARTNER/STAKEHOLDER COMMENTS ... 66 ANNEX 7. SUPPORTING DOCUMENTS .................................................................................... 67 ANNEX 8. SPLIT RATING CALCULATION .................................................................................. 68

The World Bank Public Financial Management Reform II (P120427)

Page 1 of 68

DATA SHEET

BASIC INFORMATION Product Information Project ID Project Name

P120427 Public Financial Management Reform II

Country Financing Instrument

Afghanistan Investment Project Financing

Original EA Category Revised EA Category

Not Required (C) Not Required (C)

Related Projects Relationship Project Approval Product Line Additional Financing P150632‐Second Public

Financial Management Reform Project (PFMRP II) ‐ Additional Financing

10‐Feb‐2016 Recipient Executed Activities

Organizations

Borrower Implementing Agency

Islamic Republic of Afghanistan The Islamic Republic of Afghanistan

Project Development Objective (PDO) Original PDO The proposed PDO is to strengthen public financial management through effective procurement, treasury and audit structures and systems in line with sound financial management standards of monitoring, reporting and control. Revised PDO To further strengthen the efficiency and effectiveness of Afghanistan’s procurement, treasury, and audit systems.


Page 2 of 68

FINANCING Original Amount (US$) Revised Amount (US$) Actual Disbursed (US$) World Bank Financing TF‐10024 73,000,000 114,070,775 109,209,939

Total 73,000,000 114,070,775 109,209,939

Non‐World Bank Financing Borrower 0 0 0

Total 0 0 0

Total Project Cost 73,000,000 114,070,775 109,209,939

KEY DATES

Approval Effectiveness MTR Review Original Closing Actual Closing 14‐Jun‐2011 09‐Aug‐2011 01‐Nov‐2013 31‐Dec‐2014 31‐Dec‐2017 RESTRUCTURING AND/OR ADDITIONAL FINANCING

Date(s) Amount Disbursed (US$M) Key Revisions 01‐Sep‐2014 45.47 19‐Dec‐2015 66.20 Change in Loan Closing Date(s) 27‐Jan‐2016 68.72 Additional Financing 19‐Apr‐2017 90.49 Change in Loan Closing Date(s) KEY RATINGS

Outcome Bank Performance M&E Quality

Moderately Satisfactory Satisfactory Substantial


Page 3 of 68

RATINGS OF PROJECT PERFORMANCE IN ISRs

No. Date ISR Archived DO Rating IP Rating Actual

Disbursements (US$M)

01 20‐Sep‐2011 Satisfactory Satisfactory 0

02 18‐Apr‐2012 Satisfactory Satisfactory 7.00

03 24‐Dec‐2012 Moderately Satisfactory Moderately Satisfactory 15.89

04 24‐Jun‐2013 Moderately Satisfactory Moderately Satisfactory 27.58

05 17‐Dec‐2013 Moderately Satisfactory Moderately Satisfactory 33.73


07 02‐Feb‐2015 Moderately Satisfactory Moderately Satisfactory 49.81


09 08‐Feb‐2016 Moderately Satisfactory Moderately Satisfactory 68.72


11 16‐Nov‐2016 Satisfactory Satisfactory 77.90

12 15‐May‐2017 Satisfactory Satisfactory 91.88

13 18‐Apr‐2018 Satisfactory Satisfactory 106.21 SECTORS AND THEMES

Sectors Major Sector/Sector (%) Financial Sector 100

Public Administration ‐ Financial Sector 100 Themes Major Theme/ Theme (Level 2)/ Theme (Level 3) (%)


Page 4 of 68

Public Sector Management 101 Public Finance Management 48

Public Expenditure Management 48

Public Administration 53 Transparency, Accountability and Good Governance 53

ADM STAFF

Role At Approval At ICR

Regional Vice President: Isabel M. Guerrero Ethel Sennhauser

Country Director: Nicholas J. Krafft Shubham Chaudhuri

Senior Global Practice Director: Onno Ruhl Deborah L. Wetzel

Practice Manager: Jennifer K. Thomson Ismaila B. Ceesay

Task Team Leader(s): Paul Edwin Sisk Syed Waseem Abbas Kazmi, Adenike Sherifat Oyeyiola

ICR Contributing Author:


Page 5 of 68

Abstract

The Second Public Financial Management Reform Project (PFMR‐II, 2011‐2017) sought to enhance public financial management by strengthening Afghanistan’s procurement, treasury, and audit systems. A revenue mobilization component was added to the project in 2016. The project was completed in 6.5 years at a total cost of $106.2 million, which included a three‐year extension and $41.13 million additional financing. The project achieved measurable results across the targeted areas in a challenging context. Procurement – A National Procurement Commission and National Procurement Authority were established during the project. This was a two‐dimensional reform model which addressed political and technical dimensions of procurement reform and has proven beneficial in the fight against corruption. Treasury ‐ The implementation of an upgraded Afghanistan Financial Management Information System (AFMIS) was completed, providing real‐time access to financial data allowing organizations to manage budgets and financial transactions. The AFMIS is built around a centralized payment‐processing model, in which all budget executions are managed at the Central Treasury in Kabul, and the ‘Mustofiats’ in the provinces. Audit ‐ The project provided employees with training in Internal Audit through the Professional Certification Program (PCP), raising employee skills and abilities within the Ministry of Finance (MOF) and the line ministries. External audit coverage increased, including coverage of on‐budget donor financing. Revenue ‐ The Afghanistan Revenue Department adopted a new organizational structure. Risk‐based compliance, audit, legal and HR reforms were either fully completed or are at an advanced stage. The decline in domestic revenue collection was reversed and revenues have recovered. As a result of the outcomes achieved under these project components, donor confidence in procurement, treasury and audit systems grew. At the time of project design, approximately $1.8 billion of donor financing was off budget. By the end of the project, 65% of this baseline off‐budget ODA was shifted on‐budget. The lessons learned in this project will inform similar operations in the sector in Afghanistan and in other countries. For example: (i) reducing aid dependency requires a long‐term transition plan; (ii) the shift from projects to programs should be planned from the outset; and (iii) PFM is a key ingredient for a transition to on‐budget support, and (iv) in a fragile context, activities should be primarily driven by problem solving, rather than international best practices. While project objectives have largely been met, the sustainability of some activities remains a challenge. For example, while the project successfully transitioned many tasks from advisors to government employees, many of those employees are paid a significant premium through donor funding. Even though the project has met and is likely to meet all its targets, the overall outcome is rated as ‘Moderately Satisfactory’, because of separate outcome ratings based on the share of actual disbursements made in the periods before and after the restructuring.


Page 6 of 68

I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES A. CONTEXT AT APPRAISAL

Context

1. Building an effective state was at the heart of the reconstruction effort in Afghanistan at the time of project design. After decades of war and civil strife, Afghanistan remained fragile and security was a serious obstacle to the implementation of reconstruction programs. Strengthening public financial management to accelerate aid utilization, deliver faster and better services, and to improve transparency and accountability of public expenditure was a priority for the government from the outset of reconstruction. 2. The Government had made progress in establishing its Public Financial Management (PFM) system by the time of project design. The legal framework underpinning financial management was established with the adoption of the PFM law in 2005. Over the period that followed, international firms were engaged to support the Treasury Department, the former Afghanistan Reconstruction and Development Services (ARDS), and the Control and Audit office (CAO) in the areas of treasury operations, procurement, and audit.

3. While this progress was encouraging, Public Expenditure and Financial Accountability (PEFA) Assessments in 2006 and 2007 highlighted important PFM weaknesses. Critical actions were proposed in the following areas: (i) development of PFM capacity in the line ministries (outside MOF); and (ii) strengthening of internal and external audit capacity, including formal academic/professional training; and revision of legal foundation based on international standards.

4. The macroeconomic context provided a significant challenge during project implementation. Economic growth averaged 9.4 percent per year from 2003 to 2012 and key social and infrastructure indicators, including school enrollment, life expectancy, and access to water improved. However, economic growth fell sharply to 1.5 percent in 2014 following an international military transition and a fiscal crisis unfolded with declining revenues leading to depleted cash reserves and accumulation of arrears. Rationale for the Project 5. During the reconstruction period, support for the PFM was coordinated between multiple donors. The Bank led provision of technical assistance in Treasury, Procurement, and Audit. USAID, US Treasury, DFID, UNDP, and ADB provided assistance for budget, revenue management, and decentralization. The government and donors requested that the project be financed under ARTF with IDA providing supervision and implementation support as it represented a natural continuation and deepening of reforms supported by prior IDA grant operations. After another donor concluded financing for support for revenue management, this component was added to the project. 6. Building core PFM capacity in the government was a key component of the Bank’s interim support strategy for Afghanistan. The Bank had gained valuable insight on PFM issues in Afghanistan through the IDA‐funded technical assistance projects and through extensive analytical work on PFM. The Bank was well‐positioned to assist the government in implementing the PFM Roadmap that called for reforms to ensure efficient, transparent and accountable use of public resources, increase aid absorption capacity and secure on‐budget donor aid, and improve development outcomes.


Page 7 of 68

Theory of Change (Results Chain)

7. The original project development objective was “to strengthen the efficiency and effectiveness of Afghanistan’s procurement, treasury, and audit systems to ensure transparent use of public resources.” The project was divided into four components to meet this overall objective: procurement, treasury, audit, and reform management. This fourth component supported improvement in monitoring and management of technical assistance and human resources in the MOF. Addressing these areas of reform together was seen as a complementary approach that would strengthen PFM throughout the budget cycle within the context of support provided by other donors. Consequently, a Results Framework was prepared which articulated project’s pathway from planned interventions to the intended outcomes. The logic of this results chain was valid and prudential. See diagram below:

Theory of Change

PDO: to strengthen public financial management through effective procurement, treasury and audit structures and systems Strengthen Procurement Efficiency

Activities Outputs PDO/Outcomes o Consultancy Services for PFC to

assist ARDS o Capacity Building in line

ministries and Provinces o Institutional Development for

legal framework

o Procurement Law passed by the Parliament

o Procurement rules of procedures approved, including debarment, appeal and review, domestic and foreign preference policy, and zero corruption tolerance policies

o Implementation of Procurement Reform Framework to achieve more economical public procurement and greater transparency

o Ministries fully capable of independent procurement processing and facilitating transparent utilization of donor resources for reconstruction and development

o Procurement capacity built in Line Ministries and Provinces o Procurement regulation and oversight strengthened; o Procurement responsibility transferred to all Line Ministries

Underlying Assumptions: These outcomes will result in more economical public procurement and greater transparency Strengthen Treasury

Activities Outputs PDO/Outcomes o Treasury Operations Support and

System Development Activities o Treasury HR Capacity Building

Activities o Professional Accounting

Organizations Development

o FMIS Maintenance and Enhancement in all line ministries and provinces

o Production of Qatia (statutory govt. FM statements)

o Moving to web based freelance Version 7

o Improved Treasury procedures to achieve full benefits of enhanced AFMIS

Underlying Assumptions: These outcomes will result in High level PFM performance with improved capacity Strengthen Audit System

Activities Outputs PDO/Outcomes o Capacity Building Activities o Institutional Development o IT Support

o Professional Certifications: o 50 Internal Auditors o 20 Fraud Examiners o 10 IS Auditors o 20 Project Mgmt. Professionals

o Improved "Internal” and “External” Audit coverage

Underlying Assumptions: These outcomes will result in sound standards of monitoring, reporting and control

Project Development Objectives (PDOs)

8. The original project development objective was “to strengthen public financial management through effective procurement, treasury and audit structures and systems” in line with sound financial management standards of monitoring, reporting and control.


Page 8 of 68

Key Expected Outcomes and Outcome Indicators Indicator Name Baseline End of Project Target 1. Donors shift 20% of their off‐budget official development assistance to Afghanistan to on‐budget

0% (at $1.8 billion)

20% cumulative increase

2. PEFA ratings for internal audit improve to B Rating of C: coverage under 50%, not done to recognized standard and no action by management

75% cumulative coverage of expenditure and 75 % revenue done to recognized standard and responses given to all audits done by MoF and action taken on most

3. PEFA ratings for external audit improve to B Rating of C: coverage under 50%, not done to a recognized standard and no evidence of follow‐up

75% cumulative coverage of expenditure done to an acceptable standard and evidence presented of follow‐up

4. At least 50% of procurement under the budget is done by line ministries conducting stand‐alone procurement

Two ministries

Procurement under these should equal 50% of procurement

5. Increased number of functions carried out by regular ministry staff that were previously carried out by contracted staff

0

20 separate functions in MOF

Components

9. Component 1: Procurement Reform (Original estimated cost: $21.47 million; actual cumulative cost $32.46 million): The component provided support to build procurement management capacity throughout the government and institutional capacity in the public and private sectors. There were three sub‐components:

Sub‐component 1.1 ‐ Procurement Facilitation: The activities included consultancy services of a Procurement Facilitation Consultant (PFC) to assist the ARDS in: (i) processing civilian on‐budget public procurement; (ii) completing procurement regulatory and institutional frameworks; and (iii) capacity building through knowledge transfer to staff in the Line Ministries. Sub‐component 1.2 ‐ Capacity Building: The activities included (i) support to the former Procurement Policy Unit (PPU) and Afghanistan Civil Service Institute (ACSI) to lead capacity building and institution building activities; and (ii) deployment of three groups of procurement specialists in the Line Ministries to: (a) liaise with the central authorities on training and support the application of training in operations; (b) act as procurement controllers to contribute to the preparation of procurement plans, report on progress and coordinate input into the procurement management information system (PMIS); and (c) as procurement facilitators, assist on complex procurement and accelerate budget execution. Sub‐component 1.3 ‐ Institutional Development: Support was provided to complete the legal framework, preparing a collection of all relevant legal documents and for operation of the Appeal & Review Committee, and training on dispute resolution. Support was also provided for designing an effective and strategic communication and procurement disclosure policy to raise awareness, and to enhance and operate PMIS.

10. Component 2: Financial Management Reform (Original estimated cost: $23.88 million; actual cumulative cost $39.84 million): The component supported high level PFM performance; and building staff and institutional PFM capacity throughout government through training, systems development and process renewal. There were four sub‐components:


Page 9 of 68

Sub‐component 2.1 ‐ Treasury Operations and Systems Development: Through the services of a Financial Management Consultant, support was provided to develop treasury functions and strengthen internal controls and to provide counterpart training. Specific activities included establishment of a disaster recovery plan and equipping the Afghanistan Financial Management Information System (AFMIS) IT back‐up facility. Sub‐component 2.2 ‐ Human Resource Capacity Development: Support was provided to develop job‐specific certifiable finance and general skills of treasury and finance staff of MOF and the line ministries. Support was also provided to develop a Treasury Communications Plan and to implement support reforms. Sub‐component 2.3 ‐ Professional Accountants Organization Development: The activities included support to establish the legal framework and structures for development and regulation of the accounting and auditing profession and for training of professional accountants in Afghanistan. Sub‐component 2.4 ‐ Line Ministry PFM Assessments: This sub‐component financed a detailed review of internal control and PFM performance of seven line ministries.

11. Component 3: Audit Reform and Performance (Original estimated cost: $24.70 million; actual cumulative cost: $34.27 million). The component improved public sector governance through nine sub‐components, grouped under the following Internal and External Audits:

Internal Audit: Through three sub‐components, support was provided for institutional reforms in the internal audit and capacity building activities to improve management oversight. Support was also provided to develop information systems and training in automated auditing tools. External Audit: Through six sub‐components, support was provided for capacity building and carrying out of independent reviews of on‐budget donor‐funded operations, including all operations under the budget of nine line ministries over the project period. Additional support was provided for the Public Accounts Committee and the purchase of the office and IT equipment for the new Chief Audit Officer.

12. Component 4: Reform Management (Original estimated cost: $2.95 million; actual cumulative cost: $3.72 million). The component supported activities to improve monitoring and management of technical assistance and human resources in MOF, through: (i) strengthening monitoring and evaluation (M&E); (ii) strengthening MOF Human Resources (HR) Management Department; (iii) project management; and (iv) other institutional strengthening and capacity building to strengthen M&E, HR management development, and project executive management functions in the MOF in addition to other related technical assistance and analytical work.

B. SIGNIFICANT CHANGES DURING IMPLEMENTATION

Revised PDO, Results Framework, and PDO Indicators

13. Revised Project Development Objective: The PDO was revised to make it more specific and measurable by linking it to the PFM Strategy as follows: “To further strengthen the efficiency and effectiveness of Afghanistan’s procurement, treasury, and audit systems.” 14. Revised Results Framework: During implementation, the Results Framework was revised to reflect organizational changes in the government. Most notably, the National Procurement Authority (NPA) was


Page 10 of 68

included as the successor organization to ARDS/PPU. Changes were also introduced to enhance results monitoring to reflect citizen engagement and gender issues. 15. Revised PDO Indicators: Revisions were introduced to audit reform indicators to eliminate any reference to the Public Expenditure and Financial Accountability methodology. The new wording of the indicators focused more on the quality and scope of audit coverage that was in the spirit of the original PDO indicators. A new, more relevant indicator related to the Procurement Reform component, was introduced to replace the one that had become irrelevant and difficult to measure. Another indicator was introduced on the theme of accountability and transparency in publishing procurement decisions. The indicator pertaining to the increased number of functions carried out by regular ministry staff was further refined to focus on the development and implementation of plans to transit posts to programs such as Capacity Building for Results. The Results Framework and monitoring indicators were revised and updated as below:

Indicator Name Baseline End Target (2017) Donors shift their off‐budget official development assistance to Afghanistan to on‐budget.

0%

20%

Reduction in number of days taken by the National Procurement Commission to dispose of cases presented to it.

10 days

7 days

Establish a database of, and publicly disclose, National Procurement Commission decisions.

2 days

1 day

The Supreme Audit Office commences development of a mechanism for citizen participation in audit.

No Citizen participation in audit

Preparation for citizen participation in audit during FY16 leading to development of mechanism for citizen participation in audits and issuance of policy governing citizen participation in public oversight of public funds by June 30, 2017.

Increased number of functions carried out by regular ministry staff that were previously carried out by contracted staff.

Plans currently work in progress

Plan in place for FY17 to modify Tashkeel to include functions currently undertaken by national consultants. FY16 Plan implemented.

Improved coverage and quality of internal audit 49 50 Improved coverage and quality of external audit 49 75

16. Revised Components: The original components were not revised during implementation. However, a new component related to ‘Revenue Mobilization’ was added to the project design. DFID‐financed support for revenue concluded in 2015, and follow‐on assistance was requested by Government and donors in this area following a significant decline in revenues. This component complemented revenue mobilization targets under the ARTF incentive program and the IMF program. The activities supported were related to the maintenance of computerized tax systems (SIGTAS), the implementation of risk‐based compliance, preparation for value added tax (VAT), and establishing a new organizational structure. The new Revenue Mobilization component ($3.85 million) was added under the Additional Financing when another donor concluded its program in this area.

Other Changes:

17. Additional Financing (AF): Additional Financing of $41.13 million1/ was provided to finance costs associated with scaling up of successful activities and to sustain achieved results under the project. The government requested the Bank to develop a follow‐on project to support the implementation of key aspects of the new PFM Strategy. The AF acted as a bridge to avoid any gap in the Bank’s PFM support and losing momentum

1 Effective on January 27, 2017


Page 11 of 68

on key results that had been achieved under the project. The AF was also utilized to finance the new component (Revenue Mobilization), which was at the top of the government’s priorities, requiring around $3.85 million. 18. Extension of Grant Closing Dates: The project was extended three times: (i) from the original December 31, 2014 closing to December 31, 2015, to provide the government more time to complete implementation of ongoing activities; (ii) from December 31, 2015 to June 30, 2017 to give additional time to complete ongoing activities, and to prepare AF for the project; and (iii) from June 30, 2017 to December 31, 2017, as some activities were still unfinished and required additional time to be completed while a follow‐on project was prepared. 19. Revised Cost Estimate: With the inclusion of AF, the original costs were revised as below:

Comparison of Original Amount vs. Revised Costs by Component (US$ Million)

Components

Original Amount

% of Total AF

% of Total with AF

Revised Total

Overall Percentag

e Financing

1: Procurement Reform 21.47 29.4% 11.00 26.8% 32.47 28.4% 2: Financial Management Reform 23.88 32.7% 15.96 38.8% 39.84 35.0% 3: Audit Reform and Performance Comprising: Internal Audit External Audit

24.70

09.60 15.10

33.8%

13.2% 20.6%

9.57

4.07 5.50

23.2%

9.8% 13.4%

34.27

13.67 20.60

30.0%

12.0% 18.0%

4: Reform Management 2.95 4.1% 0.77 1.9% 3.72 3.3% 5: Revenue Mobilization ‐ ‐ 3.84 9.3% 3.84 3.3% Total 73.00 100% 41.13 100% 114.13 100%

Rationale for Changes and their Implication on the Original Theory of Change

20. The provision of AF did not have any major implication on the theory of change. In fact, it provided additional resources that contributed to more effective linkages in the result chains. It also addressed a shift in complementary financing from other donors, as revenue was moved from DFID support into PFMR‐II.

II. OUTCOME A. RELEVANCE OF PDOs

21. The relevance of the project development objective is rated as High. The PDO was relevant to the Bank’s goals and donor‐supported PFM reform objective. It reflected government’s key priorities as given in the PFM Roadmap presented at the Kabul Donor Conference in June 2010. The PDO remains highly relevant to the Country Partnership Framework (2017‐20). The PDO was also in line with the government’s Strategic Plan, prepared by MOF (as custodian of PFM), that proposed reforms to produce greater efficiency in managing public resources and maintaining standards for oversight and governance. The strategy called for a participatory process and regular support to MOF decision‐makers to ensure alignment of actions, improved management of technical assistance (TA), and regular contact and communications with partner ministries and agencies. However, two gaps were identified in the coverage of PDO indicators. Treasury operations were not sufficiently covered by the indicators and the addition of support for revenue mobilization with additional financing did not result in the addition of any indicators.


Page 12 of 68

B. ACHIEVEMENT OF PDOs (EFFICACY) Assessment of Achievement of Each Objective/Outcome

22. Since the original PDO and some of its associated outcome targets were formally revised through restructuring, separate outcome ratings have been weighted in proportion to the share of actual disbursements made in the periods before and after the restructuring. Based on the two ratings, as shown Annex 7, the overall achievement of the project is rated Moderately Satisfactory. The project’s achievements in terms of PDO indicators before and after restructuring are given below:

PDO Indicators Original Project After Restructuring/AF Target Actual Target Actual

1. Increased number of functions carried out by regular ministry staff that was previously carried out by contracted staff

20 separate functions in MOF

Plans work in progress

Plan in place for FY17 to modify Tashkeel to include functions that are currently undertaken by national consultants. Equivalent FY16 Plan implemented.

Plans currently work in progress. On target: Treasury has succeeded in transitioning some key functions from advisors to civil servants; NPA now carries out key managerial functions through civil servants and number of Tashkeel staff now exceed the number of TAs.

2. Donors shift their off‐budget official development assistance to Afghanistan to on‐budget

20% cumulative increase

62% cumulative increase 20% cumulative increase 65% cumulative increase

3. The Supreme Audit Office commences development of a mechanism for citizen participation in audit (New Indicator)

N/A N/A

Preparation for citizen participation in audit during FY16 leading to development of mechanism for citizen participation in audits and issuance of policy governing citizen participation in public oversight of public funds by June 30, 2017.

SAO has developed a mechanism for citizen participation in audit / issued a policy governing citizen participation in public oversight. SAO has also developed the annual work plan for public participation in audit process, and is organizing a one‐day seminar for relevant stakeholders

4. Establish a database of, and publicly disclose, National Procurement Commission (NPC) decisions (New Indicator).

N/A N/A One One

5. Reduction in number of days taken by the National Procurement Commission to dispose of cases presented to it (New Indicator).

N/A N/A 10 Days to dispose of cases 7 Days

to dispose of cases

6. Improved coverage and quality of internal audit (Revised)

75% cumulative coverage of expenditure and 75% revenue done

50 Audits

MOF IAD audits 50% of budgeted expenditure & (for entities which collect) revenue 25%,

56% coverage of expenditure and 42% coverage of revenue completed.


Page 13 of 68

to recognized standard and responses given to all audits done by MOF and action taken on most.

within its mandate, as per recognized standards such as International Standards for the Professional Practice of Internal Auditing, issued by the Institute of Internal Auditors. Unit of Measure = %

7. Improved coverage and quality of external audit (Revised)

75% cumulative coverage of expenditure done to an acceptable standard and evidence presented of follow‐up

74% 75% 71%

8. % of procurement done by LM using stand‐alone procurement (Deleted)

50 48 N/A N/A

9. Number of functions carried out by regular ministry staff (Deleted)

20 17 N/A N/A

23. Project outcomes before Restructuring: The original PDO had three components. “To strengthen the efficiency and effectiveness of Afghanistan’s (1) procurement, (2) treasury, and (3) audit systems to ensure transparent use of public resources.” The overall theory of change was that these reforms would have more impact when implemented together, in concert with budget and revenue support provided by other donors. 24. Since effectiveness, the project performed well and disbursed nearly $68.72 million (94%) out of the total original commitment of $73 million. Satisfactory progress was made on three of the five PDO Indicators. Overall, the indicator on donor contributions to the budget was satisfactory (62% of baseline off‐budget ODA was shifted on‐budget, against 20% targeted). 25. Outcome 1 – Procurement – Substantial: Standalone procurement capacity was satisfactory. Although the NPC had 28 days to decide on a case in law, it was taking on average one week for a case to go NPC against seven days targeted. NPA was carrying out key managerial functions through civil servants, meeting its planned target in 2017. The NPA began posting all procurement on a single website, improving transparency.

26. Outcome 2 – Treasury – Substantial: PDO indicators did not cover the implementation of the upgraded treasury systems. However, Treasury was successful in transitioning some key functions from advisors to civil servants, albeit civil servants paid a special premium (the National Technical Assistance (NTA) pay scale) leading to gains in cost efficiency. Transaction time was reduced as ministries and agencies have been able to directly access Treasury systems.

27. Outcome 3 – Audit – Modest: Only moderate progress was made on the internal and external audits2/ and on the development of the professional accountancy organization. The Professional Accountants Organization 2 Mainly due to the change in the legal framework on internal audit and a delay in mobilizing technical assistance for the external audits of the line ministries


Page 14 of 68

Development sub‐component faced considerable challenges in establishing a legal framework and structure for development and regulation of the profession and training of professional accountants. The draft Accountancy Law remained pending with the Ministry of Justice for approval as it stayed with the President’s office for extensive discussions. Furthermore, the change of mandate of the Internal Audit Department adversely affected the prospect of achievement of the PDO Indicator for internal audit.3 Regarding the external audit subcomponent, while good progress was observed, there were delays in mobilizing TA for the external audits of the Line Ministries.

28. The project results framework and component activities were amended through restructuring and for Additional Financing to accommodate the performance issues discussed above. Project performance was rated ‘Moderately Satisfactory’ for PDO and Implementation Progress. 29. Project outcomes after Restructuring: After restructuring, PDO had three components. “To further strengthen the efficiency and effectiveness of Afghanistan’s (1) procurement, (2) treasury, and (3) audit systems.” The overall theory of change remained the same ‐‐ these reforms would have more impact when implemented together.

30. Progress after restructuring improved and most targets have been met. Overall, the indicator on shifting off‐budget official development assistance on‐budget met its target ‐ 65% achieved against 65% targeted. Additionally, the increased number of functions carried out by regular ministry staff previously carried out by contracted staff (consultants) met its target. Many functions of the MOF previously carried out by consultants are now performed by civil servants. However, many positions continue to require special pay arrangements, and consultants continue to advise on key functions in the ministry. It is not clear from the PDO indicator what percentage of key functions of the MoF have been transitioned. This issue is discussed further in Lessons Learned.

31. Outcome 1 – Procurement – Substantial: The targets for procurement were met. Establish a database of, and publicly disclose, National Procurement Commission (NPC) decisions. This target is fully met. The NPC decisions are reported immediately after each session, against one day targeted. The NPA has made substantial progress in improving transparency through the publication of tenders, awards, standard bidding documents, debarments, decisions, and laws on a comprehensive website. Decision forms are available on the NPA website within the same day. Reduction in number of days taken by the NPC to dispose of cases presented to it. This indicator has exceeded its target. On average now, seven days is taken against seven days targeted. By law, NPC has 28 days to decide on a case unless the package is submitted incomplete. In FY2017, the NPC held 51 weekly meetings, approving 446 projects with a total value of $2.2 billion.

32. Outcome 2 – Treasury – Substantial: In Treasury, the presence of civil servants has increased for all functions including FM activities, allotment and commitment, cash management, debt management as well as AFMIS operation. Hiring of local Tashkeel staff is ongoing. As Treasury Department capacity increased, reliance on external firms was reduced. No external firms have been contracted for treasury operations since 2014. However, many civil servants with technical capacity in these areas continue to be paid special premiums. This is discussed further in Lessons Learned. Development partners have received the AFMIS dataset for seven fiscal years, providing for a detailed review of transactions (over 400,000 lines of data) and increased confidence in on‐budget financing. This data has been used to improve third‐party monitoring such as employment verification to improve public perceptions of the use of public resources. The web‐based system allows for real‐time access to live data 3 As a consequence of the amendment of Article 61, responsibility for internal audit remains a subject of discussion between MoF and SAO.


Page 15 of 68

eliminating previous delays in relaying information to ministries and agencies. 33. Outcome 3 – Audit – Substantial: The Supreme Audit Office (SAO) commences development of a mechanism for citizen participation in audit. This target is fully met. The SAO has issued a policy in this regard. Improved coverage and quality of “internal audit”. This target is fully met. The progress was initially slower than expected as limited audits were performed in the provinces due to factors outside of the control of the Bank; however, the improved expenditure coverage is now 56% and improved revenue coverage is 42%. Improved coverage and quality of “external audit”. This target is not fully met. The improved audit coverage now stands at 71% against 75% targeted. The SAO is presently working to increase the audit coverage.

34. Rating Before Project Restructuring (August 9, 2011 ‐ January 1, 2016): The Efficacy before restructuring is rated as Modest. While there was satisfactory progress on three of the five PDO Indicators, only moderate progress was achieved on internal and external audit aspects due to a change in the legal framework on internal audit procedures and delay in mobilizing TA for external audits of the Line Ministries. As a result, the project team continued to rate the project’s performance as ‘Moderately Satisfactory’, for both PDO and Implementation Progress.

35. Rating After Project Restructuring (January 2, 2016 – December 31, 2017): The Efficacy after restructuring is rated Substantial. Despite political challenges, the project responded well by keeping its focus on developing PFM systems in the Line Ministries and providing intensive training in procurement, financial management and audit, across various government departments. Targets were mostly fully met, and in few areas, exceeded goals.

Justification of Overall Efficacy Rating

36. The overall efficacy is rated as Substantial. This is based on individual ratings of Relevance of PDO (High), and Achievement of PDO (Substantial).

C. EFFICIENCY Assessment of Efficiency and Rating

37. The Efficiency is rated as Substantial. The implementation of the project is considered efficient. An economic analysis was not performed at appraisal and there is no original projection for the Net Present Value (NPV) or Economic Rate of Return (ERR). This limits the extent to which a comprehensive analysis can be performed at the ICR stage. Credible economic benefit calculation leading to NPV and external rate of return is challenging with broad‐based PFM reforms, as it is difficult to claim attribution to all downstream benefits. Nevertheless, an estimate of project costs and benefits is estimated in Annex 4. The main benefits of the project include efficiencies gained from shifting ODA on‐budget and savings from efficiency of public procurement. The shift in ODA depended on implementation of all project components. The NPV of the project at a 5 percent discount rate is $1,021 million and at a 12 percent discount rate is $832 million. The EIRR is calculated at 236.5 percent. As discussed in the annex, this approach provides an indicative analysis only, as the outcomes supported in this project cannot be solely attributed to project activities. 38. The project also yielded non‐financial benefits. Line Ministries are increasingly able to execute budget and deliver services more efficiently and Government is increasingly empowered to manage and direct development partner resources through the budget cycle. Procurement is more transparent, treasury management is more


Page 16 of 68

efficient and is accessible within line ministries, and audit coverage and quality has increased. Data access in AFMIS is now real‐time, with no delays after data entry. These reforms have supported increased use of national systems by donors with a higher proportion of ODA flowing through the budget. Capacity building of civil servants to operate PFM systems has contributed to the sustainability of these systems. Below are the benefits realized under various components:

Procurement Efficiency 39. Structural Efficiency: The procurement structure in the MoF and the Ministry of Economy was scattered prior to initiation of reform. The of functions under the NPA, as one single regulatory body, resulted in an efficient structure. Also, NPA acting as the Secretariat of the NPC has further strengthened the efficiency. 40. Efficiency in Accountability (Cultural Efficiency): Because of NPC oversight and the follow up activities by NPA, there is more responsiveness and accountability within procurement entities and the Line Ministries.

41. System/Process Efficiency: By law, the service standard for NPC to decide on award of a project submission of documents is 28 days; however, track records show that award decisions are made more efficiently and in less than 28 days. Moreover, Bid Evaluations are now done by procurement entities, which in the past were an open‐ended process, which generally took more than two years. After the reform, the service standard is set to be 30 days plus one‐time possibility of extension.

42. Savings, Safeguarding and Contribution to Revenue: The establishment of the NPA has resulted in increased confidence in national procurement systems. The use of unified Rules of Procedures required vendors who participate in the bidding to settle their tax arrears as a condition of participation. This has positively affected revenues.

43. NPA is also in early stage of initiating the assessment of Afghanistan’s readiness for, and implementation of, Electronic Government Procurement (eGP) as part of the government’s vision and efforts toward e‐governance.

Treasury Efficiency

44. The upgraded AFMIS is providing real‐time access to financial data which allows organizations to keep track of their budgets and financial transactions. The web‐based AFMIS system was implemented in all 34 provinces and all line ministries in Kabul. The system was built and deployed around a centralized payment‐processing model, where all budget executions are managed at the Central Treasury in Kabul and the ‘Mustofiats’ in the provinces. 45. Based on the continued efforts of Treasury General Directorate, local banks have been able to integrate their systems with communication companies, which provide their customers with access to their salary accounts by using mobile banking facility. This is particularly helpful in areas that do not have access to the bank branches.

46. The Pilot project of mobile banking started in Kapisa province in north of Kabul in July 2015, and the first payment of salary to Afghan Local Police through mobile took place in August 2015. The new payment mechanism is greatly appreciated by Afghan Local Police leadership and the funding donor, and the Combined Security


Page 17 of 68

Transition Command Afghanistan (CSTCA), as the process helps remove ghost employees and strictly limits the payment of salary to a second party by capturing the employees fingerprint into the system.

Audit Reform and Performance Efficiency

47. The project supported the adoption of good practices in internal audit through the adoption of the Institute of Internal Auditors (IIA) standards. Manuals, guidelines and similar materials have been developed, resulting in a common approach to internal audit amongst audit professionals employed at the Ministries and the Independent Directorates. Skills were improved through the Professional Certification Program (PCP) provided to around 100 staff, including cadres of Internal Auditors of the MOF, all the Line Ministries and the State Administered Directorates.4/

48. This program has resulted in immediate cost savings for government. In 2017, 962 staff were trained against 500 envisaged and 175 internal audits were performed per standards as compared with 34 envisaged. As a result, out of $11.2 million collectable revenue, around $0.9 million was already recovered and deposited into the government’s account. Similarly, 52 civil servants were terminated due to fraudulent activities.

Revenue Mobilization Efficiency 49. The revenue component was introduced to the project in 2016. Following a significant reduction during the military transition, revenues recovered during project implementation, increasing 18% in 2016. In percent of GDP, the revenues increased from 8.7% in 2014 to over 11% in 20165/, which may be attributed to the revenue administration reforms, simplified returns filing requirements (5 steps to 3), functional appeals department, as well as some policy reforms undertaken by ARD (increase in Business Receipts Tax from 2% to 4%, and introduction of a 10% fee on telecommunications in 2015). The decline and recovery of revenues is presented in the chart below.

4 (i) Certified Internal Auditors (50); (ii) Certified Fraud Examiner (20); (iii) Certified Information Systems Auditor (10); and (iv) Project Management Professionals (20). 5 Afghanistan’s fiscal year runs from approximately December 21 to December 20. https://www.usip.org/publications/2017/02/revenue‐growth‐afghanistan‐continues‐strong‐future‐uncertain


Page 18 of 68

50. Notable achievements under this component are:

A clear strategy and roadmap developed for its reorganization to sustain reforms;

SIGTAS implementation in all major tax offices in Kabul and the provinces has helped progressively increase transaction volume as core functions are automated;

Fast‐track filing system completed to facilitate large taxpayers;

Risk‐based audit implemented; Local capacity built in ICT, audit, legal and HR

reforms; Taxpayers’ awareness and education program

conducted.

Other Factors 51. There were only two Task Team Leaders (TTLs) for the implementation period of the project. Both were based in Kabul, which enabled them to maintain regular contact with the government and stakeholders. This factor was critical for providing the client with timely and consistent technical and operational support and capacity building, as well as facilitating project implementation significantly.

D. JUSTIFICATION OF OVERALL OUTCOME RATING

52. The overall outcome is rated as Moderately Satisfactory. This is based on: (i) High rating for Relevance of PDO (based on relevance at closing); (ii) Modest rating for Efficacy before restructuring; (iii) Substantial rating for Efficacy after restructuring; (iv); Substantial rating for Efficiency (based on efficiency at closing). Using the ICR Guidelines Annexes H and I, the overall outcome rating is (v) Moderately Unsatisfactory before restructuring; (v) Satisfactory after restructuring. Annex 8 of this report presents the split rating calculation.

Split

Ratings Relevance of

Development Objectives Efficacy

Assessment Efficiency Assessment

Overall Outcome Rating

Before Restructuring High

Modest Substantial

Moderately Unsatisfactory

After Restructuring Substantial Satisfactory

Overall Outcome Rating Moderately Satisfactory

E. OTHER OUTCOMES AND IMPACTS

Gender

53. The original project did not specifically target women, however during restructuring; changes were introduced in the Results Framework to include monitoring of gender involvement. Under the Audit Reform and Performance component, internship programs targeting women were offered to 60 female graduates from the MOF, four Line Ministries, and various Directorates. Under the Procurement Reform component, tailored courses

0%1%2%3%4%5%6%7%8%9%

10%11%12%

Domestic Revenues as Percentage of GDP

Direct taxes Sales tax (BRT)

Customs duty Non‐tax revenues


Page 19 of 68

were offered to 3,024 personnel in the National Procurement Institute (NPI), which included 329 women (10%). Similarly, under Financial Management Reform component, 527 Treasury employees were offered Masters and Bachelor level courses of which 77 were female (15%).

Institutional Strengthening

Procurement

54. The NPA is implementing reforms to professionalize and digitize public procurement to address corruption and enhancing accountability. These efforts are divided into the restructuring of procuring entities, the recruitment of procurement professionals and the professionalization of procurement cadres through training. 55. Restructuring procurement entities ‐ Under the leadership of the Presidency, the NPA and the National Procurement Institute (NPI) were established to lead these reforms. The NPA is the main body for strengthening of procurement systems in Afghanistan. NPA staff, many of which are consultants, are being moved into the civil service. The NPI works with 56 procuring entities to focus on capacity building activities. 56. Recruiting procurement professionals ‐ To professionalize the procurement profession, a mass recruitment of civil service staff was completed in 43 procuring entities of the government agencies. Out of 36,787 applicants, 8,175 completed written exams and 697 procurement professionals were hired. 57. Professionalization ‐ To build the capacity of the procurement profession, an intensive capacity building program has been undertaken. The NPA staff was trained on World Bank procurement procedures. Key staff have been selected to complete master’s degree in sustainable procurement contract management, e‐procurement, and supply chain management in various countries including India, Italy, and Afghanistan. The NPI has been established to sustainably continue capacity building in Afghanistan by providing institutional support and Training, developing a training curriculum and training trainers. Training of Trainers has been completed on training techniques, procurement skills enhancement, and selected procurement topics. The NPI plans to conduct regular and special procurement training for the newly hired procurement personnel through the mass recruitment.

Treasury

58. AFMIS has been upgraded and a web‐based payroll system was developed. Study visits were organized to Timor‐Leste and Kosovo, countries which have upgraded Free‐balance system to the web‐based version. Subsequently, MoF implemented the upgraded system in Afghanistan. Additionally, an electronic payroll system was implemented. All 34 provinces now have unified payroll data for Treasury, including all the Line Ministries, except the Ministry of Education. 59. Treasury coverage extended to Kabul Municipality. The extension of AFMIS to Kabul Municipality has improved efficiency and compliance. Payroll calculations have been automated. It now takes only a day to process all salaries as compared to ten days previously. Legal changes to rates of pay have now been implemented. Workers who were not present were removed from the payroll, resulting in cost savings.

60. A capacity building program has been introduced for employees. Treasury employees can pursue their Bachelor’s and Master’s degrees in Accounting, Finance, Business Management and Computer Science after working hours, with reimbursement of tuition costs. The scheme aims to increase self–reliance, confidence and


Page 20 of 68

skills of government staff and decrease the organization’s dependency on outside consultants. Over the past five years, more than 600 employees have enrolled in the program and 260 employees have successfully completed their course at universities across Afghanistan. As a result of this program, employees have been able to take on progressively higher responsibilities. Under civil service rules, employees at Grade 5 and above require a degree in relevant field. As a result of this program, capacity within the Treasury Department is improving and core functions have been carried out by staff (rather than consulting firms) since 2014.

Employees moving to higher positions within treasury/ MoF 40 Employees gaining higher positions in other ministries 10 Employees retaining positions post‐graduation course 210

61. Technical Support to Develop Afghanistan Accountancy law: The Accounting Law was developed and approved with technical support from the project. The Afghanistan CPA was formed under the Professional Accountants Organizations Development component. 62. Scholarship programs for Association of Chartered Certified Accountants (ACCA) students: MOU were signed with the Kabul University to establish an Accounting Department for curriculum development. A long‐term contract with the University of Malaysia has also been signed for capacity development of lecturers. So far, more than 1,000 people have begun the process of qualifying (around a third of them women) for the scholarship program, which is encouraging in a country where less than fifth of the women over the age of 15 are in the workforce.

Audit

63. Internal Audit ‐ Establishment of Internal Audit Committee: The Internal Audit Department has established an Audit Committee in the MOF. Technical Audit Manuals, Audit Tool Kits, step‐by‐step guidelines on performing audits, including audit planning, programming, risk assessment, and preparing reports were provided. Audit report template based on 5‐Cs (criteria, conditions, cause, consequence, and corrective actions) were developed. Digitized audit reports were also introduced.

64. External Audit ‐ Institutional Development for SAO: Auditing standards with baseline for all audits were provided; professionalized audit unit were set‐up in each department – grant audit, specific audit, local budget audit, municipal audit. Twelve departments now have their own audit units; Audit follow‐up unit were established to follow‐up on compliances issues; Citizen engagement in audit; Information dissemination, citizen consultation, reporting fraud and investigation units have been established.

65. Support to Public Accounts Committee (PAC): The project supported establishment of PAC, which meets daily. Once SAO submits report, PAC must approve it before the next budget. 66. The SAO has completed the entire audit of World Bank projects for the fiscal year 2015, and submitted the report to World Bank and MOF. The SAO has completed the audit of Grants project and issued nine audit reports for various line ministries.

Revenue

67. IT System Operational Nationally: The implementation of Standard Integrated Government Tax Administration System (SIGTAS) has been completed in all major tax offices in Kabul and the provinces. Together with the installation of the new system and procurement of the required IT equipment (laptops, desktops etc.), the field staff were provided on‐the‐job training in effectively using the system. This has helped progressively


Page 21 of 68

increase transaction volume as core functions are automated. As a result of this system, the reliability of taxpayer information has increased and audit and enforcement measures have been reinforced while simplifying procedures for taxpayers. This has underpinned an increase in revenues while supporting an improved business climate. 68. Risk‐based Audit Introduced in the Revenue Department: During the project, operating guidelines/manuals were developed; risk criteria were established; and compliance improvement plan and risk register were finalized. A compliance improvement committee was established to provide guidance in case selection for audits. Performance Indicators have been developed to monitor performance over 3‐year period. As a result of this reform, audit resources can be targeted on high‐risk taxpayers. This is supporting increased revenue while supporting an improved business climate.

Other

69. The PFMR‐II Reform Implementation Management Unit (RIMU) supported initialization of good practices: Since the outset of the project, the RIMU has supported training and policy development to institutionalize good practices developed under the project. At the provincial level, human resource policies and procedures were developed and training on these policies was completed. Policies included code of conduct; merit based appointment; leave management; guidelines relatives’ employment; occupational health and safety; job rotation; overseas training management; performance appraisal; employment incentives; internship policy; policy for contracted staff employment; and policy for scholarship committee. Induction Manuals were prepared in three languages – Dari, Pashto, and English. A “Questions Bank Book” was also prepared which was comprised of 2,700 questions on various subject matter, along with “answer‐key” from the MOF.

Poverty Reduction and Shared Prosperity

70. It is difficult to attribute the effect of the project on poverty reduction as it is an enabling function for service delivery. The project provided tools to the government to better plan, execute and monitor the budget. Improvements in internal and external audits provide better assurance on the use of budget and address waste and abuse. The SAO has also started performance audits that evaluate economy, efficiency and effectiveness of government spending. In the long run, the project would benefit the poor with improved public finance performance and investment in social sectors.

Other Unintended Outcomes and Impacts

71. The number of approved learning providers and institutions for the courses offered by ACCA has increased from one to seven during project period, which is mainly attributed to the scholarships provided by the project to ACCA students, which has led to more demand for this course. This increase in the supply of training is likely to support the nascent accounting profession in Afghanistan.

III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME A. KEY FACTORS DURING PREPARATION

72. In the context of weak public sector governance, the project preparation team recognized the challenges of implementation and prepared the project design with sound background analysis and comprehensive assessment of the government’s commitment. Lessons learned during the PFM operations in other jurisdictions as well as from the ongoing operations, at the time, were also considered to assess potential risks and mitigation measures.


Page 22 of 68

Lessons Learned during project preparation

73. The ICR of the Public Administration Capacity Building Project (IDA‐H1440) highlighted several issues that persisted under the then ongoing PFMR project that were considered in the design of this project. The issues and lessons learned are as follows:

(i) Inadequate Human Resources Capacity in PFM: Engaging international firms and local consultants to perform basic PFM functions was a default strategy. Sustainability was difficult as local specialized advisors moved from the ministries to work on projects outside the budget or in the private sector while civil servants could not take over related duties. (ii) Durability of PFM Reforms: The Government, while generally dedicated to making improvements, has taken unilateral action to reverse or restrain progress in the legal framework. Donor coordination around the ARTF Incentive Program and consultations on the PFM Roadmap have led to firm commitments to maintain and improve the legal framework. (iii) Limited Automation of PFM Processes: The prevailing systems to support service delivery were largely manual, and lacked some basic functions. The project supported the development of the related service delivery administrative processes. (iv) Corruption Risk: The country environment carries high risk to perceived corruption. Strengthening of procurement and audit of Public Accounts Committee should reduce this risk. (v) Internal Audit: Management oversight is constrained by lack of effectively operating internal audit function, except in the MOF. Building internal audit capacity cannot be done in the short term so implementing the approach provided in the Public Expenditure and Finance Law which entrusts MOF to establish internal audit throughout the government would be the most effective and expeditious route to establish the function across government, while at the same time supporting the development of internal audit in the line ministries, where feasible. (vi) External Audit: The external accountability of the executive must be strengthened. Many achievements in PFM are discounted because there is little independent oversight or assurance that the reports and decisions of the executive deserve credibility, particularly given the perception of rising corruption. A new legal framework, training for members of a public accounts committee and training and technical support to the CAO are needed. (vii) Accounting Profession: There is a limited accounting and audit profession in Afghanistan due to an acute shortage of qualified professionals. There is no recognized body that represents or promotes the profession in Afghanistan, nor regulation of the profession. Accounting and auditing services are provided by expatriate accountants, without any oversight. Support is needed to develop the capacity of accounting and auditing in Afghanistan and establish a sustainable structure of effective regulation and oversight of the profession.

74. Risks and Risk Mitigation Measures: During project preparation, the Bank’s task team identified key risks and adopted appropriate mitigation measures. The overall risk rating for implementation was considered high. Based on the experience of the then ongoing IDA projects in Afghanistan, major challenges the project could face were related to the capacity and coordination within the government and among donors, as well as delays in procurement or mis‐procurement. The deteriorating security could also slow down implementation and discourage contractors from bidding, and agencies may not be able to source high level TA on a timely basis and/or at reasonable cost. The project oversight and adequate Bank supervision helped address the issues that


Page 23 of 68

arose and specific steps, including long lead‐times and engagement of specialist support for procurement addressed potential delays were considered. The project was designed to specifically address capacity issues in accounting, auditing in general and PFM specifically.

75. Adequacy of government commitment: After decades of wars and civil strife, building an effective state ‐ one that could provide security and services to the people ‐ was at the heart of the reconstruction effort in Afghanistan. The government in this context made significant progress in establishing a functioning PFM system under the direction of MOF, which is the single most crucial enabling factor for the budget implementation. The legal framework underpinning PFM (Public Finance and Expenditure Management Law and Public Procurement Law) was established. Through a series of IDA grant operations, the government had engaged specialized international firms to provide direct operational support to the Treasury of MOF, the ARDS for public sector procurement management, and the CAO for carrying out external audits for all IDA and ARTF funded operations in Afghanistan.

B. KEY FACTORS DURING IMPLEMENTATION

Positive Factors

76. The government’s ownership of PFM reform program was noteworthy. Effective PFM remained of key importance to the government as it openly promoted the reform agenda, proclaiming this was the backbone of the country and that the government would support the agenda. Despite challenges of fragility, the government was successful in making significant progress in establishing a functioning PFM system under the direction of the MOF. In particularly, the support from the President of Afghanistan, as the head of the National Procurement Commission, was a major positive factor during implementation. 77. As a follow‐on project, the agencies involved in the implementation of each component had experience in implementing Bank projects and following Bank procedures. 78. The two task team leaders were based in Kabul, which enabled them to maintain regular contacts with the government and the stakeholders. This facilitated project implementation significantly.

Negative Factors

79. Delays were experienced in the procurement of goods and consultancy packages as the government was not able to prepare a timely and comprehensive procurement plan. Requests for procurement plan revisions were made only on ad hoc basis which resulted in duplication of procurement of similar items. Had these packages been combined, the benefit from the economy of scale could have been realized. 80. In the NPA, staff faced implementation challenges because of non‐familiarity with Bank’s procurement policies and procedures, which improved with time as the staff gained experience. Implementation was by close coordination between the Bank and the RIMU and hiring of competent and professional staff familiar with Bank’s procurement guidelines. 81. Because of the country’s unstable and volatile security concerns, field visits outside of Kabul were difficult or impossible. Similarly, because of fragile country environment, interest from foreign contractors for bidding contracts remained limited which resulted in the Bank adjusting procurement methodology aligning to the


Page 24 of 68

market realities during implementation. 82. Revenue mobilization was relatively a new component and experienced implementation delays because of inadequate coordination between MOF and ARD; staff turnover; and low capacity of staff in handling procurement complexities.

IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME A. QUALITY OF MONITORING AND EVALUATION (M&E)

M&E Design

83. M&E Design: The project’s Results Framework was prepared as per the Bank’s guidelines. The PDO indicators and targets were compatible with the government’s approach, donor inputs, and overall PFM reforms agenda. This approach ensured that M&E framework was grounded in reality and ev

document of the world bank · 2018. 7. 6. · document of the world bank for official use only...

Documents