expenditure analysis supported by...

February 25, 2014

Draft version

Expenditure Analysis Supported by BOOST:

Standard Tables, and Selected Tools and Techniques

Guidance Note to PER Teams

http://iteam.worldbank.org/spaces/2013BOOST/

Table of Contents

Foreword..........................................................................................................................................................4

Section 1. The Application of Efficiency and Effectiveness Measures in Public Expenditure Analysis..........5

Section 1.1. Expenditure Analysis supported by BOOST–Getting the Analytical Dimensions and the Standard Tables Identified...........................................................................................................................8

Section 1.1.1. Identification of BOOST Standard Tables on Expenditure (descriptive stats).............8

Section 1.1.2. Identification of BOOST Tables for further Analytical Purposes................................9

Section 1.1.3. Framework on Expenditure Analysis using Standard Tables.....................................10

Section 2. Descriptive Statistics – Profile, Trends and Composition Analysis of Expenditures.....................12

Section 2.1. Descriptive Statistics in PERs...............................................................................................12

Section 2.2. Profile Analysis.....................................................................................................................12

Section 2.3. Trend Analysis......................................................................................................................14

Section 2.4. Budget Composition Analysis...............................................................................................16

Section 2.5. Descriptive Statistical Analysis --- Examples from selected PERs......................................19

Section 2.5.1. Peru PER...................................................................................................................19

Section 2.5.2. Uganda PER..............................................................................................................20

Section 2.5.3. Armenia PER.............................................................................................................21

Section 2.5.4. llustrations of BOOST standard tables and charts [Work in progress]......................22

Section 2.6. Definitions and Data Sources..............................................................................................22

Section 3. Selected Analytical Dimensions using BOOST Databases– Efficiency, Effectiveness and Equity analysis of expenditures........................................................................................................................24

Section 3.1. Effectiveness.........................................................................................................................27

Section 3.2. Equity.....................................................................................................................................27

Section 3.3. Efficiency..............................................................................................................................28

Section 3.3.1. Technical Efficiency..................................................................................................28

References.........................................................................................................................................................34

Annex 1: Institutional Coverage in BOOST.....................................................................................................35

Annex 2: Devolved Local Government Entities versus Deconcentrated Local Administration Bodies..........37

Annex 3: BOOST Core Variable Definitions Aligned With Internationally Agreed Standards.....................38


Appendix 4: BOOST Standard and Quality Check Table...............................................................................39

Foreword

The search for improvements in public expenditure efficiency, effectiveness and equity is at the core of PERs at the Bank. This purpose of this note is to improve the efficiency of PER work, by providing guidance to teams on a selected number of techniques, tools and standard data tables related to the preparation of a PER.

The note was prepared in the context of a Bank-wide review of PER – the Public Expenditure Review Stocktaking and Guidance. The Review intends to issue a number of specific guidance notes, covering in more details also some of the techniques and tools mentioned in this paper. As an example, a Guidance Note is being prepared on the DEA and SFA models, to present the various applications in details, including the related data requirements, and to provide hands-on guidance on how to conduct analysis.

The BOOST Technical Advisory Group provided comments on earlier drafts, as well as contributions were received from… Overall guidance was provided by…

The draft paper is presented to the BOOST Technical Advisory Group at its meeting on March 4, 2014

The TAG is asked to:

= review and endorse the proposed Standard Tables on BOOST (Sections 2-3)

= provide overall guidance and comments on the draft paper, in particularly on:

o The overall contents and directions of the papero The relevance of the selected tools and techniques. Any others?o Standard tables – should be paper be generalized, to focus on standard tables and standard data in

PERs, rather than the current, narrower perspective on standard tables using BOOST data?o Country cases. Would there be other examples of PERs to bring into the paper?o Views on dissemination and out reacho Any other views on how to complete the draft paper? Would TAG members be interested in joining

this work, including preparing country cases?

Further to the discussion at the TAG meeting, PRMPS will continue drafting (in particularly Section 3 needs more work) and will insert comments from TAG members. A completed draft version will be circulated to the TAG, for a final review. Final version of the paper is planned to be issued in May.


Section 1. The Application of Efficiency and Effectiveness Measures in Public Expenditure Analysis.

The application of concepts on efficiency, effectiveness and equity often prove difficult in PERs for a number of reasons. These include:

Difficulties in measuring efficiency and effectiveness – defining output in public administration or service delivery still poses problems; the various outputs may contribute to several policy objectives at one and the same time; and the costs of producing any given output are difficult to identify, since a clear understanding of the cost functions is absent and/or costs only partially captured in the accounting systems.

While progress has been made in developing measurement techniques, often good quality data is lacking to apply these techniques.

In some cases the various terms may have a different meaning when applied in different contexts, as well as exogenous factors vary depending on the level of analysis. As an example, in an analysis of technical, allocative and scale efficiency on education, the wage setting framework is seen as an exogenous factor, while in an analysis of the overall public expenditure levels and composition, the wage setting framework would be considered a variable to use, to address inefficiencies. The “allocative efficiency” term would also have been used differently at this level of analysis, as compared to an analysis on education, (or of a specific school for that matter).

The application of different levels of aggregation of analysis, as often determined by the poor availability and quality of data, may imply an uneven understanding across countries of efficiency issues, including level of policy makers control to make changes. A high level of aggregation of analysis may conceal inefficiencies, while on the other hand, a granular sector-related analysis could succeed in unbundling the various drivers behind inefficiencies, including identifying those which may be in the government’ immediate control to change.

Contextualization. PERs on MIC, LIC or FCS will vary in the application of efficiency and effectiveness measures, in great part due to variances in data availability and quality. There is a need for customization and careful attention to setting the scope of the analysis, the tools and techniques to be applied, and the data availability to be expected. 1

These issues illustrate the importance of correctly defining and understanding the scope of an efficiency, effectiveness or equity analysis, including selecting the appropriate techniques and data sources. It is equally important to situate the PER in the context of the overall expenditure policy framework for the country as such, including to capture how the analysis relate to the various ‘levels of analysis’. This point is illustrated in Figure 1 below.

1 Please refer to Anand Rajaram (2007), and European Central Bank (2006) on such contextualized analyses on clusters of countries.


Figure 1: Public Expenditure Analysis – Analytical Questions and Selected Techniques

Macro-fiscal• Spending towards economic policy priorities – • Expenditure analysis at aggregate growth, poverty reduction, equity • MTFF- fiscal rules• Overall fiscal sustainability • Sector reviews in macrofiscal optique

Public Expenditure Policy• Agility in overall level and composition- fiscal space • Spending Reviews• Drivers on effectiveness and efficiency for public • Review of expenditure drivers i.e demography expenditures (wage setting mechanisms; user charges.) • DEA benchmarking on total public sector

• Review MTBF in terms of design

Sector Effectiveness and Equity• Expenditures achieving impact? • Benefit incidence• Expenditures reaching target groups? • Benchmarking by sector indicators

Efficiency – ‘Get more for less’• Identifying inefficiencies- technical, • Efficiency analysis at unit/sector/project/ allocative, scale program level. DEA. • Identifying issues on budget execution • Budget execution analysis. Budget vs. actuals

Figure 1 illustrates the nested structure expenditure analysis, including application of the core measurements on efficiency and effectiveness. As an example, looking bottom-up, an effectiveness assessment on a specific sector, including a discussion of determinants, will remain incomplete or constrained to the extent that the efficiency review is missing or incomplete. Moving further up, a review of public expenditure matters on public sector, including improvement in public expenditure management efforts, usually require the efficiency and effectiveness reviews already completed at sector, program/project or facility unit-levels. It follows thereof that the complexity of analytical design in the PER and the related data requirements increases, as the analysis moves from one level to the next. Much more institutional context and qualitative data are required when going from efficiency to effectiveness or to the analysis of aspects of the overall expenditure policy. The preparation of a PER needs to take these aspects into account in order to ensure relevance and impact of the exercise.2

In this perspective, the purpose of the paper is to assist PER teams in clarifying scope and comprehensiveness of the PER, by using standard tables, and tools and techniques well adapted to the analytical questions at the various levels of analysis as outlined in Figure 1. The paper will come back to this issue in details in Sections 2 and 3.

The paper is organized as follows: Within the overall context as outlined in Figure 1, the remainder part of Section 1 identifies the analytical questions and the related tools, techniques and data, to support presentation and analysis PERs on two levels: descriptive- and analytics statistics. Section 2 then further presents the descriptive statistics, by identifying the standard tables and the various analytical questions

2 These issues are presented in further details in Dino Merotto and Leif Jensen, “What Makes a Good Public Expenditure Review”. Presentation at AFTP2 event, November 2013.


on ‘profile’, ‘trends’ and ‘composition’ presentations, while Section 3 presents and discuss similar issues on analytics statistics – ‘efficiency, effectiveness and equity’.

Please note that while the paper mainly defines tables and analytical questions in the context of BOOST data, the intention of the paper is nonetheless to provide guidance to PER teams on tools, techniques and data in general.

Section 1.1. Expenditure Analysis supported by BOOST – Getting the Analytical Dimensions and the Standard Tables Identified.

The construction of BOOST datasets is being guided by a set of BOOST standards, which outline the optimal institutional coverage and data structure, as well as some procedural steps to ensure high data quality (see Annex 4 - BOOST Standards and Quality Checks). The identification of BOOST standard tables is thus a logical next step in ensuring quality support to PERs and the relevance of BOOST databases. While the main purpose of BOOST standard tables is to improve efficiency of the PER preparation, the standard tables also supports a continued high quality of BOOST data. With the tables defined ex ante and independent of the specific PER, the tables guide the PER teams in defining their data queries, but the tables also give indications to the BOOST assembly team on the desired structure and coverage of the BOOST. In a broader and medium-term perspective, the standard tables may lead to better opportunities for cross-country expenditure comparisons.

Section 1.1.1. Identification of BOOST Standard Tables on Expenditure (descriptive stats).

To date there are 21 BOOSTs delivered and each BOOST database supports in varying degrees the preparation of standard tables. The databases can be found on the iTeam site. Once the design of the standard tables has been approved, such tables on the 21 completed databases will be posted on the iTeam site.

The standard tables are based on the different cuts of the three main dimensions of expenditure analysis or the core BOOST database variables (see Figure 2). These are the Administrative variable which provides information about which spending unit incurred the expense; the Economic type variable which provide information about the category or type of expense incurred; and the Function/sector variables which provide information about the sector or purpose for which an expense was incurred. In addition to these core variables, it also includes the expenditure type (recurrent, personnel, capital, non-personnel) and the financing source variables.


Figure 2: Dimensions of a Basic BOOST Expenditure Database

The numbers of analytical tables (standard and customized tables) that can be produced vary from database to database. Ideally one should be able to produce most of the standard tables as outlined in Section 2 from any given basic BOOST expenditure database. Within the basic expenditure database, the number of customized tables one can produce depends on the granularity of the data and the number of custom variables included in the BOOST database. In some databases the granularity of the data is such that it allows the user to track expenditures at the point of service delivery - districts, hospitals, schools, etc. In others, pro-poor expenditure data are already coded within the database enabling poverty related analysis.

Section 1.1.2. Identification of BOOST Tables for further Analytical Purposes.

The breadth and depth of analysis that can be done with the basic BOOST expenditure database can be further enhanced in three ways. First, customized modules such as Pay-roll module can supplement the basic expenditure database to answer questions like number of staff paid by program (see example on Mauritius in Table 6) or a Capital expenditure module can be prepared to investigate the dramatic decline in capital expenditure in a country (see example on Moldova in Table 5). Second, a BOOST expenditure database can also be supplemented with socio-economic data such as population to figure out per capita spending on education etc. Third, the BOOST dataset may be supplemented by specific sector input- and performance data, in cases where an in-depth analysis of a sector is desired. Most of the technical efficiency, effectiveness and (to some extent) equity analysis require output and/or outcome data, and for which reasons, the BOOST expenditure data set needs to be supplemented with this data. Currently Education

(i) Fund Source: Government/Donors.(ii) Fund types: recurrent, capital, personnel, non-personnel (iii) Budget – Approved - Released. (iv) Period of time


and health sector BOOSTs are routinely prepared in support of PERs (see example on Serbia in Table 6).

Figure 3: Integrated Policy Analysis Platform

Section 1.1.3. Framework on Expenditure Analysis using Standard Tables.

The standard tables in this note are organized on the basis of their application on the most commonly used analytical techniques in PERs. These techniques are profile analysis, trend analysis, budget composition analysis, efficiency analysis, effectiveness analysis and equity analysis. These techniques are further grouped into two main perspectives: descriptive (profile, trend, and composition), and analytical (efficiency, effectiveness and equity). Table 1 summarizes the presentation of expenditure analysis, by purpose, tools and techniques and data, as discussed in Section 1.1.

Basic Expenditure data by dimensions in the ‘cube’ Customized modules. As examples: (i) Pay-roll module; (ii) Pro-poor expenditures; (iii) Coding for Budget Rigidity analysis. Revenue data included as per country choice. Source: Min of Finance - Treasury

Socio-economic data: GDP; population; house-hold; spatial; urban/rural;

Source: Statistical Office

Sector: Data on activities and performance on core functions or programs. Linking expenditure and sector data using IDs

Source: Sector ministries. International sources: WHO, OECD, UNESCO, IFAD, etc


TABLE 1: FRAMEWORK FOR EXPENDITURE ANALYSIS USING BOOST

Analytical Purpose Tools & technique Data requirement

Descriptive stats

Profile analysis Profile analysis is used to lay out the expenditure landscape of the country at a given point in time. It is also used to gauge how a country performs relative to its peers. These questions in turn will lead to how is the trend of issues.

Presenting expenditure and/or components of expenditure in levels or as a share of GDP. Comparing/benchmarking using peer countries.

Basic BOOST database containing expenditure data by functional, economic, or administrative classification of the budget or any combination of each. Including information on the financing source and expenditure type is important. Minimum of 1-year (most current year) data is required. If presenting the average overtime makes more analytical sense, then will require data for more than one year.Note: to do benchmarking with peer countries requires there be consistency among datasets in economic classification and institutional coverage.

Trend analysis Trend analysis is used to gauging whether a country significantly worse (or better) than “expected,” or that improvements are slow in coming. Would the country reach its target and how quickly.These observations then provoke deeper exploration into the causes of problems.

Presenting time series of total expenditure and/or components of expenditure growth rate or in levels or as a share of GDP. Comparison/Benchmarking with peer countries.

Basic BOOST database containing expenditure data by functional, economic, or administrative classification of the budget or any combination of each. Including information on the financing source and expenditure type is important. Minimum of 5-10 years data is required.Note: to do benchmarking with peer countries requires there be consistency among datasets in economic classification and institutional coverage.

Budget Composition analysis

The Budget composition analysis help identify the government’s priority sectors and activities. This motivates efficiency, effectiveness and equity questions.

Presenting expenditure as a share of total expenditure by sector and/or by economic type. Comparison/Benchmarking with peer countries.

Basic BOOST database containing expenditure data by functional, economic, or administrative classification of the budget or any combination of each. In addition, information on the financing source and expenditure type can be included. Minimum of 1-year depending on the type of analysis done: Profile (minimum 1-year) and trend (5-10yrs).

Analytical stats

Allocative efficiency

Allocative Efficiency analysis provides information on whether a country or sector is achieving optimal mix of inputs.

Benchmarking with comparator countries and international norms. Budget deviation analysis.

Basic BOOST database preferably containing all the 3 dimensions and additional variables. The more granular the data the better.

Technical efficiency Technical efficiency gives insight into the efficiency with which inputs are converted into outputs.

DEA analysis Four quadrant analysis Bi/multivariate regression residuals analysis

Basic BOOST database and Sector module BOOST database containing multiple input/output measures.

Effectiveness The analysis of effectiveness is about the relationships between inputs, outputs and outcomes.

Basic BOOST database and data ib output/outcome indicators

Equity Equity analysis is done to see whether public money reaches those with the greatest need.

Presenting geographic distribution of expenditure Presenting expenditure and demographic data Presenting outcome data and demographic data

Basic BOOST database, demographic data, and output/outcome indicators


Section 2. Descriptive Statistics – Profile, Trends and Composition Analysis of Expenditures.

Section 2.1. Descriptive Statistics in PERs.

Descriptive expenditure analyses, as the name implies, are descriptive, giving the first account of the expenditure landscape of a given country which will lead to a declaration of the issues. This in turn will lead to deeper examination into the causes of the issues. They do not provide diagnosis or explanation. Basically they answer broadly one of the core PER question “where does the money go?” Often in PERs the descriptive expenditure analysis used are profile, trend, and composition analysis. Profile analysis captures expenditure at a point in time; trend measures progress over time while composition shows government priority areas presented as a share of total expenditure.

Usually in profile analysis expenditure or components of expenditure data are presented in levels (in local currency unit) or as a share of GDP. In trend analysis, they are presented as levels or as growth rate, or as shares of GDP. For composition analysis, on the other hand they are presented as a share of total expenditure to show the governments’ priority sectors or areas.

Another way these three analytical methods are presented is by using country comparisons or benchmarking as a tool/technique. In most cases they show profile or trend analysis for a given country as compared to other countries in the same region, income level or ethnic mix or even with countries to which the country aspires to. Here it is very important to make sure that comparisons are done apples to apples. There needs to be consistency in economic classification and institutional coverage in order to compare the expenditure performance of two countries. Please refer to Annex1. Finally, it is worth noting that these comparisons or benchmarking techniques are not substitutes for deeper analysis but rather a step towards further framing the question of performance to be further investigated using efficiency, effectiveness and equity analysis which will be covered in Section 3.

Section 2.2. Profile Analysis.

Profile analysis sets the stage providing information on how money is spent on what and by whom it is spent at a point in time. In PERs different techniques are used for profile analysis. In some cases profiles are presented as expenditure and/or components of expenditure in levels, as a share of total expenditures or as a share of GDP. Other times benchmarking technique is used by comparing the data to peer countries or groups. The profile analysis provides answers to often dealt PER questions like how much was public expenditure in 2010? Who spend the most, on what and for what function? How recurrent spending is compared to investment spending? How much was allocated for education? Did the country spend more in education than regional peer countries? How much was spent on priority sectors such as health? What is the expenditure profile of this priority program? etc.


Below are examples of profile analytical table and figure headings from actual PERs and other analytical documents that can be prepared using BOOST database:

Table: Total Expenditure and Selected Components of Expenditure for Selected African Countries, in percent of GDP, 2011

Table: Public investment by sector

Table: Allocation of Education Spending Figure: Country X spends relatively more in education than others east and central Asia transitionCountries

Table. Recurrent Spending Too Low Relative to Investment

Figure: Distribution of public spending on education by category (primary and secondary by econ type)

Table: Total Expenditure in Public Health by Major Economic Category FY 2011

Figure: Teachers’ salaries as percentage of current public expenditure in education, primary and secondary, 2008-2011 average – compare to other countries in the World

Table: Costs of Priority Health NGPES Programs. 2010

Appendix: Budget expenditure by economic types and by sector, by central and local government 2010

Profile standard tables supported by BOOST databases present spending broken down by administrative, functional, or economic classification and any combination of those. In addition, information on the financing source and expenditure type are included. Depending on the availability of granular data, analysis can delve down to the project/program level. There are two techniques used to present profile analysis. First expenditure and/or components of expenditure are presented in levels or as a share of GDP. Second, the same analysis is presented by comparing/benchmarking with peer countries.

The minimum data requirement to do this type of analysis includes the basic BOOST database containing expenditure data by functional, economic, or administrative classification and any combination of each. Including information on the financing source and expenditure type is also important. A minimum of 1-year (most current year) data is required. If presenting the average overtime makes more analytical sense, then more than one year data is required. Also it is important to note that to do benchmarking with peer countries there needs to be consistency across country datasets in economic classification and institutional coverage. Below are the BOOST standard tables using profile analysis.


TABLE 2. PROFILE ANALYSIS- BOOST STANDARD TABLES

Expenditures By Administrative Classification (Level 1), current yearExpenditures By Administrative Classification (Level 2), current yearExpenditures By Administrative Classification and Functional Classification (level 1), current yearExpenditures By Administrative Classification and Economic Classification (level 1), current yearExpenditures By Administrative Classification and Financing Source, current yearExpenditures By Administrative Classification and Expenditure Type, current yearExpenditures By Functional Classification ( Level 1)Expenditures By Functional Classification (Level 2)Expenditures By Functional Classification and Administrative ClassificationExpenditures By Functional Classification and Economic ClassificationExpenditures By Functional Classification and Financing SourceExpenditures By Functional Classification and Expenditure TypeExpenditures By Economic Classification (Level 1)Expenditures By Economic Classification (Level 2)Expenditures By Economic Classification and Administrative ClassificationExpenditures By Economic Classification and Functional ClassificationExpenditures By Economic Classification and Financing SourceExpenditures By Economic Classification and Expenditure TypeExpenditures By Project/program3 (Level 1)Expenditures By Project/program (Level 2)Expenditures By Project/program and Administrative ClassificationExpenditures By Project/program and Functional ClassificationExpenditures By Project/program and Financing SourceExpenditures By Project/program and Expenditure Type

Section 2.3. Trend Analysis.

Trend analysis demonstrates development over time providing information on how money is spent on what and by whom it is spent over a period of time. The Trend standard tables show trends in spending - broken down by the economic, functional, or administrative classification and any combination of those. In addition, information on the financing source and expenditure type can be included. The minimum time-period covered in BOOST database is 3 to 5-years however, most BOOST databases on average cover 5-years.

PERs often address the questions what is driving the growth/decline of public expenditure for the last five years? Is it Investment, wages, or social transfers? If wages, which sectors wages exhibit the most growth, defense? Health sector? How is the trend in the health expenditure across the region? Are donors financing this sector? How volatile is donor funding in the sector? What has been the trend of recurrent expenditure in this sector?

3 In some cases Project/programs are included within the Administrative classification in other cases they have their own headings.


The below are examples of trend analytical table and figure headings from actual PERs and other analytical documents that can be prepared using BOOST database:

Table: Analysis of Public Expenditure Trends and Future Resource Needs

Figure: Wages are rising but outlays on maintenance and operations are trending down

Table: Expenditure by economic classification and functional classification time series data covering 5 years or more

Figure: Wage outlays rose sharply in 2009, largely due to increases for defense and security staff

Table: Trend in Health Expenditures across Regions

Figure: Recurrent Expenditure by Sector, 1997-2002 (percent of sector’s total State Budget expenditure)

Table: Volatility of Donor Funding Figure: Functional Composition of Budget Expenditure - 1997 and 2002

Table: The Composition of Government Expenditures

Figure: Public spending is increasing rapidly in real terms driven by investment and wages, and social transfers

Trend standard tables supported by BOOST databases present spending broken down by administrative, functional, or economic classification of the budget or any combination of each. In addition, information on the financing source and expenditure type are included. Depending on the availability of granular data, analysis can delve down to the project/program level. There are two techniques used to present trend analysis. First a time series of total expenditure and/or components of expenditure are presented as growth rate, in levels or as a share of GDP. Second, the analysis is presented by comparing/benchmarking using peer countries.

The minimum BOOST data requirement to do this type of analysis includes the basic BOOST database containing expenditure data by functional, economic, or administrative classification of the budget or any combination of each. In addition, information on the financing source and expenditure type can be included. Minimum of 5 to 10 years data is required. It is important to note that to do benchmarking with peer countries there need to be consistency among datasets in economic classification and institutional coverage. Below are the BOOST standard tables using trend analysis.


TABLE 3. TRENDS ANALYSIS - BOOST STANDARD TABLES

Expenditures By Administrative Classification (Level 1)Expenditures By Administrative Classification (Level 2)Expenditures By Administrative Classification and Functional Classification (level 1)Expenditures By Administrative Classification and Economic Classification (level 1)Expenditures By Administrative Classification and Financing SourceExpenditures By Administrative Classification and Expenditure TypeExpenditures By Functional Classification ( Level 1)Expenditures By Functional Classification (Level 2)Expenditures By Functional Classification and Administrative ClassificationExpenditures By Functional Classification and Economic ClassificationExpenditures By Functional Classification and Financing SourceExpenditures By Functional Classification and Expenditure TypeExpenditures By Economic Classification (Level 1)Expenditures By Economic Classification (Level 2)Expenditures By Economic Classification and Administrative ClassificationExpenditures By Economic Classification and Functional ClassificationExpenditures By Economic Classification and Financing SourceExpenditures By Economic Classification and Expenditure TypeExpenditures By Project/program (Level 1)Expenditures By Project/program (Level 2)Expenditures By Project/program and Administrative ClassificationExpenditures By Project/program and Functional ClassificationExpenditures By Project/program and Financing SourceExpenditures By Project/program and Expenditure Type

Section 2.4. Budget Composition Analysis.

Budget composition analysis lays out the government’s priorities based on its expenditure outcome. The technique mostly used in PERs is simple and involves presenting expenditure as a share of total expenditure by sector and by economic type. PER’s often ask the questions such as what are the government’s priorities based on its expenditure Outcomes? Is the government improving the allocation of government health expenditures? Within the Health Sector specifically where is the money spent (on salary & wages or on goods and services?) and which sector institutions are spending the money (clinical hospitals or district hospitals etc.)?


The below are examples of trend analytical table and figure headings from actual PERs and other analytical documents that can be prepared using BOOST database:

Table: Selected Budgetary Allocations, 2008-10, Share of Budget (%)

Table: Line item composition for different health sector institutions by Economic Type

Table: National Priorities not reflected in Allocation of Recurrent Expenditures

Table: Government health expenditure by line item

Table: Capital and Maintenance expenditure for selected African countries

Figure: Capital and Maintenance Spending, 2010 and 2011—Maintenance broken down IT equipment, vehicles, Plant and Equipment, Buildings/structures

Table: Central government spending Figure: Maintenance Spending and Capital Stock, 2010 and 2011

Table: Local government spending (financed by local tax and VAT)

Figure: On average, priorities are given to social sectors (social spending, health, education)

Table: Local government spending (financed by central government transfers)

Figure: Local government expenditures were reduced in accordance with the transfer of service deliveryResponsibilities to the Center

Table: Sources of funding of Higher Education Institutions (HEIs)

Figure: Improving the allocation of government health expenditures

Budget composition standard tables supported by BOOST databases present spending broken down by administrative, functional, or economic classification and any combination of those. In addition, information on the financing source and expenditure type are included. Depending on the availability of granular data, analysis can delve down to the project/program level. There are two techniques used to present expenditure composition analysis. First the components of expenditure as a share of total expenditure are presented as a share of total expenditure. Second, the analysis is presented by comparing/benchmarking using peer countries.

The minimum BOOST data requirement to do this type of analysis includes the basic BOOST database containing expenditure data by functional, economic, or administrative classification and any combination of those. In addition, information on the financing source and expenditure type can be included. Minimum of 1-year data is required depending on the depth of analysis desired and data availability. Below are the BOOST standard tables using expenditure composition analysis.


TABLE 4. EXPENDITURE COMPOSITION – BOOST STANDARD TABLES

Expenditures By Administrative Classification (Level 1) as a share of Total ExpenditureExpenditures By Administrative Classification (Level 2) as a share of Total ExpenditureExpenditures By Administrative Classification and Functional Classification as a share of Total ExpenditureExpenditures By Administrative Classification and Economic Classification as a share of Total ExpenditureExpenditures By Administrative Classification and Financing Source as a share of Total ExpenditureExpenditures By Administrative Classification and Expenditure Type as a share of Total ExpenditureExpenditures By Functional Classification (Level 1) as a share of Total ExpenditureExpenditures By Functional Classification (Level 2) as a share of Total ExpenditureExpenditures By Functional Classification and Administrative Classification as a share of Total ExpenditureExpenditures By Functional Classification and Economic Classification as a share of Total ExpenditureExpenditures By Functional Classification and Financing Source as a share of Total ExpenditureExpenditures By Functional Classification and Expenditure Type as a share of Total ExpenditureExpenditures By Economic Classification (Level 1) as a share of Total ExpenditureExpenditures By Economic Classification (Level 2) as a share of Total ExpenditureExpenditures By Economic Classification and Administrative Classification as a share of Total ExpenditureExpenditures By Economic Classification and Functional Classification as a share of Total ExpenditureExpenditures By Economic Classification and Financing Source as a share of Total ExpenditureExpenditures By Expenditure Type and Economic Classification as a share of Total ExpenditureExpenditures By Project/program (Level 1) as a share of Total ExpenditureExpenditures By Project/program (Level 2) as a share of Total ExpenditureExpenditures By Project/program and Administrative Classification as a share of Total ExpenditureExpenditures By Project/program and Functional Classification as a share of Total ExpenditureExpenditures By Project/program and Financing Source as a share of Total ExpenditureExpenditures By Project/program and Expenditure Type as a share of Total Expenditure


Section 2.5. Descriptive Statistical Analysis --- Examples from selected PERs.

Section 2.5.1. Peru PER4

The PERU 2012 PER provides an overall story line of the challenges faced in public expenditure management. It deals with the three central questions in expenditure analysis. What are public resources being spent on? Who is spending the public resources? Where are the public resources being spent? While central government data provides overall spending figures, disaggregated data that provide invaluable insight into key developments in public expenditure can better be captured through the BOOST database. Summary of the descriptive analytical flow using BOOST data is as follows.

What are public resources being spent on?Trend analysis of on the aggregate level of the functional classification “Trend table: Expenditures By Functional Classification ( Level 1)” shows that public expenditure in Peru has risen by 96 percent from 2005 to 2010.

Analysis of the trend in interest payments “Trend table: Expenditures By Economic Classification (Level 1)” reveal the decline in administration and planning expenditures.

Trend analysis of the economic classification “Trend table: Expenditures By Economic Classification and Expenditure Type” shows that there has been a significant increase in public investment. However, profile analysis using benchmarking indicates that Peru’s capital spending is also high by international standards. Further profile analysis of expenditure by economic classification and by geographic areas reveals that there are large differences in the capital-to-current expenditure ratio across these geographic areas.

The budget composition analysis of the changes in the relative shares of public expenditure by different functions (“Budget Composition table: Expenditures By Functional Classification ( Level 1) as a share of total expenditures”) reveals the priority sectors and the shifts of these priorities over the years.

Profile analysis using benchmarking reveal that Peru spends little in social sectors including education, health and social protection compared to its regional peer countries. Different sources have been used for comparator country data but for Peru the “Profile table: Expenditures by Functional Classification (Level 1)” was used in all cases. Profile analysis of interest payments, goods and services, subsidies and transfers and the wage bill was also compared with international comparator countries. Here also data for comparator countries used different sources, however for Peru “Profile table: Expenditures by Economic Classification ( Level 1)” was used in all cases.

4 Public Expenditure Review for Peru: Spending for Results (July 6, 2012)


Who is spending the public resources?Trend analysis of public budget by level of government (“Trend Table: Expenditures By Administrative Classification (Level 1) as a share of Total Expenditure”) reveal that local governments took on an increasingly important share of public spending. Looking at local government spending by economic classification, half of local governments’ expenditure is investment. This in turn raises concerns about whether local levels of government are adequately equipped to perform such high levels of public investment which is further investigated using budget deviation analysis.

Where are the public resources being spent?Profile analysis of how spending is distributed across the territory by looking at public spending per capita, by department (“Profile Table: Expenditures By Administrative Classification (Level 2)”) and level of government (“Profile Table: Expenditures By Administrative Classification (Level 1)”) reveals that distribution of public expenditure across departments reinforces the income inequality gap. Furthermore, the three levels of government spending reinforce the unequal distribution of public expenditure which warrants further analysis on equity.

Section 2.5.2. Uganda PER5

Uganda BOOST was one of the first database prepared to boost expenditure analysis in PERs. Uganda’s expenditure at the time had many weaknesses as outlined in the PER. Nevertheless, the available information still presented a fair assessment of the most comprehensive and current disaggregated budget dataset in use at the time. Summary of the descriptive analytical flow using BOOST data is as follows.

Structure and Trends in the Ugandan Budget

Trend analysis of expenditure by type of budget (“Trend Table: Expenditures By Expenditure Type”) reveal that Ugandan budget estimates are presented in three sections clarifying the structure of the budget. Analysis done on “Trend Table: Expenditure by Type and Financing source” shows the decline of donor funding to development. Looking at “Trend table: Expenditure by Type and Economic classification”, one can quickly see that the capital share of the development budget has declined. Furthermore from the same table, one can see that the falling budget share of capital expenditures was offset by rising shares of staff cost, transfers and other recurrent spending.

Analysis of trend by service level (“Trend Table: Expenditure by Administrative classification (level 1)”) reveals that Ministries dominated total spending.

Analysis: What Lies Behind the Trends: Are They Healthy for Service Delivery?Analysis of spending by “Trend Table: Expenditure by economic classification and by Administrative (level 2)” indicates the rapid deterioration of funding service delivery. Looking at

5 Uganda - A public expenditure review 2008 : with a focus on affordability of pay reform and health sector (May 30, 2009)


district expenditure broken down by type of expenditure, one can see that recurrent spending has risen during the time period.

Trend analysis of “Trend Table: Expenditure by functional classification and economic type (level 1)”, show that for both health and education sectors wage to non-wage ratios deteriorated. Further study reveal that hiring more workers to expand access, and awarding them pay raises seem to have eaten into sector non-salary budgets. Further analysis of “Trend Table: Expenditure by functional classification and economic type (level 2)”, show that health and construction budgets fail during the same period of time. At the same time budget composition analysis by program type (“Composition Table: Expenditures By Project/program (Level 1) as a share of Total Expenditure”) indicate that the local government development program, water and Uganda national agriculture advisory services have become priority projects.

Section 2.5.3. Armenia PER6

The Armenia PER delves into public sector wage bill, health, education and road transport expenditures to better inform the fiscal consolidation process after the crisis. The BOOST database was prepared and used to better assess allocation efficiency of public expenditure and to easily identify priority programs. Summary of the descriptive analytical flow using BOOST data is as follows.

Expenditure trends

Composition trend analysis of expenditure by functional classification (“Composition Table: Expenditure by functional classification (level 1) as a share of Total Expenditure”) reveals that social security and social protection remain key priority sectors before and after the crisis. Also it shows that the real sector expenditure experienced the largest variations from year to year which indicates that this sector has been used as the adjustment variable. The sector was used to inject fiscal stimulus in the economy during the crisis.

Composition analysis of the expenditure on education by subsector (“Composition Table: Expenditure by functional classification (level 2)”) reveal that focus is on general primary and secondary general education as well as capital expenditure.

Composition analysis of expenditure on social services by project (“Composition Table: Expenditure by functional classification and by program/project”) shows that old age pension and the family benefit program are the main targets of government.

Expenditure trend analysis by economic classification (“Trend table: Expenditures By functional classification (level 1)and Economic Classification ( Level 1)”) reveal that spending in

6 Republic of Armenia - Fiscal consolidation and recovery (Vol. 1 of 2) : Synthesis report (English) (Nov 1, 2011)


general public services was driven by subsidies and grants, personnel expenses and interest payments during and pre-crisis periods. Analysis of spending on public order (“Trend table: Expenditures By functional classification (level 2) and Economic Classification (level 1)” ) shows that personnel expenses was mainly driving the increase in spending in this subsector.

Public Sector Wage Bill

A profile comparative analysis of the wage bill with other ECA countries and other international comparators is done using the expenditure by economic classification as a percent of GDP. The size of the civil service and pay regimes is also analyzed by supplementing data from other databases.

Health Expenditure

Composition analysis of “Composition Table: Expenditure by functional classification (level 1)” shows that the share of the government budget allocated for health has been gradually declining. This decline is despite of a series of MTEFs in which the health share was expected to increase. Further analysis was done by comparing the share of public and private spending in the health sector by supplementing data from other sources. The report also discusses effectiveness of the sector looking at the relationship between health spending and health outcomes and poverty.

Education Expenditure

A trend analysis of total public spending on education (“Trend table: Expenditures By economic Classification ( Level 1)”) reveal that education spending is continuously declining which appears to be a reversal of the government’s long term vision for developing the education sector.

A profile analysis of public spending on education by subsector (primary, secondary etc.), by economic category and by spending authority (Ministry of education, Other ministries) (“Profile Table: Expenditure by functional classification (level 1) and by administrative level 1 and by economic classification”) allow to review the type of education spending and set the stage for efficiency and allocative issues in public education.

Section 2.5.4. illustrations of BOOST standard tables and charts [Work in progress]

In Annex 5, a range of examples of standard tables and standard charts will be prepared, by using BOOST datasets already completed.

Section 2.6. Definitions and Data Sources

As regards definitions and data sources to establish the BOOST standard tables, please refer to the annexes, where:

Annex 1, Institutional Coverage in BOOST gives a brief description of institutional coverage based on the GFSM 2001. More often than not, PERs are not explicit about the coverage of the public sector


in the analysis and the definition of public sector is not consistent. To bring in more transparency and consistency on institutional coverage, this annex gives a brief definition of the public sector, describes why it is important to have a standard definition and gives an overview of coverage in BOOST databases.

Annex 2: Devolved local government entities versus deconcentrated local administration bodies presents a summary of important definitions on decentralization. Understanding the difference between devolved and deconcentrated local governments is essential both for database construction as well as doing analysis.

Annex 3, BOOST variables gives a brief definition of the BOOST variables to benefit those who are not familiar with BOOST variables. While the BOOST database does not impose standardization of classification systems, the generic examples provided here use economic classification from GFSM2001/12 and functional classification from COFOG.


Section 3. Selected Analytical Dimensions using BOOST Databases– Efficiency, Effectiveness and Equity analysis of expenditures.

Analysis of the performance of public expenditure is the main objective of PERs. For the most part PERs identify the sources of poor sector performance and outcomes for a given level of spending and explore reasons for observed inefficiencies and inequities. Addressing these inefficiencies and inequities can be important contributions to policy and program reforms or to suggesting ways to improve performance. To this end efficiency, effectiveness, and equity analysis are routinely done in PERs. Efficiency analysis measures how well public resources are being spent; effectiveness relates inputs, outputs and outcomes and equity looks into whether public money reaches those with the greatest need.

Measuring efficiency, effectiveness and (to less extent) equity as mentioned above is intrinsically difficult. The scarcity of sector specific input and output/outcome data and the lack of a unified guidance in performing this type of analysis manifest itself in PERs. To some extent BOOST has facilitated data collection as seen both in Serbia and Moldova where sector specific BOOST were prepared to supplement the basic BOOST database enabling thorough efficiency and effectiveness analysis in the education sector for Serbia and on capital accounts for Moldova.

While the BOOST tool is being used to improve on the data side, the demand for guidance in performing this type of analysis is still unmet. As an initial step to support PER teams in these areas, Tables 4 and 5 have been prepared to summarize the application of efficiency, effectiveness and equity analysis in the BOOST databases completed as of September 2013.


Table 5: Summary of BOOST Supported Efficiency/Equity Analysis in PERs

Country Report name & type Type of analysis Brief description of analysis Topic/sector SourceMoldova PER FY13: Capital

Expenditures:Making Public InvestmentWork for Competitivenessand Inclusive Growthin Moldova

Trend, composition, efficiency and equity (regional equity)

- In depth trend and composition analysis of capital expenditur by sector. - Allocative efficiency using deviation analysis. - Per Capita Spending by District to analyze equity.

Capital expenditure in the transport, utilities and housing, education, health, and agricultural sectors.

Capital expenditure module BOOST to the detailed object level and performance information monitored by the MoF.

Armenia PER FY11: Fiscal Consolidation and Recovery

Technical efficiency and equity

- Education Spending and Student Performance. Benchmarking both indicators to regional peers. - Regression analyses of student performance on various educational inputs. - Analysis of performance disparity between rural and urban students in university entrance exames.

Education BOOST database, BOOST edcation module, EDstat etc.

Uganda PER FY08: With a Focus on Affordability of Pay Reform and Health Sector

Allocative efficiency -Efficiency in service delivery. Increasing recurrent expenditure driven by the deteriorating wage to non-wage ratio. - Deviation/variance analysis: calculated by adding underspending releases with overspending releases, and comparing the total to the budget

Public sector, service delivery

BOOST database

Peru PER FY12: Spending for Results Allocative efficiency, Technical efficiency

-Is public expenditure sufficiently targeted? Results based budgeting. - Measurement of the public sector performance (PSP) which is the outcomes generated by public sector actives - Public sector efficiency (PSE)which is the outcomes relative to the resources employed (following the strategy proposed by Afonso, Schuknecht and Tanzi (2005)).

Social programs, education, health, public investment.

BOOST and other data sources

Belarus PER FY13: Enhancing Public Services in Times of Austerity

Technical efficiency Regression analysis looking at the relationship between spending and student performance. What would be the efficiency gain if student-teacher ratios were at OECD average level?

Education Education Module BOOST and other data sources.

Guatemala PER FY13: Towards Better Expenditure Quality

Allocative efficiency/equity

- Incidence analysis looking at the progressivity of primary education expenditure (i.e. disproportionately benefiting the poor), vs the regressivity of spending on secondary and tertiary education. The high share of education expenditure allocated to the primary level is therefore beneficial for poverty reduction.

Education/pro-poor expenditure

BOOST database.

Guatemala PER FY13: Towards Better Expenditure Quality

Technical efficiency/equity - Comparisons between financial inputs and key outputs-

correlations. - (DEA) methodology to derive efficiency scores - OLS regressions to estimate determinants of student achievement at the municipal level.

Education BOOST database, Ministries of Education and Health.

Niger Niger - Public Expenditure Review (PER) 2012 (English)

Allocative efficiency/equity

- Budget execution rate - Per capita public health expenditure on goods and services by region. Education/Health BOOST database

Note: All the above PERs can be accessed through the PER portal at http://search.worldbank.org/per

http://search.worldbank.org/per


Table 6: Summary of BOOST Analytical Work in Reports and Presentations


Note: All reports and presentations can be found on the BOOST iTeam site.


Section 3.1. Effectiveness

The analysis of effectiveness is about the relationships between inputs, outputs and outcomes. The following are most commonly used output/outcome measures for the education and health sectors.Education: completion rates for primary school, gross and net enrollments, secondary enrollment, secondary completion rates, measures of learning, ratio improvement --student-teacher, pupil-textbook, pupil classroom; Health: immunization rates, indicators such as children sleeping under insecticide treated bed nets, children under age 5 who are underweight, proportion of births with skilled attendant, share of population with access to improved water sources.

The below are examples of effectiveness analytical table and figure headings from actual PERs and other documents that can be prepared using BOOST database:

Table: Effectiveness of Public spending in Agriculture: poverty headcounts, levels of per capita agriculture, agricultural labor productivity (as measured as agriculture sector value added per agricultural worker, is a proxy for the effectiveness of public spending on agriculture)

Figure: Global v Local Comparisons: relationship between life expectancy, GDP per capita, and health expenditure per capita—comparing district in Country X with countries in the world.

Table: Targeting Effectiveness of HEPR (National Target Programs)

Figure: Relative effectiveness: completion rate, adult literacy rate primary education expenditure per capita per district

Section 3.2. Equity

Equity analysis looks into whether public money reaches those with the greatest need. Most countries build equity into the provision of budgetary resources and so the standard equity analytical tables can be used to examine the extent to which resources have actually penetrated areas where the need is greatest. PER’s often address questions such as how pro-poor was the composition of public spending? Regional equity can be demonstrated by doing regional/district level expenditure maps. How are per capita grants per province? How are budget transfers related to poverty measures across provinces?

Analysis of regional equity based on the distribution of expenditure by geographic location is routinely done in PERs. One of the simplest and most often used techniques is presenting the distribution of spending by regions and lower levels on maps or tables. For examples the Niger PER - BOOST presents a chart of per capita spending on public health on goods and services by region. Demographic data can be included to further enhance analysis. Of course, to do this type of analysis, the BOOST database needs to include spending data by geographic location.

A more complex equity analysis can be prepared by linking spending data to demographic data and supplementing it with detailed sector input/output data. For example for Serbia the analysis on the education sector show that distribution of teacher quality is skewed toward urban and better-off


municipalities. This was done by linking percent of teacher with university degree by municipal welfare quintiles and urban-rural location.

Some countries have pro-poor expenditure built into their budgets and are able to identify each expenditure line item as such in the BOOST database. With this type of data it is possible to prepare profile, trend and sectoral composition of pro-poor expenditures by administrative unit, by economic type and by function. There is on-going work on this type of analysis in francophone African countries like Guinea and Niger. At the minimum these analysis and level of attention to data could shade light into misclassification issues in pro-poor spending.

The below are examples of budget deviation analytical table and figure headings from actual PERs and other analytical documents that can be prepared using BOOST database:

Table: How Pro-Poor was the Composition of public spending?

Figure: Per-student expenditure across Districts

Table: Expenditure by Region Figure: The geographic distribution of provincial grants

Table: Geographic Distribution of Subsidy Expenditure

Figure: Provincial Grants per Capita in 2010

Figure: Average annual expenditure per student. By maternal education quartiles and urban-rural location

Figure: Budget Transfers and Poverty across Provinces in 2002

Figure: Percent of teacher with a university degree, by municipal welfare quartiles and urban-rural location

Figure: Per Capita Expenditure in Education by Districts

Section 3.3. Efficiency

Efficiency is a measure of how well public resources are being spent. There are many different approaches to measuring efficiency but the most used in PERs are technical efficiency and allocative efficiency.

Section 3.3.1. Technical Efficiency

Technical efficiency is the efficiency with which inputs are converted into outputs often done by linking budget management centers, both within a sector and across geographic areas, to the inputs they use and the results (whether outputs or outcomes) they generate. The efficiency of public spending is measured by comparing actual spending with the minimum spending theoretically sufficient to produce the same actual output.


This section will cover some of the technical efficiency analysis techniques used in PERs starting with the DEA frontier analysis.


Frontier Analysis Using Data Envelopment Analysis

Numerous techniques have been developed over the past decades to tackle the empirical problem of estimating the unknown and unobservable efficient frontier. One of the two methods that have commonly been used is the Data Envelopment Analysis (DEA)7. This methodology defines efficiency as the distance from the observed input-output combinations to an efficient frontier. This frontier, defined as the maximum attainable output for a given input level, is estimated using the Data Envelopment Analysis (DEA) techniques. This method uses non-parametric methods to avoid assuming specific functional forms for the relationship between inputs and outputs or for the inefficiency terms. The Data Envelopment Analysis (DEA), assumes that linear combinations of the observed input-output bundles are feasible8.

Once the DEA efficiency frontier, which is the maximum attainable output for a given level of inputs, is estimated, both input-inefficiency (excess input consumption to achieve a level of output) and output-inefficiency (output shortfall for a given level of inputs) are scored.

Both input and output/outcome data are required to do technical efficiency analysis using DEA. Table 8 lists the input and output indicators that are commonly used for DEA frontier analysis.

There are numerous examples of efficiency analysis done using the DEA frontier analysis as outlined on Tables 4 and 5. One example that is mentioned in the tables is efficiency analysis done on the education sector presented in the Guatemala PER. By running simple correlation analysis the team found that there is very weak relationship between spending and sector performance for the large majority of municipalities in Guatemala. For example, higher public spending on education is not associated with higher school completion rates. Similarly, the relationship between education spending and test scores appears to be weak. This suggests that demographics and other socio-economic factors – rather than the level of public education spending – may be responsible for driving the variations in outcomes across municipalities. To try and capture the combined influence of these factors in determining variations in outputs, the team used the data envelopment analysis (DEA) methodology to derive efficiency scores using multiple outputs and multiple inputs (including “control” variables). Results show that technical efficiency is not necessarily higher among richer municipalities.

Box on Paraguay and Mexico PERs, with DEA applications?

It is worth noting here that more often than not, data limitations prevent the development of a robust efficiency frontier analysis and therefore technical efficiency analysis is done at a stylized

7 The Free Disposable Hull (FDH) is the other approach commonly used. See Afonso, Schuknecht, and Tanzi 2003.8 It is worth noting that the limitations of the non-parametric method (vis a vis parametric method) derive mostly from the sensitivity of the results to sampling variability, to the quality of the data and to the presence of outliers. As mentioned in Section 1, the data requirements are significant when applying these measures to assess spending efficiency


facts level, mainly describing observed trends in the allocation of spending and outcomes, and the potential correlation between them. Please refer to the Box for a discussion on simpler ways to model the production frontier to facilitate relative efficiency analysis.

Public Sector Performance (PSP) and Public Sector Efficiency (PSE) indicators9

PSP links sector input data to data on sector results and PSE goes on to link PSP data to expenditure data to assess the efficiency or performance of a sector. In the Peru PER the team presents the PSP and PSE indices for the different regions covering the education, health, and transport sectors. The performance indicators are drawn mainly on the baseline results indicators collected as part of the performance-oriented budgeting. Combining results data with expenditure data the team is able to compute how the expenditure of the three levels of government in a given region is correlated or not with results on the ground. The result is presented on a map covering most regions in Peru. See Peru PER for more details specially Annex E where .

Analysis on relative efficiency [work in progress]

Efficiency is measured in relative terms.

It thereby integrates the two basic problems of a) defining a performance standard, the technology, and b) evaluating achievements against the established standard. Box 1 summarizes the different methods used to measure relative efficiency ranging from the simplest quadrant analysis and bivariate regression residuals analysis to multivariate regression residual analysis and DEA approach.

Explaining differences in efficiency -- Regression Analysis

What makes some spending units more relatively efficient at achieving high level of outputs with a given level of inputs? Once the top performers are identified using relative efficiency analysis, then the next question is which factors make them efficient and how do we use this knowledge to help low performers. (Of course there are other characteristics that one needs to consider.) In Armenia to examine the determinants of student performance a multivariate regression analysis using OLS was done. The results show the importance of the teacher quality on student performance. Holding constant the student background and community characteristics, an increase in the share of teachers with a university degree by one standard deviation (SD) is shown to correspond to a 0.13 to 0.25 SD increase in average 8th grade exam results.

9 It is assumed that PSP in a given sector or activity can be linearly approximated by a weighted average of outcome indicators (Ik) related to this activity. For simplicity, all indicators are assumed to carry equal weights. Therefore, for a region j and an activity i:

PSPij ¿∑k=1

n

I Iijk

PSE in a given activity can be expressed as the ratio between the PSP for this activity and the relevant public expenditure (PEX). For a region j and activity i:PSEij = PSPij / PEXij


However, as mentioned earlier, the requirement of data to do such analysis is high. For example to conduct this analysis data was compiled from a variety of national sources. These include the Ministry of Education and Science (MOES), Ministry of Finance (MOF), Ministry of Culture, Media and Information Society (MCMIS), and the Republican Statistical Office (RSO), among others. The school-level indicators are then assembled into a Serbia BOOST education module. See “Serbia World Bank Report School Spending and Student Performance: BOOST Analysis of Resource Allocation in Serbian Primary Education by Igor Kheyfets” for the full report.

Box 1: Four ways to measure relative efficiency

How efficient are different schools / districts / municipalities at providing quality education with the given level of resources? Identify “top performers” who are most efficient relative to others in achieving results? To answer questions like these find below a brief description of the four techniques that can be used.

Four quadrants and bivariate regression residual analysis- Simpler approaches

The four quadrants is the simplest approach followed by bivariate regression residual analysis. Both these techniques only require one measure of “inputs” and one measure of “outputs”. As demonstrated in the charts the first approach uses averages of the input and output line is against which the relative efficiency of a unit is measured. The bivariate regression approach on the other hand uses the best-fit line from which relative efficiency is measured. With this method important trends can be observed if variables are picked carefully

Multivariate regression residual and Frontier analysis- Complex approaches

Multivariate regression residual analysis need to account for multiple inputs and control for background characteristics such as initial endowments. Then the relative efficiency is measured as the distance from the best-fit line. This approach is also used to determine why certain districts are top performers (see section on Regression Analysis). Frontier analysis like DEA/SFA is the most complex technique and requires more data. DEA can construct multi-input/multi-output frontiers. Please refer to the previous section for more details on DEA.

Source: Igor Kheyfets, ECSH2, AFTP2 workshop, December 2013.


Allocative Efficiency – By facility unit, and/or by sector/projects/program

Allocative Efficiency is achieving optimal mix of inputs. In other words one way in which productive agents could be inefficient is they could use a sub-optimal input combination given the input prices and their marginal productivities. It is often done by looking at discretionary expenditures which cover general administrative expenditures. This could be support level functions done by a ministry or agency or it can be done for a specific project.

Comparing/benchmarking budget allocation/prioritization is one of the most commonly used techniques used to prepare allocative efficiency analysis. If sufficient input data is available comparison analysis can be done within a country, overtime within a country and across geographic locations within a country. For example PERs often try to identify inefficiencies in service delivery. So one way to do that is to compare wage expenditure to non-wage ratio and compare that within a country if regional data is available or to other comparator countries to shade light into the relative allocative inefficiency in service delivery.

The composition analysis standard tables in the previous section is a good place to start to do allocative efficiency analysis although more detailed tables by sector/subsector and program/project level may need to be produced depending on the analytical needs at hand. Trend analysis using these tables can also be used to gauge efficiency gains or losses.

Budget deviation analysis is another technique used to determine the efficiency of budget executing units and it deals with the question is budgets over- or under-spent. PERs often address the questions how good are the government units at executing the planned budget? Which issues are coming through when comparing expenditure out-turn and budgeted expenditures – by overall composition, by sectors, by program/project and by econ item? How can we explain discrepancies – due to budget management issues and/or technical capacity constraints of the implementing agency?10 The budget deviation standard tables which is the deviation between planned and actual budget expenditures- broken down by the economic, functional, or administrative classification of the budget is the first place to start when doing this type of analysis.

TABLE 7. BUDGET DEVIATION – BOOST STANDARD TABLES

Budget Deviation By Administrative Classification (Level 1)Budget Deviation By Administrative Classification (Level 2)Budget Deviation By Functional ClassificationBudget Deviation By Economic ClassificationBudget Deviation By Financing SourceBudget Deviation By Expenditure Type

10 A recent PER on Indonesia provides good examples on budget execution analysis. Please refer to “Identifying the Constraints to Budget Execution in the Infrastructure Sector”, DIPA Tracking Study, May 2012

Box 1: Four ways to measure relative efficiency

How efficient are different schools / districts / municipalities at providing quality education with the given level of resources? Identify “top performers” who are most efficient relative to others in achieving results? To answer questions like these find below a brief description of the four techniques that can be used.

Four quadrants and bivariate regression residual analysis- Simpler approaches

The four quadrants is the simplest approach followed by bivariate regression residual analysis. Both these techniques only require one measure of “inputs” and one measure of “outputs”. As demonstrated in the charts the first approach uses averages of the input and output line is against which the relative efficiency of a unit is measured. The bivariate regression approach on the other hand uses the best-fit line from which relative efficiency is measured. With this method important trends can be observed if variables are picked carefully

Multivariate regression residual and Frontier analysis- Complex approaches

Multivariate regression residual analysis need to account for multiple inputs and control for background characteristics such as initial endowments. Then the relative efficiency is measured as the distance from the best-fit line. This approach is also used to determine why certain districts are top performers (see section on Regression Analysis). Frontier analysis like DEA/SFA is the most complex technique and requires more data. DEA can construct multi-input/multi-output frontiers. Please refer to the previous section for more details on DEA.

Source: Igor Kheyfets, ECSH2, AFTP2 workshop, December 2013.


Table 8: Input and Output Variables by Sector (Needs updates, links also to ISPMS indicators)

Table B.6. Definition and Source of VariablesDefinition of Variable

Source

Output variables for educationSchool enrollment, primary (% gross) World Bank WDISchool enrollment, primary (% net) World Bank WDISchool enrollment, secondary (% gross) World Bank WDISchool enrollment, secondary (% net) World Bank WDILiteracy rate, youth total (% of people ages 15-24) World Bank WDIAverage years of school, ages 15+ Barro-Lee DatabaseFirst level complete, ages 15+ Barro-Lee Databasesecond level complete, ages 15+ Barro-Lee DatabaseLearning scores Crouch and Fasih (2004)Input variables for educationPublic education spending per capita in PPP terms, calculated World Bank WDILiteracy rate, adult total (% of people ages 15 and above) World Bank WDITeachers per pupil, equal the reciprocal of pupils per teacher World Bank WDIOutput variables for healthLife expectancy at birth, total (years) World Bank WDIImmunization, DPT (% of children ages 12-23 months) World Bank WDIImmunization, measles (% of children ages 12-23 months) World Bank WDIDisability Adjusted Life Expectancy Mathers et al (2000)Input variables for healthLiteracy rate, adult total (% of people ages 15 and above) World Bank WDIpublic spending on health per capita in PPP terms, calculated World Bank WDIpublic spending on health per capita in PPP terms, calculated World Bank WDIVariables used in the calculation World Bank WDIPupil-teacher ratio, primary World Bank WDIPublic spending on education, total (% of GDP) World Bank WDIGDP per capita, PPP (constant 1995 international $) World Bank WDIHealth expenditure, private (% of GDP) World Bank WDIHealth expenditure, public (% of GDP) World Bank WDIVariables used in the Panel Tobit regressionWages and salaries (% of total public expenditure) World Bank WDITotal government expenditure (% of GDP) World Bank WDIShare of expenditures publicly financed (public/total) World Bank WDIGDP per capita in constant 1995 US dollars World Bank WDIUrban population (% of total) World Bank WDIDummy variable for HIV/AIDS WHO mappings of diseasesGini Coefficient World Bank WDIAid (% of fiscal revenue) calculated as Official development assistanceand official aid (current US$) *official exchange rate * PPP conversion factor/ Revenue, excluding grants (current LCU)

World Bank WDI

Institutional Indicators includinga. State Failure data

b. ICRG International Country Risk Indicatorsc. Worldwide Governance Research Indicators

a. The State Failure Task Forceb. ICRG Online Websitec. Kaufmann, et al. 1999a,b and 2002

Source: http://siteresources.worldbank.org/INTQFA/Resources/EfficiencyofPublicSpendinginDevelopingCountries_MAY05.pdf

http://siteresources.worldbank.org/INTQFA/Resources/EfficiencyofPublicSpendinginDevelopingCountries_MAY05.pdf


References

Afonso, A., L. Schuknecht, and V. Tanzi (2005), “Public Sector Efficiency: An International Comparison,” Public Choice, Springer, Vol. 123(3), pp. 321-347, June.

Anand Rajaram (author), Fiscal Policy for Growth and Development, World Bank 2007

Anwar Shah (ed), Public Expenditure Analysis, World Bank, 2005

Benchmarking the Efficiency of Public Expenditure in the Russian Federation, IMF, WP07246, 2007

Dino Merotto and Leif Jensen, ““What Makes a Good Public Expenditure Review”. Presentation at AFTP2 event, November 2013.

“Government Finance Statistics Compilation Guide for Developing Countries”, IMF, September 2011

Igor Kheyfets, School Spending and Student Performance: BOOST Analysis of Resource Allocation in Serbian Primary Education. World Bank, 2012

Joe Zhu, Quantitative Models for Performance Evaluation and Benchmarking, Springer Publications, 2009

J. Harrison et al, “Medium-Term Budget Frameworks in Advanced Economies: Objectives, Design and Performance”, in M Cangiano et al (ed), Public Financial Management and Its Emerging Architecture, IMF, 2013

Lionel Demery, “Analyzing the Incidence of Public Spending” (further ref?)

Marc Robinson, “Spending Reviews in the OECD, unpublished paper, 2013

Marijn Verhoeven, Victoria Gunnarsson and Stephane Carcillo, “Education and Health in G7 Countries: Achieving Better Outcomes with Less Spending”, IMF WP 07/263

“PEFA Performance Framework at Sub National Government level”, PEFA

Peter Bogetoft and Lars Otto, Benchmarking with DEA, SFA and R, Springer Publications, 2011

Public Sector Efficiency. Evidence for New EU Member States. European Central Bank, WPS581, January 2006

Santiago Herrera and Gaobo Pang, “Efficiency of Public Spending in Developing Countries: An Efficiency Frontier Approach” , May 2005

Stefano Paternostro, Anand Rajaram and Erwin Tiongson, “How Does the Composition of Public Spending Matter?” World Bank WPS 3555, 2005


ANNEXES

Annex 1: Institutional Coverage in BOOST

In addition to ensuring high analytical quality of the PER, having a clear definition of the public sector and therefore understanding the data coverage becomes particularly important when BOOST databases are being used to support expenditure dialog with the government and development partners over a certain period of time, as well as country comparison or benchmarking technique requires consistency in coverage across units and over time.

BOOST databases explicitly identify the data coverage of a database which is based on the definition in the GFSM2012 (See BOOST Standards in Annex 4). To get a clearer picture of the institutional coverage, the GFS institutional tables are used as a guide. The GFS institutional table gives a list of the units of the general government for all countries covered in the GFSY. It also provides information on the coverage of the GFSY database. This information is in fact used as an ex-ante assessment of the coverage to be expected during the identification step of the BOOST process.

The institutional coverage of BOOSTs varies from country to country and region to region. ECA country BOOSTs as expected, have the highest coverage at the general government level. Out of the 11 countries BOOSTs delivered in FY13, data coverage for seven of them is general government. The rest cover the central government. On the contrary, in AFR countries, the highest coverage is in most cases the budgetary central government with a few countries covering some of the extra budgetary units and even fewer covering the local governments. It is worth noting however, most of these African countries have deconcentrated local governments which do not have their own budgets and rely on transfers from the budgetary central government. See definition in Annex2. However, as a measure of equity or spatial distribution, the utmost effort is made to capture expenditure by geographic location. For example Kenya, Niger and Togo have such data included within the BOOST.

Figure: The Public Sector and its subsectors

Public Sector

General Government

Central Government

Budgetary Central

Government

Extrabudgetary Funds

Social Security Funds

State Government

Local Government

Public Corporations

Financial

Public Nonmonetary

Corps

Public Monetary

Corps

Nonfinancial


Definitions

Public Sector: All general government units and all corporations owned or controlled by government units.

General Government (GG): All government units and nonmarket non-profit entities (created for the purpose of producing goods and services). controlled by government units. The General government sector is made up of the Central government, State government, and Local govt.

Central Government (CG): is made up of the Budgetary Central Government, Extrabudgetary units and Social security funds.

Budgetary CG: All government entities fully covered by the country’s general budget, usually approved by the national legislative. These entities are usual referred to as votes, budget heads, or budget execution units. E.g. Entities of the executive, legislative, and judiciary branches of government etc..

Extrabudgetary Units: All government unities with separate budgets. Eg. Public universities, research institutes, government boards, funds, and regulatory bodies.

Social Security Funds: All government units that manage social security schemes. E.g. National social security institutes, national pension funds.

State Government (SG): All government units with power and responsibility over large geographic areas. Eg. States, provinces, or regions.

Local Government (LG): All government units with power and responsibility over small geographic areas. Eg. Municipalities, villages.

Source: IMF GFS Compiling Guide for Developing Countries.


Annex 2: Devolved Local Government Entities versus Deconcentrated Local Administration Bodies

Source: PEFA Performance Framework at Sub National Government level


Annex 3: BOOST Core Variable Definitions Aligned With Internationally Agreed Standards

The core BOOST database variables are based on the three main dimensions of expenditure analysis. These are the Economic type variable, Function/sector variables, and Administrative variable11. While a BOOST database is following the country’s own set of classification, as outlined usually in the Chart of Account, the BOOST standard template is aligned with the

Economic type variable provides information about the category or type of expense incurred. Typically in BOOSTs in addition to the main category of an expenditure item, there will be subcategories and these subcategories are further subdivided into other categories. There can be as many as 6 sub-levels of the econ level depending on the classification system the country uses. The top level economic classification, econ1 refers to the main category of expense incurred and econ2 refers to lower level referring to subcategories of the top-level and so on. For example in the GFSM2001, expense has up to 3 levels of subcategories. If we look at wages, econ1 would correspond to “21 Compensation of employees”, econ2 would be “211 Wages and Salaries” and econ3 would be “2111 Wages and Salaries in Cash”.

Function variable provides information about the sector or purpose for which an expense was incurred. Similar to economic classification the main category of the functional classification has subcategories and these subcategories are further subdivided into other categories and so forth. The number of levels depends on the classification system the country uses. The top level functional classification, func1, refers to the division (sector) for which an expense was incurred and func2 refers to second level functional classification referring to group (sub-sector) and so on. For example in the COFOG system which is also GFSM2001 compatible, function has up to 3 levels of subcategories. If we look at the health sector for example, func1 would correspond to “707 Health”, func2 would correspond to “7072 Outpatient services” and func3 would correspond to “70721 General medical services”.

The COFOG system of functional classification is the most widely used system. Most countries that compile data by function use this system. Some countries do not have expenditure data by function and therefore MTEF categories or other set of classifications are used instead.

Administrative variable provides information about which spending unit incurred the expense. The organization of spending units varies from country to country and there are no international standards. However, typically, Admin1 corresponds to top level of administrative classification, referring to the budget or government level that incurred the expense (e.g. central or local government) and Admin2 corresponds to second level of administrative classification, typically referring to the top-level spending unit (e.g., ministry or agency at the central level, top-level subnational authority at the local level).

11 In addition to these core variables, BOOST databases also include the expenditure type (recurrent, personnel, capital, non-personnel) and the financing source variables.


Appendix 4: BOOST Standard and Quality Check Table

Item BOOST Standards Planned at launch of BOOST activity

Completed

Coverage: Public sector units

General government units

Public corporations, owned or controlled by government

Sub-national tiers (how many, intermediate situations?)

All units included in the database

Disaggregate data to the lowest public unit, as available

(Use GFSY Institutional list to get an understanding of GG’ structure and the potential scope of BOOST)

Appraised and agreed with client in relation to PER/country work program.

The suggested BOOST proposal to reflect:

-The planned design of the database. Fit to PER and role in country dialog

-Are all BOOST standards met. Explain why this may not be the case.

-IFMIS status: Is an IFMIS available to provide similar analytical support? Any assistance envisaged to support the client in improving/enhancing such functionality?

For further details on requirements at launch, please refer to the Guidance Note to Country Teams (March 2013)

V.1 delivered and signed-off by government:

Coverage; classification; fund sources and periods as planned?

Reconciled with official figures

Specify deviances (coverage; reconciliation issues > 1%)

Documentation delivered to Government, and posted on BOOST iTeam site:

o User Manual, including CoA

o Reconciliation files

o ‘Do files’

Time-period 3-5 years, incl latest FY

Classification of Expenses:

Type: Current, capital

Type: Econ

Functional /program

Disaggregate data to lowest level of functions/program/projects

BOOST pre-defined template

Transactions Budget/Approved/Actuals

Classification: Revenues Include, if relevant and value-added, as compared to using GFS

Indicate why Revenue data has been included, and any data quality issues

Any comments as regards completed version versus planned, if appropriate

Funds – Extrabudgetary (i.e., foreign aid, social funds, other extra-budgetary funds)

Include, as appropriate Indicate which funds, why, and any data quality issues

Any comments as regards completed version versus planned, if appropriate

Sector Performance data

Complete expenditure version of BOOST, before linkages to additional datasets are considered established

GDP and census data could be included, if accessible. Sector input, output and performance data also.

Any plans to expand ta in next database in next version?

Was GDP and census data included in final version?

House hold data/ Socioeconomic data/Other relevant data set

N/A


expenditure analysis supported by...

Documents