1

Techniques for Evaluating Public Policies in Developing Countries (DCs)

Luiz Awazu Pereira da Silva

Ministry of Finance (Brazil)

University of Palma de Mallorca

February 4th 2005

Based on “The Impact of of Economic Policies on Poverty and Distribution”

by François Bourguignon, Luiz A. Pereira da Silva, eds.

The World Bank, Oxford University Press (2003)

2

Outline of this Presentation

• Policy Challenges for DCs: the evaluation of public expenditures and economic policies from aggregate macro to micro “distribution/poverty”

• Framework for evaluating public policies• Part 1 : Microeconomic evaluation techniques• Part 2 : Macro evaluation techniques (micro-macro

linkages)• Future directions for more complex techniques• Practical difficulties for DCs (institutional set-up)

3

Evaluation techniques in DCs have evolved

together with:

1. Development Economics, e.g., goals and theory

2. Data Availability and Econometric Techniques, e.g., HHS, Firm data

3. Modeling techniques, e.g., CBA, CGEs

4. Challenges of Globalization , e.g., political economy in DCs

5. Policy Challenges for DCs now: linking the evaluation of public expenditures, economic policies to “distribution/poverty”

4

Policy Challenges: 1950-1970s “old” vision of Evaluation of Public Policies

1. Maximize Aggregate Growth and Minimize Risks of BoP crisis, under old BW international financial arquitecture (fixed ERs, K controls, etc.) and import-substituting development strategies

2. Evaluation = from aggregate growth models level of external savings needed for target growth, Kflows (public), find the best set of projects doing project analysis in partial equilibrium (CBA) using if need be shadow-pricing

3. Most DCs with institutional structure for evaluation with strong Min. Planning and Project Analysis Unit (World Bank, IMF) and MoF in control of ER, BoP

5

Policy Results (1970s-1980s) of “old” vision of Evaluation of Public Policies

1. Some successes but also booms and busts Policy instability, structural adjustments, external vulnerability

2. Fiscal and/or BoP crises high or hyper inflation, devaluations

3. Poverty and distributional challenges Political instability

4. Shift in institutional balance: MoFs vs. Planning

5. Obsolescence of most project analysis units and of CBA in planning & evaluation methods

6

Policy Challenges 1990s-2000s (1): the most common economic policies and structural reforms in DCs

change in scope for Evaluation of Public Policies

1. Macro-economic policies, ST Fiscal monetary policy stance, exchange rate regime, public debt management strategy, etc.

2. Public Expenditure and Revenue, Micro-social-policies, ST - MTTax policy reform, composition of public expenditure, design of social programs (CCTs) civil service reform, pension reform, decentralization

3. Structural Reforms, LTTrade liberalization, liberalization of specific markets, financial sector reforms, improving the investment climate, land reform, privatization etc.

7

Within point 2., in particular, most common policy challenge for DCs is Evaluation of Public Expenditure

a) Adequate Aggregate Level Is Deficit, Public debt Sustainable?– Definition of PS, Hidden Contingent Liabilities?– Methodology (mechanical ratios or stochastic)?

b) Adequate dynamics, counter-cyclicality of public spending?– Macro policy fiscal stance and “credibility” – Programs to off-set effect of volatility, financial crises

c) Is there crowding-out or crowding in private/public?– Complementarity of PE, Externalities w/ private sector?– Market failures? Lobbies?

d) Are allocations adequate?– Inter-sectoral allocation, Capacity-building– Input mix (Capital/Recurrent; Wage/Non-Wage)

e) What are Poverty and Distributional Impact of PE?– Cost-Efficiency of social programs – Outcome indicators, Evaluation methods

8

Illustration of Typical Set of Challenges for DCs: Example of Brazil

• High PS Debt and unsustainable Tax Burden Need to Generate Primary Fiscal Surpluses

• High and persistent inequality, poverty and Budget rigidity Need to Improve Targeting of Social Policies

9

30,4 33,6 33,2

25,3 24,5 23,5 22,0

18,0 17,9 16,8 17,1

9,7 11,3 10,7

9,0 9,4 9,5

33,6

10,1

8,97,15,85,07,6

0,0

10,0

20,0

30,0

40,0

50,0

60,0

70,0

80,0

90,0

100,0

2002 2003 2004 2005

Outras Discricionárias

Despesas Saúde eEducação

Outras Obrigatórias

Transferências a Estadose Municípios

Despesas de Pessoal

DespesasPrevidenciárias

25,0030,0035,0040,0045,00

50,0055,0060,0065,00

jan/91 jan/95 jan/99 jan/03

24

26

28

30

32

34

36

Public Debt/GDP (%) Tax Burden/GDP (%)

Tax Burden /GDP (%)Debt/GDP (%)

-2

-1

0

1

2

3

4

5

1995 1996 1997 1998 1999 2000 2001 2002 2003 2004

-2

-1

0

1

2

3

4

5Primary Balance (in perent of GDP)

Figure 2: Time Series of Inequality in Brazil

0

10

20

30

40

50

60

70

80

90

100

1977 1978 1979 1981 1983 1985 1987 1989 1993 1995 1997 1999

Year

Gin

i

Source: Pesquisa Nacional por Amostra de Domicílios

Public Debt/GDP

Tax/GDP

Gini

10

• Scope and objective, challenges increase: evaluate the economic feasibility of public programs and policies and their overall ‘development” impact.

• Aggregate and first principle analysis insufficient: heterogeneity of individuals and households, microeconomic behavior do not add up into aggregate nor “average”, specificities of economic structures and local political economy, transmission of shocks and policies

• Policy objectives and social demand increasingly focusing on distributional effects and poverty reduction, essentially micro concepts (e.g., Post-WC IFIs, new types of Governments, etc.)

• Micro data bases (household surveys HHS) increasingly available as the natural analytical environment for distributional and poverty analysis

• Hence the natural idea to link the effect of economic policies to the corresponding changes in the income and/or expenditure of individuals, households, social groups and the poor in particular

• Impact evaluation allows to think about “scaling-up” and pro-poor, redistributive development strategies

Evaluation of Policies in DCs with these new policy challenges: broader range of micro

programs to macro policies

11

Important dimensions in the evaluation of Public Policies in DCs with these new policy challenges

• Counterfactual is needed (the world with and without the program or policy being evaluated, sometimes difficult)

• Ex-ante or ex-post (ex-ante evaluates the design of non-existing programs and policies, ex-post focus on outcomes)

• Partial or General Equilibrium (taking or not into account the effect of programs and policies on price systems and economic equilibria)

• Behavioral or “Arithmetic” (based or not into some representation of economic behavior of agents “reacting” to the program or policy )

12

);(/);(

)0()1(:

pCPAELwRELwy

PyPyy

iiiiiiiii

iiiii

yi : real income

wi : wage rate

Li : labor supply

Ei : self-employment, non-wage income

Ri : net transfer income

Ai : socio-economic characteristics

Ci : consumption characteristics

household-specific P price indexp : general price index

Framework for EvaluationDefine impact for individual i as the difference in income yi with and

without the program, denominated yi :

13

Household Survey (HHS), i individual households

Compare the distribution of y|P=1 with the distribution of y|P=0.Calculate changes in inequality or poverty across the two distributionsDifferent tools/methods differ in how they construct the counterfactual distribution and the data that are neededRank results according to some agreed upon rule and/or objective

p

prices

w wage

L employt.

R transfers

A households character.

)0()1(:

);(/);(

iiiii

iiiiiiiii

PyPyy

pCPAELwRELwy

Program or policy will shock one or more components that explain theindividual income yi

Evaluation of Program and/or Economic Policy

14

An illustration of one criteria for evaluationAn incidence effect curve (say on income/ expenditure changes) showing the

percent change in per capita income of a macroeconomic policy (here, Indonesia financial crisis, changes in pc income by percentiles of the

distribution)

FU

LL

perc

ent ch

ange in

per

capita

inco

me

Simdev1 - Without Rerankingcentiles

0 50 100

-14

-12

-10

-8

-6

-4

-2

0

poor wealthy

15

Part 1 “Microeconomic techniques”

1. Average Incidence Analysis a) Tax Incidence Analysis (Sahn & Younger)

b) Public Expenditure Incidence Analysis (Demery)

2. Marginal Incidence Analysisa) Behavioral response to changes (Van de Walle)

b) Poverty mapping (Lanjouw)

3. Impact Evaluation (randomization, matching, double-dif)a) Ex-post (Ravallion)

b) Ex-ante (Bourguignon & Ferreira)

4. Data and Measurement (not covered here)

a) Multi-topic Household Surveys (Scott)

b) Qualitative surveys (Rao & Woolcock)

c) Performance in Service Delivery (Dehn, Reinikka & Svensson)

16

Average Incidence Analysis (Sahn & Younger; Demery)

• Suitable for taxes or public expenditures.

• Aims to answer: “Who pays for / receives how much?”

• Counterfactual is simply

• So that

• This is equivalent to assuming:– No behavioral response (perfectly inelastic demand for goods, perfectly

inelastic supply of factors. • Fine for marginal changes.

• But only a first order approximation to large taxes and/or transfers.

– No general equilibrium effects.

0ii Py iii sPy )1(

iiiiii sPyPyy )0()1(:

17

Average Incidence Analysis (Sahn & Younger; Demery)

• A practical example from education expenditures: the incidence of public spending in schooling category i which accrues to group j depends on:– groups j’s relative enrolment rates across schooling types i.– Relative spending across categories i.

I

iiij

I

i

i

i

ijj

I

i i

iijj

seS

S

E

Ex

E

SEX

11

1

Once again: purely arithmetic. No behavioral response, no gen. eq. effects.

18

Indonesia, Benefit Incidence of Education Spending, 1989

0

10

20

30

40

50

60

70

80

90

100

0 20 40 60 80 100

Cumulative distribution of population (%)

P rimary

J unior secondary

Senior secondary

Tertiary

All education

Expenditure

19

Benefit Incidence of Education SpendingFig C1: Incidence of Public Expenditure on Primary Education by Income Quintiles

0%10%20%30%40%50%60%70%80%90%

100%

Ecuador(1994)

Brazil(2000)

Jamaica(1992)

Colombia(1997)

Argentina(1994)

Chile(1992)

Malaysia(1989)

Uruguay(1995)

Q5

Q4

Q3

Q2

Q1

Fig. C3: Incidence of Public Expenditure on Tertiary Education by Income Quintiles

0%

20%

40%

60%

80%

100%

Brazil (2000) Argentina(1994)

Ecuador(1994)

Colombia(1997)

Uruguay(1995)

Chile (1992) Malaysia(1989)

Jamaica(1992)

Q5

Q4

Q3

Q2

Q1

20

Marginal Incidence Analysis(van de Walle)

• Suitable for taxes or public expenditures.• Aims to answer: “How has the distribution of tax burden /

program benefits changed in the recent past?”

• Assumptions are less demanding than for average incidence analysis

• Requires either panel or repeated cross-section data.– Although some have suggested using spatial variation in programs

/ taxes to proxy for temporal variation (Lanjouw & Ravallion)

1)1()0()1(: 11 tittitiiiii PyPyPyPyy

21

Poverty (and Expenditure) Maps (Lanjouw)

Reliable poverty maps = combining sample survey data with census data to yield predicted poverty rates for all households covered by the census.

 1) Estimating Models of Consumption A model of consumption or standard of living using household survey data is estimated using the variables which are available both in the census and in the survey.

 2) Predicting Poverty. The parameter estimates from the regressions (using the full household sample) are used to predict consumption or standard of living in the census data. For each household in the census, the parameter estimates from the applicable regression (conditional on geographical location) are combined with the household's characteristics in order to obtain an imputed value for per capita consumption expenditure.

3) Comparing with the map of public expenditure spending. The poverty map that is obtained can then be “super-imposed” on the map for any public spending

22

468 - 1707.21707.2 - 2216.12216.1 - 2639.22639.2 - 4040.84040.8 - 13777.3

Madagascar: Per capita health spending per district

Madagascar: Incidence of Poverty per district

0.191 - 0.5660.566 - 0.6960.696 - 0.7660.766 - 0.8330.833 - 0.946

23

Ex-Post Evaluation of public programs (Ravallion )

 Randomization: Only a random sample is allowed to participate to the program. “Randomized out” group is the counterfactual.

• Experiments may be either designed or natural: Progresa vs. Bolsa Alimentacao• Delayed participation of part of the population may be used to reach the same objective.

– But beware of anticipation bias…

• Randomization ensures that treatment and control groups are “alike” along all dimensions relevant for program selection, observable and unobservable.

• Takes into account all partial and general equilibrium effects of program, as well as all behavioral responses. Ideal for measuring. Not so great at “explaining”.

 

0,1,)0()1(: PCjyEPTiyEPyPyy jiiiiiTi

24

Ex-Post Evaluation of public programs (Ravallion )

  Matching: When no randomization is available, must construct a comparison group. Objective is to approximate a control: match participants to non-participants from a larger survey, on the basis of similarities in observed characteristics.

• The most common method is to match people on the basis of their ex-ante probability to participate to the program, these probabilities depending on their characteristics as well as those of the communities they live in (Propensity-score matching):

 

• Draws on seminal work by Rosenbaum and Rubin (1983)

iii XPXP 1Pr

C

jijjijiiiiiTi YXPWPyPyPyy

101)0()1(:

25

Ex-Post Evaluation of public programs (Ravallion )

• Key problem with non-experimental data is that if any variables which affect selection into the program are not observed, they can not be included in X, and the approximation to the ideal counterfactual fails.

• If two waves of data are available in time (I.e. with a baseline survey and a follow-up survey), then at least the time-invariant unobserved variables may be netted out through double differencing:

 

0,0,

0,1,)0()1(:

1

1

PDjyEPDjyE

PTiyEPTiyEPyPyy

jtjt

ititiiiiTi

26

Ex-Ante Evaluation of public programs (Bourguignon & Ferreira)

• Aims to simulate programs or program reforms which are not yet in existence. Complement to ex-post approach.

• In this approach, the treatment –rather than the control – is the counterfactual.

• The counterfactual incomes may be generated through:– Arithmetic micro-simulations (based on program rules)

– Behavioral micro-simulations (based on a model)

01)0()1(: PyPyPyPyyii

ssiiiiTi

27

Ex-Ante Evaluation of public programs (Bourguignon & Ferreira)

Original Bolsa escola's program Scenario 1 Scenario 2 Scenario 3 Scenario 4 Scenario 5Poverty measures

Poverty headcount 30.1% 28.8% 27.5% 24.6% 27.7% 28.8% 28.9%Poverty gap 13.2% 11.9% 10.8% 8.8% 10.9% 11.9% 12.0%Total square deviation from poverty line 7.9% 6.8% 5.9% 4.6% 6.0% 6.8% 6.8%

Annual cost of the program (million Reais) 2076 4201 8487 3905 2549 2009Source: PNAD/IBGE 1999 and author's calculation

note: Scenario 1: transfer equal R$30, maximum per household R$90 and means test R$90 Scenario 2: transfer equal R$60, maximum per household R$180 and means test R$90 Scenario 3: diferent values for each age, no household ceiling and means test R$90 Scenario 4: transfer equal R$15, maximum per household R$45 and means test R$120 Scenario 5: Bolsa escola without conditionality

Table 2. Simulated distributional effect of alternative specifications of the conditional cash transfer program

28

Public Expenditure Tracking Surveys Dehn, Reinnika and Svensson [2001]

  The need for special Public Expenditure Tracking Survey (PETS) comes primarily from the increasing evidence that budget allocations to social services (the basis for traditional “benefice incidence analysis”) are not consistent with the casual observation of what is really happening in the ground.   More evidence of government failures (corruption, leakages). Little known about transformation of budgets into services (the public sector

production function) Household surveys show that quality of service important determinant of demand

  PETS gathers information on “flow of funds” within the public sector from: Participatory poverty assessments Service delivery surveys of households Public officials surveys

29

Example : Education sector in Uganda 1996

• Data from 250 schools and administrative units

• Only 13 percent of intended capitation grant actually reached schools (1991-95).

• Mass information campaign by Ministry of Finance (the press, posters)

• Follow-up surveys (PETS, provider surveys, integrity surveys, etc.)

• High leakage has also been found in other countries (Tanzania, Ghana, Zambia, Peru)

Leakage of non-wage capitation grant to schools

0%

20%

40%

60%

80%

100%

1991-1995 2001

30

Part 2 “Macroeconomic techniques”, from robust to more speculative….

1. Standard RHG approaches to macro-micro linkage: a) "Micro-accounting"/RHG approach based on aggregate macro predictions

(PovStat-SimSip-PAMS)

b) The disaggregated SAM-CGE/RHG approach (Adelman-Robinson, Bourguignon and al. in the "Maquette“, Loefgren, Robinson or Agenor and al. with IMMPA.)

2. Top-down "micro-simulation" approaches (micro-macro linkages)a) "Micro-accounting modules" linked to disaggregated macro models (Chen-

Ravallion, McCulloch-Winters)

b) "Micro-simulation modules" linked to disaggregated macro models (Bourguignon-Robilliard-Robinson, Ferreira-Leite-Pereira-Picchetti, Cogneau-Robilliard-van der Mensbrugghe)

3.Other issues for research and applications:

a) Fully integrated models (Townsend, Heckman, Browning-Hansen-Heckman)

b) Accounting for general equilibrium effects of public expenditure programs

c) Dynamic modeling and the proper treatment of growth

31

Household Survey (HHS), i individual households

iii

iiiiiiiii

yyy

pCPAELwRELwy

1,

);(/);(

p

prices

w wage

L employt.

R transfers

A households character.

Macro framework, general/partial equilibrium

Evaluation of macro economic policies.Macro to micro linkages

Instead of « exogenous and independent » shocks like in Part 1, in Part 2 use « endogenous and  dependent» shocks to 'microsimulate' the effect of policies on all

individuals in the micro data sets, and the poor some consistency constraints will be « binding » (e.g., budget envelope for social programs, real GDP growth, etc.)

LAVs =Linkage Aggregate Variables

32

Evaluation of macro economic policies. General approaches and problems

• Before/after : evaluation based on the observation of changes in standards of living y inputed to some policy change affecting « jointly » (L, R, w, p, etc.)

Problems Before/after evaluation techniques include other changes (X) than policy (L, R,..) being evaluated difficulty to evaluate alternative policies by attributing changes to the effects of policy

• Counterfactuals: Ideally possible to smulate changes in standard of living due to alternative macroeconomic policies, e.g., (E, r) during a BoP crisis

Problems Program design/implementation in crisis time - credibility?

• Top-to-Bottom approach: Linking macro to micro data using Linkage Aggregate Variables (LAVs) to simulate macro-to-micro effects consistently

Problems Weakest link (macro? micro?), garbage in, garbage out

and…

33

Evaluation of macro economic policies. General approaches and problems

1995 Nobel Laureate in Economics Robert E. Lucas Jr.

34

1. Standard RHG approaches to a macro-micro linkage

a) "Micro-accounting"/RHG and aggregate macro predictions i. An elementary procedure

Growth rate of output in sector k : gk

Growth rate of employment in sector k : nk

Effect on distribution (using a micro data base) given by:

Multiply income of all hhs in sector k (or RHG) by:

Reweigh all hhs in sector k (or RHG) by:

Evaluate new distribution, all poverty and inequality measures

ii. More elaborated models

Change arbitrarily distribution within sector k

Change distribution endogeneously by distinguishing labor/non-labor income, so that gk is not uniform anymore (PAMS, Pereira da Silva and alii)

iii. Main problems = very much heterogeneity still missing + likely strong selection behind nk

k

k

n

g

1

1

n

nk

1

1

35

i. Basic idea Aggregation properties allow separating the household population into

groups. Only the aggregate behavior of these groups matters for the (general) equilibrium of the economy.

Overall distribution of income or earnings studied under the assumption that the distribution of 'relative' income within Representative Household Groups is constant – as given in a household survey - and also that their demographic weight is given.

These approaches thus essentially focus on changes in the distribution between RHGs.

ii. Working of standard (CGE) models (e.g., Robinson)Full integration of RHGs' behavior within the model:

interaction of heterogeneous behavior in labor supply, consumption, savings, portfolio choice in the household sector with the production side and public policies through good and factor markets

1. Standard RHG approaches to a macro-micro linkage

b) The SAM-CGE/RHG approach

36

iii. Recent and current extensionsIntroduction of the monetary and financial sectors (IMMPA, Agenor and

alii, Lewis & Robinson):Limited by current theoretical knowledge of the working of financial markets.

Introducing imperfect competition in different ways : Economies of scale, economies of scope, oligopolistic behavior, bargaining on the labor market, …

Dynamics represented through a sequence of temporary equilibria linked by asset accumulation and demographics

1. Standard RHG approaches to a macro-micro linkage

b) The SAM-CGE/RHG approach

37

iii. Limitations

Miss 'true' intertemporal behavior and important sources of growth ( public expenditures in particular)

Constant “within RHG” distribution limitative in a dynamic framework All improvements over simple static Walrasian case make all the more acute

the issue of empirically 'calibrating' the model and the confidence one may have on predictions

The 'black box' risk

iv. Final Remarks

These techniques are 'simple', yet they are not widely used

They capture only the 'between' (RHG) dimension of distributional changes, which empirically proves limitative

They are ill adapted to the distributional aspects of growth

1. Standard RHG approaches to the macro-micro linkage

b) The SAM-CGE/RHG approach

38

2. Top-down "micro-simulation" approach within a macro-micro linkage approach

Macro model

Linkage AggregatedVariables

(prices, wages, employment levels)

Household income micro-simulation model

39

2. Top-down "micro-simulation" approach within a macro-

micro linkage approach

Two distinct approaches to micro-module:

- "micro-accounting" : no explicit change in behavior (envelope theorem argument), e.g., Chen-Ravallion

- "micro-simulation" : change in behavior, possibly linked to (labor) market imperfections, e.g., Robillard, Bourguignon, Robinson and Ferreira, Leite, Pereira da Silva, Picchetti

LAVs from above

Household income micro-simulation model

40

i. Basic principles

p, w, R obtained from macro model (CGE or other)

observed in reference household surveyStandard envelope theorem:

Where yi and stand for welfare income equivalent

"Mobility" and distribution analysis can then be conducted on the set of yi and

iii

iiiiiii

yyy

Rw

wwL

p

ppcqy

01

00000 )(

0000 ,,, iiii Lcqy

1iy

1iy

2. Top-down "micro-simulation" approach a) "Micro-accounting modules" linked to disaggregated macro models

41

ii. Example : Evaluating the distributional consequences of WTO accession for China

Representing WTO Accession for China

• Reduce China’s own protection to the lesser of the tariff binding or the 2001 applied rate

• Effect of trade reforms in China since 1995 viewed as part of China’s WTO accession process (counterfactual?)

• Separate impacts of tariff reductions to 2001 and the remaining reductions to ‘2007’

• Elimination of textile & clothing quotas for China’s exports

• Removal of agricultural export subsidies for feedgrains (32%) and plant-based fibers (10%) (Huang and Rozelle, 2002).

• Liberalization of the service sectors (Francois, 2002)

42

Example Incidence Curve from Chen and Ravallion (China accession to WTO)

43

2. Top-down "micro-simulation" approacha) "Micro-simulation modules" linked to disaggregated macro models

i. Micro-simulation model, basic idea

• Micro-simulation equivalent to introducing imperfect labor markets and occupation allocation models in previous framework. More behavorial content than micro-accounting

• Econometric model of household income is estimated allowing for full individual heterogeneity

– Income model (individual households)

– Occupational choice (e.g., multi-logit)

• Simulates the effect on household income of modifying a subset of this model in accordance with predictions of the macro-model.

44

ii. Link with macro model (CGE or other): counterfactual analysis

Linkage aggregate variables (LAVs) given by macro model : wages, prices, employment levels by status and labor segment

Consistency 1: apply price changes as in accounting approach

Consistency 2 : Make occupational status consistent with macro employment levels by changing multi-logit intercepts

Analogy with the operation of 'grossing up' a sample

No feedback = no explicit link with actual prices in macro model

45

iii. Summing-Up: layer structure macro-micro linkages approach

From what precedes, proceeding top-down with three successive layers:

-Aggregate model determining the standard macro aggregates (GDP, price level, exchange rate, interest rate), possibly in a dynamic way

-Disaggregated real CGE-type model, using the variables of the aggregate model as an input

-Micro-simulation module using output of previous models as linkage variables to make micro-simulation consistent with macro counterfactuals.

46

Household Survey (HHS), i individual households, Macro "consistent" changes in real household incomes and change in the distribution of welfare

(yi) with poverty line, z, indicator of poverty Pi for each household i and indicators of within-group inequality (e.g., Gini, etc.)

),(),(,

);(/);(

1, AYgwYfLyyy

pCPAELwRELwy

iiiii

iiiiiiiii

Sectoral Disaggregation, Factor Markets Linkage Aggregate VarFor k representative groups of households

kkkk PwLy ,,,

General Equilibrium Macroeconomic ModelCGE, Macro-Econometric

Recall: Top-down "micro-simulation" approach Objective: reality test can approach replicate real outcomes (HHS)?

Layer 1: Macro

Layer 2: Meso

Layer 3: Micro

47

2. Top-down "micro-simulation" approach vs. standard CGE/RGH approach and actual outcomes

iv. Comparing the top-down “microsimulation” approach with actual outcomes and the GCE/RHG approach: what is more accurate?

As a test, we compare counterfactual distributions obtained from the micro-macro model (Brazil) with actual outcomes from an existing HHS and then with the CGE/RH approach

As a test, we compare counterfactual distributions obtained from the Indonesian CGE and the Brazilian micro-macro models:

a) Under the assumption that distribution of income within RHG (defined by the occupation of HH head) is constant

b) With the top-down micro-simulation framework shown earlier.

48

Brazil Results: Aggregate Poverty and Inequality Indices (on aggregate, good results)

Departure: Actual data from the HHS (PNAD)

Simulated by the Macro & Micro

Models (B)

Comparison: Actual data from the HHS (PNAD)

(a) (b) (c) (b)/(c)P0 Poverty Headcount

28.1% 29.8% 29.4% 1.013

P1 Poverty Gap 11.6% 12.6% 12.3% 1.026

p2 6.5% 7.1% 6.9% 1.037

e0 0.662 0.655 0.649 1.010

e1 0.715 0.695 0.695 1.000

e2 1.731 1.630 1.563 1.043

Gini 0.593 0.588 0.588 1.001

Mean of HPCI in R$ / month

257.30 260.46 257.75 1.011 (0.9971)

note 1: ratio computed using real terms

Brazil - Nominal Terms

49

Example 1: Brazil, 1999 Financial Crisis, Results of Simulationnominal changes in per capita income after floating ER

Figure 7 - Comparison betweenActual Observed Changes &

Experiment 1 - using Representative Households Groups (RHG)Experiment 2- using Pure Micro Simulation model

Experiment 3 - using Full Macro-Micro Linkage model

-0.12

-0.1

-0.08

-0.06

-0.04

-0.02

0

0.02

0.04

0.06

0.08

0 10 20 30 40 50 60 70 80 90 100

Percentiles

Lo

g d

iffe

ren

ce

Actual Experiment 1 - RHG

Experiment 2 - Pure Micro Simulation Experiment 3 - Full Macro-Micro Linkage

Percent changes between 1999 and 1998 in Nominal Income (in Reais, R$) / Month for each percentile of the distribution in Brazil

50

Example 2:RHG vs. Micro-simulation in the Indonesian model

FULL (microsimulation) and RHG without and with reranking

perc

ent c

hang

e in

per

cap

ita in

com

e

Simdev1 - Without Rerankingcentiles

FULL RHG

0 50 100

-14

-12

-10

-8

-6

-4

-2

0

perc

ent c

hang

e in

per

cap

ita in

com

e

Simdev1 - With Rerankingcentiles

FULL RHG

0 50 100

-14

-12

-10

-8

-6

-4

-2

0

Conclusions: 1. Aggregate results good, through complex LAV procedure2. counterfactuals are indeed different and macro-micro with

microsimulation approach closer to actual outcome than RHG approach

51

2. Top-down "micro-simulation" approach

Macro model

Linkage AggregatedVariables

(prices, wages, employment levels)

Household income micro-simulation model

vii. Final Remarks: introducing feedbacks

Feedback, e.g., micro-transfers, minimum wage

52

a) Fully integrated modelsFull heterogeneity of households estimated through panel data and interactions between

them in labor and asset markets : Heckman, Townsend, Browning, Hansen and Heckman. (Necessarily limited detail in structure of productive sector presently makes this approach unfit to the study of many development issues).

b) Taking into account general equilibrium effects of public expenditure programs

Spending on education, health or cash/in kind transfers to households has no direct productive effect in standard CGE or macro-econometric modeling.

Possible to analyze distributional effect using microsimulation framework if some behavior is introduced – demand for schooling or health services.

But two difficulties arise: i) most actual effects on distribution will be in the long-run (when kids will be adult); ii) initial policies likely to generate future general equilibrium effects at macro level (earning structure, growth rate) depending on the demand side of the economy

We are presently not well equipped to handle these points

3. Other issues on the techniques….

53

c) the issue of dynamic modeling and the proper treatment of growth

Micro-simulation techniques largely remain comparisons of two cross-sections of households: they describe what happens to a individual household which is itself representative of other actual households

Dynamic modeling may involve simulating what happens to a given household after some policy has been decided at the macro level!

Dynamic micro-simulation models used by demographers would permit to go in that direction. Also, integrated models, alluded to above, follow this kind of approach.

These are not small undertakings. Yet it is necessary to continue research in that direction to be able to say something on the long run distributional effects of growth and macro volatility and some aspects of growth policies.

3. Other issues on the techniques….

54

Final: Practical considerations: Usage of the techniques to meet DCs policy challenges, if, when, where and how• Are these techniques for evaluation of public policies used? Some of

these techniques are 'simple', yet not all of them are widely used, why? Costs (training) and institutional implications for Ministries and agencies (Finance vs. Planning, political economy of budget process, etc…). Aid agencies (IDA, DFID, AFD, etc…) promoting evaluation, (e.g. F. Bourguignon’s DIME group)

• When these techniques are used, are they useful for policy-makers? They capture only the 'between' (RHG) dimension of distributional changes, which empirically proves limitative. They are ill adapted to the distributional aspects of growth, but important in putting broader (poverty) perspective to decisions

• Where are these techniques for evaluation of public policies used? Examples below

55

Usage of these “techniques” in connection with policy-making (with external assistance)

Average Incidence Analysis (Tax Incidence Analysis and Public Expenditure Incidence Analysis)

Most OECD countries. Also in many DCs particularly IDA [Ghana (ISSER), Madagascar (INSTAT & Cornell Univ.), Uganda (EPRC)]

Marginal Incidence AnalysisMany OECD countries and India, using NSS 1994; Indonesia, using

SUSENAS 1981 & 1987; Vietnam using panel from VLSS 1993 & 1998; Argentina,using public spending and census data, Ministry of labor team; Brazil using PNADs

Poverty mapping Many OECD countries Ecuador, Bolivia, Mexico, Panama, Nicaragua, Guatemala, South Africa, Madagascar, Kenya, Uganda, Malawi, Mozambique, Tanzania, Bulgaria, Albania, Thailand, Vietnam, Cambodia, Indonesia, China., Brazil, etc.

Ex-post impact evaluation methods (randomization, PSM, double-dif)Many OECD countries. Also in many DCs, Argentina, Brazil, Kenya

56

"Micro-accounting"/RHG approach based on aggregate macro predictions SimSIP/PAMS

Latin America, Burkina-Faso, Thailand, Indonesia

The disaggregated SAM-CGE/RHG approach

IFPRI (US) country models, IMMPA-Cameroon, Brazil, many countries have GTAP+LAVs+HHS

"Micro-accounting modules" linked to disaggregated macro models China, Colombia, Brazil, Many countries have GTAP+LAVs+HHS

"Micro-simulation modules" linked to disaggregated macro models Indonesia, Brasil

More sophisticated?

Fully integrated models (Thailand, Madagascar)

Accounting for general equilibrium effects of public expenditure programs (???)

Dynamic modeling and the proper treatment of growth (???)

57

Usage of these “techniques” in connection with policy-making, Brazil example with Macro-stabilization + Zero-Hunger, Bolsa-Familia

program• Done on first principles: Macro-stabilization necessary condition for

poverty reduction (growth)– Control inflation utmost importance for growth, poverty reduction– Fiscal responsibility and Debt reduction are natural instruments– Reduction of vulnerabilities (external)

• Done with Monitoring & Evaluation: Social programs are needed, limited resources– Unification of several Federal programs (Bolsa-Escola, Vale-Gas, Bolsa-

Alimentacao) more eficiency (e.g., Oportunidades), but… LOAS (old-age rural pension not done on PPP-basis by region and Social Security reform still a problem

– Emergence of CCT programs with incdentives and “exit” options– Statistical apparatus available for good evaluation “Cadastro Unico”,

PNADs and POF with Census and PIA

58

So…..main messages?

• Micro-incidence analysis (average, behavioral) is easy to use, HHS are available, ex-ante and ex-post analysis can be conducted– Many DCs using it

– Next steps: inclusion in current policy frameworks

• Micro-macro linkages more costly in time, resources and skills (Indonesia, Brazil experiences entail cooperation of academia, IFIs, Government agencies)– Important when most policies have macro content

– Difficult to maintain and implement (crisis-time is not a time for DIME, usually first-principles are used, e.g., Asian crises in 1998)

59

END

Caminante no hay camino, se hace camino al andar…

(Antonio Machado)