The Performance of Decentralized School Systems: Evidence from Fe y

Alegria in Venezuela

Public-Private Partnerships in Education Conference, World BankJune 7, 2007 Washington, DC

Hunt Allcott

Harvard University

Daniel Ortega

IESA

Introduction and motivation• Fe y Alegria is a private subsidized confederation of Jesuit schools that

targets disadvantaged youths

• It is a form of government contracting educational services:– Central Governments pay the salaries of teachers and the principal– Local and international foundations and agencies as well as voluntary fees

from the community pay for infrastructure expenses– FyA maintains complete managerial autonomy and significant curricular

flexibility– Less than 3% of the people who work there are of a religious order

• The school principal and board can hire and fire teachers without public-sector union constraints, and the national FyA coordination appoints principals

• Fundraising and major infrastructure projects are coordinated at the national level (some efforts are coordinated by the international federation)

Introduction and motivation• Founded in 1955 in Caracas by Father Jose Maria Velaz

as an outreach program of Andres Bello Catholic University

• Its expansion throughout the region after its inception in 1955 has led many observers to view it as a significant success (515,000 students in formal education in 2005)

• Descriptive academic studies have documented its positive performance relative to the public school system (Swope and Latorre, 2000; Navarro and De La Cruz, 1998, Gonzalez and Arevalo, 2005)

• However, no attempt at a rigorous econometric evaluation of its effects has been undertaken

Data• In 2003, 413,607 high school graduates took the

mandatory “Prueba de Aptitud Academica”, which is Venezuela’s SAT for college admission

• Test administrators collect extensive socioeconomic data on each individual and their families (mother’s education, family income, house quality, etc.)

• We use 48,697 of these exam takers in that year: Fe y Alegria or public school graduates who finished their studies that year and who were between 14 and 22 years of age

• Our final dataset includes 46,460 public school students and 2,237 FyA students (4,5% of total)

Key information for our strategy• FyA schools are oversubscribed (admit rates are around 35% and

schools choose based on observables (wealth and geographic location)

• We construct a socio-economic status (SES) variable from factor analysis on family characteristics and find that FyA students are of the same SES that public school students within each municipality

• Although school placement was originally targeted at lower income areas, many of these areas have developed over time, so it is not clear that they are correlated with test scores within a municipality

• Thus we assume that unobservables do not substantially affect both FyA enrollment and test scores

Estimation Strategy• We originally constructed an instrument based on the number of

FyA schools in a municipality, but it had too little variation to obtain meaningful estimates (program intensity across the 330 municipalities is very low and even if restricted to those where it is >0, it is about 5%)

• We estimate the Average Treatment Effect (ATE) via OLS controlling for a set of 54 dummy variables capturing family characteristics. OLS is consistent if there are no omitted variables and if the treatment effect is homogeneous

• The OLS model is where T is the treatment indicator and X is a vector of dummy

variables indicating {Venezuelan, Male, Married, Age, Student Works, Father's Profession, Mother's Education, House Quality, Income, Number of Siblings, How School Fees Are Paid, Transportation to School, Social Class}

iiii XTY 210

Estimation strategy

• Given the OLS baseline, we estimate the ATE via propensity score matching

• The propensity score is estimated using a standard probit of participation on the observables:

• The matching estimator simulates the counterfactual outcomes based on the “nearest neighbor” (J=4):

)()1Pr( ii XT

i

i

Nlli

Nlli

YJ

TY

YJ

TY

1)0|ˆ(

1)1|ˆ(

1

0

PSM• We drop observations in the control group that have propensity

scores that are outside the support of the distribution of the treatment group, but don’t otherwise trim the sample, since we have a significant number of observations with P(X)<0.1

• All the data comes from the administration of the same test, with the same demographic questions asked of each student (no treatment heterogeneity; HIT(1997) “common economic environment”)

• We drop observations in municipalities without a FyA school since there are significant cross-municipality differences in test scores and SES [ P(X)>0 ]

• The test is conditional on high school graduation, but there is no selection into the test itself

Distribution of propensity scores• The support of the propensity score distributions in

treatment and control groups is very similar

Results• The OLS results controlling for a number of observables

give a 5% of a standard deviation on the verbal section of the test and a 6% of a SD on the math section

• Coefficients on control variables are interesting: – Younger students tend to do better, as do students with fewer

siblings– However, the effects of family income and house quality seem to

be in an inverted-U shape, the middle income group does best

• PSM results are qualitatively similar: 11% of a SD in verbal (not significant) and 8% of a SD in math

• The difference between OLS and PSM can be due to heterogeneous treatment effects (Angrist, 1998)

Heterogeneous treatment effect

Verbal Math ObsClass 4,5 ATE 0.242 0.249 13,768

SE 0.101 0.095

Class 1-3 ATE 0.056 0.079 32,519SE 0.059 0.032

Mother's Ed 4,5 ATE 0.071 0.186 19,105SE 0.042 0.049

Mother's Ed 1-3 ATE 0.052 0.074 27,182SE 0.062 0.033

Concluding Remarks

• FyA treatment improves performance by about 10% of the average, which is quantitatively significant

• The difference between OLS and PSM arises due to heterogeneous treatment effects: the poor benefit the most from FyA treatment

• We believe FyA performs better because of higher school-level autonomy, labor flexibility and a high esprit de corps

An expanding organizationFe y Alegria growth in Latin America

(Students in formal education)

-

20,000

40,000

60,000

80,000

100,000

120,000

140,000

160,00019

80

1982

1984

1986

1988

1990

1992

1994

1996

1998

2000

2002

Num

ber

of s

tude

nts

Argentina

Bolivia

Brasil

Colombia

Ecuador

El Salvador

Guatemala

Honduras

Nicaragua

Panamá

Paraguay

Perú

R. Domin.

Venezuela