The Performance of Decentralized School Systems: Evidence from Fe y
Alegria in Venezuela
Public-Private Partnerships in Education Conference, World BankJune 7, 2007 Washington, DC
Hunt Allcott
Harvard University
Daniel Ortega
IESA
Introduction and motivation• Fe y Alegria is a private subsidized confederation of Jesuit schools that
targets disadvantaged youths
• It is a form of government contracting educational services:– Central Governments pay the salaries of teachers and the principal– Local and international foundations and agencies as well as voluntary fees
from the community pay for infrastructure expenses– FyA maintains complete managerial autonomy and significant curricular
flexibility– Less than 3% of the people who work there are of a religious order
• The school principal and board can hire and fire teachers without public-sector union constraints, and the national FyA coordination appoints principals
• Fundraising and major infrastructure projects are coordinated at the national level (some efforts are coordinated by the international federation)
Introduction and motivation• Founded in 1955 in Caracas by Father Jose Maria Velaz
as an outreach program of Andres Bello Catholic University
• Its expansion throughout the region after its inception in 1955 has led many observers to view it as a significant success (515,000 students in formal education in 2005)
• Descriptive academic studies have documented its positive performance relative to the public school system (Swope and Latorre, 2000; Navarro and De La Cruz, 1998, Gonzalez and Arevalo, 2005)
• However, no attempt at a rigorous econometric evaluation of its effects has been undertaken
Data• In 2003, 413,607 high school graduates took the
mandatory “Prueba de Aptitud Academica”, which is Venezuela’s SAT for college admission
• Test administrators collect extensive socioeconomic data on each individual and their families (mother’s education, family income, house quality, etc.)
• We use 48,697 of these exam takers in that year: Fe y Alegria or public school graduates who finished their studies that year and who were between 14 and 22 years of age
• Our final dataset includes 46,460 public school students and 2,237 FyA students (4,5% of total)
Key information for our strategy• FyA schools are oversubscribed (admit rates are around 35% and
schools choose based on observables (wealth and geographic location)
• We construct a socio-economic status (SES) variable from factor analysis on family characteristics and find that FyA students are of the same SES that public school students within each municipality
• Although school placement was originally targeted at lower income areas, many of these areas have developed over time, so it is not clear that they are correlated with test scores within a municipality
• Thus we assume that unobservables do not substantially affect both FyA enrollment and test scores
Estimation Strategy• We originally constructed an instrument based on the number of
FyA schools in a municipality, but it had too little variation to obtain meaningful estimates (program intensity across the 330 municipalities is very low and even if restricted to those where it is >0, it is about 5%)
• We estimate the Average Treatment Effect (ATE) via OLS controlling for a set of 54 dummy variables capturing family characteristics. OLS is consistent if there are no omitted variables and if the treatment effect is homogeneous
• The OLS model is where T is the treatment indicator and X is a vector of dummy
variables indicating {Venezuelan, Male, Married, Age, Student Works, Father's Profession, Mother's Education, House Quality, Income, Number of Siblings, How School Fees Are Paid, Transportation to School, Social Class}
iiii XTY 210
Estimation strategy
• Given the OLS baseline, we estimate the ATE via propensity score matching
• The propensity score is estimated using a standard probit of participation on the observables:
• The matching estimator simulates the counterfactual outcomes based on the “nearest neighbor” (J=4):
)()1Pr( ii XT
i
i
Nlli
Nlli
YJ
TY
YJ
TY
1)0|ˆ(
1)1|ˆ(
1
0
PSM• We drop observations in the control group that have propensity
scores that are outside the support of the distribution of the treatment group, but don’t otherwise trim the sample, since we have a significant number of observations with P(X)<0.1
• All the data comes from the administration of the same test, with the same demographic questions asked of each student (no treatment heterogeneity; HIT(1997) “common economic environment”)
• We drop observations in municipalities without a FyA school since there are significant cross-municipality differences in test scores and SES [ P(X)>0 ]
• The test is conditional on high school graduation, but there is no selection into the test itself
Distribution of propensity scores• The support of the propensity score distributions in
treatment and control groups is very similar
Results• The OLS results controlling for a number of observables
give a 5% of a standard deviation on the verbal section of the test and a 6% of a SD on the math section
• Coefficients on control variables are interesting: – Younger students tend to do better, as do students with fewer
siblings– However, the effects of family income and house quality seem to
be in an inverted-U shape, the middle income group does best
• PSM results are qualitatively similar: 11% of a SD in verbal (not significant) and 8% of a SD in math
• The difference between OLS and PSM can be due to heterogeneous treatment effects (Angrist, 1998)
Heterogeneous treatment effect
Verbal Math ObsClass 4,5 ATE 0.242 0.249 13,768
SE 0.101 0.095
Class 1-3 ATE 0.056 0.079 32,519SE 0.059 0.032
Mother's Ed 4,5 ATE 0.071 0.186 19,105SE 0.042 0.049
Mother's Ed 1-3 ATE 0.052 0.074 27,182SE 0.062 0.033
Concluding Remarks
• FyA treatment improves performance by about 10% of the average, which is quantitatively significant
• The difference between OLS and PSM arises due to heterogeneous treatment effects: the poor benefit the most from FyA treatment
• We believe FyA performs better because of higher school-level autonomy, labor flexibility and a high esprit de corps
An expanding organizationFe y Alegria growth in Latin America
(Students in formal education)
-
20,000
40,000
60,000
80,000
100,000
120,000
140,000
160,00019
80
1982
1984
1986
1988
1990
1992
1994
1996
1998
2000
2002
Num
ber
of s
tude
nts
Argentina
Bolivia
Brasil
Colombia
Ecuador
El Salvador
Guatemala
Honduras
Nicaragua
Panamá
Paraguay
Perú
R. Domin.
Venezuela