1 an evaluation of the performance of regression discontinuity design on progresa hielke buddelmeyer...
TRANSCRIPT
1
An Evaluation of the Performance of Regression
Discontinuity Design on PROGRESA
Hielke Buddelmeyer
Melbourne Institute of Applied Economic and
Social [email protected]
Emmanuel Skoufias
The World [email protected]
Introduction
By consensus, a randomized design provides the most credible method of evaluating program impact.
But experimental designs are difficult to implement and are accompanied by political risks that jeopardize the chances of implementing them The idea of having a comparison/control group
is very unappealing to program managers and governments
ethical issues involved in withholding benefits for a certain group of households
Introduction Alternative: Quasi-experimental methods
attempting to equalize selection bias between treatment and control groups
Objective: Evaluate the performance of the RDD (a quasi-experimental estimator) by comparing the treatment effects estimated using RDD to the experimental estimates.
Focus on school attendance and work of 12-16 yr old boys and girls.
Introduction This analysis is one of the first to evaluate
the performance of RDD in a setting where it can be compared to experimental estimates. The PROGRESA experiment provides a unique opportunity to analyze this issue.
The RDD approach is economical (requires relatively little info), and potentially has many applications in different contexts and types of programs (microfinance, labor market training, education, CCT programs, etc.).
Some Background on PROGRESA
What is PROGRESA? Targeted cash transfer program conditioned on
families visiting health centers regularly and on children attending school regularly.
Cash transfer-alleviates short-term poverty Human capital investment-alleviates poverty
in the long-term By the end of 2004: program (renamed
Oportunidades) covered nearly 5 million families, in 72,000 localities in all 31 states (budget of about US$2.5 billion).
Some Background on PROGRESA
Two-stage Selection process: Geographic targeting (used census data to
identify poor localities) Within Village household-level targeting
(village household census)Used hh income, assets, and demographic
composition to estimate the probability of being poor (Inc per cap<Standard Food basket).
Discriminant analysis applied separately by regionDiscriminant score of each household compared to a
threshold value (high DS=Noneligible, low DS=Eligible)
Initially 52% eligible, then revised selection process so that 78% eligible. But many of the “new poor” households did not receive benefits
7
Figure 1: Kernel Densities of Discriminant Scores and Threshold points by region
De
nsi
ty
Region 3Discriminant Score
7593.9e-06
.003412
De
nsi
ty
Region 4Discriminant Score
7532.8e-06
.00329
De
nsi
ty
Region 5Discriminant Score
7510
.002918
De
nsi
ty
Region 6Discriminant Score
7525.5e-06
.004142
De
nsi
ty
Region 12Discriminant Score
5718.0e-06
.004625
De
nsi
ty
Region 27Discriminant Score
6914.5e-06
.003639
De
nsi
ty
Region 28Discriminant Score
757.000015
.002937
8
The RDD method-1
A quasi-experimental approach based on the discontinuity of the treatment assignment mechanism.
Sharp RD design Individuals/households are assigned to treatment (T) and
control (NT) groups based solely on the basis of an observed continuous measure such as the discriminate score DS. For example, B =1 if and only if DS<=COS (B=1 eligible beneficiary) and B=0 otherwise . Propensity is a step function that is discontinuous at the point DS=COS.
Analogous to selection on observables only. Violates the strong ignorability assumption of Rosenbaum
and Rubin (1983) which also requires the overlap condition.
9
______ Sharp Design
--------- Fuzzy design
Regression Discontinuity Design; treatment assignment in sharp (solid) and fuzzy (dashed)
designs.
10
The RDD method-2
Fuzzy RD design Treatment assignment depends on an
observed continuous variable such as the discriminate score DS but in a stochastic manner. Propensity score is S-shaped and is discontinuous at the point DS=COS.
Analogous to selection on observables and unobservables.
Allows for imperfect compliance (self-selection, attrition) among eligible beneficiaries and contamination of the comparison group by non-compliance (substitution bias).
11
Table 1: Sampled Households and Discriminant Scores by Region
(1)
Region
(2)
Number of households in
sample
(3)
Maximum value of
discriminant score among
eligible households
(4)
Minimum value of
discriminant score among non-eligible households
(5) Number of non-eligible households
with discriminant
score less than value in column (3)
Sierra Negra-Zongolica-Mazateca (code=3)
3,031 759.36 576 14
Sierra Norte-Otomi Tepehua (code=4)
4,559 753.14 653 15
Sierra Gorda (code=5)
10,790 751.5 610 14
Montana (Guerrero) (code=6)
1,907 752 693 3
Huasteca (San Luis Potosi) (code=12)
383 571 573 0
Tierra Caliente (Michoacan) (code=27)
2,934 691 546 213
Altiplano (San Luis Potosi) (code=28)*
472 856 757 116
Note: * In region 28 there were only 15 households with a discriminant score greater than the minimum value of the score among non-eligible households
12
Evaluating the Performance of RDD
A true evaluation of the RDD approach can only be done if the experimental estimates are
unbiased; and both estimators estimate the same
impact. Important to keep in mind what are the treatment effects/parameters estimated by the experimental approach and by the RDD
13
Treatment effects estimated from an experimental design
Experimental approach yields an estimate of the Average Treatment Effect on the Treated (ATT)
Depending on the success of the randomized design one may estimate the ATT using CSDIF or 2DIF
There may be heterogeneity of impacts, in which case it is wiser to use a “local” ATT among individuals close to the cutoff score for eligibility for the program (CSDIF-50)
In fact, we estimate the Average Intent to Treat effect (AIT) which provides an estimate of the average impact of the availability of the program to eligible households (in treatment communities) The binary variable B identifies whether a household has
been classified as eligible for the program. AIT is a lower bound estimate of the impact of the program ATT=AIT/(% of eligible households actually receiving benefits)
1,|1,| 01 BXYEBXYEATT
14
Treatment effects estimated by a RD design
Sharp design: The treatment effect estimated by a sharp
RDD is an Average Local Treatment Effect (to be distinguished from the LATE)
can be estimated by a simple comparison of the mean values of Y of individuals to the left (eligible) and to the right of the threshold score COS (noneligible)
Fuzzy design: The treatment effect estimated is a local
version of the LATE of Angrist et al. (1996).
15
Kernel Regression Estimator of Treatment Effect with a Sharp
RDD
COSDSYECOSDSYEYYCOS iiCOSDS
iiCOSDS
|lim|lim
n
i ii
n
i iii
uK
uKYY
1
1
)(*
)(**
where
and
n
i ii
n
i iii
uK
uKYY
1
1
)(*)1(
)(*)1(*
Alternative estimators (differ in the way local information is exploited and in the set of regularity conditions required to achieve asymptotic properties): Local Linear Regression (HTV, 2001) Partially Linear Model (Porter, 2003)
EXPERIMENTAL DESIGN: Program randomized at the locality level
Sample of 506 localities– 186 control (no PROGRESA)– 320 treatment (PROGRESA)
24, 077 Households (hh)
PROGRESA Evaluation Design
PROGRESA Evaluation Surveys/Data
BEFORE initiation of program:– Oct/Nov 97:
Household census to select beneficiaries
– March 98: consumption, school attendance, health
AFTER initiation of program– Nov 98– June 99– Nov/Dec 99Included survey
of beneficiary households regarding operations
Issues for consideration
Benchmark/Experimental estimates CSDIF or 2DIF?
Choice of bandwidth (+/-50, Appendix)Differences in program impacts across
regions (Appendix)Heterogeneity of program impacts
RDD is a local estimate of program impact Best to compare it with CSDIF-50
Treatment Effect estimates within a regression framework.
Using only the eligible (i.e. Poor) and running the regression in each survey round
Then
i
J
jijjiiii XRTRRTRTY
155330 5*53*3 ,
30)13( RCSDIF (6a)
50)15( RCSDIF (6b)
CSDIF-50: equation above estimated on hh within zone of 50 points below threshold.
22
2DIF CSDIF CSDIF-50 Uniform Biweight Epanechnik Triangular Quartic GuassianSCHOOL (1) (2) (3) (4) (5) (6) (7) (8) (9)Round 1 n.a 0.013 -0.001 -0.053 -0.016 -0.031 -0.018 -0.016 -0.050
st. error 0.018 0.028 0.027 0.031 0.029 0.031 0.031 0.021
Round 3 0.050 0.064 0.071 0.020 0.008 0.010 0.008 0.008 0.005st. error 0.017 0.019 0.028 0.028 0.034 0.031 0.033 0.034 0.022
Round 5 0.048 0.061 0.099 0.052 0.072 0.066 0.069 0.072 0.057st. error 0.020 0.019 0.030 0.028 0.032 0.030 0.032 0.032 0.021
Nobs 4279R-Squared 0.25
WORKRound 1 n.a. 0.018 0.007 0.012 -0.016 -0.004 -0.013 -0.016 0.025
st. error 0.019 0.029 0.027 0.032 0.029 0.031 0.032 0.021
Round 3 -0.037 -0.018 -0.007 0.007 -0.004 0.002 0.001 -0.004 0.005st. error 0.023 0.017 0.029 0.024 0.028 0.026 0.028 0.028 0.019
Round 5 -0.046 -0.028 -0.037 -0.031 -0.029 -0.030 -0.029 -0.029 -0.028st. error 0.025 0.017 0.025 0.024 0.028 0.026 0.027 0.028 0.019
Nobs 4279R-Squared 0.19
NOTES: Estimates in bold have t-values >=2Treatment Group for Experimental & RDD Estimates: Beneficiary Households in Treatment Villages (Group A)Comparison Group for Experimental Estimates: Eligible Households in Control Villages (Group B)Comparison Group for RDD Estimates: NonEligible Households in Treatment Villages (Group C)
163310.21
163310.16
TABLE 3aEstimates of Program Impact By Round (BOYS 12-16 yrs old)
RDD Impact Estimates using different kernel functions
Experimental Estimates
23
2DIF CSDIF CSDIF-50 Uniform Biweight Epanechnik. Triangular Quartic GuassianSCHOOL (1) (2) (3) (4) (5) (6) (7) (8) (9)Round 1 n.a. -0.001 0.000 -0.027 -0.025 -0.026 -0.025 -0.025 -0.035
st. error 0.020 0.030 0.029 0.036 0.033 0.034 0.036 0.023
Round 3 0.086 0.085 0.082 0.038 0.039 0.041 0.039 0.039 0.054st. error 0.017 0.020 0.029 0.030 0.036 0.033 0.034 0.036 0.024
Round 5 0.099 0.098 0.099 0.078 0.114 0.097 0.107 0.114 0.084st. error 0.020 0.019 0.028 0.031 0.036 0.033 0.035 0.036 0.025
Nobs 3865R-Squared 0.23
WORKRound 1 n.a. 0.034 0.000 0.033 0.026 0.027 0.027 0.026 0.030
st. error 0.017 0.024 0.019 0.022 0.020 0.021 0.022 0.015
Round 3 -0.034 0.000 0.001 0.005 0.001 0.003 0.002 0.001 -0.008st. error 0.017 0.009 0.016 0.015 0.018 0.016 0.017 0.018 0.012
Round 5 -0.042 -0.008 -0.025 -0.019 -0.034 -0.029 -0.033 -0.034 -0.025st. error 0.019 0.009 0.018 0.015 0.018 0.017 0.018 0.018 0.013
Nobs 3865R-Squared 0.07
NOTES: Estimates in bold have t-values >=2Treatment Group for Experimental & RDD Estimates: Beneficiary Households in Treatment Villages (Group A)Comparison Group for Experimental Estimates: Eligible Households in Control Villages (Group B)Comparison Group for RDD Estimates: NonEligible Households in Treatment Villages (Group C)
150460.22
150460.05
TABLE 3bEstimates of Program Impact By Round (GIRLS 12-16 yrs old)
RDD Impact Estimates using different kernel functions
Experimental Estimates
24
Main Results
Overall the performance of the RDD is remarkably good. The RDD estimates of program impact agree
with the experimental estimates in 10 out of the 12 possible cases.
The two cases in which the RDD method failed to reveal any significant program impact on the school attendance of boys and girls are in the first year of the program (round 3).
27
2DIF CSDIF CSDIF+50 CSDIF+75 2DIF CSDIF CSDIF+50 CSDIF+75 SCHOOL (1) (2) (3) (4) (5) (6) (7) (8) Round 1 n.a 0.041 0.067 0.063 n.a. 0.033 0.043 0.037
st. error 0.022 0.034 0.030 0.022 0.035 0.030
Round 3 0.017 0.058 0.071 0.067 -0.013 0.020 0.041 0.006 st. error 0.020 0.024 0.033 0.030 0.021 0.023 0.036 0.029
Round 5 -0.021 0.020 0.065 0.035 -0.012 0.021 0.015 0.016 st. error 0.023 0.024 0.036 0.032 0.024 0.023 0.036 0.031
Nobs 2762 3933 2636 3694 R-Squared 0.25 0.25 0.23 0.24
WORK Round 1 n.a. 0.026 0.062 0.033 n.a. 0.006 -0.022 -0.008
st. error 0.022 0.032 0.028 0.014 0.026 0.020
Round 3 -0.033 -0.007 -0.010 0.001 -0.003 0.003 -0.026 -0.004 st. error 0.027 0.020 0.031 0.026 0.018 0.012 0.019 0.016
Round 5 -0.025 0.001 -0.009 0.004 0.005 0.011 0.017 0.031 st. error 0.028 0.019 0.032 0.027 0.020 0.013 0.021 0.020
Nobs 2762 3933 2636 3694 R-Squared 0.18 0.19 0.61 0.57
NOTES: Estimates in bold have t-values >=2 Treatment Group: Non-Eligible Households in Treatment Villages (Group C) Comparison Group: Non-Eligible Households in Control Villages (Group D)
Experimental Estimates-Boys 12-16 yrs old Experimental Estimates-Girls 12-16 yrs old
TABLE 4
7314 0.23
7614
Program Impacts By Round on Non-Eligible Households in Treatment Localities using Group D as a comparison
0.24 7935
0.04 7935 0.19
30
RDD RDD RDD RDD RDD RDD2DIF CSDIF CSDIF+/-50 Uniform Triangular Gaussian 2DIF CSDIF CSDIF+/-50 Uniform Triangular Gaussian
SCHOOL (1) (2) (3) (4) (5) (6) (7) (8) (9) (10) (11) (12)Round 1 n.a. 0.081 0.060 0.042 0.044 0.055 n.a. 0.048 -0.010 0.000 -0.034 0.021
st. error 0.029 0.051 0.031 0.036 0.026 0.029 0.049 0.034 0.038 0.027
Round 3 -0.001 0.081 0.049 0.024 0.033 0.056 -0.002 0.046 0.001 0.022 0.031 0.014st. error 0.019 0.028 0.050 0.031 0.035 0.024 0.021 0.030 0.050 0.031 0.037 0.025
Round 5 -0.033 0.048 0.030 0.033 0.026 0.014 -0.023 0.025 -0.011 0.010 0.011 0.013st. error 0.022 0.028 0.053 0.032 0.037 0.025 0.023 0.028 0.049 0.033 0.037 0.026Nobs 2950 2971
R-Squared 0.25 0.21
WORKRound 1 n.a. -0.014 -0.014 -0.002 0.004 -0.030 n.a. 0.001 -0.028 -0.016 -0.021 -0.012
st. error 0.025 0.039 0.030 0.034 0.023 0.018 0.030 0.020 0.023 0.016
Round 3 -0.015 -0.030 -0.023 0.002 -0.005 -0.019 0.007 0.007 0.000 0.007 0.005 0.013st. error 0.026 0.023 0.038 0.028 0.031 0.022 0.018 0.014 0.026 0.015 0.017 0.012
Round 5 -0.015 -0.030 -0.001 0.001 -0.021 0.007 -0.004 -0.003 -0.006 -0.002 -0.003 0.004st. error 0.028 0.021 0.037 0.026 0.030 0.021 0.020 0.016 0.031 0.017 0.021 0.015Nobs 2950 2971
R-Squared 0.19 0.06
NOTES:Estimates in bold have t-values >=2Treatment Group: Non-Eligible Households in Tretament Villages (group C)Comparison Group: Eligible Households in Control Villages (group B)
Table 5
GIRLS 12-16 yrs oldBOYS 12-16 yrs old
Program Impacts By Round on Non-Eligible Households in Treatment Localities using Group B as a comparison
103780.16
98670.20
98670.04
103780.21
33
RDD RDD RDD RDD RDD RDD2DIF CSDIF CSIDF+/-50 Uniform Triangular Gaussian 2DIF CSDIF CSIDF+/-50 Uniform Triangular Gaussian
SCHOOL (1) (2) (3) (4) (5) (6) (7) (8) (9) (10) (11) (12)Round 1 n.a. -0.028 0.016 0.030 0.014 0.009 n.a. 0.009 0.104 0.070 0.113 0.045
st. error 0.023 0.041 0.030 0.035 0.024 0.026 0.048 0.033 0.037 0.026
Round 3 0.017 -0.011 0.026 0.050 0.063 0.025 -0.011 -0.002 0.092 0.056 0.063 0.041st. error 0.018 0.022 0.042 0.031 0.035 0.025 0.019 0.024 0.044 0.032 0.036 0.027
Round 5 0.013 -0.016 0.039 0.063 0.044 0.035 0.012 0.021 0.075 0.025 0.014 0.012st. error 0.021 0.024 0.045 0.032 0.035 0.025 0.023 0.026 0.047 0.031 0.036 0.025Nobs 2986 2937R-squared 0.25 0.23
WORKRound 1 n.a. 0.050 0.068 0.060 0.062 0.068 n.a. 0.003 -0.022 -0.009 -0.022 -0.003
st. error 0.023 0.038 0.029 0.033 0.023 0.016 0.034 0.019 0.023 0.016
Round 3 -0.021 0.029 0.005 -0.018 -0.014 0.002 -0.008 -0.006 -0.049 -0.031 -0.042 -0.021st. error 0.024 0.019 0.037 0.027 0.032 0.022 0.017 0.013 0.030 0.017 0.021 0.014
Round 5 -0.011 0.039 -0.014 -0.044 -0.022 -0.029 0.010 0.012 0.000 0.024 0.029 0.029st. error 0.024 0.018 0.038 0.028 0.031 0.023 0.017 0.012 0.031 0.015 0.018 0.012Nobs 2986 2937R-squared 0.18 0.08
NOTES: Estimates in bold have a t-value >=2Treatment Group: Eligible Households in Control Villages (Group B)Comparison Group: Non-Eligible Households in Control Villages (Group D)
94590.04
94590.21
98370.21
98370.16
BOYS 12-16 yrs old GIRLS 12-16 yrs old
Table 6 Testing the Integrity of the Control Groups: Program Impacts on Eligible Households By Gender and by Round in the Control Villages
36
RDD RDD RDD RDD RDD RDD2DIF CSDIF CSDIF+/-50 Uniform Triangular Gaussian 2DIF CSDIF CSDIF+/-50 Uniform Triangular Gaussian
SCHOOL (1) (2) (3) (4) (5) (6) (7) (8) (9) (10) (11) (12)Round 1 n.a. -0.002 0.080 0.020 0.039 0.014 n.a. -0.011 0.080 0.044 0.053 0.031
st. error 0.024 0.040 0.028 0.031 0.022 0.026 0.046 0.029 0.033 0.023
Round 3 0.068 0.066 0.161 0.094 0.105 0.086 0.070 0.059 0.142 0.117 0.132 0.108st. error 0.018 0.025 0.041 0.029 0.032 0.023 0.017 0.025 0.043 0.029 0.033 0.023
Round 5 0.060 0.059 0.196 0.147 0.139 0.107 0.105 0.094 0.149 0.112 0.131 0.108st. error 0.021 0.025 0.041 0.027 0.030 0.022 0.020 0.026 0.046 0.032 0.037 0.024
Nobs 4121 3541R-Squared 0.25 0.24
WORKRound 1 n.a. 0.067 0.026 0.070 0.053 0.063 n.a. 0.031 -0.045 0.009 -0.015 0.015
st. error 0.024 0.036 0.025 0.028 0.020 0.017 0.036 0.020 0.023 0.016
Round 3 -0.059 0.008 -0.057 -0.009 -0.018 -0.012 -0.041 -0.010 -0.070 -0.019 -0.034 -0.016st. error 0.024 0.021 0.037 0.024 0.026 0.019 0.017 0.013 0.030 0.017 0.020 0.013
Round 5 -0.059 0.008 -0.099 -0.074 -0.072 -0.050 -0.032 -0.001 -0.048 0.004 -0.007 0.008st. error 0.024 0.021 0.034 0.023 0.026 0.018 0.019 0.012 0.028 0.014 0.015 0.011
Nobs 4121 3541R-Squared 0.18 0.08
NOTES:Estimates in bold have t-values >=2
Treatment Group: Beneficiary Households in Treatment Villages (group A)Comparison Group: Non-Eligible Households in Control Villages (group D)
138880.18
127930.23
127930.06
138880.22
Table 7Estimates of Program Impact Using Non-Elligible Households in Control Villages as a Comparison Group
GIRLS 12-16 yrs oldBOYS 12-16 yrs old
Concluding Remarks
Our analysis reveals that in the PROGRESA sample, the RDD performs very well (i.e. yields program impacts close to the ideal experimental impact estimates).
Critical to be aware of some of the limitations of the RDD approach: Estimates treatment effects at the point
of discontinuity (eligibility threshold). Impact on this group of households may be of less interest than impact of the program on the poorer households
Concluding Remarks
The integrity/quality of the control/comparison group is of vital importance. Spillover effects do not necessarily lead to a violation of the
RDD approach. As long as the local continuity assumption continues to hold even though there are spillover effects the presence of spillover effects would only affect the interpretation of the RD effect: It is the effect of being eligible for program participation in treatment villages net of spillover effects.
Social programs at the national scale may be very difficult to evaluate ex-post because of the difficulty in finding an adequate comparison group
39
2DIF CSDIF CSDIF-75 Uniform Biweight Epanechnik Triangular Quartic GuassianSCHOOL (1) (2) (3) (4) (5) (6) (7) (8) (9)Round 1 n.a 0.013 0.004 -0.063 -0.042 -0.053 -0.042 -0.042 -0.050
st. error 0.018 0.024 0.022 0.027 0.024 0.026 0.027 0.019
Round 3 0.050 0.064 0.086 0.007 0.012 0.013 0.012 0.012 -0.001st. error 0.017 0.019 0.025 0.023 0.027 0.025 0.026 0.027 0.019
Round 5 0.048 0.061 0.102 0.051 0.063 0.059 0.062 0.063 0.053st. error 0.020 0.019 0.026 0.023 0.028 0.025 0.027 0.028 0.019
Nobs 6198R-Squared 0.25
WORKRound 1 n.a. 0.018 -0.009 0.049 0.010 0.024 0.013 0.010 0.028
st. error 0.019 0.025 0.022 0.026 0.024 0.025 0.026 0.019
Round 3 -0.037 -0.018 -0.028 0.006 0.002 0.004 0.003 0.002 0.006st. error 0.023 0.017 0.025 0.020 0.024 0.022 0.023 0.024 0.017
Round 5 -0.046 -0.028 -0.039 -0.030 -0.035 -0.035 -0.034 -0.035 -0.020st. error 0.025 0.017 0.022 0.020 0.023 0.021 0.022 0.023 0.016
Nobs 6198R-Squared 0.20
NOTES: Estimates in bold have t-values >=2
Treatment Group for Experimental & RDD Estimates: Eligible Households in Treatment Villages (Group A)Comparison Group for Experimental Estimates: Eligible Households in Control Villages (Group B)Comparison Group for RDD Estimates: NonEligible Households in Treatment Villages (Group C)
APPENDIX TABLE A.1a--Bandwidth=75Estimates of Program Impact By Round (BOYS 12-16 yrs old)
RDD Impact Estimates using different kernel functions
Experimental Estimates
163310.21
163310.16
40
2DIF CSDIF CSDIF-75 Uniform Biweight Epanechnik. Triangular Quartic GuassianSCHOOL (1) (2) (3) (4) (5) (6) (7) (8) (9)Round 1 n.a. -0.001 0.020 -0.035 -0.023 -0.025 -0.024 -0.023 -0.043
st. error 0.020 0.027 0.025 0.029 0.027 0.028 0.029 0.020
Round 3 0.086 0.085 0.092 0.060 0.050 0.056 0.052 0.050 0.050st. error 0.017 0.020 0.026 0.024 0.029 0.027 0.029 0.029 0.020
Round 5 0.099 0.098 0.108 0.076 0.092 0.085 0.092 0.092 0.079st. error 0.020 0.019 0.025 0.027 0.030 0.028 0.029 0.030 0.022
Nobs 5554R-Squared 0.23
WORKRound 1 n.a. 0.034 0.010 0.034 0.029 0.031 0.030 0.029 0.029
st. error 0.017 0.022 0.016 0.018 0.017 0.018 0.018 0.013
Round 3 -0.034 0.000 -0.012 -0.006 -0.001 -0.003 -0.002 -0.001 -0.014st. error 0.017 0.009 0.014 0.013 0.015 0.014 0.014 0.015 0.011
Round 5 -0.042 -0.008 -0.029 -0.025 -0.029 -0.028 -0.030 -0.029 -0.022st. error 0.019 0.009 0.015 0.014 0.015 0.015 0.015 0.015 0.012
Nobs 5554R-Squared 0.06
NOTES: Estimates in bold have t-values >=2Treatment Group for Experimental & RDD Estimates: Eligible Households in Treatment Villages (Group A)Comparison Group for Experimental Estimates: Eligible Households in Control Villages (Group B)Comparison Group for RDD Estimates: NonEligible Households in Treatment Villages (Group C)
APPENDIX TABLE A.1b--Bandwidth=75Estimates of Program Impact By Round (GIRLS 12-16 yrs old)
RDD Impact Estimates using different kernel functions
Experimental Estimates
150460.22
150460.05
41
2DIF CSDIF CSDIF-100 Uniform Biweight Epanechnik Triangular Quartic GuassianSCHOOL (1) (2) (3) (4) (5) (6) (7) (8) (9)Round 1 n.a 0.013 0.006 -0.051 -0.052 -0.055 -0.049 -0.052 -0.049
st. error 0.018 0.023 0.020 0.023 0.022 0.023 0.023 0.017
Round 3 0.050 0.064 0.090 -0.007 0.009 0.004 0.006 0.009 -0.005st. error 0.017 0.019 0.023 0.021 0.024 0.023 0.023 0.024 0.018
Round 5 0.048 0.061 0.080 0.059 0.058 0.056 0.059 0.058 0.049st. error 0.020 0.019 0.023 0.021 0.024 0.022 0.023 0.024 0.018
Nobs 7937R-Squared 0.24
WORKRound 1 n.a. 0.018 -0.004 0.023 0.025 0.030 0.022 0.025 0.027
st. error 0.019 0.024 0.020 0.023 0.021 0.022 0.023 0.018
Round 3 -0.037 -0.018 -0.028 0.012 0.005 0.007 0.006 0.005 0.007st. error 0.023 0.017 0.022 0.018 0.021 0.020 0.020 0.021 0.016
Round 5 -0.046 -0.028 -0.020 -0.023 -0.033 -0.029 -0.030 -0.033 -0.015st. error 0.025 0.017 0.020 0.017 0.020 0.019 0.020 0.020 0.015
Nobs 7937R-Squared 0.19
NOTES: Estimates in bold have t-values >=2Treatment Group for Experimental & RDD Estimates: Eligible Households in Treatment Villages (Group A)Comparison Group for Experimental Estimates: Eligible Households in Control Villages (Group B)Comparison Group for RDD Estimates: NonEligible Households in Treatment Villages (Group C)
APPENDIX TABLE A.2a--Bandwidth=100Estimates of Program Impact By Round (BOYS 12-16 yrs old)
RDD Impact Estimates using different kernel functions
Experimental Estimates
163310.21
163310.16
42
2DIF CSDIF CSDIF-100 Uniform Biweight Epanechnik. Triangular Quartic GuassianSCHOOL (1) (2) (3) (4) (5) (6) (7) (8) (9)Round 1 n.a. -0.001 0.016 -0.048 -0.029 -0.034 -0.031 -0.029 -0.049
st. error 0.020 0.025 0.022 0.025 0.023 0.025 0.025 0.019
Round 3 0.086 0.085 0.081 0.054 0.056 0.060 0.056 0.056 0.045st. error 0.017 0.020 0.024 0.022 0.026 0.024 0.025 0.026 0.018
Round 5 0.099 0.098 0.096 0.081 0.084 0.083 0.087 0.084 0.076st. error 0.020 0.019 0.025 0.024 0.027 0.026 0.027 0.027 0.021
Nobs 7221R-Squared 0.24
WORKRound 1 n.a. 0.034 0.012 0.033 0.030 0.030 0.030 0.030 0.028
st. error 0.017 0.020 0.014 0.016 0.015 0.016 0.016 0.012
Round 3 -0.034 0.000 -0.007 -0.014 -0.005 -0.008 -0.006 -0.005 -0.016st. error 0.017 0.009 0.012 0.012 0.013 0.012 0.013 0.013 0.010
Round 5 -0.042 -0.008 -0.028 -0.026 -0.027 -0.026 -0.027 -0.027 -0.019st. error 0.019 0.009 0.014 0.013 0.014 0.013 0.014 0.014 0.011
Nobs 7221R-Squared 0.06
NOTES: Estimates in bold have t-values >=2Treatment Group for Experimental & RDD Estimates: Eligible Households in Treatment Villages (Group A)Comparison Group for Experimental Estimates: Eligible Households in Control Villages (Group B)Comparison Group for RDD Estimates: NonEligible Households in Treatment Villages (Group C)
APPENDIX TABLE A.2b--Bandwidth=100Estimates of Program Impact By Round (GIRLS 12-16 yrs old)
RDD Impact Estimates using different kernel functions
Experimental Estimates
150460.22
150460.05
43
2DIF CSDIF CSDIF-50 Uniform Biweight Epanechnik Triangular Quartic GuassianSCHOOL (1) (2) (3) (4) (5) (6) (7) (8) (9)Round 1-coeff n.a 0.013 0.011 -0.067 -0.047 -0.055 -0.050 -0.047 -0.060
st. error 0.020 0.031 0.030 0.037 0.034 0.036 0.037 0.023
Round 3-coeff 0.042 0.054 0.070 -0.001 -0.023 -0.017 -0.022 -0.023 -0.013st. error 0.019 0.021 0.030 0.032 0.036 0.034 0.035 0.036 0.024
Round 5 0.042 0.055 0.117 0.062 0.070 0.066 0.068 0.070 0.057st. error 0.022 0.021 0.033 0.031 0.038 0.034 0.037 0.038 0.023
Nobs 3529R-Squared 0.24
WORKRound 1 n.a. 0.020 0.006 0.032 0.008 0.019 0.010 0.008 0.038
st. error 0.022 0.032 0.030 0.036 0.033 0.034 0.036 0.023
Round 3 -0.037 -0.017 -0.001 0.016 0.020 0.020 0.022 0.020 0.009st. error 0.026 0.019 0.031 0.026 0.031 0.028 0.030 0.031 0.020
Round 5 -0.046 -0.027 -0.033 -0.043 -0.049 -0.047 -0.048 -0.049 -0.037st. error 0.028 0.018 0.027 0.027 0.034 0.031 0.032 0.034 0.021
Nobs 3529R-Squared 0.20
NOTES: Estimates in bold have t-values >=2Treatment group for Experimental & RDD Estimates: Eligible Households in Tretament Villages (Group A)Comparison Group for Experimental Estimates: Eligible Households in Control Villages (Group B)Comparison Group for RDD Estimates: NonEligible Households in Treatment Villages (Group C)
APPENDIX TABLE A.3a--Regions 3,4,5 & 6, Bandwisth=50Estimates of Program Impact By Round (BOYS 12-16 yrs old)
RDD Impact Estimates using different kernel functions
Experimental Estimates
135090.42
135090.39
44
2DIF CSDIF CSDIF-50 Uniform Biweight Epanechnik. Triangular Quartic GuassianSCHOOL (1) (2) (3) (4) (5) (6) (7) (8) (9)Round 1 n.a. 0.012 0.023 -0.003 -0.002 -0.003 -0.003 -0.002 -0.011
st. error 0.022 0.033 0.035 0.042 0.038 0.041 0.042 0.028
Round 3 0.066 0.079 0.079 0.034 0.032 0.032 0.032 0.032 0.054st. error 0.018 0.022 0.032 0.031 0.037 0.034 0.035 0.037 0.024
Round 5 0.087 0.099 0.121 0.059 0.093 0.075 0.086 0.093 0.072st. error 0.022 0.021 0.032 0.032 0.038 0.034 0.037 0.038 0.026
Nobs 3248R-Squared 0.22
WORKRound 1 n.a. 0.034 0.011 0.025 0.016 0.018 0.017 0.016 0.023
st. error 0.019 0.027 0.022 0.026 0.024 0.026 0.026 0.017
Round 3 -0.035 -0.002 -0.001 0.008 0.005 0.008 0.006 0.005 -0.005st. error 0.020 0.010 0.018 0.016 0.020 0.017 0.019 0.020 0.014
Round 5 -0.043 -0.009 -0.028 -0.024 -0.044 -0.039 -0.042 -0.044 -0.030st. error 0.023 0.011 0.020 0.018 0.022 0.020 0.021 0.022 0.014
Nobs 3248R-Squared 0.07
NOTES: Estimates in bold have t-values >=2Treatment group for Experimental & RDD Estimates: Eligible Households in Tretament Villages (Group A)Comparison Group for Experimental Estimates: Eligible Households in Control Villages (Group B)Comparison Group for RDD Estimates: NonEligible Households in Treatment Villages (Group C)
APPENDIX TABLE A.3b--Regions 3,4,5 & 6, Bandwisth=50Estimates of Program Impact By Round (BOYS 12-16 yrs old)
RDD Impact Estimates using different kernel functions
Experimental Estimates
123440.44
123440.26
45
All Boys 12-17 Years Old
0.2
0.3
0.4
0.5
0.6
0.7
Nov-97 Nov-98 Jun-99 Nov-99
Survey Round
Per
cen
t A
tten
din
g S
choo
l L
ast
Wee
k
Treatment Control
46
All Girls 12-17 Years Old
0.2
0.3
0.4
0.5
0.6
0.7
Nov-97 Nov-98 Jun-99 Nov-99
Survey Round
Per
cent
Att
endi
ng S
choo
l L
ast
Wee
k
Treatment Control
47
All Boys 12-17 Years Old
0.20
0.25
0.30
0.35
0.40
Nov-97 Nov-98 Jun-99 Nov-99
Survey Round
Per
cent
wor
king
Las
t W
eek
Treatment Control