Lessons for Education in Africa

Evidence from Randomized Evaluations in Kenya

Esther DufloJ-PAL

A B D U L L A T I F J A M E E L P O V E R T Y A C T I O N L A B

Two Challenges in education

(1) Bring children to school– MDGs for education seek to get 100%

participation in primary school and gender equality in education participation more general

(2)Teach them something when they are there.

The situation in many developing countries, including Africa

(1) Progress has been made on the first goal• Rapid improvement in enrollment rates, at least in

primary school (with UPE)• Situation for girls has also improved. • Secondary school?

(2) Quality is still lagging behind:• Chaudhury et al. Teachers are frequently absent. • Large class sizes (average of 80 children per

teacher in grade 1)

How to make progress?

• Faced with these two challenges, one is tempted to come up with silver bullets (teacher training; school committees; vouchers; etc.)

• There is probably no silver bullet, one needs to learn from experience what has worked and what has not worked, and what problem what solutions can and cannot fix.

How do we know what has worked?

• There is no market test for service delivery; we cannot measure effectiveness by “sales”.

• No automatic way of knowing whether anything useful is being delivered. Hence there is no guarantee that the money is well-spent.

• In part this is a matter of making sure that the program is doing what it was supposed to do: Process Evaluation.

• In part it is matter of making sure that the program is having an impact: Impact Evaluation

What have we learnt from impact evaluations about education in Africa

• Participation• Quality:

– Inputs– Reform Strategies

Some findings from randomized evaluation: Education in Developing Countries

• Participation• Quality:

– Inputs– Reform Strategies

Participation in education

• Reducing the cost of education: Uniforms– Some countries have implemented free primary education– But school uniforms required in many countries, including Kenya– Uniforms cost $6, GDP/capita about $340

• Provision of free uniforms– Lower grades: Reduce absence rate by 26% (Evans and Kremer,

2005)• Data suggest effect stronger for girls, younger pupils

– Reduce dropout rate by 15% for older pupils (Duflo, Dupas, Kremer and Sinei, 2006)

• Girls and boys similar• 10% decrease in teen childbearing• Part of a larger project to evaluate HIV-AIDS prevention in Schools:

more on Saturday!

Participation in education

• School meals – Evaluation for Pre-schools in Kenya:

participation was 30% higher in schools were free breakfast was given

– But: Higher fees and payments to teachers in project schools,

– Lowered fees in nearby competing schools: – Teachers benefit?

Participation in education

• School health: deworming• One billion infected with worms; yet there is cheap,

easy mass treatment• Miguel and Kremer (2004): Schools in rural Kenya:

twice-yearly school-based mass treatment – Health and school participation improved for treated

students, externalities to non-treated students– Absenteeism 25% lower in treatment schools– Increased schooling by 0.15 years per pupil/year treated– Calculate long-term returns to increased school due to

deworming as additional $30 in wages for $0.49, or, with teacher salary, $9.55

Participation in education: Comparing Costs

• With results that are based on similar methodologies, often in similar settings, and reliable estimates of the causal impact of the intervention, can combine the cost per extra year of education induced across program.

• This is different from the cost per child of the program (depends on the number of infra-marginal children).

• Take the overall cost and divide by the increase in the number of kid-year that can be attributed to the program.

• Can get some interesting surprises.

Improving school quality

• School quality remains very low: – Descriptive evidence– Many of the interventions just described

did not lead to an increase in test scores for the new students who came in: was it useful to send them to school if they learnt nothing?

Evidence is building up• A number of randomized evaluations have been

conducted on how to improve school quality• While many of the early results were

disappointing, we have learnt from them and this has informed the design of new programs (and the sense of what might work)

• Randomized evaluations with test scores as an outcomes allow to compare the cost effectiveness of different programs expressed in a constant unit ($ per standard deviation)

Providing Inputs

• Disappointing results from– Textbooks (Kenya: Glewwe et al.)– Flipcharts (Kenya: Glewwe et al.)– Extra days in school (e.g. deworming)

• Common thread: More of the same—and nothing works Perhaps a change in pedagogy/incentives is needed instead?

Providing Inputs—And Change• Extra Teacher program (Dupas, Duflo and Kremer)

– Large class sizes in grade 1 after UPE– School committees were given money to hire extra

(qualified) teacher to split grade one into two groups (or 3 if schools had 2 already)

– The program led to increase in test scores after 18 months but:

• Increase larger if the students are put in more homogenous classes (separated by preparedness to school)

• Increase larger if school committee is trained to provide oversight.

• Increase larger if the students get assigned to the extra teachers (incentives; local teacher)

• Effect is insignificant for regular teachers, heterogeneous classes, and no extra oversight by school committees.

Incentives for Teachers• Paying for inputs:

• Incentive can work: Camera project (Duflo and Hanna). Absenteeism reduced by 50%, test scores went up by 0.17 SD after a year. Slide 9

• Incentives can be perverted: Incentives administered by headteachers in Kenya led to no increase in presence, despite increase in reported presence (reporting problem)

• Paying for output—incentives based on test scores• Teacher incentive program in kenya based on scores at KCPE• Short run increases in test scores but no improvement in learning• Apparently more success in India: large experiment conducted with

the government of Andhra Pradesh found changes in two measures of test score

Incentives for students: Merit based scholarships

• Program in Kenya for girls scoring in top 15% on gov’t tests (Kremer, Miguel and Thornton, 2006)

• Positive effect on student test scores and teacher attendance in both districts

• Significantly higher test scores (0.12 SD, 0.19 for girls)

• Teacher attendance decreased (3.5%)• For district where program successful:

– Improvements for boys (not eligible) and girls with low pre-test scores (unlikely to win)

– Increased use of textbooks at home in last week– Reduced absenteeism by 30%, slightly higher for girls

Cost Benefit Comparisons

• Since all programs are evaluated in a similar way, and all effects are expressed in terms of standard deviation, we can compute and compare the Cost per 0.1 Sd increase in test scores

• (The graphs include only programs that had positive effects)

• This is does not tell us about the welfare effects of these programs, but this can tell us where to invest scarce resources to arrive at a given objective

Conclusions • Rigorous evidence is starting to accumulate on what

works and what does not in education access and quality.

• Results are influencing policy (PROGRESA, deworming)• Many governments in Africa are now interested in trying

this approach, often with significant World Bank help (AGEPA evaluation in Madagascar, Niger)

• We can start comparing the cost effectiveness of different approaches

• But we still need to accumulate more evidence of the same type, to try out the same programs in different places, to understand better why something may or many not work… all to make progress to get more children in school and learning well!

Cost per 0.1 Sd increase in test scores

012345

Bal

sakh

iy2

Bal

sakh

iy1

GS

P B

GS

P B

&T

Ince

ntiv

es

Cam

era

CA

L

Text

book