the role of school quality in shaping learning gaps rolleston cies
TRANSCRIPT
![Page 1: The role of school quality in shaping learning gaps rolleston cies](https://reader030.vdocuments.mx/reader030/viewer/2022032422/55a9e50e1a28abbd0b8b459a/html5/thumbnails/1.jpg)
Equitable Progress? The Role of School Quality in
Shaping Learning Gaps: A Comparative Study of Two
Developing Countries.
Caine Rolleston
Abhijeet Singh
![Page 2: The role of school quality in shaping learning gaps rolleston cies](https://reader030.vdocuments.mx/reader030/viewer/2022032422/55a9e50e1a28abbd0b8b459a/html5/thumbnails/2.jpg)
• Large gaps between countries on common tests point to school system effects
• But also large gaps in educational performance by SES in some developing countries
• SES gaps due to both background and school quality effects (Heyneman-Loxley debate) - which are correlated
• Several YL papers address issues of the role of school quality compared to home backgrounds within/across countries
• E.g. inequality by backgrounds AND school quality especially high in Peru, more equitable in Vietnam (Glewwe, Krutikova & Rolleston 2015)
• A key equity question surrounds whether less advantaged pupils are ‘sorted’ into poorer quality schools, potentially widening SES gaps
• Ideally, independent data on learning progress and school quality required
• Availability of data from R4 and Ethiopia & Vietnam school surveys provide new evidence
• How much of SES gaps are due to ‘sorting on school quality’? • What is the impact of differences in school quality on learning
progress?
MOTIVATION AND RESEARCH QUESTIONS
![Page 3: The role of school quality in shaping learning gaps rolleston cies](https://reader030.vdocuments.mx/reader030/viewer/2022032422/55a9e50e1a28abbd0b8b459a/html5/thumbnails/3.jpg)
Young Lives longitudinal survey of children, households &
communities every 3 years since 2002
• 12,000 index children
• Ethiopia, India, Peru, Vietnam
• 20 sentinel sites in each country
• Qualitative component for a sub-sample
• Tested in maths at each round with common items
allowing IRT scaling of scores
School surveys implemented since 2010
• Focus on learning & learning progress
• School and teacher effectiveness
• Index children and their class peers
• Tested in curricular maths at beginning and end of
school year (common items/scale within country)
YOUNG LIVES STUDY
![Page 4: The role of school quality in shaping learning gaps rolleston cies](https://reader030.vdocuments.mx/reader030/viewer/2022032422/55a9e50e1a28abbd0b8b459a/html5/thumbnails/4.jpg)
AGES: 1 5 8 12 15
YOU
NG
ER C
OH
OR
T
Following 2,000 children
OLD
ER C
OH
OR
T
Following 1,000 children
AGES: 8 12 15 19 22
Round 1 Round 2 Round 3 Round 4 Round 5 2002 2006 2009 2013 2016
YL HOUSEHOLD DATA
Same age children at
different time points
![Page 5: The role of school quality in shaping learning gaps rolleston cies](https://reader030.vdocuments.mx/reader030/viewer/2022032422/55a9e50e1a28abbd0b8b459a/html5/thumbnails/5.jpg)
• 2012-2013 school year
• 7 regions
• 30 sites
• Site-level school census
• All pupils in all G4 & G5
classes (~12,000 total)
• Start and end of school
year surveys
• Child, class, teacher,
principal and school
data
• Allows identification of
learning progress over
school year
ETHIOPIA SCHOOL SURVEY
![Page 6: The role of school quality in shaping learning gaps rolleston cies](https://reader030.vdocuments.mx/reader030/viewer/2022032422/55a9e50e1a28abbd0b8b459a/html5/thumbnails/6.jpg)
VIETNAM SCHOOL SURVEY
• 2011-12 School year
• 5 provinces
• Grade 5 pupils in classes
attended by YL index
pupils
• Plus class peers to a total
of 20 per class
• Total 3,600 pupils
• Start and end of school
year survey (as Ethiopia)
![Page 7: The role of school quality in shaping learning gaps rolleston cies](https://reader030.vdocuments.mx/reader030/viewer/2022032422/55a9e50e1a28abbd0b8b459a/html5/thumbnails/7.jpg)
DISTRIBUTION OF MATHS SCORES (ETHIOPIA)
.000
5.0
01
.001
5.0
02
.002
5
Den
sity
200 400 600 800Score
Test 1 Test 2
Maths Scores at First and Second Round Tests
• Mean Test 1 = 500, SD= 100. Mean Test 2=530 (gain 0.3 SD)
![Page 8: The role of school quality in shaping learning gaps rolleston cies](https://reader030.vdocuments.mx/reader030/viewer/2022032422/55a9e50e1a28abbd0b8b459a/html5/thumbnails/8.jpg)
• School survey data for non-index children only used to estimate school value-added independently from household sample
• Approx. 90 schools in each country• Approx. 2,300 pupils Vietnam and 11,500 Ethiopia • Estimate school fixed effects for maths using beginning and
end of year test data (lagged value-added model) separately by country (IRT scales)
• Using squared & cubic terms plus pupils’ backgrounds not due to schooling
• Linked maths scores from household tests on cross-country scale (IRT)
• Household wealth on common scale & comparable controls• Insert school FE into regressions using household data• Allows identification of wealth gaps
DATA AND METHODS
![Page 9: The role of school quality in shaping learning gaps rolleston cies](https://reader030.vdocuments.mx/reader030/viewer/2022032422/55a9e50e1a28abbd0b8b459a/html5/thumbnails/9.jpg)
• Normalised scale within each country, not directly comparable• Conditioned on child backgrounds not due to schooling
0.1
.2.3
.4
kde
nsity s
choo
l_fe
-2 -1 0 1 2 3School Value Added Estimate
Ethiopia
0.1
.2.3
.4
kde
nsity s
choo
l_fe
-4 -2 0 2 4School Value Added Estimate
Vietnam
SCHOOL VALUE-ADDED BY COUNTRY
![Page 10: The role of school quality in shaping learning gaps rolleston cies](https://reader030.vdocuments.mx/reader030/viewer/2022032422/55a9e50e1a28abbd0b8b459a/html5/thumbnails/10.jpg)
-100
-50
050
10
0
Sch
ool V
alu
e-A
dd
ed (
90%
CI)
0 10 20 30 40 50School Value-Added Rank
SCHOOL VALUE-ADDED VIETNAM
• Fixed-effects on test-score scale (mean 500 at baseline)
![Page 11: The role of school quality in shaping learning gaps rolleston cies](https://reader030.vdocuments.mx/reader030/viewer/2022032422/55a9e50e1a28abbd0b8b459a/html5/thumbnails/11.jpg)
-10
12
3
Ethiopia Vietnam
1 2 3 1 2 3
2009 2013
MATHS SCORES 2009 & 13 BY WEALTH TERCILE
• 1: Poorest 2: Middle Wealth 3: Least Poor
![Page 12: The role of school quality in shaping learning gaps rolleston cies](https://reader030.vdocuments.mx/reader030/viewer/2022032422/55a9e50e1a28abbd0b8b459a/html5/thumbnails/12.jpg)
0.1
.2.3
.4.5
-2 0 2 4 6Math scores
Ethiopia 2009 Vietnam 2009
Ethiopia 2013 Vietnam 2013
Distribution of Test Scores 2009 and 2013
• Maths scores from household surveys R3 and R4• Standardised using Ethiopia R3 mean and SD• Ethiopian pupils typically exposed to fewer years of schooling
![Page 13: The role of school quality in shaping learning gaps rolleston cies](https://reader030.vdocuments.mx/reader030/viewer/2022032422/55a9e50e1a28abbd0b8b459a/html5/thumbnails/13.jpg)
0.2
.4.6
01
23
4
-2 0 2 4 -2 0 2 4
Ethiopia Vietnam
Most Poor Least Poor
Math
score
s (
201
3)
Math scores (2009)
Graphs by country
Maths scores
Learning divergence by wealth groups
• Kernel density R3 (2009)• Stronger relationship with prior score in Ethiopia, broadly similar gaps by wealth• Pupils typically make more gains over time in Vietnam (not linked to prior scores)
![Page 14: The role of school quality in shaping learning gaps rolleston cies](https://reader030.vdocuments.mx/reader030/viewer/2022032422/55a9e50e1a28abbd0b8b459a/html5/thumbnails/14.jpg)
LEARNING GAPS
• Most poor group is reference category
• Compare wealth gaps (middle and least poor):
1. No controls, dummy variables only
2. Add backgrounds – parental education etc.
3. Add prior test scores
4. Add school effectiveness in mathematics
5. Add site fixed effects
• Differences in scores in 2013 strongly correlated with wealth
• Average gain score is 0.86 SD over 4 years, similar to gap in Ethiopia in 2013 between poorest and least poor (4 years’ schooling) more like 3 years in Vietnam
![Page 15: The role of school quality in shaping learning gaps rolleston cies](https://reader030.vdocuments.mx/reader030/viewer/2022032422/55a9e50e1a28abbd0b8b459a/html5/thumbnails/15.jpg)
No Controls + Backgrounds + Prior Scores + School Eff + Site FE
Middle Wealth Group 0.3914 0.1567 0.1548 0.1590 0.1398
(6.59)*** (2.52)** (2.56)** (2.64)*** (4.69)***
Least Poor Group 0.7406 0.2835 0.2289 0.2146 0.0913
(12.15)*** (3.87)*** (3.21)*** (3.01)*** (1.42)
School Effectiveness 0.0747 -0.0315
(2.85)*** (-1.04)
Constant 2.4685 2.6394 2.8336 2.7113 2.1602
(57.22)*** (2.22)** (2.83)*** (2.71)*** (7.44)***
Observations 1,317 1,315 1,307 1,307 1,276
R-squared 0.101 0.239 0.285 0.290 0.200
Number of sites 19
VIETNAM
• Backgrounds reduces wealth gaps by >half• Prior scores small effect – wealth gaps not strongly linked to earlier test score gaps (some
catch-up)• School effectiveness no change - little evidence of sorting• Site fixed effects reduces gaps for the rich group – suggests clustering of progress in sites• No evidence of school sorting/differences in school effectiveness within sites• Relatively equitable regarding ‘school choice’
![Page 16: The role of school quality in shaping learning gaps rolleston cies](https://reader030.vdocuments.mx/reader030/viewer/2022032422/55a9e50e1a28abbd0b8b459a/html5/thumbnails/16.jpg)
-1-.
50
.51
mea
n o
f scho
ol_
fe
1 2 3 4 5 6 7 8 9 10 11 12 13 15 16 17 18 19 20
VARIATION IN SCHOOL EFFECTIVENESS BY SITE
![Page 17: The role of school quality in shaping learning gaps rolleston cies](https://reader030.vdocuments.mx/reader030/viewer/2022032422/55a9e50e1a28abbd0b8b459a/html5/thumbnails/17.jpg)
• Similar wealth gaps & reduction with backgrounds
• Reduced further by half when prior scores added (and no longer significant) - wealth gaps linked to earlier test score gap persistence
• No further reduction when school effectiveness is added – little evidence of sorting
• Similar results with site fixed effects – little evidence of site-level clustering
• Strong effects of differences in ‘school effectiveness’ including within sites
• Again ‘school-choice’ not leading to inequity
VARIABLES No Controls + Backgrounds + Prior Scores + School FE Site FE
Middle Wealth Group 0.3700 0.1886 0.0833 0.1141 0.1017
(4.40)*** (2.16)** (1.09) (1.52) (1.52)
Least Poor Group 0.8447 0.2549 0.1325 0.1487 0.1242
(9.95)*** (2.23)** (1.33) (1.52) (1.14)
School Effectiveness 0.1826 0.1566
(5.38)*** (2.34)**
Constant 0.4863 1.9980 1.2013 1.1298 2.1691
(7.75)*** (1.67)* (1.65) (1.57) (3.40)***
Observations 863 863 843 843 839
R-squared 0.105 0.213 0.422 0.442 0.301
Number of sites 17
ETHIOPIA
![Page 18: The role of school quality in shaping learning gaps rolleston cies](https://reader030.vdocuments.mx/reader030/viewer/2022032422/55a9e50e1a28abbd0b8b459a/html5/thumbnails/18.jpg)
• School VA estimate 0.157 – a one standard deviation change in school effectiveness increases 2013 score by 0.157 SDs (conditional on Xs)
• Approx. equivalent to moving from 25th to 55th
percentile of school effectiveness distribution
• Apparently effect at least as large as effect of moving from poorest to least poor wealth tercile
• Much less evidence of variation in effectiveness in Vietnam
IMPACT OF SCHOOL QUALITY -ETHIOPIA
![Page 19: The role of school quality in shaping learning gaps rolleston cies](https://reader030.vdocuments.mx/reader030/viewer/2022032422/55a9e50e1a28abbd0b8b459a/html5/thumbnails/19.jpg)
-2
-1
01
mea
n o
f sch
oo
l_fe
Addis AbabaAmhara Oromiya SNNP Tigray Somali Afar
-.5
0.5
11.5
mea
n o
f sch
oo
l_fe
government private community faith-based ngo
VARIATION IN SCHOOL QUALITY (ETHIOPIA)
![Page 20: The role of school quality in shaping learning gaps rolleston cies](https://reader030.vdocuments.mx/reader030/viewer/2022032422/55a9e50e1a28abbd0b8b459a/html5/thumbnails/20.jpg)
-.5
0.5
1
mea
n o
f sch
oo
l_fe
shift school full-day school Mixed school
VARIATION IN SCHOOL QUALITY (ETHIOPIA)
![Page 21: The role of school quality in shaping learning gaps rolleston cies](https://reader030.vdocuments.mx/reader030/viewer/2022032422/55a9e50e1a28abbd0b8b459a/html5/thumbnails/21.jpg)
FINDING OUT MORE…
www.younglives.org.uk
• methodology
• datasets (ESDS International)
• publications
• child profiles and photos
• e-newsletter