ASER data – informing policy debates

Round Table Discussion : Cambridge University – 12 June 2014

Speakers: Baela Raza Jamil, Monazza Aslam and Shaheen Sardar Ali

Evidence based policy

• Education research – undergoing revolution.• ASER – example of large scale initiatives driven by

desire to base policies on scientific evidence.• An important aspect of evidence-based policy is the

use of scientifically rigorous studies using quality data to identify programs and practices capable of improving policy-relevant outcomes.

Fostering Evidence-based policy and research

Quality Data

Political Support

Good Analytical

Skills

Using ASER data to highlight key messages through…

• Annual Reports;• Policy Briefs;• Working Papers;• Studies using ASER data published in peer reviewed

journals;• Newspaper articles;• Economic Survey of Pakistan;• Blogs, discussion forums, twitter & other social media

Policy briefs – food for thought?

Education and Learning in Pakistan

• A Story of Disparities (Monazza Aslam and Shenila Rawal)

Expanding Equitable Early Childhood Care

• An Urgent Need! (Sahar Saeed)

Standardizing Early Childhood Education

• Student Achievement and Teacher Qualifications (Zara Khan)

Right to Education

• The Impact of Parents’ Literacy on Children’s Education Access and Learning

The Link of Pupil Teacher Ratio (PTR)

• To Student Learning Achievements in Pakistan (Ravish Amjad)

Comparative Analysis of Urban

Polarization • Education for

Sustainable Development (Huma Zia)

Learning – a story of disparities

• Education systems around the world are characterised by persistent inequalities that manifest in different ways - by gender, by class, by religion, caste or ethnicity and in others through socio-economic status or disability.

• Inequalities in access and in differences in learning and achievement and consequently, differences in the life they eventually lead.

• Pakistan – beset by inequalities since 1947.

Until recently…

• it has not been possible to document the exact extent of differences in learning outcomes by gender and especially across different districts in the country.

• The availability of the Annual Status of Education Report (ASER) data, however, allows us to overcome this constraint and paint a picture of the nature of gender gaps in key learning outcomes across 32 districts in the country.

Progressive data collection…

• The ASER team has collected data on learning outcomes (as well as other indicators such as proportion of out of school children, the incidence of tuition-taking, parental education levels etc.) effectively since 2010.

• The coverage of districts across the country has been progressive with 32 districts covered in 2010, 85 in 2011, 136 in 2012 and 138 in 2013.

• For the first time, it has become possible to generate a data set that has collected information over consecutive years on crucial education indicators.

Aslam and Rawal (2014)

• Use data from 2011, 2012 and 2013 to document gender gaps in key learning outcomes.

• Data rendered to averages at the district level, resultant 32 data points (representing the 32 districts) on which we have data across the two time periods.

• Note: ASER survey does not revisit the same households across two time periods and even villages are replaced over any two consecutive years. While not a perfect panel, the datasets at hand do allow for a descriptive analysis that can nevertheless be informative and indicative if not necessarily able to establish causal linkages.

• ASER selects its villages through Probability Proportional to Size (PPS) methodology. A sample of 30 villages is selected from one district such that 20 new villages are selected every year and 10 villages from the previous year are retained for comparative analysis purposes.

A story of disparities in learning levels across districts: % unable to read in 2011 and 2013

What can we say?

While we cannot draw causal inferences from these basic descriptive statistics, two findings are very clear from the charts:• The pattern indicates wide differences across districts in terms of

childrens’ competencies in both their local language (Urdu/Sindhi/Pashto) and mathematics outcomes.

• While there appears to be some improvement in both reading and mathematics competencies among the tested children over the short time period being studied, it is clear that children in these rural districts face severe learning challenges in basic literacy and numeracy competencies – a very large proportion of children of school-going age are illiterate and do not possess the basic literacy and numeracy skills needed to make them productive contributors to society.

A story of wide, widening and persistent gender gaps in learning outcomes

Mathematics Achievement LevelOf Child

2011 2012 2013

M F M F M F

Nothing 17.1 24.5 *** 20.3 28.1 *** 15.6 20.9 ***

Numbers 1-9 14.8 16.8 ** 13.8 13.1 14.4 16.6 **

Numbers 10-99 21.0 21.6 20.9 20.0 23.1 21.6 **

Subtraction (2 digit) 18.1 16.1 ** 15.9 12.5 ** 18.4 16.3 **

Division (2 Digits) 27.4 22.2 *** 29.7 24.4 *** 28.8 24.7 ***

A story of differences in access to private tuition

• Shadow education sector – emerging ‘industry’ in the region.• children are coached, out of school hours, to achieve better in state-

wide exams. • Has raised pertinent questions regarding the equity, efficiency and

social justice implications of the rise of this education sector. • The option of giving (for the teachers) and receiving (for the pupils)

tuition outside of normal school hours changes the incentive structure of the provision of high quality instruction within the standard school system which in turn has implications for equity and social justice.

• The relationship between private tutoring and student achievement is also increasingly gaining policy attention as it calls into question the quality of schooling during usual school-hours.

District-wise data reveal:

• The incidence of private tuition taking is systematically high, especially in certain districts.

• For example, as many as 25% of the male children aged 5-16 sampled in Sheikhupura report taking private tuitions outside school hours.

• There is significant diversity in uptake of private tuitions across the districts with some displaying very high uptake as compared to others.

• In almost all of the districts in 2013, the incidence of tuition taking appears to display a pro-male gender bias in that boys are on average more likely to be taking private tuitions as compared to girls.

• In some instances the gaps are especially large – for example in Sheikhupura, Multan, Rawalpindi, Mianwali, Islamabad, Faisalabad and Gotki among others.

% taking private tuition(aged 6-16)by gender and district

Analyses such as above raise questions and generate debate…

• Differences in access to human capital are perhaps one of the most critical dimension of inequality of opportunity.

• The short descriptive analysis in the brief discussed here has shown the extent to which educational inequalities exist and manifest themselves in the form of large gender gaps or differences across districts, even within the same provinces.

• What is more, the analysis has clearly shown that the last few years have seen no systematic improvement in key educational indicators. Gender and regional differences are two of the most persistent contenders generating multiple disadvantages.

Glass half full?

• The glass half full picture certainly shows Pakistan having made great strides in improving educational access to girls and more broadly to children in different regions in rural areas, the glass half empty picture certainly indicates worryingly persistent gaps that still prevail.

• This is especially worrying because these gaps exist in what children know (or rather do not know), something that has the ability to alter their life chances.

Targeted policy

• Policy clearly needs to target the more systemic and deep rooted issues that both generate and perpetuate biases.

• In terms of educational access (or even access to private tutoring) for instance it could mean using targeted social policy and media campaigns to alter cultural prejudice against girls.

• There is clearly a need for policy reform to target girls’ learning outcomes and identify at-risk individuals to provide support to ensure meaningful learning.

What does this mean for global learning goals?

Data like ASER and the subsequent analyses highlight:• The need to focus on the marginalised – be it by gender,

by caste, by socio-economic status etc;• Measuring outcomes is critical – having well defined

targets essential;• Measuring progress (or lack thereof) is critical;• Debate is crucial – but it needs to be based on scientific,

evidence-based research that feeds further into discussions surrounding the post 2015 agenda for education indicators.