latin american experience with enhancing quality and measuring quality south asia regional...
TRANSCRIPT
Latin American Experience with Latin American Experience with Enhancing Quality and Measuring Enhancing Quality and Measuring QualityQuality
South Asia Regional Conference on Education QualityNew Delhi, India
October 24-26, 2007
Eduardo VelezSector Manager for EducationHuman Development SectorLatin America and the Caribbean
Current Situation in Latin AmericaCurrent Situation in Latin AmericaAssessment SystemsAssessment Systems
- Most countries have at least an incipient national assessment system based on standardized student achievement test, periodically applied to samples or all students of certain key grades in core academic subjects.- Some countries have sub-national assessments systems-Most have participated in one or more international text--A few countries and sub-national entities have been producing school- and system-report cards
Report CardsReport Cards
-Various models-Different variables-Different processes-All lead to focusing on outcomes, some give more space for consideration of processes-All promote improvement and accountability among various stakeholders-
Assessment Systems. What kinds Assessment Systems. What kinds of results are found? of results are found?
-Lower than expected-Huge differences in averages between rural and urban population, public and private schools, poor and non-poor, indigenous and non-indigenous populations-Increasingly small, if any, differences between girls and boys.-Slow change in outcomes
Assessment Systems. What kinds of Assessment Systems. What kinds of results are found? Positive resultsresults are found? Positive results
The following are important inputs: school climate, high expectations, principal’s leadership and permanence, homework, peer effects, educational materials, teacher’s satisfaction and knowledge of subject matter, active pedagogy, parents’ SES and participation, use of classroom assessment as a pedagogical tool, Time on homework, interest in subject, student perception of relationship with teacher, understanding that science & math associated w/ better job opportunities & future financial security, Mother’s education, home educational resources, assessment systems, school autonomy (process and human resources), less influence from unions, …….. just like everywhere else!!!!!!!!!!!!!!
Assessment Systems. What kinds Assessment Systems. What kinds of results are found? Negative of results are found? Negative
resultsresults
The following characteristics have a negative impact: -Memorization, rote learning-Mother’s employment-Number of siblings-High student-teacher ratio-Frontal teaching method-Technology has mixed results
Assessment Systems & Report Assessment Systems & Report Cards. Big challenge. Cards. Big challenge.
Quality of Education is the challenge
It should take 2 to 3 (at very most) grades to learn to read. If it is taking 4 to 6, are systems working at about 50% effectiveness?Poorest 53 countries spend $16 billion on primary education, are they “wasting” $8 billion of it?
PISA 2000: Math Scores & Dispersion
ISR
ARG
ROMBUL
BEL
GRC
PER
DEU
ALB
RUS
LIE
LVA
CHE
POL
NZL
USACZE
HUN
BRA
MKD
SWE
CHL
HKG
AUT
GBR
NO
LUX
AUS
PRTITA
ESP
FRA
NLDJPN
DNK
CAN
ISL
IRL
IDN
THA
MEX
KORFIN
Mean
Score
Dispersion
200
400
600
280240 340 380 420
300
350
400
450
500
550
600
240 260 280 300 320 340 360 380
Dispersion
Sc
ore
MEX
Finland
IDN
Thailand
Tunisia
Serbia
Uruguay
BZL
TUR
Greece
Belgium
US
Italy
HK
GermanyIreland
Canada
Latvia
Macao
Korea
NLD
RUSPurtugal
JPN
CZK
AUS
HUN
SWEIcelandDEN
POL
ESP
SwitNZD
NOR
Austria
FRAFRA
PISA 2003: Math Scores & DispersionPISA 2003: Math Scores & Dispersion
Assessment and AccountabilityAssessment and AccountabilityThe Uruguay Experience (1)The Uruguay Experience (1)
Participation, consensus building and face to face discussion with teachers, principals, and supervisors (sample, all schools get results, all teachers can apply the test to their students, and can compare with national results)
Assessment and AccountabilityAssessment and AccountabilityThe Uruguay Experience (2)The Uruguay Experience (2)
In-service teacher training as the first consequence of the assessment (starting with assessments’ results, voluntary and collective --involves the teacher team of a school--, exchange of experience with other teams, all year round—once a fortnight; participants receive 20% of salary; schools in poor areas are priority; focus on how to teach; and emphasis on practical activities in the classroom)
Assessment and AccountabilityAssessment and AccountabilityThe Uruguay Experience (3)The Uruguay Experience (3)
Evidence on the impact-70% of teachers support national assessment-70.2% read MOE’s publications-55% changed teaching and evaluating practices-78% apply school based assessment
“When MOE appeared I had a brick in each hand.. Little by little they convinced us.. Now it is a valuable experience. We have changed our practice”
Evolución de resultados por contexto - Lenguaje en porcentajes de alumnos suficientes
51,4
46,7
37,1
54,8
60,8
48,5
66,4
62,3
58,4
75,5
73,970,2
87,7
88,085,4
66,5
61,6
57,1
30
40
50
60
70
80
90
100
Muy Desfav. Desfavorable Medio
Favorable Muy Favorable TOTAL PAÍSEVOLUTION OF THE RESULTS BY SOCIAL
CONTEXT
Language, percentage of students achieving an acceptable level Language, percentage of students achieving an acceptable level in the test /Primary 6in the test /Primary 6thth grade grade
Poor ………………………………………Rich
Assessment and AccountabilityAssessment and AccountabilityThe Mexican Experience (1)The Mexican Experience (1)
Increase AutonomyTo improve quality, efforts are needed to move decision-making to the school level, thus increasing school autonomy
Increasing school autonomy can compensate disadvantaged schools
Autonomy can help raise the schooling outcomes of indigenous peoples
School autonomy reinforces the role of homework, learning styles and future value of education
With more autonomy, schools could determine the appropriate mix of technology for their students
Assessment and AccountabilityAssessment and AccountabilityThe Mexican Experience (2)The Mexican Experience (2)
Improve AccountabilityAccountability mechanisms can improve school quality
Accountability mechanisms that put people at the center of service provision can go a long way in making services work and improving outcomes
Flexible and wide-ranging accountability mechanisms could encompass various types of services
To improve quality, efforts are needed to move decision-making to the school level, thus increasing school autonomy
Assessment and AccountabilityAssessment and AccountabilityThe Mexican Experience (3)The Mexican Experience (3)
Continue learning from the assessmentAssessment testing can be used to inform policy decisions. Analysis of assessments can foster public and civil society involvement in education reform. However, governments must be proactive in encouraging public debate using assessment results. Expand coverage of the national assessments. National and international assessments could be used to inform school reform process
PISA 2003: MathematicsPISA 2003: Mathematics
GreciaDinamarca
Portugal
Colima DF
AguascalientesUruguay Serbia TurquiaTailandia
Mexico
350
400
450
500
Assessment and AccountabilityUsing Early Grade Reading (EGR)
Some start to use it to monitor reading but also to increase involvement of parents and other stakeholders (an accountability mechanism). It is not an alternative to assessment systems. Let’s see some pros and cons.
Math and literacy among 15-year-olds,
or around grade 9 Early-grade reading Closer to labor market in time Can measure skills closer to real competencies Maybe more meaningful to “moneyed” stakeholders (MinFin, cabinet)
Con: a score on a test is not exactly a standard or goal, at least not until the test has gained very wide currency
Measures at time when can remediate More quickly remediable if start now Easier for parents to use for localized accountability Easier to create social movement around it Probably easier to build into teaching strategies
Where are the countries?
Chile, Grade 1: “We expect children to read fluently and with
comprehension. This means that they should be able to:
read, fast enough not to impede comprehension, stories of about 200 words;
identify the type of text read; comprehend literal meaning; and make simple inferences.” (Paraphrase.)
This is measurable, or can lead to something measurable
Peru, Haiti, Honduras
Are there any results (from EGR)?
To early to say. But in Peru there is some evidence that introducing EGR at the school level has a significant return. In six schools in five municipalities in Peru, after six months of introducing EGR the changes were on average 80% and the worst off the school (in terms of education quality) the biggest the impact.
Also in Peru the Government decided to use the results of an assessment system to evaluate its education policy starting with a baseline at the beginning of the Administration.
Some lessons for Assessment PolicySome lessons for Assessment Policy
Assessment Unit must be committed with producing Assessment Unit must be committed with producing materials useful for teachers and with dissemination materials useful for teachers and with dissemination and use of results. (School reports with useful and use of results. (School reports with useful information about their performance and activities)information about their performance and activities)
Technical legitimacy of tests and frameworks is Technical legitimacy of tests and frameworks is crucial crucial
Timely and accurate data to inform policymakingTimely and accurate data to inform policymaking
Unit must be autonomous from political interests Unit must be autonomous from political interests
Importance of detailed planning of actions and Importance of detailed planning of actions and coherent implementationcoherent implementation
Some lessons for Assessment PolicySome lessons for Assessment Policy
You need an assessment strategy, not just You need an assessment strategy, not just to administrate tests… policymakers use the to administrate tests… policymakers use the results of evaluation of existing interventions results of evaluation of existing interventions to inform design and implementation of to inform design and implementation of policiespolicies
A A "teacher-friendly" approach to "teacher-friendly" approach to assessment facilitates its use by teachersassessment facilitates its use by teachers
Articulate dissemination of results with an Articulate dissemination of results with an effective in-service training programeffective in-service training program
Teachers need space and time to meet, Teachers need space and time to meet, study, discuss, try new thingsstudy, discuss, try new things
Some lessons for Assessment PolicySome lessons for Assessment Policy
Autonomy… with support. More Autonomy… with support. More autonomous schools can implement autonomous schools can implement appropriate education policiesappropriate education policies
Accountability. A more accountable system Accountability. A more accountable system will encourage more active participation by will encourage more active participation by parents, teachers, and others, which is key to parents, teachers, and others, which is key to improving learning outcomesimproving learning outcomes
Assessment. A system that is based in Assessment. A system that is based in constant assessment and participation in constant assessment and participation in international benchmarking exercises will international benchmarking exercises will improve cost-effectivenessimprove cost-effectiveness