the impact of decentralization on schools and student performance elizabeth m. king development...

The impact of decentralization on schools and student

performance

Elizabeth M. KingDevelopment Research Group

The World BankFebruary 23, 1999

Elizabeth M. KingThe World Bank, 2/3/99

Presentation Outline

Three cases of decentralization to the local level

Evaluation issues Evaluation strategies Results Lessons

Reforms in school management Premise is that “if local participants regain

a sense of agency … they can be a powerful force for school improvement.”

Why?


Because actors with the best information about a

subject could make better decisions more quickly

about that subject ...


… and because teachers often work autonomously

inside the classroom since centralized school systems

are seldom tightly monitored.


Reforming school management

A key policy issue is how to design incentives and contracts to ensure that teachers and school heads behave in accordance with their mandate.

Public School

Principals

Teachers


Reforming school management

Reduce agency problems within schools by increasing monitoring, control, and influence by others.

Public SchoolPrincipals

Teachers

Parents

Local Community


Cases of school management reform

Chicago School Reform, 1989/90 El Salvador’s community-managed

schools or EDUCO (Educacion con Participacion de la Communidad), 1991

Nicaragua School Autonomy Reform, 1993



Chicago School Reform, 1989/90» Local School Councils (LSCs): representatives of

parents, community members and teachers» LSCs hire, fire and evaluate school principal;

advise on curriculum, textbook selection, and student discipline; approve budget and School Improvement Plan

» Additional resources to support school improvements



El Salvador’s community-managed schools or EDUCO (Educacion con Participacion de la Communidad), 1991» Community education associations (ACEs):

parents and community members» Legally responsible for operating EDUCO

schools» ACEs hire, fire, supervise teachers, administer

fund transfers from central government, raise additional resources



Nicaragua School Autonomy Reform, 1993» Consejo Directivo: principal or director, elected

teachers, parents, and students» Makes decisions pertaining to personnel, school

budget and plan, finance, and pedagogy» Revenues from students retained in the school

Nicaragua PreviousRegime

Present Regime

Function All PublicSchools

PublicSchools

AutonomousSchools

PrivateSchools

Structuring education system Ministry Ministry Ministry MinistrySetting the curriculum Ministry Ministry Ministry MinistryFormulating annual pedagogicalplan

Ministry Ministry School School

Hiring and firing teachers Ministry Ministry School SchoolHiring and firing director Ministry Ministry School SchoolPromotions policy Ministry Ministry Ministry MinistrySetting classroom hours bysubject

Ministry School Schools School

Programming extracurricularactivities

Ministry School School School

Selecting textbooks Ministry Ministry School SchoolSetting equivalencies Ministry Ministry School SchoolEvaluating students Ministry Ministry School SchoolEstablishing pedagogicalmethods

Ministry School School School

Setting school fees Ministry Ministry School School



References

Chicago» Consortium on Chicago School Research

El Salvador» DEC Research Group in collaboration with

Ministry of Education of El Salvador

Nicaragua» DEC Research Group in collaboration with

Ministry of Education of Nicaragua and international consultants

Evaluating decentralization reform

We know less about decentralization

than we claim to know.


Questions for evaluation

How are the new structures functioning?

Is there really greater local decision-making? More local participation?

How are the schools using their greater autonomy and accountability?

Has the reform influenced the classroom and student outcomes?


Evaluation issues The logic of the reform is that the

mandated change in school governance will expand local decision-making and participation and thereby lead to advances in student engagement and learning.

Indicators» Local decision-making» Local participation» Student outcomes


Evaluation issues

If participation in a reform is voluntary or if the reform content is subject to choice, it may be difficult to measure impact.» Need to explain reform participation separately

from impact» Choose appropriate evaluation design


Evaluation issues

Decentralization is often not the only reform taking place. How to isolate its effect from other changes, such as an increase in available school resources?» Identify other relevant policy changes» Need for appropriate evaluation design


Evaluation issues

Considerable diversity among communities and schools -- in resources and circumstances -- will mean variability in response to the reform.» Examine diversity» Identify factors explaining diversity


Evaluation issues

By its nature, decentralization itself can take a variety of different forms (e.g., school improvement plans).» Need to describe new management structures and

define evaluation problem accordingly


Evaluation issues

The reform may be fairly new at the start of an evaluation. When is it too early to assess the reform?» Mostly a process evaluation, a mid-course

assessment» May be difficult to assess long-term impact on

student outcomes


Evaluating impact

Evaluation method» Chicago: Surveys with recall, and case study

method» El Salvador: Matched and reflexive comparison

using school and parents survey data» Nicaragua: Matched and reflexive comparison

using school and parents survey data, plus qualitative methods


Evaluating impact

Instruments (Chicago)» Quantitative evaluation

– Survey of teachers (1991) and principals (1992)

» Case study through observation and interviews in selected schools (1993)


Evaluating impact

Instruments (El Salvador & Nicaragua)» Quantitative evaluation

– School survey with questionnaires applied to principals, teachers, and students

– Parents survey linked to school survey

– Sample of reform and non-reform schools

Nicaragua: Focus group meetings with principals, teachers, council members


Evaluating impact

Performance indicators» Local decision-making: by whom and which

decisions» Local participation: by whom and for what» Perceptions and attitudes toward the reform by

principals, teachers, parents, council members» Student outcomes -- enrollment, repetition,

continuation, achievement test scores, and attendance


Evaluation findings


Chicago, IL

Local decision-making and participation» One-half of teachers agree they have more

influence on policy» Teachers more involved with reform also more

positive about school operation» In small schools (<350), teachers significantly

more positive about the LSCs and the reform

Source: Consortium on Chicago School Research


Chicago, IL

Local decision-making and participation» Composite index of principals’ attitudes:

– 36% very positive

– 31% moderately positive

– 22% somewhat negative

– 11% very negative

» More positive when teacher collegiality high and community relations positive; small schools



Chicago, IL

Student engagement» No significant change in attendance rates after

reform; already high prior to the reform» Substantial declines in student mobility» A few schools showed gains in enrollment



Chicago, IL

Learning» Drop in grade repetition in all case schools» Principals report some improvement in test

scores, but principals cautious about interpreting short-term gains



El Salvador

Local participation» Higher participation by parents and school

councils in EDUCO schools than in traditional schools


El Salvador

Local participation

0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

5

Hours/month teachers meet with parents

5

3.5

EDUCO Traditional

0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

5

4.5

1.5

Number of ACE visits to the classroom

EDUCO Traditional


El Salvador

Student enrollment and learning» EDUCO students have 3 fewer days of absence

per year than non-EDUCO students; stronger result for newer EDUCO

» Despite being poorer, EDUCO students do as well as non-EDUCO students in achievement tests


El Salvador

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

EDUCO Traditional

Assets of students’ families

Own House With Electricity With Sanit. With Water


El Salvador

Student engagement and learning

-2

-1

0

1

2

3

4

EDUCO Traditional

Days missed Math score Language score


El Salvador

Student engagement and learning» EDUCO students have 3 fewer days of absence

per year than non-EDUCO students; stronger results for newer EDUCO

» Despite being poorer, EDUCO students do as well as non-EDUCO students in achievement tests

» ACE visits have positive effect on language scores and number of days present


Nicaragua

Questions for evaluation» Did the reform promote school autonomy?

Why would it not?» School may choose not to exercise autonomy or

lack of resources to do so» Reform may not be clear» Local actors may diverge in interpretation


Nicaragua

Questions for evaluation» Did the reform promote school autonomy?» Is the level of autonomy related to the degree of

influence perceived by principals, teachers, council members?

While participants and power-sharing are part of the reform, previous governance style may be obstructive.


Nicaragua

Dec. 1995 Dec. 1996 Dec. 1997School-household School-

householdsurvey I survey II

Gr 3 & Yr 2 Grs 4,5 & Yrs 3,4

Achievement Achievement tests I test IIGr 4 & Yr 3 Gr 4 & Yr 3

Qualitative evaluation Visit I Directors, teachers, parents

•Evaluation design

Areas of school decision-making

Salaries and incentives» Setting salaries» Establishing incentives for

teachers & administrators

Personnel» Hiring & firing teachers,

directors, administrative personnel

Classroom & pedagogy» Determining class size» Designing curriculum» Selecting textbooks

Maintenance & infrastructure» Developing infrastructure

projects» Repair buildings

Administration» Planning school budgets» Distributing textbooks» Informing community» Relations with teachers’ union

Teacher supervision, evaluation & training



Nicaragua

Local decision-making and participation» Data source: Questionnaire for principals, teachers,

council members on 25 decision areas» De jure autonomy not equal to de facto autonomy» Considerable diversity in de facto autonomy

among program schools» Evidence of spillover to non-program schools


Nicaragua

Local decision-making and participationIs School Principal the Decision-maker?

Secondary Schools, 1995 and 1997 (% Yes)

Traditional Autonomous PrivateDecision Areas1995 1997 1995 1997 1995 1997

Classroom & pedagogy 35 55 44 64 59 73

Personnel 19 25 66 74 79 84

Supervision & evaluationof teachers

64 81 71 81 78 92

Salaries & incentives 34 32 59 59 79 82

School budget & plan 50 67 88 92 88 92

Teacher training 14 45 50 79 50 79


Nicaragua

Local decision-making and participation» Additional findings:

– High consistency in reports by principals, teachers and council members

– De jure autonomy is the only statistically significant determinant of de facto autonomy, especially with respect to administrative functions


Nicaragua

Local decision-making and participation» Perceptions of local actors

– Less than one-half of respondents were positive about parent participation

– Positive view correlated with duration of de jure autonomy


Nicaragua

Local decision-making and participation

Findings from focus group meetings» Disparate understanding of role of council and

parents in new structure» Directors and teachers see reform as mainly

administrative» Parent participation on the rise -- but a few

instances of hostility between parents and teachers


Nicaragua

Student engagement and learningFindings from focus group meetings

» Reform interpreted as putting pressure on teachers and parents to improve student performance

» Focus on achievement led teachers to be more innovative, to assess student learning more regularly, to have more contact with parents


Nicaragua

Student engagement and learningPerceptions of local actors:

» About half of respondents (40-60%) were positive about student achievement. Small percentage (5-14%) were negative.

» Over half (55-65%) were positive about teachers’ attendance; parents least positive.

» Positive view correlated with duration of de jure autonomy.


Lessons from the field


About the process of reform

Time essential to clarify new roles in new governance structures

Time essential to effect real systemic change

There is no one reform across schools Conflict and anxiety may increase as

change occurs Past governance style and school-

community relationship influence adaptation to reform


About impact on learning

Short-term perceptions and test gains suggest improvement

But cautious optimism about long-term gains recommended

Leading indicators may be important -- such as decline in student absenteeism

Important to consider diversity among schools and implications about equity


About evaluating decentralization

Three-part data collection important» School and household characteristics» Student achievement tests» Focus group meetings on specific issues: Perceptions

by school personnel and parents

Different stakeholders contribute different perspectives about reform

the impact of decentralization on schools and student performance elizabeth m. king development...

Documents