decentralization and the quality of teaching

33
Decentralization and the quality of teaching Steven Rivkin April 2014

Upload: micah-vance

Post on 03-Jan-2016

18 views

Category:

Documents


0 download

DESCRIPTION

Decentralization and the quality of teaching. Steven Rivkin April 2014. The decentralization of public schooling typically shifts authority and responsibility from the ministry of education to municipalities - PowerPoint PPT Presentation

TRANSCRIPT

Page 1: Decentralization and the quality of teaching

Decentralization and the quality of teaching

Steven Rivkin

April 2014

Page 2: Decentralization and the quality of teaching

• The decentralization of public schooling typically shifts authority and responsibility from the ministry of education to municipalities

• Decentralization affects many aspects of schooling, but I focus on its effects on the quality of teaching and instruction

Page 3: Decentralization and the quality of teaching

Outline of Talk• Discuss selected evidence on the effects of

high-quality instruction and professional development

• Discuss potential advantages and challenges of decentralization, focusing on the quality of teaching

• Compare decentralization and achievement trends in the Czech Republic and Poland

• Outline plan for teacher and principal evaluation efforts in Dallas, Texas

Page 4: Decentralization and the quality of teaching

Evidence on the effects of teacher quality on future earnings

• Investigates the effect of 4th-8th grade teacher quality in New York City on earnings– quality is measured as the contribution of a

teacher to test score growth• This is only one dimension of teacher quality• math score 5th grade minus math score 4th grade• control for differences in family income and special

needs

– compares teachers within the same school who teach children with similar backgrounds

Page 5: Decentralization and the quality of teaching

Rich data to learn about schools

• New York tests all children in grades 3 to 8 every spring in math and reading

• Information on income and special needs

• School data link teachers to their students– Can follow students over time

• Enables measurement of teacher quality

• Tax data on earnings for these students as adults is matched with school data

Page 6: Decentralization and the quality of teaching

Results

• Value added to achievement is clearly a partial and flawed measure of teacher quality, BUT these results show it is strongly related to future student earnings

• Having a teacher who is one standard deviation better in terms of raising achievement (moving from the middle of the distribution past 1/3rd of teachers) a student’s increases annual earnings in the future by roughly 1% on average– Multiply this effect by 25 students in the class and by

the number of years a student works to calculate the total benefit of a better teacher (many $1,000s)

Page 7: Decentralization and the quality of teaching

Benefits of Data on Schooling and Outcomes

• Data make such research possible• Data enable schools to evaluate teacher effectiveness

as part of the hiring, retention and pay processes• Data enable schools to evaluate principal effectiveness

as part of the hiring, retention and pay processes• Data also provide information for parents and the public

– In the US laws protect the privacy of students– Personnel information on employees including measures of

teacher performance is generally private– Publicly available information on school performance shows

average achievement, achievement growth and pass rates, graduation and attendance rates, and other averages

Page 8: Decentralization and the quality of teaching

Relevant teacher effects publications of the research by Raj Chetty, John Friedman and Jonah Rockoff

• “Great Teaching: Measuring its Effects on Students’ Future Earnings,” Education Next, Summer 2012 (not technical)

• Measuring the Impacts of Teachers II: Teacher Value-Added and Student Outcomes in Adulthood,” American Economic Review, forthcoming (complete analysis)

• “Big Study Links Good Teachers to Lasting Gain,” by Annie Lowrey, New York Times, January 6, 2012

Page 9: Decentralization and the quality of teaching

Research on Benefit of Teacher Evaluation

• Evaluation that potentially affects teacher pay and job retention provides incentives for teachers to increase effort

• Evaluation of teachers also provides information on teacher performance that can be used by school administrators the mentoring of teachers in order to support their efforts to improve

Page 10: Decentralization and the quality of teaching

Study of Teacher Improvement

• A large US city put in a teacher evaluation plan as part of a performance pay system– teachers were observed and then met with the

principal to discuss their performance and specific areas that needed improvement

– average test score growth in math and reading for each classroom teacher was calculated

• This study compares average growth after evaluation with average growth before evaluation

Page 11: Decentralization and the quality of teaching
Page 12: Decentralization and the quality of teaching

Relevant evaluation effects publications of the research by Eric Taylor and John Tyler

• “Can Teacher Evaluation Improve Teaching?,” Education Next, Fall 2012 (not technical)– Source for plot

• “The Effect of Evaluation on Teacher Performance,” American Economic Review, forthcoming (complete analysis)

Page 13: Decentralization and the quality of teaching

US Department of Education funded study on intensive professional development for

mathematics instruction

• Average differences between the teachers in the treatment and control schools are unbiased estimates of program effects– Randomization eliminates systematic differences

• Schools were randomly assigned to two groups– Teachers in the treatment schools received summer

workshops, seminars and coaching (114 hours total)– Teachers in the control schools participated in any

existing programs at the school but not the intensive professional development

Page 14: Decentralization and the quality of teaching

Results after second year of program• No significant effect on mathematics achievement

– Average score in treatment schools is not significantly higher than the average in control schools

• No significant effect on teacher knowledge• These results cast doubt on the idea that in-service

training is effective at improving the quality of teaching– These were expensive, highly regarded professional

development programs

• In the US there are a number of random assignment studies done by both academics and research firms that provide valuable information that can guide policy– In many cases these avoid expensive but ineffective programs

Page 15: Decentralization and the quality of teaching

Citation

• Garet, M., et al, (2011). Middle School Mathematics Professional Development Impact Study: Findings After the Second Year of Implementation (NCEE 2011-4024). Washington, DC: National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, U.S. Department of Education.

• Report available on the IES website at http://ncee.ed.gov• This study was completed by analysts at policy research

firms

Page 16: Decentralization and the quality of teaching

Summary of Results

• These three studies suggest that school leaders have an important role to play in raising the quality of instruction through the selection, mentoring, evaluation, and contract renewal decisions– Evidence suggests that large professional development

programs or simple rules to govern teacher hiring are not likely to be effective

– Rather it is a leader’s skill at identifying good teachers to hire, providing good feedback on teacher performance, and making decisions about which teachers to keep and which to let go that appears to be much more important

• If school decentralization improves the quality of school leadership it may be a very good policy

Page 17: Decentralization and the quality of teaching

School Decentralization

• Responsibility for many decisions shifts from central authority to local institutions, often in response to dissatisfaction with the centralized system

• Municipalities must develop the capacity to produce high-quality schools– A particularly important decision is the hiring

of a principal, who will have responsibility for virtually all aspects of school operations

Page 18: Decentralization and the quality of teaching

One view of the decentralization experiences of the Czech Republic and

Poland• Much authority was transferred from the

central education ministry to local municipalities in both countries

• However– Central ministry retained much more

involvement with regards to curriculum and evaluation of schooling in Poland

Page 19: Decentralization and the quality of teaching

Percentage of decisions taken at each level of government in public lower secondary education (2011)

Page 20: Decentralization and the quality of teaching

• Decline in the Czech central ministry support for school administration and involvement in evaluation and curriculum may have been harmful– There was likely a loss in the capacity to support high-

quality instruction– As Daniel Münich showed, the decline in support

likely caused many school leaders to focus more attention on school operations and less on instruction

• This does not appear to have been the case in Poland

Page 21: Decentralization and the quality of teaching

• Some municipalities in the Czech Republic may have devoted less effort to ensuring high education quality, particularly in terms of school principal selection and retention decisions

– This may have been more of a problem in lower-income communities where parents may be less involved in pushing for a higher quality of education

Page 22: Decentralization and the quality of teaching

• The absence of national monitoring of educational outcomes in the Czech Republic complicates efforts to evaluate school performance– It is difficult to identify a decline in quality in a

school, municipality or the country as a whole without evidence of outcomes that can be compared with other schools or years

– It complicates efforts to manage and support teachers

• Areas where students are struggling are not illuminated

Page 23: Decentralization and the quality of teaching

The Czech response to salary increases in other occupations following labor-market

decentralization was far weaker than in Poland

• As shown by Professor Munich, teacher salaries relative to other occupations that employ college graduates increased much more in Poland than in the Czech Republic during the 1990s and 2000s– Many older teachers may choose to remain in

teaching, because it is more difficult to switch occupations as you get older

Page 24: Decentralization and the quality of teaching

– But younger teachers and students with better opportunities are more likely to move away from teaching if salaries of teachers fall further below salaries in other occupations

• Professor Munich will provide some evidence

• The sharp decline in the school-age population in the Czech Republic during the late 1990s and 2000s likely masked the problem of teacher retention– Fewer new teachers were needed because of the

decrease in the number of school-age children

Page 25: Decentralization and the quality of teaching
Daniel Munich
These are averages. Cannot you get average score differece of the bottom quartiles or somethin like that? From a presentation in Budapest last week by the former vise minister for schooling in Poland I noted that great deal of the Polish success was due to bringing up low performing students who had been (before the reform) thrown into low quality upper-secondary vocational schools and when tested by PISA they already lost one year there. The reform extended their studies in the mainstream schools and one additional year helped them a lot.
Page 26: Decentralization and the quality of teaching

Evidence highlights importance of high-quality instruction and school leadership

• High quality teaching cannot be produced simply through regulations, including mandatory professional development or teacher education programs

• Rather it requires good decisions regarding teacher hiring, teacher support and teacher mentoring– Good systems of evaluation of both teachers

and school leaders support good decisions

Page 27: Decentralization and the quality of teaching

Dallas Independent School District

• Dallas school administration– Elected school board hires and evaluates the

superintendent– Superintendent supervises area

superintendents who govern different parts of the city

– Area superintendents supervise school principals

– This is a typical structure for the US

Page 28: Decentralization and the quality of teaching

Dallas Evaluation Systems

• Principal evaluation by school district administrators has been in place since 2013

• A teacher evaluation system has been proposed but not yet approved or implemented

• Both systems combine supervisor observations of performance with data on student outcomes

Page 29: Decentralization and the quality of teaching

Availability of Assessments is Crucial

• The district is developing assessments available to supervisors in order to ensure that there is student outcome data for all subjects and grades– State test score data are available only for

certain subjects and grades– District assessments cover all subjects

including music, art and physical education• This is a very ambitious undertaking• The goal is to treat all teachers similarly

Page 30: Decentralization and the quality of teaching

Teacher evaluations

• Teacher evaluation combines the ratings based on observations of teaching completed by the principal typically with the assessment data– Assessments include estimates of student

achievement growth– School principals have an incentive not to give very

high ratings based on observations, because their own evaluation suffers if teacher ratings differ substantially from the student achievement growth and other outcomes

Page 31: Decentralization and the quality of teaching

Addressing the serious problem of rating inflation

• In previous evaluation systems it was typical for almost all teachers to be rated as good or excellent– There was little incentive for high standards– Student outcomes were not considered typically

• This district requires some fraction of teachers and some fraction of principals to receive lower ratings and punishes principals who give a teacher with low student achievement growth or low outcomes in other areas a high rating

Page 32: Decentralization and the quality of teaching

Basing pay on performance

• The proposed evaluation system would eliminate the previous practice of basing teacher salary entirely on experience and level of education (often specified in union contracts)

• Salary would be influenced by experience but determined largely by performance

Page 33: Decentralization and the quality of teaching

Benefits of strong evaluation systems

• Teachers and principals have strong incentives to work hard to raise the quality of instruction and produce better outcomes for students

• More skilled individuals who expect to be successful as teachers or principals have a stronger incentive to enter teaching