how we did the story “grading the teachers” molly bloom / stateimpact ohio * patrick o’donnell...
TRANSCRIPT
How We Did The Story“Grading the Teachers”
Molly Bloom / StateImpact Ohio * Patrick O’Donnell / The Cleveland Plain Dealer@m_bloom * [email protected] * [email protected]
The Series
Read● Overview: Using Data To Evaluate Teachers● Pay vs. Value-Added Performance● Secrets Of Two “Most Effective” Teachers● Value-Added’s Poverty Factor● How is Value-Added Calculated?
Multimedia● Audio: Measuring Performance Through Growth● Audio: Push for Performance Pay● Video: Guide to Ohio’s New Way of Evaluating Teach
ers
Data● Search Ratings For 4,200 Ohio Teachers● Map: See Teacher Ratings at Your School● Why We Published Teachers’ Ratings● About Our Analysis
Grading the Teachers: Where the
“Most Effective” Teachers Are (MAP)
Search Teacher Ratings
http://www.cleveland.com/datacentral/index.ssf/2013/06/find_ohio_teacher_ratings_base.html
http://stateimpact.npr.org/ohio/2013/06/16/grading-the-teachers-search-value-added-grades-for-4200-ohio-teachers/
What We Found
● Some teachers find new evaluations useful, some don’t
● Little relationship between value-added scores and teacher pay
● Clear relationship between value-added scores and teachers’ ages
● Clear relationship between the % of poor kids in a school and teachers’ value-added scores
The department’s top research official, Matt Cohen,
acknowledged that he can’t explain the details of
exactly how Ohio’s value-added model works. He
said that’s not a problem.
“It’s not important for me to be able to be the expert,”
he said. “I rely on the expertise of people who have
been involved in the field.” �
How It Began
We had some questions about value-added…
1) Is it really independent of student family income?2) Does it show teachers getting better over time? In
keeping with their pay increases?3) Does it show any difference between teacher prep
programs?4) Does a masters degree improve teacher performance,
as shown by value-added?
State’s contention: Performance may be related to income…
But value-added isn’t.
Teachers who receive top value-added rating are more likely to be in schools with fewer poor students
How We Did It
● Talking with teachers
● Interviewing experts
● Working with data
Teachers
Meet Maria Plecnik
Experts
Seriously: Value-Added is complicated
Data
● Know what you want to ask the data
● Get outside advice
● Bulletproof it
● Few people care about the details
This was our post explaining the analysis. Almost no one read it.
StateImpact Ohio and The Plain Dealer analyzed the relationships between a statistical measure called value-added and teachers’ pay, and between value-added and teachers’ ages using data obtained from the Ohio Department of Education.
Value-added looks at how much an individual teacher’s students learn in a given year, regardless of their level at the start of the year. Value-added is a key part of Ohio’s new teacher evaluation system, but is not an overall measure of teacher performance.The analysis included value-added scores and ratings for about 16,000 reading and math teachers in grades four through eight in more than 450 districts across the state.
StateImpact Ohio and The Plain Dealer consulted with Ohio University researchers Anirudh Ruhil and Marsha Lewis, who are leading a statewide study of value-added and related measures, on the analysis.
The pay analysis looked district-by-district at the correlation between teachers’ pay and value-added scores. It found significant correlations in just three districts across the state. None of those districts had more than 40 teachers rated.
That analysis did not take into account other factors that might influence student learning, such as poverty levels, or other factors related to teachers, such as advanced degrees. It also did not take into account the small numbers of teachers who received value-added scores in some districts or the fact that some teachers had only a single year of value-added data available. Ruhil and Lewis cautioned that additional analysis is necessary to more completely describe the relationship between teacher pay and value-added.
Scale matters
The Naming of the Names
● If you publish, know why
● Explain to readers/listeners
● Engage online/offline