nyc teacher data initiative: an introduction for principals eso focus on professional development...
TRANSCRIPT
NYC Teacher Data Initiative: An introduction for Principals
ESO Focus on Professional DevelopmentOctober 2008
2
Session Objectives for Principals
Become familiar with the new Teacher Data Reports
Consider ways to incorporate this new tool into school-wide professional development
Plan for sharing Teacher Data Reports with teachers
Locate support resources
Take a closer look at two teachers
Read the scenarios
How effective is Ms. Brooks as a teacher? How effective is Mr. Capstone?
What differences, if any, will there be with the students’ ELA scores in January?
3
4
We use an array of instruments to determine teacher and school-wide professional development needs
Classroom observations
Lesson plans
Participation in professional development
Quality of student work products
Student performance on state assessments
No one measure gives us the full story, but the various pieces come together to create a more reliable picture
Areas of convergence and dissonance in our observations are equally useful
5
Teacher Data Initiative (TDI) provides a new lens that will soon be available to schools
Purpose: To contribute another lens through which to look at teacher contributions to student learning
Rationale: Teachers make a big difference, and value-added data provides a lens to focus on what teachers bring to students rather than what students bring to the classroom
Framing Question: How might the TDI data tool fit into existing school plans for instructional improvement and professional development?
TDI should not be viewed as a silver bullet, big initiative, or accountability metric that will be forced upon schools. Rather it is a new tool available to principals to incorporate into their larger instructional and professional development plans.
6
TVI draws on 10 years of city-wide data (test scores, student, teacher, and school characteristics) to predict individual student gains
The predicted gains are compared to the actual gains for each student to determine the teacher’s contributions or the “value added”
The teacher’s contribution for each student is averaged, and then compared to other 4-8 ELA and Math teachers or rank ordered top 20%, middle 60%, and bottom 30%.
How Teacher Data Works
Predicted Score
Mathematically isolates factors beyond teacher control e.g. prior year
test scores
Predicted Score
Mathematically isolates factors beyond teacher control e.g. prior year
test scores
Teacher Contribution
Factors within teacher control e.g. quality of
instruction & high expectations
Teacher Contribution
Factors within teacher control e.g. quality of
instruction & high expectations
Actual Test Score
Student scores on ELA & Math tests
Actual Test Score
Student scores on ELA & Math tests
7
How TDI Works: The Model
Value added for one student
Pro
fici
ency
rat
ing
3rd Grade 4th Grade
3 -
-
2-
PredictedPredicted
Gain
Actual Value Added
Baseline (Previous Year’s) Score
Teacher A
Teacher B
Teacher E
Teacher D
Teacher C
Least Gain
Most Gain
• The “value added is the difference between the predicted and actual scores
• Value added is averaged for all students in a class
• The value added is measured in proficiencies
• TDI orders teacher from least to most gain to determine a percentile rank
8
TDI mathematically factors in measurable characteristics to predict student scores
Student characteristics Classroom characteristics School characteristics
Prior year reading
Prior year math
Free or reduced price lunch
Special education status
English Language Learner status
Number of suspensions and absences (prior-year)
Student retained in grade
Attended summer school
New to school
Race
Gender
Prior year teacher
Average prior year reading and math
Percent free or reduced price lunch
Percent special education status
Percent English Language Learner status
Average number of suspensions and absences (prior)
Percent of students retained in grade
Percent attended summer school
Class size
Percent by race
Percent by gender
Average classroom characteristics
Average class size
Total tested by grade/subject
Year starting and ending school
Teacher characteristics
(used when comparing teachers to peer teachers)
Years of experience
Years teaching in the same grade and subject
Principals will receive a summary report and analysis by student subgroups
10
Individual teacher reports will also be available for principal use and to share with teachers
Similar to the Progress Reports, TDI compares teachers to the following groups:
1. All teachers on the same grade level, City-wide
2. Peer teachers (similarly situated in terms of teacher experience, and student, school, and classroom characteristics)
0% 25% 50% 75%
2007-08 66% 95%
Last 3 years 69% 92%
100%My percentile (0%-100%)
79%
Range*
81%
My percentile
0% 25% 50% 75%
2007-08 55% 85%
Last 3 years 58% 78%
100%My percentile (0%-100%)
68%
Range*
70%
My percentile
Analyze sample reports to look for trends and consider key questions
Key Questions
What is being taught?
How is it taught?
Are the students learning?
How are teachers learning?
How are resources invested?
Potential Trends
Clumps of teachers scoring low with a particular subgroup
Individual teachers consistently low/high across many groups
Sizeable difference between math and ELA
Similar scores among all teachers on a team or in a grade
Example: Think through a specific trend
School summary report reveals much higher scores on math than ELA
What is being taught?
Is our math curriculum stronger than ELA?
Consult with schools using similar curriculum
How are resources invested?
Are more push-in resources allocated for math?
Teacher report reveals high scores on everything except for ELL students
What is being taught?
Might ELL students require additional instruction
Analyze test items for trends in ELL responses
How is it taught?
Is teacher differentiating instruction?
Analyze quantity and quality of math PD
Request peer observations
Consult with others who have high ELL score
Does this teacher receive adequate ELL Support?
Pair this teacher with ELL coach
13
Principals can use TDI in numerous ways, but with some cautions
Potential Uses
> Look for strengths, areas for development, surprises and wonderings
> Emphasize instructional improvement
> Triangulate with other insight Consider factors you know about the teachers
or the classrooms that may not be measurable
> Consider professional development approaches for individual teachers or groups
> Help teachers connect these results with insights from their periodic assessments, student work, and item analysis
> Consider implications for classroom assignment
> Consider implications for curriculum or instructional programs
> Consider implications for staffing needs
> Prioritizing principal observation and coaching
> Inform principal and teacher goal setting
Cautions
> DO NOT Use TDI for teacher evaluation
> Avoid replacing principal judgment and other forms of information
> Not all negative value-added results are bad and all positive results are good
Use the performance ranges to see how strong a positive result is or how weak a negative result is.
Be aware of small sample sizes or few years of data
> Remember to consider context that is not easily measured and not in the model for example:
Push-in/pull-out teachers
AIS services
Life events for teachers, students
School context
> Secure the teacher’s permission before sharing the report with coaches and others
> Consider individual teacher information confidential and thus not sharable available to parents
14
Next Steps to Consider
How will use Teacher Data Reports to improve instruction?
How will you involve others within the school?
How can you introduce TDI to your staff?
How can you share individual reports with teachers?
What additional supports might you need?