measuring up: effective strategies for teacher evaluation patricia a. popp, ph.d. state coordinator,...
TRANSCRIPT
Measuring Up: Effective Strategies for Teacher
Evaluation
Patricia A. Popp, Ph.D.State Coordinator, Project HOPE-VA
Clinical Associate [email protected]
Xianxuan Xu, Ph.D.Post-Doctoral Research Associate
School of EducationThe College of William and Mary
Teacher Evaluation in an Era of Accountability
• Currently, ESEA flexibility has been granted to 34 states and the District of Columbia. As part of the flexibility requirements, the states were required to establish new teacher evaluation systems that factor in student achievement progress for statewide implementation by the end of the 2013-2014 school year.
• All states that received Race to the Top funding are undertaking substantial reforms with teacher evaluation.
• “The quality of an education system cannot exceed the quality of its teachers” (Barber & Mourshed, 2008).
Share of At-Risk Students on PISA(Not reaching PISA baselines): Reading
Canada Finland Shanghai Singapore South Korea
United States
0
5
10
15
20
25
11
8
5
108
21
Share of At-Risk Students on PISA(Not reaching PISA baselines): Math
Canad
a
Finl
and
Shan
ghai
Sing
apor
e
Sout
h Ko
rea
Unite
d St
ates
0
5
10
15
20
25
118
5
108
21
Percentage of Country’s Students in PISA Top Performing Groups:
Reading
0
4
8
12
16
20
1315
19
16
13
10
Percentage of Country’s Students in PISA Top Performing Groups:
Math
Canad
a
Finl
and
Shan
ghai
Sing
apre
Sout
h Ko
rea
Unite
d St
ates
0
10
20
30
40
50
60
18 21
50
36
26
10
What do we do about it?
How Long Do Students Attend School? Days in the Academic Year
Country Days in an Academic Year
Canada Average: 188Finland 187
Singapore 200Shanghai 180
South Korea 204United States 180
How Long Do Students Attend Schools?Minutes in the Day
Location Minutes in a School Day
Canada 304Finland 240
Shanghai 390Singapore 330
South Korea 264United States 402
How Much Do We Spend?
Location Annual Expenditures Per Pupil
Canada 8,045
Finland 7,216
Shanghai N/A
Singapore N/A
South Korea
6,663
United States
10,259
Student/Teacher Ratio
CountryAverage
Student/Teacher Ratio
Canada 25
Finland 19
Shanghai 39
Singapore 35
South Korea 36
United States 24Source: Available at worldbank.org.
Which factor is a strong predictor of student achievement gains?
Sources: Wright, Horn, & Sanders, 1997;
Hanushek, Kain, & Rivkin, 1998,
Class size
Classroom heterogeneity
School resource
differences
It’s the teacher.
Influences on Student Achievement:Explained Variance
Students50%
Peers5-10%
School5-10%
Home5-10%
Teachers30%
Source: Hattie, J. Teachers make a difference: What is the research evidence.Retrieved 20Nov08 from http://acer.edu.au/documents
Dallas Research: Teacher Quality
1st
gra
de
4th
gra
de
0
20
40
60
80
1004th gr. Math Achievement
Highly Effective
Ineffective
Dallas, Texas data: 2800-3200 students per cohort
Comparison of 3 “highly effective” & 3 “ineffective” teachers (Jordan, Mendro, & Weerasinghe, 1997)
Dallas Research: Teacher Quality
1st
G..
.
4th
...
0
10
20
30
40
50
60
70
4th gr. Reading Achievement
Highly Effective
Ineffective
Time in School Year Needed to Achieve the Same Amount of Learning
Leigh, Economics of Education Review (2010)
0 1/4 1/2 3/4 1
25th PercentileTeacher
75th PercentileTeacher
Years Needed
Time in School Year Needed to Achieve the Same Amount of Learning
0 1/ 4 1/ 2 3/ 4 1
10th PercentileTeacher
90th PercentileTeacher
Years Needed
Source: Leigh, A. (n.d.). Estimating teacher effectiveness from two-year changes in students’ test scores. Retrieved May 22, 2007, from http://econrsss.anu.edu.au/~aleigh/.
Time in School Needed to Achieve the Same Amount of Learning
Sequence of Effective Teachers
Low
High
52-54percentile
points difference
Low Low
High High
Sanders & Rivers (1996)
Sequence of Effective Teachers
Low
High
13percentile
points difference
Low
High High
Sanders & Rivers (1996)
High
Revised Teacher Evaluation System
in Virginia: An Overview
Primary Purposes of the Evaluation System
• Improve student achievement through the quality of instruction by assuring accountability for classroom performance
• Contribute to the successful achievement of the goals and objectives defined in a school division’s educational plans
• Provide a basis for instructional improvement through productive teacher appraisal and professional growth
• Share responsibility for evaluation between the teacher and the evaluation team in a collaborative process that promotes self-growth, instructional effectiveness, and improvement of overall job performance
A Flawed System
Problem No. 1: Observation equals evaluation
Problem No. 2: Likely to rely on intuition, not evidence, to make judgments about teacher performance
Problem No. 3: One size fits all
Problem No. 4: Don’t communicate
Problem No. 5: Fragmented evaluation process
Problem No. 6: Irrelevant evaluation
Problem No. 7: One-point rating scales
Problem No. 8: No impact evaluation
Question 1
What is the basis of the teachers’ evaluation?
Standard 2: Instructional PlanningThe teacher plans using the Virginia Standards of Learning, the school’s curriculum, effective strategies, resources, and data to meet the learning needs of all students.
Main Components
Sample Performance IndicatorsExamples may include, but are not limited to:
The teacher: 2.1 Uses student learning data to guide planning.2.2 Plans time realistically for pacing, content mastery, and transitions.2.3 Plans for differentiated instruction.2.4 Aligns lesson objectives to the school’s curriculum and student learning needs.2.5 Develops appropriate long- and short-range plans, and adapts plans when needed.
ExemplaryProficient
Proficient is the expected level of performance.
Developing/Needs Improvement
Unacceptable
In addition to meeting the standard, the teacher actively seeks and uses alternative data and resources and consistently differentiates plans to meet the needs of all students.
The teacher plans using the Virginia Standards of Learning, the school’s curriculum, effective strategies, resources, and data to meet the needs of all students.
The teacher inconsistently uses the school’s curriculum, effective strategies, resources, and data in planning to meet the needs of all students.
The teacher does not plan, or plans without adequately using the school’s curriculum, effective strategies, resources, and data.
Performance Appraisal
Rubric
Performance Standard
Performance Indicators
Performance Standards
Professional
Knowledge
Instructional
Planning
Instructional
Delivery
Assessment of
and for Student Learnin
gLearning
Environment
ProfessionalismStudent
Academic
Progress
The teacher demonstrates an understanding of the curriculum, subject content, and the developmental needs of students by providing relevant learning experiences.
Teacher Performance Standard 1: Professional Knowledge
Teacher Performance Standard 2: Instructional Planning
The teacher plans using the Virginia Standards of Learning, the school’s curriculum, effective strategies, resources, and data to meet the needs of all students.
Teacher Performance Standard 3: Instructional Delivery
The teacher effectively engages students in learning by using a variety of instructional strategies in order to meet individual learning needs.
Teacher Performance Standard 4: Assessment of and for Student Learning
The teacher systematically gathers, analyzes, and uses all relevant data to measure student academic progress, guide instructional content and delivery methods, and provide timely feedback to both students and parents throughout the school year.
Teacher Performance Standard 5: Learning Environment
The teacher uses resources, routines, and procedures to provide a respectful, positive, safe, student-centered environment that is conducive to learning.
Teacher Performance Standard 6: Professionalism
The teacher maintains a commitment to professional ethics, communicates effectively, and takes responsibility for and participates in professional growth that results in enhanced student learning.
Teacher Performance Standard 7: Student Academic Progress
The work of the teacher results in acceptable, measurable, and appropriate student academic progress.
Question 2
How will teacher performance be documented?
Multiple Data Sources
Data Source Evaluator Teacher
Informal Observations
Formal Observations
Student Surveys
Portfolios/Document Logs
Self-Evaluation Measures of Academic Progress
Reviews/approves Selects/develops
Measures of Academic Progress
Teachers
Percentage of Evaluation based on
Student Growth Percentiles (SGPs)
Percentage of Evaluation based on
Other Growth Measures
Teachers of reading and mathematics for whom SGPs are available
20 20
Teachers who support instruction in reading and mathematics for whom SGPs are available
No more than 20 20 to 40
Teachers who have no direct or indirect role in teaching reading or mathematics in grades where SGPs are available
N/A 40
Student Achievement Goal Setting
Step 1:Determine
needs
Step 2:Create specific learning
goals based on pre-
assessment
Step 3: Create and implement
teaching and learning
strategies
Step 4: Monitor student progress through ongoing
formative assessment
Step 5:Determine
whether the students
achieved the goal
What are the Purposes ofStudent Achievement Goal Setting?
Focus on student results
Explicitly connect teaching and learning
Improve instructional practices and teacher performance
Tool for school improvement
Question 3
How will teacher performance be
rated?
Evaluations
Interim Evaluation • Used to document evidence of meeting standards• Does NOT include rating of performance
Summative Evaluation• Comes at end of evaluation cycle
- One year for probationary teachers
- Three years for continuing contract teachers
• Assessment of performance quality
- Four point rating scale
- Performance rubric for every standard
Evaluating Performance
Exemplary The teacher maintains performance, accomplishments, and behaviors that consistently and considerably surpass the established standard.
Exceptional Performance• Sustains high performance over period of time• Behaviors have strong positive impact on
learners and school climate• Serves as role model to others
Category Description Definition
Proficient The teacher meets the standard in a manner that is consistent with the school’s mission and goals.
Effective Performance• Meets the requirements contained in job
description as expressed in evaluation criteria• Behaviors have positive impact on learners and
school climate• Willing to learn and apply new skills
Developing/Needs Improvement
The teacher often performs below the established standard or in a manner that is inconsistent with the school’s missions and goals.
Below Acceptable Performance• Requires support in meeting the standards• Results in less than quality work performance• Leads to areas for teacher improvement being
jointly identified and planned between teacher and evaluator
Unacceptable The teacher consistently performs below the established standards or in a manner that is inconsistent with the school’s missions and goals.
Ineffective Performance• Does not meet requirements contained in job
description as expressed in evaluation criteria• Results in minimal student learning• May result in employee not being
recommended for continued employment
Sample Performance Appraisal Rubric
ExemplaryProficient
Proficient is the expected level of performance.
Developing/Needs Improvement Unacceptable
In addition to meeting the standard, the teacher consistently demonstrates extensive knowledge of the subject matter and continually enriches the curriculum.
The teacher demonstrates an understanding of the curriculum, subject content, and the developmental needs of students by providing relevant learning experiences.
The teacher inconsistently demonstrates understanding of the curriculum, content, and student development or lacks fluidity in using the knowledge in practice.
The teacher bases instruction on material that is inaccurate or out-of-date and/or inadequately addresses the developmental needs of students.
Standard I: Professional Knowledge
The teacher demonstrates an understanding of the curriculum, subject content, and the developmental needs of students by providing relevant learning experiences.
Summative ratings should apply the rating for each of the seven performance expectations, with the most significant weight given to Standard 7 - Student Academic Progress.
Summative Rating
• Weight each of the first six standards equally at 10 percent each
• Weight Standard 7 – Student Academic Progress at 40 percent
Lessons Learned
Most practitioners believe new models provide specific measures of teacher effectiveness that are useful for distinguishing effective from less effective teachers.
ExemplaryProficient
Proficient is the expected level of performance.
Developing/Needs Improvement
Unacceptable
In addition to meeting the standard, the teacher actively seeks and uses alternative data and resources and consistently differentiates plans to meet the needs of all students.
The teacher plans using the state’s standards, the school’s curriculum, effective strategies, resources, and data to meet the needs of all students.
The teacher inconsistently uses the school’s curriculum, effective strategies, resources, and data in planning to meet the needs of all students.
The teacher does not plan, or plans without adequately using the school’s curriculum, effective strategies, resources, and data.
Most practitioners believe that new models have the potential to improve teaching and learning by providing useful feedback that can be used to diagnose and guide teacher improvement.
Formal Observation Post-Conference
Student Learning Objectives
Teacher Self-Assessment
Student Surveys
Lessons Learned
Lessons Learned
With more rigorous assessment to identify problems and recognize excellence, investments in teacher development can be better related to school and division goals for improvement.
Evaluation
Strengths and
Weaknesses Identified
Targeted Professional Developme
nt
Lessons Learned
Practitioners appreciate the value in using multiple data sources to provide evidence of performance standards.
Teacher Performance
Standards
Observations
Documentation
Logs
Student Learning
Objectives
Student Surveys
Lessons Learned
Practitioners believe that new models set up realistic expectations for teacher performance, and they reflect the most important elements of effective teaching.
Professional Knowledge
Instructional Planning
Assessment of/forLearning
Instructional DeliveryLearning
Environment
Professionalism
Student Progress
Lessons Learned
The evaluation framework is valid in terms that the process standard ratings of teacher have a moderate ability to predict the student academic progress. In addition, there is a significant correlation between each of the six process standards and student academic progress.
Lessons Learned
New models can be time-consuming to implement.
Practitioners mistrust the validity of student progress models as a measure of student growth.
Teachers believe that the nature, quality, and credibility of the evaluation process vary depending on the qualifications of the evaluators.
Concluding Thoughts on Transforming Teacher Evaluation
State
• Convene stakeholders across the state to design, implement, and improve evaluation system.
• Develop validated and reliable evaluation measures.
• Provide incentives and on-going support to school divisions.
• Make knowledge of new developments in teacher evaluation part of leadership and teacher preparation programs.
• Make evaluation count.
Concluding Thoughts on Transforming Teacher Evaluation (Continued)
Schools and School Divisions/Districts
• Clear expectations of the WHAT will be evaluated.
• Clear communication of the evaluative criteria for effective performance.
• Instruments and procedures for teachers to provide evidence from multiple sources of the HOW of their performance.
• Increase the use of evaluation results to inform professional development.