washington state teacher and principal evaluation project combining multiple measures into a...
TRANSCRIPT
1
Washington State Teacher and Principal
Evaluation Project
Combining Multiple Measures Into a Summative Rating
Updated April 2014
2 2
Entry TaskAs you enter, please take a moment to place a sticky note under the appropriate place on the chart paper. Make sure to put your district name on the note, along with which instructional framework you are using.
Four options include:
1. We know the types of evidence that we will use for criterion scoring and how to derive a criterion score using that evidence.
2. We know the types of evidence that we will use for criterion scoring, but are still working out how to derive a criterion score using that evidence.
3. We know some of the types of evidence that we will use for criterion scoring.
4. We do not know the types of evidence that we will collect for criterion scoring.
3
Introductions Logistics Agenda
Agenda Connecting Learning I Learning II Implementing Reflecting Wrap-Up
Welcome!
Modules
Introduction to Educator Evaluation in Washington Using Instructional and Leadership Frameworks in
Educator Evaluation Preparing and Applying Formative Multiple Measures
of Performance: An Introduction to Self-Assessment, Goal Setting, and Criterion Scoring
Including Student Growth in Educator Evaluation Conducting High-Quality Observations and
Maximizing Rater Agreement Providing High-Quality Feedback for Continuous
Professional Growth and Development Combining Multiple Measures Into a Summative
Rating
4
5
Examine the most appropriate types of evidence needed to assign each criterion score.
Translate multiple forms of evidence to the eight criterion scores.
Understand how the instructional framework rubric and student growth rubrics are used within the summative scoring methodology for the focused and comprehensive evaluations.
Assign a summative score to EXAMPLE teachers or principals using the instructional AND leadership framework rubrics and student growth rubrics.
Overview of Intended Outcomes for Module
6
Guidance Icon Key
A capital “G!” indicates that the guidance represents Washington state law.
A lower-case “g” indicates that the guidance represents research-based best practice but is not mandated by law.
gG!RCW 28A.405.100
G!RCW 28A.405.100
7
Pausing Paraphrasing Posing Questions Putting Ideas on the Table Providing Data Paying Attention to Self and Others Presuming Positive Intentions
What Else?
Session Norms
8
Connecting
Builds community, prepares the team for learning, and links to prior knowledge, other modules, and
current work
9
Let’s look at the chart papers from the entry task
Where are most districts at in using multiple forms of evidence to create criterion scores? What is the distribution of responses? What is this distribution telling us?
Where Are We?
The Year-Long Evaluation Cycle
10
Standards G!
8 Criteria
Frameworks+
Student Growth Rubrics
Summative Rating
G!State- determined process
DistinguishedProficientBasicUnsatisfactory
Criterion Rating gDistrict-determined process
DistinguishedProficientBasicUnsatisfactory
• Observation
• Student Growth
• Evidence
Evidence g
Step 1:• Criteria aligned to
instructional /leadership and student growth rubrics
• Professional goals (g )
• Instructional/leadership goals (g )
• Student growth goals (G!)
Step 2 & 3: Select and collect
evidence• 2 observations
(G!)• Student growth
(G!)• Other evidence
(g )
Step 4: Determi
ne 8 Criteria
Scores (g )
Step 5: Summative Score
(G!)
11
Review the goal sheet for Tom Wilson On your handout (Handout 3), write down two
pieces of evidence that could be collected to show progress toward that goal.
Think about how the evidence aligns to the criteria.
From Plan to Actiong
12
Learning I: Transforming Evidence
Review the type of evidence needed to assign multiple
criterion scores
Review how to translate multiple forms of evidence to all
eight criterion scores
13
The primary goal of any system of teacher evaluation is to promote teacher and student learning.
Accurate teacher evaluation requires trained observers using a research-based instructional framework. Trained observers make accurate assessments of practice based on evidence.
The value of accurate assessments of practice is to shape the conversations that lead to improved practice.
Embedded in each instructional framework is a system for growth in teaching practice.
Reliability and validity of the instructional framework relies on implementation of the full framework rather than individual components/indicators.
It is imperative to remain in the formative mindset until the final summative rating is determined.
OSPI’s Guiding Principles for Criterion Scoring for Teacher Evaluation
g
14
Three sources of information
1. Observations based on your chosen instructional framework
2. Student growth data as measured by student growth rubrics
3. Other evidence relevant to the frameworks
Sources of Evidence for Summative Scoring
G!RCW 28A.405.100
g
15
The Evidence Cycle – Roles and Responsibilities
Step of the Evidence Cycle Teacher Role Evaluator Role
1. Collect X
2. Sort and align X X
3. Interpret and clarify X
4. Draw conclusions X
1. Collect
2. Sort and align
3. Interpret
and clarify
4. Draw conclusio
ns
The Evidence Cycle
G!RCW 28A.405.100
16
Identify pieces of evidence that the majority of educators will need to collect (e.g., lesson plan, parent communications)
Staff share examples of high-quality pieces of evidence Discuss how they provide evidence of a criterion
score OR how they can cut across multiple criteria
Think about how the pieces of evidence align to the OSPI Guiding Principles
Remember, it’s about collecting quality and a variety of evidence that you already use in your classroom
Strategies for Collecting Pieces of Evidence
g
17
Included are five pieces of evidence Evidence cover pages are missing
Alignment to criteria Evidence statements
Artifacts From Tom Wilson
Set Evidence Location
A A two-day lesson plan Handout 3
B Unit assessment dataTeam meeting minutes
Handouts 4 and 5
C Parent communication logE-mail exchange
Handouts 6 and 7
18
Within your school team, divide into pairs Each pair will do the following:
Review one set of evidence Complete the evidence cover page for the missing
components Consider these questions:
After reviewing these pieces of evidence, do you have enough evidence to make an accurate assessment of practice?
Do these pieces of evidence provide sufficient evidence to help shape a conversation that will improve practice?
How can these pieces of evidence be used for teacher growth in practice?
Identifying Evidence
19
Learning II: Summative Scoring
Understand how the instructional framework rubric and student growth rubrics are used within the
summative scoring methodology for the formative and comprehensive evaluations
Assign a summative score to teachers or principals using the instructional framework rubrics and student growth rubrics for the formative and
comprehensive evaluations
20
Things to remember: It is a process, not a final rating! Balance between professional judgment and
transparent rating process Uniformity and transparency in developing the
summative rating
Summative Performance Ratingg
21
Assesses all eight evaluation criteria All criteria contribute to the comprehensive
summative evaluation rating Student growth rubrics embedded in criteria
(3, 6, and 8) All provisional classroom teachers and
classroom teachers not on level 3 or level 4 receive comprehensive evaluation
All classroom teachers shall receive a comprehensive summative evaluation at least once every four years
Comprehensive Evaluation: Teachers
G!RCW 28A.405.100
22
Assesses all eight evaluation criteria All criteria contribute to the comprehensive
summative evaluation rating Student growth rubrics embedded in criteria
(3, 5, and 8) “Due to the importance of instructional leadership
and assuring rater agreement among evaluators, particularly those evaluating teacher performance, school districts are encouraged to conduct comprehensive summative evaluations of principal performance on an annual basis.”
— Section 1, (12 c(v))
Comprehensive Evaluation: Principals
G!RCW 28A.405.100
23
Three Steps to Assign Comprehensive Evaluation System Step 1: Assign Preliminary Summative Score Step 2: Determine Impact on Student Learning Step 3: Use the Summative Rating and Impact on
Student Learning Matrix to determine summative score
Comprehensive Evaluation
24
Step 1: Assign Preliminary Summative Score 1a: Transfer criterion scores to summative scoring
sheet 1b: Add the eight criterion scores to create a sum 1c: Compare the sum score to the scoring band 1d: Assign a preliminary summative score
Step 1: Comprehensive Evaluation
25
The RAW Score Model: Preliminary Summative ScoreTeaching Criteria* Indicates criterion embedded with student growth rubrics
Overall Criterion Score
Criterion 1: Centering instruction on high expectations for student achievement 3Criterion 2: Demonstrating effective teaching practices 4*Criterion 3: Recognizing individual student learning needs and developing strategies to address those needs 3Criterion 4: Providing clear and intentional focus on subject matter content and curriculum 2Criterion 5: Fostering and managing a safe, positive learning environment 3*Criterion 6: Using multiple student data elements to modify instruction and improve student learning 2Criterion 7: Communicating and collaborating with parents and school community 3*Criterion 8: Exhibiting collaborative and collegial practices focused on improving instructional practice and student learning 2
Total Summative Score 22
Evaluators place teachers into preliminary summative rating categories based on score bands. As shown, this teacher would receive a preliminary overall summative rating of proficient.
OSPI-Approved Summative Scoring Band8–14 15–21 22–28 29–32
1Unsatisfactory
2Basic
3Proficient
4Distinguishe
d25
26
Step 2: Determine Impact on Student Learning 2a: Transfer student growth rubric scores to
student growth summative scoring sheet 2b: Add the five student growth rubric scores 2c: Compare the sum to the student growth
scoring band 2d: Assign impact on student learning score
Step 2: Comprehensive Evaluation
27
Student Growth
Goal-Setting Score Based
on Rubric
Student Growth*
Score Based on Rubric
Overall Student Growth
Criterion Score
Criterion 3 3 2** 5Criterion 6 2 2** 4Criterion 8 2 N/A 2Student Growth Score 7 4 11
Student Growth Rubric and Rating(Teachers Only)
*Must include a minimum of two student growth measures (i.e., state-, district-, school-, and classroom-based measures).
** A student growth score of “1” in any of the student growth rubrics will result in a low growth rating.
OSPI-Approved Student Growth Impact Rating Scoring Band
5-12 13-17 18-20Low Average High
27
Evaluators place teachers into summative rating categories based on score bands. As shown here, this teacher would receive a low student growth rating.
28
Summative Rating & Impact on Student Learning Matrix
Summative Rating
DistinguishedProficient
RatingStudent Growth Inquiry
Distinguished Rating
ProficientProficient
RatingStudent Growth Inquiry
Proficient Rating
Basic Basic RatingStudent Growth Inquiry
Basic Rating
Unsatisfactory Unsatisfactory RatingPlan of improvement
Consequences as a result of intersection between summative rating and impact on student learning rating
Low Average High
Impact on student learning
28
29
Step 3: Use the Summative Rating and Impact on Student Learning Matrix to determine summative score Educators with preliminary rating of distinguished
with average or high student growth rating: These educators will receive an overall distinguished rating and will be formally recognized and/or rewarded (per regulations).
Educators with preliminary rating of unsatisfactory and high student growth rating: These evaluations will be reviewed by the evaluator’s supervisor when an educator is rated unsatisfactory and receives a high student growth rating. The supervisor will take these discrepancies into account in the evaluator’s evaluation.
Educators who receive a score of 1 on the achievement of student growth goals will automatically receive a low student growth rating.
Step 3: Comprehensive EvaluationG!
RCW 28A.405.100
30
Step 1: Determine framework dimension/component score Use evidence collection alignment form (instructional
framework-specific; Handout 14a, b or c) Step 2: Determine criterion score
Use criterion scoring sheet (instructional framework-specific; Handout 12, and Handout 13a, b, or c)
Step 3: Determine preliminary summative score Use summative scoring sheet (Handout 8)
Step 4: Determine student growth impact rating Use student growth rubric and rating form (Handout 9)
Step 5: Determine comprehensive evaluation score Use summative rating and impact on student learning matrix
(Handout 10)
Learning Activity: Putting the Pieces Together
31
Find an individual from another district using the same framework
Discuss three main questions Did you come up with the same summative
score? Why or why not?
How do/how will the five steps fit in with your district's current process? Will you have to make big changes or adjust a few
things? How can/will the summative scoring process be
used as a way to promote professional growth?
Putting the Pieces Together Debrief
32
Includes an assessment of one of the eight criteria
Student growth rubrics from one of the three criteria If a teacher chooses criterion 3, 6, or 8, their
accompanying student growth rubrics will be used.
If a teacher chooses criterion 1, 2, 4, 5, or 7, the accompanying student growth rubrics from criterion 3 or 6 will be used.
Approved by the teacher’s evaluator
A focused evaluation must be performed in any year that a comprehensive evaluation is not scheduled
Focused EvaluationCertificated Classroom Teachers
G!RCW 28A.405.100
33
Includes an assessment of one of the eight criteria
Student growth rubrics from one of the three criteria The focused evaluation will include the student
growth rubric row selected by the principal or assistant principal.
Criterion and student growth rubric rows must be approved by the principal’s evaluator
A focused evaluation must be performed in any year that a comprehensive evaluation is not scheduled
Focused EvaluationPrincipals and Assistant Principals
G!RCW 28A.405.100
34
Similar process to criterion scoring
Criterion 3: MarzanoFormative evaluation score based on two Marzano components aligned to criterion 3 and two student growth rubric scores aligned with criterion 3
Focused Evaluation Example: Selected Criterion Includes Student Growth Score (Criterion 3, 6, 8)
Indicator Unsatisfactory 1
Basic2
Proficient3
Distinguished4 Summative Score
3.1 Effective scaffolding of information
X
2
3.2 Planning and preparing for needs of all students
X
Criterion 3: Goal-setting student rubric score X
Criterion 3: Student growth rubric score X
35
To create formative score, the following are needed: Component scores for the criterion (similar to criterion score) Growth rubric scores from criterion 3 or 6 (from teachers); from criterion 3 or 5
(from principals)
Criterion 4: MarzanoFormative evaluation score based on two Marzano components aligned to criterion 4 and two student growth rubric scores aligned with criterion 6
Focused Evaluation Example: Selected Criterion Does Not Include Student Growth Score (Criterion 1, 2, 4, 5,
7)
Indicator Unsatisfactory 1
Basic2
Proficient3
Distinguished4 Summative Score
4.1 Attention to est. content standards
X
? (District
determined process
based on the evidence)
4.2 Use available resources and technology
X
SG 6.1: Establish Student growth goal(s)
X
SG 6.2: Achievement of Student growth goal(s)
X
36
Split into pairs. You will reteach each other the process of the focused evaluation process. When you reteach, provide an example from the
teacher or principal framework your district uses.
Partner 1: Reteach the focused evaluation process when educator selects a criterion that includes student growth.
Partner 2: Reteach the focused evaluation process when educator selects a criterion that does NOT include student growth.
Learning Activity IIB: Reteach
37
Implementing
Supports teams in problem solving and planning next steps for schools and districts
38
Create a plan for implementing the data collection process and how that will be used for summative evaluation.
Use Handout 11 (Implementation Planning – Three Steps) from your packet to structure your conversation with your school team.
This packet is similar what you did at the end of the Preparing and Applying Formative Multiple Measures of Performance module If you have completed part of it, go back through
your decisions and refine them based on this module, and any other modules you have completed since then.
Identifying Tools and Processes for Gathering and Organizing Evidence
39
Each team shares one decision that was made today to increase the clarity and feasibility of the teacher evaluation process.
Implementing Activities Debrief
40
Reflecting
Engages participants in providing feedback, reflecting on learning, and closing the session
41
Plant your hand on a piece of blank paper and trace it.
On each finger, write the five most important facts to remember and teach others about combining multiple measures into a summative rating.
Share at your tabletops and be prepared to share one with the large group.
Debrief: Hand Plant
42
Homework Options District: Continue to work on a district teacher
evaluation guidebook that includes all of the nuts and bolts of the teacher evaluation process. Use the information you recorded on the “Implementation Planning” handout as a starting place.
School or Teams: Identify the processes and procedures at your school for how the evidence teachers collect will be organized and stored for effective implementation of the summative scoring process.
What’s Next?
43
Thank you!
Presenter’s Name
XXX-XXX-XXXX
1234 Street Address
City, State 12345-1234
800-123-1234