connecting teaching practices to student data documents/dr robert...marzano causal teacher...

4/19/2013

1

Marzano Causal Teacher

Evaluation Model

Systematic, Measurable Development of

Every Teacher

Beth Carr, Director of District Partnerships

Goals:

2

1. Review the Marzano Causal Teacher Evaluation

Model

2. Review the Marzano Leader Evaluation Model

3. Answer Questions

4. Review iObservation Features

5. Review the Marzano Model for determining the

instructional practice score

6. Other

Founded by Dr. Robert Marzano and Learning Sciences

International to:

o Conduct research and develop the next generation

of tools and supports

o Advance the field of teacher and leadership

evaluation

o Support districts throughout their evaluation model

adoption and implementation phases including

professional development and next generation tools

o Provide resources and assistance

About the Learning Sciences

Marzano Center

4/19/2013

2

What should be the focus of your teacher evaluation

system?

1. Purely to develop teachers

2. Emphasizes development but also measures

3. Equal emphasis measurement and development

4. Emphasizes measurement but also develops

5. Purely to measure teachers

Development or Measurement?

Development focus requires:

• A comprehensive framework that reflects the complexity

of teaching (teaching is cognitively complex – the

number of decisions that need to be made for each

student)

• Specificity in the framework for diagnostics and

actionable feedback

• Validation to actually improve teaching as measured in

student learning gains at both the element and

framework level

4/19/2013

3

The Goal: An expectation that all teachers can

increase their expertise from year to year which

produces gains in student achievement from

year to year with a powerful cumulative effect

© 2010 Learning Sciences International

877.411.7114 www.iObservation.com

ES=0

Typical bell curve of student results from

teachers using a research-based strategy

Decreased

Student

Achievement

Increased

Student

Achievement

4/19/2013

4

Typical Bell Curve of Student Results from

Teachers using a Research-Based Strategy

“The simple presence or absence

of an instructional strategy does

Not define effectiveness, but it

Is rather the teacher’s expertise

In adapting that strategy to the

Classroom within the context of

Lesson segments that produces

Gains in student achievement.”

— Dr. Robert Marzano

11 © 2010 Learning Sciences International


Marzano Teacher Evaluation Model

When these strategies are used, here is the typical

effect on raising student achievement

(percentile gain corrected):

Building Vocabulary 20%

Effort and Recognition 14%

Graphic Organizers 13%

Homework 15%

Identifying Similarities and Differences

20%

Interactive Games 20%

Nonlinguistic Representations 17%

Note Taking 17%

Practice 14%

Setting Goals/Objectives 25%

Student Discussion/Chunking 17%

Summarizing 19%

Tracking Student Progress and Using Scoring Scales

34%

The Research Base:

Meta-analysis

1,036 classroom studies, 38 schools

On average, when teachers used the

classroom strategies and behaviors in the

Marzano Causal Teacher Evaluation Model,

typical student achievement increased by 16

percentile points.

4/19/2013

5

A Correlational Study:

What Works in Oklahoma Schools

Correlations were associated with a 31 percentile

point increase in student achievement.

Apache Elementary Study

Correlations between implementation of the

Marzano Model and percentage of students

scoring proficient or advanced on state tests was

.43 for reading and .71 for mathematics.

The more strategies teachers used and the better

they executed them, the greater their students’

achievement in terms of both status and growth.

Adams 50 Study

4/19/2013

6

3rd Party Research

Dr. Paul Mielke, 2012: Qualitative comparative case study

of teacher evaluation models:

o Purpose of study: to discover impact of a potential teacher

supervision and evaluation process focusing

on developing self-directed teachers

o Marzano Model was found to have greater emphasis

on current research and alignment to the new demands of

Common Core

o Marzano Model found to have greater emphasis

on classroom strategies and behaviors

o Marzano Model teachers more likely to identify specific

elements of improvement, compared to the general insights from

the other model P.G. Mielke, 2012, “Investigating a Systematic Process to Develop Teacher Expertise:

A Comparative Case Study”

Overview of the Marzano Causal

Teacher Evaluation Model and the

Three Continuous Improvement

Cycles

18

Marzano Teacher

Evaluation Model

18

STUDENT

ACHIEVEMENT

Domain 1: Classroom Strategies and Behaviors (41 Elements) Routine Segments (5 Elements) Content Segments (18 Elements) On the Spot Segments (18 Elements)

Domain 2: Planning and Preparing (8 Elements) Lesson and Units (3 Elements) Use of Materials and Technology (2 Elements) Special Needs of Students (3 Elements)

Domain 3: Reflecting on Teaching (5 Elements) Evaluating Personal Performance (3 Elements) Professional Growth Plan (2 Elements)

Domain 4:

Collegiality and Professionalism (6

Elements)

Promoting a Positive

Environment (2 Elements)

Promoting Exchange of Ideas

(2 Elements)

Promoting District and School

Development (2 Elements)

Domain 4:


Elements)




(2 Elements)



4/19/2013

7

19

Marzano Teacher

Evaluation Model

19

STUDENT

ACHIEVEMENT

Domain 1: Classroom Strategies and Behaviors (41 Elements) Routine Segments (5 Elements) Content Segments (18 Elements) On the Spot Segments (18 Elements)

Domain 2: Planning and Preparing (8 Elements) Lesson and Units (3 Elements) Use of Materials and Technology (2 Elements) Special Needs of Students (3 Elements)

Domain 3: Reflecting on Teaching (5 Elements) Evaluating Personal Performance (3 Elements) Professional Growth Plan (2 Elements)

Domain 4:


Elements)




(2 Elements)



Domain 4:


Elements)




(2 Elements)



The Marzano Domain 1 organizes 9 of 10 Design

Questions into Three Lesson Segments:

Lesson Segment Involving Routine Events

Design Question 1: What will I do to establish and communicate learning

goals, track student progress, and celebrate success?

Design Question 6: What will I do to establish or maintain classroom

rules and procedures?

Lesson Segment Addressing Content

Design Question 2: What will I do to help students effectively interact

with the new knowledge?

Design Question 3: What will I do to help students practice and deepen

their understanding of new knowledge?

Design Question 4: What will I do to help students generate and test

hypotheses about new knowledge?

Lesson Segment Enacted on the Spot

Design Question 5: What will I do to engage students?

Design Question 7: What will I do to recognize and acknowledge

adherence and lack of adherence to classroom rules and procedures?

Design Question 8: What will I do to establish and maintain effective

relationships with students?

Design Question 9: What will I do to communicate high expectations for

all students?

4/19/2013

8

Domain 1: Classroom Strategies and Behaviors Lesson Segment

Involving Routine Events

Learning Goals & Feedback • Providing Clear Learning Goals and Scales to Measure those Goals

• Tracking Student Progress

• Celebrating Student Success

Rules & Procedures • Establishing Classroom Routines

• Organizing Physical Layout of the Classroom for Learning

Lesson Segments Addressing Content

Interacting With New Knowledge • Identifying Critical Information • Organizing Students to Interact with New Knowledge • Previewing New Content • Chunking Content into “Digestible Bites” • Processing of New Information • Elaborating on New Information • Recording and Representing Knowledge • Reflecting on Learning

Practicing & Deepening Knowledge • Reviewing Content • Organizing Students to Practice and Deepen Knowledge • Using Homework • Examining Similarities and Difference • Examining Errors in Reasoning • Practicing Skills, Strategies, and Processes • Revising Knowledge

Generating & Testing Hypothesis • Organizing Students for Cognitively Complex Tasks • Engaging Students in Cognitively Complex Tasks Involving Hypothesis Generation and Testing • Providing Resources and Guidance

Lesson Segments Enacted on the Spot

Student Engagement • Noticing When Students are Not Engaged • Using Academic Games • Managing Response Rates • Using Physical Movement • Maintaining a Lively Pace • Demonstrating Intensity and Enthusiasm • Using Friendly Controversy • Providing Opportunities for Students to Talk about Themselves • Presenting Unusual or Intriguing Information

Adherence to Rules & Procedures • Demonstrating “Withitness” • Applying Consequences for Lack of Adherence to Rules and Procedures • Acknowledging Adherence to Rules and Procedures

Teacher/Student Relationships • Understanding Students’ Interests and Backgrounds • Using Verbal and Nonverbal Behaviors that Indicate Affection for Students • Displaying Objectivity and Control

High Expectations • Demonstrating Value and Respect for Low Expectancy Students • Asking Questions of Low Expectancy Students • Probing Incorrect Answers with Low Expectancy Students

Domain 1: Classroom Strategies and Behaviors (Common Core Alignment)









Practicing & Deepening Knowledge

• Reviewing Content • Organizing Students to Practice and Deepen Knowledge • Using Homework • Examining Similarities and Difference • Examining Errors in Reasoning • Practicing Skills, Strategies, and Processes • Revising Knowledge








Practicing & Deepening Knowledge

• Reviewing Content

• Organizing Students to Practice

and Deepen Knowledge

• Using Homework

• Examining Similarities and

Difference

• Examining Errors in Reasoning

• Practicing Skills, Strategies,

and Processes

• Revising Knowledge

Generating & Testing Hypothesis

• Organizing Students for Cognitively

Complex Tasks

• Engaging Students in Cognitively

Complex Tasks Involving Hypothesis

Generation and Testing

• Providing Resources and Guidance


4/19/2013

9

Domain 1: Instructional Support Strategies and Behaviors Lesson Segment
















Developmental Scale

All 41 elements in Domain 1: Classroom

Strategies and Behaviors use the same

developmental scale construct.

4/19/2013

10

Generic Scale for

All 41 Elements in Domain 1

Innovating 4

Applying 3

Developing 2

Beginning 1

Not Using 0

New strategies are created to meet needs of specific students or class as a whole in order for the desired effect to be evident in all students.

Strategy is used and monitored to see if it has desired effect with the majority of students.

Strategy is used correctly but the majority of students are not monitored for the desired effect of the strategy.

Strategy is used but pieces or components are missing.

Strategy is called for, but not used.

Domain 2-4

29

Domain 1: Classroom Strategies and Behaviors Lesson Segment
















4/19/2013

11

Deliberate Practice Involves:

Focused Feedback

Focused Practice

Cycle of Continuous Improvement

Systematic Annual Growth

4/19/2013

12

Evaluation

An expectation that all teachers can increase their expertise from year to year which produces gains in student achievement from year to year with a powerful cumulative effect.

Remember the Goal

The Marzano Framework organizes 9 of 10 Design Questions

into Three Lesson Segments:

36


Design Question 1: What will I do to establish and communicate learning goals, track student

progress, and celebrate success?

Design Question 6: What will I do to establish or maintain classroom

rules and procedures?

Lesson Segment Addressing Content

Design Question 2: What will I do to help students effectively interact

with the new knowledge?

Design Question 3: What will I do to help students practice and

deepen their understanding of new knowledge?

Design Question 4: What will I do to help students generate and test hypotheses about new knowledge?

Lesson Segment Enacted on the Spot

Design Question 5: What will I do to engage students?

Design Question 7: What will I do to recognize and acknowledge

adherence and lack of adherence to classroom rules and procedures?

Design Question 8: What will I do to establish and maintain effective

relationships with students?

Design Question 9: What will I do to communicate high expectations

for all students?

4/19/2013

13

Domain 2:

Planning and Preparing

Planning and Preparing for Lessons and Units

Planning and Preparing for Use of Resources and

Technology

Planning and Preparing for the Needs of English Language Learners

Planning and Preparing for the Needs of Students

Receiving Special Education

Planning and Preparing for the Needs of Students Who Lack Support for Schooling

Domain 3:

Reflecting on Teaching

Evaluating Personal Performance

Developing and Implementing a

Professional Growth Plan

Domain 4:

Collegiality and Professionalism

Promoting a Positive Environment

Promoting Exchange of Ideas and Strategies

Promoting District and School Development

The Marzano Framework Domains 2-4

Scoring of a Teacher’s

Instructional Practice

Rating the Elements

• 5-Level scale used to rate and provide feedback to teachers on their use of the 60 Elements of the Art and Science of Teaching Framework

Instructional Practice Score

• Scale used to determine Teacher Instructional Practice Score

38

Learning Goal 1:

Understand the Relationship of the Rating Levels for Elements to the


39

4/19/2013

14

Rating Scale for Domain

Elements (Marzano Scale)

40

4 3 2 1 0

Formative

Ratings

Used for

Each

Domain

Element

Innovating Applying Developing Beginning Not Using

Michigan 4-Level Scale

41

4 3 2 1

Ratings

Used for

Each

Domain

Element

Highly

Effective Effective

Minimally

Effective Ineffective

Learning Goal 2: Understand the

Marzano Model for Determining the


42

4/19/2013

15

Marzano Model Implementation

Approach

Year 1 Year 2

43

Status Score

Deliberate Practice Score

Status Score




=

Status Score combined with

Deliberate Practice Score

Student

Growth

Student

Growth

What is the Status Score?

• All feedback ratings roll into the Status Score.

• Status Score is an aggregation of a teacher’s

ratings across all elements within the framework

that result in a score.

• Status refers to one point in time (end-of-year

evaluation.)

• We recommend using only status score for Year 1.

44

Recap:


Reflects teachers’ performance across all

elements within the framework (Domains 1-4)

Accounts for teachers’ experience levels

Assigns weight to the domain with greatest

impact on student achievement (Domain 1)

Recommended in year 2, it acknowledges

teachers’ focus on deliberate practice by

measuring teacher improvement over time on

specific elements within the framework

45

4/19/2013

16

Process Begins with

Sources of Evidence

46

Purposes of Teacher Evaluation

Formative

Shape, form or improve teacher

practice

Summative

Quality Assurance

Sources of Evidence for Domains 1 -4

47

Group Think:

Brainstorm ideas for different sources of

evidence for:

Domain I: Classroom Strategies & Behaviors

Domain II: Planning & Preparing

Domain III: Reflecting on Teaching

Domain IV: Collegiality & Professionalism

4/19/2013

17

Recommended Sources of Evidence

Domain 1: Classroom Strategies & Behaviors

49

•Formal observation(s) • Informal, announced observation • Informal unannounced observation •Walkthroughs •Student work from observation •Student surveys •Videos of classroom practice •Artifacts


Domain 2: Planning and Preparing

50

•Planning conference or preconference •Lesson and unit planning documents •Pacing guides/Curriculum maps •Classroom artifacts


Domain 3: Reflecting on Teaching

51

•Self-assessment •Reflection conference •Professional Growth Plan •Conferences •Discussions •Artifacts

4/19/2013

18


Domain 4: Collegiality & Professionalism

52

•Professional development logs

•Team and faculty meetings

•Curriculum committees

•Lesson study

•Conferences

•Discussions

•Artifacts

Learning Goal 3:

Use a Process of Assigning the

Instructional Practice Score Based on

the Results of Multiple Data Sources

and Ratings on Elements.

53

Benefits of the Instructional

Practice Status Score (Conjunctive Approach)

• Proficiency level determined by whether

teachers have achieved minimum requirements

on the scale (rather than by averaging scores)

• Uses a scale that is sensitive to teachers’ time in

service

• Category I: 1-3 years

• Category II: 4 or more years

• Category III: 10 or more years (optional)

• Recognizes development of expertise

throughout teachers’ careers while also

celebrating milestones along the way 54

4/19/2013

19

Proficiency Scale for

All 3 Categories

55

CI Highly Effective (4) Effective (3)

Minimally Effective (2)

Ineffective (1)

D1:

At least 65% at Level 4

At least 65% at Level 3 or higher

Less than 65% at Level 3 or higher and Less than

50% at Level 1, 0

Greater than

or equal to

50% at Level

1, 0

D2:

D3:

D4:

CII Highly Effective (4) Effective (3)


Ineffective (1)

D1:

At least 75% at

Level 4

At least 75% at Level 3 or higher

Less than 75%at Level 3 or higher and Less

than 50% at Level 1, 0

Greater than or equal to 50% at

Level 1, 0

D2:

D3:

D4:

CIII

Highly Effective

(4) Effective (3)


Ineffective (1)

D1:

At least 85% at

Level 4

At least 85% at

Level 3 or higher

Less than 85% at

Level 3 or higher and

Less than 50% at

Level 2, 1, 0

Greater than or equal to 50% at

Level 2, 1, 0

D2:

D3:

D4:

Creating a Soft Landing for Your Teachers

Possibility for Phasing:

56

Implementation

Year

Category I Category II Category

III

Year 1 55% 65% 75%

Year 2 60% 70% 80%

Year 3 65% 75% 85%

Weighting System Reflects “Causal” Approach

• Recommended weight for each domain (60 Total

Elements)

• Domain 1: 68%, 41 Elements




57

D1 68%

D2 14%

D3 8%

D4 10%

4/19/2013

20

58

Example Step 2: Count the number of ratings at each level for Domain 1

Domain 1: Elements with Data Evidence Sources

Frequency at Level 4





59

Directions:

1. Using the Domain Forms, count the number of times each scale level has been recorded

2. Enter the frequency in the yellow highlighted cells

Frequency D1 D2 D3 D4

Level 4 12 7 4 3

Level 3 34 14 8 6

Level 2 15 7 5 4

Level 1 3 4 3 2

Level 0 2 3 2 -

Total Elements Used 66 35 22 15

Percentages D1 D2 D3 D4

Level 4 18% 20% 18% 20%

Level 3 52% 40% 36% 40%

Level 2 23% 20% 23% 27%

Level 1 5% 11% 14% 13%

Level 0 3% 9% 9% 0%

100% 100% 100% 100%

3. Adjust weights in gray highlighted cells; must add up to 100%

Category I Teachers (View Scale) D1 D2 D3 D4 Total

Status Score 3 2 2 2

Weight 68% 13% 8% 10% 100%

Weighted Score 2.05 0.27 0.17 0.20

Overall Status Score:

Overall Status: Effective

STATUS SCORE

2.68

Category I Teacher Sample

60

3. Adjust weights in gray highlighted cells; must add up to 100%

Category I Teachers (View Scale) D1 D2 D3 D4 Total

Status Score 3 2 2 2

Weight 68% 13% 8% 10% 100%

Weighted Score 2.05 0.27 0.17 0.20

Overall Status Score:

Overall Status: Effective

2.68

14%

4/19/2013

21

61

ALIGNED TEACHER PD AND

CAPACITY BUILDING AT ALL

LEVELS

Implementation Phases

Professional Development Track

4/19/2013

22

Recommended Observer Growth Progression

© 2010 Learning Sciences International


MISALIGNED

SYSTEM No Common Language or Model of

Instruction

ALIGNED SYSTEM Common Language or Model of

Instruction

COMMON LANGUAGE OF INSTRUCTION ALIGNS

MISALIGNED SYSTEMS

June 19-21 • 2013

MarzanoConference.com

Join us at this year’s Marzano

Conference 2013 for a deep dive

into teacher development,

evaluation, and Common Core

with Dr. Robert Marzano.

Register now:

4/19/2013

23

Contact

Beth Carr

(717)818-3973

[email protected]

www.marzanocenter.com

mailto:[email protected]

connecting teaching practices to student data documents/dr robert...marzano causal teacher...

Documents