Providing Effective, Supportive Feedback on Teacher Observations

Virginia Department of EducationOffice of School Improvement

2013-2014

Agenda1. Why is lesson observation important?

2. What is evidence and how do we objectively collect it?

3. How do we decrease bias and increase reliability in observations?

4. How do we use a model to give constructive feedback?

Why is Lesson Observation Important?

Why do we observe teachers?

“…[T]he purpose of supervision should be the enhancement of teachers’ pedagogical skills, with the ultimate goal of enhancing student achievement.”

4Marzano, R.J., Fronteri, T., & Livingston, D. (2011). Effective supervision: Supporting the art and science of teaching. Alexandria, Virginia: ASCD..

“My ideal evaluator would care deeply aboutthe work I do and would have a vision for how that work, done artfully, contributes to

the betterment of people.

Tomlinson, C.A. (2012). “The evaluation of my dreams.” Educational Leadership, 70(3). ASCD.

Teacher evaluation is not an end in itself, but a means to an end —

TEACHER IMPROVEMENT.

Davis, D. R., Ellett, C. D., & Annunziata, J. (2002). Teacher evaluation, leadership and learning organizations. Journal of Personnel Evaluation in Education, 16(4), 287-301. p. 288.

Why Evaluation Only Is Not Enough

Making Permanent Changes: The Importance of Leadership

Explicit Training and Expectations on Needed

Skills

Explicit, Sustained

Emphasis and Monitoring of New

Skills

Change in Practice

Question 2

What is evidence, and how do we collect it objectively?

What is Evidence?

“The available body of facts or information

indicating whether a belief or proposition is true

or valid.”

-Dictionary.com

Evidence considered

The teacher is Proficient in the

teaching standard.

Focus of Evidence

STANDARDS & INDICATORS

Teacher Practice

Student Learning

Evidence in Observations…

• Is standards-based• Can include both examples of meeting

or not meeting the expectation• Does not include judgment

• 3-4 observations a year = less than 1% of overall teaching time

• Observations = snapshots

Main Areas for Evidence Collection during ObservationsStandard 1: Professional KnowledgeStandard 2: Instructional PlanningStandard 3: Instructional DeliveryStandard 4: Assessment of/for Learning

(formative assessment)Standard 5: Learning EnvironmentStandard 6: ProfessionalismStandard 7: Student Academic Progress

How do we collect evidence objectively?

Documenting Evidence Tips• Avoid terms that express judgment

(“neat classroom,” “fun activity,” “caring attitude”)

• Avoid words that imply, but do not specify, quantity (“most,” “few,” “several”)

• Stick to the five senses• Remember Who, What, When, Where,

How

How can this evidence be improved?

3.3 Differentiates instruction to meet students’ needs.

Spent the majority of the time on whole class instruction for math though several of the students demonstrated proficiency on the long division process in the first few minutes of class.

How much?How many?

How?

Better Example

3.3 Differentiates instruction to meet students’ needs.

The teacher spent 45 minutes reviewing the steps in the long division process with the whole class though four students correctly completed all steps in the process on the problem introduced during the warm-up activity.

How can this evidence be improved?

5.1 Arranges the classroom to maximize learning while providing a safe environment.

Classroom neatly arranged so students can work in small groups.

How? Desks in groups? Center

areas?

When during the lesson do students

work in small groups?

How is this defined?

Improve this Evidence: One Example

5.1 Arranges the classroom to maximize learning while providing a safe environment.

All desks are arranged in clusters of 5-6. During Explore activity, teacher distributed task cards to each cluster and had them work together to solve the Explore problem.

Using an Standards-Based Observation Form

Standard 1: Professional Knowledge

1.1 Effectively addresses appropriate curriculum standards.1.2 Integrates key content elements and facilitates students’ use of

higher level thinking skills in instruction.1.3 Demonstrates an ability to link present content with past and future

learning experiences, other subject areas, and real world experiences and applications.

1.4 Demonstrates an accurate knowledge of the subject matter.1.5 Demonstrates skills relevant to the subject area(s) taught.1.6 Bases instruction on goals that reflect high expectations and an

understanding of the subject.1.7 Demonstrates an understanding of the intellectual, social, emotional,

and physical development of the age group.1.8 Communicates clearly and checks for understanding.

Example Observation1. Professional KnowledgeThe teacher demonstrates an understanding of the curriculum, subject content, and the developmental needs of students by providing relevant learning experiences. Effectively addresses appropriate

curriculum standards. Integrates key content elements and

facilitates students’ use of higher level thinking skills in instruction.

Demonstrates ability to link present content with past and future learning experiences, other subject areas, and real world experiences and applications.

Demonstrates an accurate knowledge of the subject matter.

Demonstrates skills relevant to the subject area(s) taught.

Bases instruction on goals that reflect high expectations and an understanding of the subject.

Demonstrates an understanding of the intellectual, social, emotional, and physical development of the age group.

Communicates clearly and checks for understanding.

Evidence:• Facilitates students’ use of higher level thinking skills in instruction: During

Focus Lesson group discussions, many students state their answers followed by “because” and a verbal justification of how they found the answer. Teacher Uses open-ended questions that encourage student thinking during Focus Lesson whole group debrief (“Can you please tell me what your group found out about your Entrance Cards?”)

Sample IndicatorsSample Performance Indicators:Examples may include, but are not limited to:

The teacher: 3.1 Engages and maintains students in active learning.3.2 Builds upon students’ existing knowledge and skills.3.3 Differentiates instruction to meet the students’ needs.3.4 Reinforces learning goals consistently throughout the

lesson.3.5 Uses a variety of effective instructional strategies and

resources.3.6 Uses instructional technology to enhance student

learning.3.7 Communicates clearly and checks for understanding.

Example Observation3. Instructional DeliveryThe teacher effectively engages students in learning by using a variety of instructional strategies in order to meet individual learning needs. Engages and maintains students in active

learning. Builds upon students’ existing knowledge

and skills. Differentiates instruction to meet the

students’ needs. Reinforces learning goals consistently

throughout lessons.

Uses a variety of effective instructional strategies and resources.

Uses instructional technology to enhance student learning.

Communicates clearly and checks for understanding.

Evidence:• Communicates clearly and checks for understanding: Directions for Focus

Lesson posted on chart paper and on student worksheets. Walks around and informally checks independent work on Entrance Card/Warm-Up and provides a sticky note. Uses Exit Cards during Whole Group Closure.

• Differentiates instruction to meet students’ needs: All students work on the same activities and receive the same lesson during small group/center time. Teacher spends 15 minutes working with groups A and B and 20 minutes working with group C, who showed difficulty with the material during the Entrance Card/Warm-up activity.

Sample IndicatorsSample Performance Indicators:Examples may include, but are not limited to:

The teacher: 4.1 Uses pre-assessment data to develop expectations for students, to

differentiate instruction, and to document learning.4.2 Involves students in setting learning goals and monitoring their own

progress.4.3 Uses a variety of assessment strategies and instruments that are valid

and appropriate for the content and for the student population.4.4 Aligns student assessment with established curriculum standards and

benchmarks.4.5 Uses assessment tools for both formative and summative purposes, and

uses grading practices that report final mastery in relationship to content goals and objectives.

4.6 Uses assessment tools for both formative and summative purposes to inform, guide, and adjust students’ learning.

4.7 Gives constructive and frequent feedback to students on their learning.

Example Observation4. Assessment of/for LearningThe teacher systematically gathers, analyzes, and uses all relevant data to measure student academic progress, guide instructional content and delivery methods, and provide timely feedback to both students and parents throughout the school year. Uses pre-assessment data to develop

expectations for students, to differentiate instruction, and to document learning.

Involves students in setting learning goals and monitoring their own progress.

Uses a variety of assessment strategies and instruments that are valid and appropriate for the content and for the student population.

Aligns student assessment with established curriculum standards and benchmarks.

Uses assessment tools for both formative and summative purposes and uses grading practices that report final mastery in relationship to content goals and objectives.

Uses assessment tools for both formative and summative purposes to inform, guide, and adjust students’ learning.

Gives constructive and frequent feedback to students on their learning.

Evidence: Uses pre-assessment data to differentiate instruction: Uses student

independent responses on Warm-Up/Entrance slip to form small groups Gives constructive and frequent feedback to students on their learning:

During Guided Group: Independent Practice, after meeting with the pink group teacher walks around and monitors the other groups, checking on their progress and asking how they are doing.

Sample IndicatorsSample Performance Indicators:Examples may include, but are not limited to:

The teacher: 5.1 Arranges the classroom to maximize learning while providing a safe

environment.5.2 Establishes clear expectations, with student input, for classroom rules

and procedures early in the school year, and enforces them consistently and fairly.

5.3 Maximizes instructional time and minimizes disruptions.5.4 Establishes a climate of trust and teamwork by being fair, caring,

respectful, and enthusiastic.5.5 Promotes cultural sensitivity.5.6 Respects students’ diversity, including language, culture, race, gender,

and special needs.5.7 Actively listens and pays attention to students’ needs and responses.5.8 Maximizes instructional learning time by working with students

individually as well as in small groups or whole groups.

Example Observation5. Learning EnvironmentThe teacher uses resources, routines, and procedures to provide a respectful, positive, safe, student-centered environment that is conducive to learning.• Arranges the classroom to maximize learning

while providing a safe environment. • Establishes clear expectations, with student

input, for classroom rules and procedures early in the school year, and enforces them consistently and fairly.

• Maximizes instructional time and minimizes disruptions.

• Establishes a climate of trust and teamwork by being fair, caring, respectful, and enthusiastic.

• Promotes cultural sensitivity. • Respects students’ diversity, including language,

culture, race, gender, and special needs. • Actively listens and pays attention to students’

needs and responses. • Maximizes instructional learning time by working

with students individually as well as in small groups or whole groups.

Evidence:• Establishes clear expectations: Students enter classroom quietly and in a line

before lesson. Students raise their hands quietly and wait to be called upon during Focus Lesson group debrief.

• Arranges classroom to maximize learning: During Guided Groups: Independent Practice, there are 3 groups and about 7-9 students per group; there are enough materials for students to separate themselves into even smaller groups of 2-4. Center materials are in folders and bags allowing students to work anywhere in the room (at desks, on the floor, etc.).

Increasing Reliability, Decreasing Bias in Observations

What are some of the most common reliability issues in observations?

Problem:Lack of Criterion Reliability

How consistent are the evaluator’s observations with those of an expert

observer?

The MET Project, 2013

Solution:Establishing Criterion Reliability

• Training with an expert observer• Practice scoring using videos,

documentation, etc.• Instruction on how to interpret

evidence• Information about common sources of

systemic observer error

The MET Project, 2013

Problem:Establishing Intra-Evaluator

Reliability

How consistent is the evaluator in his/her own observations?

The Met Project, 2013

Solution:Establishing Intra-Evaluator Reliability

• Watch the same lesson multiple times with time between

• Give ratings based on multiple pieces of evidence rather than single observations

The Met Project, 2013

Problem:Lack of Inter-Evaluator Reliability

How consistent is evidence collected between two or more evaluators?

The Met Project, 2013

Solution:Establishing Inter-Evaluator Reliability

• Calibrate evidence-collection during initial training

• Conduct tandem observations and performance reviews with multiple evaluators

The Met Project, 2013

How do we make observations more reliable overall?

• Acknowledging the existence of bias• Understanding the different types of

bias and how to counteract them

The Met Project, 2013

How do we decrease bias?

Problem: Observer Personal Bias

Observers collect idiosyncratic or irrelevant evidence during lesson observations

Definition

Well, you used higher level questioning, the students seemed engaged, and your formative assessment was on-point.

Unfortunately, I noticed that your Unifix cubes weren’t organized by color, so you lost a few

points there.

Solution

Train evaluators on objective ways to collect evidence on uniform, research-based criteria.

Problem: Halo and Pitchfork Effect

Early impressions of the teacher influence all further observations and evidence-collection

Definition

Don’t worry, Mr. Williams. You can receive three more poor lesson

observations before you use up all the points you earned at your interview for

being well-spoken and using that heavy-bond resume paper.

Stronge and Associates Educational Consulting,

LLC

That was a great lesson

you just taught, Ms. Johnson. It’s really too

bad you came late to that

meeting back in September and I can only mark you negatively

now.

Solutions

• Train evaluators on objective ways to collect evidence on uniform, research-based criteria

• Multiple observers provide various perspectives

Problem: Observer Drift

Tendency of observers who originally had a desired level of agreement and accuracy to gradually and unconsciously redefine criteria

Definition

Student learning is measured by the number of

higher level questions students ask during the

lesson.

No, it’s measured by the number of

students who are awake at any given

moment.

I think it’s more to do with quality of

the products student create

during a lesson.

Overall Ways to Increase Reliability and Decrease Bias

Overall Ways to Increase Reliability

• Same evaluator observing multiple lessons by the same teachers

Overall Ways to Increase Reliability

• Adding more evaluators within the school to the same observation

Overall Ways to Increase Reliability

Add observers or evaluators from outside the school to either observe or review observations

Evaluators’ performance increases when they know they are being monitored (Center of Educational Compensation Reform, 2009)

Can be a small sample of teachers’ observations

Summary of this Section

• Be aware of and look out for bias

• Train, train, train• Reliability in numbers

Question 4

How do we use a model to give constructive feedback?

What Makes Feedback Informative and Effective?

Gordon, S. P. (2006). Teacher evaluation and professional development. In J. H. Stronge. (Ed.). Evaluating teaching: A guide to current thinking and best practice (2nd ed.) (pp. 268-290). Thousand Oaks, CA: Corwin.

Informative and effective feedback is:• Focused on teaching and learning rather than other areas• Aligned to performance standards• Contextual rather than context free; contexts that should be

considered include: School’s mission and improvement goals Curriculum and instructional goals Teachers’ instructional goals, level of expertise, and

career history career histories Students’ cultural background, prior learning, current

needs

What Makes Feedback Informative and Effective?

Informative and effective feedback is:• Generated through analysis of deep, rich

evaluation data• Generated based on long-term, continuous

data gathering rather than “one-shot” evaluations

Gordon, S. P. (2006). Teacher evaluation and professional development. In J. H. Stronge. (Ed.). Evaluating teaching: A guide to current thinking and best practice (2nd ed.) (pp. 268-290). Thousand Oaks, CA: Corwin.

Tips for the Post-Observation Conference

Think back to a post-observation conference you

had that was particularly EFFECTIVE.

What about it was effective? Write down 2-4 things.

56

Do any of these sound familiar?

Effective conferencing skills:

• Conducting a focused observation during which data are gathered

• Knowing about effective teaching methods

• Understanding how teaching relates to learning

• Analyzing data (both from an observation and from assessments)

• Knowing how to make a conference reflective and nonthreatening

57Blase, J., & Blase, J. (2004). The dark side of school leadership: Implications for administrator preparation. Leadership and Policy in Schools, 3(4), 245-273.

Think back to a post-observation conference you

had that was particularly INEFFECTIVE.

What about it was ineffective? Write down 2-4 things.

58

Do any of these sound familiar?

Research shows that ineffective conferences tend to:• Be dominated by the individual leading the

conference• Be focused on narrow concerns• Provide a teacher with short, prescriptive

feedback• Threaten teacher self-esteem• Be influenced by an unequal power relationship• Lack teacher reflection and self-evaluation• Lack reasoning and critical thinking

59Blase, J., & Blase, J. (2004). The dark side of school leadership: Implications for administrator preparation. Leadership and Policy in Schools, 3(4), 245-273.

Feedback that GIVES…

• Growth-oriented

• Instructionally-sound

• Values the recipient

• Evidence-based

• Scope

Growth-Oriented

• Does the feedback encourage specific reflective, and measurable growth from the recipient?

• Does the feedback provide ideas for future growth?

The Widget Effect Study

• Evaluation practices in 12 divisions across four states, and 15,000 teachers, 1,300 administrators.

• No areas for development cited in 73% of teacher evaluations

• Of those who did have areas for development cited, only 45% reported receiving useful support

• Only 12.15% of all teachers in the study said they received useful support for professional growth

Weisburg, D., Sexton, S., Mulhern, J., & Kelling, D. (2009). The widget effect: Our national failure to acknowledge and act on differences in teacher effectiveness. Brooklyn, NY: New Teacher Project, p. 4.

Teachers are not Widgets“The Widget Effect describes the

tendency of school districts to assume classroom effectiveness is the same from teacher to teacher. … In it’s denial of individual strengths and weaknesses, it is deeply disrespectful to teachers; in its indifference to instructional effectiveness, it gambles with the lives of students.”

Weisburg, D., Sexton, S., Mulhern, J., & Kelling, D. (2009). The widget effect: Our national failure to acknowledge and act on differences in teacher effectiveness. Brooklyn, NY: New Teacher Project, p. 4.

Meet Teacher A

• Grade 2 teacher• Difficulty transitioning between

subjects• Principal conducts several formal and

informal observations and walkthroughs, then conducts a conference to provide feedback and focus on growth

Instructionally Sound

Does the feedback demonstrate an appropriate understanding of content, effective instructional strategies, students’developmental needs, and/or the learningenvironment?

Resources for Instructionally-Sound Feedback

• Virginia Department of Education Curriculum Framework

• Division supervisors, specialists, or coaches

• Lead teacher or teacher leaders

Value

• Does the feedback value the strengths of the recipient?

• Does the feedback display appropriate tone and content that values the recipient’s feelings and point of view?

Evidence-Based

Does the feedback cite or analyze specificevidence or data?

Scope

• Is the feedback neither too broad nor too narrow?

• Does the feedback focus on things the recipient can control?

Feedback Scenario 1

Feedback that GIVES…

Growth-Oriented Instructionally Sound Values the Recipient Evidence-Based Scope

During the observation, I noted that you called on girls 15 times and boys 7, even though you have an equal number of each in your class. This is very discouraging for the boys.

Feedback Scenario 2Most students stayed on-task during the experiment. Four of the five groups’ paper helicopters completed more rotations when they added paper clips, but one group did not. Have you considered showing students the expected result ahead of time so they will all achieve it?

Feedback that GIVES…

Growth-Oriented Instructionally Sound Values the Recipient Evidence-Based Scope

Feedback Scenario 3

The lesson was clear, organized, and focused. Students appeared engaged in relevant learning. The resources used were appropriate for the developmental needs of students. Consider differentiating for an even greater impact.

Feedback that GIVES…

Growth-Oriented Instructionally Sound Values the Recipient Evidence-Based Scope

“Teacher growth requires a careful balance of constructive and critical performance feedback, creative coaching in new practices, and insistent optimism.”

Donaldson, M.L. & Donaldson, G.A. (2012) “Strengthening teacher evaluation: What district leaders can do. Educational Leadership, 69(8). ASCD. Retrieved from: http://www.ascd.org/publications/educational-leadership/may12/vol69/num08/Strengthening-Teacher-Evaluation@-What-District-Leaders-Can-Do.aspx