using the teaching pyramid observation tool (tpot™) for preschool classrooms

Teaching Pyramid Observation Tool (TPOT)

Mary Louise HemmeterPatricia Snyder

Lise Fox

Overview of TPOT

• Observation- and interview-informed, judgment-based rating scale

• Developed to assess teachers’ implementation of universal and targeted Pyramid model practices and teachers’ capacity to implement tertiary/individualized practices

Tertiary Intervention: Few Children

Secondary Prevention:

Some Children

Universal Promotion: All Children

The Pyramid Model: Promoting Social and Emotional Competence and Addressing

Challenging Behavior

3

Teaching Pyramid Observation Tool (TPOT)

• The TPOT was developed to measure the fidelity with which teachers implement Pyramid Model practices

• Provides information that can be used to:– Describe “quality” of implementation of TPOT practices– Compare implementation within and across

teachers/classrooms– Identify needs of teachers for training and support

TPOT• Scoring booklet

– Classroom information– Schedule– Items/indicators– Space for notes

• Manual– Definitions– Items/indicators with scoring guidance, rules, and clarifications

Using the TPOT• Observations

–Conducted for a minimum of 2 hours• Must observe centers or free play, at least one

teacher-directed activity and transition between activities

• Focus of observation is primarily lead teacher’s behavior, but consider all adults

• Interviews– For those practices that might not be observed during a

2-hour observation

Organization of the TPOT: Subscales

• Key Practice Items: Multiple indicators associated with each item

– Each indicator rated yes, no, or N/0 (only when noted)

• Red Flags– Each item rated yes or no

• Using Effective Strategies for Responding to Challenging Behavior– Includes three indicators that represent “essential” strategies for responding to

challenging behavior and three additional strategies that might be used– Each item rated as yes or no– Only scored when challenging behavior is observed during the observation

period

Key Practice Item Subscale

Observation items1. Schedules, routines, & activities (SR)2. Transitions between activities (TR)3. Supportive conversations (SC)4. Promoting engagement (ENG)5. Providing directions (PD)6. Collaborative teaming (CT)7. Teaching behavior expectations (TBE)8. Teaching social skills and emotional

competencies (TSC)

Observation and interview items9. Teaching friendship skills (FR)10. Teaching children to express emotions (TEE)11. Teaching problem-solving

(TPS)

Interview items12. Interventions for children with

persistent challenging behavior (PCB)

13. Connecting with families (COM)14. Supporting Families in using

Pyramid Model practices (INF)

Key Practice Item

9

Item

Indicator

Score

Columns

No Opportunity

Key Practice Item: Observation Only

10

Key Practice Item: Observation AND Interview Item

11

Red Flags Subscale

• Practices that are contraindicated in relation to Pyramid Model implementation

Red Flags

14

Challenging Behavior Subscale:Using Effective Strategies to Respond to Challenging Behavior (SCB)

• Only scored when challenging behavior occurs. If challenging behavior occurs during the observation, score for each incident of challenging behavior.

• If one or more incidences of challenging behavior occur that meet the definition for challenging behavior, record a short-hand note in the ‘incident’ box to describe the type of challenging behavior you observed

• For each incident that occurred, mark ‘yes’ if you observed the teacher using a practice related to the SCB indicator or mark ‘no’ if you did not observe the teacher using the practice to address the challenging behavior

Defining the Behavior – What to Note

Challenging Behavior Subscale

17

What is different about this version

of the TPOT?

Studies to Examine the Psychometric Integrity of Teaching Pyramid Observation Tool (TPOT) Scores

• Primarily Head Start Classrooms• TPOT observations

– 50 classrooms– 2 raters– 3 occasions

• 300 total TPOTs• Generalizability study to look at

dependability of scores across items, raters, occasions of measurement

• CLASS (Pianta, LaParo, & Hamre, 2008)

– 50 classrooms– Administered between 2nd and 3rd

TPOT observation– Inter-observer agreement for 34% (n

= 17)– Convergent score validity

Generalizability Study• Examine potential sources of

error in TPOT scores• Teachers X Raters X Indicators

X Occasion • Less than .01% of variance on

key practice indicators due to raters

• .22% of variance on key practice indicators due to occasion

• 5.1% of variance on key practice indicators due to classroom/teacher

• Phi coefficient for key practice indicators = .89

• G coefficient for key practice indicators = .95

• Good news! • Dependability in rank

ordering of classrooms and dependability in scores across raters, occasions

• Phi coefficient = important because want teachers to reach criterion with respect to implementation of TPOT indicators

Snyder, P., Hemmeter, M.L., Fox, L., Bishop, C., & Miller, M.D. (2013). Developing and gathering psychometric evidence for a fidelity instrument. Journal of Early Intervention, 35, 150-172.

TPOT and CLASS Correlations

N = 50TPOT Key Practices

Emotional Support (ES)

.70

Classroom Organization (CO)

.73

Instructional Support (IS)

.76

Snyder, P., Hemmeter, M.L., Fox, L., Bishop, C., & Miller, M.D. (2013). Developing and gathering psychometric evidence for a fidelity instrument. Journal of Early Intervention, 35, 150-172.

Noteworthy Correlations:TPOT and CLASS*

• TPOT Key Practices subscale and every CLASS dimension and domain

• TPOT Red Flags subscale and every CLASS dimension and domain (negative relationships)

• General teaching items on TPOT (SR, TR, SC, ENG, PD) and each CLASS dimension and domain

• Most targeted teaching items on TPOT (TBE, TSC, TEE, TPS, FR) and Instructional Support CLASS domain

• TPOT Connecting with Families with each dimension and domain on CLASS

Additional detail in Chapter 7 in TPOT manual

Noteworthy CorrelationsTPOT and ECERS-R

• TPOT Key Practices subscale and overall ECERS-R• 10 of the 14 TPOT key practice items and overall

ECERS-R• TPOT Red Flags subscale and overall ECERS-R

(negative relationships) • TPOT Supporting Family Use of Pyramid Model

Practices and 5 of 7 ECERS-R subscales• TPOT Strategies to Promote Engagement and 6

of 7 ECERS-R subscalesAdditional detail in Chapter 7 in TPOT manual

Pre-SET and Select TPOT-Pilot Version Key Practice Items(N = 31 Classrooms)

Note. Adapted from Steed and Pomerleau (2012). N = 31 classrooms. a = Seven environmental items included on pilot version of TPOT. * = p < .05 ** = p < .01

Additional detail in Chapter 7 in TPOT manual

Figure 1. Mean Teaching Pyramid Observation Tool implementation scores across waves by experimental condition. Total number of TPOT indicators = 108.Statistically significant and noteworthy differences at Wave 4 [t(40.03)=6.80, p<.001, Cohen’s d=2.6)

Results: Implementation of Teaching Pyramid Practices

Hemmeter, M.L., Fox, L., Snyder, P., & Algina, J. (2011, April). Efficacy of a classroom-wide model for promoting social-emotional development and preventing challenging behavior. Paper presented at the annual meeting of the American Educational Research Association, New Orleans, LA.

Results: SSIS Child Outcomes

Adjusted Means Cohen’s d Effect SizeIntervention Control

Target ChildrenSocial 88.6 84 .41

Problem Behavior 108.7 115.5 -.52*Non Target Children

Social 103.8 96.4 .46*Problem Behavior 95.2 99 -.29

Hemmeter, M.L., Fox, L., Snyder, P., & Algina, J. (2011, April). Efficacy of a classroom-wide model for promoting social-emotional development and preventing challenging behavior. Paper presented at the annual meeting of the American Educational Research Association, New Orleans, LA.

Figure 2. Mean frequency of positive social interactions during 60 min observation session across waves for Cohort 1 target children whose teachers were in the intervention or control condition. An average of the frequency of positive social interactions for the 2 to 3 target children in each classroom was used to derive the means reported for each group at each wave.

Results: Target Child Social Interactions

Hemmeter, M.L., Fox, L., Snyder, P., & Algina, J. (2011, April). Efficacy of a classroom-wide model for promoting social-emotional development and preventing challenging behavior. Paper presented at the annual meeting of the American Educational Research Association, New Orleans, LA.

Using the TPOT to Inform Decision Making

• Using the TPOT in coaching– Running TPOT– Formal TPOT– Goal setting/action planning

• Using the TPOT program wide– Monitor implementation of PW implementation– Plan professional development

• Using the TPOT in monitoring/evaluation

Sched

ules, ro

utines, an

d activiti

es

Transiti

ons betw

een ac

tivities

Supporti

ve co

nversa

tions

Promoting c

hildren

’s enga

gemen

t

Provid

ing dire

ctions

Collaborati

ve te

aming

Teach

ing child

ren beh

avior e

xpect

ations

Teach

ing socia

l skills

and em

otional co

mpetencie

s

Teach

ing frie

ndship sk

ills

Teach

ing child

ren to

expres

s emotions

Teach

ing pro

blem so

lving

Interve

ntions for c

hildren

with

persist

ent c

hallen

ging b

ehav

ior

Connecting w

ith fa

milies

Supporti

ng fam

ily use

of the P

yramid M

odel prac

tices

Key Prac

tices Su

bscale

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Percentage of Indicators Observed for Key Practice Items

Initial TPOT Spring TPOT

Sched

ules, ro

utines, an

d activiti

es

Transiti

ons betw

een ac

tivities

Supporti

ve co

nversa

tions

Promoting c

hildren

’s enga

gemen

t

Provid

ing dire

ctions

Collaborati

ve te

aming

Teach

ing beh

avior e

xpect

ations

Teach

ing socia

l skills

and em

otional co

mpetencie

s

Teach

ing frie

ndship sk

ills

Teach

ing child

ren to

expres

s emotions

Teach

ing pro

blem so

lving

Interve

ntions for c

hildren

with

persist

ent c

hallen

ging b

ehav

ior

Connecting w

ith fa

milies

Supporti

ng Fam

ily use

of the P

yramid M

odel prac

tices

Key Prac

tices Su

bscale

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Percentage of Indicators Observed for Key Practice ItemsInitial - 8 Teachers November - 8 Teachers April - 8 Teachers

To order:www.brookespublishing.com1-800-638-3775

To learn more about other social-emotional development products: http://products.brookespublishing.com/Social-Emotional-Development-C1033.aspx

Thanks for joining us!

Questions?