Using the PBIS Tiered Fidelity Inventory (TFI)

E-12Rob Horner & Kelsey Morris

University of Oregonwww.pbis.org

Goals• Introduce a new PBIS Fidelity Measure

• Purpose, Format, Functions of TFI

• Provide examples of how TIF is being useo With schoolso To define “exemplars”

Maximizing Your Session Participation

Work with your team

Consider 4 questions:

– Where are we in our implementation?

– What do I hope to learn?– What did I learn?– What will I do with what I learned?

V 2.0 b

Why Another Fidelity Measure?

SETTIC

BoQ SAS

ISSETBAT

MATT

Phases of Implementation

(POI)

Tiered Fidelity Inventory

(TFI)

V 2.0 b

The purpose of the School-wide PBIS Tiered Fidelity Inventory is to provide an efficient and valid index of the extent to which PBIS core features are in place within a school.

Tier I (Universal PBIS) Whole School Universal Prevention

Tier II (Targeted PBIS) Secondary, Small Group Prevention

Tier III (Intensive PBIS) Tertiary, Individual Support Prevention

Purpose of the PBIS Tiered Fidelity Inventory

V 2.0 b

Primary purpose of the instrument is to help school teams improve

Primary audience for instrument results is the team, faculty, families and administrators of the school.

Effective use of the instrument requires multiple administrations (progress monitoring)

SWPBIS Tiered Fidelity Inventory is a Self-Assessment (done with a COACH)

V 2.0 b

Formative Assessment Determine current PBIS practices in place and needed prior

to launching implementation

Progress monitoring Self-assess PBIS practices by tier to guide implementation

efforts, and assess progress by tier Build action plan to focus implementation efforts

Annual Self-Assessment Self-assess annually to facilitate sustained implementation

of PBIS

State Recognition Determine schools warranting recognition for their fidelity

of PBIS implementation.

Uses of the SWPBIS Tiered Fidelity Inventory

V 2.0 b

Currently on www.fieldtest.pbisassessment.org

Will be transitioned to www.pbisassessment.org

Access

V 2.0 b

Technical Adequacy

Time for Completion (average): Tier I: 15 minutes Tier II: 11 minutes Tier III: 12.5 minutes Inter-rater Reliability (IRR): Coaches completed the TFI alone prior to meeting with school teams. Then, the school teams completed the initial administration of the TFI. These results were compared to determine IRR (see Table 1). Table 1. Inter-Rater Reliability (Pearson Correlations)

Tier I Scale Tier II Scale Tier III Scale Overall 0.950 0.964 0.892 0.955

1. Content Validity Study (COMPLETE). Results showed strong content validity for the measure (average CVI = .93), and items with lower CVIs were changed based on feedback.

V 2.0 b

Test-retest Reliability (TRR): School teams completed an initial TFI administration and a follow-up administration two weeks later. These results were compared to determine TRR (see Table 2). Table 2. Test-Retest Reliability (Pearson Correlations)

Tier I Scale Tier II Scale Tier III Scale Overall 0.975 0.992 0.985 0.995

Usability Testing On-Going

V 2.0 b

Total Score

Subscale

Sub-subscale

Individual Items

Summary Data and Reports

Note: Pending Beta Test we continue to use the 80%

benchmark at each tier to indicate “implementation at

criterion”

V 2.0 b

Total score (TFI)

V 2.0 b

Sub-scale report

V 2.0 b

Sub-subscale report

Tier ITeamsImplementationEvaluation

Tier IITeamsInterventionsEvaluation

Tier IIITeamsResourcesAssessmentSupport planMonitoring and adaptation

V 2.0 b

Item Report

V 2.0 b

Action Planning

TFI Administration and Action Planning

TFI Walkthrough Tool

Tier I: Teams

• Composition• Operating Procedures

Tier I: Implementation

• Behavioral Expectations• Teaching Expectations• Problem Behavior Definitions• Discipline Policies• Professional Development• Classroom Procedures• Feedback &

Acknowledgement• Faculty Involvement• Student/Family/Community

Involvement

Tier I: Evaluation

• Discipline Data• Data-based Decision Making• Fidelity Data• Annual Evaluation

Tier II: Teams

• Composition• Operating

Procedures• Screening• Request for

Assistance

Tier II: Interventions

• Tier II Intervention Options

• Tier II Critical Features

• Practices Matched to Student Need

• Access to Tier I Supports

• Professional Development

Tier II: Evaluation

• Level of Use• Student

Performance Data

• Fidelity Data• Annual

Evaluation

Tier III: Teams

• Composition• Operating Procedures• Screening• Student Support Team

Tier III: Interventions

• Staffing• Student/Family/Community

Involvement• Professional Development

Tier III: Support Plans

• Quality of Life Indicators• Academic, Social, Physical

Indicators• Hypothesis Statement• Comprehensive Support• Formal & Natural Supports• Access to Tier I & Tier II

Supports

Tier III: Evaluation

• Data System• Data-based Decision Making• Level of Use• Annual Evaluation

TFI Tier III Support Plan Scoring

Item Current Score Action Who Whe

n1.1 Team Composition

1.2 Team Operating Procedures

1.3 Behavioral Expectations

1.4 Teaching Expectations

1.5 Problem Behavior Definitions

1.6 Discipline Policies

1.7 Professional Development

1.8 Classroom Procedures

1.9 Feedback & Acknowledgment

1.10 Faculty Involvement

1.11 Student/Family/Community Involvement1.12 Discipline Data

1.13 Data-based Decision Making

1.14 Fidelity Data

1.15 Annual Evaluation

Item Current Score Action Who Whe

n2.1 Team Composition

2.2 Team Operating Procedures

2.3 Screening

2.4 Request for Assistance

2.5 Options for Tier II Interventions2.6 Tier II Critical Features

2.7 Practices Matched to Student Need2.8 Access to Tier I Supports

2.9 Professional Development

2.10 Level of Use

2.11 Student Performance Data

2.12 Fidelity Data

2.13 Annual Evaluation

Item Current Score Action Who When3.1 Team Composition

3.2 Team Operating Procedures

3.3 Screening

3.4 Student Support Team

3.5 Staffing

3.6 Student/Family/Community Involvement3.7 Professional Development

3.8 Quality of Life Indicators

3.9 Academic, Social, & Physical Indicators

3.10 Hypothesis Statement

3.11 Comprehensive Support

3.12 Formal & Natural Supports

3.13 Access to Tier I & Tier II Supports

3.14 Data System

3.15 Data-based Decision Making

3.16 Level of Use

3.17 Annual Evaluation

Jan-14 May-14 Sep-140

10

20

30

40

50

60

70

80

90

100

36

62

71

Riverbend ElementarySubscales

Tier ITier IITier III

Evaluation—SWIS was installed for data

collection and analysis

Riverbend Action Planning

• TFI “subgroup” completed the TFI and developed goals and action items for subcommittees

• Subcommittees for:1.3 & 1.4 Behavioral Expectations & Teaching Expectations1.6 Discipline Policies1.9 Feedback & Acknowledgement1.11 Student/Family/Community Involvement

• TFI “subgroup” members led subcommittee work– Communication– Recruiting workers– Setting meetings

Riverbend Action Planning• Behavioral Expectations

– Established school-wide behavior matrix– Drafted lesson plans– Designed and ordered signage

• Discipline Policies– Established a response to problem behavior flowchart– Established a student-friendly flowchart– Developed training on referral form completion and problem behavior definitions

• Feedback & Acknowledgement– Established a system for how an individual’s acknowledgement benefits the group (e.g., Party in

a Bucket)– Created signage to advertise the acknowledgement system – Established language to be used by all staff when delivering acknowledgement

• Student/Family/Community Involvement– Established calendars that can be distributed to all families; each month of the calendar

highlights a different PBIS component– Developed a survey to garner community feedback

Subcommittees prepared 20-30 minute rollout trainings for all staff

Jan-14 May-14 Sep-140

10

20

30

40

50

60

70

80

90

100

36

71

62

46

7165

Riverbend Elementary Subscales

Tier ITier IITier III

Jan-14 May-14 Sep-140

10

20

30

40

50

60

70

80

90

100

36

71

90

62

46

7165

Riverbend ElementarySubscales

Tier ITier IITier III

Jan-14 May-14 Sep-140

10

20

30

40

50

60

70

80

90

100

39

54

32

Hamlin Middle SchoolSubscales

Tier ITier IITier III

Evaluation—SWIS was installed for data

collection and analysis

Hamlin Action Planning

• PBIS Team used TFI results to identify items of importance1.3 & 1.4 Behavioral Expectations & Teaching Expectations1.6 Discipline Policies (e.g., flowchart)1.9 Feedback & Acknowledgement (e.g., menu of reinforcers)

• PBIS Team worked on items during team meetings– Solicited feedback from whole faculty – Revised as needed

• Created lesson plans to teach all staff the aspects of school-wide PBIS during in-service week

Jan-14 May-14 Sep-140

10

20

30

40

50

60

70

80

90

100

39

71

54

46

32

65

Hamlin Middle School Subscales

Tier ITier IITier III

Jan-14 May-14 Sep-140

10

20

30

40

50

60

70

80

90

100

39

71

90

5450

32

44

Hamlin Middle SchoolSubscales

Tier ITier IITier III

Training & Administration Resources

• Tier I—TFI Walkthrough Tool• Tier III—Student Support Plan Scoring Guide• PowerPoint Training Slides