HOW TO EFFECTIVELY USE DATA WITHIN THE PBIS FRAMEWORK

Kelly M. Vaillancourt, School Psychologist

Kim A. Comrie, Assistant Principal

Rolling Ridge Elementary School, Loudoun County Public Schools, VA

OBJECTIVES OF TODAY’S PRESENTATION

Participants will: Review the essential elements of PBIS

Learn essential elements of effective data systems

Learn effective data collection, analysis and reporting methods

Discuss how to design and analyze Tier 1 and Tier 2 supports using data to drive decisions

ESSENTIAL ELEMENTS OF PBIS

PBIS….

Is an approach for teaching appropriate behaviors and providing the necessary supports to sustain those behaviors

Is a framework to identify needs, develop strategies, and evaluate practice toward success

Addresses the behavior of ALL students

Team

that is re

pre

senta

tive

of th

e sta

ff

Behavioral

expectations

are clearly

defined and taughtContinuum of procedures

for supporting positive and addressing

negative behavior

Continuous

monito

ring

and revi

ew for

effectiv

eness

Data-driven

decision-

making

Essential Featuresof PBIS

Research-based approach

offers a common approach to

discipline

Universal

Targeted

IntensiveContinuum of Support for

ALLProb Sol.

Coop play

Adult rel.

Anger man.

Attend.

Peer interac

Ind. play

Label behavior…not people

EFFECTIVE DATA COLLECTION ANALYSIS, AND REPORTING

CHARACTERISTICS OF EFFECTIVE DATA SYSTEMS• The data are accurate and valid• The data are very easy to collect (1% of staff time)• Data are presented in picture (graph) format• Data are current (no more than 48 hours old)• Data are used for decision-making

• The data must be available when decisions need to be made (weekly?)

• Difference between data needs at a school building versus data needs for a district

• The people who collect the data must see the information used for decision-making

• Data tied to measurable outcomes

THE NEED FOR CONSISTENCY

CREATE A RESPONSE FRAMEWORKAs a team… Create a system to document/report

behavior infractions Set criteria to differentiate classroom

managed vs. office managed behaviorClassroom managed behaviors handled with

teacher chosen consequences Establish a continuum of supports and

consequences Embed systematic decision points to

identify at-risk students

BEHAVIORAL RESPONSE PROCESS

More than 4 Class Reports

Classroom Reports and

Office Referrals

Teacher contacted; classroom

observation; behavior monitored

ODR

?

Teacher contacted; classroom

observation

At next data review

Continue to Monitor

At next data review

ODR

?Remove from

monitor list until 4 more class

reports/or ODR

3+ODR

Yes

Child Study Begin CICO

Consult with teacher; counselor

contactedno

yes

no

yes

At next data review

No

3+ODR

yes

Continue to consultation/ monitor behavior

no

HOW DO WE KEEP DATA COLLECTION CONSISTENT

Provide initial and booster staff training

Keep it simple yet comprehensive

If you collect it, you must report it

Tie data to interventions

Use data to highlight the success stories

SCENARIO 1

Comments on an end of year survey at a middle school revealed that teachers felt there were too many fights and arguments over the course of the school year. In addition, many teachers indicated that they felt they could not adequately cover all of the academic standards since instructional time was often devoted to diffusing conflict and calming the class down before they were ready to learn. Cursory review of discipline data revealed there were 75 incidences of fighting. The principal decided to implement PBIS as a way to address the behavior of the school.

SCENARIO 2

After a monthly review of the PBIS data regarding the number of class reports and office referrals revealed that Joseph had 4 class reports and 1 office referral in the last 4 weeks for disruptive behavior during instructional time. The school psychologist initiated a consultative relationship with his classroom teacher, who felt that his behavior was impacting his academics and was open to suggestions and interventions.

SCENARIO 3

A monthly review of behavior data revealed that Casey had 3 office referrals and 4 class reports in one month. Casey entered the CICO for 5 weeks. A review of her daily report card data showed that she had inconsistently met 2 of her daily behavioral goals, and had consistently met the third one. Despite receiving this intervention during this 5 week period, Casey had 2 office referrals and 5 class reports during this period. Her teacher reported that she was not seeing progress and was continually frustrated with her behavior.

DATA COLLECTION

WHY IS DATA COLLECTION IMPORTANT?

Documents the “Big 5”Who, what, when, where, how many

Useful in discovering trendsAllows for analysis of intervention

and support outcomesReminds you what you are doing

well!

THE IMPORTANCE OF BASELINE DATA

Baseline data helps to:

Identify areas in need of improvement Focus efforts to the areas most in need of

change (prioritize concerns) Determine the amount of time/support needed to

implement strategies Track progress over time

WHAT KINDS OF DATA SHOULD WE COLLECT?

School-wide Systems Discipline data

Office referrals and classroom reports Increase in instructional time

Attendance Tardies and truancy

Academic achievement Benchmarks, grade level assessments, graduation

rate etc.

Climate survey data Informal and formal staff/student surveys

Student Recognition Tickets, gotchas, etc.

WHAT KINDS OF DATA SHOULD WE COLLECT?

Student Level

Discipline data Office referrals and classroom reports

Attendance Tardies and truancy

Academic achievement Progress monitoring, grades

Response to Interventions CICO Data Individualized Behavior Charts

SCENARIO 1

Comments on an end of year survey at a middle school revealed that teachers felt there were too many fights and arguments over the course of the school year. In addition, many teachers indicated that they felt they could not adequately cover all of the academic standards since instructional time was often devoted to diffusing conflict and calming the class down before they were ready to learn. Cursory review of discipline data revealed there were 100 incidences of fighting. The principal decided to implement PBIS as a way to address the behavior of the school.

What data do you need?

SAMPLE DATA COLLECTION FORM

DATA COLLECTION FORMS

INTERVENTION DATA COLLECTION

INTERVENTION DATA COLLECTION

DATA ANALYSISWhat do you do with all your data?

USING DATA EFFICIENTLY

Review the data at every team meeting

Maintain consistency when entering data

Keep data entry current (weekly)

Rotate responsibility of data entry

Track both classroom reports and office referrals

Examine, don’t just enter and admire the data

Track the data across time

SCHOOL LEVEL DATA (SCENARIO 1)

Cafe-teria

Hallway Gym Outside0

10

20

30

40

50

60Pre-PBIS

Pre-PBIS

SCHOOL LEVEL DATA-WHERE

Cafe-teria

Hallway Gym Outside0

2

4

6

8

10

12

3 months Post PBIS

3 months Post PBIS

SCHOOL LEVEL DATA-WHEN

8:30-99-9:30

9:30-1010-10:3010:30-1111-11:3011:30-1212-12:3012:30-11-1:301:30-22-2:302:30-3

0 1 2 3 4 5 6 7 8

Time of Day

Time of Day

Lunch

SCHOOL LEVEL DATA- BEHAVIORS

0

10

20

30

40

September-January January-February

SCHOOL LEVEL DATA- TRENDS

Sept

embe

r

Octob

er

Novem

ber

Decem

ber

Janu

ary

Febr

uary

Mar

chAp

rilMay

June

0

5

10

15

20

25

2008-20092009-20102010-2011

UNIVERSAL SCHOOL-WIDE STRATEGIES

Consistent behavioral feedback, re-teaching, and modeling

Daily reminders of anchors and expectations Behavior of the week Mentoring program Watch Dog Dads Muffins with Mom Parent Workshops Consistent Language

UNIVERSAL TIER 1 INTERVENTION IDEAS

Cafeteria

•Table Tents•Music•Inclusion of Hostesses•Increased Adults

Library

•Expectations posted at entrances and exists•Expectations in book jackets

Hallway

•Street Signs•Stop signs•Adults in high traffic areas

Bus

•Include bus drivers in PBIS training•Post expectations by exits•Expectations on the bus

IDENTIFYING STUDENTS IN NEED

Discipline data Office referrals and classroom reports

Attendance Tardies and truancy

Academic achievement Progress monitoring, grades

Response to Interventions CICO Data Individualized Behavior Charts

Come up with a system that works for your school.

IDENTIFYING STUDENTS IN NEED

Office Referrals Only

IDENTIFYING STUDENTS IN NEED

Office Referrals and Class Reports

USING DATA AT THE STUDENT LEVEL

After a monthly review of the PBIS data regarding the number of class reports and office referrals revealed that Joseph had 6 class reports and 1 office referral in the last 4 weeks for disruptive behavior during instructional time. The school psychologist initiated a consultative relationship with his classroom teacher, who felt that his behavior was impacting his academics and was open to suggestions and interventions.

JOSEPH P. STUDENT

5/6 class reports occurred during language arts

ODR-altercation with a peer for making fun of him being “stupid”

Observations: he was frustrated and asking for help in inappropriate ways

Did not pass PALS Below grade level on

DRA; receiving some support from the reading specialist

Poor sight words Slow progress in writing

Behavioral Data Academic Data

Interventions: Initiate Child Study Team for behavior concerns that appear to be secondary to academic concerns; peer buddy, increased reading support; 4 week sight word intervention with progress monitoring.

JOSEPH P. STUDENT-FOLLOW UP

2 new class reports; during reading and writing

No new ODR Teacher reports the peer

buddy seems to be helping, but is not solving the problem.

Progressed 1 level on DRA

Increase in sight words recognition

Slow progress in writing

Behavioral Data Academic Data

Interventions: Continue with the peer buddy and increased support in reading. Add 30 minutes a week of paired reading time with assistant principal.

MONITORING INTERVENTION EFFECTIVENESS

A monthly review of behavior data revealed that Casey had 3 office referrals and 4 class reports in one month. She has a total of 7 class reports and three office referrals. Casey entered the CICO for 4 weeks. A review of her daily report card data showed that she had inconsistently met 2 of her daily behavioral goals, and had consistently met the third one. Despite receiving this intervention Casey had 2 office referrals and 5 class reports during this period. Her teacher reported that she was not seeing progress and was continually frustrated with her behavior.

MONITORING INTERVENTION EFFECTIVENESS- CICO

Set entrance and exit criteria

Set decision points

Monitor the data

CASEY P. DISRUPTION

31-Ja

n 3 7 9 11 15 17 23 28 2 40

1

2

3

4

CICO

Goal 1Goal 2Goal 3

Tota

l P

oin

ts

CASEY P. DISRUPTION

4060

7 6 7 8

4061

3 12 13 14 15

4062

0 19 20 21 220

1

2

3

4

Goal 2Goal 3

SCHOOL LEVEL DATA

Cafe-teria

Hallway Gym Outside0

2

4

6

8

10

12

3 months Post PBIS3 months Post In-tervention

USING FIDELITY ASSESSMENTS TO MONITOR PROGRESS

School-wide Evaluation Tool (SET) Pre and Post 80% or more Interview format with administrators, students,

teachers, and team members

Benchmarks of Quality (BOQ) 70% or more Completed by the PBIS team

Team Implementation Checklist (TIC)

REPORTING YOUR DATA

MAP YOUR DATA

•Maps are a great way to localize your data and make it seem relevant

•This is also helpful for showing ‘hot spots’

Student Office Discipline Referrals

Actual dataTheory

1%

10-15%

1.4%

80-90% 97.6%

Intensive, Individual InterventionsIndividual StudentsAssessment-basedIntense, durable procedures

Targeted Group InterventionsSome Students (at-risk)High EfficiencyRapid Response

Universal InterventionsAll SettingsAll Students, Preventive, proactive

1-5%

Students with 3 or more referrals

Students with 2 referrals

Students with 1 or less referrals

Your School PBIS as of May 2010

SELF EVALUATION At the end of the year, ask:

What worked/what didn’t work? What areas will continue to be targeted? Will we

add new targeted areas? How will we keep the momentum going into next

year especially during specific times of the year? Compared to the baseline data, were our efforts

reflected in positive change?

QUESTIONS

Kelly.vaillancourt@lcps.org

Kim.comrie@lcps.org