scaling up sw pbs implementation: different journeys to the same destination
DESCRIPTION
Scaling Up SW PBS Implementation: Different Journeys to the Same Destination. Heather Peshak George, Ph.D. APBS Conference, St. Louis, MO: March 26, 2010. Florida’s PBS Project . Our Mission - PowerPoint PPT PresentationTRANSCRIPT
Scaling Up SW PBS Implementation:
Different Journeys to the Same Destination
Heather Peshak George, Ph.D.APBS Conference, St. Louis, MO: March 26, 2010
2
Florida’s PBS Project
Our Mission• Increasing the capacity of Florida’s school districts to
address problem behavior using positive behavior support through a Response to Intervention framework.
What we do…• Provide training and technical assistance to districts across
the state in the development and implementation of positive behavior supports at the school-wide, targeted group, classroom and individual student levels
3
Exploration Phase• Is there a need for change?• What current practices and initiatives exist
that are facilitators or barriers?• What is innovation and does it address
our problem?• How do we plan for implementation?• Is the team ready to begin installation of
innovation?
4
Installation Phase• What individuals have authority to
reallocate resources/facilitate implementation and connect with state improvement plan?
• Who will guide implementation?• What does implementation of the
innovation involve?
5
Initial Implementation Phase
• Who are initial implementers?• How do we begin implementation?• How do we monitor fidelity and outcomes?
6
Full Implementation Phase
• How do we build local training/coaching capacity?
• What resources do we need to add more schools?
• How do we sustain fidelity and positive outcomes over time?
7
Innovation and Sustainability Phase
• How do we braid innovation with new/existing initiatives?
• How do we become more efficient and effective?
• How do we share what we have learned?
9
Funding• IDEA Part B Discretionary Grant
• Around 1997: $33,000/year
• Current: Almost $1.8 million/year
10
VisibilityBefore:• Present at every state conference
• Special education• Expanded across disciplines
• ABA, Safe Schools, RtI, Reading, School Psychologists, Superintendent, Evaluation
Now:• Present when requested or “invited”
11
Click Here…
12
http://flpbs.fmhi.usf.edu/resources_newsletter.asp
Newsletter:
Read directly from the website or download to print.
Use to find useful tips, data, important dates, etc.
13
Political Support and Policy
• Expanded funding of project• State RtI Implementation Plan• Active member on the FL DOE’s State
Transformation Team for RtI• Title of Project to include RtI:B• RtI• Bullying & Harassment Policy• Zero Tolerance
State Infrastructure• State Management Team
– Role: Provide leadership and facilitate policy-level changes to support implementation of effective educational practice
• State Transformation Team– Role: Analyze progress toward statewide efforts, recommend actions for
improvement, and support District Based Leadership Teams DBLT to build the capacity of districts to implement evidence-based practices and to establish integrated RtI academic and behavior systems in each school
• District Based Leadership Team– Role: Provide leadership, advisement, and training at the district level and
assist schools in their implementation efforts• School Based Leadership Team
– Role: Develop a school implementation plan. The school based team will become “trainers” and “coaches” for the school staff and will be responsible for school wide implementation
• Advisory Group– Role: Provide on-going stakeholder input– pp. 11-13 of the Statewide RtI Implementation Plan
District ResponsibilitiesBased on self-assessment results, and in conjunction
with the student progression plan and K-12 Comprehensive Reading Plan, develop an RtI implementation plan organized around building consensus, infrastructure, and implementation. Plans should also address:– How current resources will be used to implement
RtI and identify additional resources needed– How stakeholders will be educated– How stakeholders will be involvedp. 8 of the Statewide RtI Implementation Plan
Alignment with Existing InitiativesThe basic elements of RtI are required by
NCLB and IDEA, therefore, they are included in all broad-based initiatives for schools striving to meet (AYP), such as:– K-12 Reading Plan and Reading First– Florida’s Continuous Improvement Model (FCIM)– Florida’s Positive Behavior Supports (PBS)– PS/RtI Pilot Project– Bright Beginningsp. 1 of the Statewide RtI Implementation Plan
Tiered Model of School Supports & the Problem-Solving Process
ACADEMIC and BEHAVIOR SYSTEMS
Tier 3: Intensive, Individualized Interventions & Supports
The most intense (increased time, narrowed focus, reduced group size) instruction and intervention based upon individual student need provided in addition to and aligned with Tier 1 & 2 academic
and behavior instruction and supports.
Tier 2: Targeted, Supplemental Interventions & Supports.
More targeted instruction/intervention and supplemental support in addition to and aligned with the core academic and behavior curriculum.
Tier 1: Core, Universal Instruction & Supports.
General academic and behavior instruction and support provided to all students in all settings.
FL RtI State Transformation Team, Dec. 2009)
18
19
Training• School-Wide/Initial (Tier 1)• Booster/Retraining (Tier 1)• Classroom (Tiers 1 & 2) – online only• Targeted Group (Tier 2)• Individual Student (Tier 3) • Team Leader• Principal/Administrator• SWIS• Coaches’
Italics indicates both on-site & online modules available
20
Tier 1 Training Sequence
Day 1
Intro to RtI:BTeamingDeveloping Expectations & RulesDeveloping a System for Teaching
Day 3
EvaluationClassroom PBSFacilitated work time
Day 2
Developing a Reward SystemEffective Discipline Procedures•Definitions •Forms•Process•Responses
Implementing PBS
# Florida Schools' Initial PBS Training
12
61
8781
53
77
119
219
0
50
100
150
200
250
2001-2002 2002-2003 2003-2004 2004-2005 2005-2006 2006-2007 2007-2008 2008-2009
Year
Tota
l Sch
ools
Tra
ined
22
Number of Schools Trained
(As of March, 15 2010)6 Pre-K schools387 Elementary schools167 Middle schools82 High schools56 Alt/Center schools50 Other (e.g. K-8)
748 TOTAL SCHOOLS
Coaching & Behavioral Expertise
RtI:B Support StructureProject Contact
District Coordinator(District Leadership Team)
COACH
School-based Team Leader
PBS Team Members
25
Factors to Consider in Developing Comprehensive
Evaluation Systems1) Systems Preparation
– Readiness activities2) Service Provision
– Training and technical assistance3) Identification and Assessment of Behavior Problems
– Possible data sources4) Evaluation Process
– Timelines, data systems5) Evaluation Data (Across all three Tiers)
– Implementation Fidelity, Impact on Students, Attrition, Client Satisfaction
6) Products and Dissemination– Reports, materials, presentations, etc.
(modified from Childs, Kincaid & George, in press)
26
Florida’s Evaluation Model
TrainingOn-going technical assistance
FLPBS↓
Districts↓
Coaches↓
Schools
End-Year
ImpactOutcome data (ODR, ISS, OSS)FL Comprehensive Assessment TestBenchmarks of QualitySchool Demographic DataPBS WalkthroughDaily Progress ReportsBehavior Rating ScalesClimate Surveys
Implementation FidelityPBS Implementation Checklist (PIC)Benchmarks of Quality (BoQ)Benchmarks for Advanced Tiers (BAT)School Demographic DataSchool-wide Implementation FactorsTier 3 plan fidelity checklistBEP Fidelity checklist
Project ImpactAttrition Survey/Attrition RatesDistrict Action Plans
Client SatisfactionSchool-Wide Implementation FactorsDistrict Coordinator’s SurveyTraining Evaluations
Annual Reports Revisions to
training and technical assistance process
National, State, district, school dissemination activities
Website On-line training
modules
Systems Preparation
Service Provision
Evaluation Process
Evaluation Data
Products and Dissemination
MidYear
I
MidYearII
Identification/Assessment
Discipline Records ESE Referrals Surveys Walkthroughs PIC Classroom
Assessment Tool Student rank/rating Teacher requests Lack of response BAT Behavior Rating
Scale Daily Progress
Report Charts
• District Action Plan
• District Readiness
Checklist• SchoolReadinessChecklist• New School Profile (includes ODR, ISS, OSS)
27
(1) Systems Preparation• Tier 1
• District Readiness Checklist• District Action Plan
– District baseline and goals• School Readiness Checklist
• Baseline data• Tier 2
• School Readiness• Implementation of Tier 1• School Infrastructure
• Tier 3– District Action Plan
• Systems change• Evaluation of products and processes• Establish vision and goals
28
29
(2) Service ProvisionTraining and ongoing technical assistance
• Training• Tier 1- District and multi-district
on-site• Tier 2 –District, multi-district, web-
based• Tier 3- Post assessment, goal
setting, systems/process established
(3) Identification & Assessment
• Tier 1– Discipline records, attendance, ESE referrals, baseline BoQ, action
plans, climate surveys, coaches surveys, walkthrough (mini SET), PBS Implementation Checklist (PIC)
• Classroom– Discipline records, teacher requests, student rankings/ratings, ESE
referrals, observations, Classroom Assessment Tool• Tier 2
– Discipline records, teacher requests, student rankings/ratings (SSBD, TRF, etc…), lack of response to Tier 1, Daily Progress Reports, PBS Implementation Checklist (PIC), Benchmarks for Advanced Tiers (BAT)
• Tier 3– Above items, lack of response to Tier 2, Behavior Rating Scale,
observation data, intervention fidelity checklist, PBS Implementation Checklist (PIC), Benchmarks for Advanced Tiers (BAT)
31
Referrals per Student
0
10
20
Num
ber o
f Ref
erra
ls p
er S
tude
nt
Students
Office Discipline Referrals
Student Initials Grade/Period I or E (Step 2) _________ ________ _________________ ________ _________________ ________ ________
Academic Personally Taught Personally Given Externalizing Concerns Expectations SW Reward 1. __________________ ____ Yes ____ Yes ____ Yes
2. __________________ ____ Yes ____ Yes ____ Yes
3. __________________ ____ Yes ____ Yes ____ Yes
• Rank top 3 externalizing and top 3 internalizing students
• Check “YES” if personally taught expectations to the student
• Check “YES” if personally given a SW-PBS reward to student
Teacher Nomination
Tier 2 Progress Monitoring
Behavior Rating ScaleBehavior Date
Hitting 8 or more6-7 times4-5 times2-3 times0-1 times
54321
54321
54321
54321
54321
54321
54321
54321
54321
54321
54321
54321
54321
54321
54321
54321
54321
54321
Profanity 16 or more times12-15 times
8-11 times4-7 times0-3 times
54321
54321
54321
54321
54321
54321
54321
54321
54321
54321
54321
54321
54321
54321
54321
54321
54321
54321
Requesting Attention/Assistance
55% or more40-55%25-40%10-25%0-10%
54321
54321
54321
54321
54321
54321
54321
54321
54321
54321
54321
54321
54321
54321
54321
54321
54321
54321
(4) Evaluation Process• Timelines for State Evaluation
– Baseline (due date varies)– Mid Year I – due 10/31
• School Profile• PBS Implementation Checklist (PIC) (Tiers 1-3)
– Mid Year II – due 2/28• PBS Implementation Checklist (PIC) (Tiers 1-3)
– End Year – due 6/15• Benchmarks of Quality (BoQ) (Tier 1)• Benchmarks for Advanced Tiers (BAT) (Tiers 2-3)• Outcome Data Summary • School-wide Implementation Factors (SWIF)
• Web-based Data Entry and Reporting• PBSES• Statewide Student Database – Academic/Behavior
36
(5) Evaluation Dataa) Implementation Fidelity
– PIC (All Tiers)– BoQ (Tier 1)– BAT (Tiers 2-3)– SWIF (All Tiers)– Walkthrough (Tier 1)– Tier 2 & 3 intervention
specific fidelity measuresc) Attrition
– Attrition Survey (All)
b) Impact on Students– Outcome data (ODR, ISS,
OSS) – Academic achievement– School Demographic
Data (e.g. ethnicity)– Attendance– DPR charting– Behavior Rating Scale
d) Client Satisfaction– SWIF– Climate surveys– Social validation
37
(a) Implementation Fidelity
1. Are schools trained in SWPBS implementing with fidelity? Across years? Across school types?– BoQ, BAT, School Demographic Data
2. What factors are related to implementing with fidelity? – SWIF survey, BoQ, BAT
38
Tier 1 Critical Element Implementation Level chart
39
PBS Implementation Level chart
40
High Implementing Florida PBS Schools(Scoring 70 or Above on BoQ)
57 5660
7774
0
10
20
30
40
50
60
70
80
90
100
School Year
Perc
ent o
f Sch
ools
Sco
ring
70+
2004-2005 2005-2006 2006-2007 2007-2008 2008-2009
Average Score
66
Average Score
66
Average Score
72
Average Score
77
Average Score
76
BoQ Totals by School Type Across Years
Average BoQ Total Score by School Type
6964
70 6971
66 66
7674 73
63
8178
74 758080
74
67
80
0
10
20
30
40
50
60
70
80
90
Elementary Middle High Alt/Center
School Type
Ave
rage
BoQ
Tot
al S
core
2004-2005 2005-2006 2006-2007 2007-2008 2008-2009
38Sc
hool
s
77Sc
hool
s10
4Sc
hool
s
27Sc
hool
s
54Sc
hool
s
55Sc
hool
s
6Sc
hool
s
14Sc
hool
s
17Sc
hool
s
15Sc
hool
s
26Sc
hool
s
28Sc
hool
s
149
Scho
ols
221
Scho
ols
59Sc
hool
s
93Sc
hool
s
17Sc
hool
s
25Sc
hool
s
32Sc
hool
s
37Sc
hool
s
Brief Walk-through
Benchmarks for Advanced TiersScore Summary for BAT Sections
0
10
20
30
40
50
60
70
80
90
100
Tier
1
T2&
3:C
omm
it
T2&
3:St
u Id
ent
T2&
3:M
onito
r
T2:S
uppo
rt
T2:M
ain
Impl
T2:M
ain
Mon
itor
T2:2
nd Im
pl
T2:2
nd M
onito
r
T2:3
rd Im
pl
T2:3
rd M
onito
r
T2:4
th Im
pl
T2:4
th M
onito
r
T3:S
uppo
rt
T3:A
sses
s
T3:M
onito
r
Tota
lBAT Sections
Perc
ent P
ossi
ble
Scor
e
44
(b) Impact on Student Behavior
1. Do schools implementing SWPBS decrease ODRs, days of ISS, and days of OSS?– ODRs, ISS, OSS
2. Do schools implementing SWPBS realize an increase in academic achievement?– FCAT scores
3. Is there a difference in outcomes across school types?– ODRs, ISS, OSS, FCAT scores, school demographic data
4. Do schools implementing with high fidelity have greater outcomes implementers with low fidelity?– BoQ, ODRs, ISS, OSS
5. Do teams that work well together have greater outcomes than those that don’t work as well together?– Team Process Evaluation, ODRs, ISS, OSS
Percentage Change in Discipline OutcomeBetween Baseline and Year 1
-24%
-18%
-8%
-30%
-25%
-20%
-15%
-10%
-5%
0%
ODR ISS OSS
Perc
enta
ge C
hang
e 110
Scho
ols
89Sc
hool
s
108
Scho
ols
higher implementing schools reported 37% fewer ODRs per 100 students
162
114115
94
105100
107
85
115
73
0
20
40
60
80
100
120
140
160
180
Low (BoQ <70) High (BoQ >=70)
Implementation Level
Ave
rage
OD
Rs
per 1
00 S
tude
nts 2004-2005 2005-2006 2006-2007 2007-2008 2008-2009
35Sc
hool
s
77Sc
hool
s
83Sc
hool
s
39Sc
hool
s
76Sc
hool
s
107
Scho
ols
67Sc
hool
s
162
Scho
ols
ODR Rates by Implementation Level Across School Years
102
Scho
ols
256
Scho
ols
ISS Rates by Implementation Level Across School Years
52
29
45
28
43
32
54
25
39
26
0
10
20
30
40
50
60
Low (BoQ <70) High (BoQ >=70)
Implementation Level
Ave
rage
Day
s IS
S pe
r 100
Stu
dent
s
2004-2005 2005-2006 2006-2007 2007-2008 2008-2009
35Sc
hool
s
77Sc
hool
s
83Sc
hool
s
39Sc
hool
s
76Sc
hool
s
107
Scho
ols
67Sc
hool
s
162
Scho
ols
67Sc
hool
s
162
Scho
ols
overall average difference of 40% across the five school years
73
43
61
34
56
37
51
28
50
25
0
10
20
30
40
50
60
70
80
Low (BoQ <70) High (BoQ >=70)
Implementation Level
Ave
rage
Day
s O
SS p
er 1
00 S
tude
nts
2004-2005 2005-2006 2006-2007 2007-2008 2008-200935
Scho
ols
77Sc
hool
s
83Sc
hool
s
39Sc
hool
s
76Sc
hool
s
107
Scho
ols
67Sc
hool
s
162
Scho
ols
OSS Rates by Implementation LevelAcross School Years
67Sc
hool
s
162
Scho
ols
overall average difference of 43% across the five school years
49
(c) Attrition1. Why do schools discontinue
implementation of SWPBS?– Attrition Survey
50
(d) Consumer Satisfaction
1. Are our consumers satisfied with the training, technical assistance, products and support received?– SWIF survey – District Coordinators survey– Training evaluation– Climate surveys
School-Wide Implementation Factors (SWIF)
Higher Implementing Lower Implementing
(70+ on BoQ) (-70 on BoQ)90% + respondents from high implementing schools identified these factors as Helpful:
80%+ respondents from low implementing schools identified these factors as Helpful:
Factors MOST
Helpful to Implementation of
SWPBS
Expectations and rules clearly definedAdministrator committed to PBS, willing to teach and model PBS, willing to reward studentsRepresentative and committed PBS TeamReward system worksPBS Coach’s guidance with processStudents responses to rewards and activities
Expectations and rules clearly definedAdministrator willing to reward studentsRepresentative PBS Team
25%+ respondents from high implementing schools identified these factors as Problematic:
50%+ respondents from low implementing schools identified these factors as Problematic:
FactorsMOST
Problematic to Implementation of
SWPBS
Adequate fundingTeam recognizes faculty participationStaff stability from year to yearStudent stability from year to year
Staff time for PBSStaff belief about effectiveness of PBSStaff philosophyStaff consistency in teachingStaff consistency in discipline procedures
53
(6) Products and Dissemination
• Annual Reports• Revisions to Training• Revisions to Technical Assistance process• Dissemination activities:
– National, state, district, school levels• Revisions to Website• On-line training modules
Model SchoolsFlorida's Model Schools
17 39 5691
13395
184221
280
410
050
100150200250300350400450
2004-2005 2005-2006 2006-2007 2007-2008 2008-2009
Number of Model Schools Number of Active Schools
32% of eligible schools achieved model school status in 2008-2009. Of the 133 schools = 27 Gold, 72 Silver, 34 Bronze
55
What’s New• Train-the-Trainer Evaluation Process• Expansion of Online Training Modules• State-wide Behavioral Database• Possible State RtI Center• High Schools• Further Research• ???
56
Current Research• PBS Implementation Checklist (PIC)
• Benchmarks of Quality (Revised)
• Benchmarks for Advanced Tiers (BAT) • Walkthrough• Tier 3
• Model Demonstrations at Tier 2/3
57
Challenge
58
Contact
Heather Peshak George, Ph.D.• Phone: (813) 974-6440• Fax: (813) 974-6115• Email: [email protected]• State Website: http://flpbs.fmhi.usf.edu• National Website: www.pbis.org• Association on PBS: www.apbs.org