the vision - florida problem solving & response to ... ec ialdu ton“ v s” why...
TRANSCRIPT
NASP 2008
Problem-Solving/RtI: Implementing anIntervention Data Collection System with
Integrity
NASP Annual ConventionNew Orleans, LA
George M. BatscheProfessor and Co-DirectorInstitute for School Reform
Florida Problem-Solving RtI Statewide ProjectJose’ M. Castillo
PS/RtI Project EvaluatorRtI Coach - Pasco County School District
University of South Florida
The Vision• 95% of students at “proficient” level
• Students possess social and emotional behaviors that support“active” learning
• A “unified” system of educational services– One “ED”
• Student Support Services perceived as a necessary componentfor successful schooling
Components of theOrganizational Delivery System
• Academic and Behavior Instruction
• Learning Supports
• Leadership
Response to Intervention
• RtI is the practice of (1) providing high-qualityinstruction/intervention matched to studentneeds and (2) using learning rate over timeand level of performance to (3) makeimportant educational decisions.
(Batsche, et al., 2005)
• Problem-solving is the process that is used todevelop effective instruction/interventions.
NASP 2008
Stages of ImplementingProblem-Solving/RtI
• Consensus– Belief is shared– Vision is agreed upon– Implementation requirements understood
• Infrastructure Development– Problem-Solving Process– Data System– Policies/Procedures– Training– Tier I and II intervention systems
• E.g., K-3 Academic Support Plan– Technology support– Decision-making criteria established
• Implementation
The Process of Systems Change
• Until, and unless, Consensus (understanding the need and trusting inthe support) is reached no support will exist to establish theInfrastructure. Until, and unless, the Infrastructure is in placeImplementation will not take place.
• A fatal flaw is to attempt Implementation without Consensus andInfrastructure
• Leadership must come both from the Principal and from the educatorsin the building.
Consensus Development:Methods
• Knowledge
• Data
Consensus Development:Knowledge
• Rationale for PS/RtI– Impact on students– Reduces disproportionality– Equity in Educaiton
• Research– NASDSE Book
• Law and Regulations
NASP 2008
Building Consensus
• Beliefs• Understanding the
“Need”• Skills and/or Support
Essential Beliefs• Student performance is
influenced most by the quality ofthe interventions we deliver andhow well we deliver them- notpreconceived notions aboutchild characteristics
• Decisions are best made withdata
• Our expectations for studentperformance should bedependent on a student’sresponse to intervention, not onthe basis of a “score” that“predicts” what they are“capable” of doing.
How Do We Know If This is aGeneral Education Initiative?
• Priority of superintendent and schoolboard– District Leadership Team– Strategic Plan
• Focus is on effectiveness of Tier 1 fordisaggregated groups– Unit of Analysis is the BUILDING
How Do We Know If This is aGeneral Education Initiative?
• Principal Led– Regular data analysis– Data Days– Team focuses in improving impact of core
instruction• Prevention and Early Intervention
– Screening and early intervention withKindergarten students
NASP 2008
Contextual Issues Affecting The Problem-Solving Process in General and
Special Education
• IDEA Re-Authorization– Focus on academic outcomes– General education as baseline metric– Labeling as a “last resort”– Increasing general education options– Pooling building-based resources– Flexible funding patterns– RtI Introduced as option for LD eligibility
• ESEA Legislation-No Child Left Behind• National Emphasis on Reading• Evidence-based Interventions
Is It All About Reading?Maybe At First!
• 52% of IDEA $$ go to LD Programs• 70% +/- of special education “activities” (e.g.,
evaluations, staffings, IEPs) related to LD cases• 94% of students in LD because of reading/language
arts• 46% of IDEA $$ go to improve reading• Changes in LD Rules will affect the vast majority of
special education “activities”
Why Problem-Solving ?BIG IDEAS
• AYP and Disaggregated Data (NCLB) move focus of attention tostudent progress, not student labels
• Building principals and superintendents want to know if students areachieving benchmarks, regardless of the students “type”
• Accurate “placements” do not guarantee that students will be exposedto interventions that maximize their rate of progress
• Effective interventions result from good problem-solving, rather thangood “testing”
• Progress monitoring is done best with “authentic” assessment that issensitive to small changes in student academic and social behavior
Big Ideas (con’d)• Interventions must be “evidence based” (IDEA/NCLB)• Response to Intervention(RtI) is the best measure of
problem “severity”• Program eligibility (initial and continued) decisions
are best made based on RtI• Staff training and support (e.g., coaching) improve
intervention skills• “Tiered” implementation improves service efficiency
NASP 2008
New Regulations: LD• The child does not achieve adequately for the• child’s age or to meet State-approved grade-level standards• in one or more of the following areas, when provided with• learning experiences and instruction appropriate for the• child’s age or State-approved grade–level standards:
• The child does not make sufficient progress to• meet age or State-approved grade-level standards in one or• more of the areas identified in paragraph (a)(1) of this• section when using a process based on the child’s response• to scientific, research-based intervention;
New Regulations: LD
• Data that demonstrate that prior to, or as a part of, the referralprocess, the child was provided appropriate instruction inregular education settings, delivered by qualified personnel;and
• (2) Data-based documentation of repeated assessments ofachievement at reasonable intervals, reflecting formalassessment of student progress during instruction, which wasprovided to the child’s parents.
New Regulations: LD
• If the child has participated in a process that• assesses the child’s response to scientific, research-based• intervention-–• (i) The instructional strategies used and the• student-centered data collected; and
• (ii) The documentation that the child’s parents were• notified about--• (A) The State’s policies regarding the amount and• nature of student performance data that would be collected• and the general education services that would be provided;• (B) Strategies for increasing the child’s rate of Learning AND the relationship
between student behavior and academic performance.
What Does the USDOE Say?• “The Department does not believe that an
assessment of psychological or cognitive processingshould be required in determining whether a child hasan SLD. There is no current evidence that suchassessments are necessary or sufficient foridentifying SLD. Further, in many cases, theseassessments have not been used to makeappropriate intervention decisions.” (IDEIA, 2004, p.46651)
NASP 2008
Criteria for SpecialEducation Eligibility
• Significant gap exists between studentand benchmark/peer performance
• The Response to Intervention isinsufficient to predict attainingbenchmark
• Student is not a functionallyindependent learner
• Complete comprehensive evaluation
Implications
• Tier 1 Decision Making– Ensure that the “core curriculum” is
effective– What does “effective” mean?
• 80% of students achieving benchmarks• Disaggregated data
– Race, SES, LEP
– Who determines “effective?”• Principal, Teacher, Data “Person”
Implications• Poor/lack of instruction must be ruled out• Curricular access blocked by any of the following
must be addressed– Attendance– Health– Mobility
• Sufficient exposure to and focus on the curriculummust occur
• Frequent, repeated assessment must be conducted
Consensus Development:Data
• Are you happy with your data?
• Building/Grade Level StudentOutcomes– Disaggregated– AYP
NASP 2008
Infrastructure:Critical Issues
• Policies and Procedures– The Model– Steps in the Model– Decision Rules– Decision Rules and Impact on Intervention
Development• Expectation for Tier Functions/Integration• Data Collection and Interpretation• Intervention Development• Intervention Integrity and Documentation
Infrastructure:Policies and Procedures
• Clearly delineate the components of themodel– Triangle– 4-Step Model
• Identify steps/skills required for eachcomponent
• Decision Rules
Problem Solving Process
EvaluateResponse to
Intervention (RtI)
Problem AnalysisValidating ProblemIdent Variables that
Contribute to ProblemDevelop Plan
Define the ProblemDefining Problem/Directly Measuring Behavior
Implement PlanImplement As Intended
Progress MonitorModify as Necessary
NASP 2008
Steps in the Problem-SolvingProcess
1. PROBLEM IDENTIFICATION• Identify replacement behavior• Data- current level of performance• Data- benchmark level(s)• Data- peer performance• Data- GAP analysis
2. PROBLEM ANALYSIS• Develop hypotheses( brainstorming)• Develop predictions/assessment
3. INTERVENTION DEVELOPMENT• Develop interventions in those areas for which data are available
and hypotheses verified• Proximal/Distal• Implementation support
4. Response to Intervention (RtI)• Frequently collected data• Type of Response- good, questionable, poor
Problem-Solving/RtIResource Management
• Public EducationResourceDeployment– Support staff cannot
resource more than20% of the students
– Service vsEffectiveness--BIGISSUE
1-5% 1-5%
5-10% 5-10%
80-90% 80-90%
Students
Academic Behavior
Intervention Framework
• Intensive Interventions– A few
• Supplemental Interventions– Some
• Core/Universal Interventions– All
1-5% 1-5%
5-10% 5-10%
80-90% 80-90%
Students
Academic Behavior
How Does it Fit Together?Standard Treatment Protocol
Addl.Diagnostic
Assessment
InstructionResults
Monitoring
IndividualDiagnostic
IndividualizedIntensive
weekly
All Students at a grade level
ODRsMonthly
Bx Screening
Bench-Mark
Assessment
AnnualTesting
Behavior Academics
None ContinueWithCore
Instruction
GradesClassroom
AssessmentsYearly Assessments
StandardProtocol
SmallGroupDifferen-tiatedBy Skill
2 times/month
Step 1Step 2 Step 3 Step 4
Supplemental
1-5%
5-10%
80-90%
Core
Intensive
NASP 2008
Tier Functions/Integration
• How the Tiers work
• Time aggregation
• Tier integration
How the Tiers Work• Goal: Student is successful with Tier 1 level of support-
academic or behavioral• Greater the tier, greater support and “severity”• Increase level of support (Tier level) until you identify an
intervention that results in a positive response to intervention• Continue until student strengthens response significantly• Systematically reduce support (Lower Tier Level)• Determine the relationship between sustained growth and
sustained support.
Integrating the Tiers
• Tier 1 (Core) instruction present at allthree levels
• Purpose of Tier 2 is to improve successin Tier 1
• Purpose of Tier 3 is to improve successin Tier 2
• Is there a single “intervention” planmade up of different Tier services?
Integrating the Tiers
• 5th grade student reading at the 2nd grade level– Tier 3
• Direct Instruction, Targeted, Narrow Focus (e.g., phonemicawareness, phonics, some fluency)
– Tier 2• Fluency, vocabulary, comprehension, pre-teach for Tier 1
– Tier 1• Focus on comprehension, participation, scripted decoding
• Use core materials for content• Progress monitor both instructional level and grade
placement level skills
NASP 2008
Cascade of Interventions
• Entire staff understands “triangle” and the availableinterventions at each Tier.
• Supplemental and intensive interventions are inaddition to core instruction.
• A student intervention plan is a single document thatis integrated across the tiers.
• Different tiers ensure that outcomes in Tier 1 areimproved
• Tier 1 progress monitoring data are used foreffectiveness determination for all Tiers
Academic Systems Behavioral Systems
1-5%
Tier 3: Comprehensive and IntensiveInterventionsIndividual Students or Small Group (2-3)Reading: Scholastic Program,Reading,Mastery, ALL, Soar to Success, LeapTrack, Fundations
1-5%
Tier 3: Intensive InterventionsIndividual CounselingFBA/BIPTeach, Reinforce, and Prevent (TRP)Assessment-basedIntense, durable procedures
5-10%Tier 2: Strategic InterventionsStudents that don’t respond to the corecurriculumReading: Soar to Success, Leap Frog,CRISS strategies, CCC Lab Math:Extended DayWriting: Small Group, CRISS strategies,and “Just Write Narrative” by K.Robinson
5-10% Tier 2: Targeted Group InterventionsSome students (at-risk)Small Group CounselingParent Training (Behavior & Academic)Bullying Prevention ProgramFBA/BIP Classroom ManagementTechniques, Professional Development Small Group Parent Training ,Data
80-90%Tier 1: Core CurriculumAll studentsReading: Houghton MifflinMath: HarcourtWriting: Six Traits Of WritingLearning Focus Strategies
80-90% Tier 1: Universal InterventionsAll settings, all studentsCommittee, Preventive, proactivestrategiesSchool Wide Rules/ ExpectationsPositive Reinforcement System(Tickets & 200 Club)School Wide ConsequenceSystem School Wide Social SkillsProgram, Data (Discipline,Surveys, etc.) ProfessionalDevelopment (behavior)Classroom ManagementTechniques,Parent Training
Three Tiered Model of School Supports:Anclote Elementary-Pasco County
Students
Decision Rules
• Response to Intervention Rules
• Linking RtI to Intervention Decisions
Decision Rules: What is a “Good”Response to Intervention?
• Positive Response– Gap is closing– Can extrapolate point at which target student will
“come in range” of peers--even if this is long range• Questionable Response
– Rate at which gap is widening slows considerably, butgap is still widening
– Gap stops widening but closure does not occur• Poor Response
– Gap continues to widen with no change in rate.
NASP 2008
Decision Rules: Linking RtI toIntervention Decisions
• Positive, Questionable, Poor Response• Intervention Decision Based on RtI (General Guidelines)
– Positive• Continue intervention until student reaches benchmark (at
least).• Fade intervention to determine if student has acquired
functional independence.– Questionable
• Increase intensity of current intervention for a short period oftime and assess impact. If rate improves, continue. If rate doesnot improve, return to problem solving.
– Poor• Return to problem solving for new intervention
Data Collection and Interpretation:Issues and Models
• Where to data come from• Which data are used across tiers• High school applications• Data analysis and display
Data Coach
• Gathers and Organizes Tier 1 and Tier2 Data
• Supports staff for small group andindividual data
• Provides coaching for datainterpretation
• Facilitates regular data meetings forbuilding and grade levels
Skills Required• Collaborative problem solving
– Interpersonal– Problem solving process
• Data-based decision making– Collection– Management– Display & analysis– Interpretation
• Action planning– Assessment & intervention– Support
NASP 2008
Skill Assessment Examples
• Measure skills– Problem-solving process– Data-based decision making
• Examples– Problem ID - Tier I focus– Hypothesis evaluation– Hypothesis production & validation
NASP 2008
Data For Each Tier - WhereDo They Come From?
• Tier 1: Universal Screening, accountability assessments, grades,classroom assessments, referral patterns, discipline referrals, commonassessments
• Tier 2: Universal Screening - Group Level Diagnostics (maybe),systematic progress monitoring, large-scale assessment data andclassroom assessment, common assessments
• Tier 3: Universal Screenings, Individual Diagnostics, intensive andsystematic progress monitoring, formative assessment, other informalassessments
High School Data
• Skill or Content Assessment• Skill
– Use existing reading/math skillassessments
• Content– Use “Common Assessments
Common Assessments
• Based on State-Approved, ContentArea Standards
• Syllabus expected to reflect thosestandards
• Common assessment given every 3rdweek
• Data aggregated and disaggregated
NASP 2008
Common Assessments:Interpretation
• Mean level of performance of all students– Disaggregated by section– Disaggregated by demographics– Are 80% of students attaining 70% accuracy? If
not, implications for core instruction--ProblemSolving
– Is syllabus implementation on track?• Compare individual student performance to
group data.
Intervention Development
• Tiers 1 and 2
• Critical Components
• Evidence-based
Data Infrastructure: Using ExistingData to Predict Intervention Needs
• Previous referral history predicts future referralhistory
• How do we interpret teacher referrals?• Previous intervention history predicts future
intervention history• How do we use this information to establish an
infrastructure for change?
Data-Driven Infrastructure:Establishing a Building Baseline
• Code referrals (reasons) for past 2-3 years– Identifies problems teachers feel they do not have the
skills/support to handle– Referral pattern reflects skill pattern of the staff, the
resources currently in place and the “history” of whatconstitutes a referral in that building
– Identifies likely referral types for next 2 years– Identifies focus of Professional Development Activities AND
potential Tier II and III interventions– Present data to staff. Reinforces “Need” concept
NASP 2008
Data-Driven Infrastructure:Establishing a Building Baseline
• Assess current “Supplemental Interventions”– Identify all students receiving supplemental
interventions– For those interventions, identify
• Type and Focus (academic, direct instruction, etc)• Duration (minutes/week)• Provider
– Aggregate• Identifies instructional support types in building• This constitutes Tier II and III intervention needs
Implications for High Schools
• Focus is on BOTH skill and content.• Primary problems:
– Students who cannot read well enough to get the content.– Students who do not have the skills to study and maintain
content OR do not have sufficient time and organizationalskills.
• High school difficulties should be identified andpredicted in middle school.
• High school interventions should be a continuation ofmiddle school interventions--not separate.
Data Cannot Be a Barrier
• Data should be:– Collected efficiently– Easy to manage– Displayed graphically– Easily interpretable
Benchmark Data Made Easy
NASP 2008
H
www.swis.org
www.swis.org Intervention Development• Criteria for “Appropriate” and “Effective” Interventions:
– Evidence-based• Type of Problem• Population• Setting• Levels of Support
• Focused on most important needs• Group interventions have priority• Interventions MUST be linked to Tier 1 focus, materials,
performance criteria
NASP 2008
Interventions: Tier 2
• First resource is TIME (AET)– HOW much more time is needed?
• Second resource is curriculum– WHAT does the student need?
• Third resource is personnel– WHO or WHERE will it be provided?
Tier 2: Getting TIME• “Free” time--does not require additional personnel
– Staggering instruction– Differentiating instruction– Cross grade instruction– Skill-based instruction
• Standard Protocol Grouping• Reduced range of “standard” curriculum• After-School• Home-Based
Tier 2: Curriculum
• Standard protocol approach• Focus on essential skills• Most likely, more EXPOSURE and more FOCUS of
core instruction• Linked directly to core instruction materials and
benchmarks• Criterion for effectiveness is 70% of students
receiving Tier 2 will reach benchmarks
Tier 2: Personnel• EVERYONE in the building is a potential resource• Re-conceptualize who does what• Personnel deployed AFTER needs are identified• WHERE matters less and less• REMEMBER, student performance matters more than labels,
locations and staff needs.• A school cannot deliver intensive services to more than 7% of
the population
NASP 2008
Evidence-Based
• Nationally Evidenced– Select to increase probability of success
• Locally Validated
– Local outcome data used to evaluate degree towhich interventions “worked”
– Local outcome data trumps national “evidence.”
Intervention Support andDocumentation
• Intervention Integrity• Intervention Support• Intervention Documentation
Intervention Integrity
• Enhanced through two practices
– Intervention Support System– Intervention lmplementation
Documentation
Intervention Support
• Intervention plans should be developedbased on student need and skills of staff
• All intervention plans should have interventionsupport
• Principals should ensure that interventionplans have intervention support
• Teachers should not be expected toimplement plans for which there is no support
NASP 2008
Critical Components ofIntervention Support
• Support for Intervention Integrity
• Documentation of InterventionImplementation
• Intervention and Eligibility decisions andoutcomes cannot be supported in an RtImodel without these two critical components
Intervention Support
• Pre-meeting– Review data– Review steps to intervention– Determine logistics
• First 2 weeks– 2-3 meetings/week– Review data– Review steps to intervention– Revise, if necessary
Intervention Support• Second Two Weeks
– Meet twice each week
• Following weeks– Meet at least weekly– Review data– Review steps– Discuss Revisions
• Approaching benchmark– Review data– Schedule for intervention fading– Review data
NASP 2008
Program Evaluation
Evaluation of PS/RtIImplementation Should
• Respect the complexity of schools associal systems
• Be organized and systematic• Be used to inform implementers
– Formative data collection necessary• Include multiple methods and
informants
FL PS/RtI Evaluation Model
• IPO model used• Variables included
– Levels– Inputs– Processes– Outcomes– Contextual factors– External factors– Goals & objectives
NASP 2008
Levels• Students
– Receiving Tiers I, II, & III• Educators
– Teachers– Administrators– Coaches– Student and instructional support personnel
• System– District– Building– Grade levels– Classrooms
Inputs (What We Don’t Control)
• Students– Demographics– Previous learning experiences & achievement
• Educators– Roles– Experience– Previous PS/RtI training– Previous beliefs about services
• System– Previous consensus regarding PS/RtI– Previous PS/RtI infrastructure
• Assessments• Interventions• Procedures• Technology
Processes (What We Do)
• Students– Assessment participation (e.g., DIBELS screening)– Instruction/intervention participation
• Educators– Frequency and duration of participation in PS/RtI Project training– Content of Project training in which they participated
• System– Frequency & duration of professional development offered by the
Project– Content of professional development offered– Stakeholders participating in professional development activities– Communication between Project and districts/buildings
Implementation IntegrityChecklists
• Implementation integrity measures developed• Measure
– Steps of problem solving– Focus on Tiers I, II, & III
• Data come from:– Permanent products (e.g., meeting notes, reports)– Problem Solving Team meetings
NASP 2008
Outcomes (What We Hope toImpact)
• Educators– Consensus regarding PS/RtI
• Beliefs• Satisfaction
– PS/RtI Skills– PS/RtI Practices
Outcomes cont.
• System– PS/RtI Infrastructure
• Assessments• Interventions• Procedures• Technology• Costs
– PS/RtI Implementation
NASP 2008
Outcomes cont.• Students
– Academic achievement– Behavioral outcomes
• Systemic– Discipline referrals– Referrals for problem solving– Referrals for SPED evaluations– SPED placements
NASP 2008
Reading Instruction - Tier IGrade Level
Anclote NWF-3
Low Risk
59%
Moderate
Risk
21%
High Risk
20%
Anclote NWF-4
Low Risk
70%
Moderate
Risk
21%
High Risk
9%
Systemic Outcomes - OfficeDiscipline Referrals
10
30
38
3031
24
30
54
26
18
8
6
1716 16
26
0
10
20
30
40
50
60
Au
gu
st
Sep
tem
ber
Octo
ber
Novem
ber
Decem
ber
Jan
uary
Feb
ruary
Ma
rch
Ap
ril
May
Month
Nu
mb
er
of
OD
Rs
2005-2006 2006-2007
Other Variables to Keep in Mind
• Contextual factors– Leadership– School climate– Stakeholder buy-in
• External factors– Legislation– Regulations– Policy
School Goals & Objectives• Content Area Targets
– Reading– Math– Behavior
• Majority focusing on reading• Some selected math and/or behavior as well• Grade levels targeted varied
– Some chose K or K-1– Some chose K-5