how to do classwide intervention within rti amanda vanderheyden education research and consulting,...

How To Do Classwide Intervention within RTI Amanda VanDerHeyden Education Research and Consulting, Inc.

Objectives Today Overview of RTI, RTI decision making, and expected outcomes Specific How-To for Classwide Math Intervention Implementing intervention for sustenance and system change

Disparities in Achievement Substantially lower level of performance in Reading at first and second grades for African-American students relative to their Caucasian peers (approximately 20 wc/min) Slower growth rate (approximately half) at both grade levels Differences not observed in Math

In Low-Achieving Classrooms (more than 50% of class scored in the frustrational range on probes) STEEPTeacher Referral Sensitivity.75.55 Specificity.88.68 Positive Predictive Power.69.35 Negative Predictive Power.91.82 VanDerHeyden & Witt, 2005

In Average to High-Achieving Classrooms (less than 20% of class scored in frustrational range on probes) STEEPTeacher Referral Sensitivity.670 Specificity1.0.67 Positive Predictive Power 1.00 Negative Predictive Power.97.95 VanDerHeyden & Witt, 2005

% Accurate ID by Race MinorityCaucasian STEEP9086 Teacher Referral 7861 Use of RTI with STEEP approximated base rate by race and gender AA students showed a disproportionate RTI (.50 versus.07) VanDerHeyden & Witt, 2005

Research Finding VanDerHeyden, Broussard, et al. (2004). Prior to single instructional session, children receiving special ed services were significantly lower performing on math probes. Following single instructional session, no significant difference was observed.

Mixed Mult/Div/Fractions Probe Classroom F

Sore Thumb Test

Response to Classwide Intervention Student A

Individual Math Intervention Cant Do Problem

Successful Math Intervention BaselineIntervention Novel, grade-level probe

Why do Classwide Intervention Efficiency Accuracy Efficacy

STEEP Model Screening to Enhance Educational Progress

Tier 1: Screening Screening Math Screening 2 minutes. Scored for Digits Correct Writing Screening 3 Minutes. Scored for Words Written Correctly Reading Screening 1 Minute. Scored for Words Read Correctly

Class-wide Screening

Feedback to Teachers

Tier 2: Class-wide Intervention

No Class-wide Problem Detected

Tier 2: Cant Do/Wont Do Assessment Cant Do/Wont Do Individually-administered Materials Academic material that student performed poorly during class assessment. Treasure chest: plastic box filled with tangible items. 3-7 minutes per child

Cant Do/Wont Do Assessment

Decision Rule Following Cant Do/Wont Do Assessment

Tier 3: Individual Intervention

Response to Intervention Before Intervention During Intervention Avg. for his Class Intervention in Reading #Correct Intervention Sessions Each Dot is one Day of Intervention

Before Intervention During Intervention #Correct Avg. for his Class Response to Intervention

Instructional range Frustrational range Vehicle for System Change: System-wide Math Problem Each bar is a students performance

Re-screening Indicates No Systemic Problem Fourth Grade

Rest of Grade at Standard ABCDEFClassroom

Spring 2003 Classroom F F

Teacher moved to lower grade in Fall 2003

Class-wide Intervention Teacher F Mult 0-12 0 20 40 60 80 100 120 10/24/200310/31/2003 11/7/2003 11/14/200311/18/2003 Weeks Digits Correct Two Minutes

Increased Difficulty- Intervention Continues

Mixed Mult/Div/Fractions Probe Classroom F

Growth Obtained aimline actual growth

Effect on High-Stakes Scores VanDerHeyden, in prep

District-wide Implementation Data Vail Unified School District www.vail.k12.az.us Three years, system-wide implementation of STEEP grades 1-8

System Outcomes Referrals reduced greater than half % who qualify from 50% stable baseline over three years to nearly 100% SLD down from 6% of children in district in 2001-2002 (with baseline upward trend) to 3.5% in 2003-2004 school year Corresponding gains on high-stakes tests (VanDerHeyden & Burns, 2005) Intervention successful for about 95 to 98% of children screened VanDerHeyden, Witt, & Gilbertson, 2007

Cost Reduction VanDerHeyden, Witt, & Gilbertson, 2007

Findings Number of Evaluations dramatically reduced 70% at highest referral school Diverse settings, psychologists of diverse backgrounds and no prior experience with CBM or functional academic assessment Percentage qualify increased at 4 of 5 schools Disproportionate representation of males positively affected Number of children placed dramatically reduced VanDerHeyden, Witt, & Gilbertson, 2007

Team Decision-Making Agreement RTI + and Evaluated RTI- and Did Not Evaluate 2003-2004 (3 schools) 100%41% 2004-2005 (5 schools) 100%87% VanDerHeyden, Witt, & Gilbertson, 2007

Team Decision-Making VanDerHeyden, Witt, & Gilbertson, 2007

Fall to Spring Reading Growth VanDerHeyden & Witt, 2005

What Proportion of Ethnicity Represented Before and After Intervention in Risk Category? VanDerHeyden & Witt, 2005

Identification Accuracy High-achieving classrooms (50%) Procedures paired with RTI criterion were more accurate than other commonly used screening devices VanDerHeyden & Witt, 2005

Weighing a cow doesnt make it fatter.

Using Screening Data to Identify Class-wide and System-wide Instructional Problems

Consider The Task Integrity of Administration Reliability of Scoring Use software to organize the data

Mult 0-9 4 th Grade Fall Screening

Mult/Div/Fractions 4 th Grade Winter

Grade-wide Data

3 rd Grade Mult 0-9 Spring

Guided Practice

Fourth Grade Reading Level: Math Skill 1: Math Skill 2:

Questions Is there a classwide problem? Is there a gradewide problem? Whats the most efficient way to deliver intervention?

What Data do you Want for Principal?

Questions Is there a classwide problem? Is there a gradewide problem? Whats the most efficient way to deliver intervention?

Independent Practice

First Grade Reading What do you want to know? Is there a class-wide problem? Is there a grade-wide or systemic problem? Whats the most efficient way to deliver intervention? (whole class, small group, individual) What is the next step for Class 1, 2, 3, 4?

Class 1

Class 2

Class 3

Class 4

Grade-wide Data

Class 4

Screening tells you How is the core instruction working? What problems might exist that could be addressed? Most bang-for-the-buck activity Next most high-yield activity is classwide intervention at Tier 2.

Screening Guidelines Efforts at Tier 1 pay off with fewer children needing individual intervention 3 times per year, single probe Use small team of trained coaches Prepare all needed materials in a packet for each teacher Score and return within 1 week on graph Use data to generate aimlines, can be used to set benchmarks

Digits Correct Two Minutes Weeks 121 Pass the AIMS

Academic SystemsBehavioral Systems 1-5% 5-10% 80-90% Intensive, Individual Interventions Individual Students Assessment-based High Intensity Of longer duration Intensive, Individual Interventions Individual Students Assessment-based Intense, durable procedures Targeted Group Interventions Some students (at-risk) High efficiency Rapid response Targeted Group Interventions Some students (at-risk) High efficiency Rapid response Universal Interventions All students Preventive, proactive Universal Interventions All settings, all students Preventive, proactive Any Curriculum Area Students Dave Tilly, 2005

Class-wide Intervention Use pair-peered practice (classwide peer tutoring, PALS) Model, Guided Practice, Independent timed practice with delayed error correction, reward contingency

Unsupported means. Team recommends intervention. All materials have to be created from scratch Teachers sent to organize class and train No one may be an expert to help the teacher No or not enough frequent objective data to collect or interpret

With teacher support Consider time, resources, materials Remove skill barriers with classroom training for students classroom coaching for teachers Remove implementation barriers after use new steps follow-up supportive meetings to problem solve. frequent acknowledgment of a teachers efforts

Time consuming Complex not yet fluent Cant Do Poor management No child change Wont do No reinforcement for teacher behaviors No teacher change prevents Address Common Reasons for Resistance Lack of materials prevents Too much work avoidance prevents

Select a Few Good Interventions to Keep it Simple ClasswideIndividual MathFlash card Practice Cover copy compare Cue Cards Highlighted errors ReadingListening PreviewRepeated Readings Error Correction Key Words

Prepare for Training Day Locate probes (e.g., worksheet factory, intervention central, basic skill builders) Identify Graphing Program (excel) Locate or develop scripts (gosbr.net; interventioncentral.org) Develop quick access to materials for teachers Determine integrity monitoring plan Identify common time for intervention Set start date

Set a daily routine. Time, location of materials, process for weekly assessment. Set a date and time for 30-min training Set a date for a later 15-minute first practice time with teacher

Materials needed Computer and software to organize data Student data imported. Clerical person to enter data on- site for tier 1 screen only. Color printer to print graphs + extra color cartridges Probe materials, digital count-down timers Intervention protocols, intervention materials (e.g., flashcard sets, reading materials) Access to copier and some assistance with copying Reinforcers for treasure chest (no more than $500 per school)

Usually the higher-level reader, reads (models) first. Rotating high level readers helps maintain motivation

How-To Classwide Math

Intervention Plan- 15 Min per Day Protocol-based classwide peer tutoring, randomized integrity checks by direct observation Model, Guide Practice, Independent Timed Practice with delayed error correction Group performance contingency Teachers encouraged to Scan papers for high error rates Do 5-min re-teach for those with high-error rates Provide applied practice using mastery-level computational skill

Measurement Plan Weekly probe of Intervention skill Weekly probe of Retention of previously mastered computational skills Monthly probe using GOM approach to monitor progress toward year-end computational goals To this you might add an application measure

Sample Sequence

Intervention Plan Class Median reaches mastery range for skill, next skill is introduced Following promising results at one site in 2002-2003, lead to implementation district- wide grades 1-8 for all children by 2004- 2005.

Acquisition Fluency Generalization Instructional Hierarchy To gain the steepest growth, introduction of new skills should happen here Core Instruction- Not manipulated But fluency building should happen here with an instructional level skill Intervention Focus was here Finally, problem-solving/ application practice should occur here with a mastery level skill Core Instruction- Not Manipulated but could be

Class-wide Math Intervention

..\..\Math Assessment 04_05\Skill Sequence 04-05.doc..\..\Math Assessment 04_05\Skill Sequence 04-05.doc

..\..\Math Assessment 04_05\Data\FINAL 04--05\Cottonwood 5-27.xls..\..\Math Assessment 04_05\Data\FINAL 04--05\Cottonwood 5-27.xls

Provided teacher/student a script that tells. what the student has to do and when what the teacher should do to support student how the student will know how he/she is doing Treatment considerations for integrity issues All steps are clearly needed Includes lots of student response opportunities Disrupts class as little as possible Requires little teacher time ( < 15 min/day) Considers resources to decrease teacher effort Used simple language All the materials are available Rationale

Trainer 1.Observe the teacher using the steps on the intervention script 2. Check off steps used. 3. Prompt the teacher to do any missed step. 4. Problem Solve any noted blockers 5. Continue until accurately implemented without prompts

Lack of practice with feedback Non-specific steps In adequate classroom management Not enough child assistance for bx change Lack of reinforcement For teacher behaviors Low implementation Low frequencies interferes prevents Why verbal and modeling training alone do not work: Inadequate materials prevents No instructions when problems arise

Math Partners Progress Chart Count every digit that is not circled. This is your score! Write your score on your math sheet. Find todays date on this page and write your score on the line. Put a Star on the graph to mark todays score.

Weekly Progress Monitoring Administer classwide math worksheet Target skill once per week Criterion skills periodically to monitor growth Use incentives to maximize performance Apply decision rules

Progress Review Review folders to ensure that intervention was used correctly for at least 4 days that week If this is not the case, conduct another in- class training day. Graph weekly progress monitoring assessment data

Decision making Review data to make decisions: DATA OUTCOME 1: Class median is below mastery range and most students gaining digits correct per week. ACTION: Consider implementing intervention for an additional week and then review progress again.

Decision making DATA OUTCOME 2: Class median is below mastery range and most students are not gaining digits correct per week: ACTION: Check Integrity first and address with training if needed. Consider implementing intervention for an additional week with incentives or easier task and then review progress again.

Decision making DATA OUTCOME 3: If the class median is above mastery range then consider: ACTION: Increasing task difficulty and continuing classwide intervention. ACTION: For students performing in the frustration range, consider Tier 3 assessment and intervention.

Tell Rational Step by step protocol Show Model Do Train students Implement with guided practice Implement independently with support Training Package

> 80% of interventions are not used without support

Troubleshoot Intervention SupportYesNo Was the intervention developed to ensure that it required minimal classroom time and resources and fit within daily classroom routines? Are materials readily available to the teacher? Was a step-by-step coach card provided? Was the teacher shown how to implement the intervention by a coach? Did the coach observe implementation of the intervention to ensure that the teacher could use the intervention correctly and had all needed materials? Was weekly follow-up support provided to the teacher after initial training? Are integrity data graphed to show used correctly? Is an administrator involved?

Results

Tier 1 Screening Indicates Class- wide Problem

Tier 2: Class-wide Intervention Teacher F Mult 0-12 0 20 40 60 80 100 120 10/24/200310/31/2003 11/7/2003 11/14/200311/18/2003 Weeks Digits Correct Two Minutes

Increased Difficulty- Intervention Continues

Contextually-Relevant Comparisons and Use of Trend Data

5 th Grade Math Intervention

Instructional range Frustrational range Pre-post changes to performance detected by CBM Each bar is a students performance

Fourth Grade

Effect on SAT-9 Performance

Effect on CBM Scores

Computation Gains Generalized to High Stakes Test Improvements (Gains within Multiple Baseline shown as pre-post data)

Gains within Multiple Baseline (shown as pre-post data)

Additional Research Questions What level of performance predicted strongest subsequent growth given intervention? What level of performance predicted skill would be retained about 3 months after it was taught? Did mastery of foundation skills shorten the number of trials required to master more complex related skills?

What level of performance predicted strongest subsequent growth given intervention? Across 4 weeks of intervention (4 datapoints) OLS used to estimate slope Children achieving slopes equal to or greater than the 66 th percentile were identified as strong responders Starting fluency (prior to intervention) was identified for the group of strong responders and range was estimated as average starting fluency +/- 1 standard deviation Tested new criterion on second set of scores Burns & VanDerHeyden, 2006

New Range Digits Correc t/Min Reliabilit y (tau) Validity (rho) FrusInstMast 2 nd -3 rd 14-31.35.0822%70%8% 4 th -5 th 24-49.63.5022%67%11% 2 nd - 3 rd 1.772.011.55 4 th - 5 th 1.161.441.25

General Findings Growth rates and trials to criterion varied dramatically across skills Retention probe was strongest predictor of year- end SAT-9 performance Mastery level performance on early skills predicted fewer trials to criterion on future related complex skills Fluency scores higher than mastery predicted retention of skill over time (about +20 dc/min) VanDerHeyden & Burns, 2008; VanDerHeyden & Burns, in submission

Identification Accuracy CBA + RTI CriterionITBSWJ-R STEEP Sensitivity.761.58 Specificity.89.99.77 Positive Predictive Power.59.67.44 Negative Predictive Power.951.86 Teacher Referral Sensitivity.46.33.42 Specificity.69.94.85 Positive Predictive Power.19.17.45 Negative Predictive Power.89.97.83 VanDerHeyden, et al., 2003

Percent Identified at each Tier Identified CBM (Classwide Assessment) 55 (15%) CBM + Reward (Performance/skill Deficit Assessment) 40 (11%) CBM + Reward + Instruction (STEEP +) 22 (6%) Teacher Referral 32 (19%) CIBS-R 64 (18%) DRA 17 (9%) RTI Criterion Assessment 17 (5%) WJ-R 12 ITBS deficit 3 (4%) VanDerHeyden, et al., 2003

Academic SystemsBehavioral Systems 1-5% 5-10% 80-90% Intensive, Individual Interventions Individual Students Assessment-based High Intensity Of longer duration Intensive, Individual Interventions Individual Students Assessment-based Intense, durable procedures Targeted Group Interventions Some students (at-risk) High efficiency Rapid response Targeted Group Interventions Some students (at-risk) High efficiency Rapid response Universal Interventions All students Preventive, proactive Universal Interventions All settings, all students Preventive, proactive Any Curriculum Area Students Dave Tilly, 2005

To work smart, we must ask What is the purpose of our assessment? How do we know it serves our purpose? Is this the cheapest way to do it?

Our Goal Collect the best information in the shortest possible period of time

Tier 3 Assessment Data Instructional level performance Error analysis (high errors, low errors, pattern) Effect of incentives, practice, easier task Verify intervention effect Same implementation support as Tier 2 Instructional-level materials; Criterion-level materials

Tier 3 Implement for 5-15 consecutive sessions with 100% integrity Link to referral decision Weekly graphs to teacher and weekly generalization probes outside of classroom, supply new materials Troubleshoot implementation weekly

Strategy in a Nutshell Identify the goal (DVs) Behavior to increase (fluency, comprehension) Behavior to decrease (errors) Match the strategy to the goal (Daly et al., 1996) Monitor the DVs and the IVs (intervention variables)

Find instructional level (sampling back) Identify the root of the problem (e.g., division is difficult because subtraction is not fluent or multiplication is not fluent; poor decoding skills v. dolce words; production v. accuracy in writing) This is the most important part of the process

Accurate Letter Naming Fluent Letter Naming Association of Letters with phonemes Accurate Letter Sound Production Fluent Letter Sound Production Pronounce beginning word sounds Define the Behaviors/skills

Other Sample Hierarchies Reading 5 th Grade, 2 nd Semester Reading 5 th grade, 1 st semester Reading 4 th grade, 2 nd semester Reading 4 th grade, 1 st semester Reading 3 rd grade, 2 nd semester Math 2 nd Grade Subtraction 0-9 Addition 0-18 Addition 0-9

Identify Reinforcers and Logical Consequences Use a treasure chest Use an activity survey or reinforcer checklist Use incidental teaching strategy Use logical or natural consequences

Measure Baseline Performance and Set Goals

Acquisition Fluency Generalization This is the Instructional Hierarchy To gain the steepest growth, introduction of new skills should happen here But fluency building should happen here with an instructional level skill Finally, problem-solving/ application practice should occur here with a mastery level skill

Functional Assessment What is an effective intervention?

Functional Assessment

BLInterventionPerformance Feedback

Troubleshooting Intervention Effects at Tier 3

Teachers must weigh the following What outcomes does not completing work produce? Escape from assignment, from classroom setting Peer attention Adult attention (even if it is negative). Some students are so motivated to obtain adult attention that it does not matter if the attention is negative or positive. What outcomes does completing work produce? Positive feedback from the teacher Positive attention from peers, status Access to fun activities or reinforcement Avoidance of punitive consequences

Antecedent Variables Task Difficulty, Sequencing of Skills Time actively engaged in learning (AET) Opportunities to respond Other lesson variables (pacing, exemplars) Behaviors interfering with instruction (teacher and child)

Task Difficulty, Sequencing Sample back measuring fluency of performance on basic skills The idea is to identify the weak point in the chain Define the target skill for intervention and the criterion skill (goal)

Academic Engaged Time Impacts opportunities to respond Robust predictor of achievement Average 2 nd grader (Rosenshine) spent less than 1 hour AET per day. Check transitions, classroom management, time allocated for independent practice, active monitoring/scanning

Other Lesson Variables Presentation of materials and Sequencing of Lesson Organized Clear, redundant examples Exemplars sufficient S+ and S- Checking for student understanding Pacing of lesson

Behaviors Interfering with Instruction/Intervention Teacher behaviors Implementation accuracy and consistency Fuchs & Fuchs, 1987; Gresham, 1991; Happe, 1982; Wickstrom, Jones, LaFleur, & Witt, 1998 Teacher understanding/adequately trained Train to fluency criterion (Chandler, Lubeck, & Fowler, 1992) Teacher acceptability of intervention (prospective, ongoing, link to changes) Adequate resources to conduct intervention

Behaviors Interfering with Instruction Child behaviors Disruptive or inattentive behaviors Concurrent options available (access to reinforcing outcomes by not completing intervention) Consider cant do/wont do (although programming for motivation is important anyway)

Consequences Reinforcing consequences (for correct and incorrect performance) Escaping task Extra attention (staying in at recess may be reinforcing) Feedback Frequency Immediacy Accuracy Correct error immediately, have student repeat response correctly, match response to instructional situation & learner (Heubusch & Lloyd, 1998)

Tier 3 Intervention >5% of children screened (total population) IF solid Tier 1 Possibly as low as 2% IF solid Tier 1 and Tier 2 About 1-2% failed RTI; 10% of most at-risk VanDerHeyden et al., 2007

Tier 3 Findings Most interventions for reading Math is next Math is at least two-dimensional--- computational and operational fluency plus application or conceptual understanding Most interventions are not implemented well and thats why they fail Tier 3 interventions are likely to occur on below grade level tasks AND require acquisition-type instruction (discrimination training to establish accurate responding)

Successful Math Intervention

Unsuccessful Math Intervention

Integrity Matters 59% Integ96% Integrity

Integrity Matters

Integrity Untreated integrity problems become student learning deficits, schoolwide learning problems, and false positive decision errors Integ problems affect dose and quality of the treatment (an intervention implemented with fidelity is a functionally different intervention than one implemented inconsistently Integ positively correlated with student learning gains, amount of intervention covered Even veteran sites require monitoring and follow-up

Tips for Effective Implementation

Our Recipe for Intervention Success PREPARATION Identify and Use standard protocols for intervention Develop all needed materials Develop packets or put on a central web site Determine graphing program

Our Recipe for Intervention Success TRAIN Explain Watch the teacher do it with the actual child before you leave Call or meet teacher after first day to problem solve

DATA COLLECTION AND SUPPORT Each week, graph intervention performance and do a generalization check with the child. Graphed feedback to teachers with generalization checks for individual intervention once per week Response-dependent performance feedback to sustain implementation accuracy Monthly CBM to track growth and enhance existing Tier 1 Programs or advise new Tier 1 Data to principal weekly. Summarize effects and integrity of procedures. Our Recipe for Intervention Success

DATA DECISION -MAKING RTI successful if child performs criterion-level probe (from screening) in the instructional range. RTI unsuccessful if 15 consecutive intervention sessions and criterion probe is not in the instructional range. Increase task difficulty for intervention if child scores at mastery on task during intervention sessions

Guidelines for Implementers Use single trial scores for screening Following screening, grade-wide graphs to principal Return data to teachers within 48 hours with personal interpretation at grade-level team meeting Include principal in critical meetings Involve teachers at all stages

Guidelines for Implementers Learn about curriculum and instruction. Integrate RTI with ongoing school and system reform efforts Thoughtfully merge to subtract duplicate activities and to enhance more comprehensive supplemental and core instructional support activities that may be in place Use RTI data to evaluate the value of ALL instructional programs or resource allocation decisions. Quantify bang for the buck using student performance data.

Infrastructure for Implementation Grade-level planning periods can be utilized Special education team at school can be utilized School Psych must be on-site 1 day/week Developing master schedule for Tier 1, 2, and 3 intervention times is useful Integrate efforts with evaluation referral team efforts (consider major reduction in meeting time and shift to intervention efforts!) Use existing instructional periods to target student needs more effectively See NASDSE blueprint for implementation Brown-Chidsey book coming from Guilford

For More Information [email protected] www.isteep.com Thank you to the US Dept of Education for providing all film clips shown in this presentation

how to do classwide intervention within rti amanda vanderheyden education research and consulting,...

Documents

math slide

assessment slide

individual intervention

day of intervention

problem slide

child slide

classwide intervention

class intervention