rti and intervention fidelity tools

Dr. John D. Barge, State School Superintendent“Making Education Work for All Georgians”www.gadoe.org

RTI and Intervention Fidelity Tools

GACIS ConferenceDecember 2012

Paula Freer, PhD RTI, SST, Psychological Services GaDOE


Session ObjectivesTools to Support RTI Framework/Process

• Curriculum and Instruction• Assessment• Data Decision Making and Interpretation• Problem-Solving Process• Interventions

–Tools to Help You Build an Intervention Fidelity Framework in Your School and/or System


21st Century Context (Horner adapted, 2008)

Science guided by our values and visionPrograms and practices guided by our science

Early Intervention

Literacy

Math

School Improvement Positive Behavioral

Interventions & SupportsParent

Engagement

Response to Intervention

UDLGeorgia

Pyramid of

Interventio

n DifferentiationCommon Core GPS

College and Career Ready

Dr. John D. Barge, State School Superintendent“Making Education Work for All Georgians”www.gadoe.org(Fixsen, Blase, Horner, Sugai, 2008)

Using RTI to Align Align Resources

Literacy

Wraparound

Math

Family Support

Behavior Support (PBIS)

AL

IGN

ME

NT

Early Intervention

Resp

on

se t

o

Inte

rven

tion

/Pre

ven

tion

Student Outcomes

Primary Prevention

Universal Screening

Multi-tiered Interv Support

Early Intervention

Progress Monitoring

Systems to support practices


KKey elements: Ga Pyramid of Interventions ey elements: Ga Pyramid of Interventions (SSTAGE,

2010)

4

Tier 3

Tier 2

Tier 1Standards-Based Classroom Learning : All students; implementation of the GPS through research-based practices, differentiated instruction and progress monitoring through multiple formative assessment and analysis of student work. School-Wide Positive Behavioral Interventions and Supports.

Needs-Based Learning: In addition to Tier 1 Different by including specialized pyramids of intervention, greater frequency of progress monitoring of learning

SST-Driven Learning: In addition to Tier 1 and Tier 2 Different by including individualized assessments, formal progress monitoring, interventions tailored to individual needs, referral for specially- designed instruction if needed.

Specially-Designed Learning: GPS access/extension, greater frequency of progress monitoring, specialized programs, methodology or instructional delivery ((Sped, EL, Gifted..)


Tier 1 is the Foundation for ALL Tiers

Tier 1 is the foundation of the Pyramid. Tier 1 academicand behavioral supports are vital to the success of all Tiers.

School-wide, grade level, class- wide data guides: Selection of Tier 1 research based strategies Universal Design for LearningDifferentiationInstructional planning including core foundational

skills Ongoing formative assessment Focus for coaching, consultation, feedbackCommon Core GPS


RTI, Differentiation, and UDL

Tier 4

Tier 3

Tier 2

Statewide: CCGPS and GPS—WHAT

District-wide/School-wide: Local curricula incorporate UDL Principles to maximize student access

Classrooms: Instruction is differentiated based on the readiness, interests, or learner profile data of specific students in the class HOW

Tier 1


Universal Design for Learning (UDL) is a set of principles for curriculum development that give all individuals equal opportunities to learn at Tier 1.

UDL provides a blueprint for creating instructional goals, methods, materials, and assessments that work for all students—not a single one-size-all solution but rather flexible approaches that can be customized and adjusted for individual needs.

Source: Cast.org


Universal Design for LearningIS incorporated in the initial unit and lesson planning to meet the needs of a wide diversity of students from the start.

IS NOT retrofitted adjustments or modifications to units or lessons for specific students or groups of students.

DOES provide access to a rigorous curriculum for all students, including ELs, SWDs, gifted students and the kids next door.

DOES NOT lower expectations or achievement for certain students or groups of students.

IS a framework for all students. IS NOT a special education initiative.


UDL PD Resources on GPB

UDL, Part 1: Principles of Universal Design http://www.gpb.org/education/common-core/udl-part-1

UDL, Part 2: Scaffolding Texts & Providing Explicit Strategy Instructionhttp://www.gpb.org/education/common-core/udl-part-2

10

http://www.gpb.org/education/common-core/udl-part-1




Connecting RTI to New Priorities: Aligning Educational Initiatives

(Center on Instruction, 2011)

The Center on Instruction hosted a webinar Oct-2011 discussing states’ implementation of new priorities and how these initiatives align with Response to Intervention (RTI):

School Improvement Grants (SIG), College & Career Ready Standards (including Common Core

State Standards), Meeting the needs of diverse student populations,

Tools:• Connecting RTI to New Priorities: Aligning Educ. Initiatives/pdf • Connecting RTI to New Priorities: Aligning Educ. Initiatives/ppt• RTI Alignment Planning Packet/doc

http://centeroninstruction.org/webinar-connecting-rti-to-new-priorities-aligning-educational-initiatives



Essential & Core Concepts of RTI

TIER 1•Students receive high quality, research-based instructional strategies by qualified staff in general and special education.•Instructors assume an active role in students’ assessment in that curriculum.•School staff conduct universal screening of both academics and behavior.

TIERs 2, 3, & 4•School staff implement specific, research-based interventions to address the student’s difficulties.•Regular progress monitoring of student performance occurs.•School staff use progress-monitoring data and data decision rules to determine interventions’ effectiveness and needed modifications.•PL, Coaching and systematic assessment of the intervention fidelity or integrity of instructional implementation are ongoing and documented.

(NRCLD,adapted for GA POI/RTI)


Positive Student Outcomes areDependent Upon

• Fidelity of implementation of RTI Framework (at the school and system levels)

•Degree to which interventions are empirically supported (evidence and/or research-based)

• Fidelity of intervention implementation(at the interventionist and teacher level- classroom level) (Pierangelo & Giuliani, 2008)


What is Fidelity?Fidelity of implementation refers to how closely the prescribed procedures of a process or intervention are followed (Mellard & Johnson, 2007).

Fidelity and Integrity are two major terms used in RTI research. They are often used interchangeably.


RTI Fidelity Matters Tier 4 Problem-Solving ProcessFidelity of Progress MonitoringFidelity of most intensive interventions

Tier 3SST Problem-Solving ProcessIndividualized interventions Progress Monitoring

Tier 2Data teams Targeted group, standard protocol interventions Progress Monitoring

Tier 1 Assessment and Universal ScreeningInstruction, CurriculumWalk-Through (e.g., instructional fidelity)


RTI has the potential to help a school make better use of its resources for increasing overall

student achievement

However, achievement cannot be realized if screening, interventions, and progress monitoring procedures are not implemented with fidelity (NRCLD, 2006).

SSTAGE RTI/POI Best Practice Rubric: Learning, Self-assessment and Planning Tool

Includes five major components of the RTI/ POI framework:

1. Problem Solving Process

2. Assessment and Progress Monitoring

3. Instructional/Behavioral supports

4. Professional Learning / Teacher support

5. Parent/Family Communication & Involvement17


What is Fidelity in Curriculum and Instruction?

• Fidelity of implementation is the delivery of instruction and interventions in the way in which they are designed to be delivered

(Gresham, MacMillan, Boebe-Frankenberger, & Bocian, 2000). • Examples of assessing instructional fidelity

include:– Walk-Throughs– Peer observations

(Barringer, ret. 2011)


What is Fidelity of Assessment?

Universal Screening -US & Progress Monitoring-PM

Fidelity of the data collection process meansthat all individuals are collecting data following exactly the same procedures.

(Barringer, 2011)


Universal Screening and Progress MonitoringUniversal screenersUniversal screeners help show the “big picture”Whole school or large groups of students: Universally screened- using grade level CBM Probes:•Reading (1min. fluency, 2-5 min Maze Rdg. Comp)•Math (2-4 min computation fluency; concepts/applic 5-20 min) •Writing (5 min fluency)•Behavior (Frequency/type of office referrals, attendance…)Essential Q’s Universal Screening:•How are our 9th graders performing in reading using 9th grade probe?

Progress MonitoringProgress Monitoring assesses targeted student skills to measure response to intervention using CBM ProbesProgress monitoring- CBM Probes at student’s performance level

Essential Q’s Universal Screening:•How is John- a 9th grader (who is at 6th gr level math) responding to PALS- Math intervention using normed 6th grade math probe?


Key Components (Barringer, 2011)

Students’ Universally Screened & Progress monitored• CBM assessments (normed)

• Results graphed against goals, comparison groups, and expected rates of weekly progress- all based on research/norms

• Decisions regarding curriculum and instruction based on data

(NRCLD, 2006)


What is Fidelity of Data Decision-Making and Interpretation?

• Fidelity means that the same decision making processes/rules are being applied to every case, across settings and across time

• Fidelity means that the data is being

interpreted the same way by all individuals engaged in interpretation

(Barringer, 2011)

Fidelity Activity -Universal Screening

Interpreting Results Between and Within Campuses

• What does the universal screening data tell you about these 6 schools?

– What questions would you ask about certain school’s curriculum and instructional practices?

– The readiness of each school for the Common Core?


What is Fidelity of the Problem-Solving Process?

• Fidelity means that the data is being interpreted the same way by all individuals engaged in interpretation. – Tier 1- School-wide and grade level data trends– Tier 2- Data teams, targeted small group school

needs- standard protocol interventions – Tier 3- SST- Individualized student support plans– Tier 4- Specialized Programs- Gifted, EL, SWD…

(Barringer, 2011)


TIPS-Fidelity-Checklist-Revised.pdf


What is Fidelity ofIntervention Implementation?

• Degree to which interventions are empiricallysupported (evidence and/or research-based)

• Fidelity of intervention implementation(at the interventionist and teacher level- classroom level)

(Barringer, 2011)


There is a great deal of confusing language being used to ‘qualify’ strategies, interventions,

programs and practices

Which is which?• Strategies ______________

• Interventions:– Scientifically-Based ______________– Research-Based ______________– Evidence-Based ______________

27


StrategiesDefinition of Strategy A loosely defined collective term that is often used interchangeably with the word “intervention”; however strategies are generally considered effective instructional and behavioral practices rather than a set of prescribed instructional procedures, systematically implemented (GaDOE RTI Guidance).

Examples (Classroom Instruction that Works, Marzano)–Cooperative learning–Reinforcing effort and providing recognition–Setting objectives and providing feedback–Nonlinguistic representations –Graphic organizers


Interventions are NOT… (John McCook, 2006)

• Preferential seating• Shortened assignments• Parent contacts• Classroom observations• Suspension• Doing MORE of the same• Retention• Peer helpers (informal)


ESEA Defines Scientifically Based Reading Research as:– (A) applies rigorous, systematic, and objective procedures to

obtain valid knowledge relevant to reading development, reading instruction, and reading difficulties; and

– (B) includes research that:• (i) employs systematic, empirical methods that draw on observation or

experiment; • (ii) involves rigorous data analyses that are adequate to test the stated hypotheses

and justify the general conclusions drawn;• (iii) relies on measurements or observational methods that provide valid data

across evaluators and observers and across multiple measurements and observations; and

• (iv) has been accepted by a peer-reviewed journal or approved by a panel of independent experts through a comparably rigorous, objective, and scientific review.

(20 U. S. C. § 6368(6))

Understanding Scientifically-Based Interventions (NASP-Harn, 2007)


Intervention ScienceScientists have produced programs and practicesthat can help students, communities & education systems

– What Works Clearinghouse http://ies.ed.gov/ncee/wwc/

– Best Evidence Encyclopedia www.bestevidence.org

– Promising Practices Network promisingpractices.net

– SAMHSA- NREPP http://www.nrepp.samhsa.gov/ – Colorado Blueprints

http://www.colorado.edu/cspv/blueprints/


Interventions

• Definition of an intervention– Targeted instruction that is based on student needs.

Interventions supplement the general education curriculum. Interventions are a systematic compilation of well researched or evidence-based specific instructional strategies and techniques (GaDOE RTI Guidance).


Interventions Must Haves …• Be connected to a specific goal that is well-defined,

observable and measurable• Be Matched to Student Need (skill or motivation deficit?;

acquisition, proficiency/fluency, generalization, adaptation)• Have specific, defined, step-by-step directions so they

can be: Implemented consistently Can be replicated so it can be researched

• Include ongoing progress monitoring of the student’s response to the intervention

• Fidelity, training, and coaching of the intervention• Scheduling to support interventions

(Burns,M., Chris Riley-Tillman, T., & VanDerHeyden, A., 2012)


Top Reasons for Academic Problems (Daly & Martens, 1997; EB Interv ret www. 2012 )

• The task is too hard for the student -

Acquisition Interventions

• They have not had enough help doing the task

Proficiency/Accuracy Interv’s

• The student has not spent enough time doing the academic activity -

Proficiency/Speed Interv’s

• The student has demonstrated the skill before, but has difficulty applying the skill in a new manner -

Generalization Interventions

• The student does not want to do the academic task

Motivation Interventions


Intervention Tools

Evidence Based Intervention NetworkDr. T. Chris Riley-Tillman Educ., School, and Counseling Psych-Univ. of Missouri

TOOLS*Evidence Briefs *Intervention Scripts *Videos

http://ebi.missouri.edu/


Intervention Tools Intervention Centralwww.interventioncentral.org

Reading Intervention Manual http://www.jimwrightonline.com/pdfdocs/brouge/rdngManual.PDF

Peer Tutoring Reading Manualhttp://www.jimwrightonline.com/pdfdocs/prtutor/peerTutorManual.pdf


Reading Interventionwww.helpsprogram.org

•(HELPS) Program is to assist students with their reading fluency. Multiple studies evaluating the effects of the HELPS Program show that students who receive this program, compared to those who do not, significantly improve their reading fluency and reading comprehension (references and articles available upon request [email protected] )

•In Georgia, with a four-tiered model, HELPS may be appropriate at Tiers 1-4. There is also a HELPS Program that can be used with small groups of students (typically used as part of Tier 1 and Tier 2 instruction), and a Spanish version of the program (Leamos para Avanzar) also exists.


PALS TA: Iris Vanderbilt Website

Gr K-1 PALShttp://iris.peabody.vanderbilt.edu/palsk1/chalcycle.htm Gr 2-6 Reading PALShttp://iris.peabody.vanderbilt.edu/pals26/chalcycle.htm HS PALS Modulehttp://iris.peabody.vanderbilt.edu/palshs/chalcycle.htm


Integrity of Progress Monitoring (Windram & Gibbons, 2011)


Top reasons why interventions FAIL?• Not implemented with FidelityFidelity

– Implemented Inconsistently– Implemented Incorrectly- missing steps, not implemented

in the time it was designed (Ex: Do intervention for 20 min 1x wk when research

designed for 45 min intervention- 3x wk)

• Not matched to student need• Lack of Progress Monitoring • Lack of training and coaching to support• Lack of scheduling supports


Essential Questions: Assessing Instructional Contexts and the Fidelity of Implementation

What is fidelity? Whether an intervention was implemented as

planned (Moncher & Prinz, 1991)

Surface fidelity (Gersten, Fuchs, Compton,et al., 2005)

Were key components implemented? Was adequate time allowed? Was the specified amount of material covered?

Quality of delivery (Gersten, Fuchs, Compton,et al., 2005)

Teacher behaviors Student behaviors(Parisi, Potter & Whitcomb, NASP 2007)


Intervention Fidelity/Integrity Forms

• Many research based interventions have those fidelity tools available

http://www.aea11.k12.ia.us/educators/idm/checkists.html

http://www.oregonrti.org/node/33/


Intervention Implementation (Windram & Gibbons, 2011)

How will intervention implementation fidelity be ensured?

Select an intervention with high probability of successCommunicate a clear plan to interventionistsProvide specific training and support to interventionists. Directly observe intervention in action.Make adjustments to the plan if needed.Collect and graph data on the goal.


Activity: Review Fidelity Data

With your elbow partner, discuss:•What does your fidelity data tell you?

•What follow-up steps would you see are needed?

•How is this data important and how could you envision using it?


Response to “Failure to Implement” Intervention?

If the intervention is not implemented as designed, progress (or lack thereof) cannot be attributed to the specific plan or to student failure to respond (Windram & Gibbons, 2011;Kaufman & Flicek, 1995).


How to Create an Intervention Checklist (Windram & Gibbons, 2011)

Write out a brief description of the intervention

Break down intervention into key steps

Use concrete and practical language

Put in table format (possible columns:intervention sequence: yes, no, NA)

Dr. John D. Barge, State School Superintendent“Making Education Work for All Georgians”www.gadoe.org (Windram & Gibbons, 2011)


Factors That Can Reduce Fidelity of Implementation of Intervention NRCLD 2006

Complexity—The more complex it is, the lower the fidelity (includes time to implement)

Materials and Resources Required— Must be readily accessible

Perceived and Actual Effectiveness— Intervention must beperceived by the teacher as effective and must match the teacher’s style

Interventionists-- Number, expertise, and motivation matter


Who Monitors Fidelity?

• Someone trained in the intervention being monitored

• Someone trained in structured observation

• Someone trained in giving feedback and coaching

• Someone who can develop positive, supportive relationships with teachers


When Do You Monitor Fidelity/Integrity?

• At the beginning, frequently• Provide staff immediate, brief, constructive

feedback• Follow up with written feedback• After a solid protocol is established, less

frequently• Always when the interventionist asks for help


Positive Student Outcomes areDependent Upon

• Fidelity of implementation of RTI Framework (at the school and system levels)

•Degree to which interventions are empirically supported (evidence and/or research-based)

• Fidelity of intervention implementation(at the interventionist and teacher level- classroom level) (Pierangelo & Giuliani, 2008)


Georgia UpdatesGaDOE- GPB RTI Series Coming Soon:

Georgia’s Promising Practices in RTI which includes an Georgia’s RTILeaders’ Panel Discussion and four teams representing the 2012SSTAGE RTI STAR Award winning system , elementary, middle and high school. • Georgia’s RTI Leaders’ Panel: Realizing the Potential of RTI • System Level- Griffin Spalding County• High School Level- Ware Co High School• Middle School Level- Teasley Middle, Cherokee Co • Elementary School Level- Woodstock Elementary, Cherokee Co


PL Fidelity Tools

Fidelity of RTI process forms:http://www.rtinetwork.org/getstarted/checklists-and-forms

RTI Integrity Framework Tools/Webinar (27 min)

http://www.rti4success.org/webinar/rti-integrity-framework-tool-monitoring-and-evaluating-rti-implementation-2721


PL Fidelity ToolsNational Center on RTI

Fidelity of Implementation -RTI Framework Dr. Mellard

• Webinar (56 min)• Power Point • Fidelity forms/tools

http://www.rti4success.org/webinar/monitoring-fidelity-rti-5615


IRIS Vanderbilt-PL Tools• The IRIS Center for Training Enhancements. (2010). Fidelity

of Implementation: Selecting and Implementing Evidence-Based Practices and Programs Retrieved on [July 2012] from

http://iris.peabody.vanderbilt.edu/fid/chalcycle.htm


Intervention Central PLFidelity Toolkit

Intervention Integrity Part 1: Building Integrity into the Academic Intervention in the Design Phase

http://www.interventioncentral.org/blog/rti-20-elements-effective-interventions/intervention-integrity-part-1-building-integrity-academ


Intervention Central PLFidelity Toolkit

Intervention Integrity Part 2: Using Multiple Measures to Track the Quality With Which Interventions Are Carried Out

http://www.interventioncentral.org/blog/rti-20-elements-effective-interventions/intervention-integrity-part-2-using-multiple-measures-t


GaDOE and Fidelity

GaDOE Chapter on RTI and Fidelityhttp://archives.gadoe.org/DMGetDocument.aspx/Chapter%209%20Fidelity%20of%20Implementation.pdf?p=6CC6799F8C1371F629C13CBF50C40F3C933BFD2C32CE92F150B5DDF13E2D2061&Type=D

GaDOE/SERVE Chapter on R-B Interventionshttp://archives.gadoe.org/DMGetDocument.aspx/Chapter%208%20Section%208.1%20Interventions.pdf?p=6CC6799F8C1371F6D1DCF9259BC8B9BC6C274BBA0BE05CD558A2342FEFB41093&Type=D


State Guidance onResearch and Evidence Based Resources

GaDOE RTI Webpagehttp://www.gadoe.org/Curriculum-Instruction-and-Assessment/Curriculum-and-Instruction/Pages/Response-to-Intervention.aspx

RTI Guidance Manual 2011http://www.gadoe.org/Curriculum-Instruction-and-Assessment/Curriculum-and-Instruction/Documents/RTI%20document%20Full%20Text.pdf

Parent GaPOI/RTI Brochure/Parent Chapterhttp://archives.gadoe.org/DMGetDocument.aspx/RTI%20for%20Parents%20Sep%2018%202009%20Newsletter%20format.pdf?p=6CC6799F8C1371F6809F76CEBC2D1082E99C433A5A790A1B42C22BA634A989A4&Type=D

GaDOE SST Webpagehttp://www.gadoe.org/Curriculum-Instruction-and-Assessment/Student-Support-Teams/Pages/default.aspx


National Center on Response to Interventionhttp://www.rti4success.org/ This free site is supported by the US Department of Education, the National Center on Response to Intervention (RTI) provides information to individuals and technical assistance to state educational agencies across the country about RTI.

Institute of Education Sciences (IES) - National Center for Educational Evaluation and Regional Assistance http://ies.ed.gov/ncee/wwc/publications/practiceguides/#rti Spring of 2009 two practice guides on RTI were released, one on reading and one on mathematics. Other helpful information ranges from improving adolescent literacy to reducing behavior problems at the elementary school level.


National Center on Student Progress Monitoring http://www.studentprogress.org/ This site provides information on the scientifically based practice of screening and monitoring students' skill development. Many different tools are available.

Research Institute on Progress Monitoringwww.progressmonitoring.org

Florida Center on Reading Research http://www.fcrr.org This site provides multiple intervention ideas linked to the five essential components of effective early reading instruction. Interventions for individuals or small groups can be effective early reading in printed in their entirety.


Thank you!

rti and intervention fidelity tools

Documents

school staff

fidelity of implementation

intervention fidelity

fidelity nrcld

researchbased interventions

progressmonitoring data

progress monitoring

interventions effectiveness