universal screening and progress monitoring indiana department of education response to instruction...

Universal Screening and Progress Monitoring

Indiana Department of EducationResponse to Instruction Pilot Training

Friday, July 23, 2010

Rebecca S. Martínez, Ph.D.Associate Professor

School Psychology ProgramIndiana University – Bloomington

Our learning goals for today:Our learning goals for today:

• Understand how screening and progress monitoring are similar and Understand how screening and progress monitoring are similar and how they are differenthow they are different

• Appreciate the value of assessment data in making informed Appreciate the value of assessment data in making informed instructional and placement decisionsinstructional and placement decisions

• Learn what important questions you can answer with screening and Learn what important questions you can answer with screening and progress monitoring dataprogress monitoring data

• Discover resources to assist in selecting screening and progress Discover resources to assist in selecting screening and progress monitoring assessment toolsmonitoring assessment tools

• Determine who is struggling using screening dataDetermine who is struggling using screening data

• Examine a child’s RTI using progress monitoring dataExamine a child’s RTI using progress monitoring data

For data analysis techniques to be valuable to For data analysis techniques to be valuable to educators, the techniques must enhance the chance educators, the techniques must enhance the chance that educators gain insight into student performance that educators gain insight into student performance

and that they translate this insight into improved and that they translate this insight into improved educational experiences for children.educational experiences for children.

Screening and Progress Monitoring do this!Screening and Progress Monitoring do this!

Another way to look at it:Another way to look at it:

All assessment activities MUST relate to efforts to appropriately All assessment activities MUST relate to efforts to appropriately differentiate instruction based on students’ needs. This may differentiate instruction based on students’ needs. This may

include consideration of include consideration of group size, instructional period, group size, instructional period, frequency, and duration frequency, and duration (Deschler, 2008) to ensure concept (Deschler, 2008) to ensure concept

mastery.mastery.

55

• If your assessment efforts are notIf your assessment efforts are not• aligned to curriculum and instructionaligned to curriculum and instruction• used in the context of general school used in the context of general school

improvement effortsimprovement efforts

…….then your assessment efforts are essentially .then your assessment efforts are essentially worthless…worthless…

Ikeda, Neesen, & Witt, 2008Ikeda, Neesen, & Witt, 2008

• We can only appropriately differentiate We can only appropriately differentiate instruction id we know instruction id we know what what to address and to address and with whomwith whom

• We can only know this information if we assess We can only know this information if we assess our studentsour students

• We need data to substantiate teacher’s or We need data to substantiate teacher’s or parents’ “gut feelings” about struggling parents’ “gut feelings” about struggling studentsstudents

““Without data, Without data, you are just another person with you are just another person with

an opinion.”an opinion.”

Boeing Aircraft CompanyBoeing Aircraft Company

Two Response to Instruction Two Response to Instruction assessment “types”assessment “types”

1.1. Universal ScreeningUniversal Screening• BenchmarkingBenchmarking

2.2. Progress MonitoringProgress Monitoring

Universal Screening: Universal Screening: Definition and PurposeDefinition and Purpose

• Definition: Universal screening is the administration Definition: Universal screening is the administration of an assessment to of an assessment to allall students in the classroom, students in the classroom, grade, school or district.grade, school or district.

• Frequency: 2-3 times a yearFrequency: 2-3 times a year

• Purpose: To determine which students may be Purpose: To determine which students may be struggling with certain skills (e.g., phonological struggling with certain skills (e.g., phonological awareness or reading fluency)awareness or reading fluency)

Adapted from http://iris.peabody.vanderbilt.edu/rti02_assessment/cresource.h

tm

Progress Monitoring: Progress Monitoring: Definition and PurposeDefinition and Purpose

• Definition: Progress Monitoring is the administration Definition: Progress Monitoring is the administration of an assessment to of an assessment to some some students in the classroom, students in the classroom, grade, school or district. grade, school or district.

• Frequency: Usually every week, but at least every Frequency: Usually every week, but at least every monthmonth

• Purpose: To determine which students respond to Purpose: To determine which students respond to intervention and which students are in need of more intervention and which students are in need of more intensive supportsintensive supports

Adapted from http://www.rtinetwork.org/Essential/Assessment/Progress/ar/MutlilevelPreventi

on

A word of caution:A word of caution:

• if you are progress monitoring, but not doing so with fidelity if you are progress monitoring, but not doing so with fidelity you WILL loose the value of the datayou WILL loose the value of the data

• Said another way, if you don’t assess with fidelity, your efforts Said another way, if you don’t assess with fidelity, your efforts are not worth the time you spend assessing! are not worth the time you spend assessing!

1212

1313

Tier 3:Twice a month to once a week

Regarding assessment frequency…

Tier 2:Monthly to twice a month

Tier I:2-3 times a year

Benefits of universal screeningBenefits of universal screening

• Global indicators of the overall academic or behavioral Global indicators of the overall academic or behavioral “health” of a system and the individuals within the “health” of a system and the individuals within the systemsystem• % at risk, % some risk, % low risk% at risk, % some risk, % low risk

• Helps identify prevalence and severity of problems so Helps identify prevalence and severity of problems so resources can be applied efficientlyresources can be applied efficiently

• Allows personnel to adjust intervention intensity or Allows personnel to adjust intervention intensity or rigorrigor

Ikeda, Neesen, & Witt, 2008

Good Universal Screeners Help Data Teams Good Universal Screeners Help Data Teams Answer All the Following…Answer All the Following…

• How is each student responding to instruction?How is each student responding to instruction?

• How many students are at risk for failure?How many students are at risk for failure?

• Is the current core instruction effective?Is the current core instruction effective?

• Which students may need additional assessment?Which students may need additional assessment?

• What levels of resource support might be needed to What levels of resource support might be needed to promote desired (criterion-level) performance?promote desired (criterion-level) performance?


Benefits of Benefits of Progress MonitoringProgress Monitoring

• Accelerates learning because more appropriate Accelerates learning because more appropriate instruction that has been differentiated for their individual instruction that has been differentiated for their individual student needsstudent needs

• Allows for more informed instructional decisionsAllows for more informed instructional decisions• Documents student progress for accountability purposesDocuments student progress for accountability purposes• Yields more efficient communication with families and Yields more efficient communication with families and

other professionals about students’ progressother professionals about students’ progress• Promotes higher expectations for students by teachersPromotes higher expectations for students by teachers• Leads to fewer Special Education referrals Leads to fewer Special Education referrals

http://www.studentprogress.org/progresmon.asp#3

Good Progress Monitoring Helps Set SMART Good Progress Monitoring Helps Set SMART Goals!Goals!

• RRealistic, ealistic, MMeasureable, easureable, AAttainable, ttainable, RRealistic ealistic TTimelyimely

• Example: (Student name) will be able to read (#) Example: (Student name) will be able to read (#) grade level reading passages at a rate of (#) words grade level reading passages at a rate of (#) words read correctly per minute with an accuracy rate of read correctly per minute with an accuracy rate of 95% or above on 3 consecutive reading passages by 95% or above on 3 consecutive reading passages by (date intervention will end). (date intervention will end).

Assessment, Curriculum & InstructionAssessment, Curriculum & Instruction

• Assessment (screening and progress monitoring) is a Assessment (screening and progress monitoring) is a necessary, but not sufficient component of an aligned necessary, but not sufficient component of an aligned service delivery systemservice delivery system

• Must also have curriculum and instruction that Must also have curriculum and instruction that complements each other and assessment practices or complements each other and assessment practices or findingsfindings


Assessment … Assessment … Whom?Whom?

CurriculumCurriculum …… What?What?

InstructionInstruction …… How?How?

Criteria for Selecting Universal Screening Criteria for Selecting Universal Screening and PM Instrumentsand PM Instruments

• Easily administered to groups of studentsEasily administered to groups of students

• Brief in administration timeBrief in administration time

• Repeatable over timeRepeatable over time

• Sensitive to changeSensitive to change

• Valid and reliableValid and reliable


Resource to assist in selecting Resource to assist in selecting assessment toolsassessment tools

http://www.studentprogress.org/chart/chart.asp

http://www.studentprogress.org/chart/chart.asp

Academic ScreenersAcademic Screeners

Frequently use CBM (name brands include DIBELS, Frequently use CBM (name brands include DIBELS, AIMSweb, STEEP), but others are acceptable:AIMSweb, STEEP), but others are acceptable:

• CBA: 1 minute readings of Dolch Word ListCBA: 1 minute readings of Dolch Word List• Norm-Referenced Assessments: WJ Reading Norm-Referenced Assessments: WJ Reading

Mastery Test-Word Identification Test, Yopp-Singer Mastery Test-Word Identification Test, Yopp-Singer Test of Phoneme Segmentation, CTOPP, ISTEP, Test of Phoneme Segmentation, CTOPP, ISTEP, NWEANWEA

• ISTARISTAR• AcuityAcuity

School Options Regarding AssessmentSchool Options Regarding Assessment

• SelectionSelection of the assessment instruments of the assessment instruments• Use criteria discussed in previous slideUse criteria discussed in previous slide

• FrequencyFrequency of the assessments of the assessments• Recommendations offered in researchRecommendations offered in research

• CriteriaCriteria used to determine which students are in need used to determine which students are in need of interventionof intervention

Adapted from http://iris.peabody.vanderbilt.edu/rti02_assessment/cresource.ht

m

A Closer Look…A Closer Look…

Criteria used to determine which students are in Criteria used to determine which students are in need of intervention…need of intervention…

• Somewhat subjectiveSomewhat subjective• Subject to available resourcesSubject to available resources

Regardless of the school or district's plan for Regardless of the school or district's plan for implementing a universal screening and progress implementing a universal screening and progress

monitoring process, the school's resources need to be monitoring process, the school's resources need to be organized so students who are struggling can be organized so students who are struggling can be

identified and can receive the services they need in identified and can receive the services they need in order to be successful. order to be successful.


Universal Screening CriteriaUniversal Screening Criteria

1.1. Regardless of the type of universal screening Regardless of the type of universal screening measure used, rank ordering according to scores measure used, rank ordering according to scores (highest to lowest) is used to identify the lowest (highest to lowest) is used to identify the lowest performing students in a class or grade levelperforming students in a class or grade level

2.2. Students with the lowest scores are identified as Students with the lowest scores are identified as struggling students struggling students

• Struggling students are those who may benefit Struggling students are those who may benefit from additional supportfrom additional support


m

Universal Screening CriteriaUniversal Screening Criteria

3.3. Depending on available resources, teachers, time, Depending on available resources, teachers, time, etc. select:etc. select:

1.1. A set number of students from among the lowest A set number of students from among the lowest ranked (e.g., the bottom eight students), orranked (e.g., the bottom eight students), or

2.2. A certain percentage of the lowest performing A certain percentage of the lowest performing students (e.g., the bottom 20 percent of the class students (e.g., the bottom 20 percent of the class or grade). or grade).


m

Sample Sample class wide class wide universal universal screening screening

datadata

Note: When Determining Who StrugglesNote: When Determining Who Struggles

• If the standard is set too high, students who do not If the standard is set too high, students who do not need intervention may be identified, and the school's need intervention may be identified, and the school's resources could be strained because too many resources could be strained because too many students are receiving intensive, expensive services students are receiving intensive, expensive services that they do not need.that they do not need.

• On the other hand, if the standard is set too low, On the other hand, if the standard is set too low, students who are struggling may not receive needed students who are struggling may not receive needed intervention.intervention.


m

At the conclusion of screening, how does At the conclusion of screening, how does progress monitoring come into play?progress monitoring come into play?

• Students identified as struggling based on the Students identified as struggling based on the universal screening receive universal screening receive frequent progress frequent progress monitoringmonitoring of their performance in Tier 1 of their performance in Tier 1

• Intervention ingredients can be tweaked to increase Intervention ingredients can be tweaked to increase rigor (rigor (group size, instructional period, frequency, group size, instructional period, frequency, and duration)and duration)

Adapted from http://iris.peabody.vanderbilt.edu/rti02_assessment/cresource.

htmDeschler, 2008

http://iris.peabody.vanderbilt.edu/rti02_assessment/cresource.htm

http://iris.peabody.vanderbilt.edu/rti02_assessment/cresource.htm

• After each intervention session or on a regularly After each intervention session or on a regularly scheduled time frame, a progress monitor scheduled time frame, a progress monitor assessment is administeredassessment is administered

• The teacher or student plots the score on an The teacher or student plots the score on an individual graphindividual graph


m

Sample CBM Progress Monitoring Sample CBM Progress Monitoring GraphGraph


m

• Examination of the data on the graph allows teachers Examination of the data on the graph allows teachers to determine whether a student is making adequate to determine whether a student is making adequate progressprogress

• Assuming that students are receiving high-quality Assuming that students are receiving high-quality instruction in Tier 1, those who do not respond instruction in Tier 1, those who do not respond adequately are provided additional adequately are provided additional interventions/support in Tier interventions/support in Tier

• Fidelity of instruction AND leadership involvement are Fidelity of instruction AND leadership involvement are key componentskey components


m

Tier 2 assessment and interventionTier 2 assessment and intervention

• At Tier 2, intervention intensifies (in addition to sound At Tier 2, intervention intensifies (in addition to sound instruction in Tier 1) and progress monitoring instruction in Tier 1) and progress monitoring frequency increasesfrequency increases

Sample CBM Sample CBM Progress Monitoring GraphProgress Monitoring Graph

• At the end of the progress At the end of the progress monitoring period, the teacher monitoring period, the teacher compares each student's compares each student's performance level to the performance level to the benchmark specified by the benchmark specified by the measure being usedmeasure being used

Adapted from http://iris.peabody.vanderbilt.edu/rti02_assessment/cresource.htm

Sample CBM Progress Monitoring Sample CBM Progress Monitoring GraphGraph

• If a student's average score on the last three probes is If a student's average score on the last three probes is equal to or greater than the benchmark, the student equal to or greater than the benchmark, the student is responding adequately to instruction.is responding adequately to instruction.

• On the other hand, if a student's score is less than the On the other hand, if a student's score is less than the specified benchmark, more intensive instruction, like specified benchmark, more intensive instruction, like that provided in Tier 3, is warranted.that provided in Tier 3, is warranted.


m

• By using data generated through progress monitoring By using data generated through progress monitoring process, you gain information about a student that is process, you gain information about a student that is SMARTSMART

• When you use data to determine a student’s RTI, you When you use data to determine a student’s RTI, you begin to see progress/lack of progress in a new, begin to see progress/lack of progress in a new, objective lightobjective light

Using Data to Determine ProgressUsing Data to Determine Progress

• Student is not Student is not adequately responding adequately responding to instruction or to instruction or interventionintervention

• Will receive Tier 3 Will receive Tier 3 intervention intervention


m

Using Data to Determine ProgressUsing Data to Determine Progress

• Benefiting from Benefiting from instruction or instruction or interventionintervention

• No instructional No instructional changes requiredchanges required


m

““When you change the way you When you change the way you look at things, the things you look at things, the things you

look at change.”look at change.”

Wayne DwyerWayne Dwyer

Action Plan for Screening and Progress Action Plan for Screening and Progress Monitoring DataMonitoring Data

AssessAssess

AnalyzeAnalyze

ActAct

AssessAssess

• Teachers need frequent information about students’ Teachers need frequent information about students’ individual strengths and weaknesses to guide individual strengths and weaknesses to guide instruction.instruction.• Frequent feedback on student Frequent feedback on student

progress to know if what you’re doing is workingprogress to know if what you’re doing is working

AnalyzeAnalyze

• Teachers and school building administrators need Teachers and school building administrators need multiple opportunities to reflect on and discuss multiple opportunities to reflect on and discuss data.data.• PD on how to understand student achievement PD on how to understand student achievement

datadata • Structured, data driven discussions focused on Structured, data driven discussions focused on

achievement gap dataachievement gap data

ActAct

• Teachers need structure and support to make targeted Teachers need structure and support to make targeted changes to meet the needs of all students. changes to meet the needs of all students. • School-wide focus School-wide focus • Professional development on how to take action on Professional development on how to take action on

datadata• Collaboration time for teachers to work on best Collaboration time for teachers to work on best

practicespractices

What have we learned from the places that are

improving?

• No excuses: Everybody takes responsibility for No excuses: Everybody takes responsibility for student learningstudent learning

• Student performance isn’t blamed on kids Student performance isn’t blamed on kids and their families.and their families.

• Embrace state standards and assessments as Embrace state standards and assessments as benchmarks and leveragebenchmarks and leverage

• Build systems that support all involvedBuild systems that support all involved• Clear, specific and SMART goalsClear, specific and SMART goals

How to Start (NCREL)How to Start (NCREL)

1.1. Develop a Leadership Team Develop a Leadership Team

2.2. Collect and Organize Data Collect and Organize Data

3.3. Analyze Data Patterns to Define Problems/Needs Analyze Data Patterns to Define Problems/Needs

4.4. Pose Hypotheses Pose Hypotheses

5.5. Set Improvement Goals Set Improvement Goals

6.6. Identify Specific Strategies Identify Specific Strategies

7.7. Define Evaluation Criteria Define Evaluation Criteria

8.8. Make the Commitment to: Make the Commitment to: 1.1. Adjust Strategies as NecessaryAdjust Strategies as Necessary

Mr. Beckwith, a beginning teacherMr. Beckwith, a beginning teacher

““...I’ve really come to understand the need for assessment. ...I’ve really come to understand the need for assessment. Without assessing our students, we really don’t know what Without assessing our students, we really don’t know what they know and we don’t know what they’re learning... Certain they know and we don’t know what they’re learning... Certain approaches that may be effective for most of the class may be approaches that may be effective for most of the class may be leaving one or several students in the dark... I would suspect leaving one or several students in the dark... I would suspect that they are often capable, but the instruction itself is not that they are often capable, but the instruction itself is not meeting their needs. Maybe they need more practice, maybe meeting their needs. Maybe they need more practice, maybe they need to have the material presented in an alternative they need to have the material presented in an alternative form, maybe they just need a little one-on-one instruction from form, maybe they just need a little one-on-one instruction from a teacher or a peer.” a teacher or a peer.”

Take Home Message Take Home Message

• Screening and progress monitoring data are used to Screening and progress monitoring data are used to make decisions at all stages of RTI implementationmake decisions at all stages of RTI implementation

• Assessment data must be used to improve student Assessment data must be used to improve student achievement or it is worthlessachievement or it is worthless

• Educators must put the data into action to accomplish Educators must put the data into action to accomplish building wide and individual student improvement building wide and individual student improvement goalsgoals

Questions & AnswersQuestions & Answers

5252