SCREENING AND PROGRESS MONITORING: THE HOW TO’S…..

BEAVERTON SCHOOL DISTRICT10-23-09

Dean Richards, Lisa Bates, Jon Potter

Objectives

Provide the philosophy behind universal screening

Provide overview of why to use progress monitoring and its essential features

Show how to use progress monitoring Developing an aimline and phaselines

How to use the information in Tier 1 and Tier 2 meetings

Managing Complex Change

= Change

Confusion=

=

=

=

=

+ + + +Vision Incentives ResourcesAction Plan

+ + + +Skills Incentives Resources Action Plan

+ + + +Vision

Incentives ResourcesAction Plan

+ + + +Vision Skills Resources

Action Plan

+ + + +Vision Skills Incentives Action

Plan

+ + + +Vision Skills Incentives Resources

Adapted from Knoster, T.

Anxiety

Resistance

False Starts

Frustration

Skills

The Process is Ongoing and Long-Term

CONSENSUS

Oregon RTI project’s role today Create knowledge of one component of

the RTI infrastructure (assessment) Provide knowledge and skills of Universal

Screening and Progress Monitoring

Expectations

Let us know when you need a break Write questions on sticky notes provided

and we will answer them when we can

The Importance of Data

Data are important

Assessments

Universal Screening Tools Why? What?

What is CBM?

Curriculum Based Measures are usually composed of a set of standard directions, a timing device, set of materials, scoring rules, standards for judging performance, and record form or charts.

These are also called General Outcome Measures (GOM)

Purposes of CBMs

Evaluate overall effectiveness of program Select students who need additional

support Monitor progress of students

A universal screener should over-identify students who might need something more!

Essential Features of CBM Tools• Robust indicator of academic health • Brief and easy to administer• Can be administered frequently• Must have multiple, equivalent forms

(If the metric isn’t the same, the data are meaningless)

• Must be sensitive to growth

Universal Screening Quick general outcome measures Should occur for ALL students 3x per

year Used for data-based decision making

about: How to create instructional change for

ALL Changes for Tier 1

Which students need a closer look and/or intervention Changes for Tier 2

Why use a Universal Screener to Track Progress?

• Reading trajectories are established early.

• Readers on a low trajectory tend to stay on that trajectory without intervention.

• Students on a low trajectory tend to fall further and further behind.

• Early identification leads to early intervention. The later children are identified as needing support, the more difficult it is to catch up!

Maze EasyCBM AIMSweb ORF DIBELS Math Computation Math Applications Math Tests of Early

Numeracy Writing (Total Words

Written) Writing (Correct Word

Sequences)

Quick Phonics assessment

QRI-IV CORE Multiple Measures

Assessment DRA2 Fountas and Pinnell Report cards Meeting OAKS standards Read Well Unit Tests,

core curriculum weekly tests on skills that are learned

Screeners Not Screeners

•The major purpose for administering diagnostic tests is to provide information that is useful in planning more effective instruction.

Purpose of Diagnostic Assessments

• Diagnostic tests should only be given when there is a clear expectation that they will provide new information about a child’s difficulties learning to read that can be used to provide more focused, or more powerful instruction.

OAKS Passing OAKS 3rd grade (204) places a child in the 17th percentile

Passing 3rd grade is not enough.

By 5th grade, if these students stay at the same percentile they will fail OAKS

By 10th grade, if these students stay at the same percentile they will fail OAKS

This student will score a 236 and pass at the 37th percentile

Based on 2008-2009 ODE percentiles

Storytown Assessments

Assesses what was taught that week/unit The metric changes each week (i.e.

inferencing is more difficult than summarizing)

Not enough questions to truly guide individual student instruction

Oral Reading Fluency

• Same measure middle of first grade through eighth grade

• ORF is not designed to provide an exhaustive assessment.

• You can be fluent enough, unless you want to be an auctioneer!

• Strong link to comprehension• Accuracy matters!

Oral reading fluency and comprehension activity• Meet a new friend. Find a person who

you do not know in the room. (This is important because this is how these assessments are often given to students.)

• The person with the next birthday is the Test Administrator, paper A. The other person will be the Test Taker, paper B.

• Test Administrator, read your directions to yourself. Give the test and score.

• Test Taker, read the passage aloud and be prepared to answer questions.

Oral Reading Fluency

95% 98% 99%

The Secret Life of Bees

18.5 7.4 3.6

My Brother Sam is Dead

15 6 3

The Magic School Bus

6 2.4 1.2

Oral Reading Errors per Page at Different Levels of Accuracy

Richard Allington, 2009

Evaluate Overall Effectiveness of Program Are 80% of students reaching

benchmarks and “on track” for next goal? 90 minutes of reading daily? Protected allocated reading time each

day? Skill grouping by class or grade? Core and supplemental programs

implemented with fidelity? More professional development needed?

Indicators of the health of the core

Assessment Walls

13

10 3

16

121 3

66

6 6000

Intensive at Beginning of the year Strategic at Beginning of the year Benchmark at Beginning of the year

Which students do you need to take a closer look at? Lowest 20% (?) of students What do your resources allow?

Find the Students Belowthe Proficiency Guide Point

About 65% Meeting Minimum Proficiency

Lowest 20% identified to receive interventions

5th Grade MathMultiplication & Division 0-9

0

20

40

60

80

100

120

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 9910

010

110

210

310

410

510

610

710

810

911

0

Dig

its C

orre

ct T

wo

Min

utes

Fifth Grade Math

About 60% Meeting Minimum Proficiency

Screening Indicates Math Problem in Grade 5 Given The Standard – Do We Have a Problem With Our Core?

Lowest 20% identified to receive interventions

Make a School Plan

• Who will conduct Universal Screening? • Who will train the screeners?• Who will prepare materials?• Who will organize at the school?• Where will the data go?• Who will organize the data and present it

to teaching teams?• Who will keep track of which students are

in interventions?

Purpose of a Tier 1 meeting

•To determine the effectiveness of the core program•Make necessary adjustments if the core is not meeting the needs of most students

Tier One Meeting agenda

Purpose of meeting: To determine the effectiveness of the core program and make necessary adjustments. Determine if your system is moving children toward benchmark goals.

Review team norms Review purpose for meeting

1) Is our core program sufficient for most students? a) Review and analyze benchmark screening data.

i) How does this round of screening data compare to screening data from earlier in the year or last year? ii) What trends do you see in these data?

b) Review and analyze core program assessments. i) Are students making adequate progress? ii) What is the class average score in each category? iii) When analyzing weekly tests, look at the total questions given weekly in the unit and compute percentage

passed. Are 80% of students passing at 80%? c) Review and analyze OAKS assessments (in Fall review final spring data, Winter and Spring

progress made thus far) i) Review percentile needed for passing at the grade level. ii) Review of grade level percentage passing in the first try, second try, and third try. iii) Which strands need additional instruction?

2) What instructional adjustments are needed to improve the health of the core?

a) Discuss grade level wide opportunities to make core program more robust for this cohort. i) What agreements can the grade level make on common instructional strategies?

(1) How are you going to make the instruction more explicit? (2) How are you going to provide extra guided practice when needed? (3) How are you going to provide more corrective feedback to students? (4) How and when are you going to have students practice skills independently?

ii) What agreements can the grade level make on common active engagement strategies? How will you increase active engagement to increase the effectiveness of the instruction?

iii) What has worked? iv) What can you do to improve your teaching of the core to fidelity? v) How is your team using fidelity checklists to improve the instruction within the core? vi) What are the professional development needs?

b) What are the common instructional needs of the students in this grade level? c) Establish an end of the year goal to work toward for the percentage of student you would like

to see in each tier based on assessment data.

3) How can we support the students for whom the core is not sufficient? a) Identify the percentage of students who are falling in each tier based on the assessment data.

i) What trends do you see? ii) Are students in need of strategic instruction progressing towards benchmark standards? iii) Are students in intensive instruction progressing towards strategic or benchmark standards?

b) Identify the initial list of students who should be monitored more frequently than 3x per year using fluency measures.

c) Discuss what you are doing to challenge your highest skilled students.

Talk Time

Directions for 3-2-1 Activity

3 connections between the video and what you heard today

2 things you think would work in your school

1 question

Oregon RTI Video Clip

Talk Time

Directions for 3-2-1 Activity

3 connections between the video and what you heard today

2 things you think would work in your school

1 question

Assessments

Progress Monitoring Tools Why? What?

Maze EasyCBM AIMSweb ORF DIBELS Math Computation Math Applications Math Tests of Early

Numeracy Writing (Total Words

Written) Writing (Correct Word

Sequences)

Quick Phonics Screener QRI-IV CORE Multiple Measures

Assessment DRA2 Fountas and Pinnell Report cards Meeting OAKS standards Read Well Unit Tests, core

curriculum weekly tests on skills that are learned

Progress Monitoring Tools Not Progress Monitoring Tools

Progress Monitoring: Key Features

• Administration is standardized • Student performance is measured frequently (e.g.,

once a week, or every other week) and results in quantitative data

• Progress is monitored toward an observable, measurable, and ambitious goal

• Progress is graphed and viewed regularly• Data decision rules are used consistently

Essential Features of Reliable and Valid Progress Monitoring Tools

• Robust indicator of academic health • Brief and easy to administer• Can be administered frequently• Must have multiple, equivalent forms

(If the metric isn’t the same, the data are meaningless)

• Must be sensitive to growth

Why Monitor Progress?

It’s the GPS for educators It improves achievement It’s the law

Where are we?What is our goal?What course should we follow?How are we doing?

ActualCourse

DesiredCourse

Our Goal

We are Here

Effects of Progress Monitoring

• Progress monitoring has been extensively researched in Special Education (Fuchs & Fuchs, 1986)

• Students showed improved reading scores when teachers:• monitored their progress (+.70 effect size; ≈ 25th 50th %ile. Like it!)• graphed their reading scores (+.80 effect size. Love it!)• used decisions rules to determine whether to make a

change in instruction (+.90 effect size. Gotta have it!)

Progress Monitoring leads to good things…..

It’s the Law!

Teams must include for all SLD evaluations“data that demonstrate that prior to or as

part of the referral process the child was provided appropriate instruction in regular education settings, delivered by qualified personnel; and

Data based documentation of repeated assessments of achievement at reasonable intervals, reflecting formal assessment of student progress during instruction, which was provided to the child’s parents.”

This information is to be used to prompt evaluation as appropriate.

Sources for Reliable and Valid Monitoring Tools

National Center on Student Progress Monitoring (studentprogress.org)

DIBELS (dibels.uoregon.edu - Free to Oregon Schools)

AIMSWEB (www.aimsweb.com) Easy CBM (www.easycbm.com) Monitoring Progress of Basic Skills (Fuchs &

Fuchs; Reproducible masters) The ABC’s of CBM (Hosp, Hosp,& Howell)

TTSD EBIS DECISION RULES - Grades 1-5

So.. How do you progress monitor?

Key components Aimlines

Tell you where you need to go

Phase lines Tell you that you made a change

Trendlines Tell you if you are o the right track

Aimlines: Tell you where you need to go

Step 1: Aimlines

1. Plot the student’s score from the screening data using a X.

2. Draw a red line from the screening score to the end of year benchmark score (bullseye).

Brandon, first grade/Identify Goal and Draw Aimline

The aimline connects where you are to where you want to get to, and shows the rate of progress needed to get there.

10

20

30

40

Dec.Scores

Feb.Scores

Jan.Scores

M archScores

AprilScores

M ayScores

JuneScores

60

50

Aimline

Cor

rect

wor

ds p

er M

inut

eAimlines

Step 2: Phase Lines

1. Plot the data on the chart by week 2. When there are four data points in a

row below the aimline (TTSD rule, BSD will make own decision rules) you need to make a change in your intervention.

Step 2: Phaselines tell you when to make a change

Whoops! Time to make a change!

10

20

30

40

Dec.Scores

Feb.Scores

Jan.Scores

M archScores

AprilScores

MayScores

JuneScores

60

50

Cor

rect

Pho

nem

es

Aimline

10

20

30

40

Dec.Scores

Feb.Scores

Jan.Scores

MarchScores

AprilScores

MayScores

JuneScores

60

50

Aimline

Determine Response to InterventionWhen a student fails to make adequate progress we should change their instruction to help her meet the goal

Let’s Practice

19151812

18172129

374346

Step 3: Trendlines

Why do we need trendlines? To tell you where you are To tell you how intensive of a

modification/individualization you need to make to get back on track

Step 3: Draw a Trendline

1. Start with at least 7 data points2. Approximately divide the data points into

three equal sections using vertical lines. The outer two sections should have at least three data points

3. In the first and third sections, calculate the mid-date (draw a vertical line) and mid-rate (draw a horizontal line).

4. Mark the points on the graph where the two values intersect.

5. Connect the points to draw a trendline.

Trendline Example: Tukey Method

10

20

30

40

Dec.Scores

Feb.Scores

Jan.Scores

M archScores

AprilScores

MayScores

JuneScores

60

50

Cor

rect

Pho

nem

es

Aimline

10 data points after baseline data divide points

Step 3: Draw a Trendline

1. Start with at least 7 data points2. Approximately divide the data points into

three equal sections using vertical lines. The outer two sections should have at least three data points

3. In the first and third sections, calculate the mid-date (draw a vertical line) and mid-rate (draw a horizontal line).

4. Mark the points on the graph where the two values intersect.

5. Connect the points to draw a trendline.

Trendline Example: Tukey Method

10

20

30

40

Dec.Scores

Feb.Scores

Jan.Scores

M archScores

AprilScores

MayScores

JuneScores

60

50

Cor

rect

Pho

nem

es

Aimline

Mid-date & Mid-Rate

Step 3: Draw a Trendline

1. Start with at least 7 data points2. Approximately divide the data points into

three equal sections using vertical lines. The outer two sections should have at least three data points

3. In the first and third sections, calculate the mid-date (draw a vertical line) and mid-rate (draw a horizontal line).

4. Mark the points on the graph where the two values intersect.

5. Connect the points to draw a trendline.

Trendline Example: Tukey Method

10

20

30

40

Dec.Scores

Feb.Scores

Jan.Scores

M archScores

AprilScores

MayScores

JuneScores

60

50

Cor

rect

Pho

nem

es

Aimline

Connect the points wiith a trendline

Trendlines

302730

34

3328

3247

283931

221920

22253038

31413245

5155

RN 30 min

+ Phonics for Reading

Trendline

Sam

Trendline FYIs

Some research suggests that you should have at least 14 data points to develop a reliable trendline

Step 4: Redraw aimline

• When an intervention needs to be changed, the aimline needs to be redrawn

1. Using the last three data points, find the intersection of the mid-date & mid-rate.

2. Draw a new aimline from this point to the end of the year benchmark.

3. Draw a phase line to reflect that the intervention has changed (after that point).

Redrawing Aimlines

302730

34

3328

3247

283931

Use mid-date & mid-rate of last 3 data points & redraw aimine

Progress Monitoring Chant

Remember, you have to share this information with the rest of your school.

This is a GLAD (Guided Language Acquisition Design) strategy

Progress Monitoring Chant

We all know ‘cause we’ve been toldProgress monitoring is worth its weight in goldWatch the data, I’ll tell you whyIt won’t leave you high and dry

Progress (Progress)Monitoring (Monitoring)Make informed instructional decisions

everyday (everyday)

Progress Monitoring Chant

When we monitor, students learn moreImproved decision making for all we adoreIt doesn’t measure just one skillMonitoring measures pace towards the goals we

need to fulfill

Progress (Progress)Monitoring (Monitoring)Make informed instructional decisions everyday

(everyday)

Important Factors to Examine Match between skill need and

intervention Group Size is small

provides enough explicit instruction and feedback

Time in program is sufficient

Organizing Fluency Data:Making the Instructional

Match

Group 1: Accurate

and Fluent

Group 2:Accurate but Slow

Rate

Group 3:Inaccurate and Slow

Rate

Group 4:Inaccurate but High

Rate

Regardless of the skill focus, organizing student data by looking at accuracy and fluency will assist teachers in making an appropriate instructional match!

Organizing Fluency Data:Making the Instructional Match

Group 1:

Accurate and Fluent

Group 2:

Accurate but Slow

Rate

Group 3:

Inaccurate and Slow

Rate

Group 4:

Inaccurate but High

Rate

Group 1: Dig Deeper in the areas of reading comprehension, includingvocabulary and specific comprehension strategies.Group 2: Build reading fluency skills. (Repeated Reading, Paired Reading, etc.) Embed comprehension checks/strategies.Group 3: Conduct an error analysis todetermine instructional need. Teach to the instructional need paired with fluency buildingstrategies. Embed comprehension checks/strategies.Group 4: Conduct Table-Tap Method. If studentcan correct error easily, teach student to self-monitor reading accuracy. If reader cannot self-correct errors, complete an error analysis to Determine instructional need. Teach to the instructional need.

Purpose of a Tier 2 Meeting

• To determine which students are in need of intervention, select appropriate interventions based on protocol, schedule intervention groups, and schedule progress monitoring.

• Determine effectiveness of interventions• To review progress monitoring for all

students in interventions (20% group) and make decisions about whether to proceed with, discontinue, or change an intervention.

Reading Protocol

Contacts

Dean Richards drichards@ttsd.k12.or.us 503-431-4135

Lisa Bates lbates@ttsd.k12.or.us 503-431-4079

Jon Potter jpotter@ttsd.k12.or.us 503-431-4149