calling ms. cleo:what can dibels tell us about the future ben clarke, scott baker, and ed...

Calling Ms. Cleo:What Can DIBELS Tell Us About the Future

Ben Clarke, Scott Baker, and Ed Kame’enui

Oregon Reading First Center

February 3, 2004

“The Pleasure of Reading” by Carol Tyx

“While part of the pleasure of reading lies in going somewhere else, that’s only one piece of the story. Reading a novel is like watching a potter at work: out of a lump of words, a story emerges. Or maybe it is more like being a potter yourself, thestory taking shape right between your hands. As you pull thewords up from the page, you see the contours of lives—the curves,the contradictions. It is more difficult to discern the shape of our lives, one chapter overlapping with another. A long read allows us to hold the beginning and the ending of the story in our hands, see the start and the finish at one sitting, and in the process, we touch what is ordinarily too unstable, too shifting to grasp. Momentarily, we witness the shaping power of time. And through that witness, we understand more fully what it means to be alive in time”.

Big Ideas of Today’s Presentation

• Remembering trajectories: Why they matter.

• Reading success is built upon a foundation of skills.

• DIBELS provides information about performance today and performance tomorrow.

• Educators can use this information to make a difference at the student, classroom, school, and district level.

Trajectories: The Predictions

(Good, Simmons, & Smith, 1998)

• Students on a poor reading trajectory are at risk for poor academic and behavioral outcomes in school and beyond.•Students who start out on the right track tend to stay on it.

Longitudinal Outcomes forDIBELS Benchmark Assessment

• Odds of achieving subsequent early literacy goals for DIBELS Benchmark Assessments at the beginning, middle, and end of kindergarten, first, second, and third grades (12 screening points across K - 3) are available at

dibels.uoregon.edu/techreports/decision_rule_summary.pdf• Students are at risk if the odds are against achieving

subsequent early literacy goals.• The purpose of screening is to provide additional instructional

support -- strategic or intensive -- sufficient to thwart the prediction achieve reading outcomes.

Sample Cutoffs for Low Risk, Some Risk, At Risk for Kinder DIBELS Performance

DIBELS 3 Benchmark Goals and Indicators of Risk Kindergarten

Beginning of Year

Month 1 - 3

Middle of Year

Month 4 - 6

End of Year

Month 7 - 10 DIBELS Measure Scores Status Scores Status Scores Status

DIBELS Initial Sound Fluency

ISF < 4

4 <= ISF < 8

ISF >= 8

At risk

Some risk

Low risk

ISF < 10

10 <= ISF < 25

ISF >= 25

Deficit

Emerging

Established

DIBELS Letter Naming Fluency

LNF < 2

2 <= LNF < 8

LNF >= 8

At risk

Some risk

Low risk

LNF < 15

15 <= LNF < 27

LNF >= 27

At risk

Some risk

Low risk

LNF < 29

29 <= LNF < 40

LNF >= 40

At risk

Some risk

Low risk

DIBELS Phoneme Segmentation Fluency

PSF < 7

7 <= PSF < 18

PSF >= 18

At risk

Some risk

Low risk

PSF < 10

10 <= PSF < 35

PSF >= 35

Deficit

Emerging

Established

DIBELS Nonsense Word Fluency

NWF < 5

5 <= NWF < 13

NWF >= 13

At risk

Some risk

Low risk

NWF < 15

15 <= NWF < 25

NWF >= 25

At risk

Some risk

Low risk

Sample Odds of Achieving Early Literacy Goals for Different Patterns of DIBELS Performance

Table 4 Instructional Recommendations for Individual Patterns of Performance on Middle of Kindergarten DIBELS Benchmark Assessment

Percent Meeting Later Goals

Initial Sound Fluency

Letter Naming Fluency

Phoneme Segmentation

Fluency Pctile End K PSF

Mid 1 NWF

End 1 ORF Avg. Incidence Instructional Support Recommendation

Deficit At Risk At Risk 3 18 14 19 17 More Common Intensive - Needs Substantial Intervention Deficit At Risk Some Risk 7 34 13 21 23 Unusual Intensive - Needs Substantial Intervention Emerging At Risk At Risk 9 28 20 28 25 More Common Intensive - Needs Substantial Intervention Emerging At Risk Some Risk 11 41 17 22 27 More Common Intensive - Needs Substantial Intervention Deficit Some Risk At Risk 13 24 28 48 33 More Common Intensive - Needs Substantial Intervention Deficit At Risk Low Risk 15 60 21 25 35 Unusual Intensive - Needs Substantial Intervention Deficit Some Risk Some Risk 16 37 30 40 36 Unusual Strategic - Additional Intervention Established At Risk At Risk 17 45 32 31 36 Extremely Rare Strategic - Additional Intervention Emerging Some Risk At Risk 18 37 30 49 38 Unusual Strategic - Additional Intervention Deficit Low Risk At Risk 20 30 37 58 42 Unusual Strategic - Additional Intervention Established Some Risk At Risk 21 42 38 49 43 Extremely Rare Strategic - Additional Intervention Emerging Some Risk Some Risk 22 47 36 51 45 More Common Strategic - Additional Intervention Established At Risk Some Risk 24 52 38 47 45 Extremely Rare Strategic - Additional Intervention Emerging At Risk Low Risk 26 75 29 36 47 More Common Strategic - Additional Intervention Deficit Low Risk Some Risk 28 43 42 68 51 Unusual Strategic - Additional Intervention Deficit Some Risk Low Risk 29 66 41 55 54 Extremely Rare Strategic - Additional Intervention Emerging Low Risk At Risk 31 42 50 70 54 More Common Strategic - Additional Intervention Established Some Risk Some Risk 33 55 44 64 54 Unusual Strategic - Additional Intervention Established At Risk Low Risk 34 82 34 47 54 Unusual Strategic - Additional Intervention Emerging Low Risk Some Risk 38 53 53 80 62 More Common Strategic - Additional Intervention Emerging Some Risk Low Risk 44 82 47 59 63 More Common Strategic - Additional Intervention Established Low Risk At Risk 47 51 58 89 66 Extremely Rare Benchmark - At grade level Established Low Risk Some Risk 49 58 62 87 69 More Common Benchmark - At grade level Deficit Low Risk Low Risk 52 74 60 75 70 Unusual Benchmark - At grade level Established Some Risk Low Risk 54 88 56 69 71 More Common Benchmark - At grade level Emerging Low Risk Low Risk 64 88 68 83 80 More Common Benchmark - At grade level Established Low Risk Low Risk 86 93 80 93 89 More Common Benchmark - At grade level

Note. Percent meeting goal is the conditional percent of children who meet the end of first grade goal of 40 or more on DIBELS ORF. Based on n of approximately 32000 students, 638 schools, and 255 school districts.

Instructional Recommendations for Individual Patterns of Performance on Middle of Kindergarten

DIBELS Benchmark Assessment




Fluency Pctile

Deficit At Risk At Risk 3 Deficit At Risk Some Risk 7 Emerging At Risk At Risk 9 Emerging At Risk Some Risk 11 Deficit Some Risk At Risk 13

[Table Continues]

Established Some Risk Low Risk 54 Emerging Low Risk Low Risk 64 Established Low Risk Low Risk 86

Pattern of performance based on the DIBELS Benchmark Assessment

[Tab

le C

on

tin

ues

]

Percentile Rank for the pattern of performance. For example, a child with established ISF, some risk on LNF, and low risk on PSF is at the 54th percentile compared to other children in the middle of kindergarten. He or she achieved as well or better than 54% of children in participating schools on DIBELS. dibels.uoregon.edu/techreports/decision_rule_summary.pdf

Average Percent achieving subsequent early literacy goals. For example, a student with a Deficit, Some Risk, At Risk pattern on DIBELS has 33% odds of achieving later literacy goals on average.

Percent Meeting Later Goals




Fluency End K

PSF Mid 1 NWF

End 1 ORF Avg.

Deficit At Risk At Risk 18 14 19 17 Deficit At Risk Some Risk 34 13 21 23 Emerging At Risk At Risk 28 20 28 25 Emerging At Risk Some Risk 41 17 22 27 Deficit Some Risk At Risk 24 28 48 33

[Table Continues]

Established Some Risk Low Risk 88 56 69 71 Emerging Low Risk Low Risk 88 68 83 80 Established Low Risk Low Risk 93 80 93 89


DIBELS Benchmark Assessment (continued)

Odds of achieving specific early literacy goal. For example, 83% of students with Established, Some Risk, Low Risk pattern in the middle of kindergarten achieved the end of first grade DIBELSOral Reading Fluency goal of 40 or more words read correct per minute.

[Tab

le C

on

tin

ues

]

dibels.uoregon.edu/techreports/decision_rule_summary.pdf




Fluency Incidence

Deficit At Risk At Risk More Common Deficit At Risk Some Risk Unusual Emerging At Risk At Risk More Common Emerging At Risk Some Risk More Common Deficit Some Risk At Risk More Common

[Table Continues]

Established Some Risk Low Risk More Common Emerging Low Risk Low Risk More Common Established Low Risk Low Risk More Common


DIBELS Benchmark Assessment (continued)

Extremely rare patterns may indicate a need to retest. For example, it would be extremely rare for a student to have Established ISF, Low Risk on LNF, and At Risk status on PSF. Their PSF score may not be accurately estimating their phonemic awareness skill.

[Tab

le C

on

tin

ues

]

Incidence or how often a pattern of performance occurs. For example, among students with a Deficit on ISF and Some Risk on LNF, achieving in the At Risk range on PSF would be a more common pattern, but achieving in the Some Risk range would be an unusual pattern.





Fluency Instructional Support Recommendation

Deficit At Risk At Risk Intensive - Needs Substantial Intervention Deficit At Risk Some Risk Intensive - Needs Substantial Intervention Emerging At Risk At Risk Intensive - Needs Substantial Intervention

[Table Continues]

Emerging Some Risk Some Risk Strategic - Additional Intervention Established At Risk Some Risk Strategic - Additional Intervention Emerging At Risk Low Risk Strategic - Additional Intervention

[Table Continues]

Established Some Risk Low Risk Benchmark - At grade level Emerging Low Risk Low Risk Benchmark - At grade level Established Low Risk Low Risk Benchmark - At grade level

Instructional Recommendations for Individual Patterns of Performance on Middle of Kindergarten DIBELS Benchmark

Assessment (continued)

Instructional Support Recommendation. For students with odds in favor of achieving subsequent literacy goals, benchmark instruction is recommended. For students with odds against achieving subsequent literacy goals, intensive support is recommended. For about 50 – 50 odds, strategic support is recommended.

[Tab

le C

on

tin

ues

]


Beginning of Kindergarten

Middle of Kindergarten

End of Kindergarten

Decision Utility of DIBELS • Pattern of performance on DIBELS measures determines overall

risk status and instructional recommendation. In fall of first grade, for example,

• LNF >= 37, DIBELS PSF >= 35, DIBELS NWF >= 24Instructional Recommendation: Benchmark - At grade level. Effective core curriculum and instruction recommended,– Odds of reading 40 or more words correct per minute at the end of first grade: 84%– Odds of reading less than 20 words correct per minute at the end of first grade: 2%

• LNF < 25, DIBELS PSF < 10, DIBELS NWF < 13 Instructional Rec: Intensive - Needs substantial intervention: – Odds of reading 40 or more words correct per minute at the end of first grade: 18%

(unless given intensive intervention)– Odds of reading less than 20 words correct per minute at the end of first grade:

48% (unless given intensive intervention)• Value of knowing the instructional recommendation and the goal early enough

to change the outcome: Priceless.

Instructional Goals for Core Components of Beginning Reading

Benchmark Goals to be On Grade Level

• Step 1: Phonological Awareness with 25 - 35 on DIBELS Initial Sound Fluency by mid kindergarten (and 18 on PSF)

• Step 2: Phonemic Awareness with 35 - 45 on DIBELS Phoneme Segmentation Fluency (and 25 on NWF)

• Step 3: Alphabetic principle 50 - 60 on DIBELS Nonsense Word Fluency by mid first grade (and 20 on DORF)

• Step 4: Fluency with 40 - 50 on DIBELS Oral reading fluency by end of first grade.

• Step 5: Fluency with 90 + on DIBELS Oral reading fluency by end of second grade

• Step 6: Fluency with 110 + on DIBELS Oral reading fluency by end of third grade

Instructional Steps from Kindergarten to Successful Reading Outcomes

Big Ideas inBeginningReading

DynamicIndicators ofBig Ideas inBeginningReading

Benchmark GoalTimeline forAssessing BigIdeas K-3

Accuracy &Fluency with

Connected Text

High-StakesReadingOutcome

AlphabeticPrinciple

PhonologicalAwareness

ISF PSF NWF ORF ORF ORF HSA

Fall Winter Spring

Kindergarten

Fall Winter Spring

First Grade

Fall Winter Spring

Second Grade Third Grade

InstructionalStep

Step1

Step2

Step3

Step4

Step5

Fall Winter Spring

Step6

The outcome of each step depends on (a) students beginning skills, (b) effectiveness of core curriculum and instruction, and (c) effectiveness of system of additional instructional support.

Stepping Stones of Early Literacy

Video of Dr. Roland Good

QuickTime™ and aYUV420 codec decompressorare needed to see this picture.

Reviewing Outcomes: Effectiveness of Benchmark Instruction (Core Curriculum)

• For each step toward literacy outcomes, a school with an effective core curriculum and instruction supports students who are on track (i.e., low risk or benchmark) to achieve the goal.

• For students with the odds in favor of achieving literacy goals, it is the job of the core to teach the core components so well that all students achieve the goals.

Reviewing Outcomes: Effectiveness of Strategic and Intensive Intervention

• For each step toward literacy outcomes, a school with an effective system of effective interventions supports students who are not on track (i.e., at some risk or at risk of difficulty achieving literacy goals) to achieve the goal.

• For students with the odds against achieving literacy goals unless we provide an effective intervention, it is the job of the system of additional support to augment the core curriculum so well that all students achieve the same benchmark goals.

Step by Step, Core and Intervention

Step Effectiveness

of Core

Effectiveness of Strategic

Support

Effectiveness of Intensive

Support

Step 1: Phonemic Awareness

Step 2: Phonemic Awareness and Phonics

Step 3: Phonics and Fluency

Step 4: Fluency and Comprehension

Step 5: Fluenc y and Comprehension


AI

AS

AB

Effectiveness of Benchmark (core) for School A

Effectiveness of Strategic support for School A

Effectiveness of Intensive support for School A

Evaluating the Effectiveness of the Core Curriculum and Instruction

1. Is the core curriculum and instruction getting at least 90% to 95% of Benchmark students to the next early literacy goal?– If children are on track, the core should keep them on track.– What would it take to achieve 100%?

2. Is the core curriculum and instruction as effective as other schools in getting Benchmark students to the goal?– If typical schools are not getting 100% of Benchmark

students to the goal, then supplementing the core in this area can improve reading outcomes.

Step 3: Beginning First to Middle First

0

50

100

150

200

250

300

0 - 1011 - 2021 - 3031 - 4041 - 5051 - 6061 - 7071 - 8081 - 9091 - 100Conditional Percent Reaching NWF Goal

Number of Schools

Intensive

Strategic

Benchmark

A typical (middle) school had 68% of children with a beginning first grade benchmark recommendation achieve the middle of first grade goal, and 0% of children with intensive support recommendation.

Benchmark Median School

Intensive Median School

Middle of first grade outcomes for students with benchmark, strategic, and intensive instructional recommendations in the beginning of first grade

AI

AS

AB


Step Effectiveness

of Core


Support


Support





Step 5 a: Fluenc y and Comprehension

Step 5b: Fluency and Comprehension

Step 6 a: Fluency and Comprehension

Step 6 b: Fluency and Comprehension

Strength – Effectiveness is at goal or greater than typical of other schools

Support – Effectiveness is less than a typical school and less than goal.Typical – Effectiveness is less than goal, but typical of other schools.


Step Effectiveness

of Core


Support


Support









Support Strength Strength

SupportStrengthStrength

TypicalStrengthTypical



Step 4: Middle First to End First

0

50

100

150

200

250

300

350

0 - 1011 - 2021 - 3031 - 4041 - 5051 - 6061 - 7071 - 8081 - 9091 - 100Conditional Percent Reaching ORF Goal

Number of Schools

Intensive

Strategic

Benchmark

A typical (middle) school had 96% of children with a middle first grade benchmark recommendation achieve the end of first grade goal, and 0% of children with intensive support recommendation.

Benchmark Median SchoolIntensive

Median School

End of first grade outcomes for students with benchmark, strategic, and intensive instructional recommendations in the middle of first grade

AI

AS

AB


Step Effectiveness

of Core


Support


Support









Support Strength Strength

SupportStrengthStrength

TypicalStrengthTypical

Support TypicalTypical



Instructional Steps from Kindergarten to Successful Reading Outcomes

Big Ideas inBeginningReading

DynamicIndicators ofBig Ideas inBeginningReading

Benchmark GoalTimeline forAssessing BigIdeas K-3

Accuracy &Fluency with

Connected Text

High-StakesReadingOutcome

AlphabeticPrinciple

PhonologicalAwareness

ISF PSF NWF ORF ORF ORF HSA

Fall Winter Spring

Kindergarten

Fall Winter Spring

First Grade

Fall Winter Spring

Second Grade Third Grade

InstructionalStep

Step1

Step2

Step3

Step4

Step5

Fall Winter Spring

Step6

Step by step to important reading goals and outcomes. Implicit in this logic is a linkage to High Stakes Reading Outcomes.

Third Grade Oral Reading Fluency to Oregon Statewide Assessment Test

• Odds of “meets expectation” on OSAT given 3rd grade TORF of 110 : 90 of 91 or 99%.

• Odds of “meets expectation” on OSAT given 3rd grade TORF below 70: 4 of 23 or 17%.

160

170

180

190

200

210

220

230

240

250

0 20 40 60 80 100 120 140 160 180 200 220 240TORF May Grade 3

OSAT Total Score

r = .7353% of Variance

Meets

Does not meetExpectations

Exceeds

Linkage of Third-Grade TORF to Illinois State Assessment Test (ISAT)

• Odds of “meets standards” on ISAT given Third-Grade TORF of 110 or above: 73 of 74 or 99%.

• Odds of “meets standards” on ISAT given Third-Grade TORF of 70 or below: 1 of 8 or 12%.

r = .7963% of Variance

Sibley, D., Biwer, D., & Hesch, A. (2001). Unpublished Data. Arlington Heights, IL: Arlington Heights School District 25.

3rd Grade Benchmark in Reading - CBM

Above 110, the odds are strong the student will rank “proficient” on the AK State Benchmark.

Ala

ska

Sta

te B

ench

mar

k in

Rea

din

g

Below Proficient

Below 70, the odds are low the student will rank “proficient” on the AK State Benchmark.

100

125

150

175

200

225

250

275

300

325

350

375

400

425

450

475

500

525

550

25 50 75 100 125 150 175 200 225 250

Words Per Minute: TORF 3.3

Not Proficient

Proficient

Advanced

Proficient

Linner, S. (2001, January). Curriculum Based Assessment in reading used as a predictor for the Alaska Benchmark Test. Paper presented at the Alaska Special Education Conference, Anchorage, AK.

Linkage of Oral Reading Fluency to State Reading Outcome Assessments

Oral Reading Fluency

240220200180160140120100806040200

Reading FCAT-SSS Score

550

500

450

400

350

300

250

200

150

100

50

Buck, J., & Torgesen, J. (2003). The relationship between performance on a measure of oral reading fluency and performance on the Florida Comprehensive Assessment Test (Technical Report 1). Tallahassee, FL: Florida Center for Reading Research,.

Above 110, the odds are 91% the student will rank “adequate” on the FL State Assessment.

Below 80, the odds are 19% the student will rank “adequate” on the FL State Assessment.

Important Themes

Curriculum and instruction should teach the most important stills that are most predictive of later reading achievement.

It’s not enough to teach -- students must learn what was taught in the program.

All students must learn essential foundation skills.

Important themes cont. All students must perform at high criterion

levels of performance on the essential foundation skills.

All students must learn essential skills well enough in time.

If students meet important goals during specific times, we can predict later reading success.

calling ms. cleo:what can dibels tell us about the future ben clarke, scott baker, and ed...

Documents