decision-making framework from screening to action
TRANSCRIPT
![Page 1: Decision-making framework From screening to action](https://reader036.vdocuments.mx/reader036/viewer/2022062409/5697c0291a28abf838cd781f/html5/thumbnails/1.jpg)
Decision-making framework
From screening to action
![Page 2: Decision-making framework From screening to action](https://reader036.vdocuments.mx/reader036/viewer/2022062409/5697c0291a28abf838cd781f/html5/thumbnails/2.jpg)
Procrustes
![Page 3: Decision-making framework From screening to action](https://reader036.vdocuments.mx/reader036/viewer/2022062409/5697c0291a28abf838cd781f/html5/thumbnails/3.jpg)
Objectives
• 1.5 hours (one before lunch & half after)• Understand and reflect on the purpose of measures and
key assessment vocabulary– Screening and Benchmarking– Sharpening Instructional Targets– Progress monitoring
• Use problem-solving model to – Identify a reading problem– Analyze data for intervention planning
• Generate instructional targets• Create instructional groups
![Page 4: Decision-making framework From screening to action](https://reader036.vdocuments.mx/reader036/viewer/2022062409/5697c0291a28abf838cd781f/html5/thumbnails/4.jpg)
Problem-Solving ProcessWhat is the problem?
What should be done?
Why is it happening?Did it work?
![Page 5: Decision-making framework From screening to action](https://reader036.vdocuments.mx/reader036/viewer/2022062409/5697c0291a28abf838cd781f/html5/thumbnails/5.jpg)
Problem-Solving ProcessWhat is the problem?
What should be done?
Why is it happening?Did it work?
= Expectation – Level of Performance
![Page 6: Decision-making framework From screening to action](https://reader036.vdocuments.mx/reader036/viewer/2022062409/5697c0291a28abf838cd781f/html5/thumbnails/6.jpg)
• Is there a discrepancy between what is expected and what is occurring?– Expectation = > 140 WRC per
minute with 95% accuracy– Current performance = 95 WRC
with 80% accuracy– Is there a problem?– What do the data tell us?– Any other information you would
want to know?
Case Study: Katie
![Page 7: Decision-making framework From screening to action](https://reader036.vdocuments.mx/reader036/viewer/2022062409/5697c0291a28abf838cd781f/html5/thumbnails/7.jpg)
Problem-Solving ProcessWhat is the problem?
What should be done?
Why is it happening?
Did it work?
What alterable factors
underlie or
maintain problem?
![Page 8: Decision-making framework From screening to action](https://reader036.vdocuments.mx/reader036/viewer/2022062409/5697c0291a28abf838cd781f/html5/thumbnails/8.jpg)
ICEL/RIOT Matrix: Gives us clues as to why
Instruction Curriculum Environment Learner
Review
Interview
Observe
Test
Focus on educationally relevant info
![Page 9: Decision-making framework From screening to action](https://reader036.vdocuments.mx/reader036/viewer/2022062409/5697c0291a28abf838cd781f/html5/thumbnails/9.jpg)
• Goal: To formulate hypothesis about why the problem is occurring. I.E. Find out why she is reading so slow and with such poor accuracy.– Can she see well enough? (Review, Interview, Test)– What instruction has she been given? (Review, Interview)– Could she do it in the right environment?
Disengaged/Unmotivated/Distracted? (Observe, Interview, Test)– Does she have robust decoding skills? (Review, Interview, Test)– Is she fluent with decoding strategies? (Observe, Test)– Does she think about meaning as she reads? (Interview, Test)– Does she have appropriate background knowledge? Vocabulary?
English? (Review, Observe, Interview, Test)
Case Study: Katie
![Page 10: Decision-making framework From screening to action](https://reader036.vdocuments.mx/reader036/viewer/2022062409/5697c0291a28abf838cd781f/html5/thumbnails/10.jpg)
Problem-Solving ProcessWhat is the problem?
What should be done?
Why is it happening?Did it work?
How can we most effectively change those alterable factors that underlie or maintain problem?
What targeted interventions have the highest probability of success (EB)? What resources do we have to hit instructional targets?What is the goal?How will progress be monitored?
![Page 11: Decision-making framework From screening to action](https://reader036.vdocuments.mx/reader036/viewer/2022062409/5697c0291a28abf838cd781f/html5/thumbnails/11.jpg)
Case Study: Katie
• Ease back on Fluency Training Program– Practicing too many errors
• Increase time in word study and multi-syllabic words. Implement Rewards.
• Increase intervention time small group instruction.• Monitor Progress weekly with a goal of increasing accuracy to
97% on fourth grade-level text by end of term.
![Page 12: Decision-making framework From screening to action](https://reader036.vdocuments.mx/reader036/viewer/2022062409/5697c0291a28abf838cd781f/html5/thumbnails/12.jpg)
Problem-Solving ProcessWhat is the problem?
What should be done?
Why is it happening?Did it work?
How can we most effectively change those alterable factors that underlie or maintain problem?
Did Katie meet our goal?If not, revisit problem-solving process.If so but she is not at grade-level, can we be more ambitious?If so and she is at grade level, can we reduce supports?
![Page 13: Decision-making framework From screening to action](https://reader036.vdocuments.mx/reader036/viewer/2022062409/5697c0291a28abf838cd781f/html5/thumbnails/13.jpg)
![Page 14: Decision-making framework From screening to action](https://reader036.vdocuments.mx/reader036/viewer/2022062409/5697c0291a28abf838cd781f/html5/thumbnails/14.jpg)
Activity: Think, Write, Share
1. Write (5 minutes): Without looking at notes, label the flowing
problem-solving diagram In your own words, give a quick description of
each step. Check your work with your notes and adjust.
2. Share (5 minutes) Ones share descriptions of each step with 2s. Twos offer constructive feedback.
![Page 15: Decision-making framework From screening to action](https://reader036.vdocuments.mx/reader036/viewer/2022062409/5697c0291a28abf838cd781f/html5/thumbnails/15.jpg)
You have
10minutes left.
![Page 16: Decision-making framework From screening to action](https://reader036.vdocuments.mx/reader036/viewer/2022062409/5697c0291a28abf838cd781f/html5/thumbnails/16.jpg)
You have
9minutes left.
![Page 17: Decision-making framework From screening to action](https://reader036.vdocuments.mx/reader036/viewer/2022062409/5697c0291a28abf838cd781f/html5/thumbnails/17.jpg)
You have
8minutes left.
![Page 18: Decision-making framework From screening to action](https://reader036.vdocuments.mx/reader036/viewer/2022062409/5697c0291a28abf838cd781f/html5/thumbnails/18.jpg)
You have
7minutes left.
![Page 19: Decision-making framework From screening to action](https://reader036.vdocuments.mx/reader036/viewer/2022062409/5697c0291a28abf838cd781f/html5/thumbnails/19.jpg)
You have
6minutes left.
![Page 20: Decision-making framework From screening to action](https://reader036.vdocuments.mx/reader036/viewer/2022062409/5697c0291a28abf838cd781f/html5/thumbnails/20.jpg)
You have
5minutes left.
![Page 21: Decision-making framework From screening to action](https://reader036.vdocuments.mx/reader036/viewer/2022062409/5697c0291a28abf838cd781f/html5/thumbnails/21.jpg)
You have
4minutes left.
![Page 22: Decision-making framework From screening to action](https://reader036.vdocuments.mx/reader036/viewer/2022062409/5697c0291a28abf838cd781f/html5/thumbnails/22.jpg)
You have
3minutes left.
![Page 23: Decision-making framework From screening to action](https://reader036.vdocuments.mx/reader036/viewer/2022062409/5697c0291a28abf838cd781f/html5/thumbnails/23.jpg)
You have
2minutes left.
![Page 24: Decision-making framework From screening to action](https://reader036.vdocuments.mx/reader036/viewer/2022062409/5697c0291a28abf838cd781f/html5/thumbnails/24.jpg)
You have
1minute left.
![Page 25: Decision-making framework From screening to action](https://reader036.vdocuments.mx/reader036/viewer/2022062409/5697c0291a28abf838cd781f/html5/thumbnails/25.jpg)
Attention Please
Time’s up!
![Page 26: Decision-making framework From screening to action](https://reader036.vdocuments.mx/reader036/viewer/2022062409/5697c0291a28abf838cd781f/html5/thumbnails/26.jpg)
Problem-Solving Process_________________________
______________________________
____________________________
![Page 27: Decision-making framework From screening to action](https://reader036.vdocuments.mx/reader036/viewer/2022062409/5697c0291a28abf838cd781f/html5/thumbnails/27.jpg)
Problem-Solving/Learning Theory
YesBenchmark
Met?No
Continue general
curriculum: Provide challenging work and extensions
to generalize skills
Can do/won’t
doSubstantial
Improvement with motivation?
No
YesPlan motivational
enhancement
Validate Skill Deficit
Screening Measure
Accuracy in skills?
No Teach skills &
practice to high accuracy
Practice skills to fluency/automaticity
Application and Generalization:
use in context.
Yes
![Page 28: Decision-making framework From screening to action](https://reader036.vdocuments.mx/reader036/viewer/2022062409/5697c0291a28abf838cd781f/html5/thumbnails/28.jpg)
Labeling Activity (5 Min)
Where do each of the objects on the previous flow-chart fit in the problem-solving model?
1. What is the problem?2. Why is it occurring?3. What can we do about it?4. Did it work?5. Not tied in to PSM
![Page 29: Decision-making framework From screening to action](https://reader036.vdocuments.mx/reader036/viewer/2022062409/5697c0291a28abf838cd781f/html5/thumbnails/29.jpg)
Reading Measures: Table of Specifications
Decoding/Word Study
Fluency Vocabulary Literal Cmprhnsn
InferenceCmprhnsn
LA-content
ELA-CRT X X X X X X
SRI X X X X X
Maze X X X X (X)
RCBM-WRC
X X (X) (X) (X)
RCBM- Errors
X (X) (X)
Decoding Inventory
X
Vocabulary Inventory
X
![Page 30: Decision-making framework From screening to action](https://reader036.vdocuments.mx/reader036/viewer/2022062409/5697c0291a28abf838cd781f/html5/thumbnails/30.jpg)
SRI & Lexiles
• Strengths:• Gives proficiency levels for more advanced students• Computer Scored• Repeated Measure• Tests both inferential and literal comprehension• Good overall screening tool for secondary students
• Weaknesses:• Reliability: computer issues, motivation…• Doesn’t give many clues concerning what to target
instructionally• Not appropriate for frequent progress monitoring
![Page 31: Decision-making framework From screening to action](https://reader036.vdocuments.mx/reader036/viewer/2022062409/5697c0291a28abf838cd781f/html5/thumbnails/31.jpg)
SRI Pointers
• Let students know their scores and set goals• Consider using as an initial screening measure
to determine who needs remedial reading instruction
• Use scores to help content area teachers scaffold
• Have a rational for score decision points
![Page 32: Decision-making framework From screening to action](https://reader036.vdocuments.mx/reader036/viewer/2022062409/5697c0291a28abf838cd781f/html5/thumbnails/32.jpg)
![Page 33: Decision-making framework From screening to action](https://reader036.vdocuments.mx/reader036/viewer/2022062409/5697c0291a28abf838cd781f/html5/thumbnails/33.jpg)
General Guidelines for Lexile Scores
Text Level
Magic Schl Bus
Charolette’s Web
Johnny Tremain
Animal Farm
Great Expectations
![Page 34: Decision-making framework From screening to action](https://reader036.vdocuments.mx/reader036/viewer/2022062409/5697c0291a28abf838cd781f/html5/thumbnails/34.jpg)
Example: Middle School Reading
YesBenchmark
Met?No
Continue general
curriculum: Provide challenging work and extensions
to generalize skills
CBM: Can do/won’t do
Substantial Improvement with
motivation?
No
YesPlan motivational
enhancement
R-CBM
Scholastic Reading Inventory (SRI)
> 95% accuracy
No Teach skills &
practice to high accuracy
Practice skills to fluency/automaticity
Application and Generalization:
use in context.
Yes
R-CBM
![Page 35: Decision-making framework From screening to action](https://reader036.vdocuments.mx/reader036/viewer/2022062409/5697c0291a28abf838cd781f/html5/thumbnails/35.jpg)
CVMS Reading Classes
• Can do/ won’t do: 83% (39/47) improved 68% significantly, 26% moved up to benchmark
• Inaccurate: 15% (<95% accuracy on RCBM)• Dysfluent: 44% (<25%ile on RCBM)• Poor literal comprehension: 26% (<25%ile
Maze)• Poor vocabulary and/or knowledge: ? %
![Page 36: Decision-making framework From screening to action](https://reader036.vdocuments.mx/reader036/viewer/2022062409/5697c0291a28abf838cd781f/html5/thumbnails/36.jpg)
Implications
• Lots of students in reading classes who don’t appear to need reading intervention
• Motivation a significant problem– many ersatz reading problems are really motivation problems and many real reading problems are compounded by motivation issues. (Pocket of low motivation)
• Clear skill deficits that we can measure and remediate: phonics, fluency, vocabulary
![Page 37: Decision-making framework From screening to action](https://reader036.vdocuments.mx/reader036/viewer/2022062409/5697c0291a28abf838cd781f/html5/thumbnails/37.jpg)
Incoming Students
What are we going to do with them?
![Page 38: Decision-making framework From screening to action](https://reader036.vdocuments.mx/reader036/viewer/2022062409/5697c0291a28abf838cd781f/html5/thumbnails/38.jpg)
Students in Reading Classes:Who will need what?
1. What kinds of data do we currently have?2. What does it tell us about who will need
help?3. What sort of help will they need?4. What sort of structures do we need to have
in place to make sure these needs are met?
![Page 39: Decision-making framework From screening to action](https://reader036.vdocuments.mx/reader036/viewer/2022062409/5697c0291a28abf838cd781f/html5/thumbnails/39.jpg)
Maze (multiple-choice close fluency)
• Can student read silently and understand text accurately and at a good pace?
• Hint: look at rate (better than 17 correct replacements)…but also look at errors (better than 90% accuracy).
![Page 40: Decision-making framework From screening to action](https://reader036.vdocuments.mx/reader036/viewer/2022062409/5697c0291a28abf838cd781f/html5/thumbnails/40.jpg)
Oral Reading Fluency:Four Quadrant Instructional Support
Automatic: Rate ≥ 140 WRCPM
Slow: Rate < 140 WRCPM
Accurate: 95% or higher
Quadrant 1 Accurate
And Good Pace
Quadrant 2 Accurate
And Labored
Inaccurate: 95% or lower
Quadrant 4 Inaccurate
And Good Pace
Quadrant 3 Inaccurate
And Labored
![Page 41: Decision-making framework From screening to action](https://reader036.vdocuments.mx/reader036/viewer/2022062409/5697c0291a28abf838cd781f/html5/thumbnails/41.jpg)
Who needs decoding support?
• 104 Students (5.9%)—minimum.• Measure: 95% accuracy—RCBM
![Page 42: Decision-making framework From screening to action](https://reader036.vdocuments.mx/reader036/viewer/2022062409/5697c0291a28abf838cd781f/html5/thumbnails/42.jpg)
Who needs fluency support?
• 275 Students (16%)• Needing support defined as <114 WRC…, <25th%ile
![Page 43: Decision-making framework From screening to action](https://reader036.vdocuments.mx/reader036/viewer/2022062409/5697c0291a28abf838cd781f/html5/thumbnails/43.jpg)
Relationship between ORF and passing a state high-stakes test (McGlinchey & Hixson, 2004)
![Page 44: Decision-making framework From screening to action](https://reader036.vdocuments.mx/reader036/viewer/2022062409/5697c0291a28abf838cd781f/html5/thumbnails/44.jpg)
Who needs comprehension, vocabulary and background knowledge support?
• Everyone…that is what content courses are about…• But some are more behind
– Deliberate direct instruction of important academic vocabulary
• How do we identify the most behind?– Non-readers– Common formative assessments– SRI
![Page 45: Decision-making framework From screening to action](https://reader036.vdocuments.mx/reader036/viewer/2022062409/5697c0291a28abf838cd781f/html5/thumbnails/45.jpg)
Who needs motivational support
• Unknown from data at hand…but we extrapolate that there are many and that there will be many who slough.
• Success is the best motivation (self perpetuating & natural)
• Perky pace at a level that is neither too difficult nor too easy is motivational
• Social interaction is motivational• School stores and silly prizes are motivating
![Page 46: Decision-making framework From screening to action](https://reader036.vdocuments.mx/reader036/viewer/2022062409/5697c0291a28abf838cd781f/html5/thumbnails/46.jpg)
BL
Significance
Behavior Instruction
Reading Instruction
Reading and Behavior Instruction
R BR BR B
The Need for Academic & Behavioral Integration
Source: Shepard Kellam, Ph.D, Senior Research Fellow, American Institutes for Research (AIR)
![Page 47: Decision-making framework From screening to action](https://reader036.vdocuments.mx/reader036/viewer/2022062409/5697c0291a28abf838cd781f/html5/thumbnails/47.jpg)
Discussion Question
• Would we want to group students by motivation?
![Page 48: Decision-making framework From screening to action](https://reader036.vdocuments.mx/reader036/viewer/2022062409/5697c0291a28abf838cd781f/html5/thumbnails/48.jpg)
Reading Comprehension: Literal and Inferential (SRI)
• 16 % (329 students)
![Page 49: Decision-making framework From screening to action](https://reader036.vdocuments.mx/reader036/viewer/2022062409/5697c0291a28abf838cd781f/html5/thumbnails/49.jpg)
Color Key
SRI, RCBM, & Maze CRT Level of Mastery
< 10%ile 1 Minimal10th to 25%ile 2 Partial25th to 50%ile 3 Sufficient
> 50%ile 4 Substantial
SRI Only50 to 75%ile 75 to 100%ile
![Page 50: Decision-making framework From screening to action](https://reader036.vdocuments.mx/reader036/viewer/2022062409/5697c0291a28abf838cd781f/html5/thumbnails/50.jpg)
Individual Student Example
LEXILE Prc_Rnk WRC Er_CBM Maze Er_MzLACRT_
PC LACRTSCCRT_P
CSCCRT_UPA
SS_5
231 91 99% 8 73% 37 1a 63 2b
176 100% 36 100% 92 4 83 4
300 80 80% 48 1b 74 3
1110 86 176 100% 26 96% 90 4 76 3
946 61 172 98% 27 96% 80 3 69 3
1105 85 143 99% 24 100% 86 4 81 3
400 5 150 85% 80 3 76 3
983 67 164 99% 30 100% 82 3 87 4
0 1 75 96% 14 88% 49 1b 53 2a
932 58 112 100% 19 95% 80 3 83 4
![Page 51: Decision-making framework From screening to action](https://reader036.vdocuments.mx/reader036/viewer/2022062409/5697c0291a28abf838cd781f/html5/thumbnails/51.jpg)
![Page 52: Decision-making framework From screening to action](https://reader036.vdocuments.mx/reader036/viewer/2022062409/5697c0291a28abf838cd781f/html5/thumbnails/52.jpg)
Rough Guideline for identifying readers for skills based intervention classes
• Below 140 words read correctly per minute or below 95% accuracy on R-CBM.
AND
• Below 17 correct replacements on Maze or below 25th percentile on SRI
When in doubt:1. look at LA-CRT scores. 2. Ideally you should do some additional testing.
![Page 53: Decision-making framework From screening to action](https://reader036.vdocuments.mx/reader036/viewer/2022062409/5697c0291a28abf838cd781f/html5/thumbnails/53.jpg)
Your Turn
• Find practice materials at the back of your “Tools for Assessment”… handout.
• Get in groups and look at your data and complete the question sheet with your group
• Discuss– Student need– How school might meet that need– Grouping for reading class (using quadrant & SRI/Maze)– What additional data might be needed