curriculum based measurement writing progress monitoring presenters kathleen begeny-johnson &...
TRANSCRIPT
Curriculum Based Measurement Writing Progress Monitoring
Presenters
Kathleen Begeny-Johnson & Jerry Purles
June 12, 2013
What is your Professional Role?
General Education Teacher Special Education Teacher School/District Administrators Coaches/Consultants/Coordinators
Objectives
Learn how to monitor a student’s writing performance to indicate what the teacher needs to specifically instruct each student
Administer, score, and graph data from Curriculum Based Measurement – Writing (CBM-W)
Use CBM-W for data-based instructional decision making
What Is the Difference Between Traditional Assessments and Progress Monitoring (PM)?
Curriculum-Based Measurement (CBM) is one type of PM:CBM provides an easy and quick method to
gathering student progressTeachers can analyze student scores and adjust
student goals and instructional programsStudent data can be compared to teacher’s
classroom or school district data
Curriculum-Based Measurement (CBM) Overview CBM was designed to be a set of simple, efficient
standard procedures that Are objective Allow for comparison of students to peers and to
grade-level benchmarks Allow for repeated measurement Show student growth Serve as indicators of student performance and
progress in academic areas
Scoring Categories
Category Notation Definition
Correct Letter Sequence
CLS
Total Words Written
TWW A count of the number of words written. A word is defined as any letter or group of letters separated by a space, even if the word is misspelled or is a nonsense word.
Words Spelled Correctly
WSC A count of the number of words that are spelled correctly. A word is spelled correctly if it can stand alone as a word in the English Language.
Correct Writing Sequence
CWS A count of correct writing sequences found in the sample. A correct writing sequence is defined as two adjacent writing units (i.e., word-word or word-punctuation) that are acceptable within the context of what is written. Correct writing sequence takes into account correct spelling, grammar, punctuation, capitalization, syntax, and semantics.
Correct Minus Incorrect Word Sequence
CIWS Number of incorrect word sequences subtracted from number of correct sequence
University of Minnesota Research Institute for Problem Solving Finding Grade 1 (2009)
Measures Scoring
Sentence Copying (3-5 min) TWW, WSC, CWS, CIWS, CLS
Story Prompt (5 min) TWW, WSC, CWS, CLS
Picture Word (3-5 min) TWW, WSC, CWS, CLS
Photo Prompt (5 min) CWS, CLS
Sample Expository Prompts
Describe a game you like to play and tell why you like it.
Describe your favorite day of the week and tell why you like it.
Think of the most valuable think you own that was not bought in a store. Why is it important to you?
Think of something you made with your own hands. How did you do it? What would you differently next time?
Sources for Prompts
Research Institute on Progress Monitoring www.progressmonitoring.org
Write Source http://wwwthewritesource.com
NWREL http://
educationnorthwest.org/resource/514-
Select Materials
Select type of measures based on Grade level Purpose of progress monitoring What will provide you with the most meaningful information
Determine the number of prompts needed How many weeks do you plan to monitor progress How often do you plan to monitor progress
Create That tap students background knowledge The end of a mid-sentence (for narrative/story starters) A sentence that includes a “tell why” component (expository)
TIP
Keep materials consistent throughout the monitoring period
Materials Needed for Written Expression CBMCBM Writing Manual with story starters and
scoring criteriaStory starter appropriate for the students grade
levelLined paper for student responsesStop watchData recording sheet (optional)Graph paper or computer graphing program
General Finding within and Across Grades
Grade Level Type of Prompt Duration Scoring Procedures
1-2 Sentence Copying 3-5 min WW, WSC, CWS
Picture-word WW, WSC, CWS
Narrative WW, WSC, CWS
3-4 Narrative 3-5 min CWS, CIWS
5-6 Narrative 3-5 min CWS, CIWS,
Expository 5 min CWS, CIWS
7-9 Narrative 5-10 min CWS, CIWS, %CWS
ExpositoryCWS, CIWS
10-11 Narrative 7-10 min CWS, CIWS
Selection of Prompts
• Narrative Prompts (story starters) Primary Narrative Prompts Grade 1-3 Intermediate Narrative Prompts Grade 4-6
(Included in the scoring manual)
The story starters should represent the experiences and background knowledge of the students
Administering CBM Writing Say these specific directions to the students:“You are going to write a story. First, I will read a
sentence, and then you will write a story about what happens next. You will have 1 minute to think about what you will write, and 3 minutes to write your story. Remember to do your best work. If you don't know how to spell a word, you should guess. “Are there any questions?” (Pause). “Put your pencils down and listen.”
“For the next minute, think about ... (insert story starter).”
Administering CBM Writing
After reading the story starter, begin your stop watch and allow 1 minute for students to “think.” (Monitor students so that they do not begin writing).
After 30 more seconds say: “You should be thinking about . . . . (insert story starter)”. Let 30 more seconds pass . . .
At the end of the 1 minute say: “Now begin writing.” Restart your stopwatch (set for 3 minutes).
Monitor students' participation. If individual students pause for about 10 seconds or say they are done before the test is finished, move close to them and say “Keep writing the best story you can.” This prompt can be repeated to students should they pause again.
Administering CBM Writing
After 90 seconds say: “You should be writing about… (insert story starter).”
At the end of 3 minutes say: “Stop. Put your pencils down.” If students want to finish their story, they may do so on a separate piece of paper or continue on the same sheet after a slash mark has been inserted at the end of 3 minutes.
Administering CBM Writing
Testing ConsiderationsTesting not teaching Best vs. fastestMonitor student performanceDo not answer questions
Scoring Total Words Written (TWW)
What Is A Word? Any letter or group of letters separated by a space is
defined as a word, even if the word is misspelled or is a nonsense word.
The total number of words written are counted regardless of spelling or context.
Correct spelling, word usage, capitalization, and punctuation are ignored when calculating the number of words written.
Underline each word written when scoring Students can score their own TWW
Example Scoring Total Words Written (TWW)
Sentence Total Words WrittenThe sky was blue TWW = 4The sky was blew TWW = 4I tuk a baf TWW = 4I tuka baf TWW = 3Iv graqz zznip TWW = 3In your manual there are specific scoring procedures
to follow
______ ___ ____ __________ ___ ______
____ ___ ___ _____ ___ ____ ____ __
________ ______ ____ __ ____ ___ ______
__________ ____ ___ ____ ___ _______
___ ______ ____ _____ ___ _____ __ __
______ ___ ________ ______ ___ _ ______
TWW = 42
Scoring Words Spelled Correctly (WSC)
Number of correctly spelled words regardless of context
A word is counted correct if it can be found in the English language
Incorrectly spelled words are circled WSC is calculated by subtracting the total
number of circled words from the TWW Teachers score the WSC
Scoring Words Spelled Correctly (WSC)
What is a correctly spelled word? A word is spelled correctly if it can stand alone as a
common word in the English Language regardless of context
Sentence WSC
Bill will reed the book. WSC = 5And can not pake. WSC = 3Daz ran down the road. WSC = 4
______ ___ ____ __________ ___ ______
____ ___ ___ _____ ___ ____ ____ __
________ ______ ____ __ ____ ___ ______
__________ ____ ___ ____ ___ _______
___ ______ ____ _____ ___ _____ __ __
______ ___ ________ ______ ___ _ ______TWW = 42
WSC = 39
Found in English
Language, regardless of context
Scoring WSC
How to Score CWS
Correct Word Sequences (CWS): CWS is any two adjacent, correctly spelled words
acceptable within the context of the sample to a native English speaker.
The teacher considers the units of writing and their relations to one another
The two words must be syntactically and semantically correct
Acceptable word sequence: “the car” Unacceptable word sequence: “car eyebrow”
EX: Nobody could see the trees of the forest .
^ ^ ^ ^ ^ ^^ ^
How to Score CWS
Correct Word Sequences (CWS): Carat is placed ABOVE two words if it
represents a CWS “the ^ car”
Correct carats placed between: Between any two correct word sequence Between a word and the line at the beginning of
a sentence Between a word and the correct punctuation at
the end of a sentence
How to Score CWS
Correct Word Sequences (CWS): Carat is placed BELOW two words if it represents
an incorrect word sequence “car V eyebrow”
Incorrect carats placed between: Any two incorrect word sequence Between a misspelled circled word and non-
circled word Between an un-capitalized word and line at
beginning of a sentence Between an incorrect word and punctuation at
end of a sentence
How to Score CWS
Correct Word Sequences (CWS): Teacher must read entire sample before scoring Vertical line placed where a sentence should end
^ Mary ^ asked ^ if ^ I ^ would ^ come ^ over
^. ^ I ^ said ^ no ^. CWS = 12
Judgment calls may have to be made about where sentences end: Make decision rules and stick to them when scoring.
Misspelled words are circled
How to Score CWS
^ It ^ was ^ dark ^ . ^ Nobody ^ could seen the ^ trees ^ of ^ the forrast .
Since the first word is correct, it is marked as a correct writing sequence
Because the end mark is considered essential punctuation, a carat is placed between the word and the end mark to make a correct writing sequences.
Grammatical or syntactical errors are not counted. A down carat is place between a incorrect writing sequences. (Note: incorrect writing sequence within the same sentence).
Misspelled words are not counted.
CWS = 10
CWS Scoring Practice
One day, we were playing outside the school and … I Shrunk a Person olmost Steped on me But I Ran to fast ten David nodest me. I seid Can You help me with Everthing he sied Yes! Ov course I sead Ya!
Steps to Score CBM1. Read entire sample before scoring, place vertical lines where
sentence ends and circle misspelled words 2. CWS- Two adjoined words are:
Spelled correctlySemantically correctSyntactically correctCorrectly punctuation
How to Administer and Score Written Expression CBM
v
One day, we were playing outside the school and … I
ShrunkÚ aPersonÚ olmostÚ StepedÚ onme But
I RanÚtoÚfastÚ ÚtenÚDavidÚnodestÚ me. IÚ seid
ÚCan You helpme withÚEverthingÚ heÚ sied Ú
Yes ! ÚOvÚcourse IÚseadÚYa Ú!
Ú
Ú
CWS = 7
CIWS Scoring Procedure
Correct Minus Incorrect Word Sequence (CIWS) The number of incorrect word sequences subtracted from the number of correct sequences.
^A ^cat chasd the ^mouse^ into ^ the hoose .
CWS = 5 IWS = 4 CIWS = 1
Grade Time of year TWW WSC CWS
1 Fall 7-8 5
Spring 14-20 10
2 Fall 12-24 20
Spring 25-30 27
3 Fall 23-36 32 14
Spring 34-36 33 26
4 Fall 33-41 38 23
Spring 41-46 44 37
5 Fall 37-51 48 34
Spring 42-57 55 41
6 Fall 41-47 42 41
Spring 53-58 56 51
7 Fall 48-51 49 53
Spring 58 56 64
8 Fall 59-74 70 48
Spring 58-67 66 67
ESTIMATED TARGETS BASED ON SELECTED SCORES FROM MULTIPLE RESOURCES (Best Practices in School Psychology V; The ABCs of CBM) These scores represent averages of the selected scores. Use with caution. As we collect data, we will develop local norms
Example of a Baseline
CBM Writing Fall 2011
0
10
20
30
40
50
60
70
80
90
100
Date
TW
W &
WS
C
Baseline TWW
Baseline
Baseline
(Median Data-point)
• Administer 3 separate CBM writing probes
• Plot 3 baseline data points on the graph
• Identifying the median score
The values are 15, 22, 10
How to Set and Graph Goals
Once baseline data have been collected (best practice is to administer three probes and use the median score), the teacher decides on an end-of-year performance goal for each student.
Three options for making performance goals: Published norms (AIMSweb www.aimsweb.com)
http://fehb.org/CSE/CCSEConference2011/Wright/wright_Pre_Conference_AIMSweb%20Norms_resource.pdf Intra-individual framework
*Tip*
Set reasonable/attainable yet ambitious goals
Setting Goal
Student baseline performance Example: 24 CWS (median of 25, 24, 23)
Desired rate of progress Example: 1.5 CWS per week
Amount of the time the student will be monitored Example: 35 week
Take the growth per week (1.5) X the number of weeks (35) = 1.5 X 35 = 52.5
Add total to baseline (24 CWS)
24 CWS + 52.5 = 76.5 goal in 35 weeks
Example of a Graphed Goal
CBM Writing Fall 2011
0
10
20
30
40
50
60
70
80
90
100
Date
TW
W &
WS
C
Baseline TWW
Baseline
Baseline
(Median Data-point)
Performance Goal
Goal Iine is something to compare data against
Using an Intra-Individual Framework to Set Goals
Intra-individual framework: Weekly rate of improvement is calculated using at least eight data
points. Subtract the lowest from the highest score
EX. 12, 16, 15, 19, 16, 21, 26, 24 (26-12=14) Divide the difference by the number of weeks the data was collected 14
÷ 8 = 1.75 This baseline rate of growth is multiplied by 1.5 (1.75 x 1.5= 2.625) This number is multiplied by the number of weeks left until the end of
the year 2.625x16 weeks = 42 This number is then added to the median score of the first eight data
points used to calculate the baseline growth rate 12, 15, 16, 16, 19, 21, 24, 26 = 17.5 median score 42 + 17.5 = 59.5
This is our end of year performance goal ( 59.5 round up to 60)
Example: Using an Intra-Individual Framework to Set Goals
First eight scores: 3, 2, 5, 6, 5, 5, 7, 4. Difference between the lowest and highest
score 7 - 2 = 5 Divide difference by number of weeks of data
points 5 ÷ 8 = .625 Multiply by 1.5 (1.5 x 0.625 = 0.9375) Multiply by weeks left: 0.9375 × 14 = 13.125 Product is added to the median: 13.125 + 5 =
18.125. The end-of-year performance goal is 18.
Summary
CBM-W is designed to be a simple, efficient approach to monitoring student progress in writing
CBM-W provides useful information about whether instruction is benefiting student or whether a change is necessary