administration and scoring of spelling curriculum-based measurement (cbm) for use in general outcome...
TRANSCRIPT
![Page 1: Administration and Scoring of Spelling Curriculum-Based Measurement (CBM) for use in General Outcome Measurement](https://reader036.vdocuments.mx/reader036/viewer/2022062511/5513cf765503463a298b5074/html5/thumbnails/1.jpg)
![Page 2: Administration and Scoring of Spelling Curriculum-Based Measurement (CBM) for use in General Outcome Measurement](https://reader036.vdocuments.mx/reader036/viewer/2022062511/5513cf765503463a298b5074/html5/thumbnails/2.jpg)
Administration and Scoring of
Spelling
Curriculum-Based Measurement (CBM)for use in General Outcome Measurement
![Page 3: Administration and Scoring of Spelling Curriculum-Based Measurement (CBM) for use in General Outcome Measurement](https://reader036.vdocuments.mx/reader036/viewer/2022062511/5513cf765503463a298b5074/html5/thumbnails/3.jpg)
Part of a training series developed to accompany the AIMSweb Part of a training series developed to accompany the AIMSweb Improvement System.Improvement System.
Purpose is to provide the background information and data collection Purpose is to provide the background information and data collection procedures necessary to administer and score Spelling Curriculum procedures necessary to administer and score Spelling Curriculum Based Measurement (S-CBM).Based Measurement (S-CBM).
Designed to accompany: Designed to accompany:
• AIMSWeb Training Workbook: Administration and Scoring of AIMSWeb Training Workbook: Administration and Scoring of Spelling CBM for Use in General Outcome MeasurementSpelling CBM for Use in General Outcome Measurement
• Standard Spelling ProbesStandard Spelling Probes
• AIMSweb Web-based SoftwareAIMSweb Web-based Software
• Training VideoTraining Video
Overview of Spelling-CBM Assessment Training Session
![Page 4: Administration and Scoring of Spelling Curriculum-Based Measurement (CBM) for use in General Outcome Measurement](https://reader036.vdocuments.mx/reader036/viewer/2022062511/5513cf765503463a298b5074/html5/thumbnails/4.jpg)
Brief review of Curriculum Based Measurement (CBM) Brief review of Curriculum Based Measurement (CBM) and General Outcome Measurement (GOM).and General Outcome Measurement (GOM).• Its PurposeIts Purpose
• Its OriginsIts Origins
Learn how to administer and score Spelling - Curriculum Based Learn how to administer and score Spelling - Curriculum Based Measurement (S-CBM) through applied practice.Measurement (S-CBM) through applied practice.
Training Session Goals
![Page 5: Administration and Scoring of Spelling Curriculum-Based Measurement (CBM) for use in General Outcome Measurement](https://reader036.vdocuments.mx/reader036/viewer/2022062511/5513cf765503463a298b5074/html5/thumbnails/5.jpg)
General Outcome Measurements from Other Fields
• Medicine measures blood pressure, weight, temperature, weight.Medicine measures blood pressure, weight, temperature, weight.
• Federal Reserve Board measures the Consumer Price Index.Federal Reserve Board measures the Consumer Price Index.
• Wall Street measures the Dow-Jones Industrial Average.Wall Street measures the Dow-Jones Industrial Average.
• Companies report earnings per share.Companies report earnings per share.
• McDonald’s measures how many hamburgers they sell.McDonald’s measures how many hamburgers they sell.
![Page 6: Administration and Scoring of Spelling Curriculum-Based Measurement (CBM) for use in General Outcome Measurement](https://reader036.vdocuments.mx/reader036/viewer/2022062511/5513cf765503463a298b5074/html5/thumbnails/6.jpg)
Common Characteristics of General Outcome Measures
• Simple, accurate, and reasonably inexpensive in terms of time and Simple, accurate, and reasonably inexpensive in terms of time and
materials. materials.
• Considered so important to doing business well that they are Considered so important to doing business well that they are
routine. routine.
• Are collected on an ongoing and frequent basis. Are collected on an ongoing and frequent basis.
• Shape/inform a variety of important decisions.Shape/inform a variety of important decisions.
![Page 7: Administration and Scoring of Spelling Curriculum-Based Measurement (CBM) for use in General Outcome Measurement](https://reader036.vdocuments.mx/reader036/viewer/2022062511/5513cf765503463a298b5074/html5/thumbnails/7.jpg)
CBM is Designed to Provide Educators With…
The same kind of evaluation technology as other professions!The same kind of evaluation technology as other professions!
Powerful measures that are:Powerful measures that are:
• SimpleSimple
• AccurateAccurate
• Efficient indicators of student achievement that guide/inform a variety of Efficient indicators of student achievement that guide/inform a variety of decisionsdecisions
![Page 8: Administration and Scoring of Spelling Curriculum-Based Measurement (CBM) for use in General Outcome Measurement](https://reader036.vdocuments.mx/reader036/viewer/2022062511/5513cf765503463a298b5074/html5/thumbnails/8.jpg)
Not a measure of performance on a particular lesson or unitNot a measure of performance on a particular lesson or unit
A broader indicatorA broader indicator
– Level of general spelling skills relative to peersLevel of general spelling skills relative to peers
– Degree of progress in spelling relative to self or peersDegree of progress in spelling relative to self or peers
General Outcome Measures in Spelling
![Page 9: Administration and Scoring of Spelling Curriculum-Based Measurement (CBM) for use in General Outcome Measurement](https://reader036.vdocuments.mx/reader036/viewer/2022062511/5513cf765503463a298b5074/html5/thumbnails/9.jpg)
• ““Signs” of general achievementSigns” of general achievement– Not everythingNot everything– Important thingsImportant things
• Standardized testsStandardized tests– AdministrationAdministration– Scoring Scoring – InterpretationInterpretation
• ResearchedResearched– Reliability and validityReliability and validity– Accuracy of measurementAccuracy of measurement
• Sensitive to improvement in short periods of timeSensitive to improvement in short periods of time
• Designed to be as short as possible (do-ability)Designed to be as short as possible (do-ability)
• Linked to decision making for promoting positive achievementLinked to decision making for promoting positive achievement
Things to Remember About General Outcome Measures in Spelling
![Page 10: Administration and Scoring of Spelling Curriculum-Based Measurement (CBM) for use in General Outcome Measurement](https://reader036.vdocuments.mx/reader036/viewer/2022062511/5513cf765503463a298b5074/html5/thumbnails/10.jpg)
Traditional spelling tests versus S-CBMTraditional spelling tests versus S-CBM
• Shorter duration (2 minutes)Shorter duration (2 minutes)
• Spelling words dictated carefully at set paceSpelling words dictated carefully at set pace
• Random sample from a large pool of graded spelling wordsRandom sample from a large pool of graded spelling words
• Combination of words Combination of words
– Words students have already learned Words students have already learned
– Words they will learn to spell in upcoming weeks or monthsWords they will learn to spell in upcoming weeks or months
• Exact and sensitive way to assess spelling improvementExact and sensitive way to assess spelling improvement
– Correct letter sequencesCorrect letter sequences
– Correct word spellingsCorrect word spellings
Spelling CBM
![Page 11: Administration and Scoring of Spelling Curriculum-Based Measurement (CBM) for use in General Outcome Measurement](https://reader036.vdocuments.mx/reader036/viewer/2022062511/5513cf765503463a298b5074/html5/thumbnails/11.jpg)
![Page 12: Administration and Scoring of Spelling Curriculum-Based Measurement (CBM) for use in General Outcome Measurement](https://reader036.vdocuments.mx/reader036/viewer/2022062511/5513cf765503463a298b5074/html5/thumbnails/12.jpg)
TimingTiming
– 2 minutes2 minutes
– New word dictated every 7 seconds New word dictated every 7 seconds
(every 10 seconds: grades 1 & 2)(every 10 seconds: grades 1 & 2)
Arrangements Arrangements
– Group Group
– IndividualIndividual
What is scoredWhat is scored
– Number of letters spelled in correct sequenceNumber of letters spelled in correct sequence
– Number of words spelled correctlyNumber of words spelled correctly
Summary of Spelling Conditions
![Page 13: Administration and Scoring of Spelling Curriculum-Based Measurement (CBM) for use in General Outcome Measurement](https://reader036.vdocuments.mx/reader036/viewer/2022062511/5513cf765503463a298b5074/html5/thumbnails/13.jpg)
1. Students have lined paper numbered 1–12 (Grades 1 & 2) or 1–17
(Grade 3-8)and a pencil.
2. Say, “We’re going to take a 2-minute spelling test. I am going to
say some words that I want you to spell on the sheet of paper in
front of you. Write the first word on the first line, the second
word on the second line and so on. I’ll give you [10] [7] seconds
to spell each word. When I say the next word, write it down,
even if you haven’t finished the last one. You will receive credit
for each correct letter written. Are there any questions? [Pause]
Let’s begin.”
3. Say the first word and start your stopwatch (or tape recorder)
Standard Directions
![Page 14: Administration and Scoring of Spelling Curriculum-Based Measurement (CBM) for use in General Outcome Measurement](https://reader036.vdocuments.mx/reader036/viewer/2022062511/5513cf765503463a298b5074/html5/thumbnails/14.jpg)
4. Say each word twice. Use homonyms in a sentence (provided)
5. Say a new word every 10 or 7 seconds
6. Say, “You should be on the fifth word which is…”
7. Monitor students to ensure they are writing on the correct line
8. Do not respond to student questions during the test
9. After two minutes, say, “Stop. Put your pencils down.”
Good to begin with short practice test to reduce novelty.
Shortened directions available for well-experienced students.
Standard Directions
![Page 15: Administration and Scoring of Spelling Curriculum-Based Measurement (CBM) for use in General Outcome Measurement](https://reader036.vdocuments.mx/reader036/viewer/2022062511/5513cf765503463a298b5074/html5/thumbnails/15.jpg)
Benchmark: 3 standards lists for each grade (1–8)
• Screen and identify students who are at risk
• Monitor progress and improvement in fall, winter, and spring
• Guide program evaluation decisions & improve accountability
Progress Monitor: 30 standard lists for each grade
• Provide practical way to write individualized progress goals
• Monitor progress and determine rate of improvements
• Provide accountability data
Standard Assessment Spelling Lists
![Page 16: Administration and Scoring of Spelling Curriculum-Based Measurement (CBM) for use in General Outcome Measurement](https://reader036.vdocuments.mx/reader036/viewer/2022062511/5513cf765503463a298b5074/html5/thumbnails/16.jpg)
An Example
![Page 17: Administration and Scoring of Spelling Curriculum-Based Measurement (CBM) for use in General Outcome Measurement](https://reader036.vdocuments.mx/reader036/viewer/2022062511/5513cf765503463a298b5074/html5/thumbnails/17.jpg)
Administration and Scoring of Spelling-CBM
What Examiners Need To Do…
• Before testing students
• While testing students
• After testing students
![Page 18: Administration and Scoring of Spelling Curriculum-Based Measurement (CBM) for use in General Outcome Measurement](https://reader036.vdocuments.mx/reader036/viewer/2022062511/5513cf765503463a298b5074/html5/thumbnails/18.jpg)
BEFORE Testing
Student needs:• Numbered, lined paper with
the exact number of lines (12 or 17)– Saves time– Reduces potential
confusion– Makes scoring easier– Answer sheets available
as PDF documents• Pencil
![Page 19: Administration and Scoring of Spelling Curriculum-Based Measurement (CBM) for use in General Outcome Measurement](https://reader036.vdocuments.mx/reader036/viewer/2022062511/5513cf765503463a298b5074/html5/thumbnails/19.jpg)
BEFORE Testing
Examiner needs• List of words with time
interval clearly marked– Includes sentences
for homonyms– Improves ease of
scoring• Timing device
![Page 20: Administration and Scoring of Spelling Curriculum-Based Measurement (CBM) for use in General Outcome Measurement](https://reader036.vdocuments.mx/reader036/viewer/2022062511/5513cf765503463a298b5074/html5/thumbnails/20.jpg)
Gather Assessment Materials
Download and review S-CBM Manual
Download and reproduce student answer sheets
Download Standard Spelling Assessment Lists– Curriculum independent– Allow decision making – Minimize between-teacher, between-school differences
Stopwatch or timer– Clock on the wall is inaccurate, inefficient
Timing tape– Audio tape with recorded tones at prescribed intervals– Allows better attention to words and students
List of students to assess
![Page 21: Administration and Scoring of Spelling Curriculum-Based Measurement (CBM) for use in General Outcome Measurement](https://reader036.vdocuments.mx/reader036/viewer/2022062511/5513cf765503463a298b5074/html5/thumbnails/21.jpg)
Arrange Testing Environment
Quiet, without distractions
May be 1:1, small group, whole classroom
Consider a simple practice test
– Use very simple words
– Allows students to adjust to pace
– Show student how they get credit for unfinished words
Be sure you can monitor
– Especially those students who “get lost” easily
– Be available to point to the number of the next word
![Page 22: Administration and Scoring of Spelling Curriculum-Based Measurement (CBM) for use in General Outcome Measurement](https://reader036.vdocuments.mx/reader036/viewer/2022062511/5513cf765503463a298b5074/html5/thumbnails/22.jpg)
Things to Keep in Mind
Keep examiner out of the process– Keep number of words said to minimum– Some students try to spell everything said– Only homonyms have sentences (say word, say sentence, repeat
word)
Roam and project– Loud enough for all students to hear– Be sure head is turned toward students
Monitor, monitor, monitor– Alert for lost students
• Before they become frustrated and quit • Before they call out that they are lost
– Available to point to the next number
![Page 23: Administration and Scoring of Spelling Curriculum-Based Measurement (CBM) for use in General Outcome Measurement](https://reader036.vdocuments.mx/reader036/viewer/2022062511/5513cf765503463a298b5074/html5/thumbnails/23.jpg)
Things to Keep in Mind
Say the number of the word periodically– Younger students, say,
“You should be on the fifth (tenth) word which is…”– Older students find this intrusive, so don’t say the numbers
Avoid answering questions during testing– Students may comment: You’re going too fast, I’m lost, What number
are we on?– Deal with this in practice tests– Point to orient student
Adhere to end of timing
Interruptions– Re-administer later with another spelling list– Consider that some students need small group or 1:1
![Page 24: Administration and Scoring of Spelling Curriculum-Based Measurement (CBM) for use in General Outcome Measurement](https://reader036.vdocuments.mx/reader036/viewer/2022062511/5513cf765503463a298b5074/html5/thumbnails/24.jpg)
Helpful Hints
• Ensure a quiet and distraction-free testing environment – Students must be able to hear the examiner– Offer no assistance, even when you know the child is capable of more
• Complete reliability checks – Before and after data collection with all examiners to ensure reliable
administration and scoring– Additional reliability checks at least once a year– Prevent examiner “drift”
• Use same Benchmark examiner for the same students. – Younger children tend to perform better with familiar examiners.
![Page 25: Administration and Scoring of Spelling Curriculum-Based Measurement (CBM) for use in General Outcome Measurement](https://reader036.vdocuments.mx/reader036/viewer/2022062511/5513cf765503463a298b5074/html5/thumbnails/25.jpg)
Follow the standardized directions:Follow the standardized directions:
• Spelling-CBM is a standardized test
• Administer the assessment with consistency
• Remember it’s about testing, not teaching
• Don’t teach or correct
• Never practice using the standard spelling lists
Things You Need DURING Testing
![Page 26: Administration and Scoring of Spelling Curriculum-Based Measurement (CBM) for use in General Outcome Measurement](https://reader036.vdocuments.mx/reader036/viewer/2022062511/5513cf765503463a298b5074/html5/thumbnails/26.jpg)
Things to Do AFTER Testing
Score Immediately!
Determine the words spelled correctly
• Circle the number beside the words spelled correctly
• Award full points
Determine the correct letter sequences for misspelled words
(see next slide)
• More likely to show change
• Provides diagnostic and positive feedback
![Page 27: Administration and Scoring of Spelling Curriculum-Based Measurement (CBM) for use in General Outcome Measurement](https://reader036.vdocuments.mx/reader036/viewer/2022062511/5513cf765503463a298b5074/html5/thumbnails/27.jpg)
Spelling Scoring
• Correct: Count each sequence of 2 letters• Spaces before and after the word count as “letters”
– Consider: break through versus breakthrough
• Example: cat
c a t
1 2 3 4 sequences correct
![Page 28: Administration and Scoring of Spelling Curriculum-Based Measurement (CBM) for use in General Outcome Measurement](https://reader036.vdocuments.mx/reader036/viewer/2022062511/5513cf765503463a298b5074/html5/thumbnails/28.jpg)
Spelling Scoring
store _ s t o r e _ (6)
stor _ s t o r e _ (6)
fishing _ f i s h i n g _ (8)
fisshing _ f i s s h i n g _ (8)
cat _ c a t _ (4)
catz _ c a t z _ (4)
![Page 29: Administration and Scoring of Spelling Curriculum-Based Measurement (CBM) for use in General Outcome Measurement](https://reader036.vdocuments.mx/reader036/viewer/2022062511/5513cf765503463a298b5074/html5/thumbnails/29.jpg)
Spelling Scoring
store _^s^t^o^r^e^_ 6 (6)
stor _^s^t^o^r e _ 4 (6)
fishing _^f^i^s^h^i^n^g^_ 8 (8)
fisshing _^f^i^s s h^i^n^g^_ 7 (8)
cat _^c^a^t^_ 4 (4)
catz _^c^a^t z _ 3 (4)
![Page 30: Administration and Scoring of Spelling Curriculum-Based Measurement (CBM) for use in General Outcome Measurement](https://reader036.vdocuments.mx/reader036/viewer/2022062511/5513cf765503463a298b5074/html5/thumbnails/30.jpg)
Spelling Scoring, Continued
• Tricks:
– Use master sheet with all correct sequences pre-counted.
– Then you only need to count the scoring for the few incorrect sequences (errors) on each paper.
• A complete list of spelling rules can be found in the Appendix of A complete list of spelling rules can be found in the Appendix of
your workbook. your workbook.
• Please review and become more familiar with the more unusual Please review and become more familiar with the more unusual
errors.errors.
![Page 31: Administration and Scoring of Spelling Curriculum-Based Measurement (CBM) for use in General Outcome Measurement](https://reader036.vdocuments.mx/reader036/viewer/2022062511/5513cf765503463a298b5074/html5/thumbnails/31.jpg)
Summary
You now have the building You now have the building blocks to begin blocks to begin Spelling -Spelling - CBM CBM to ensure literacy to ensure literacy growth.growth.
• Practice to AutomaticityPractice to Automaticity — — You’ll get more efficientYou’ll get more efficient
• Get Checked OutGet Checked Out by a by a colleague using the colleague using the Accuracy of Implementation Accuracy of Implementation Rating Scale (Rating Scale (AIRSAIRS) for ) for accuracy and efficiencyaccuracy and efficiency
• Stay in TuneStay in Tune by periodically by periodically checking AIRSchecking AIRS
![Page 32: Administration and Scoring of Spelling Curriculum-Based Measurement (CBM) for use in General Outcome Measurement](https://reader036.vdocuments.mx/reader036/viewer/2022062511/5513cf765503463a298b5074/html5/thumbnails/32.jpg)
Let’s Practice
![Page 33: Administration and Scoring of Spelling Curriculum-Based Measurement (CBM) for use in General Outcome Measurement](https://reader036.vdocuments.mx/reader036/viewer/2022062511/5513cf765503463a298b5074/html5/thumbnails/33.jpg)
Practice: Spelling
• Open your workbooks to Spelling Practice List 1: Elida
• Score this spelling test
– Words spelled correctly
– Correct letter sequences
![Page 34: Administration and Scoring of Spelling Curriculum-Based Measurement (CBM) for use in General Outcome Measurement](https://reader036.vdocuments.mx/reader036/viewer/2022062511/5513cf765503463a298b5074/html5/thumbnails/34.jpg)
Answer:Elida
WSC = 13CLS = 121
![Page 35: Administration and Scoring of Spelling Curriculum-Based Measurement (CBM) for use in General Outcome Measurement](https://reader036.vdocuments.mx/reader036/viewer/2022062511/5513cf765503463a298b5074/html5/thumbnails/35.jpg)
If the Student Misses a Few Words…
• Any Any missing single lettermissing single letter equals 2 missing CLS. equals 2 missing CLS.
• Any Any 2 letters together that are missing2 letters together that are missing equals 3 missing CLS equals 3 missing CLS
• Any Any incorrectly inserted letterincorrectly inserted letter equals 1 missing CLS equals 1 missing CLS
NEXT — Score Practice List 2: Lester
![Page 36: Administration and Scoring of Spelling Curriculum-Based Measurement (CBM) for use in General Outcome Measurement](https://reader036.vdocuments.mx/reader036/viewer/2022062511/5513cf765503463a298b5074/html5/thumbnails/36.jpg)
Answer:Lester
WSC = 0CLS = 52
NEXT —Score Practice List 3: Calvin
![Page 37: Administration and Scoring of Spelling Curriculum-Based Measurement (CBM) for use in General Outcome Measurement](https://reader036.vdocuments.mx/reader036/viewer/2022062511/5513cf765503463a298b5074/html5/thumbnails/37.jpg)
Answer:Calvin
WSC = 9CLS = 109
![Page 38: Administration and Scoring of Spelling Curriculum-Based Measurement (CBM) for use in General Outcome Measurement](https://reader036.vdocuments.mx/reader036/viewer/2022062511/5513cf765503463a298b5074/html5/thumbnails/38.jpg)
Determining Inter-Scorer Agreement
Agreements / (Agreements + Disagreements) x 100Agreements / (Agreements + Disagreements) x 100
Example:
Examiner 1 = 90 CLS90 CLS
Examiner 2 = 92 92 CLS
Inter-Scorer Agreement = 98%98%
NEXT — Score Practice List 4: Lucinda and Compute Inter-Scorer Agreement with a partner
![Page 39: Administration and Scoring of Spelling Curriculum-Based Measurement (CBM) for use in General Outcome Measurement](https://reader036.vdocuments.mx/reader036/viewer/2022062511/5513cf765503463a298b5074/html5/thumbnails/39.jpg)
Answer:Lucinda
WSC = 2CLS = 70
How is your Inter-Scorer Agreement?
![Page 40: Administration and Scoring of Spelling Curriculum-Based Measurement (CBM) for use in General Outcome Measurement](https://reader036.vdocuments.mx/reader036/viewer/2022062511/5513cf765503463a298b5074/html5/thumbnails/40.jpg)
The End