contentssss.usf.edu/resources/presentations/2010/fasp_summer_inst2010... · contents steps for...

Using CBM for Progress Monitoring in Math i

Contents

Steps for Conducting Curriculum-Based Measurement ...................................................................... 1 Step 1: How to Place Students in a Math Curriculum-Based Measurement Task for Progress Monitoring.............................................................................................................................. 2 Step 2: How to Identify the Level of Material for Monitoring Progress........................................ 3 Step 3: How to Administer and Score Math Curriculum-Based Measurement ........................... 4 Step 4: How to Graph Scores.............................................................................................................. 49 Step 5: How to Set Ambitious Goals ................................................................................................. 50 Step 6: How to Apply Decision Rules to Graphed Scores to Know When to Revise Programs and Increase Goals............................................................................................................. 52 Step 7: How to Use the Curriculum-Based Measurement Database Qualitatively to Describe Student Strengths and Weaknesses .................................................................................. 58 Curriculum-Based Measurement Case Study 1: Alexis ................................................................. 61 Curriculum-Based Measurement Case Study 2: Marcus ............................................................... 63

Appendix A: Curriculum-Based Measurement Materials ................................................................. 65

Appendix B: Resources............................................................................................................................ 68

Using CBM for Progress Monitoring in Math 1

Steps for Monitoring Progress in Math Using Curriculum-Based Measurement

Step 1: How to Place Students in a Math Curriculum-Based Measurement Task for Progress Monitoring (page 2)

Step 2: How to Identify the Level of Material for Monitoring Progress (page 3)

Step 3: How to Administer and Score Math Curriculum-Based Measurement Probes (page 4)

• Computation (page 4)

• Concepts and Applications (page 15)

• Number Identification (page 28)

• Quantity Discrimination (page 35)

• Missing Number (page 42)

Step 4: How to Graph Scores (page 49)

Step 5: How to Set Ambitious Goals (page 50)

Step 6: How to Apply Decision Rules to Graphed Scores to Know When to Revise Programs and Increase Goals (page 52)

Step 7: How to Use the Curriculum-Based Measurement Database Qualitatively to Describe Students’ Strengths and Weaknesses (page 58)


Step 1: How to Place Students in a Math Curriculum-Based Measurement Task for Progress Monitoring

The first decision for implementing CBM in math is to decide which task is developmentally appropriate to monitor each student for over the academic year. For students who are developing at a typical rate in math, the correct CBM tasks are as follows.

At kindergarten or first grade, the following probes can be administered alone or in combination. Number Identification asks students to identify numeric characters. Quantity Discrimination asks students to identify the bigger number in a pair of numbers. Missing Number asks students to identify the missing number in a sequence of four numbers.

It should be noted that the Number Identification, Quantity Discrimination, and Missing Number tasks presented in the manual have not been approved by the Technical Review Committee at this time. However, the measures (as presented here) have been researched for the past 3 years (go to www.studentprogress.org for updated technical reports) and have demonstrated moderate to strong reliability and validity (Lembke & Foegen, in preparation). Data indicate that the measures show promise as indicators of student performance in math, and data has also been collected across a school year to demonstrate that the measures model student growth across time (Lembke, Foegen, Whittaker, & Hampton, in preparation).

Earlier work with similar math measures has been conducted by Clarke and Shinn (2004), with measures that included Number Identification, Missing Number, and Quantity Discrimination, among others. Similar findings with respect to reliability and validity were observed in the Clarke and Shinn and Lembke and Foegen research. In addition, VanDerHayden, Witt, Naquin, and Noell (2001) used measures of early math like writing numbers and circling the correct number to assess the readiness skills of kindergarten students.

For further information on technical adequacy of the measures, please e-mail Dr. Erica Lembke ([email protected]) or check the Research Institute on Progress Monitoring (RIPM) Web site (www.progressmonitoring.org). As work continues on the development of these measures, the Web site will have updated technical reports that outline reliability and validity results with respect to effects of the measures on student performances and progress.

CBM Computation probes (Grades 1–6) and CBM Concepts and Applications probes (Grades 2–6) have demonstrated reliability, validity, and utility for instructional decision making. These can be administered alone or in combination. Students in the earlier grades should use the Computation probes until the Concepts and Applications probes are appropriate for the grade-level material from the curriculum. For Grades 1–6, once you select a task for CBM progress monitoring, stick with that task (and level of probes) for the entire year.


Step 2: How to Identify the Level of Material for Monitoring Progress

For Computation and Concepts and Applications, teachers use CBM probes at the student’s current grade level. However, if a student is performing well below grade-level expectations, then he or she may need to use lower-grade probes. If teachers are worried that a student is too delayed in math for the appropriate grade-level probe, then they can find the appropriate CBM level by following these steps.

1. Determine the grade-level probe at which you expect the student to perform in math competently by year’s end. On two separate days, administer a CBM test (either Computation or Concepts and Applications) at the grade level lower than the student’s grade-appropriate level. Use the correct time limit for the test at the lower grade level, and score the tests according to the directions.

• If the student’s average score is between 10 and 15 digits or blanks, then use this lower grade-level test.

• If the student’s average score is less than 10 digits or blanks, then move down one more grade level or stay at the original lower grade level and repeat this procedure.

• If average is greater than 15 digits or blanks, then reconsider grade-appropriate material.

Maintain the student on this grade level for the purpose of progress monitoring for the entire school year.


Step 3: How to Administer and Score Math Curriculum-Based Measurement

With math CBM, students work on selected math problems for a set amount of time. After the probes are administered, the teacher scores each probe and graphs the score on a student graph. The CBM score is a general overall indicator of the student’s math competency.

In math, the following CBM tasks are available at these grade levels.

• Kindergarten and Grade 1:

– Number Identification

– Quantity Discrimination

– Missing Number

• Grades 1–6:

– Computation

• Grades 2–6:

– Concepts and Applications

A description of each of these CBM tasks follows. Information on how to obtain the CBM materials for each task is available in Appendix A. For monitoring progress, you need a different, alternate form for each test administration. These alternate forms are available as described in Appendix A. The sample probes provided in this manual are only illustrative.

Computation

The math CBM Computation probes include tests at each grade level for Grades 1–6. Each test consists of 25 math computation problems representing the year-long, grade-level math computation curriculum. Within each grade level, the type of problems represented on each test remains constant from test to test. For example, at Grade 3, each Computation test includes five multiplication facts with factors zero through five and four multiplication facts with factors six through nine. However, the facts to be tested and their positions on the test are selected randomly. Other types of problems remain similarly constant.

CBM Computation can be administered to a group of students at one time. The administrator presents each student with a CBM Computation test (Figures 1–3). Students have a set amount of time to answer the math problems on the Computation test. Timing the CBM Computation test correctly is critical to ensure consistency from test to test. See Figure 4 for the time limit for each grade. The administrator times the students during the test and scores the tests later. Student performance on the Computation test is scored as the total number of digits correct.


Figure 1. Sample First-Grade Curriculum-Based Measurement Computation Probe


Figure 2. Sample Second-Grade Curriculum-Based Measurement Computation Probe


Figure 3. Sample Third-Grade Curriculum-Based Measurement Computation Probe


Figure 4. Time Limits for Curriculum-Based Measurement Computation

Grade Time limit

1 2 minutes

2 2 minutes

3 3 minutes

4 3 minutes

5 5 minutes

6 6 minutes

Administration of CBM Computation is standardized according to a script.

Teacher: It’s time to take your weekly math test. As soon as I give you the test, write your first name, your last name, and the date. After you’ve written your name and the date on the test, turn your paper over and put your pencil down so I know you are ready.

I want you to do as many problems as you can. Work carefully and do the best you can. Remember, start at the first problem and work left to right. Some problems will be easy for you; others will be harder. When you come to a problem you know you can do, do it right away. When you come to a problem that’s hard for you, skip it, and come back to it later.

Go through the entire test doing the easy problems. Then go back and try the harder ones. Remember that you get points for getting part of the problem right. So, after you have done all the easy problems, try the harder problems. Do this even if you think you can’t get the whole problem right. (For appropriate grade levels, say, “Remember to reduce fractions to the lowest terms unless the problem specifies to do something differently. Be sure to write out your remainder if the division problem has one.”)

When I say, “Begin,” turn your test over and start to work. Work for the whole test time. You should have enough room to do your work in each block. Write your answers so I can read them. If you finish early, check your answers. When I say, “Stop,” put your pencil down and turn your test face down.

When scoring CBM Computation, students receive 1 point for each correctly answered digit. The number of correct digits within the set time limit is the student’s score. Although we can score total problems correct, scoring each digit correct in the answer is a more sensitive index of student change. We typically can evaluate overall student growth (or deterioration) earlier by evaluating correct digits in the answers (Figure 5). Scoring keys are available from the publisher.

For Computation tests, score from right to left for addition, subtraction, and multiplication problems (i.e., following the same direction the student would use when calculating the problem) (Figure 6).


For division, score from left to right (just as the student would work the problem). Students should be instructed to write out remainders when appropriate. If a remainder is present, then score the remainder portion from right to left. The remainder would be calculated by subtracting, and the student would work from right to left (Figures 7 and 8).

For decimal problems, start at the decimal point and move outward, both to the right and left of the decimal point. Placement of the decimal is the most critical feature in decimal problems. Evaluating the actual operation used (e.g., subtraction) could be evaluated from other whole-number computational problems (Figures 9 and 10).

For fractions, evaluate each digit in the whole number part separately from the fractional part, and evaluate each digit in the numerator separately from the denominator, moving from right to left for all parts (Figures 9 and 11). Add the number of digits obtained from each part to get the total correct for that fraction. Students should be instructed to reduce (or simplify) fractional answers to the lowest terms (unless specified differently in the problem).

Figure 5. Correct Digits: Evaluate Each Numeral in Every Answer


Figure 6. Scoring Different Operations

Figure 7. Division Problems With Remainders

9


Figure 8. Scoring Examples: Division With Remainders

Figure 9. Scoring Decimals and Fractions


Figure 10. Scoring Examples: Decimals

Figure 11. Scoring Examples: Fractions


Look at the following fifth-grade CBM Computation score sheet (Figure 12). Samantha answered 13 problems correctly in 5 minutes but got 49 digits correct. The teacher would graph 49 as Samantha’s score for this probe.

Figure 12. Samantha’s Computation Probe

X


Let’s practice administering the Computation probe. Use the following Computation student probe (Figure 13) and answer sheet (Figure 14). The administrator reads the directions to the student, and the student has 6 minutes to answer as many items as possible. If the student gets every digit correct on this probe, then the student earns a total of 104 digits correct (or 25 problems). Then, the teacher graphs 104 as the student’s score for this probe.

Figure 13. Practice Computation Student Probe


Figure 14. Practice Computation Answer Sheet

• Possible score of 21 digits correct in first row

• Possible score of 23 digits correct in the second row

• Possible score of 21 digits correct in the third row

• Possible score of 18 digits correct in the fourth row

• Possible score of 21 digits correct in the fifth row

• Total possible digits on this probe: 104

See Appendix A for information about obtaining CBM Computation probes.

Concepts and Applications

The math CBM Concepts and Applications probes include tests at each grade level for Grades 2–6. Each test consists of 18–25 math computation problems representing the year-long, grade-level math Concepts and Applications curriculum. Each test is 3 pages long (Figure 15). Within each grade level, the type of problems represented on each test remains constant from test to test. For example, at Grade 3, every Concepts and Applications test includes two problems dealing with charts and graphs and three problems dealing with number concepts. Other types of problems remain similarly constant. The placement of the various types of items is random from test to test, and the actual problems differ from test to test.

CBM Concepts and Applications can be administered to a group of students at one time. The administrator presents each student with a CBM Concepts and Applications test. Students have a set amount of time to answer the math problems on the test. Timing the CBM Concepts and Applications test correctly is critical to ensure consistency from test to test. See Figure 16 for the time limit at each grade. The administrator times the students during the test and scores the tests later. Student performance is scored as the total number of blanks correct.


Figure 15. Sample Curriculum-Based Measurement Concepts and Applications Probe


Figure 15. Sample Curriculum-Based Measurement Concepts and Applications Probe (continued)


Figure 16. Time Limits for Concepts and Applications Probes

Grade Time limit Number of blanks

2 8 minutes 18 blanks




6 7 minutes 24 or 25 blanks


Administration of the CBM Concepts and Applications for Grades 2–6 is standardized, based on the following script:

Teacher: It’s time to take your weekly math test. As soon as I give you the test, write your first name, your last name, and the date. After you’re written your name and the date on the test, turn your paper over and put your pencil down so I know you are ready.

I want you to do as many problems as you can. Work carefully and do the best you can. Remember, start at the first problem, work down the first column and then down the second column. Then move on to the next page. Some problems will be easy for you; others will be harder. When you come to a problem you know you can do, do it right away. When you come to a problem that’s hard for you, skip it, and come back later. Remember, some problems have more than one blank. You get credit for each blank that you answer, so be sure to fill in as many blanks as you can. The answers to some word problems may be an amount of money. When you write your answer to a money problem, be sure to use the correct symbols for money in order to get credit for your answer. (Note that no “digits” are awarded for the actual symbols indicating money. Only the digits in the answers will be scored. However, without the monetary symbol for appropriate word problems, the answer should not be scored).

Go through the entire test doing the easy problems. Then go back and try the harder ones. When I say, “Begin,” turn your test over and start to work. Work for the whole test time. Write your answers so I can read them! If you finish early, then check your answers. When I say, “Stop,” put your pencil down and turn your test face down.

When scoring CBM Concepts and Applications, students receive 1 point for each correctly answered blank. The number of correct blanks within the set time limit is the student score.

For Concepts and Applications problems that involve a multiple-choice response (e.g., selecting an equals sign or a type of inequality sign), assign one “blank” for the answer. Some problems contain more than one answer. Score each answer blank separately as a correct blank.

Look at the following fourth-grade CBM Concepts and Applications score sheet (Figure 17). Quinten answered a total of 24 blanks correctly in 6 minutes. Quinten’s math score for this probe is 24. Note that on problem 9, Quinten missed both blanks. The whole problem is marked wrong. However, Quinten’s teacher could have marked each blank separately.


Figure 17. Quinten’s Concepts and Applications Probe


Figure 17. Quinten’s Concepts and Applications Probe (continued)


Let’s practice administering the Concepts and Applications probe for fifth grade. Use the following Concepts and Applications student probe (Figures 18–20) and answer sheet Figure 21). The administrator reads the directions to the student, and the student has 7 minutes to answer as many items as possible. Subtract any incorrect blanks from the total number of blanks attempted in 7 minutes. The number of blanks the student got correct is the student’s Concepts and Applications score. If the student has every blank correct, then his or her score is 32.

Figure 18. Practice Fifth-Grade Concepts and Applications Student Probe—Page 1


Figure 21. Practice Fifth-Grade Concepts and Applications Answer Sheet

Problem Answer

1 54 sq. ft

2 66,000

3 A center C diameter

4 28.3 miles

5 7

6 P 7 N 10

7 0 $5 bills 4 $1 bills 3 quarters

8 1 millions place 3 ten thousands place

9 697

10 3

11 A ∠ADC C ∠BFE

12 0.293

13 < >

14 28 hours

15 790,053

16 451 CDLI

17 7

18 $10.00 in tips 20 more orders

19 4.4

20 > <

21 5/6 dogs or cats

22 1 m

23 12 ft See Appendix A for information about obtaining CBM Concepts and Applications probes.


Number Identification

The Number Identification test for kindergarten or first-grade students consists of 84 items that require the student to orally identify numbers between 0 and 100. Number Identification includes three alternate forms. The Number Identification measures have been researched as screening tools, but schools may want to use them for progress monitoring.

Number Identification is administered individually. The administrator presents the student with a student copy of the Number Identification test (Figure 26). The administrator places the administrator copy of the Number Identification test on a clipboard and positions it so the student cannot see what the administrator records (Figure 27).

Figure 26. Student Copy of Number Identification

The actual Number Identification student copy is 3 pages long.


Figure 27. Administrator Copy of Number Identification

Administration of Number Identification is as follows:

Teacher: The paper in front of you has boxes with numbers in them. When I say, “Begin,” I want you to tell me what number is in each box. Start here and go across the page (demonstrate by pointing). Try each one. If you come to one that you don’t know, I’ll tell you to go on to the next one. Are there any questions? Put your finger on the first one. Ready, begin. (Start stopwatch. If the student does not respond after 3 seconds, then count the item as incorrect; point to the next item and say, “Try this one.” At the end of 1 minute, draw a line under the last item completed.)

The teacher marks the student’s responses on the score sheet as the student says the answers aloud. The probe is scored after the student is finished. When scoring Number Identification, if a student correctly identified the number, then the item is scored as correct. If a student hesitated or struggled with a problem for 3 seconds or gave the wrong answer, then the item is scored as incorrect.


Look at the following Number Identification score sheet (Figure 28). Jamal answered 54 items correctly in 1 minute. Jamal’s math score for this probe is 54.

Figure 28. Jamal’s Number Identification Score Sheet


Let’s practice administering the Number Identification probe. Use the following Number Identification student sheet (Figures 29–31) and score sheet (Figure 32). The administrator reads the directions to the student, and the student has 1 minute to answer as many items as possible.

Figure 29. Practice Number Identification Student Sheet—Page 1


Figure 32. Practice Number Identification Score Sheet

Once the 1-minute time limit is over, practice scoring the Number Identification score sheet. See Appendix A for information about obtaining CBM Number Identification probes.


Quantity Discrimination

The Quantity Discrimination test for kindergarten or first-grade students consists of 63 items that require the student to orally identify the bigger number from a pair of numbers 0 through 20. Quantity Discrimination includes three alternate forms. The Quantity Discrimination measures have been researched as screening tools, but schools may want to use them for progress monitoring.

Quantity Discrimination is administered individually. The administrator presents the student with a student copy of the Quantity Discrimination test (Figure 33). The administrator places the administrator copy of the Quantity Discrimination test on a clipboard and positions it so the student cannot see what the administrator records (Figure 34).

Figure 33. Student Copy of Quantity Discrimination

The actual Quantity Discrimination student copy is 3 pages long.


Figure 34. Administrator Copy of Quantity Discrimination

Administration of Quantity Discrimination is as follows:

Teacher: The paper in front of you has boxes with two numbers in each box. When I say, “Begin,” I want you to tell me which number is bigger. Start here and go across the page (demonstrate by pointing). Try each one. If you come to one that you don’t know, I’ll tell you to go on to the next one. Are there any questions? Put your finger on the first one. Ready, begin. (Start stopwatch. If the student does not respond after 3 seconds, count the item as incorrect; point to the next item and say, “Try this one.” At the end of 1 minute, draw a line under the last item completed.)

The teacher marks the student’s responses on the score sheet as the student says the answers aloud. The probe is scored after the student is finished. When scoring Quantity Discrimination, if a student correctly identified the bigger number, the item is scored as correct. If a student hesitated or struggled with a problem for 3 seconds or gave the wrong answer, the item is scored as incorrect.


Look at the following Quantity Discrimination score sheet (Figure 35). Lin answered 33 items correctly in 1 minute. Lin’s math score for this probe is 33.

Figure 35. Lin’s Quantity Discrimination Score Sheet


Let’s practice administering the Quantity Discrimination probe. Use the following Quantity Discrimination student sheet (Figures 36–38) and score sheet (Figure 39). The administrator reads the directions to the student, and the student has 1 minute to answer as many items as possible.

Figure 36. Practice Quantity Discrimination Student Sheet—Page 1


Figure 39. Practice Quantity Discrimination Score Sheet

Once the 1-minute time limit is over, practice scoring the Quantity Discrimination score sheet. See Appendix A for information about obtaining CBM Quantity Discrimination probes.


Missing Number

The Missing Number test for kindergarten or first-grade students consists of 63 items that require the student to orally identify the missing number in a sequence of four numbers. The sequence includes counting by 1-digit increments with numbers 0 through 10, counting by 2-digit increments with numbers 0 through 20, counting by 5-digit increments with numbers 0 through 50, and counting by 10-digit increments with numbers 0 through 100. Missing Number includes three alternate forms. The Missing Number measures have been researched as screening tools, but schools may want to use them for progress monitoring.

Missing Number is administered individually. The administrator presents the student with a student copy of the Missing Number test (Figure 40). The administrator places the administrator copy of the Missing Number test on a clipboard and positions it so the student cannot see what the administrator records (Figure 41).

Figure 40. Student Copy of Missing Number

The actual Missing Number student copy is 3 pages long.


Figure 41. Administrator Copy of Missing Number

Administration of Missing Number is as follows:

Teacher: The paper in front of you has boxes with three numbers and a blank in each of them. When I say, “Begin,” I want you to tell me what number goes in the blank in each box. Start here and go across the page (demonstrate by pointing). Try each one. If you come to one that you don’t know, I’ll tell you to go on to the next one. Are there any questions? Put your finger on the first one. Ready, begin. (Start stopwatch. If the student does not respond after 3 seconds, then count the item as incorrect; point to the next item and say, “Try this one.” At the end of 1 minute, draw a line under the last item completed.)


The teacher marks the student’s responses on the score sheet as the student says the answers aloud. The probe is scored after the student is finished. When scoring Missing Number, if a student correctly identified the missing number in the sequence, then the item is scored as correct. If the student hesitated or struggled with a problem for 3 seconds or gave the wrong answer, then the item is scored as incorrect.

Look at the following Missing Number score sheet (Figure 42). Thomas answered 18 items correctly in 1 minute. Thomas’s math score for this probe is 18.

Figure 42. Thomas’s Missing Number Score Sheet


Let’s practice administering the Missing Number probe. Use the following Missing Number student sheet (Figures 43–45) and score sheet (Figure 46). The administrator reads the directions to the student, and the student has 1 minute to answer as many items as possible.

Figure 43. Practice Missing Number Student Sheet—Page 1


Figure 46. Practice Missing Number Score Sheet

Once the 1-minute time limit is over, practice scoring the Missing Number score sheet. See Appendix A for information about obtaining CBM Missing Number probes.


Step 4: How to Graph Scores

Once the CBM data for each student have been collected, teachers can begin to graph student scores. Graphing the scores of every CBM on an individual student graph is a vital aspect of the CBM program. These graphs give teachers a straightforward way of reviewing a student’s progress, monitoring the appropriateness of the student’s goals, judging the adequacy of the student’s progress, and comparing and contrasting successful and unsuccessful instructional aspects of the student’s program.

CBM graphs help teachers make decisions about the short- and long-term progress of each student. Frequently, teachers underestimate the rate at which students can improve (especially in special education classrooms); therefore, the CBM graphs help teachers set ambitious, but realistic, goals. Without graphs and decision rules about the scores on a student graph, teachers often stick with low goals. By using a CBM graph, teachers can use a set of standards to create more ambitious student goals and help improve student achievement. Also, CBM graphs provide teachers with actual data to help them revise and improve a student’s instructional program.

Teachers have two options for creating CBM graphs of the individual students in the classroom. The first option is that teachers and schools can purchase CBM graphing software that graphs student data and helps interpret the data for teachers. The second option is that teachers can create their own student graphs using graph paper and pencil.

To create student graphs, teachers create a master CBM graph, in which the vertical axis accommodates the range of the scores of all students in the class, from 0 to the highest score. A table of highest scores for the vertical axis is available in the CBM manual. On the horizontal axis, the number of weeks of instruction is listed. Once the teacher creates the master graph, it can be copied and used as a template for every student.

Teacher-made graphs make it easy to graph student CBM scores. Teachers create a student graph for each individual CBM student so they can interpret the CBM scores of every student and see their progress, or lack thereof.

Figure 47. Highest Scores for Labeling Vertical Axes on Curriculum-Based Measurement Graphs

CBM task Vertical axis: 0– __

Number Identification 84

Quantity Discrimination 63

Missing Number 63

Computation 25 (problems correct) 30–80 (digits correct)

Concepts and Applications 50


Step 5: How to Set Ambitious Goals

Once a few CBM scores have been graphed, it is time for the teacher to decide on an end-of-year performance goal for the student. There are three options. Two options are utilized after at least three CBM scores have been graphed. One option is utilized after at least eight CBM scores have been graphed. (See publishers’ manuals for norms.)

Option 1: End-of-Year Benchmarking

For typically developing students at the grade level where the student is being monitored, identify the end-of-year CBM benchmark. (Because of the limited data of the technical adequacy of the measures at this time, no kindergarten benchmarks have been established. See recommendations for Grades 1–6 in Figure 50.) This is the end-of-year performance goal. The benchmark, or end-of-year performance goal, is represented on the graph by an X at the date marking the end of the year. A goal-line is then drawn between the median of at least the first three CBM graphed scores and the end-of-year performance goal.

Figure 48. Curriculum-Based Measurement Computation/Concepts and Applications Benchmarks

Grade Probe Maximum score Benchmark

Kindergarten Data not yet available

First Computation 30 20 digits

First Data not yet available

Second Computation 45 20 digits

Second Concepts and Applications 32 20 blanks

Third Computation 45 30 digits

Third Concepts and Applications 47 30 blanks

Fourth Computation 70 40 digits

Fourth Concepts and Applications 42 30 blanks

Fifth Computation 80 30 digits

Fifth Concepts and Applications 32 15 blanks

Sixth Computation 105 35 digits

Sixth Concepts and Applications 35 15 blanks

For example, the benchmark for a first-grade student in CBM math Computation is 20 digits correct in 2 minutes. The end-of-year performance goal of 20 would be graphed on the student’s graph. The goal-line would be drawn between the median of the first few CBM scores and the end-of-year performance goal.

The benchmark for a sixth-grade student on CBM Concepts and Applications is 15 points in 7 minutes. The end-of-year performance goal of 15 would be graphed on the student’s graph. The


goal-line would be drawn between the median of the first few CBM scores and the end-of-year performance goal.

Option 2: Intra-Individual Framework

Identify the weekly rate of improvement for the target student under baseline conditions, using at least eight CBM data points. Multiply this baseline rate by 1.5. Take this product and multiply it by the number of weeks until the end of the year. Add this product to the student’s baseline score. This sum is the end-of-year goal.

For example, a student’s first eight CBM scores were 3, 2, 5, 6, 5, 5, 7, and 4. To calculate the current weekly rate of improvement, we can use the Tukey method. First, divide the scores into three roughly equal groups, and subtract the first median from the third median. In this instance, 5 is the third median and 3 is the first median: 5 – 3 = 2. Since eight scores have been collected, divide the difference between the highest and lowest scores by the number of weeks minus 1: 2 ÷ (8-1) = 0.29.

That quotient is then multiplied by 1.5: 0.29 × 1.5 = 0.435. Multiply that product by the number of weeks until the end of the year. If there are 14 weeks left until the end of the year, then 0.435 × 14 = 6.09. The median score of the first three data points was 3. The sum of 6.09 and the median score is the end-of-year performance goal: 3 + 6.09 = 9.09. The student’s end-of-year performance goal would be 9. (The end-of-year performance goal is rounded to the nearest whole number.)

Option 3: National Norms

For typically developing students at the grade level where the student is being monitored, identify the average rate of weekly increase from a national norm chart.

Figure 49. Curriculum-Based Measurement Norms for Student Growth (Slope)

Grade Computation CBM—

slope for digits correct Concepts and Applications

CBM—slope for blanks

1 .35 No data available

2 .30 .40

3 .30 .60

4 .70 .70

5 .70 .70

6 .40 .70


For example, let’s say that a fourth-grade student’s median score from his first three CBM Computation probes is 14. The norm for fourth-grade students is .70 (Figure 49). The .70 is the weekly rate of growth for fourth graders. To set an ambitious goal for the student, multiply the weekly rate of growth by the number of weeks left until the end of the year. If there are 16 weeks left, then multiply 16 by .70: 16 × .70 = 11.2. Add 11.2 to the median score of 14 (11.2 + 14 = 25.2). The student’s end-of-year performance goal is 25.

Computer Management Programs

CBM computer management programs are available for schools to purchase. The computer scoring programs create graphs for individual students after the student scores are entered into the program and aid teachers in making performance goals and instructional decisions. Other computer programs actually collect and score the data.

Various types of computer assistance are available at varying fees. Information on how to obtain the computer programs is in Appendix A.

Step 6: How to Apply Decision Rules to Graphed Scores to Know When to Revise Programs and Increase Goals

CBM can judge the adequacy of student progress and the need to change instructional programs. Researchers have demonstrated that CBM can be used to improve the scope and usefulness of program evaluation decisions (Germann & Tindal, 1985) and to develop instructional plans that enhance student achievement (Fuchs, Deno, & Mirkin, 1984; Fuchs, Fuchs, & Hamlett, 1989a).

After teachers draw CBM graphs and trend-lines, they use graphs to evaluate student progress and to formulate instructional decisions. Standard CBM decision rules guide decisions about the adequacy of student progress and the need to revise goals and instructional programs.

• Decision rules based on the four most recent consecutive scores:

– If the four most recent consecutive CBM scores are above the goal-line, the student’s end-of-year performance goal needs to be increased.

– If the four most recent consecutive CBM scores are below the goal-line, the teacher needs to revise the instructional program (the end-of-year performance goal should never decrease).

• Decision rules based on the trend-line:

– If the student’s trend-line is steeper than the goal-line, the student’s end-of-year performance goal needs to be increased.

– If the student’s trend-line is flatter than the goal-line, the teacher needs to revise the instructional program.

– If the student’s trend-line and goal-line are the same, no changes need to be made.

Let’s look at each of these decision rules and the graphs that help teachers make decisions about students’ goals and instructional programs.


Look at the graph in Figure 50.

Figure 50. Four Consecutive Scores Above Goal-Line

On this graph, the four most recent scores are above the goal-line. Therefore, the student’s end-of-year performance goal needs to be adjusted. The teacher increases the desired rate (or goal) to boost the actual rate of student progress.

The point of the goal increase is notated on the graph as a dotted vertical line. This allows teachers to visually note when the student’s goal was changed. The teacher re-evaluates the student graph in another seven to eight data points to determine whether the student’s new goal is appropriate or whether a teaching change is needed.

X

Goal-line

Most recent 4 points



Figure 51. Four Consecutive Scores Below Goal-Line

On this graph, the four most recent scores are below the goal-line. Therefore, the teacher needs to change the student’s instructional program. The end-of-year performance-goal and goal-line never decrease; they can only increase. The instructional program should be tailored to bring a student’s scores up so they match or surpass the goal-line.

The teacher draws a dotted vertical line when making an instructional change. This allows teachers to visually note when changes to the student’s instructional program were made. The teacher re-evaluates the student graph in another seven to eight data points to determine whether the change was effective.

X

Goal-line

Most recent 4 points



Figure 52. Trend-Line Above Goal-Line

On this graph, the trend-line is steeper than the goal-line. Therefore, the student’s end-of-year performance goal needs to be adjusted. The teacher increases the desired rate (or goal) to boost the actual rate of student progress. The new goal-line can be an extension of the trend-line.

The point of the goal increase is notated on the graph as a dotted vertical line. This allows teachers to visually note when the student’s goal was changed. The teacher re-evaluates the student graph in another seven to eight data points to determine whether the student’s new goal is appropriate or whether a teaching change is needed.

X

X X

Goal-line

Trend-line



Figure 53. Trend-Line Below Goal-Line

On this graph, the trend-line is flatter than the performance goal-line. The teacher needs to change the student’s instructional program. Again, the end-of-year performance goal and goal-line are never decreased! A trend-line below the goal-line indicates that student progress is inadequate to reach the end-of-year performance goal. The instructional program should be tailored to bring the student’s scores up so they match or surpass the goal-line.

The point of the instructional change is represented on the graph as a dotted vertical line. This allows teachers to visually note when the student’s instructional program was changed. The teacher re-evaluates the student graph in another seven to eight data points to determine whether the change was effective.

X X

X

Goal-line

Trend-line



Figure 54. Trend-Line Matches Goal-Line

If the trend-line matches the goal-line, then no change is currently needed.

The teacher re-evaluates the student graph in another seven to eight data points to determine whether an end-of-year performance goal increase or instructional change needs to take place.

X

X

X

Goal-line

Trend-line


Step 7: How to Use the Curriculum-Based Measurement Database Qualitatively to Describe Student Strengths and Weaknesses

Teachers can use a skills profile to describe the strengths and weaknesses of each student in the classroom. A skills profile is a visual display of a student’s progress by each skill area. Each box on the skills profile represents performance for each skill every 2-week interval for the entire school year. The skills profile can help teachers formulate instructional decisions for individual student by identifying skills to target for instruction.

Figure 55 depicts the mastery level codes for the skills profile.

Figure 55. Mastery Level Codes for Skills Profile

Cold. Not tried. Student did not attempt any problems of this type.

Cool. Trying these. Student attempted problems of this type but worked with

low accuracy.

Warm. Starting to get it. Student worked some problems of this type with fair

accuracy.

Very warm. Almost have it. Student worked these problems with high accuracy but didn’t

do enough to guarantee mastery.

Hot. You’ve got it! Student worked enough of these problems with high accuracy

to indicate mastery.

A class skills profile provides information on specific skills for each student in the class, averaged across the two most recent assessments (Figure 56). Each box represents the student’s mastery status on a particular skill, following the same mastery level codes as the individual student skills profile (Figure 57).

For instance, the class skills profile in Figure 64 provides Mrs. Marshall with information on all of her students from their Computation tests. The labels at the top of the chart (A1 through F2) stand for addition, subtraction, multiplication, division, and fractions. As Mrs. Marshall looks at the boxes for each student, she can see that increasingly dark boxes show increasing mastery of skill.


Figure 56. Class Skills Profile

On this individual skills profile, the boxes on Ashley’s Computation printout show that her performance on S1 (subtracting with regrouping) has improved over the year (Figure 57). From September to November, her performance was “cool”—that is, she was attempting these problems, but working on them with low accuracy. In December, her performance improved to “warm” and “hot”—that is, she began working these problems with some accuracy and then mastery. In January, following winter break, her performance deteriorated some, suggesting that the teacher should provide some review. The skills profile also provides a student graph with the number of digits correct from the Computation CBM for the teacher to see the student’s progress.


Figure 57. Individual Student Skills Profile


Curriculum-Based Measurement Case Study 1: Alexis

Mrs. Taylor has been monitoring her entire class using weekly CBM Computation tests. She has been graphing student scores on individual student graphs. Mrs. Taylor used the Tukey method to draw a trend-line for Alexis’s CBM Computation scores. Figure 58 presents Alexis’s graph.

Figure 58. Alexis’s Curriculum-Based Measurement Computation Graph

Since Alexis’s trend-line is flatter than her goal-line, Mrs. Taylor needs to make a change to Alexis’s instructional program. She has marked the week of the instructional change with a dotted vertical line. To decide what type of instructional change might benefit Alexis, Mrs. Taylor decides to use the skills profile provided by the Math Computation and Concepts/Applications CBM Scoring Program to find Alexis’s strengths and weaknesses as a math student.

X X

X

Alexis’s goal-line

Alexis’s trend-line


Figure 59 depicts Alexis’s skills profile.

Figure 59. Alexis’s Skills Profile

Based on the skills profile, what instructional program changes should Mrs. Taylor introduce into Alexis’s math program?


Curriculum-Based Measurement Case Study 2: Marcus

Mrs. Fritz has been using CBM to monitor the progress of all of the students in her classroom for the entire school year. She has one student, Marcus, who has been performing extremely below his classroom peers, even after two instructional changes.

Look at Marcus’s CBM graph in Figure 60.

Figure 60. Marcus’s Curriculum-Based Measurement Graph

After 8 weeks, Mrs. Fritz determined that Marcus’s trend-line was flatter than his goal-line, so she made an instructional change to Marcus’s math program. This instructional change included having Marcus work on basic math facts that he was counting on his fingers. The instructional change is the first thick, vertical line on Marcus’s graph.

After another 8 weeks, Mrs. Fritz realized that Marcus’s trend-line was still flatter than his goal-line. His graph showed that Marcus had made no improvement in math. So, Mrs. Fritz made another instructional change to Marcus’s math program. This instructional change included having Marcus work on math fact flash cards. The second instructional change is the second thick, vertical line on Marcus’s graph.

X

Marcus’s goal-line Marcus’s

trend-lines

Instructional changes


Mrs. Fritz has been conducting CBM for 20 weeks and still hasn’t seen any improvement with Marcus’s math despite two instructional teaching changes. What could the graph in Figure 61 tell Mrs. Fritz about Marcus? Pretend you are at a meeting with your principal and Individual Education Program team members. What would you say to describe Marcus’s situation? What would you recommend as the next steps? How could Mrs. Fritz use this class graph to help her with decision making about Marcus?

Figure 61. Mrs. Fritz’s Curriculum-Based Measurement Class Report

High-performing math students

Middle-performing math students

Low-performing math students


Appendix A: Curriculum-Based Measurement Materials

AIMSweb/Edformation (Curriculum-Based Measurement Math Probes and Scoring Software)

The following math probes are available:

• Oral Counting—Test of Early Numeracy (30 probes and 3 benchmarking probes) • Missing Number—Test of Early Numeracy (30 probes and 3 benchmarking probes) • Number Identification—Test of Early Numeracy (30 probes and 3 benchmarking probes) • Quantity Discrimination—Test of Early Numeracy (30 probes and 3 benchmarking

probes) • Math Computation (40 probes per grade) • Math Facts (40 probes per grade)

Scoring software is also available from AIMSweb (Figure A2). See Web site for prices.

Web site: www.aimsweb.com

Phone: 888-944-1882

Address: Edformation, Inc. 6420 Flying Cloud Drive Suite 204 Eden Prairie, MN 55344

Figure A2. AIMSweb Software


Yearly ProgressPro™/McGraw-Hill Digital Learning (Curriculum-Based Measurement Math Probes and Scoring Software)

Yearly ProgressPro™, from McGraw-Hill Digital Learning, combines ongoing formative assessment, prescriptive instruction, and a reporting and data management system to give teachers and administrators the tools they need to raise student achievement. Yearly ProgressPro™ is a computer-administered progress monitoring and instructional system to bring the power of CBM into the classroom. Students take tests on the computer, eliminating teacher time in administration and scoring (Figure A3).

Weekly 15-minute diagnostic CBM assessments provide teachers with the information they need to plan classroom instruction and meet individual student needs. Ongoing assessment across the entire curriculum allows teachers to measure the effectiveness of instruction as it takes place and track both mastery and retention of grade-level skills. Yearly ProgressPro™ reports allow teachers and administrators to track progress against state and national standards at the individual student, class, building, or district level. Administrators can track progress toward AYP goals and disaggregate data demographically to meet No Child Left Behind requirements.

See Web site for prices.

Web site: www.mhdigitallearning.com

Phone: 1-800-848-1567 ext. 4928

Figure A3. Yearly ProgressPro™ Software


Pro-Ed Math Computation and Concepts/Applications CBM

These CBM materials were developed and researched using standard CBM procedures. Curriculum-Based Math Computation Probes include 30 alternate forms at each grade level for grades 1-6. Curriculum-Based Math Concepts/Applications Probes include 30 alternate forms at each grade level for grades 2-6. Each comes with a manual that provides supporting information (e.g., technical information, directions for administration, and scoring keys).

Phone: (512) 451-3246

Web site: www.proedinc.com

Mail: 8700 Shoal Creek Blvd Austin, TX 78757

Research Institute on Progress Monitoring, University of Minnesota (OSEP Funded) Early Math Measures (Curriculum-Based Measurement Math Probes)

The following math probes are available:

• Number Identification (three probes)

• Quantity Discrimination (three probes)

• Missing Number (three probes)

The K–1 early math measures are not yet ready for full dissemination. For more information about the ongoing research related to these measures, please contact the Research Institute on Progress Monitoring: www.progressmonitoring.org.


Appendix B: Resources

Clarke, B., & Shinn, M. R. (2004). A preliminary investigation into the identification and development of early mathematics curriculum-based measurement. School Psychology Review, 33, 234–248.

Deno, S. L. (1985). Curriculum-based measurement: The emerging alternative. Exceptional Children, 52, 219–232.

Deno, S. L., Fuchs, L. S., Marston, D., & Shin, J. (2001). Using curriculum-based measurement to establish growth standards for students with learning disabilities. School Psychology Review, 30, 507–524.

Deno, S. L., & Mirkin, P. K. (1977). Data-based program modification: A manual. Reston, VA: Council for Exceptional Children.

Fuchs, L. S. (1987). Curriculum-based measurement for instructional program development. Teaching Exceptional Children, 20, 42–44.

Fuchs, L. S., & Deno, S. L. (1987). Developing curriculum-based measurement systems for data-based special education problem solving. Focus on Exceptional Children, 19, 1–16.

Fuchs, L. S., & Deno, S. L. (1991). Paradigmatic distinctions between instructionally relevant measurement models. Exceptional Children, 57, 488–501.

Fuchs, L. S., & Deno, S. L. (1994). Must instructionally useful performance assessment be based in the curriculum? Exceptional Children, 61, 15–24.

Fuchs, L. S., Deno, S. L., & Mirkin, P. K. (1984). Effects of frequent curriculum-based measurement of evaluation on pedagogy, student achievement, and student awareness of learning. American Educational Research Journal, 21, 449–460.

Fuchs, L. S., & Fuchs, D. (1990). Curriculum-based assessment. In C. Reynolds & R. Kamphaus (Eds.), Handbook of psychological and educational assessment of children (Vol. 1): Intelligence and achievement. New York: Guilford Press.

Fuchs, L. S., & Fuchs, D. (1996). Combining performance assessment and curriculum-based measurement to strengthen instructional planning. Learning Disabilities Research and Practice, 11, 183–192.

Fuchs, L. S., & Fuchs, D. (1998). Treatment validity: A unifying concept for reconceptualizing the identification of learning disabilities. Learning Disabilities Research and Practice, 13, 204–219.

Fuchs, L. S., & Fuchs, D. (2000). Curriculum-based measurement and performance assessment. In E. S. Shapiro & T. R. Kratochwill (Eds.), Behavioral assessment in schools: Theory, research, and clinical foundations (2nd ed., pp. 168–201). New York: Guilford.


Fuchs, L. S., & Fuchs, D. (2002). Curriculum-based measurement: Describing competence, enhancing outcomes, evaluating treatment effects, and identifying treatment nonresponders. Peabody Journal of Education, 77, 64–84.

Fuchs, L. S., & Fuchs, D. (2004). Determining adequate yearly progress from kindergarten through grade 6 with Curriculum-Based Measurement. Assessment for Effective Instruction, 29, 25–38.

Fuchs, L. S., Fuchs, D., & Hamlett, C. L. (1989a). Effects of alternative goal structures within curriculum-based measurement. Exceptional Children, 55, 429–438.

Fuchs, L. S., Fuchs, D., & Hamlett, C. L. (1989b). Effects of instrumental use of curriculum-based measurement to enhance instructional programs. Remedial and Special Education, 10, 43–52.

Fuchs, L. S., Fuchs, D., & Hamlett, C. L. (1990). Curriculum-based measurement: A standardized long-term goal approach to monitoring student progress. Academic Therapy, 25, 615–632.

Fuchs, L. S., Fuchs, D., & Hamlett, C. L. (1993). Technological advances linking the assessment of students’ academic proficiency to instructional planning. Journal of Special Education Technology, 12, 49–62.

Fuchs, L. S., Fuchs, D., & Hamlett, C. L. (1994). Strengthening the connection between assessment and instructional planning with expert systems. Exceptional Children, 61, 138–146.

Fuchs, L. S., Fuchs, D., & Hamlett, C. L. (2005). Using technology to facilitate and enhance curriculum-based measurement. In K. Higgins, R. Boone, & D. Edyburn (Eds.), The handbook of special education technology research and practice (pp. 663–681). Knowledge by Design, Inc.: Whitefish Bay, WI.

Fuchs, L. S., Fuchs, D., Hamlett, C. L., Phillips, N. B., & Karns, K. (1995). General educators’ specialized adaptation for students with learning disabilities. Exceptional Children, 61, 440–459.

Fuchs, L. S., Fuchs, D., Hamlett, C. L., Phillips, N. B., Karns, K., & Dutka, S. (1997). Enhancing students’ helping behavior during peer-mediated instruction with conceptual mathematical explanations. Elementary School Journal, 97, 223–250.

Fuchs, L. S., Fuchs, D., Hamlett, C. L., & Stecker, P. M. (1991). Effects of curriculum-based measurement and consultation on teacher planning and student achievement in mathematics operations. American Educational Research Journal, 28, 617–641.

Fuchs, L. S., Fuchs, D., Hamlett, C. L., Thompson, A., Roberts, P. H., Kubek, P., & Stecker, P. S. (1994). Technical features of a mathematics concepts and applications curriculum-based measurement system. Diagnostique, 19, 23–49.

Fuchs, L. S., Fuchs, D., Hamlett, C. L, Walz, L., & Germann, G. (1993). Formative evaluation of academic progress: How much growth can we expect? School Psychology Review, 22, 27–48.


Fuchs, L. S., Fuchs, D., Hosp, M., & Hamlett, C. L. (2003). The potential for diagnostic analysis within curriculum-based measurement. Assessment for Effective Intervention, 28, 13–22.

Fuchs, L. S., Fuchs, D., Karns, K., Hamlett, C. L., Dutka, S., & Katzaroff, M. (2000). The importance of providing background information on the structure and scoring of performance assessments. Applied Measurement in Education, 13, 83–121.

Fuchs, L. S., Fuchs, D., Karns, K., Hamlett, C. L., & Katzaroff, M. (1999). Mathematics performance assessment in the classroom: Effects on teacher planning and student learning. American Educational Research Journal, 36, 609–646.

Fuchs, L. S., Fuchs, D., Karns, K., Hamlett, C. L., Katzaroff, M., & Dutka, S. (1997). Effects of task-focused goals on low-achieving students with and without learning disabilities. American Educational Research Journal, 34, 513–544.

Fuchs, D., Roberts, P. H., Fuchs, L. S., & Bowers, J. (1996). Reintegrating students with learning disabilities into the mainstream: A two-year study. Learning Disabilities Research and Practice, 11, 214–229.

Germann G., & Tindal, G. (1985). An application on curriculum-based assessment: The use of direct and repeated measurement. Exceptional Children, 52, 244–265.

Gersten, R., & Dimino, J. A. (2001). The realities of translating research into classroom practice. Learning Disabilities Research and Practice, 16, 120–130.

Gickling, E. E. (1981). The forgotten learner. Nevada Public Affairs Review, 1, 19–22.

Hutton, J. B., Dubes, R., & Muir, S. (1992). Estimating trend progress in monitoring data: A comparison of simple line-fitting methods. School Psychology Review, 21, 300–312.

Jenkins, J. R., Mayhall, W., Peshka, C., & Townshend, V. (1974). Using direct and daily measures to measure learning. Journal of Learning Disabilities, 10, 604–608.

Lembke, E. S., & Foegen, A. (2005). Creating measures of early numeracy. Presentation at the annual Pacific Coast Research Conference, San Diego, CA.

Marston, D. (1988). The effectiveness of special education: A time-series analysis of reading performance in regular and special education settings. The Journal of Special Education, 21, 13–26.

Marston, D., Mirkin, P. K., & Deno, S. L. (1984). Curriculum-based measurement: An alternative to traditional screening, referral, and identification of learning disabilities of learning disabled students. The Journal of Special Education, 18, 109–118.

Phillips, N. B., Hamlett, C. L., Fuchs, L. S., & Fuchs, D. (1993). Combining classwide curriculum-based measurement and peer tutoring to help general educators provide adaptive education. Learning Disabilities Research and Practice, 8, 148–156.

Shinn, M. R. (Ed.). (1989). Curriculum-based measurement: Assessing special children. New York: Guilford Press.


Shinn, M. R., Tindal, G.A., & Stein, S. (1988). Curriculum-based measurement and the identification of mildly handicapped students: A research review. Professional School Psychology, 3, 69–86.

Stecker, P. M., & Fuchs, L. S. (2000). Effecting superior achievement using curriculum-based measurement: The importance of individual progress monitoring. Learning Disabilities Research and Practice, 15, 128–134.

Tindal, G., Wesson, C., Germann, G., Deno, S., & Mirkin, P. (1982). A data-based special education delivery system: The Pine County Mode. (Monograph No. 19). Minneapolis, MN: University of Minnesota, Institute for Research on Learning Disabilities.

Tucker, J. (1987). Curriculum-based assessment is not a fad. The Collaborative Educator, 1, 4, 10.

VanDerHeyden, A. M., Witt, J. C., Naquin, G., & Noell, G. (2001). The reliability and validity of curriculum-based measurement readiness probes for kindergarten students. School Psychology Review, 30, 368–382.

Wesson, C., Deno, S. L., Mirkin, P. K., Sevcik, B., Skiba, R., King, P. P., Tindal, G.A., & Maruyama, G. (1988). A causal analysis of the relationships among outgoing measurement and evaluation, structure of instruction, and student achievement. The Journal of Special Education, 22, 330–343.


This document was originally developed by the National Center on Student Progress Monitoring under Cooperative Agreement (#H326W0003) and updated by the National Center on Response to Intervention under Cooperative Agreement (#H326E07004) between the American Institutes for Research and the U.S. Department of Education, Office of Special Education Programs. The contents of this document do not necessarily reflect the views or policies of the Department of Education, nor does mention of trade names, commercial products, or organizations imply endorsement by the U.S. Government. This publication is copyright free. Readers are encouraged to copy and share it, but please credit the National Center on Student Progress Monitoring.

contentssss.usf.edu/resources/presentations/2010/fasp_summer_inst2010... · contents steps for...

Documents