week 7 rubrics and rating scales
TRANSCRIPT
![Page 1: Week 7 Rubrics And Rating Scales](https://reader036.vdocuments.mx/reader036/viewer/2022062512/5549cfe5b4c905856d8b4c9c/html5/thumbnails/1.jpg)
Assessment in Schools
Complex Achievement:Scoring Performance Based Assessments
![Page 2: Week 7 Rubrics And Rating Scales](https://reader036.vdocuments.mx/reader036/viewer/2022062512/5549cfe5b4c905856d8b4c9c/html5/thumbnails/2.jpg)
Question (choose the best answer)
Which statement provides the best description of an analytic scoring rubric?
A. Rating is based on the overall performance compared to exemplars.
B. Rating is based on a sum of scores for the individual parts of the performance.
C. Rating is based on the student’s analysis of their performance.
D. All the above.
![Page 3: Week 7 Rubrics And Rating Scales](https://reader036.vdocuments.mx/reader036/viewer/2022062512/5549cfe5b4c905856d8b4c9c/html5/thumbnails/3.jpg)
Types of Performance
1. Alternative – something other than traditional paper and pencil tests requiring students to demonstrate
2. Authentic – practical application of a task in real world conditions/setting(usually only approximated/simulated)
![Page 4: Week 7 Rubrics And Rating Scales](https://reader036.vdocuments.mx/reader036/viewer/2022062512/5549cfe5b4c905856d8b4c9c/html5/thumbnails/4.jpg)
Assessing Performance
Why use performances in assessment?
Why do we score/measure performance? Communication Comparison
![Page 5: Week 7 Rubrics And Rating Scales](https://reader036.vdocuments.mx/reader036/viewer/2022062512/5549cfe5b4c905856d8b4c9c/html5/thumbnails/5.jpg)
Assessing Performance
All Claims about the value of performance assessments rest on the assumption that performance can be accurately observed and reliably rated.
![Page 6: Week 7 Rubrics And Rating Scales](https://reader036.vdocuments.mx/reader036/viewer/2022062512/5549cfe5b4c905856d8b4c9c/html5/thumbnails/6.jpg)
Conducting Music Assessment
![Page 7: Week 7 Rubrics And Rating Scales](https://reader036.vdocuments.mx/reader036/viewer/2022062512/5549cfe5b4c905856d8b4c9c/html5/thumbnails/7.jpg)
Scoring Challenge
No one correct or best answer/solution
Many different performances or solutions might be judged as excellent (or poor)
Requires expert judgment and clearly specified criterion to assess properly.
![Page 8: Week 7 Rubrics And Rating Scales](https://reader036.vdocuments.mx/reader036/viewer/2022062512/5549cfe5b4c905856d8b4c9c/html5/thumbnails/8.jpg)
Scoring Limitations Scoring can be inconsistent (unreliable Rating)
To compare scores fairly, Task (learning outcome) must be clearly
defined and communicated to students Scoring criteria/rubrics must be well
defined. Time consuming to complete
Must have reasonable amount of time to do Limits the number of tasks that can be
done
![Page 9: Week 7 Rubrics And Rating Scales](https://reader036.vdocuments.mx/reader036/viewer/2022062512/5549cfe5b4c905856d8b4c9c/html5/thumbnails/9.jpg)
Scoring Issues
What are you assessing?
Process – approach used, methods & procedures, instrument use, etc.
Product – complete performance or resulting artifact
![Page 10: Week 7 Rubrics And Rating Scales](https://reader036.vdocuments.mx/reader036/viewer/2022062512/5549cfe5b4c905856d8b4c9c/html5/thumbnails/10.jpg)
Scoring Decisions
What assessment instruments will be used?
Rubrics, Rating scales, Checklists …
How will the results be used/reported?
What will you do to make sure the results are accurate (reliable)?
![Page 11: Week 7 Rubrics And Rating Scales](https://reader036.vdocuments.mx/reader036/viewer/2022062512/5549cfe5b4c905856d8b4c9c/html5/thumbnails/11.jpg)
Scoring Decisions
Who will do the assessment?
Teacher, student, peers, others
How will they be trained?
![Page 12: Week 7 Rubrics And Rating Scales](https://reader036.vdocuments.mx/reader036/viewer/2022062512/5549cfe5b4c905856d8b4c9c/html5/thumbnails/12.jpg)
Guidelines and Suggestions
1. Focus on the learning outcomes that require complex cognitive skills and performances
2. Select tasks that represent important content and skills
3. Minimize the dependence of irrelevant skills not directly related to learning outcome
![Page 13: Week 7 Rubrics And Rating Scales](https://reader036.vdocuments.mx/reader036/viewer/2022062512/5549cfe5b4c905856d8b4c9c/html5/thumbnails/13.jpg)
Guidelines and Suggestions
5. Provide scaffolding as needed6. Construct task directions that
clearly explain what students are expected to do
7. Clearly communicate performance expectations (how performance will be judged)
![Page 14: Week 7 Rubrics And Rating Scales](https://reader036.vdocuments.mx/reader036/viewer/2022062512/5549cfe5b4c905856d8b4c9c/html5/thumbnails/14.jpg)
Scoring Issues
What are your expectations?
Criteria – ideas about what is good or desirable when we judge adequacy; also used to defend that judgment.
![Page 15: Week 7 Rubrics And Rating Scales](https://reader036.vdocuments.mx/reader036/viewer/2022062512/5549cfe5b4c905856d8b4c9c/html5/thumbnails/15.jpg)
Scoring Criteria Issues
Floating Criteria – wait until you see the performance to determine acceptability
Ask yourself – Do you know what your are looking for? Can you define and describe the quality of a
performance (both good and bad)? Can you provide a defensible basis for
rating good and bad performance?
![Page 16: Week 7 Rubrics And Rating Scales](https://reader036.vdocuments.mx/reader036/viewer/2022062512/5549cfe5b4c905856d8b4c9c/html5/thumbnails/16.jpg)
Scoring Criteria Issues
Criteria – define what is acceptable and unacceptable
in ways the student can understand communicate the goal or standards not useful when vague or ambiguous make public what is considered important
[9-30] characteristics
![Page 17: Week 7 Rubrics And Rating Scales](https://reader036.vdocuments.mx/reader036/viewer/2022062512/5549cfe5b4c905856d8b4c9c/html5/thumbnails/17.jpg)
Instruments
Rubrics Rating Scales Checklists
![Page 18: Week 7 Rubrics And Rating Scales](https://reader036.vdocuments.mx/reader036/viewer/2022062512/5549cfe5b4c905856d8b4c9c/html5/thumbnails/18.jpg)
Scoring Rubrics
Rubrics are a set of guidelines that explain the criteria by which performance will be judged or rated (may include a rating scale).
Rubrics outline performance standards
![Page 19: Week 7 Rubrics And Rating Scales](https://reader036.vdocuments.mx/reader036/viewer/2022062512/5549cfe5b4c905856d8b4c9c/html5/thumbnails/19.jpg)
Scoring Rubrics
Rubrics can be
Analytic – individual aspect of the task are judged and used to determine overall score
Holistic – the performance or product is judged as a whole, compared to models or exemplars
[see chapter 10]
![Page 20: Week 7 Rubrics And Rating Scales](https://reader036.vdocuments.mx/reader036/viewer/2022062512/5549cfe5b4c905856d8b4c9c/html5/thumbnails/20.jpg)
Scoring Rubrics
Rubrics typically provide a description of how the rater should determine the quality of various performances at specific levels.
Examples [9-29,9-33, pg 272]
[9-32 rubric development]
![Page 21: Week 7 Rubrics And Rating Scales](https://reader036.vdocuments.mx/reader036/viewer/2022062512/5549cfe5b4c905856d8b4c9c/html5/thumbnails/21.jpg)
Group Task
Create an rubric you might use to rate or score the Leading Music performance.
![Page 22: Week 7 Rubrics And Rating Scales](https://reader036.vdocuments.mx/reader036/viewer/2022062512/5549cfe5b4c905856d8b4c9c/html5/thumbnails/22.jpg)
Rating Scales
They provide a convenient recording method, common frame of reference, and focus the raters attention on specific important aspects of the performance
Used for (limited to) make quality judgments Requires additional information regarding
performance expectations Examples [9-34, pg 274]
![Page 23: Week 7 Rubrics And Rating Scales](https://reader036.vdocuments.mx/reader036/viewer/2022062512/5549cfe5b4c905856d8b4c9c/html5/thumbnails/23.jpg)
Rating Scales
These take many forms (numerical and descriptive) but are used to provide a uniform way to score performances along a continuum (at least ordinal, preferably interval).
[9-42 types of rating scales]
![Page 24: Week 7 Rubrics And Rating Scales](https://reader036.vdocuments.mx/reader036/viewer/2022062512/5549cfe5b4c905856d8b4c9c/html5/thumbnails/24.jpg)
How often do you (meant to record
frequency)
Response scale 1
Daily 2-3 times per week Once a week 2-3 times a month Once a month Less than once a
month
Response scale 2
Once a day Once a week More than once a
day More than once a
week As seldom as
possibleNot Ordinal, Not IntervalOrdinal but Not Interval
![Page 25: Week 7 Rubrics And Rating Scales](https://reader036.vdocuments.mx/reader036/viewer/2022062512/5549cfe5b4c905856d8b4c9c/html5/thumbnails/25.jpg)
Question (choose the best answer)
Which of the following is NOT a good principle for constructing a graphic rating scales?
A. Characteristics should be directly observable.
B. Use 3 to 7 points on the scale.C. Points on the scale must form an
ordinal continuum.D. Each point on the scale must be
defined clearly.
![Page 26: Week 7 Rubrics And Rating Scales](https://reader036.vdocuments.mx/reader036/viewer/2022062512/5549cfe5b4c905856d8b4c9c/html5/thumbnails/26.jpg)
Rating vs. Ranking
Ranking requires a person to place in relative order
Raters assigns a specific score
Why might you rank instead of rate?
![Page 27: Week 7 Rubrics And Rating Scales](https://reader036.vdocuments.mx/reader036/viewer/2022062512/5549cfe5b4c905856d8b4c9c/html5/thumbnails/27.jpg)
Checklists More appropriate for analytic rubrics where
you can easily divide the task into a series of specific actions that must be present.
Reduces the amount of subjectivity in the judgment (dichotomous decision)
Can be problematic when aspects of the performance are valued but not represented in the criteria. (e.g., esthetically pleasing, interesting)
Examples [pg 282]
![Page 28: Week 7 Rubrics And Rating Scales](https://reader036.vdocuments.mx/reader036/viewer/2022062512/5549cfe5b4c905856d8b4c9c/html5/thumbnails/28.jpg)
Common Rating Errors
Personal Bias – Generosity error – too easy, grade inflation Severity error – too hard, no perfect papers Central tendency – rating everyone about average Halo Effect – general impression of individual
(positive or negative) influences an individual rating Logical error – rating alike or different based on the
belief that factors are related (e.g., studious and able) [see 9-44, pg. 277]
![Page 29: Week 7 Rubrics And Rating Scales](https://reader036.vdocuments.mx/reader036/viewer/2022062512/5549cfe5b4c905856d8b4c9c/html5/thumbnails/29.jpg)
Effective Rating Review
Focus on educationally significant outcomes
Characteristics should be directly observable
Clearly define key points on scale Select most appropriate type of
instrument Use an appropriate scale (# of points)
![Page 30: Week 7 Rubrics And Rating Scales](https://reader036.vdocuments.mx/reader036/viewer/2022062512/5549cfe5b4c905856d8b4c9c/html5/thumbnails/30.jpg)
Effective Rating Review
Rate all performances on one task before going on to next.
When possible rate performances without knowing the raters name
If the assessment has significant impact, several ratings should be used.
Example practice [7-22,7-23]