quality assessments-- rubrics
TRANSCRIPT
Quality Assessment Design
Constructed Response Assessments: ● Take more time to grade ● Require a specific rubric● Require inter-rater and intra-rater reliability ● Write descriptive rubrics: less than full credit should
point out misconceptions, next steps● May not be appropriate for all ages or subjects● Be consistent in administration technique (directions,
etc.)
Considerations in Selecting
Question Form
Constructed Response Assessments: ● Take more time to grade ● Require a specific rubric● Require inter-rater and intra-rater reliability ● Write descriptive rubrics: less than full credit should
point out misconceptions, next steps● May not be appropriate for all ages or subjects● Be consistent in administration technique (directions,
etc.)
Considerations in Selecting
Question Form
We will discuss in detail shortly...
Performance Assessments: ● Take more time to grade and administer● Require a specific rubric● Require inter-rater and intra-rater reliability ● Write descriptive rubrics: less than full credit should
point out misconceptions, next steps● May not be appropriate for all ages or subjects● Be consistent in administration technique (directions,
etc.)
Considerations in Selecting
Question Form
Performance Assessments: ● Take more time to grade and administer● Require a specific rubric● Require inter-rater and intra-rater reliability ● Write descriptive rubrics: less than full credit should
point out misconceptions, next steps● May not be appropriate for all ages or subjects● Be consistent in administration technique (directions,
etc.)
Considerations in Selecting
Question Form
Lets discuss this deeper now...
Understanding Rubrics
Understanding Rubrics
ROWS:Each of the criteria
you plan to measure.
Understanding Rubrics
ROWS:Each of the criteria
you plan to measure.
COLUMNS:Potential levels of
performance ranging from “ideal” to not
evident.
Understanding Rubrics
ROWS:Each of the criteria
you plan to measure.
COLUMNS:Potential levels of
performance ranging from “ideal” to not
evident.Rating Boxes:
Detailed descriptions of performance at that level for that
skill.
Understanding Rubrics
Understanding Rubrics
ROWS:Each of the criteria
you plan to measure.
Understanding Rubrics
COLUMNS:Potential levels of
performance ranging from “ideal” to not
evident.
ROWS:Each of the criteria
you plan to measure.
Understanding Rubrics
Rating Boxes:Detailed descriptions
of performance at that level for that
skill.
COLUMNS:Potential levels of
performance ranging from “ideal” to not
evident.
ROWS:Each of the criteria
you plan to measure.
Understanding Rubrics
Rating Boxes:Detailed descriptions
of performance at that level for that
skill.
COLUMNS:Potential levels of
performance ranging from “ideal” to not
evident.
ROWS:Each of the criteria
you plan to measure.
This is NOT an example of a descriptive rubric
Understanding Rubrics
Rating Boxes:Detailed descriptions
of performance at that level for that
skill.
COLUMNS:Potential levels of
performance ranging from “ideal” to not
evident.
ROWS:Each of the criteria
you plan to measure.
This is NOT an example of a descriptive rubric
Some of the timeMost of the time
FairPoor
Understanding Rubrics
ROWS:Each of the criteria
you plan to measure.
Understanding Rubrics
ROWS:Each of the criteria
you plan to measure.
Determining Appropriate Criteria:
● Areas key to student learning (essential skills)
● Definable & Observable
● Not generally characteristic of the task itself but rather characteristics of the learning outcomes the assessment is designed to measure
Effective Rubrics do not list all possible criteria; they list the right criteria for the assessment’s purpose
Understanding Rubrics
ROWS:Each of the criteria
you plan to measure.
Determining Appropriate Criteria:
● Areas key to student learning (essential skills)
● Definable & Observable
● Not generally characteristic of the task itself but rather characteristics of the learning outcomes the assessment is designed to measure
Effective Rubrics do not list all possible criteria; they list the right criteria for the assessment’s purpose
Understanding Rubrics
ROWS:Each of the criteria
you plan to measure.
Determining Appropriate Criteria:
● Areas key to student learning (essential skills)
● DOD: Definable, Observable & Distinct
● Not generally characteristic of the task itself but rather characteristics of the learning outcomes the assessment is designed to measure
Effective Rubrics do not list all possible criteria; they list the right criteria for the assessment’s purpose
Understanding Rubrics
ROWS:Each of the criteria
you plan to measure.
Determining Appropriate Criteria:
● Areas key to student learning (essential skills)
● DOD: Definable, Observable & Distinct
● Not generally characteristic of the task itself but rather characteristics of the learning outcomes the assessment is designed to measure
Effective Rubrics do not list all possible criteria; they list the right criteria for the assessment’s purpose
Definable: Each Criterion has a clear, agreed upon meaning that teacher and student understand.
Observable: Each criterion describes a quality in the performance that can be perceived (seen, heard).
Distinct: Each Criterion identifies a separate aspecting of the learning outcomes.
Understanding Rubrics
ROWS:Each of the criteria
you plan to measure.
Determining Appropriate Criteria:
● Areas key to student learning (essential skills)
● DOD: Definable, Observable & Distinct
● Not generally characteristic of the task itself but rather characteristics of the learning outcomes the assessment is designed to measure
Effective Rubrics do not list all possible criteria; they list the right criteria for the assessment’s purpose
Definable: Each Criterion has a clear, agreed upon meaning that teacher and student understand.
Observable: Each criterion describes a quality in the performance that can be perceived (seen, heard).
Distinct: Each Criterion identifies a separate aspecting of the learning outcomes.
Understanding Rubrics
ROWS:Each of the criteria
you plan to measure.
Determining Appropriate Criteria:
● Areas key to student learning (essential skills)
● DOD: Definable, Observable & Distinct
● Not generally characteristic of the task itself but rather characteristics of the learning outcomes the assessment is designed to measure
Effective Rubrics do not list all possible criteria; they list the right criteria for the assessment’s purpose
Definable: Each Criterion has a clear, agreed upon meaning that teacher and student understand.
Observable: Each criterion describes a quality in the performance that can be perceived (seen, heard).
Distinct: Each Criterion identifies a separate aspecting of the learning outcomes.
Understanding Rubrics
ROWS:Each of the criteria
you plan to measure.
Determining Appropriate Criteria:
● Areas key to student learning (essential skills)
● DOD: Definable, Observable & Distinct
● Not generally characteristic of the task itself but rather characteristics of the enduring learning outcomes the assessment is designed to measure. These measurement criteria could then apply to several prompts, taks or performances.
Effective Rubrics do not list all possible criteria; they list the right criteria for the assessment’s purpose
Understanding Rubrics
ROWS:Each of the criteria
you plan to measure.
Determining Appropriate Criteria:
● Areas key to student learning (essential skills)
● DOD: Definable, Observable & Distinct
● Not generally characteristic of the task itself but rather characteristics of the enduring learning outcomes the assessment is designed to measure. These measurement criteria could then apply to several prompts, taks or performances.
Effective Rubrics do not list all possible criteria; they list the right criteria for the assessment’s purpose.
Understanding Rubrics
COLUMNS:Potential levels of
performance ranging from “ideal” to not
evident.
ROWS:Each of the criteria
you plan to measure.
Understanding Rubrics
COLUMNS:Potential levels of
performance ranging from “ideal” to not
evident.
ROWS:Each of the criteria
you plan to measure.
Performance Level Descriptions:
Hint: Think--what does performance look like at high quality? Work backwards.
Descriptive Descriptive language that depicts what one would observe, NOT quality conclusions. (ex: good vs poor)
Clear Students and teachers both understand meanings of descriptions. Teachers use in a repeatable fashion.
Whole Range & Distinguishable Levels
Performance descriptions are described from one extreme to another extreme. Descriptions differ enough from one level to another that work can be categorized. (Ex: number of levels may be multiple or only 2)
Parallel Descriptions
Performance descriptors at each level of the continuum differ in the same aspects of the work and focus on incremental improvements in a standard or skill.
Understanding Rubrics
COLUMNS:Potential levels of
performance ranging from “ideal” to not
evident.
ROWS:Each of the criteria
you plan to measure.
Performance Level Descriptions:
Hint: Think--what does performance look like at high quality? Work backwards.
Descriptive Descriptive language that depicts what one would observe, NOT quality conclusions. (ex: good vs poor)
Clear Students and teachers both understand meanings of descriptions. Teachers use in a repeatable fashion.
Whole Range & Distinguishable Levels
Performance descriptions are described from one extreme to another extreme. Descriptions differ enough from one level to another that work can be categorized. (Ex: number of levels may be multiple or only 2)
Parallel Descriptions
Performance descriptors at each level of the continuum differ in the same aspects of the work and focus on incremental improvements in a standard or skill.
Understanding Rubrics
COLUMNS:Potential levels of
performance ranging from “ideal” to not
evident.
ROWS:Each of the criteria
you plan to measure.
Performance Level Descriptions:
Hint: Think--what does performance look like at high quality? Work backwards.
Descriptive Descriptive language that depicts what one would observe, NOT quality conclusions. (ex: good vs poor)
Clear Students and teachers both understand meanings of descriptions. Teachers use in a repeatable fashion.
Levels:Whole Range & Distinguishable
Performance descriptions are described from one extreme to another extreme. Descriptions differ enough from one level to another that work can be categorized. (Ex: number of levels may be multiple or only 2)
Parallel Descriptions
Performance descriptors at each level of the continuum differ in the same aspects of the work and focus on incremental improvements in a standard or skill.
Understanding Rubrics
COLUMNS:Potential levels of
performance ranging from “ideal” to not
evident.
ROWS:Each of the criteria
you plan to measure.
Performance Level Descriptions:
Hint: Think--what does performance look like at high quality? Work backwards.
Descriptive Descriptive language that depicts what one would observe, NOT quality conclusions. (ex: good vs poor)
Clear Students and teachers both understand meanings of descriptions. Teachers use in a repeatable fashion.
Levels:Whole Range & Distinguishable
Performance descriptions are described from one extreme to another extreme. Descriptions differ enough from one level to another that work can be categorized. (Ex: number of levels may be multiple or only 2)
Parallel Descriptions
Performance descriptors at each level of the continuum differ in the same aspects of the work and focus on incremental improvements in a standard or skill.
Understanding Rubrics: For Growth Data
COLUMNS:Potential levels of
performance ranging from “ideal” to not
evident.
ROWS:Each of the criteria
you plan to measure.
Spotlight on Growth for Evaluations
COMPARABLE DATA:● End of year Expectations ● Use Rubric Consistently● High Reliability
Gradebook Problem: How can I grade at week 2 with end of year expectations?
● Use a curve (“A” is not only for highest rating)● Allow revisions
The Evaluation Connection: Student Growth
Understanding Rubrics: For Growth Data
COLUMNS:Potential levels of
performance ranging from “ideal” to not
evident.
ROWS:Each of the criteria
you plan to measure.
Spotlight on Growth for Evaluations
COMPARABLE DATA:● End of year Expectations ● Use Rubric Consistently● High Reliability
Gradebook Problem: How can I grade at week 2 with end of year expectations?
● Use a curve (“A” is not only for highest rating)● Allow revisions
The Evaluation Connection: Student Growth
Understanding Rubrics: For Growth Data
COLUMNS:Potential levels of
performance ranging from “ideal” to not
evident.
ROWS:Each of the criteria
you plan to measure.
Spotlight on Growth for Evaluations
COMPARABLE DATA:● End of year Expectations ● Use Rubric Consistently● High Reliability
Gradebook Problem: How can I grade at week 2 with end of year expectations?
● Use a curve (“A” is not only for highest rating)● Allow revisions
The Evaluation Connection: Student Growth
Performance Assessments:Consider whenCollecting Baseline Data:
● Exasperation cut points● Safety Issues● Advertisement for class
Understanding RubricsThe Evaluation Connection: Student Growth
Rubrics as Tools for Teacher: ● Defines “quality” for a process, product or behavior● Powerful for teaching and learning
○ Descriptions help communicate what it takes to succeed (This shouldn’t be a secret kept by the teachers!)
● Increase reliability of results○ Increase consistency in grading between
students○ Increase consistency in grading between
measurement points
Understanding Rubrics
Rubrics as Tools for Teacher: ● Defines “quality” for a process, product or behavior● Powerful for teaching and learning
○ Descriptions help communicate what it takes to succeed (This shouldn’t be a secret kept by the teachers!)
● Increase reliability of results○ Increase consistency in grading between
students○ Increase consistency in grading between
measurement points
Understanding Rubrics
Rubrics as Tools for Student: ● Defines “quality” for a process, product or behavior● Encourage more thoughtful judgement of their own
(and others’) work.● Increases ability to “self-assess” ● Increases student responsibility
Understanding Rubrics
Inter-Rater Reliability: ● Same/Repeatable results when
2+ people are using 1 rubric● Practice as a team, share samples,
keep on fileIntra-Rater Reliability:
● Repeatable results when 1 person is using 1 rubric
● Photograph/record samples, keep on file
Understanding Rubrics