understand and explore assessment concepts relative to udl
DESCRIPTION
Understand and Explore Assessment Concepts Relative to UDL. UDL Institute CAST Tracey E. Hall 2013. Operational Definition of Assessment. - PowerPoint PPT PresentationTRANSCRIPT
Understand and Explore Assessment Concepts
Relative to UDL
UDL InstituteCAST
Tracey E. Hall 2013
Operational Definition of Assessment
The gathering of information about a learner from his or her performance in a variety of tasks, subjects, and learning contexts to determine abilities and knowledge for the purpose of making educational decisions (Salvia & Ysseldyke, 2009).
Summative Evaluation: Measurement to determine subject performance at the end of a specific grade level, unit, year, or instructional episode. Mastery Measurement
Formative Evaluation: Assessment of progress toward a long-term goal or major objective, used primarily to diagnose what students have learned in order to plan further instruction (i.e., the on-going assessment of progress toward an objective).
Types of Assessment:
Summative Evaluation: Measurement to determine subject performance at the end of a specific grade level, unit, year, or instructional episode. Mastery Measurement
Formative Evaluation: Assessment of progress toward a long-term goal or major objective, used primarily to diagnose what students have learned in order to plan further instruction (i.e., the on-going assessment of progress toward an objective).
Types of Assessment:
CAST’s latest definitionFormative assessment is an iterative process embedded throughout instruction that is used by educators and learners. Educators (1) collect and analyze multiple measures of learner performance, and (2) use these data to inform and adjust instruction. Learners (1) interact with measures that approximate classroom instruction; and (2) with guidance, use assessment data to better self-regulate learning. Formative assessment is used to address the variability of learners, improve the achievement of intended instructional goals by all, and build educator and learner expertise.
Educational tests are: Indirect measure of a construct or a set of closely
related constructs.
Constructs are not observable
A test provides a sample of observable behaviors believed to be the product of the intended construct
Observable behaviors are used to make an inference about the construct
What is a test supposed to do?
2 perspectives
1.Access to the test
2.Access to the construct
Accessing Assessment
Accessing the Construct
Test construct refers to the concept or the characteristic that a test is designed to measure.
e.g., In a mathematics assessment, an item designed to evaluate students' ability to show equivalence has a test construct of numerical equivalency.
American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (1999).
Test construct
Tests are designed to: Present information to stimulate the
construct.
Interact with the information as the construct is operating
Produce a response which becomes a physical product
Components of an assessment item
Tests are designed to: Present information to stimulate the
construct.
Interact with the information as the construct is operating
Produce a response which becomes a physical product
Components of an assessment item
How Items Function
Interact with Apply View Product Measure of Stimuli Construct of Construct Construct
If a challenge or barrier exists at any of these levels, inference about the construct will fail.
Present Information
Produce Response
Interact with
Construct
Inference to
Construct
AQuantita-
tiveScore
Michael Russell 2011
When there are construct irrelevant impediments, distractions, or barriers in the assessment methods it is essential to provide scaffolds, supports or accommodations in order to improve accuracy and validity, especially for individuals with
disabilities.
Keep in mind, using those same scaffolds, supports or accommodations inappropriately – where they affect the construct relevant demands of the assessment – is likely to invalidate the measure.
Construct relevant refers to the factors (e.g., mode of presentation or response) that are relevant (related) to the construct that the test is intended to measure.
Identifying the symbols for a math calculation problem is a construct relevant change to the assessment item.
Construct relevant
Construct irrelevance is the extent to which test scores are influenced by factors (e.g., mode of presentation or response) that are irrelevant (not related) to the construct that the test is intended to measure.
For Elisa, a student with severe cognitive disabilities, changing the font size and increasing sizing of images helped her to see the assessment item for counting objects and doing so was construct irrelevant to assessing ability to count.
Construct Irrelevant
Accommodations, scaffolds, and supports should not be exclusive to the instructional environment but should be a part of the whole instructional episode including assessment.
The preservation of the distinction between what is construct relevant and what is construct irrelevant is essential when making decisions about accommodations, scaffolds, and supports in the assessment environment.
Envision taking a test:
Envision taking a test:
Ability to concentrate on the
problem is one of the
constituent parts of the
successful mental work
Envision taking a test:
Many students in the United States struggle in the areas of reading and writing, with difficulties emerging early and continuing into the secondary school years and beyond (National Center for Education Statistics, 2002, 2003, 2005, 2007). As student performance levels diverge, it becomes increasingly necessary to provide greater differentiation of instruction. Two approaches are particularly suited for helping teachers and practitioners to differentiate instruction.
Many students in the United States struggle in the areas of reading and writing, with difficulties emerging early and continuing into the secondary school years and beyond (National Center for Education Statistics, 2002, 2003, 2005, 2007). As student performance levels diverge, it becomes increasingly necessary to provide greater differentiation of instruction. Two approaches are particularly suited for helping teachers and practitioners to differentiate instruction.
Adapted presentation Flexible font size (magnification)
Color contrasts
Format, White space
Easy in a digital environment, no item interference. No impact on measuring the construct.
Only change is how the item is being displayed.
“These are no brainers in a computer environment” Mike Russell 09.
On paper, more difficult to make changes such as these.., for administration not student – lots of different papers
Alternate representations: in digital environment – relatively simple
water
agua
H2O
Multiple Representations of Information
Alternate means for Expression and Action: also easily accomplished in a digital environment
Engagement Example:Options for to read and respond.
Students are (a)engaged by
choice (b) often select
based on interest or prior knowledge
(c) perform better
Limitations of Accommodationsas Retrofits
Marginal efficacy Consider:◦ Extended time◦ Read-aloud accommodation vs. independent
reading and strategic test-taking May invalidate measurements
Consider:◦ Use of read-aloud for decoding task◦ Use of calculator for arithmetic task
Thus it is often difficult to level the playing field with accommodations, especially if the playing fields was initially un-level for all students.
Some Solutions: Consider Student Diversity During Test Development & Delivery
During Test Development◦ Ensure narrowly-defined constructs◦ Conduct bias review early◦ Ensure items are suitable for appropriate delivery
to all students
During Test Delivery• Provide multiple means of recognition, expression
& engagement• Ensure matching of classroom and assessment
supports
The cartoon above depicts United States frustration with A) the Good Neighbor policy B) Dollar DiplomacyC) the Spanish-American War D) the Bay of Pigs invasion
12th Grade NAEPU.S. History Item
Broadly-Define Constructs / Bias Review:
Assumptions of Prior Knowledge
Circle the picture that starts with “B”
Broadly-Defined Constructs / Bias Review:Culture and Native Language Assumptions
Circle the picture that starts with “T”
1st Grade Phonological Awareness Tasks
Ensuring Suitable Delivery: Invalidation
“Sarah must determine the diameter of a circle that she has drawn, but doesn’t know the value of П. Can she still do it?”
8th Grade Geometry: Word Problem vs. Symbol Recognition
Consider effect of read-aloud accommodation on the following test question:
MCA-IIGrade 10 Reading Sample Item
UDL Considerations for Assessment Development Structure:
Based on the UDL Principles
Research on working memory
Research on eye movement
*handout
UDL Considerations for Assessment Development Structure
.
Linguistic Complexity
Information Density
Self-Regulation and Visio-Spatial information
UDL Considerations for Assessment Development
Linguistic Complexity (LC) The relationship between the sentence processing mechanism and the available computational resources (Gibson, 1998)
LC-1 Syntax –
LC-2 Simplifying vocabulary
LC-3 Reduced sentence length
LC-4 Language translation
LC-5 Clarify anaphoric references
Linguistic Complexity
Information Density (ID) The two critical limitations on handling information in working memory are the (1) small number of pieces of information an individual’s memory can handle and (2) the short duration of time during which information can remain in memory (Sylwester & Choo, 1992). The following modification structures help to reduce working memory load and maximize use of information read when used carefully.
ID-1 Question placement ID-2 Emphasize key information ID-3 Chunking or combining tasks (questions or response options) ID-4 Guide information processing— ID-5 Contextualizing skills –skill icons ID-6 Hint ID-7 Line numbering ID-8 Passage primer
Information Density
Self-regulation is seen by many cognitive researchers as a pivot upon which students’ achievement turns. The structures noted here when used in a non-construct relevant application help to support self-regulation and attention to tasks.
SR-1 Progress map SR-2 Self check SR-3 Optional Workspace SR-4 Skill Icon Preview
SR-5 Reduce Reflexive Eye Movement
Self-Regulation and Visio-Spatial information (SR)
Example Item 1: A Time-Distance Graph 41
Example Item 2: Features of Plant and Animal Cells 42
Example Item 3: The Boiling Points of Two Beakers of Water 43
UDL - Assessment revisions/considerations Discussion –UDL considerations
Small groups◦ Start with the task directions◦ Think UDL access to executive functioning