Aligning to standards from the "get go:"

Designing alternate assessments based on states’ standards, expanded

benchmarks, and universal design

Sue Bechard, Measured ProgressNCME, Montreal, April 14, 2005

Comparison of test development process for general and AAC

assessments

Figure 1. Test Development Process for General Large-Scale and AAC Alternate Assessments

General Large-Scale Process Alternate Assessment

1. NA

2. NA

3. NA

4. Build on state academic content standards

5. NA

6. Develop an assessment blueprint

7. Determine assessment item formats

8. Write items

9. Review items for bias and content

10. Revise items

11. Construct assessments

12. Field test assessments

13. Analyze field test data

1. Pool standards into consensus frameworks

2. Consider organization national standards

3. Identify key or essential concepts

4. Produce expanded benchmarks

5. PLT select concepts to assess

6. Development process with templates

7. CDTs identify activity, context, or materials

8. Write items linked to activity

9. Review assessments/items (bias & content)

10. Make changes abased on results of review

11. Finalize assessments

12. Pilot test assessments

13. Analyze pilot assessment data

Process used to develop aligned alternate assessments for the AAC

1. Develop crosswalks

2. Develop consensus frameworks

3. Develop expanded benchmarks

4. Develop two types of alternate assessments (performance tasks and instructionally embedded assessments)

Develop crosswalks

Collect state standards (nine states)Spreadsheet developed using structures suggested by national content organizations (NCTE, NCTM, NCES)

All state standards are compared by essential concepts/topics and grade level the topic was first introduced.

Develop Consensus Frameworks

Crosswalks were examined for consistency across states. Common topics were extrapolated.Mismatches were negotiated using consistent decision rules.Vocabulary was standardized.State content experts reviewed for coverageCAST examined for universal languagePLT reviewed and finalized

Develop Expanded Benchmarks

The consensus frameworks standards were expanded backward by grade spans (El., M.S., H.S.).

Assumes a learning continuum that has been defined by content experts.

Assumes all students pass similar learning milestones for acquisition of academic content.

Develop expanded benchmarks:Considerations of content

Content is described along a continuum in each essential concept/topic from least to most complex to capture the range of students in this population.

Content covers the small steps that typical students learn incidentally, but must be directly taught to students with significant cognitive disabilities.

Process for expanding benchmarks

1. Identify foundational skills to describe what are all students asked to do, e.g.:

Reading = comprehend graphic symbols

Writing = use graphics to convey meaning

2. Conduct task analysis, e.g., reading:Connect objects to words

Connect words to symbols

Connect symbols to print

Expanding benchmarks (cont.)

3. Lay out expanded benchmarks along learning continuum for each essential concept/topic, in measurable terms, using universal language.

4. Conduct reverse alignment from benchmarks back to standards.

Concepts and expanded benchmarks

Project leadership team determined content of assessments

Content area/grade level foci:Elementary/science: characteristics and structure of living things

Middle school/reading and writing: informational and functional text, write in a variety of genres, considering audience & purpose

High school/mathematics: patterns, relations, and functions of algebra.

Develop assessments:Considerations of context

Age – appropriate topics and materials are used:

Elementary students understand living vs. non-living using mealworms and conducting experiments with celery.Middle school students comprehend informational text using newspaper articles and biographies.High school students use manipulatives to build fences and bridges to demonstrate understanding related to linear equations.

Relationship of content and context in AAC alternate assessments

Considering content and context

High school

mathematics

Middle school English

language arts

Elementary science

Expanded benchmarks Expanded benchmarks Expanded benchmarks

Assigned

grade level

Content Context Content Context Content Context

11 X X

10 X X

9 X X

8 X X X

7 X X X

6 X X X

5 X X X X

4 X X X X

3 X X X X

2 X X X

1 X X X

K X X X

Pre-K X X X

Develop assessments: Considerations of learning modality

Principles of universal design used to construct tasks and activities.

Students with significant disabilities require multiple means of access.

Flexibility in adapting materials and providing response options is necessary for valid results.

Number of assistive technologies usedby students in AAC pilot

How many assistive technologies (AT) did the student use during day-to-day instruction?

Number of AT Items Selected

Number of Students Percent

1-4 49 29.9

5-7 41 25.0

8-11 46 28.0

12-20 28 17.1

Total164 100.0

How many words does the student communicate expressively by grade range?

(in any mode—speaking, writing, signing, gesturing,

pictures, symbols, or objects)?

Grade Range How many words does the student communicate expressively? Elementary Middle School High School Total No words Count 8 4 5 17 % within Grade Range 13.6% 8.0% 9.1% 10.4% 1-15 words Count 11 9 14 34 % within Grade Range 18.6% 18.0% 25.5% 20.7% 16-50 words Count 11 5 3 19 % within Grade Range 18.6% 10.0% 5.5% 11.6% 50-200 words Count 15 9 10 34 % within Grade Range 25.4% 18.0% 18.2% 20.7% 200+ words Count 14 23 23 60 % within Grade Range 23.7% 46.0% 41.8% 36.6% Total Count 59 50 55 164 % within Grade Range 100.0% 100.0% 100.0% 100.0%

Develop Assessments (cont.)

Templates are used by PT for each step

Step 1: Materials Needed

Script

Indicators

Set-Up

Notes about scaffolding

Expanded Benchmarks Addressed

Notes about adaptations

Develop Assessments (cont.)

Templates are used by IEA for each dayDay 1: Instructional/Assessment Activities Benchmarks & Expanded Benchmarks

Activity 1: Activity 2: Activity 3: Teacher will: Student will:

Teacher will: Student will:

Materials Provided:

Materials Needed:

Accommodations Needed:

Assessment Types: Constructed Response Product Performance Process

Process: Product:

Write steps (PT) and days’ lessons (IEA) linked to activities -

Based on principles of universal design

To elicit measurable behaviors/products linked to benchmarks

With scoring rubrics in mind

Reviewed for bias

Scoring Rubrics

PT - Level of independenceRed Orange Yellow Green Blue

Responds independently Responds with clarification or fewer options

Responds with specific prompts

Responds given exact response

Inconclusive

IEA - Number of indicators with evidence of

independent student performance

Each content area had unique numbers of indicators listed in each of the rubric levels

Conclusions

The pilot assessments would have benefited from an assessment blueprint indicating specific expanded benchmarks.

Procedures can be followed to align alternate assessments to standards at the time of test development.

More research is needed to determine if students with significant cognitive disabilities acquire skills in a sequence that has been identified by content experts.