a new vision of assessment

A New Vision of Assessment Texts Worth Reading

Problems Worth Solving

Tests Worth Taking

1

NCSA June 2013

PARCC States

PARCC Priorities

3

1. Determine whether students are college and career ready or on track

2. Connect to the Common Core State Standards 3. Measure the full range of student performance,

including that of high- and low-achieving students 4. Provide educators data throughout the year to

inform instruction 5. Create innovative 21st century, technology-based

assessments 6. Be affordable and sustainable

Getting All Students College and Career Ready

4

K–2 Grades 3–8 High

School

Voluntary K–2 assessment being

developed, aligned to the Common Core State

Standards

Timely data showing whether ALL students

are on track for college and career readiness

College readiness score to identify who is ready for college-level

coursework

Success In first-year,

credit-bearing, postsecondary

coursework

Targeted interventions and supports:

• State-developed 12th-grade bridge courses

Ongoing student support/interventions

Professional development for educators

Assessments ELA/Literacy and Mathematics, Grades 3–11

5

Beginning of School Year

End of School Year

Diagnostic Assessment

Mid-Year Assessment

Performance-Based

Assessment

End-of-Year Assessment

Speaking and Listening

Assessment

Optional Required

Key:

Flexible administration

Promoting Student Access

PARCC is committed to the following principles:

• Use Universal Design principles to create accessible assessments throughout every stage and component of the assessment

• Minimize/eliminate features of the assessment that are irrelevant to what is being measured, so that all students can more accurately demonstrate their knowledge and skills

• Measure the full range of complexity of the standards

• Use technology to make all components of the assessment as accessible as possible

• Conduct bias and sensitivity reviews of all PARCC items

PARCC Accessibility System

7

Features for All Students

Tools embedded in the test platform

Accessibility Features for All Students

Identified in advance

Accommodations

for Students with Disabilities, English

Learners, and English Learners with

Disabilities


• Features that PARCC will make available to all students, either through the online platform or through the test administration process.

• Each student should determine whether they wish to use the support on an item-by-item basis, based on the supports they use during instruction and in daily life.

• Some features must be identified in advance as part of the student’s PNP because of the concern of student overload or clash of supports.

• All of these features are based on research and universal design features principals.

8


9

Embedded Features Audio Amplification

Blank Paper (provided by test administrator)

Eliminate Answer Choices

Flag Items for Review

General Administration Directions Clarified (by test administrator)

General Administration Directions Read Aloud and Repeated as Needed

(by test administrator)

Highlight Tool

Headphones

Magnification/Enlargement Device

NotePad

Pop-up Glossary

Redirect Student to the Test (by test administrator)

Spell Checker

Writing Tools

Embedded Features Demonstration

Example of “eliminate answer choice.” *NOTE: NOT a PARCC item. Not on the PARCC delivery platform.

Embedded Features Demonstration

Example of “highlighting.” *NOTE: NOT a PARCC item. Not on the PARCC delivery platform.

Accessibility Features to be Selected in Advance

12

Accessibility Features

Adaptive and Specialized Equipment or Furniture

Answer Masking

Background/Font Color (Color Contrast)

General Masking

Line Reader Tool

Text-to-Speech for the Mathematics Assessments

13

The PARCC Assessment System:

Design, Development and Critical Advances

Model Content Frameworks and Assessment Development

• The Model Content Frameworks were developed through a state-led process that included content experts from PARCC member states and members of the Common Core State Standards writing team.

• The Model Content Frameworks were constructed based on the Common Core State Standards for use in guiding and framing item development for the PARCC assessment.

6

What is Different About PARCC’s Development Process?

15

• PARCC states first developed the Model Content Frameworks to provide guidance on key elements of excellent instruction aligned with the Standards.

• Then, those Frameworks informed the assessment blueprint design.

So, for the first time. . .

• PARCC is communicating in the same voice to teachers as it is to assessment developers!

• PARCC is designing the assessments around exactly the same critical content the standards expect of teachers and students.

Evidence-Centered Design (ECD) for the PARCC Assessments

16

Model Content Frameworks

To make claims about what students know, we must operationalize the standards

Evidence Statements

Based on analysis, evidence drive task development

Tasks

Tasks are designed to elicit specific evidence from students

Item Development

17

• Item development began in fall 2012 • Item and passage reviews take place regularly,

with teams of reviewers: o K-12 content experts o Higher education faculty o Local educators o Community members

• Item development is on schedule, and the vendors will meet the August 30 benchmark to complete all items for field testing.

PARCC Cognitive Complexity Framework Guides Item Development

• CCSS demand a new type of cognitive complexity framework.

• PARCC partnered with the Item Development contractors to develop a new cognitive complexity framework.

• New framework :

o Provides a systematic, replicable method for determining item cognitive complexity

o Provides a measurement precision at all levels of the test score scales

o Enables development of test forms with adequate score reliability to support achievement growth interpretations

PARCC’s Cognitive Complexity Framework for ELA/Literacy

19

• The Cognitive Complexity Framework guides item development and recognizes that text complexity and item/task complexity interact to determine the overall complexity of a task.

• For the reading claim, the performance levels at each grade level are differentiated by three factors: (1) text complexity; (2) the range of accuracy in expressing reading comprehension demonstrated in student responses; and (3) the quality of evidence cited from sources read

• For the writing claim, PLDs are written for the two sub-claims: (1) written expression, and (2) knowledge of language and conventions

Claims Driving Design: ELA/Literacy

20

Students are on-track or ready for college and careers

Students read and comprehend a range of sufficiently complex texts

independently

Reading Literature

Reading Informational

Text

Vocabulary Interpretation

and Use

Students write effectively when using

and/or analyzing sources.

Written Expression

Conventions and

Knowledge of Language

Students build and present

knowledge through

research and the

integration, comparison,

and synthesis of ideas.

Item Types That Showcase Students’ Command of Evidence with Complex Texts

21

• Evidence-Based Selected Response (EBSR)—Combines a traditional selected-response question with a second selected-response question that asks students to show evidence from the text that supports the answer they provided to the first question. Underscores the importance of Reading Anchor Standard 1 for implementation of the CCSS.

• Technology-Enhanced Constructed Response (TECR)—Uses technology to capture student comprehension of texts in authentic ways that have been difficult to score by machine for large scale assessments (e.g., drag and drop, cut and paste, shade text, move items to show relationships).

• Range of Prose Constructed Responses (PCR)—Elicits evidence that students have understood a text or texts they have read and can communicate that understanding well both in terms of written expression and knowledge of language and conventions. There are four of these items of varying types on each annual performance-based assessment.

Research Simulation Task (Grade 7): Amelia Earhart’s Disappearance

22

• Session 1:

o Students begin by reading an anchor text that introduces the topic. EBSR and TECR items ask students to gather key details about the passage to support their understanding.

o Then, they write a summary or short analysis of the piece.

• Session 2:

o Students read two additional sources (may include a multimedia text) and answer a few questions about each text to learn more about the topic so they are ready to write the final essay and to show their reading comprehension.

o Finally, students mirror the research process by synthesizing their understandings into an analytic essay using textual evidence from several of the sources.

Understanding the Research Simulation Task

23

• Range: Example of assessing reading across the disciplines

and helping to satisfy the 55%-45% split of informational text

to literature at the 6-8 grade band.

• Quality: The texts on Amelia Earhart represent content-rich

nonfiction on a topic that is historically significant.

• Complexity: Quantitatively and qualitatively, the passages

have been validated and deemed suitable for use at grade 7.

Texts Worth Reading?

24

ELA/Literacy: Grade 7 Sample Item

25

Part A: Highlight the claim that is supported by the most relevant and sufficient facts within “Earhart’s Final Resting Place Believed Found.”

Part B: Click on two facts within the article that best provide evidence to support the claim selected in Part A.

Claims

Earhart and Noonan lived as castaways on Nikumaroro Island.

Earhart and Noonan’s plane crashed into the Pacific Ocean

People don’t really know where Earhart and Noonan died.

Grade 7 Analytical Prose Constructed-Response Item #1

Based on the information in the text “Biography of Amelia

Earhart,” write an essay that summarizes and explains the

challenges Earhart faced throughout her life. Remember to use

textual evidence to support your ideas.

Questions Worth Answering?

26

Final Grade 7 Prose Constructed-Response Item #2

You have read three texts describing Amelia Earhart. All three include the

claim that Earhart was a brave, courageous person. The three texts are:

• “Biography of Amelia Earhart”

• “Earhart's Final Resting Place Believed Found”

• “Amelia Earhart’s Life and Disappearance”

Consider the argument each author uses to demonstrate Earhart’s bravery.

Write an essay that analyzes the strength of the arguments about Earhart’s

bravery in at least two of the texts. Remember to use textual evidence to

support your ideas.


27

28

Narrative Task (Grade 6): Jean Craighead George’s

Excerpt from Julie of the Wolves

Understanding the Narrative Writing Task

29

• Students read one or two brief texts and answer a few

questions to help clarify their understanding of the text(s).

• Students then write either a narrative story or a narrative

description (e.g., writing a historical account of important

figures; detailing a scientific process; describing an account

of events, scenes, or objects).

Texts Worth Reading?

30

• Range: Example of assessing literature and helping to satisfy the

55%-45% split of informational text to literature at the 6-8 grade-

band.

• Quality: Julie of the Wolves was a winner of the Newbery Medal in

1973. This text about a young Eskimo girl surviving on her own in

the tundra by communicating with wolves offers a story rich with

characterization and imagery that will appeal to a diverse student

population.

• Complexity: Quantitatively and qualitatively, the passages have

been validated and deemed suitable for use at grade 6.


31


32


33

Grade 6 Prose Constructed Response from Narrative Writing Task

In the passage, the author developed a strong character named Miyax. Think about Miyax and the details the author used to create that character. The passage ends with Miyax waiting for the black wolf to look at her.

Write an original story to continue where the passage ended. In your story, be sure to use what you have learned about the character Miyax as you tell what happens to her next.

Design of PARCC Math Summative Assessment

• Performance Based Assessment (PBA)

– Type I items (Machine-scorable)

– Type II items (Mathematical Reasoning/Hand-Scored – scoring rubrics are drafted but PLD development will inform final rubrics)

– Type III items (Mathematical Modeling/Hand-Scored and/or Machine-scored - scoring rubrics are drafted but PLD development will inform final rubrics)

• End-of-Year Assessment (EOY)

– Type I items only (All Machine-scorable)

Overview of PARCC Mathematics Task Types

35

Task Type Description of Task Type

I. Tasks assessing concepts, skills and procedures

• Balance of conceptual understanding, fluency, and application • Can involve any or all mathematical practice standards • Machine scorable including innovative, computer-based formats • Will appear on the End of Year and Performance Based Assessment

components • Sub-claims A, B and E

II. Tasks assessing expressing mathematical reasoning

• Each task calls for written arguments / justifications, critique of reasoning, or precision in mathematical statements (MP.3, 6).

• Can involve other mathematical practice standards • May include a mix of machine scored and hand scored responses • Included on the Performance Based Assessment component • Sub-claim C

III. Tasks assessing modeling / applications

• Each task calls for modeling/application in a real-world context or scenario (MP.4)

• Can involve other mathematical practice standards • May include a mix of machine scored and hand scored responses • Included on the Performance Based Assessment component • Sub-claim D

For more information see PARCC Task Development ITN Appendix D.

Claims Driving Design: Mathematics

36

Students solve problems involving the major

content for their grade level with connections to

practices

Students solve problems involving the additional and supporting content for their grade level with connections to practices

Students express mathematical reasoning

by constructing mathematical arguments

and critiques

Students solve real world problems

engaging particularly in the modeling practice

Student demonstrate fluency in areas set forth

in the Standards for Content in grades 3-6

Students are on-track or ready for college and careers

Factors that determine the Cognitive Complexity of PARCC Mathematics Items

37

Cognitive Complexity

Mathematical Content

Mathematical Practices

Stimulus Material

Response Mode

Processing Demand

1. Mathematical Content

2. Mathematical Practices

3. Stimulus Material

4. Response Mode

5. Processing Demand

For further reading on the PARCC Cognitive Complexity Framework see, “ Proposed Sources of Cognitive Complexity in PARCC Items and Tasks: Mathematics “ Aug. 31, 2012

Math: Grade 3 Sample Item

38

• This a fairly traditional fraction task in a computer-based setting.

• Unlike traditional multiple choice, it is difficult to guess the correct answer or use a choice elimination strategy and there is more than one correct solution.

• Unlike paper and pencil tests, students can create a visual representation even though the task is scored automatically.

Math: Grade 3 Sample Item

39

• Second part of multi-step problem, and, unlike traditional multiple choice, it is difficult to guess the correct answer or use a choice elimination strategy.

Math: High School Sample Item

40

Item has two possible solutions

Students have to recognize the nature of the equation to know how to solve

Technology prevents guessing and working backward

41

A New Vision of Assessment

June 2013

www.PARCConline.org

http://www.parcconline.org/

a new vision of assessment

Documents