NEXT GENERATION ASSESSMENTS

K-12 Reinvention SymposiumOctober 22-24, 2010

A CALL FOR NEXT GENERATION ASSESSMENTS

“I am calling on our nation’s Governors and state education chiefs to develop standards and assessments that don’t simply measure whether students can fill in a bubble on a test, but whether they possess 21st century skills like problem-solving and critical thinking, entrepreneurship and creativity.”

-- President Barack Obama March 10, 2009

HOW THE DEMAND FOR SKILLS HAS CHANGEDECONOMY-WIDE MEASURES OF ROUTINE AND NON-ROUTINE TASK INPUT (U.S.)

1960 1970 1980 1990 200240

45

50

55

60

65

Routine manualNonroutine manualRoutine cognitiveNonroutine analyticNonroutine interactive

(Levy and Murnane)Mea

n ta

sk in

put a

s per

cent

iles o

f th

e 19

60 ta

sk d

istrib

utio

n

The dilemma of schools:The skills that are easiest to teach and test are also the ones that are easiest to digitize, automate, and outsource

CONCERNS ABOUT THE EFFECTS OF HIGH-STAKES MULTIPLE-CHOICE TESTS ON INSTRUCTION “I have seen more students who can pass the [state test] but cannot apply those skills to anything if it’s not in the test format. I have students who can do the test but can’t look up words in a dictionary and understand the different meanings…. As for higher quality teaching, I’m not sure I would call it that. Because of the pressure for passing scores, more and more time is spent practicing the test and putting everything in [the test] format.”-- A Texas Teacher

WORLDWIDE REFORM INITIATIVES Expectations for higher- order skills Rich content Modes of inquiry “Teach less, learn more” Project work, tasks requiring research,

analysis, application, self assessment, production

Performance tasks Assessment of, as, and for learning Learning progressions

COMMON PRACTICES ACROSS COUNTRIES Tightly integrated system Assessments include evidence of actual

student work Teachers integrally involved Assessments inform course grades, provide

information to colleges and employer, not to determine diploma

Assessments designed to continuously improve teaching and learning

HOW ASSESSMENT SYSTEMS AIM TO IMPROVE TEACHING AND LEARNING Together on-demand and curriculum

embedded assessments evaluate analytic & performance abilities – measure the full range of knowledge and skills

Moderated teacher scoring Learning progressions (shape curriculum,

personalizes learning, know where the student is

School based assessments model good instruction

THE CHALLENGEHow do we get from here... ...to here?

All students leave high

school college and

career ready

Common Core State Standards

specify K-12 expectations for

college and career readiness

AND HOW CAN ASSESSMENT CONTRIBUTE?

How do we get from here...

All students leave high

school college and

career ready

Common Core State Standards

specify K-12 expectations for

college and career readiness

...to here?

RACE TO THE TOP ASSESSMENT PROGRAM COMPETITION $350 million of Race to the Top Fund set aside for

awards to consortia of states to design and develop common K-12 assessment systems aligned to common, college- and career-ready standards

Competition asked consortia to design assessment systems that meet dual needs of: Instructional improvements Accountability

In September 2010, the U.S. Department of Education awarded two grants: Partnership for Assessment of Readiness for College and Careers

(PARCC) Smarter Balanced Assessment Consortium (SBAC)

The winning consortia have four years to develop assessment systems, and participating states will administer new assessment statewide by 2014-2015

Partnership for Assessment of College and Career Readiness (PARCC)

12

PARCC STATES

PARCC STATES11 Governing States –Arizona–District of Columbia–Florida-Fiscal Agent–Illinois–Indiana–Louisiana–Maryland–Massachusetts-Board Chair–New York–Rhode Island–Tennessee

13

15 Participating StatesAlabamaArkansasCaliforniaColoradoDelawareGeorgiaKentuckyMississippi

New HampshireNew JerseyNorth DakotaOhioOklahomaPennsylvaniaSouth Carolina

BACKGROUNDSTATES INVOLVED

Fiscal Agent: Washington State

Governing AdvisoryCT, HI, ID, KS, ME, MI, MO, MT, NC, NM, NV, OR, UT, VT, WA, WI, WV

AL, CO, DE, GA, IA, KY, ND, NH, NJ, OH, OK, PA, SC, SD

17 14Total Number of States = 31

GOALS OF CONSORTIUMPARCC

States in the Partnership are committed to building their collective capacity to increase the rates at which students graduate from high school prepared for success in college and the workplace.

SMARTER

To ensure that all students leave high school prepared for postsecondary success in college or a career through increased student learning and improved teaching.

18

PARCC ASSESSMENT SYSTEM DESIGN: MEASURING MASTERY OF THE COMMON CORE STANDARDS

ELA/LiteracyReading complex literary and informational textWriting – both on demand and over timeResearchSpeaking and listeningLanguage , conventions, vocabularyMathematicsConceptual understanding and procedural fluencyApplication via modeling and strategic problem solvingReasoning, explaining, justifying

19

PARCC ASSESSMENT SYSTEM DESIGN: MEASURING MASTERY OF THE COMMON CORE STANDARDS

ELA/LiteracyReading complex literary and informational textWriting – both on demand and over timeResearchSpeaking and listeningLanguage , conventions, vocabularyMathematicsConceptual understanding and procedural fluencyApplication via modeling and strategic problem solvingReasoning, explaining, justifying

20

PARCC ASSESSMENT SYSTEM DESIGNThe through-course components in both subjects

will be administered after approximately 25 percent, 50 percent and 75 percent of instruction

ELA 1 and ELA 2 One or two tasks involving reading, drawing conclusions and presenting analysis in writing

Math 1 and Math 2 One to three tasks that assess one of two essential topics in mathematics (standards or clusters of standards)

PARCC ASSESSMENT SYSTEM DESIGN

ELA 3: Performance tasks) that require evaluating information from within a set of digital resources, evaluating their quality, selecting resources, and composing an essay or research paper

ELA 4 (speaking and listening) Students will present their work from ELA 3 to classmates and respond to questions. Teachers will score, using a standardized rubric, and can use results in determining students class grades.

Math 3 Performance tasks) that require conceptual understanding, procedural fluency, and application of mathematical tools and reasoning.

PARCC ASSESSMENT SYSTEM DESIGNEnd-of-Year – Comprehensive, computer-scored

assessment that includes a range of item types, including innovative, technology-enhanced items. Enables quick turnaround of student work.

A students summative score – used for accountability purposes – will include his/her performance on Through-Courses 1, 2, and 3 as well as the end of year assessment

23

PARCC ASSESSMENT SYSTEM DESIGN:FORMATIVE ASSESSMENTFormative Tools “[Thoughtful, curriculum-aligned, and valid ways of

determining what students know, rather than leaving the burden of planning and assessing on the teacher alone.” Shavelson et al. (2008):

Partnership Resource Center (PRC)―an online, digital resource that includes two supports:

released items with item data, student work, rubrics model curriculum frameworks

Text Complexity Diagnostic Tool: a computer adaptive tool to identify students’ proximate zone of development and supply suggestions for appropriate texts for students to read

K-2 Assessments in ELA/Literacy and Mathematics

24

PARCC ASSESSMENT SYSTEM DESIGNData to Support Instruction, Professional

Development, and Accountability Decisions– Types of Data

• Student achievement scores and growth measures• Information on readiness for college and careers • Item analysis of released items

– Reporting• Periodic Feedback Reports• Annual Stakeholder Reports• Item Analysis Reports• Interactive Data Tool (online)

– Available data will be aggregated at a level appropriate for each audience and report

25

PARCC ASSESSMENT SYSTEM DESIGN

Accessibility and AccommodationsStudents with disabilities and English Learners will be

considered from onset of development process Items and test forms will be created using an evidence

centered design approachUniversal design methods will be considered in every step of

the processAccessibility and Accommodations committee will be formed

to advise Partnership

PARCC TIMELINE

October 2010 Launch and

design phase

September 2011

Development and Design

begins

September 2012

First year field testing and

related research and

data collection

begin

September 2013 Second

year field testing begins

and related research and

data collection continues

September 2014 Full

administration of PARCC assessment

begins

Summer 2015 Set achievem

ent levels,

including college-ready

performance

levels

http://www.fldoe.org/parcc/

Partnership for Assessment of Readiness for College and Careers

HISTORICAL DEVELOPMENT OF THE SMARTER BALANCED CONSORTIUM

SMARTER

MOSAIC

Balanced

• Computer Adaptive

• Formative Capacity

• Integrated System

THE PURPOSE OF THE CONSORTIUM

To develop a set of comprehensive and innovative assessments for grades 3-8 and high school in English language arts and mathematics aligned to the Common Core State Standards.

The assessments shall be operational across Consortium states in the 2014-15 school year.

THEORY OF ACTION• A model of verifiable accomplishments/milestones,

leading to the desired outcome

• Accomplishments/milestones are inter-dependent

• The theory of action is closely linked to the validation argument for the assessment system

31

The SMARTER Balanced Theory of Action

All students leave high

school college and

career ready

Summative adaptive

assessments are benchmarked to college & career

readiness

Technology supports

innovative & comprehensi

ve assessments

Technology provides increased access to learning

State policies and practices

support increased

expectations

Common Core State

Standards specify K-12 expectations

for college and career

readiness

Clear communication of

expectations to stakeholders

Professional

capacity-building

PD and other supports for teachers to instruct on the CCSS

Teachers design and

score assessment

items & tasks

Teachers use formative tools and

practices to improve

instruction

Interim/Benchmark assessments are used as progress

checks

32

The SMARTER Balanced Theory of Action

All students leave high

school college and

career ready

Summative adaptive

assessments are benchmarked to college & career

readiness

Technology supports

innovative & comprehensi

ve assessments

Technology provides increased access to learning

State policies and practices

support increased

expectations

Common Core State

Standards specify K-12 expectations

for college and career

readiness

Clear communication of

expectations to stakeholders

Professional

capacity-building

PD and other supports for teachers to instruct on the CCSS

Teachers design and

score assessment

items & tasks

Interim/Benchmark assessments are used as progress

checks

Teachers use formative tools and

practices to improve

instruction

THEORY OF ACTIONAssessment system that balances

summative, interim/benchmark, performance, and formative components for ELA and mathematics:

1.Computer adaptive assessment system/summative

a. Grades 3–8 and 11 (testing window within the last 12 weeks of the instructional year)

b. Selected response, enhanced constructed response, technology enhanced, and performance events (grades 3–8, 2/year: HS up to 6)

2.Adaptive interim/benchmarka. Learning progressionsb. Administered throughout the year

3.Formative Processes and Tools

ASSESSMENT DESIGNThe Consortium will provide the following by

the 2014-15 school year:3. Formative tools and resources4. Responsible flexibility5. Distributed summative assessment

a. Content clusters throughout a courseb. Most appropriate time for each studentc. Scores rolled up

Smarter Balanced Assessment Consortium

35 AFT October 20, 2010

Assessment System Highlights

SYSTEM HIGHLIGHTSSummative assessments using online computer

adaptive technologies• Efficiently provide accurate measurement of all students,

across the spectrum of knowledge and skills• Incorporate adaptive precision into performance tasks

and events• Will assess full range of CCSS in English language arts and

mathematics• Describe both current achievement and growth across

time, showing progress toward college- and career-readiness• Scores can be reliably used for state-to-state comparability,

with standards set against research-based benchmarks • The option of giving the summative tests twice a year.

SYSTEM HIGHLIGHTSOptional interim/benchmark assessments

• Are aligned to and reported on the same scale as the summative assessments

• Help identify specific needs of each student, so teachers can provide appropriate, targeted instructional assistance

• Incorporate significant involvement of teachers in item and task design and scoring

• Are non-secure and fully accessible for use in instruction and professional development activities

• Provide students and teachers with clear examples of the expected performance on common standards.

SYSTEM HIGHLIGHTSDigital library of formative tools

Instructionally sensitive, on-demand measures that enable differentiation of instruction

Use is associated with improved teaching and increased student learning

SYSTEM HIGHLIGHTSComputer Adaptive TechnologyAllows for the breadth of the

Common Core State Standards to be comprehensively assessed while minimizing test length.

Allows increased measurement precision relative to fixed form assessments.

Allows items to be presented as a function of student ability as measured during the test.

SYSTEM HIGHLIGHTSOnline, tailored reporting system

• Supports educator access to information about student progress toward college- and career-readiness

• Allows for exchange of student performance history across districts and states

• Uses a Consortium-supported backbone, while individual states retain jurisdiction over access permissions and front-end “look” of online reports.

SYSTEM HIGHLIGHTSBenefits and efficiencies from

“economies of scale” due to a multi-state consortium • Cost savings• Shared interoperable open source software

platforms: Item generation, item banking, and adaptive testing no longer exclusive property of vendors

• Common, agreed-upon protocols for accommodations for students with disabilities and ELL students.

43 AFT October 20, 2010

Professional Capacity and

Outreach

45 AFT October 20, 2010

Collaboration with Higher Education

IHE COLLABORATIONGOAL: Better prepare students for college-

and career-readiness.

Collaborate with IHEs to create student achievement standards that define college-ready.

Students will enter IHE systems having met common, clear college-ready standards.

Students will be able to track readiness for college and careers throughout high school.

TIMELINE

TO FIND OUT MORE......the SMARTER Balanced Assessment

Consortium can be found online at

www.k12.wa.us/SMARTER

CCSS APPENDIX BText Exemplars

Stories/LiteraturePoetryDramaInformational (ELA, Science, Social Studies)

Performance Assessment Examples

PROFESSIONAL CAPACITY BUILDINGCollaboration with existing professional

development networks in each participating State in order to:• Create a sustainable structure that supports teaching,

learning, reliable scoring of performance assessments, and development of formative tools and processes, and

• Foster interdependence and opportunities for shared learning among teachers and school leaders across States.

PROFESSIONAL CAPACITY BUILDINGCollaboration with existing professional

development networks in each participating State in order to:• Provide access to authentic learning experiences

among teachers and school leaders, and• Bring added efficiency and cost effectiveness to

professional capacity building activities.

DEVELOPING A TRANSITION STRATEGY Rigor/Relevance framework

Natural fit for Common Core Standards Emphasis on Deep knowledge and applied real

world situations Designing curriculum/instruction/assessments

RIGOR/RELEVANCE IN ASSESSMENTQuadrant A – Acquisition (Low Rigor/Low

Relevance

Use verbs synonymous with recall and understanding Call for explanation of knowledge or skill

Multiple choice, true/false or short answerRequire single standard response

RIGOR/RELEVANCE IN ASSESSMENTQuadrant B – Application (Low Rigor/High

Relevance)Verbs synonymous with recall, understanding, or applicationApplication of knowledge to real-world situationsPerformance-basedFollow a routine or set of procedures

RIGOR/RELEVANCE IN ASSESSMENTQuadrant C – Assimilation (High

Rigor/Low Relevance)Verbs synonymous with analysis, synthesis or evaluationCall for the explanation of knowledge or skill, but not real-world applicationInclude multiple step problemsEssays, presentations, or portfolios

RIGOR/RELEVANCE IN ASSESSMENTQuadrant D – Adaptation (High Rigor/High

RelevanceUse verbs synonymous with analysis, synthesis or evaluationCall for unique solutions in applying knowledge to real world problemsPerformance-basedInclude multi-step problems

GRADE 4/NAEP Sam can purchase his lunch at school. Each

day he wants to have juice that costs 50¢, a sandwich that costs 90¢, and fruit that costs 35¢. His mother has only $1.00 bills. What is the least number of $1.00 bills that his mother should give him so he will have enough money to buy lunch for 5 days?

GRADE 4/NAEP

RELEASED GRADE 4/WRITING Situation: The local newspaper is having a “Good

Friend” contest. To enter your friend, you must think of an event in your life when your friend did something with you or for you that showed what a terrific friend he or she is.

 

GRADE 4 Writing Task: Select your friend. (Remember, a friend

could be a child your age or a grownup.) Choose an event that shows how your friend is a good friend to you. Write a letter to the newspaper that tells about that event so that people will know why your friend deserves to win.

GRADE 7Situation: Not only do games have rules, but there are

also rules of respect and safety at home and at school. These rules help everyone understand what to do, how and when to do the activities, and how to be safe. Think of a time that you needed to know the rules for an activity. What happened? Why were the rules important? Could someone else learn from your experience? Your school newspaper is running a series of first-person articles about lessons students have learned.

 

GRADE 7 Writing Task: In an article for your school newspaper, tell

about a time when knowing the rules was important.

STRATEGY TO CONSIDER Step 1: By content area, identify assessment

anchors that are a natural fit for each content area. Be hard-nosed. Only select what you are willing to teach until students learn it!

Step 2: School wide, track the “adoption” of assessment anchors, identify orphans, and make adjustments as needed.

Step 3: By course, design curriculum, instruction, and assessment to teach students to be proficient in all assessment anchors assigned to specific content areas/courses.

RELEASED ITEMSResources

State released itemsNAEPDepth of Knowledge Levels 3 & 4Assessment Kits

DEVELOP A THOUGHTFUL PLAN OF ACTION

SUSANGENDRON1@GMAIL.COM