bcn update for transitioning to a new assessment system

CALIFORNIA DEPARTMENT OF EDUCATIONTom Torlakson, State Superintendent of Public Instruction

Bilingual Coordinator’s Network

November 15, 2012

Patrick Traynor, Ph.D.Director, Assessment Development & Administration Division

BCN Update for Transitioning to a New Assessment

System

TOM TORLAKSONState Superintendent of Public Instruction

2

The STS Pre AB 1521

Standards-Based Tests in Spanish (STS)•Required for Spanish-speaking English learners (ELs) who either:

Receive instruction in their primary language; or Have been enrolled in a school in the U.S. less

than 12 months

•May also be administered, at the option of the school district, to any other Spanish-speaking ELs

•Not available for non-ELs in dual immersion programs


3

AB 1521 (Brownley)Approved by Governor September 21, 2012•Upon approval of the state board, the department may make the STS available to pupils enrolled in a dual language immersion program:

• Nonlimited English proficient or

• Redesignated fluent English proficient.

•Administer the assessment at own expense and enter into an agreement with the state testing contractor

•The cost for the assessment will be the same for all school districts and charter schools


4

AB 1521 (Brownley)ETS Options for Implementation:•Survey schools and districts to determine number of students

•2013 Possible Scenarios:

• High risk making edits to order management systems

• Flag assessments and answer documents

• Fold into current order management process

• Manual Operations• Add separate ordering process for dual immersion

students

• Return to ETS for hand scoring

• Provide raw score results


5

AB 1521 (Brownley)ETS Options for Implementation: (cont)•2014 Possible Scenarios:

• Incorporate in current order management system

• Provide On-line Pre-ID services

• Monitor inventory controls for smooth deliveries

• Provide a flag on the answer document to identify hand marked dual immersion test takers

• Address change in law in manuals and directions for administration


Alternate Assessment Participation

• California recently joined the National Center and State Collaborative (NCSC) as a Tier II state

• Developing alternate assessments for students with significant cognitive disabilities

• Based on alternate achievement standards

6


Reauthorization

• Standardized Testing and Reporting (STAR) Program sunsets July 2014

• California Education Code 60604.5 amended

• Extensive consultation with stakeholders– Outreach plan activities

7


8

Proposed Web Site RedesignHome Page


9

Proposed Web Site RedesignCurrent Testing Standards—Filtering Feature


10

Proposed Web Site RedesignCurrent Testing Standards—Information About the Standard


11

Proposed Web Site RedesignCurrent Testing Standards—Sample Question


12

Proposed Web Site RedesignCurrent Testing Standards—Related CCSS Information


13

Proposed Web Site RedesignCommon Core—Filtering Feature


14

Proposed Web Site RedesignCommon Core—Standards Information


15

Proposed Web Site RedesignCommon Core—Comparison to Current Testing Standards


16

Proposed Web Site RedesignCommon Core—Example Related Sample Question


Current Assessment System

• Originally designed in 1997• Modifications made over the years• Current STAR system includes:

– California Standards Tests– California Modified Assessment– Standards-based Tests in Spanish– California Alternate Performance Assessment

• Other parts of state assessment system:– California High School Exit Examination– California English Language Development Test– Physical Fitness Test

17


Smarter Balanced Participation

• In June 2011, California joined the Smarter Balanced Assessment Consortium (SBAC)

• Developing assessments in English–language arts (ELA) and mathematics in grades three through eight and grade eleven

• Optional formative assessment tools and interim assessments

18


Purposes of the New Assessment System

• Design an assessment system that includes a variety of assessment approaches and item types that has as its primary purpose to model and promote high-quality teaching and student learning activities.

19


Purposes of the New Assessment System (cont.)

In accomplishing the purpose, the system can also:•Produce scores that can be aggregated for the purpose of holding schools and districts accountable for the progress of their students in learning the California academic content standards.

•Provide assessments and/or assessment tools in multiple grades covering the full breadth of the curriculum to provide clear expectations and incentives for teaching the full curriculum.

20


Guiding Principles

• Assess subjects and learning in ways that promote high-quality instruction

• Conform to rigorous industry standards for test development

• Use resources efficiently and effectively• Provide for inclusion of all students• Provide information on the assessment

system that is readily available and understandable to parents, teachers, schools, and the public

21


Considerations for Developing California’s Future

Assessment System• Adoption of the Common Core State

Standards and sunset of the STAR Program• Presents a set of challenges and opportunities

for the future assessment system• Important to decide what information is

expected from the tests– Consider resources– Test subjects beyond ELA and mathematics– Progress or growth from year to year– Plan for how to use the results

22


Strengths of the Current System

• Paper and pencil multiple-choice assessments inexpensively developed, administered, and scored

• Reliable results• Provide secure measures of achievement• Use of multiple-choice approach has

allowed for a wide variety of tests and a high level of reliability and validity in the accountability system

23


Limitations of the Current System

• Multiple-choice assessments limits the types of knowledge and skills measured (less depth)

• Limited types of items and formats• Criticized for negatively influencing

instruction through narrowing of the curriculum

• Not sufficient for providing diagnostic information

24


Determining What to Test

• CCSS requires a more integrated approach to delivering instruction

• SBAC assessments use multiple item types such as multiple-choice, constructed response, technology enhanced, and performance tasks

• SBAC will provide optional formative assessment tools and interim assessments

• Automated scoring

25


Determining What to Test (cont.)

• Costs will likely be greater per student with SBAC assessments than the current system

• Need to consider possibly allocating additional resources for assessment, finding more efficient ways to assess subjects not included in SBAC, and reduce the number of grades and subjects assessed

• Utilizing various ways to administer, score, and report has the potential for realizing greater efficiencies

26


Determining What to Test (cont.)

• Choices beyond SBAC assessments– Accountability considerations– Interim and formative assessment– Reducing redundant testing– Matrix testing

• Balance costs of the system and the kind of assessment and reporting desired

27


Re-take option

Optional Interim assessment system—

Summative assessment for accountability

Last 12 weeks of year*

DIGITAL CLEARINGHOUSE of formative tools, processes and exemplars; released items and tasks; model curriculum units; educator training; professional development tools and resources; scorer training modules; and teacher collaboration tools.

Scope, sequence, number, and timing of interim assessments locally determined

* Time windows may be adjusted based on results from the research agenda and final implementation decisions.

Source: http://www.ets.org

Computer AdaptiveAssessment andPerformance Tasks

PERFORMANCETASKS

• Reading• Writing• Math

END OF YEARADAPTIVE

ASSESSMENT

English Language Arts and Mathematics, Grades 3–8 and High School

Computer AdaptiveAssessment andPerformance Tasks

BEGINNING OF YEAR

END OF YEAR

INTERIM ASSESSMENT INTERIM ASSESSMENT

28

Formative Assessments

SBAC Timeline


29

SBAC Schedule of Activities

• Sample item and performance task release – Oct. 9 – Early November 2012

• Small scale trial– Oct. 15 – Nov. 9, 2012

• Pilot test– Feb. 2013 – May 2013, Non-adaptive

• Field test– Spring 2014, Adaptive

• Operational system– Fall 2014, Adaptive


30

Pilot Test

• Pilot Test in early 2013

• Open to all schools in the Consortium

• 11% of students from each Governing State to be included in scientific sample

30


Purpose of Sample Items and Performance Tasks

• Demonstrate rigor and complexity of ELA/literacy and mathematics questions

• Showcase variety of item types:• Selected response

• Constructed response

• Technology enhanced

• Performance tasks

• Help teachers to begin planning for the shifts in instruction

31


Accessibility and Accommodations

• Sample items do not include accessibility and accommodations features

• Full range of accessibility tools and accommodations options under development guided by:– Magda Chia, Ph.D., Director of Support for Under-

Represented Students– Accessibility and Accommodations Work Group– Students with Disabilities Advisory Committee

• Chair: Martha Thurlow (NCEO)– English Language Learners Advisory Committee– Accessibility & Accommodations Framework

• Learn more online:– http://www.smarterbalanced.org/parents-students/

support-for-under-represented-students/

32


CDE SBAC Web page

33


Sample Items and Tasks SBAC Landing Page

34


Sample Items and Tasks Navigation

View mathematics or ELA/literacy items

Advance to next item, or go back to previous

35


Sample Items and Tasks Navigation (cont.)Content Claim

Grade band

36


Sample Items and Tasks Navigation (cont.)

Filter by item type, themes

37


Item Metadata

About this item

38


Exploring the Sample ItemsSelected response and technology enhanced items are machine scorable

39


Exploring the Mathematics Sample Items

40


Exploring the Mathematics Sample Items (cont.)

41


Exploring the ELA Sample Items

42


Exploring the ELA Sample Items (cont.)

43


44

SBAC Timeline

Field testing of summative

assessment, training school-

and district-level staff in formative

tools

Field testing of summative

assessment, training school-

and district-level staff in formative

tools

Teams of teachers evaluate formative

assessment practices and

curriculum resources

Teams of teachers evaluate formative

assessment practices and

curriculum resources

Full implementatio

n of assessment

system

Full implementatio

n of assessment

system

Formative Processes, Tools, and Practices Development Begins

Formative Processes, Tools, and Practices Development Begins

Pilot Testing of Summative and Interim Items/Tasks Conducted

Pilot Testing of Summative and Interim Items/Tasks Conducted

Writing and Review

Items/Tasks for Field Testing

(throughout the school year)

Writing and Review

Items/Tasks for Field Testing

(throughout the school year)

Technology readiness

tool available

Technology readiness

tool available

Formative tools available

to teachers

Formative tools available

to teachers

Content and Item

Specifications Development

Content and Item

Specifications Development

Writing and Review of Pilot Items/Tasks (including Cognitive

Labs and Small-Scale Trials)

Writing and Review of Pilot Items/Tasks (including Cognitive

Labs and Small-Scale Trials)


Resources• CDE/SBAC presentations and an electronic mailing

list are found on the CDE SBAC Web page at http://www.cde.ca.gov/sbac/

• Join the SBAC CDE electronic mailing list by sending a blank e-mail to [email protected]

• Visit the SBAC Website at www.smarterbalanced.org

• E-mail SBAC Questions to [email protected]

• Email IT Readiness Questions to [email protected]

45


Contact Information• Patrick Traynor, PhD, Director

Assessment Development and Administration Division – E-mail: [email protected]

• Jessica Barr, SBAC and Reauthorization Lead ConsultantAssessment Development and Administration Division– Phone: 916-319-0364– E-mail: [email protected]

• Jessica Valdez, SBAC Teacher Involvement Coordinator and NAEP CoordinatorAssessment Development and Administration Division– Phone: 916-319-0332– E-mail: [email protected]

46

bcn update for transitioning to a new assessment system

Documents