www.smcoe.org

San Mateo County Results from the 2014 SBAC Field Test Survey

Deann WalshManager, Learning Analytics & Program Evaluation

www.smcoe.org

Agenda

• Field Test Overview

• Field Test Survey Results

Student Survey

Teacher/Proctor Survey

Administrator Survey

• Next Steps

www.smcoe.org

“A field test is not designed to be a valid and reliable measure of student achievement; rather, it is designed to help the test developers evaluate whether the test, individual items, and the technology platform work as intended before the first operational administration.”

Deborah S. Delisle, U.S. Department of Education

www.smcoe.org

Field Test Overview• Timeline:

o March 25th through June 6th

o Schools assigned a five- or six- week window

• Participants:

o Grades 3-8: All students

o Grades 9 & 10: Only students selected for the scientific sample

o Grade 11: Students selected for the scientific sample; all others encouraged

• Components:

o Performance Task

o Non-performance Task

• Format: Computer-based; not computer adaptive

www.smcoe.org

2014 SBAC Field Test Surveys

Survey Number of Responses

Number of Districts Responding

Student 6400 16

Teacher/Proctor 336 17

Administrator 56 13

www.smcoe.org

Student Survey Results

• Methodology Used

o Weighted student results based on % of 2013 STAR testing population

• Approximately 28% of students encountered problems with the technology or testing system

Student Preparation50%

Completed a practice test

together as a class

53%Completed a practice test individually

54%Teacher reviewed technology

needed for testing (i.e. computer, mouse)

www.smcoe.org

Student Survey Results, Continued

Testing Conditions

33%Computer Lab

56%Classroom

with Laptops

4%Classroom

with Tablets7%

Other

Content

Approximately 56% of students polled felt comfortable or very comfortable with their understanding of the questions asked

www.smcoe.org

Teacher/Proctor Survey Results

* Highest ranked item of the PD options

Teacher Preparation

93%Overview of

the SBAC Field Test

66%Instruction on the selection and

administration of universal tools, designated supports and

accommodations

78%Demonstration of

the test administrator

interface

70%Physical

copies of required manuals

www.smcoe.org

Teacher/Proctor Survey Results

Impact on Instruction53% indicated that the use of computer devices for testing purposes impacted their ability to use technology for instructional purposes

Professional Development85% of teachers ranked “training on preparing students for the content” as

important or very important*

*Highest ranked item of the PD options

www.smcoe.org

Administrator Survey Results

Technology & Systems66%

Experienced significant glitches in the system (e.g. device failure,

missing log-in information)

89%Indicated that the network and bandwidth were reliable during

testing

Impact on Instruction67% indicated that the use of computer devices for testing

purposes impacted the school’s ability to use technology for instructional purposes

www.smcoe.org

Next Steps

• SBAC Digital Library

o Preview began on June 3rd

o Digital Library Orientation @ SMCOE August 5th

o Operational version to begin on October 1st

• Interim Assessments – Available Fall 2014

o Interim Comprehensive Assessment (ICA)

o Interim Assessment Blocks (IAB)

www.smcoe.org

Next Steps

• Summative Assessment – Spring 2015

o Administered in the last 12 weeks of the school year

o Operational test will differ from field test in terms of duration & length

www.smcoe.org

SMCOE Support

• CIS Support

o Teacher Professional Development

o Administrator Professional Development

• ESS Support

o District Assessment Leader Network

o Regional Learning Analytics Institutes

o Target Supported on Universal Tools, Designated Supports and Accommodations with Special Education & Assessment Staff

www.smcoe.org

Questions?