national conference on student assessment june 21, 2013 ccsso 2013 presentation1
TRANSCRIPT
National Conference on Student AssessmentJune 21, 2013
June 21, 2013CCSSO 2013 Presentation 1
One state’s model for using published authors to generate quality reading texts for next-generation assessments◦ Michigan’s vision◦ Process and training◦ Authors’ perspectives
June 21, 2013CCSSO 2013 Presentation 2
Andrew Middlestead, Test Development Manager, Michigan Department of Education
Kara Courtney, Data Recognition Corporation
Michael P. Spradlin, Author Shirley Neitzel, Author Patty McDivitt, Data Recognition
Corporation
June 21, 2013CCSSO 2013 Presentation 3
—Andrew Middlestead Office of Standards and Assessment in the Michigan
Department of Education
June 21, 2013CCSSO 2013 Presentation 4
Michigan currently has six assessment programs.◦ Michigan Educational Assessment Program
(MEAP)◦ Michigan Merit Examination (MME)◦ MEAP-Access (2%)◦ MI-Access (1%)◦ English Language Proficiency Assessment (ELPA)◦ Michigan’s Interim Assessment System
Five of these assessment programs use reading passages for the assessments.
June 21, 2013CCSSO 2013 Presentation 5
Similar to many states, Michigan was purchasing permissions for contexts for all programs.
Leadership made a decision to charge the test development team with transitioning to 100% commissioned passages for all programs.
Michigan wanted to look to authors from Michigan to write pieces for Michigan’s assessments.
June 21, 2013CCSSO 2013 Presentation 6
Cost Ability to modify the pieces at the time of
item writing and item review if necessary to produce high quality items and the required quantity of items
Ownership of the contexts (reduced effort in continually renewing or tracking copyright permissions)
Reduction in information transfer headaches during vendor changes!
June 21, 2013CCSSO 2013 Presentation 7
Michigan has its own item banking and authoring system (IBS).◦ Includes a passage writing and reviewing module
Authors may work online in the IBS on-site where training takes place, or they may work elsewhere.
Authors may work in a word-processing system of their choice and then enter their work into the IBS.
Flexibility for the authors is key!◦ The reading contexts are automatically in the
same system in which items are written and tests are built.
June 21, 2013CCSSO 2013 Presentation 8
Reduces information migration between various systems to get the context to the item-writing phase
Supports Michigan children’s authors Provides for flexibility with the contexts,
allowing Michigan to generate first-rate assessments for students
Reduces permissions cost and time
June 21, 2013CCSSO 2013 Presentation 9
Authors took some time to learn the ins and outs of writing for assessments just as item writers do. (“Why can’t we write about certain topics?”)
We learned how to create context-writing training and work sessions differently than we would for item writing (flexibility, space, open “check-in” opportunities).
June 21, 2013CCSSO 2013 Presentation 10
—Kara Courtney
June 21, 2013CCSSO 2013 Presentation 11
List provided by Michigan Reading Association through the Michigan Department of Education contact◦ Letter of interest◦ Communications
13 published authors out of 23 committed to attend training
Over 60 quality contexts written within one month
June 21, 2013CCSSO 2013 Presentation 12
Writing for assessments Context specifications
◦ Word counts◦ Readability programs and other resources◦ Content concerns
Bias and sensitivity training◦ Topics to avoid◦ Socioeconomic issues◦ Other sensitivity issues
Technology training◦ Web-based program
June 21, 2013CCSSO 2013 Presentation 13
Michigan Item Bank System◦ Context authoring ability◦ External access
Benefits Challenges
June 21, 2013CCSSO 2013 Presentation 14
Provides rich text for students Ownership of context Relationships built with the authors Cost-effective approach to passage
development Generation of topics for all Michigan
assessment programs
June 21, 2013CCSSO 2013 Presentation 15
Published authors and assessments Working relationships Expanding the pool
June 21, 2013CCSSO 2013 Presentation 16
—Michael P. Spradlin
June 21, 2013CCSSO 2013 Presentation 17
Educators/administrators are unaware of the authorial talent that resides in each state.
June 21, 2013CCSSO 2013 Presentation 18
June 21, 2013CCSSO 2013 Presentation 19
Who better to write about Michigan than authors from Michigan?
Authors work at honing their ability to engage readers. Writing for assessments seems to be a natural progression.
Economic standpoint—keep money in Michigan.
June 21, 2013CCSSO 2013 Presentation 20
Usually work within guidelines, restrictions based on age of readers, reading levels, etc.
Authors are used to being edited. Not all is new, but in some regards writing
for assessments can be a challenge. Working one-on-one with DRC and MDE staff
June 21, 2013CCSSO 2013 Presentation 21
On-site training Individualized training for each assessment
area (MEAP, ELPA, Access all have some similarities and differences.)
Ability to dedicate time and effort to the specific program
Receive immediate feedback Writing two to four pieces a week Assignments can be completed at home
June 21, 2013CCSSO 2013 Presentation 22
Digging into nonfiction topics Bringing forward unknown or unique stories
about Michigan and creating topics around them to engage readers
June 21, 2013CCSSO 2013 Presentation 23
—Shirley Neitzel
June 21, 2013CCSSO 2013 Presentation 24
Published writers connect with their readers.
June 21, 2013CCSSO 2013 Presentation 25
June 21, 2013CCSSO 2013 Presentation 26
Local authors are “repository for arcane information”
Authors want students to do well on assessment tests
Authors appreciate the working relationship with DRC and MDE
June 21, 2013CCSSO 2013 Presentation 27
Training sessions are collegial
Assignments are specific
Topics are mutually agreed to before writing begins
Interesting, challenging work
Flexibility
June 21, 2013CCSSO 2013 Presentation 28
On-site training
Prompt feedback
Off-site work
June 21, 2013CCSSO 2013 Presentation 29
Supplemental income
Satisfaction of making a contribution to the education of students
June 21, 2013CCSSO 2013 Presentation 30
—Patty McDivitt
June 21, 2013CCSSO 2013 Presentation 31
Benefits◦ Developers can control for such considerations
as text complexity, fairness, sensitivity, and freedom from bias, quality of writing
◦ Develops passages that will truly lend themselves well to items
Key drivers for success◦ Training◦ Item banking/authoring system◦ Flexibility for writers
June 21, 2013CCSSO 2013 Presentation 32
With careful planning and commitment, the goal of using the work of published authors can be achieved.
June 21, 2013CCSSO 2013 Presentation 33
June 21, 2013CCSSO 2013 Presentation 34