www.nedp.org 1 the national external diploma program writing assessment in the new nedp linda...
TRANSCRIPT
www.nedp.org 1
The National External Diploma Program
Writing Assessment in the New NEDP
Linda Taylor, CASAS Director of Assessment Development
National External Diploma Program Council Conference
October 2012, Providence, RI
www.nedp.org 2
The Future of NEDP Writing
• CASAS’ review of NEDP Competencies and Tasks revealed a need for increased rigor.
• In terms of Writing, there is a need for:– More writing– More challenging writing
• In current NEDP development work, CASAS is incorporating more writing.
• The CASAS Written Prompt has been piloted for use in Diagnostics.
www.nedp.org 3
Writing Assessment in the New NEDP
Four general types of writing in NEDP Tasks:
– Extended Response• Includes the Written Prompt in Diagnostics• Each Competency Area will have at least one Extended
Response item
– Short Constructed Response
– Specific Applications – e.g., letters, resume, power point presentation
– Fill Matrix
www.nedp.org 4
CASAS Written Prompt
January 2013 - June 2014
CASAS will phase in the CASAS Written Prompt to replace the current NEDP writing process for Diagnostics
• January 2013 - June 2013– Agencies may voluntarily transition to the CASAS Written Prompt
• July 2013 – June 2014– Agencies to be trained will be assigned timeframes for training
• CASAS has developed a self-paced electronic training that will be available 24/7
• Additional implementation support will be scheduled periodically
www.nedp.org 5
NEDP Written Prompt Background
• The CASAS Functional Writing Assessment (FWA) Picture Task does not assess the highest levels of ability
• The CASAS National Consortium Writing Committee identified a need to develop a Written Prompt Task to assess higher level writing ability
• The Singapore Workforce Development Agency (WDA) funded and worked with CASAS to develop this new writing assessment from 2006 – 2008
• Tryouts were also conducted in CT in 2007 and from 2010 – 2012 to ensure appropriateness in the US
www.nedp.org 6
Background
• The Written Prompt Task is a new writing assessment in the CASAS Functional Writing Assessment series.
• The task is to write an extended response generated by a written essay topic.
• Takes 45 minutes to one hour• Holistically scored on a scale of 1 – 5• Diagnostic feedback is provided• Designed to be computer-delivered
www.nedp.org 7
Development
• Target population– Advanced adult ESL and ABE learners and workforce
participants– GED preparation, high school programs, NEDP– Transition to post-secondary
• Content of written prompts must be appropriate for both employed and unemployed test takers
• Developed 10 prompts; selected 5 for US field testing
www.nedp.org 8
Written Prompt Testing Process
– Test Directions embedded in the NEDP software
– New spell check feature tutorial is included
– Client enters writing into NEDP software
– Two trained and certified NEDP assessors score independently.
• Can be two assessors or one assessor and one advisor
– If they don’t agree, they discuss and come to consensus
– Client must score 3 (out of 5) to enter General Assessment
www.nedp.org 14
Written Prompt Rubric Development
• CASAS researched academic essay rubrics
• Wanted continuity with CASAS Functional Writing Assessment Picture Task rubric
• Created/revised numerous versions
• Issues: Voice/style, key elements, formatting of rubric for holistic scoring
• Scored both Singaporean and US responses
• Trained Singapore staff and part-time scorers
– good success rate
www.nedp.org 15
Providing Diagnostic Feedback
• Holistic scoring with targeted feedback on relevant aspects of the writing sample
• Drop-down menu items AND
• Open-ended comments
• Show example
www.nedp.org 16
Results from Phase 1 Pilot (2009 – May 2010)
Collected information about the writing ability levels of the NEDP population
Adjusted the Written Prompt rubric to distinguish more clearly between rubric score points
Revised the wording in the directions to the examinee (“Write about the essay topic” instead of “Write three paragraphs…”)
www.nedp.org 17
Results from Phase 1 Pilot
• Collected writing samples for scorer training materials
• Realized that computer delivery lends itself to including a “Spell-check” feature which would make the writing assessment more realistic
• Converted to CASAS eTests software delivery platform to conduct more tryouts (2010)
www.nedp.org 18
Written Prompt -- Current Status
• Created and tried out self-paced scorer training materials (2011 – 2012)– Currently converting into Moodle platform
• Now embedding the test directions and delivery into the NEDP online software
• CASAS will work with State Departments of Ed to determine training schedule and who will score
www.nedp.org 19
Written Prompt Research Studies
• Inter-rater Reliability studies– Two local program scorers score the same
responses
• Parallel forms– At least 100 test takers take a set of two
prompts on the same day or within one or two days
www.nedp.org 20
Writing in Generalized Assessment
• Extended writing is included in all NEDP Competency Areas (except Health)
• These items require writing based on analyzing or summarizing information from higher level reading
• Scoring Guidelines are provided for each item that requires writing.
• Items requiring writing are judged on content and ability to clearly communicate in writing. Errors in mechanics are only considered if they interfere with clarity of communication.
www.nedp.org 21
NEDP Writing Foundation Competency 3
Write to:
• present, explain and examine information and ideas• state a position, with clear and coherent supporting
arguments• describe experiences and events, real or created, and• express thoughts, feelings and emotions,• using writing techniques, organizational structures,
word choices, English language conventions, and styles of expression that are effective and appropriate to the purpose.
www.nedp.org 22
Writing Assessment in the New NEDP
Four general types of writing in NEDP Tasks:
– Extended Response• Includes the Written Prompt in Diagnostics• Each Competency Area will have at least one Extended
Response item
– Short Constructed Response
– Specific Applications – e.g., letters, resume, power point presentation
– Fill Matrix
www.nedp.org 23
Extended Written Response
Type of Writing Assessment
PI Item Title Description of Writing Required
Phase 14-5 paragraphs
41.3-4
The Legal System
Write an argument for and against Miranda Rights
Phase 23-4 paragraphs
31.1-2
Critique Live Performance
Apply vocabulary; evaluate quality of cast, costumes, music, special effects
www.nedp.org 24
Short Paragraph
Phase 23-4 sentences
43.2 National Geography
Explain how geographic feature has been affected by environmental factor
Phase 12 or more sentences
2.0 Read literary texts ̶ literature
Apply vocabulary; analyze central ideas; evaluate literary elements; make inferences
www.nedp.org 25
Extended Narrative: The Legal System
4. Required: Client must write four or five paragraphs that cover components a, b and c, below. Assess this response for content only; mistakes in mechanics are to be considered only if they interfere with clarity
www.nedp.org 26
Extended Narrative: The Legal System
• Guideline: Check each box that is demonstrated. If PTA is required, the response must be rewritten; however, boxes that have already been checked will not require re-demonstration.
a. Identifies and summarizes the main idea behind the argument for the reading of the Miranda Rights to an individual under arrest
www.nedp.org 27
Extended Narrative: The Legal System
b. Identifies and summarizes the main idea behind the justification for not reading the Miranda Rights to a suspected terrorist
c. Cites two resources researched at end of the response from which main ideas were drawn for arguments for and against reading the Miranda Rights, using citation guidelines in R4
D ND You provided an:
incorrect incomplete
Written response describing circumstances under which the Miranda Rights should or should not apply to an individual, citing two sources of information.
www.nedp.org 28
Short Paragraph: Science Issues in the News
3. Required: Two sentences interpreting data in line graph of temperature change:
one sentence summarizing temperature movement between 1900 and 2000, and
one sentence that makes prediction for next 100 years; prediction may mention general trend or more specific temperature increase
Guideline: Assess this response for content only; mistakes in mechanics
are to be considered only if they interfere with clarity. For sample responses, see following box. D ND You provided an:
incorrect incompletestatement on temperature changes displayed in graph
and prediction of future temperature change.
www.nedp.org 29
Sample Responses
3. Sample response, acceptable:Between the years of 1900 and 2000, average temperatures have increased two degrees due to human effects. Based on this data, we can predict that average temperatures will increase a similar amount in the next 100 years.
Sample response, not acceptable:This graph shows effect of natural forces and human effects. Temperatures are changing.
www.nedp.org 30
One or Two Sentences
One or two sentences
53.1 Elements of the Scientific Method
Provide evidence for hypothesis, dependent and independent variables; identify correct steps in experiment
www.nedp.org 31
Specific Applications: Resume
Resume
62.3 Creating a Resume
Design and write a resume for a job notice identified in earlier assessment
(6 content criteria;
5 formatting criteria)
www.nedp.org 32
Specific Applications: Formal Letter
49.1-2 Consumer Complaints
Write a formal complaint letter (4 types of information; 6 elements of a formal letter)
62.4 Employment Cover Letter
Write an email cover letter for job notice identified in earlier assessment (9 criteria including no errors in spelling, grammar and punctuation)
www.nedp.org 33
Formal Letter: Consumer Complaints
• 4a. Required: A complaint letter to Copper City Electronics with each of the four types of information below:
• 1) stating the problem• 2) asking for specific action• 3) including copies of all related documents, if
applicable • 4) providing contact information
www.nedp.org 34
Formal Letter: Consumer Complaints
4b. Required: All six elements of a formal letter, following guidelines for written statements and placement of each convention
• 1) Date• 2) Salutation• 3) Sender Address• 4) Receiver Address• 5) Closing• 6) Signature
Guideline: Check each box that is demonstrated. If PTA is required, the letter must be rewritten; however, boxes that have already been checked will not require re-demonstration
www.nedp.org 35
Formal Letter: Consumer Complaints
Click Sample Letter Response and Letter Formats to guide evaluation. Any of the six displayed formats is acceptable.
D ND You provided a
incorrect incomplete
formal complaint letter containing four types of information and six formal elements.
www.nedp.org 36
Topics for Discussion
• Will writing mechanics (e.g., spelling, punctuation) be assessed whenever writing is required in a task?– No, in short written responses, the focus will
be on content.
• Other Topics?
www.nedp.org 37
NEDP Contact Information
• Jim HarrisonDirector, National External Diploma [email protected], ext. 191
• Melissa Dayton Linda TaylorNEDP Coordinator [email protected] Director of Assessment Development800-255-1036, ext. 181 [email protected]
800-255-1036, ext. 186• Christine O’Hara
NEDP [email protected], ext. 135
37