on common ground: 6-trait scoring for the kansas writing assessment
TRANSCRIPT
On Common Ground: 6-TRAIT Scoring for the Kansas Writing Assessment
Presentation Objectives1. Examine how the 6+1 TRAIT model was developed
for teachers by teachers.
2. Discuss the ways the 6-TRAIT model has been applied both nationally and here in Kansas.
3. Explore how the 6-TRAIT model is used within the context of the Kansas Writing Assessment.
4. Practice scoring for the Kansas Writing Assessment.
5. Discuss issues with scorer objectivity and bias.
Activity One
The Writing Sneeze:
Write continuously for three
minutes using the following
sentence starter as a
springboard:
My fears in scoring student essays for the Kansas Writing Assessment are…
Why Worry About Writing? Approximately 70% of students in grades 4-12 are low-
achieving writers. (Persky et al., 2003)
Thirty-five percent of high school graduates in college and 38% of high school graduates in the workforce feel their writing does not meet expectations for quality. (Achieve, Inc., 2005)
About half of private employers and more than 60% of state government employers say writing skills impact promotion decisions. (National Commission on Writing, 2005)
Writing remediation costs American businesses as much as $3.1 billion annually. (National Commission on Writing, 2004)
Among the Recommendations for Improving Student Writing
1. School districts should insist that writing be taught in all subjects and at all grade levels.
2. Every teacher should be required to successfully complete a course in writing theory and practice as a condition for teacher licensing.
3. Schools should aim to double the amount of time most students spend writing.
(National Commission on Writing, 2003)
Effective Practices to Improve Student Writing in Grades 4 to 12
1. Writing Strategies2. Summarization3. Collaborative Writing4. Specific Product Goals5. Word Processing6. Sentence-Combining7. Prewriting8. Inquiry Activities9. Process Writing Approach10. Study of Models11. Writing for Content
Learning
The Writing Next report highlights 11 instructional practices that research has identified as having shown strength and consistency in improving student writing quality.
Which of these practices can be facilitated and enhanced with the 6-TRAIT model?
(Graham and Perin, 2007)
Effective Practices to Improve Student Writing in Grades 4 to 12
1. Writing Strategies2. Summarization3. Collaborative Writing4. Specific Product Goals5. Word Processing6. Sentence-Combining7. Prewriting8. Inquiry Activities9. Process Writing Approach10. Study of Models11. Writing for Content
Learning
Using the 6-TRAIT
model of assessment
and instruction can
facilitate and
complement eight of
these eleven
instructional practices.
(Graham and Perin, 2007)
The 6+1 TRAIT ModelWhat it is…• a common
vocabulary for teachers and students
• a vision of the characteristics of strong writing
• an analytical scoring method
• a system for driving and managing student learning
What it is NOT…• a packaged
curriculum• a standards
document• a ‘canned’ program• a ‘convenience’ to
ease the grading load• a way to make
teaching ‘easier’
Development of the 6+1 TRAIT Modelfor teachers, by teachers
In 1984, drawing from the pioneering work of Paul
Diederich (1974) and Donald Murray (1982) and with
the help of researchers from the NWREL, a group of
17 teachers from the Beaverton, Oregon, school
district set out to create a scoring guide (a rubric) that
would describe what ‘good’ writing looks like…
Teachers spent weeks pouring over thousands of
examples of student writing, sorting them into high,
middle, and beginning levels, and documenting their
reasons for the rankings…
Next, they began sifting through their documented
reasons—combining and condensing—until they
arrived at six essential traits:
Development of the 6+1 TRAIT Modelfor teachers, by teachers
1. Ideas & Content – the heart of the message
2. Organization – the internal structure
3. Voice – the personal tone and flavor
4. Word Choice – the vocabulary a writer chooses
5. Sentence Fluency – the rhythm and flow
6. Conventions – the mechanical correctness
+1. Presentation – form, layout, eye-appeal
The 6+1 Traits of Writing
focused and clear controlling idea or
theme provides detail and
support that enriches writer is selective shows insight writes from knowledge
and experience
Ideas & Content
thread of central meaning
crafts a clear beginning, middle, and end
effective sequencing good pacing smooth transitions information is given in
the right doses at the right times
Organization
person behind the words
heart and soul appropriate for
audience conviction text is full of life personal, individual,
expressive
Voice
Word Choice rich, colorful, precise
language
strong and natural
vocabulary
energetic verbs
precise nouns and
modifiers
skill in using everyday
words well
Sentence Fluency rhythm and flow
easy to read a out loud
poetic, musical
variety of sentence
beginnings
variety of sentence structures
cadence, power, movement
mechanical correctness consistent control of
spelling, punctuation, grammar, indenting, capitalization, etc.
minimal typographical errors to distract a reader
appearance is irrelevant
Conventions
Presentation uniform spacing
legible handwriting or
appropriate use of fonts
and sizes
appealing use of white
space
effective use of bullets,
lists, graphs, maps, tables,
illustrations, sidebars,
sub-headings as necessary
Activity TwoWhat Teachers Value in Writing:
With a partner…
Read the selection of writing and generate a list of three to five characteristics that make this a ‘good’ piece of writing…
Now, for each of the characteristics you and your partner wrote down, identify under which of the traits that characteristic would fall.
1. Ideas & Content – the heart of the message
2. Organization – the internal structure
3. Voice – the personal tone and flavor
4. Word Choice – the vocabulary a writer chooses
5. Sentence Fluency – the rhythm and flow
6. Conventions – the mechanical correctness
+1. Presentation – form, layout, eye-appeal
The 6+1 Traits of Writing
Remember those 17 teachers in Oregon that spent weeks pouring over thousands of examples of student writing and documenting their reactions and comments?
Keep in mind they were also sorting that writing into three categories: high, middle, and beginning levels.
The result of their work was a draft of a rubric that would become the foundation for the 6+1 TRAIT model…
It looked a little something like this…
Development of the 6+1 TRAIT Modelfor teachers, by teachers
The 6+1 TRAIT Continuum
5 = Strong• shows skill and control; many
strengths present; writing is publishable
4 = Effective• strengths outweigh weaknesses; one
draft away from being publishable
3 = Developing• balance of strengths and weaknesses
2 = Emerging• weaknesses outweigh strengths
1 = Beginning• writer not yet showing any control
4
5
3
2
1
The 6+1 TRAIT Continuumpoints worth noting
• A ‘5’ represents a strong paper, not necessarily a perfect one.
• There may be a considerable difference in the “distance” between a ‘1’ and a ‘2’ and the distance between a ‘4’ and a ‘5.’
• The goal is for each student to improve with each subsequent draft.
4
5
3
2
1
Since 1984, both the rubric and the 6+1 TRAIT model have been revised nearly 20 times and been modified by a host of schools, districts, states, and even foreign countries.
In fact, Presentation (the seventh trait) wasn’t added to the model until 1997—based on the recommendations of classroom teachers—to help separate issues of correctness (conventions) from issues of appearance.
Development of the 6+1 TRAIT Modelfor teachers, by teachers
Flexibility and Adaptability
All this to prove a fairly simple point…
The 6-TRAIT Writing Model is just that—a model.
It is flexible and can be adapted to suit specific
assignments, specific purposes, and—most
importantly—the needs of particular students.
Flexibility and Adaptability Even a simple
Google search will reveal…• 4-point rubrics• 5-point rubrics• 6-point rubrics• Rubrics for teachers• Rubrics for students• Rubrics for assessment• Rubrics for instruction
The traits, however, remain constant:
1. Ideas and Content
2. Organization
3. Voice
4. Word Choice
5. Sentence Fluency
6. Conventions
7. Presentation
What We Value in Student Writing“We believe that any scoring model is by design a
value system. In that spirit, we hope to share this
model in a way that enables users to clarify and
define their own values about writing, and to learn a
method of sharing those values with students and
with other teachers, so that all of us who work with
the assessment of writing can understand writing
better and teach it more effectively.”
– from NWREL’s first publication of the 6-TRAIT model (1986)
Flexibility and Adaptability
Adapt the 6-Trait Model to suit your needs, the needs of your classroom, and the needs of your students.
Unlike this teacher…
The 6-TRAIT Model in Kansas
First used in 1989 in the KCK district
1992 — the first pilot of the Kansas Writing
Assessment which is built upon the 6-TRAIT model
(about one third of Kansas school districts
participate)
1994 to present — the Kansas Writing Assessment
continues biennially, requiring 6-TRAIT scoring at
both the local and state level
The 6-TRAIT Model in Kansas In 2003, 6-TRAIT rubrics were adapted to create the
Kansas Modified 6-TRAIT Rubric.
In 2007 work began on updating the 6-TRAIT rubrics
for the narrative, expository, and persuasive writing
on the Kansas Writing Assessment.
Changes to format and layout of the rubrics.
The six traits and the descriptor language,
however, remains largely the same.
Recent Changes to Rubricsfor the Kansas Writing Assessment
Changes to format:
descriptors organized under four criteria
within each trait
landscape orientation
“checkbox” bullets
Changes to content: descriptors added for the ‘2’ and ‘4’ levels point level labels of the rubric were changed to
better reflect the stages of a writing process
Rubrics are available for download on the KSDE Writing Homepage: <www.ksde.org/Default.aspx?tabid=1726>
New Point Level LabelsWhat was…
Strong
Maturing
Developing
Emerging
Beginning
Is now…
Publishing
Polishing
Drafting
Shaping
Inventing
4
5
3
2
1
The “Old” Rubric
The “New” Rubric
Two readers are always preferable to only one.
Scorers should be encouraged to discuss their ratings with other scorers.
Periodically, scorers should physically mark a copy of the rubric as they score to personally verify their ratings.
Suggested Scoring Procedures
Suggested Scoring Procedures are available on the KSDE Writing Homepage: <www.ksde.org/Default.aspx?tabid=1726>
Scoring an Individual Essay
For each trait… Determine which set of descriptors best
describe the essay for each of four criteria within a given trait.
Then, scorers should use the four criteria levels they identified to inform the rating of the overall trait.
Suggested Procedure for Scoring an Essay
Guidelines for Discussing Scores The goal is not to persuade or convince individuals
to change their scores. Keep the discussion firmly grounded in the language
of the scoring rubric. Discussion facilitates professional learning and
allows readers to better understand, clarify, and apply the criteria and descriptors contained within the scoring rubric.
The ultimate goal of is to ensure that student writing is being scored consistently and accurately.
Updated Training Manual is available on the KSDE Writing Homepage: <www.ksde.org/Default.aspx?tabid=1726>
Includes scored samples essays written by Kansas students that can be used for training/calibration purposes.
Activity Three
Practice Scoring
Using pre-scored sample essays from the
KSDE Training Manual for 6-TRAIT scoring,
let’s practice scoring student writing at the
5th grade, 8th grade, and high school levels.
How Did We Do?
Using the scoring
chart from the
training manual, let’s
check to see how we
did!
Issues with Scorer Objectivity & Bias Aspects/Characteristics of student writing NOT
considered when scoring for the Kansas Writing Assessment:
handwriting lightness or darkness of writing neatness formatting (skipping lines, unusual margins, font size/style) presence or absence of a title length of the piece of writing absence or use of technology
A number of factors can unknowingly influence a scorer’s perception of student writing in either a positive or negative direction.
An illustrative (not exhaustive) list of potential pitfalls to be mindful of…
Other Examples of Potential Scorer Bias
Personal Standards of Quality
Readers often have personal standards for what makes quality writing. Rationalizing scores with personal thoughts such as, Three misspelled words means a piece can only score a 3 or below in Conventions, or Only an essay that is completely error free can receive a top score, is a form of bias. When scoring for the Kansas Writing Assessment, the rubric must be followed.
Other Examples of Potential Scorer Bias
Other Examples of Potential Scorer Bias
Referring to the Prompt
Many writers write excellent papers but do not refer
directly in any way to the language of the prompt.
This is fine. Do not demand an obvious reference
(e.g., “My most memorable experience was the time
when…”) or demand that students include any of the
example ideas suggested in the prompt.
Other Examples of Potential Scorer Bias
Reactions to Content
Personal reactions to the specific content of the essay
may influence scoring. If the writer’s values are not
the same as the scorer’s or the writer’s choice of
overall theme or specific details are unappealing, this
may unduly affect scoring. If you cannot objectively
read such a paper, pass it on for scoring by someone
else.
Profanity
Occasionally a student will include profanity in his or
her writing. If you cannot objectively read a paper
with profanity, pass it on for scoring by someone else.
Other Examples of Potential Scorer Bias
Persona or Tone
Personal reactions to the persona or tone of the
writing can influence scoring. Scorers should take
into account any thoughts such as What a cute kid! or
How conceited! that may affect the accuracy of their
scoring.
Other Examples of Potential Scorer Bias
Style or Usage Prejudice
Some idiosyncratic preferences in style or usage (e.g.,
the use of a lot, get, or that is; the halo effect of a
well-turned phrase; the use of a particular cliché) can
create bias.
Other Examples of Potential Scorer Bias
Prior Experience with Student
Deeply ingrained personal classroom assessment
experiences with specific students can corrupt
accurate scoring. Thoughts such as, Pablo is the best
writer in the class, so his essay must receive straight
5s, or Samantha never does well with writing; I’m
sure her essay is horrible, can influence scoring.
Other Examples of Potential Scorer Bias
1. Refer often to the scoring rubric. Do not rely just on
your memory or your intuition.
2. Physically mark copies of the rubric while scoring
to ensure your judgments are being made based
upon the characteristics the rubric provides.
3. Remember to score each trait individually without
allowing the score from one trait to influence your
scoring of another trait.
Other Friendly Reminders for Scoring
4. Think of a 3 as the point on the scoring continuum
where strengths and weaknesses balance. Any score
above a 3 indicates dominant strengths; any score
below a 3 indicates dominant weaknesses.
5. Do not dwell on a particular essay’s weaknesses.
Focus your attention on identifying the set(s) of
descriptors that best describe the characteristics of
the essay.
Other Friendly Reminders for Scoring
6. Remember that you are assessing the writing—not
the writer—and only a single performance at that.
7. Keep in mind that the prompt is only meant to
motivate the writer and provide a springboard for
the student to begin generating ideas. Readers
should score the quality of the writing, not the
student’s adherence to the prompt.
Other Friendly Reminders for Scoring
8. If the essay is off-topic, score it according to the rubric to the best of your ability.
9. If you think a paper might be a crisis paper (e.g., suicide, child abuse, drug abuse), flag it and report it to your scoring session leader or testing coordinator but continue scoring.
10. If the papers are starting to blend together in your mind and all are starting to read the same, take a break.
Other Friendly Reminders for Scoring
11. When you need to share an essay or talk at length about
scoring, please do so with respect toward other scorers.
Even quiet talking can make it hard for those around you to
concentrate.
12. Your pace should be brisk but comfortable. Take time to
read each paper thoroughly; do not skim. Our priority is
scoring accuracy, not scoring speed. If you become stuck
with a particular essay, ask for help.
Other Friendly Reminders for Scoring
The 6+1 TRAIT Model in KansasKansas educators believe the 6+1 TRAIT model has:
reinforced the teaching of a writing process; helped many teachers who were unsure or
inexperienced with teaching writing to do so with confidence in their classrooms;
provided a common vocabulary to talk about writing with students and colleagues;
helped students understand what improvement a piece of writing needs;
The 6+1 TRAIT Model in KansasKansas educators believe the 6+1 TRAIT model has:
motivated students to revise and improve their writing; helped teachers to diagnose student needs; helped teachers to adjust and individualize instruction;
and provided a method for sharing information about
individual student writing ability with administrators, parents, and other non-specialists.
Other Available Resources
Glossary of Writing Terminology provides clarification on commonly
confused and/or misunderstood terms
in reference to their use on the Kansas
Writing Assessment
might be appropriate for both students
and educators
available on the KSDE Writing
Homepage: <www.ksde.org/Default.aspx?
tabid=1726>
Other Available ResourcesRubrics for Incorporating Research and Citing Sources
Grades 3-7 Grades 8-12
• Although not a part of the Kansas Writing Assessment, these
rubrics are formatted like those for the assessment and help
educators communicate our expectations for academic writing
and avoiding plagiarism.
• Available on the KSDE Writing Homepage:
<www.ksde.org/Default.aspx?tabid=1726>
Other Available Resources
• illustrative lists of the various types and genres of writing that might fit into each mode (narrative, technical, expository, persuasive)
• helps communicate that most writing often blends several modes or moves back and forth among modes
• available on the KSDE Writing Homepage: <www.ksde.org/Default.aspx?tabid=1726>
Illustration of Writing Modes Chart
Activity FourTeacher Reflection:
1. One piece of information presented today that I already knew was…
2. One piece of information presented today of which I was not aware was…
3. One idea from this presentation that I intend to use in my classroom is…
4. The most controversial idea I heard today was…
5. The idea I’d most like to hear more about is…
Communication from KSDE about Writing
• KSDE Writing Homepage—Standards, Assessment, and Resources <www.ksde.org/Default.aspx?tabid=1726>
• KSDE Writing ListServ
(email Matt to be added to this list)• Contact Matt Copeland directly
• Phone (785) 296-5060
• Email [email protected]
BibliographyAchieve, Inc. (2005). Rising to the Challenge: Are High School Graduates Prepared for College
and Work? Washington, DC: Author. Retrieved January 16, 2008, from http://www.achieve.org/node/548
Graham, S., & Perin, D. (2007). Writing Next: Effective Strategies to Improve Writing of Adolescents in Middle and High Schools—A Report to Carnegie Corporation of New York. Washington, DC: Alliance for Excellent Education. Retrieved January 16, 2008, from http://www.all4ed.org/publication_material/reports
National Commission on Writing (2003). The Neglected ‘R’: The Need for a Writing Revolution. Retrieved January 16, 2008, from http://www.writingcommission.org/report.html
National Commission on Writing. (2004). Writing: A Ticket to Work… or a Ticket Out: A Survey of Business Leaders. Retrieved January 16, 2008, from http://www.writingcommission.org/report.html
National Commission on Writing. (2005). Writing: A Powerful Message from State Government. Retrieved January 16, 2008, from http://www.writingcommission.org/report.html
Northwest Regional Education Laboratory. (1986). “The Six-Trait Analytic Model.” Portland, OR: Author.
Persky, H.R., Daane, M.C., & Jin, Y. (2003). The Nation’s Report Card: Writing 2002. (NCES 2003—529). U.S. Department of Education. Institute of Education Sciences. National Center for Education Statistics. Washington, DC: Government Printing Office.