the effects of being a reader and of observing readers on fifth-grade students’ argumentative...
Post on 25-Aug-2016
215 Views
Preview:
TRANSCRIPT
The effects of being a reader and of observing readerson fifth-grade students’ argumentative writingand revising
Noreen S. Moore • Charles A. MacArthur
Published online: 11 June 2011
� Springer Science+Business Media B.V. 2011
Abstract Opportunities to read and analyze others’ writing or to observe readers
as they analyze writing might enhance one’s own sense of audience and improve
one’s own writing. This mixed-methods study investigated whether reader and
observer activities in comparison to writing practice activities affected fifth-grade
students’ persuasive writing and revising. After writing a first draft of a persuasive
letter, 87 fifth-grade students were randomly assigned to one of three conditions:
being a reader, observing readers, or practicing writing. The reader group read and
discussed three persuasive letters, considering whether they were persuasive and
why and selecting the most persuasive. The observer group listened to the reader
group’s discussions and took notes; then they had their own discussion to generate a
list of criteria for what made the letters persuasive. The practice-writing control
group practiced writing persuasive letters. Afterwards, all groups revised their first
drafts. The reader group produced second drafts that were of better quality and
contained more evidence of audience awareness than the control group. The
observer group did not differ from either group. The groups did not differ on a
transfer task occurring 1-week later. The authors discuss implications for designing
writing curriculums that utilize reader and observer activities.
Keywords Adolescents � Observational learning � Persuasive writing �Reading-writing � Revising
N. S. Moore (&)
Department of Special Education, Language and Literacy, The College of New Jersey,
Ewing, NJ 08628, USA
e-mail: nmoore@tcnj.edu
C. A. MacArthur
School of Education, University of Delaware, Newark, DE, USA
e-mail: macarthu@udel.edu
123
Read Writ (2012) 25:1449–1478
DOI 10.1007/s11145-011-9327-6
Introduction
Cognitive and social perspectives on writing emphasize the importance of audience
awareness (Flower & Hayes, 1980, 1981; Hayes, 1996; Nystrand, 2006; Prior,
2006). Proficient writers attend to audience while composing and understand that
writing is an interactive meaning-making process among readers and writers.
Argumentative writing, in particular, demands audience awareness because the
writer must consider the potential alternative perspectives of an audience to achieve
the compositional goal of convincing an audience to adopt a certain position on an
issue. Instructional activities in which writers take the role of readers or observe
readers may help them develop audience awareness.
Review of literature
The complexities of argumentative writing
In a report from the Carnegie Corporation of New York entitled, Writing Next:Effective strategies to improve writing of adolescents in middle and high schools,
the authors assert ‘‘writing well is not just an option for young people—it is a
necessity’’ (Graham & Perin, 2007, p. 3). The ability to write well is important for
school success; it is a tool for thinking and learning (Shanahan, 2004; Sperling &
Freedman, 2001). The ability to write persuasively is necessary because it is
practiced in many school subject areas, including science and social studies
(Chambliss, 2001; MacArthur, Ferretti, & Okolo 2002). Furthermore, persuasive
writing skills are required for success in higher education (National Commission on
Writing, 2003, 2004), the workplace (National Commission on Writing, 2004,
2005), and for participation in a democratic society and a global economy
(Crowhurst, 1990).
However, national assessments indicate that elementary, middle, and high
school students have considerable difficulty writing argumentative texts (Persky,
Daane, & Jin, 2003). In particular, research has shown that school-age writers
often fail to consider audience in their persuasive writing, as evidenced by their
lack of inclusion of counterarguments or rebuttals (Felton & Kuhn, 2001;
Knudson, 1994; Pontecorvo & Girardet, 1993). Consideration of audience when
writing persuasively is a complex task that demands perspective taking, or the
ability to decenter one’s self, in order to adopt another’s point of view or way of
thinking even though it may differ from one’s own beliefs (Clark & Delia, 1977;
Golder & Coirier, 1996). Holding alternative perspectives in mind in order to
construct arguments to support a proposition and counterarguments against
opposing propositions adds to the cognitive complexities of the writing process.
Argumentative writing poses particular social (e.g., consideration of audience
view points) and cognitive challenges (e.g., orchestrating declarative, procedural,
and conditional knowledge) for writers (Coirier, Andriessen, & Chanquoy, 1999;
Oostdam, 2004).
1450 N. S. Moore, C. A. MacArthur
123
Despite the difficulties, intervention studies show that with appropriate support
elementary and middle school students are able to include elements of audience
awareness in their persuasive writing. For example, Auriac-Peyronnet (2001) found
that 10- and 11-year old students who received oral training produced longer texts,
more argumentative and fewer non-argumentative utterances, and less use of the
personal ‘‘I’’ and more use of ‘‘you’’, which the author interpreted as a distancing
technique crucial to persuasive discourse. In another study, Midgette, Haria, and
MacArthur (2005) found that when fifth- and eighth-grade students were given
specific content and audience awareness goals for revising a persuasive essay, they
produced better quality essays than students given only content goals or only a
general goal to improve their essays. These studies suggest that with instructional
support, it is possible for students as young as fifth grade to demonstrate audience
awareness in ways that improve the quality of their writing.
Developing audience awareness
Peer review
One common way to develop audience awareness is through frequent response and
review from teachers and peers, which is a key aspect of process approaches to
writing instruction (Pritchard & Honeycutt, 2006). In particular, peer review
provides writers an opportunity to find out how well their writing communicates to
readers, as well as an opportunity to learn by reading and evaluating the writing of
their peers. Thus, they may learn both from receiving feedback from readers and
from giving feedback (MacArthur, in press).
A recent meta-analysis of research on writing instruction for adolescents
(Graham & Perin, 2007) reported strong effects (d = 0.75) for peer collaboration.
Peer collaboration was defined to include assistance at any stage of the process, but
all of the studies reviewed included peer review for revision. For example, Boscolo
and Ascorti (2004) studied peer review in grades 4, 6, and 8 in Italian schools. The
readers were instructed to read their partner’s text aloud, stop at a point where
something was unclear, and provide a suggestion for making this part of the text
clearer. Compared to children who only received teacher feedback, children who
received peer feedback had fewer information gaps and problems with clarity in
their final drafts.
Typical peer review involves both giving and receiving feedback. Two recent
studies isolated the effects of giving feedback, that is, the effects of reading and
evaluating others’ writing. Lundstrom and Baker (2009) randomly assigned
students in college L2 writing classes to either give feedback to peers or to
receive it for a full semester. The students who gave feedback made greater gains
in writing. Cho and MacArthur (2011) assigned undergraduate college students to
reviewer, reader, or no-treatment control conditions. The reviewers rated and
provided comments on three different peer papers of different qualities. The
readers read the same papers without evaluating or commenting on them.
Afterwards, all students wrote papers in the same genre, but on a different topic.
The reviewers wrote significantly better quality papers than the readers and the
The effects of being readers and observing readers 1451
123
no-treatment control group. These studies illustrate that reading and reviewing
writing can impact one’s writing quality.
Reading to act
Other research has investigated the effects on writing of another type of reading
activity—reading to comprehend and perform some specific action (Holliway &
McCutchen, 2004; Traxler & Gernsbacher, 1992, 1993). Holliway and McCutchen
(2004) had fifth-grade students write descriptions of tangrams (abstract figures
composed of geometric shapes). The students were then assigned to three groups:
(a) feedback only, (b) feedback and rate, (c) feedback and read-as-a-reader. All
students received simple feedback indicating whether a reader could successfully
match their description to the correct tangram picture. In the feedback and rate
condition, students also read and rated three descriptions of tangrams by other
students for general informational adequacy. Finally, in the feedback and read-as-a-
reader condition, writers received the same three descriptions of tangrams written
by other students, but instead of rating them, they had to match the descriptions to
the correct tangram picture. Students then revised their original descriptive writing.
Analyses indicated that the read-as-a-reader group made the most significant gains
in writing quality as measured by the number of correct description-to-tangram
matches by multiple readers.
In this study (Holliway & McCutchen, 2004), reading to perform an action was
more effective than reading to evaluate in helping students to improve their own
writing. However, the study used an atypical writing task with a highly concrete
goal, so it is not clear that the results would apply to more typical academic writing
tasks for which more general evaluation criteria are relevant. In addition, the
features of the writing and revisions made were not studied; nor were writers
thought processes as they made the revisions.
Observing readers
Another way students can learn about audience is through observing readers explain
their thinking processes or try to carry out a task based on reading. Observational
learning gives learners the opportunity to reflect upon and gain metacognition about
a task or skill they are learning, which ultimately helps them successfully carry out
this new task or skill (Sonnenschein & Whitehurst, 1984). Observational learning
through modeling consists of four processes: attention, retention, production, and
motivation (Bandura, 1986). In learning through observation, a learner focuses
attention on a model, retains information given by the model, translates the
information into a behavior, and is ultimately motivated to enact the behavior or not
depending on the success of the model. Observation may be more effective than
practice writing in some circumstances because it provides information on the task
and the opportunity for reflection while freeing the learner of the cognitive burdens
of doing the writing task (Rijlaarsdam & Couzijn, 2000; Rijlaarsdam & Van den
Bergh, 2006).
1452 N. S. Moore, C. A. MacArthur
123
Several studies have examined the extent to which a combination of reading
activities (i.e., being a reader, reading a reader’s think aloud transcript, and listening
to a reader think aloud as he reads a text) affects student writing quality (Crasnich &
Lumbelli, 2005, Lumbelli, Paoletti, & Frausin, 1999; Schriver, 1992). These studies
suggest that observation of readers in combination with reading leads middle school
and junior high school writers to improve the clarity of their texts so that readers can
better access them.
Other studies have isolated the effects of observational learning (Couzijn, 1999;
Couzijn & Rijlaarsdam, 2004b). In general, the results show that observation of
readers and/or writers is a more effective instructional method than practicing
writing for improving one’s overall writing quality. In one study (Couzijn &
Rijlaarsdam, 2004a), high school students wrote instructions for a science
experiment, which they revised after participating in one of several experimental
conditions. After writing a first draft of the instructions, some students observed
readers thinking aloud while following their instructions or another students’
instructions; prior to writing instructions for the experiment, other students read the
instructions themselves while thinking aloud; and others self-evaluated their own
writing. Students who observed readers attempt to follow the instructions made
greater gains in the quality of their written directions, and it did not matter whether
the text being read was the writer’s own or another student’s. They also did a better
job providing written advice to other students about how to write science
instructions. This study suggests that observing readers think aloud as they try to
comprehend texts is an effective way for writers to reflect on audience needs and
learn about the characteristics of texts that pose difficulties for readers. However,
like the tangram studies (Holliway & McCutchen, 2004; Traxler & Gernsbacher,
1992, 1994), the writing task asked for concrete responses from readers.
A recent study (Rijlaarsdam & Braaksma, as cited in Rijlaarsdam et al., 2008)
investigated the effects both of experience as a reader and of observing readers using a
more common persuasive writing task. In this study, junior high students wrote
persuasive letters to the Yummy Yummy Candy Company about a special promotion.
A group of students simulating a review board at the company (readers) read several
letters and discussed which one was the most persuasive. Another group simulating
researchers (observers) took notes, discussed the evaluation criteria used by the board,
and presented their findings to the class. Then all students revised their own letters. The
board discussions and the poster presentations clearly demonstrated students’
audience awareness knowledge. Moreover, the results indicated that all letters
showed improvement in overall quality; however, the researchers (observers)
improved more than the review board (readers). It is important to note that the
readers did not just read and the observers did not just observe. The readers were
charged with making a decision based on reasons, and the observers were charged with
analyzing those reasons and presenting them to the class.
The current study
In this study, groups of readers read, discussed, and ranked three persuasive letters
according to overall persuasiveness. The observer groups listened to the reader
The effects of being readers and observing readers 1453
123
groups’ discussions and took notes; then they had their own discussion to generate a
list of criteria for what made the letters persuasive. The practice-writing control
group practiced writing persuasive letters. Then all groups revised a letter that they
had written previously on the same topic.
The current study builds on previous research on peer review, being a reader, and
observing readers. Like research on the effects of giving feedback, this study asked
students to analyze and comment on the writing of other students. The study also
separated the effects of being a reader from being an observer and compared both to
a practice writing control condition. The study extends prior research in several
ways. First, the procedures used in this study could be applied to any writing task.
They are not limited to tasks with concrete goals, like the tangram and science
instruction studies, or to a particular story line like the Yummy Yummy study. In
addition, this study not only investigated the immediate effects on revision of the
papers written before the intervention, but also checked for transfer to a similar
writing task the following week. Although Couzijn and Rijlaarsdam’s (2004a, b)
study on instructions for a science experiment included a far transfer task (a letter of
advice about the task), it did not include a transfer to a similar task. In addition, the
study not only documented the overall quality of the writing under various
conditions but also examined the types of revisions made following the intervention.
Furthermore, qualitative data were collected to describe the students’ thinking
during revision and to capture the nature of interactions in the group discussions.
Finally, the study investigated students’ perceptions about the various activities.
The current study used mixed-methods to investigate the extent to which being a
reader and observing readers affected the revisions made by fifth-grade students on
argumentative letters and the overall quality of their writing. We hypothesized that
readers and observers, in comparison to controls who practiced writing, would make
more revisions, particularly revisions to argumentative elements that demand
audience awareness (e.g., counterarguments), and increase the overall quality of
students’ writing. We also examined transfer to a similar writing task a week later
and gathered qualitative data to describe students’ reasons for revision, group
interactions, and perceptions about the activities.
Method
Participants
Eighty-seven fifth-grade students (47 female and 40 male) from five classrooms in
two elementary schools located in the same suburban school district in the
northeastern part of the United States participated in this study. The primary
approach to writing instruction in both schools was writers’ workshop (Calkins,
1986); however, one school’s instruction was also influenced by the six-traits of
writing (Spandel, 2005). Persuasive writing was part of the fifth-grade English
language arts curriculum in both schools. However, none of the students had
experienced persuasive writing instruction in fifth grade prior to the study.
1454 N. S. Moore, C. A. MacArthur
123
The average age of the participants was 10.7 years (see Table 1). The ethnic
backgrounds were as follows: 80.5% Caucasian, 12.6% African-American, 3.4%
Hispanic, and 3.4% Asian-American/Pacific Islander. Socioeconomic status (SES)
data was available at the school and district levels: 42.7 % of fifth-grade students
were identified as low income at the first school and 2.9% at the second school; at
the district level, 46.9% of all fifth graders were identified as low income. Students
with disabilities were excluded from the study. The results of the 2008 statewide
annual testing program were used to determine students’ literacy achievement levels
in reading and writing. The scores on the reading and writing subtests range from 0
to 5; 2.5 represents a passing score on the state standards (Delaware Student Testing
Program, 2008). The average scores were as follows: reading, M = 3.39,
SD = 1.00; writing, M = 2.81; SD = 0.61; scores for three students were not
available.
Experimental design and procedures
The participants were randomly assigned individually within classroom to the
reader, observer, and control groups. Prior to the random assignment, classroom
teachers identified 3 or 6 strong leaders (depending on the class size) in the class
to act as discussion leaders for the groups. These leaders were randomly assigned
to the reader, observer, and control conditions. Next, the remaining students were
randomly assigned so that there was at least one group each of four readers, four
observers, and four control students. Three of the classes were large enough that
two groups of readers, observers, and control students were selected. Twelve
students were randomly selected, one per condition per classroom except for the
smallest class, to participate in think alouds and semi-structured follow-up
interviews.
Table 1 Descriptive data on participants
Variable Group
Reader Observer Control
Gender
Female (N = 47) 16 (51.6%) 19 (67.9%) 12 (42.9%)
Male (N = 40) 15 (48.4%) 9 (32.1%) 16 (57.1%)
Ethnicity
Caucasian (N = 70) 27 (87.1%) 21 (75%) 22 (78.6%)
American Indian/Alaskan Native,
Asian American/Pacific Islander,
Hispanic, Black (N = 17)
4 (12.9%) 7 (25%) 6 (21.4%)
Age M (SD) N = 87 10.6 (0.4) 10.7 (0.65) 10.8 (0.34)
DSTP reading M (SD) N = 84 3.52 (1.06) 3.37 (1.08) 3.29 (0.9)
DSTP writing M (SD) N = 84 2.93 (0.46) 2.78 (0.7) 2.71 (0.66)
The reading and writing test scores were taken from the Delaware State Testing Program (2008)
The effects of being readers and observing readers 1455
123
Originally, 32 students were assigned to each condition. Nine students were lost
to the study due to absenteeism for one or more sessions. Complete data were
available for 28 participants in the control group, 28 in the observer group, and 31 in
the reader group. No group of four students lost more than one member and no
student selected for participation in the think-alouds and semi-structured interviews
was lost. Students in the three conditions were compared on reading and writing
scores, age, gender, and demographic data with a p value set at .20; no statistically
significant differences were found.
The research study was carried out over six 45-minute sessions over 2 weeks.
The first four sessions were completed in 1 week. During session I, all students
planned and wrote a persuasive letter in response to prompt A. During session II,
participants in the reader and observer groups met in discussion groups of four, with
an observer group observing each reader group. The reader and observer groups
practiced the experimental activities using narrative texts. Participants in the control
group practiced writing to a narrative prompt. The decision to use narrative texts
during this session was because the primary purpose of this session was to teach
students about the procedures of the study activities (i.e., how to be a reader,
observer, control group participant) rather than to teach students about persuasive
writing. During session III, participants in the reader and observer groups engaged
in the experimental activities using persuasive texts. Participants in the control
group practiced writing to a persuasive prompt. During session IV, all participants
revised the first draft they wrote to prompt A. At this time, think alouds and
retrospective semi-structured interviews were collected from four students in each
condition. During session V, all participants planned and wrote a persuasive letter in
response to Prompt B, which they revised in session VI. Semi-structured interviews
were collected from the same sample of students who participated in the think
alouds and interviews during session IV. The writing and revising activities in
sessions I and IV-VI and the practice writing by control students were conducted in
students’ classrooms. The discussion groups in sessions II and III and the think-
aloud interviews were held in other spaces in the school.
The investigator, assisted by three trained research assistants, conducted the
study. The lead research assistant helped the investigator conduct the experimental
activities for the reader and observer groups and conducted think alouds with
selected students. The remaining two research assistants conducted think alouds
with selected students. All research assistants attended one 45-minute training
session on how to conduct think alouds. The lead research assistant attended an
additional 60-minute training session that gave an overview of the reader and
observer group procedures; she also observed the investigator facilitate the reader
and observer activities during the first week of the study and prior to doing so
independently. All procedures for think alouds and experimental activity facilitation
were scripted.
Session II: Practice experimental activity
The primary purpose of session II was to familiarize the participants in the reader
and observer groups with their roles and the activity. The investigator or lead
1456 N. S. Moore, C. A. MacArthur
123
research assistant introduced the experimental activities, and the reader and
observer groups practiced with narrative texts. The reader groups responded to two
or three narratives written by students similar in age. A discussion guide consisting
of five questions focused the discussion. The observer groups observed the readers’
discussion and used a graphic organizer to take notes about what they heard the
readers say about each narrative. Following the readers’ discussion, the observers
had their own discussion in which they devised a list of criteria about what
characterizes an effective narrative. Throughout the activity, the investigator and
lead research assistant facilitated and monitored discussions to make sure
participants understood their roles in the activity. The control group practiced
writing to a narrative prompt during this session; the investigator-created narrative
writing prompt asked students to write about their favorite season.
Session III: Participation in the experimental activity
During session III, the reader and observer groups participated in the experimental
activities with persuasive texts. The reader group read persuasive letters written by a
group of fifth-grade students in response to prompt A; the students whose letters
were discussed were not participants in the study (see ‘‘Materials’’).
The reader groups were directed to discuss whether each letter was persuasive
and why; rate them as persuasive, somewhat persuasive, or not persuasive; and
choose the most persuasive letter. Each letter was read aloud while students
followed on their own copies. Readers used a discussion guide that contained five
guiding questions to guide their discussion and all discussions were audio-recorded.
The following questions were included on the guide in order to facilitate and focus
discussion; however, students were not limited to discussing these questions:
• Do you think this letter will persuade the teacher? Why or why not?
• Does the writer state his opinion in a strong and clear way? Does this make the
letter better or worse?
• Did the writer give good reasons for his or her opinion? Why are the reasons
good or not so good?
• Did the writer think about reasons why someone may disagree with him or her?
Does this make the letter better or worse?
• Did the writer write to the teacher with an appropriate attitude? Why or why
not?
The observer groups were instructed to listen to the reader groups’ discussions
and figure out what things the readers thought were good or bad about the letters and
what made them persuasive. To this end, the observers listened to the discussions
and took notes. Following their observation, they met to discuss what they observed
and to create a list of the characteristics the readers identified that make writing
more persuasive. To take notes, they used a graphic organizer with space for notes
on each letter and a numbered list of ways to make writing more persuasive. All
observer discussions were audio recorded.
Participants in the control group practiced writing to another persuasive prompt
on a different topic for the duration of the treatment group activities. The persuasive
The effects of being readers and observing readers 1457
123
writing prompt was taken from the state testing program: ‘‘Your school cafeteria
manager is thinking about taking ice cream and other sweets off the menu. Write a
letter to the cafeteria manager telling why you agree or disagree with this idea.’’
Materials
Writing prompts
Prompt A and B were selected to require little specific background knowledge and
to represent similar adult audiences (i.e., a teacher and a principal). Both prompts
were adapted from the state testing program.
Prompt A was, ‘‘Your teacher is thinking about giving students more homework
to get them ready for the next grade. Do you think this is a good idea? Write a letter
persuading your teacher to agree with what you think about giving students more
homework.’’
Prompt B was, ‘‘In order for students to have more time in the library, your
principal is thinking of having students attend gym class only once a month. Is this a
good idea? Write a letter convincing your principal to agree with your opinion about
students having gym class only once a month to have more library time.’’
Letters for discussion
The reader group read persuasive letters written by a group of similar fifth-grade
students from a pilot study. The investigator selected and made minor modifications
to the texts to represent a range of writing abilities, argument elements, and tones.
The texts also represented different points of view. The same set of three persuasive
letters was used in all classes.
Measures, scoring procedures, and inter-rater reliability
First and second drafts of the prompt A and B essays were typed double-spaced with
names replaced with codes. A trained rater unfamiliar with the purpose of the study
checked 20% (n = 70) of all of the writing for accuracy against participants’
original handwritten writing. The only errors were five typos that affected spelling
but not word meaning. The essays were analyzed for: (a) overall persuasive quality,
(b) revisions, (c) and inclusion of argumentative elements.
Overall persuasiveness
Overall persuasiveness of the first and second drafts of the Prompt A and B essays
was measured using a 7-point primary trait scoring rubric (Ferretti, MacArthur, &
Dowdy, 2000). The rubric defines persuasive writing as writing that ‘‘attempts to
influence readers to change their thinking or behavior’’ (Ferretti et al., 2000, p. 702).
Two trained raters unfamiliar with the purpose of the study scored all of the papers
independently. Training included discussion of the rubric and two anchor papers for
each score point and practice scoring of sample papers from a pilot study. Training
1458 N. S. Moore, C. A. MacArthur
123
continued to a criterion of 70% exact agreement on a sample of ten papers. Inter-
rater reliability was calculated both as Pearson product moment correlation and
percent agreement within one point. The correlation between raters was r = .70.
Percent agreement within one point was 94%. The scores of the two raters were
averaged and the mean scores were used in the analysis.
Revisions
First and second drafts of essays written to Prompt A were compared and revisions
were coded in the following categories: (a) type: word, phrase, sentence;
(b) operation: add, delete, substitute; (c) purpose: give new information, clarify,
elaborate, organize, emphasize, delete, change tone, no purpose; (d) argument
element: introduction, proposition, reason for proposition, elaboration of proposi-
tion or reason, alternative proposition, reason for alternative proposition, rebuttal,
conclusions; and (e) quality: successful or no change (MacArthur, Schwartz, &
Graham, 1991; Moore & MacArthur, 2008). Revisions were identified using the
compare documents feature in Microsoft Word. Raters had to decide whether a
revision was non-surface or surface and then continue coding all non-surface
revisions for the remaining categories. Three raters unfamiliar with the purpose of
the study attended a 2-h training session that included discussion of codes and
practice scoring papers from the pilot study. Training continued until raters reached
at least 70% agreement in each category when papers were scored independently.
After the training, each rater scored one-third of the sample independently. In
addition, 20% of each rater’s scored papers were also scored by the investigator to
calculate interrater reliability. However, several of the codes for purpose and
argument element were used too infrequently to permit meaningful calculation
of interrater reliability. Low frequencies would also limit analysis. Consequently,
the codes for purpose were collapsed into two codes: information (give new
information, clarify, and elaborate) and other purpose (organize, emphasize, delete,
change tone, no purpose). The codes for argument elements were collapsed into two
codes: proposition elements (proposition, reason for proposition, and elaboration of
proposition or reason) and alternative proposition elements (alternative proposition,
reason(s) for alternative proposition, and rebuttal). Interrater reliabilities (agreement
on occurrences/ [agreements ? disagreements]) were 91% for purpose and 86% for
argument element.
Argument elements
The presence of argument elements in second drafts of both prompts A and B was
measured using a coding scheme developed in previous research (Ferretti et al.,
2000). Papers were rated for the following elements: proposition about the topic,
reason(s) for the proposition, elaboration of the proposition or reason(s), alternative
proposition about the topic, reasons for the alternative proposition, elaborations of
the alternative proposition or reason(s), rebuttals of the reason(s) for the alternative
proposition, conclusions, and non-functional.
The effects of being readers and observing readers 1459
123
Three trained raters unfamiliar with the purpose of the study independently coded
the papers after attending three training sessions. During training, the raters
reviewed the codebook and examples and practiced scoring pilot-study papers until
they reached at least 70% exact agreement. After training, each rater independently
scored one-third of the total number of papers. The investigator initially scored 20%
of each rater’s batch of papers. However, the low frequency of units coded for
alternative propositions, reasons for alternative propositions, and rebuttals prevented
reasonable calculation of interrater reliability. It also limited analysis because of the
high frequency of zeros. Therefore, codes were collapsed to form three categories
for analysis: proposition elements (including propositions, reasons for propositions,
and elaboration of propositions), alternative proposition elements (including
alternative propositions, elaborations of alternative propositions, and rebuttals),
and conclusions. All drafts were re-scored by two trained raters for alternative
proposition elements. The interrater reliabilities, calculated as percents exact
agreement were: proposition elements, 88%; alternative proposition elements, 71%;
conclusion, 87%.
Think alouds and semi-structured interview data
All think aloud and semi-structured interview data were transcribed using a format
that designated the difference between writing, thinking, and composing out loud
(Prior, 2004). Triangulation of the several data sources occurred to ensure validity
of the data analysis: the investigator and a trained rater read and coded the think
aloud transcripts, semi-structured interviews, and first and second drafts of writing
in order to confirm observations and draw conclusions.
All data were analyzed using constant comparative analysis (Rossman & Rallis,
2003). The unit of analysis was an idea unit, which consisted of a proposition and
everything related to it. The analysis occurred over several steps. First, the
investigator and a trained rater read through all of the transcripts and papers
associated with the transcripts and noted themes related to the research questions.
The investigator and trained rater discussed their observations as they did the initial
read through of the transcripts. Secondly, the investigator and trained rater began
coding the transcripts for the repeating ideas they noticed. When a code appeared
with great frequency, they coded all remaining transcripts for that code only. They
repeated steps two and three until saturation was reached. Third, another rater was
trained on the definitions of the final codes and given examples of each from the
transcripts. She coded 25% of the think alouds and semi-structured interviews for
prompt A (n = 3) and 25% of the semi-structured interviews for prompt B (n = 3).
The percent exact agreement with the codes was 87% for think alouds and
interviews associated with prompt A and 85% for interviews associated with prompt
B. Finally, the investigator grouped the codes into categories and then into themes
to describe the discourse processes, strategies, thoughts, and understanding of
students in each of the three groups in the study. Groups were compared and
contrasted.
1460 N. S. Moore, C. A. MacArthur
123
Discussions
Reader and observer discussions during the experimental activities were audio-
recorded, transcribed, and analyzed for topics discussed and type of discussion
move. The same procedures for constant comparative analysis used to analyze the
think-aloud data were followed for the discussion analysis.
Results
Overall persuasiveness on prompt A
The quality of the final draft of prompt A was compared across groups using
analysis of covariance with quality of the first draft as a covariate (see Table 2 for
the quality data and the ‘‘Appendix’’ for correlations with other variables). A
significant difference by group was found (F = 5.70, df[2,86], p \ .01). Pairwise
comparisons, using the Bonferroni adjustment, indicated that the reader group had
greater overall quality scores than the control group (effect size, d = .625,
calculated as difference in mean posttest score divided by the pooled standard
deviation). The observer group did not differ significantly from either the reader
group or the control group.
Revision type on prompt A
The three groups were compared on four revision variables including two purposes
(information and other) and two types of argument elements (proposition and
alterative proposition; see Table 3 and the ‘‘Appendix’’). A violation of homoge-
neity of variance prevented the use of multivariate analysis of variance (MANO-
VA). Therefore, separate analyses were conducted for the four revision variables.
The Bonferroni procedure for four simultaneous tests was applied; alpha was set at
.012 (i.e., .05/4).
Purpose for revision
Separate one-way ANOVAs were conducted for the two purpose codes of
information and other purpose. For information, the main ANOVA revealed no
significant differences among groups (F = 1.34, df[2,84], p = .269).
Table 2 Overall persuasive quality ratings for prompt A
Group First draft Second draft Adjusted
M (SD) M (SD) M (SE)
Reader 3.26 (1.0) 3.89 (1.50) 3.74 (0.12)a
Observer 2.96 (0.68) 3.20 (0.60) 3.32 (0.13)a,b
Control 3.05 (0.94) 3.11 (0.89) 3.15 (0.13)b
Adjusted means that do not share subscripts differ significantly at p \ .05
The effects of being readers and observing readers 1461
123
For other purpose, the homogeneity assumption underlying ANOVA was not met
and scores were significantly skewed due to a high frequency of zeros. Therefore, as
recommended by Field (2005), the Kruskal-Wallis H test, the nonparametric
equivalent of the one-way ANOVA, was used to assess group differences. There
was no significant difference among groups on revising for other purpose (p = .738).
Revisions to argument elements
Separate analyses were conducted for the two argument elements of proposition and
alternative proposition. For proposition elements, the ANOVA revealed no
significant differences among groups (F = .404, df[2,84], p = .669).
For alternative proposition elements, the homogeneity assumption was not met
and scores were significantly skewed due to a high frequency of zeros, so the non-
parametric Kruskal-Wallis H test was used. There was a significant difference
among groups on revision to alternative propositions (H(2) = 8.52, p \ .01). As
recommended by Field (2005), Mann-Whitney tests were used for follow-up
comparisons between groups; a Bonferroni correction was applied. The reader
group made significantly more revisions affecting alternative propositions than the
control group (p \ .01; effect size, d = .57). The observer group did not differ
significantly from either the reader or the control group.
Argument elements on prompt A
Separate ANOVAs were conducted to assess group differences in inclusion of
argument elements coded as proposition, alternative proposition, and conclusion in
the second draft of prompt A (see Table 4 and the ‘‘Appendix’’). A violation of the
homogeneity assumption prevented the use of MANOVA, so separate analyses were
conducted for the three variables using the Bonferroni correction with alpha set at
.02 (i.e., .05/3).
For proposition elements, the ANOVA revealed no significant difference among
groups, F = 1.30, df[2,84], p = .279.
For alternative proposition elements, the assumption of homogeneity was not met
and the scores were significantly skewed due to a high frequency of zeroes.
Table 3 Number of revisions by purpose and argument elements on prompt A
Reader Observer Control
M (SD) M (SD) M (SD)
Purpose for revising
Information 5.35 (3.66) 4.04 (2.53) 4.86 (2.99)
Other 2.03 (1.87) 1.74 (1.88) 2.39 (3.14)
Argument element revised
Proposition elements 3.74 (2.65) 3.29 (2.32) 3.86 (2.58)
Alternative proposition elements .87 (1.91)a .11 (.31)a,b .07 (.26)b
Means in the same row that do not share subscripts differ significantly at p \ .01
1462 N. S. Moore, C. A. MacArthur
123
Therefore, the Kruskal-Wallis H test was used. There was a significant difference
among groups on inclusion of alternative propositions, (H(2) = 14.06, p \ .001).
Mann-Whitney tests with Bonferroni correction were used to follow up this finding.
The reader group included more alternative proposition elements than the control
group (p \ .001; effect size, d = .67) and more than the observer group (p \ .01;
effect size, d = .63).
For conclusions, the ANOVA was not significant, indicating no difference among
groups on this element, F = 1.5, df (2,84), p = .231.
Effects of treatment on prompt B, transfer task
Both the first and second draft for prompt B were written after the treatment. Thus,
treatment may have affected the quality of the first draft or the improvement in
quality due to revision. For the first drafts, we used an ANCOVA with the first draft
from prompt A as the covariate. No significant difference was found (F = .868,
df[2,86], p = .424). For the final draft, we also used ANCOVA with quality of the
first draft as a covariate (see Table 5). No significant difference was found
(F = .574, df[2,86], p = .565). In addition, analyses of variance showed no
significant differences among groups regarding the presence of argument elements
on the final draft of prompt B. The effects of the treatment did not transfer to a new
persuasive writing task 1 week following instruction.
Think alouds and semi-structured interviews
The think alouds and semi-structured interviews (12 students, 4 from each
condition) were coded for the types of revisions participants made and the rationale
that the participants gave for their revisions. The results support and extend the
Table 4 Argument elements on the final draft of prompt A
Reader Observer Control
M (SD) M (SD) M (SD)
Proposition elements 9.35 (5.14) 7.79 (2.74) 7.96 (4.0)
Alternative proposition elements .87 (1.71)a .07 (.38)b .04 (.19)b,c
Conclusion .45 (.68) .25 (.52) .50 (.51)
Means with the same subscript do not differ significantly. Means with different subscripts differ as
follows: a and b differ at p \ .01; a and c differ at p \ .001
Table 5 Overall persuasive quality ratings for prompt B
Group First draft Second draft Second adjusted
M (SD) M (SD) M (SE)
Reader 3.31 (0.99) 3.35 (1.01) 3.24 (0.09)
Observer 3.20 (0.98) 3.13 (0.95) 3.10 (0.10)
Control 2.98 (0.82) 3.05 (0.82) 3.21 (0.10)
The effects of being readers and observing readers 1463
123
quantitative results related to group differences in revision type. Two major themes
emerged from the analysis: there were group differences in audience awareness and
there were group differences in the perception of the activities.
Group differences in audience awareness
Audience awareness was made apparent in several ways. First, students made
revisions or comments to show consideration of how an audience may also benefit
by sharing the same opinion on the topic (e.g., students gave a reason why giving
less homework may benefit the teacher, not just the student). Secondly, students
made revisions or comments about making the tone of the letter more appropriate
for the audience. Finally, students made revisions or comments to show awareness
that an outside audience may have a different point of view (i.e., a
counterargument).
The participants in the reader group showed the most evidence of audience
awareness during the revision phase (i.e., 33 instances). In addition, the evidence of
audience awareness for this group was the most rich and varied. Furthermore, all
readers interviewed showed evidence of thinking about audience in some way.
For example, one reader, John (all names are pseudonyms), was concerned that
his letter might have an inappropriate tone. Therefore, one of his major revisions
was to delete a phrase that he felt sounded like he was yelling at the teacher:
John: [reading]…now I’m gonna change this because I put: [rereading what he
wrote] ‘‘You know what.’’ And that sounds like I’m kinda yelling at him.
It was clear that John became more aware of tone through his participation in the
experimental activity. During the semi-structured interview, which followed the
think aloud, John was asked about what he learned through his participation in the
study activity. He responded:
John: We were talking about persuasive writing. We were talking about what
should be changed in there and this one letter, it was saying [quoting letter]
‘‘[if] you give us more homework [and] stupid’’ and ‘‘if you give us more
homework we will hate school.’’ Everybody said that shouldn’t be persuasive.
I mean it might persuade the teacher to yell at him [the writer] or give him
detention.
Another reader, Nicole, decided that one of her main revisions would be to
include a reason about why less homework would benefit the teacher. Prior to her
participation in the activity, Nicole only included reasons about why less homework
would be beneficial to her or why more homework would be harmful to her.
Nicole: [I’m] adding something: [talking aloud as she is adding a phrase to her
rough draft] ‘‘and the teacher might not get to grade it or will stay at school for
a couple hours.’’
In the semi-structured interview that followed her think aloud, Nicole was asked
about why her letter may persuade the teacher. She responded:
1464 N. S. Moore, C. A. MacArthur
123
Nicole: It [has] a lot of detail for why it’s bad if you give us more homework
and what it could do to affect the students and the teacher.
Through her revision activity and her reflection on her revisions, it is clear that
Nicole is beginning to think about how her opinion may benefit her audience in
addition to herself.
Finally, another reader, Maura, thought about her audience in a different way.
She decided to add a counterargument and a rebuttal to the first draft of her letter.
She began her think aloud by saying:
Maura: Well, somewhere at the end I think. [scanning her writing] Here we go.
I think I might add something about how people [who] don’t agree with me
would feel. And, I said, [summarizing what she already wrote] yes, because it
would prepare us for middle school. [thinking about what she will write] And I
think some…a lot of people wouldn’t agree. So, it will probably be good to
share that point of view because yesterday there were one of the letters didn’t
share the other point of view…and it was okay, but I think if it shared the other
point of view it would have been a little better. So, I think if I added that other
point of view to mine it might be a little better. I’m just gonna write: [writing]
‘‘Even though people might not agree it will help prepare them and in the end
they’ll be glad.’’ I wrote [rereading what she wrote] ‘‘Even the students who
do not agree with me would be more prepared and in the sixth grade will be
glad you gave them more homework.’’
In summary, audience awareness was evident in the reader group’s thoughts in
multiple ways: they thought about tone, they thought about reasons to benefit their
audience, and they thought about counterarguments and rebuttals.
The observer group also showed evidence of audience awareness (i.e., a total of
22 instances), but it was not as rich and varied as the examples found from the
reader group participants. Of the four observers interviewed, only two showed
evidence of audience awareness. In addition, audience awareness for the observer
group focused on tone only. Their revision activity focused primarily on word
choice and clarifying or explaining what they already wrote in their first draft in
order to make their letters make more sense. For example, Timothy changed word
choice to make the tone more appropriate.
Timothy: Right here…where in the beginning where I say: [rereading what he
wrote] ‘‘I think it is a very bad idea to give us a lot more homework.’’ And I
think that…I don’t want to seem mean to the teacher. [summarizing] But like
saying it’s a very bad idea to give us more a lot more homework doesn’t sound
quite right to talk to a teacher with. So I will change it to: [writing] ‘‘I disagree
with the idea that we should have more homework.’’
Like John from the reader group, Timothy was concerned with sounding mean to
his teacher. However, Timothy spent the majority of his revision focusing on word
choice and clarifying his writing in order to make it sound right instead of on
concern for his audience. For example, a typical revision for Timothy sounded like
the following:
The effects of being readers and observing readers 1465
123
Timothy: I’m moving this sentence up here to: [rereading what he wrote]
‘‘Finally, I believe that we are being prepared for the next grade throughout
the school year.’’ But I want to change the wording around. I want to say:
[writing] ‘‘Finally I believe that we are being prepared throughout the school
year for the next grade.’’
Thus, this more typical revision illustrates Timothy rearranging two phrases in
the same sentence. Overall, Timothy’s transcript represents a blend of revisions
geared toward making tone more appropriate for the audience and surface revisions.
These are characteristic of the observer group’s revision practices. Whereas
audience awareness is apparent, it is not as strongly emphasized or evident in as
wide a variety as in the reader group revisions.
The control group only had two instances of audience awareness and they were
by the same participant. Instead, the control group transcripts were rife with changes
made because the word choice did not sound right to the writer. For example,
Brittany (who was the same participant that considered her audience twice) started
her think aloud by saying:
Brittany: Well, I already made this one change…’cause I said: [rereading what
she wrote] ‘‘Just give us more homework already.’’ And then I said: [rereading
what she wrote] ‘‘Why? I’m glad you asked.’’ It just…when I read it out loud
it didn’t sound right to me so I thought I’d change it so I put: [rereading what
she wrote] ‘‘I’ll tell you why.’’ Because it just didn’t sound right to me. And I
don’t know what else I’m gonna do yet…. probably read it out loud again to
see if anything sounds weird…’cause I wrote it on Monday. I’ll read it out
loud and see if anything sounds weird. [rereading silently] I don’t like that
sentence but I don’t know how to change it.
Brittany lacks strategies for revising, stating, ‘‘I don’t like that sentence but I
don’t know how to change it.’’ The lack of specific direction for revision apparent in
the control group transcripts may contribute to their tendency to make surface
revisions (i.e., rewording parts of their text) and revisions that do not alter the
meaning or persuasiveness of their letters.
Group differences in perception of activity
Follow-up semi-structured interviews conducted immediately after think alouds
confirmed and clarified students’ think alouds and allowed time for students to
discuss what they thought about the activity. Results show that students from the
reader group were much more enthusiastic about their experience than students in
the observer and control groups and also saw more value in the activity. For
example, Maura from the reader group found value in reading and evaluating
others’ writing:
Maura: When we were going over the writing, I realized that I didn’t have
some of those things [in the letters discussed] and that I should go back and
revise and make sure I added them or made sure that I had them. Yeah and I
1466 N. S. Moore, C. A. MacArthur
123
think that later in other writing pieces, I’ll probably remember to do that other
stuff.
Nicole, another reader, thought the activity was fun because she had the chance
to offer advice to the authors of the letters. She stated: ‘‘It was just fun to see what
stories [letters] they had and how you could fix it.’’
Although the observers found their activity fun and valuable too, stating ideas like
‘‘I thought it was gonna help my writing first of all’’ and ‘‘It was kind of a fun thing to do
I thought,’’ the observers often asked the investigators to have a turn as a reader. In
their interviews, observers stated this again. For example, Owen said, ‘‘I would switch
jobs…so we could each have the experience [of being a reader].’’ In contrast, the
control group did not have much to say about their participation in practice writing
activities. Students in this group were silent or stated that they could not think of an
answer when asked about whether they liked the writing activity. One student did not
even perceive practice writing as an activity to help him learn to write.
Group discussions
Both the reader groups and the observer groups were asked to discuss the
characteristics of effective persuasive texts. The readers were asked to discuss three
persuasive letters, and the observers were asked to discuss the readers’ discussions.
Reader and observer discussions were coded for the evaluation criteria students
identified and the type of discussion move students utilized. Both the reader and
observer groups identified the same criteria with the exception of one criterion. On
the other hand, the reader and observer groups utilized different discussion moves
during their discussion tasks.
Group similarities regarding effective persuasive writing
Both the readers and the observers mentioned evaluation criteria for good writing in
general and for persuasive writing in particular. The evaluation criteria that
appeared with great frequency in the discussions included: considers the audience’s
perspective, written in an appropriate tone, contains details, has a clear focus,
includes an introduction, states a clear opinion, mentions an opposing view, has a
clear organization, includes reasons, includes a conclusion, and uses good word
choice. For example, in one reader group discussion several of these ideas are
discussed:
Jenna: Do you think this letter will persuade the teacher, why or why not?
Ethan: I think it will…well…semi-sort of because if they did…well…they
gave good reasons but they still should have put more details and stuff so…[included reasons]
Maura: I think it kind of would because it was talking about what it would
affect in their [the teacher] lives…not just what it would affect in our lives so
probably [it would] matter more to them [the teacher] then if it were just like
about him or her [audience consideration]
The effects of being readers and observing readers 1467
123
Caitlin: I think that it would because they’re…as well as persuading that it
would be good for the students, just like you said, they were telling the
teachers that it would be good for them because it would give them less papers
to grade and I think that the teachers might have not thought about that and
then they’re like, ‘‘Oh, that’s a good point. Maybe we should give less
homework.’’ So I think they have really persuasive reasons. [audience
consideration]
Jenna: I agree with Ethan and Caitlin and Maura…I agree with everyone but
like I think it …there are some things about it they could change and make
better…there are some things that are good about…there are some things that
aren’t good…like they put everything in one paragraph when it should be like
three paragraphs and they don’t have like a closing really or a lead…so they
don’t really like wrap up or anything and so it could persuade the teacher or it
could not persuade the teacher …yeah half and half. [organization]
Similarly, in the observer group evaluation criteria are identified:
Nicole: …let’s see… a little more detail? [includes details]
Owen: Yeah…Sara: lots more …Owen: More good…lots more detail…Sara: maybe less general [includes details]
Nicole: Yeah…Owen: maybe more good manners [written in an appropriate tone]
Despite similarities in criteria discussed, there was one point of differentiation
between the reader and observer group discussion. The readers identified
consideration of opposing viewpoints as one characteristic of strong persuasive
writing, but the observers did not mention this idea. Instead they identified other
criteria for consideration of audience such as providing reasons why the audience
may side with the opinion and using the appropriate tone. The excerpt below
illustrates how one reader group identified and evaluated the idea of including
opposing viewpoints to strengthen an argument:
Zoe: yeah it says [reading] ‘‘some people like you might disagree with me and
say that we need… might learn more if we had more homework…’’
Anna: yeah I think that’s a reason.
Despite this important difference between readers and observers regarding
discussion about counterarguments, readers and observer groups were both
successful in identifying many criteria for effective persuasive writing. Still, as
illustrated in the excerpts above from the reader and observer groups, the process by
which the groups discussed evaluation criteria was much different and will be
discussed in detail next.
Group differences in discussion moves and quality
Overall the readers’ discussions were longer, more elaborate and more specific than
the observers’ discussions. The readers’ discussions included a variety of discussion
1468 N. S. Moore, C. A. MacArthur
123
moves such as critiquing written text, suggesting ways for the writer to improve the
text, and comparing texts with other texts. On the other hand, observers spent most
of their time just listing the evaluation criteria for persuasive writing.
The readers’ discussions were characterized by 12 dominant discussion moves,
which demanded reflection, evaluation, and analysis of the persuasive letters they
were comparing and contrasting. The discussion moves can be organized into
several cateogories: moves related to the discussion of the letters, moves related to
group management, and moves that did not serve a functional purpose. Discussion
moves related specifically to analysis of the letters included: critiquing the letters,
suggesting areas of improvement, comparing letters to other texts or objects,
elaborating an idea or comment, analyzing the effect of the letters on readers, and
analyzing the author’s purpose. Discussion moves related to group management
included: clarifying or explaining a comment made during discussion, correcting
peers’ analysis or evaluation, disagreeing with peers, and keeping peers on task.
Nonfunctional discussion moves, although infrequent, included not responding to or
answering a question because one did not have anything to say or did not
understand, and repeating what someone else already said or what the discussion
question asked because one did not have anything else to add or did not understand.
Several of these discussion moves are illustrated below in an excerpt from one
reader group small-group discussion:
Anna: I think the writer gave good reasons but one of them wasn’t so good,
like falling asleep in class, because I don’t really know much kids that would
fall asleep in class because they would probably get in trouble…I don’t think
that kids would fall asleep in class so I don’t even think that should be in
there…[critiquing]
Reanna: I think that the reasons are good, but some of them really aren’t that
good…[repeating]
Lloyd: well I disagree with what A said about how that…well they might fall
asleep in class …[disagreeing]
Anna: but it’s not very likely [talking over L] [clarifying]
Zoe: wait wait wait….Lloyd? [keeping on track]
Lloyd: I think it because, yeah, if they do stay up til 11:30 they might fall
asleep in class and that might be a problem [clarifying]
In contrast to the reader discussions, observer discussions contained one
discussion move: listing. Discussions typically began with one participant sharing
one of the evaluation criterion on his or her list of observational notes. Next, another
participant would share a different evaluation criteria extracted from the readers’
discussion. There was rarely more than one comment made on a single aspect, or
evaluation criteria, during observer discussion. In addition, there was not much
elaboration or specificity given to each evaluation criteria named. The excerpt
below is from one observer group small group discussion and illustrates the
discussion move evident in this group as well as the difference between the observer
and reader group discussions in terms of elaboration, variation, and specificity:.
Sam: vocabulary…
The effects of being readers and observing readers 1469
123
Max: yeah, vocabulary….
Paul: the last one held my interest…Max: you have to be honest…Amanda: you are supposed to be persuasive
Paul: you shouldn’t try to speed through your writing
Sam: good reasons…Paul: you could read them without stopping…Max: let’s do one more
Amanda: good examples. [listing]
In summary, although the readers’ and observers’ discussions focused on the
same topics, they differed in the way in which the topics were discussed.
Through rich discussions, readers grappled with understanding what aspects of
writing contributed to a text’s persuasiveness. In contrast, although observers
were asked to discuss what they had observed and generate a list of evaluation
criteria, they engaged in little discussion beyond listing or recounting what they
heard. This difference helps to explain the differences obtained from the
quantitative analysis of writing quality, revision type, and inclusion of argument
elements.
Discussion
The overall purpose of the current study was to investigate the effects of reader and
observational learning activities in comparison to practice writing activities on fifth-
grade students’ persuasive writing. Writing quality after revision was significantly
higher for students in the reader group than for students in the control group. The
observers did not differ significantly from either the readers or the control group.
However, on transfer papers written 1-week after the instructional activities, the
three groups did not differ on writing quality.
Analyses of revisions and argumentative elements included in final drafts found
differences between the reader and control groups in inclusion of persuasive text
elements that are markers of audience awareness. Readers made significantly more
revisions related to alternative proposition elements than control students. Also,
readers included significantly more alternative proposition elements in their revised
papers than observers and control students. On the other hand, no significant
differences among groups were found on purpose for revising or inclusion of
propositions.
Qualitative analyses shed additional light on the quantitative findings. Think-
aloud protocols during revision showed that students in the reader groups made
many of their revisions based on their goals to communicate with and convince the
audience. Few students in the observer or control groups commented on audience
issues as they were revising their drafts or during the semi-structured interviews that
followed the think alouds. Moreover, analyses of student semi-structured interviews
also illustrated that the reader group had a more positive perception of their
1470 N. S. Moore, C. A. MacArthur
123
participation in the reader activities than the observer or control groups. Finally, an
analysis of reader and observer discussions illustrated that readers engaged in much
richer and more elaborated discussions than the observers and extracted one
additional criterion, use of counterarguments, which the observers missed.
The quantitative and qualitative analyses offer a rich, in-depth look at how
students’ writing and revising are affected by participation in reader and observer
activities as well as information on students’ cognitive processes during revision.
The combination of quantitative and qualitative analyses allows a comprehensive
evaluation of students’ writing performance; analyses illustrate differences in
students’ written products as well as differences in students’ writing processes.
Therefore, this study not only offers insight into what changes students make as the
result of instruction, but why and how they make these changes. The results of the
study provide important information for both practitioners and researchers in the
areas of theory, practice, and future study.
Implications for theory
The finding that readers were able to write better quality papers than the control
group is consistent with previous research (e.g., Holliway & McCutchen, 2004;
Traxler & Gernsbacher, 1992, 1993). As previous studies show, readers can take
many stances when reading. Therefore, it is important to note that the readers’ role
in this study was to rate or rank texts according to overall persuasiveness. Readers
were acting primarily as raters rather than as readers who would be persuaded to
take some action or change their opinions. Skilled writers evaluate their own writing
in terms of evaluation criteria appropriate for the writing task (MacArthur, in press).
Through reading other students’ persuasive letters and discussing whether and why
they were persuasive, the readers in this study learned evaluation criteria that they
could use to revise and improve the overall quality of their writing.
The current study extends previous findings through quantitative analysis of the
types of revisions made and qualitative analysis of students’ explanations of the
reasons for their revisions. Analysis of revisions showed that readers made more
revisions than controls in argumentative elements specifically geared towards
accommodating the perspectives of the audience (i.e., counterarguments and
rebuttals). Analysis of the think-aloud data and interviews showed that readers
demonstrated greater audience awareness, as revealed in comments about reasons
that would appeal to the audience, appropriate tone for addressing the audience, and
potential opposing positions.
In addition, whereas the previous research on being a reader used a writing task
with a concrete goal (i.e., describe a tangram so that a reader can identify it;
Holliway & McCutchen, 2004; Traxler & Gernsbacher, 1992, 1993), the current
study used a common school writing task, a persuasive letter. Therefore, the findings
suggest that reader activities are an effective way for upper elementary school-age
students to develop knowledge about audience specifically and quality writing in
general, which can be applied during revision to improve the overall quality of their
writing.
The effects of being readers and observing readers 1471
123
The finding that the observer group did not improve significantly over the reader
or control groups on any of the measures is inconsistent with previous research on
observational learning (Rijlaarsdam et al., 2004; Sonnenschein & Whitehurst,
1984). One explanation for this finding is that previous research on observation of
readers as a way to learn about writing has been conducted with junior high school
students rather than upper elementary school students. Age may be a factor in
observational learning.
Another explanation may be that the observational learning activities in
previous research involved more analysis. For example, in a closely related
study, Rijlaarsdam et al. (2008) asked observers to listen as readers discussed a
set of persuasive letters, to analyze the evaluation criteria that readers used, and
to make a poster and presentation to their class about what they had learned. In
the current study, students listened as readers discussed papers, discussed their
observations, and created a list of criteria for effective persuasive letters, but
they did considerably less analysis of evaluation criteria than in the previous
study. In fact, the qualitative analyses of the reader and observer group
discussions shows that the readers were more actively engaged in discussing
evaluation criteria. Readers were asked to read and evaluate each piece of
writing, whereas observers were asked to discuss what they heard the readers
say. Readers’ discussions were elaborate and specific to the texts, whereas
observers’ discussions were list-like and general. The analysis of observers’
discussions showed that they did not extract counterarguments as an evaluation
criterion. Therefore, it is not surprising that they did not include this element in
their revisions. The effects of reader and observer activities may depend on the
cognitive activities involved.
It is also important to consider why there were no group differences in
writing quality or inclusion of argument elements on the transfer writing task.
The most probable explanation for this finding is the short duration of the
intervention. The reader and observer activities lasted only 45 min on one day
and students had no prior instruction in persuasive writing during fifth grade.
Although Couzijn and Rijlaarsdam (2004a, b) did find transfer effects in their
short intervention study, the transfer task in their study was much different than
the transfer task in the current study. Couzijn and Rijlaarsdam’s (2004a, b)
transfer task required students to write an advice letter for how to write an
effective instructional text; on the other hand, the transfer task in the current
study required students to apply what they learned in writing their own
persuasive texts. Future studies should implement reader and observer activities
for an extended period of time to determine whether time would lead to greater
transfer. In addition, future studies should examine the effects of instruction on
different types of transfer activities.
Implications for practice
The study was carried out using typical academic writing tasks in small groups that
would be feasible in regular classrooms. Past research on the effects of being a
reader or observing readers has been conducted in artificial settings (i.e., researcher
1472 N. S. Moore, C. A. MacArthur
123
and participants work one-on-one) and with concrete writing tasks of limited
generalizability such as describing tangrams (Holliway & McCutchen, 2004) or
instructions for a science experiment (Couzijn, 1999; Couzijn & Rijlaarsdam,
2004b). The design of the current study was in line with Rijlaarsdam et al. (2008) in
its goal of finding ways to study reader and observer activities in a real classroom
setting. Although the readers did write more persuasive essays than both the
observer and control groups, it is important to note that the readers’ essays did not
score very highly on the overall quality rubric in general. The average score for a
revised persuasive essay was a 3.89 on a 7-point scale. In addition, it is important to
note that not all readers were able to include argumentative elements geared towards
audience awareness after revision. Thus, the results imply that some learners may
require more extended or more explicit instruction in using evaluation criteria
effectively.
Implications for future research
Reader and observer activities hold promise as effective instructional methods for
teaching writing. However, future research is needed to understand the benefits and
challenges of studying these types of activities and using these methods. Future
research should use larger samples to permit nested analysis and examination of the
role of group discussion. In addition, it is necessary that future research
acknowledge that reader and observer activities can vary on many dimensions.
One particularly important consideration in designing activities is the nature of the
cognitive and social requirements. Reading to perform a task, such as identifying an
object from a description or carrying out a science experiment from written
instructions, is different from reading to evaluate as in peer review. The nature of
the analytic task set for readers or observers is critical to the effects, as shown in the
contrast between the results of the current study and a related previous study
(Rijlaarsdam et al., 2008). The social interaction in groups is also important in
stimulating engagement and developing understanding.
In conclusion, the present study addressed the call for more research on
effective writing instructional practices for adolescents (Graham & Perin, 2007).
Instruction must address both the social and cognitive complexities in writing
and it must hold teachers and students to high standards. Despite the difficulties
inherent in designing effective writing instruction, literacy scholars are respon-
sible for meeting the challenge. Adolescents deserve carefully designed
instruction to prepare them to write clearly and logically for a range of
purposes and audiences so that they can be successful students, workers, and
citizens of a democratic society.
Appendix
See Table 6.
The effects of being readers and observing readers 1473
123
Tab
le6
Co
rrel
atio
nm
atri
x
Var
iab
le1
23
45
67
89
10
11
12
13
14
1.
Qu
alit
yA
11
2.
Qu
alit
yA
2.7
72*
*1
3.
Qu
alit
yB
1.5
33*
*.5
64*
*1
4.
Qu
alit
yB
2.5
32*
*.6
17*
*.8
38*
*1
5.
To
tal
rev
isio
ns
.02
1-
.08
7.1
60
.10
71
6.
Rev
ise
for
info
.-
.18
8-
.15
8.0
86
-.0
06
.74
4*
*1
7.
Rev
ise
for
oth
er.1
43
.03
7.2
35*
.14
7.7
45*
*.2
89*
*1
8.
Rev
ise
pro
p.
elem
.-
.07
6-
.15
1.0
57
.04
1.8
69*
*.8
57*
*.3
22*
*1
9.
Rev
ise
Alt
.p
rop
.
elem
.
.13
4-
.01
9-
.03
4.0
58
.43
0*
*.0
68
.14
8.4
86*
*1
10
.A
rgu
men
tp
rop
.
elem
.
.48
9*
*.6
24*
*.3
60*
*.4
39*
*.1
17
-.0
80
.12
0.0
57
.29
9*
*1
11
.A
rgu
men
tal
t.el
em.0
67
.28
8*
*.2
19*
.33
5*
*-
.02
5-
.00
7-
.03
3-
.00
4-
.05
5-
.00
11
12
.A
rgu
men
t
con
clu
sio
ns
.01
8.0
13
.17
9.1
50
.03
3.0
09
.06
7.0
00
-.0
40
.08
4-
.08
81
13
.D
ST
Pre
adin
g.1
03
.07
3.2
37*
.15
6.1
42
.05
8.1
72
.06
7.1
37
.07
4-
.08
8.0
84
1
1474 N. S. Moore, C. A. MacArthur
123
Tab
le6
con
tin
ued
Var
iab
le1
23
45
67
89
10
11
12
13
14
14
.D
ST
Pw
riti
ng
-.0
75
-.0
32
.15
5.1
25
.26
0*
.10
2.2
46*
.17
6.1
61
-.0
54
.09
1.0
86
.45
7*
*1
n=
87
exce
pt
for
cell
sre
late
dto
the
DS
TP
sco
res
wh
ere
n=
84
**
Corr
elat
ion
issi
gn
ifica
nt
atth
e0
.01
lev
el(2
-tai
led
)
*C
orr
elat
ion
issi
gn
ifica
nt
atth
e0
.05
lev
el(2
-tai
led
)
Fu
llv
aria
ble
nam
esb
yn
um
ber
1.
Qu
alit
yra
ting
,p
rom
pt
A,
dra
ft1
2.
Qu
alit
yra
ting
,p
rom
pt
A,
dra
ft2
3.
Qu
alit
yra
ting
,p
rom
pt
B,
dra
ft1
4.
Qu
alit
yra
ting
,p
rom
pt
B,
dra
ft2
5.
To
tal
revis
ion
s
6.
Pu
rpo
sefo
rre
vis
ing
,p
rom
pt
A:
info
rmat
ion
7.
Pu
rpo
sefo
rre
vis
ing
,p
rom
pt
A:
oth
er
8.
Arg
um
ent
elem
ent
revis
ed,
pro
mp
tA
:p
rop
osi
tio
nel
emen
ts
9.
Arg
um
ent
elem
ent
revis
ed,
pro
mp
tA
:al
tern
ativ
ep
rop
osi
tio
nel
emen
ts
10.
Arg
um
ent
elem
ents
incl
uded
inpro
mpt
A:
pro
posi
tion
elem
ents
11.
Arg
um
ent
elem
ents
incl
uded
inpro
mpt
A:
alte
rnat
ive
pro
posi
tion
elem
ents
12.
Arg
um
ent
elem
ents
incl
uded
inpro
mpt
A:
concl
usi
ons
13
.D
elaw
are
stat
ete
stin
gp
rog
ram
(DS
TP
)re
adin
g
14
.D
elaw
are
stat
ete
stin
gp
rog
ram
(DS
TP
)w
riti
ng
The effects of being readers and observing readers 1475
123
References
Auriac-Peyronnet, E. (2001). The impact of oral training on argumentative texts produced by ten- and
eleven-year old children: Exploring the relation between narration and argumentation. EuropeanJournal of Psychology of Education, 16, 299–317.
Bandura, A. (1986). Social foundations of thought and action: A social cognitive theory. Englewood
Cliffs, NJ: Prentice-Hall.
Boscolo, P., & Ascorti, K. (2004). Effects of collaborative revision on children’s ability to write
understandable narrative texts. In L. Allal, L. Chanquoy, & P. Largy (Eds.), Revision: Cognitive andinstructional processes (pp. 157–170). Boston, MA: Kluwer.
Calkins, L. M. (1986). The art of teaching writing. Portsmouth, NH: Heinemann.
Chambliss, M. J. (2001). Analyzing science textbooks materials to determine how ‘‘persuasive’’ they are.
Theory into Practice, 40, 255–264.
Cho, K., & MacArthur, C. (2011). Learning by reviewing. Journal of Educational Psychology, 103, 73–84.
Clark, R. A., & Delia, J. G. (1977). Cognitive complexity, social perspective-taking and functional
persuasive skills in second- to ninth-grade children. Human Communication Research, 3, 128–134.
Coirier, P., Andriessen, J., & Chanquoy, L. (1999). From planning to translating: The specificity of
argumentative writing. In J. Andriessen & P. Coirier (Eds.), Foundation of argumentative textprocessing (pp. 1–28). Amsterdam, Netherlands: Amsterdam University Press.
Couzijn, M. (1999). Learning to write by observation of writing and reading processes: Effects on
learning and transfer. Learning and Instruction, 9, 109–142.
Couzijn, M., & Rijlaarsdam, G. (2004a). Learning to read and write argumentative text by observation of
peer learners. In G. Rijlaarsdam, H. Van den Bergh, & M. Couzijn (Eds.), Studies in writing: Vol.14. Effective learning and teaching of writing. A handbook of writing in education (2nd ed.,
pp. 241–258). Boston, MA: Kluwer.
Couzijn, M., & Rijlaarsdam, G. (2004b). Learning to write instructive texts by reader observation and
written feedback. In G. Rijlaarsdam, H. Van den Bergh, & M. Couzijn (Eds.), Studies in writing:Vol. 14. Effective learning and teaching of writing. A handbook of writing in education (2nd ed.,
pp. 209–240). Boston, MA: Kluwer Academic Publishers.
Crasnich, S., & Lumbelli, L. (2005). Improving argumentative writing by fostering argumentative speech.
In G. Rijlaarsdam, H. Van den Bergh, & M. Couzijn (Eds.), Studies in writing: Vol. 14. Effectivelearning and teaching of writing (2nd ed., pp. 181–197). Boston, MA: Kluwer.
Crowhurst, M. (1990). Teaching and learning the writing of persuasive/argumentative discourse.
Canadian Journal of Education, 15, 348–359.
Delaware State Testing Program (2008). Delaware comprehensive assessment portal. Retrieved from
http://www.doe.k12.de.us/aab/default.shtml.
Felton, M., & Kuhn, D. (2001). The development of argumentative discourse skill. Discourse Processes,32, 135–153.
Ferretti, R. P., MacArthur, C. A., & Dowdy, N. S. (2000). The effects of an elaborated goal on the
persuasive writing of students with learning disabilities and their normally achieving peers. Journalof Educational Psychology, 92, 694–702.
Field, A. (2005). Discovering statistics using SPSS (2nd ed.). London: Sage.
Flower, L. S., & Hayes, J. R. (1980). The dynamics of composing: Making plans and juggling constraints.
In L. Gregg & E. R. Steinberg (Eds.), Cognitive processes in writing (pp. 31–50). Hillsdale, NJ:
Lawrence Erlbaum.
Flower, L. S., & Hayes, J. R. (1981). Plans that guide the composing process. In C. H. Frederiksen & J.
F. Dominic (Eds.), Writing: Vol. 2. The nature, development, and teaching of written composition(pp. 39–58). Hillsdale, NJ: Erlbaum.
Golder, C., & Coirier, P. (1996). The production and recognition of typological argumentative text
markers. Argumentation, 10, 271–295.
Graham, S., & Perin, D. (2007). Writing next: Effective strategies to improve writing of adolescents inmiddle and high school. Washington, DC: Alliance for Excellent Education.
Hayes, J. R. (1996). A new framework for understanding cognition and affect in writing. In C. M. Levy &
S. Ransdell (Eds.), The science of writing (pp. 1–27). Mahwah, NJ: Erlbaum.
Holliway, R. D., & McCutchen, D. (2004). Audience perspective in young writers’ composing and
revising. In L. Allal, L. Chanquoy, & P. Largy (Eds.), Revision: Cognitive and instructionalprocesses (pp. 157–170). Boston, MA: Kluwer.
1476 N. S. Moore, C. A. MacArthur
123
Knudson, R. E. (1994). An analysis of persuasive discourse: Learning how to take a stand. DiscourseProcesses, 18, 211–230.
Lumbelli, L., Paoletti, G., & Frausin, T. (1999). Improving the ability to detect comprehension problems:
From revising to writing. Learning and Instruction, 9, 143–166.
Lundstrom, K., & Baker, W. (2009). To give is better than to receive: The benefits of peer review to the
reviewers’ own writing. Journal of Second Language Writing, 18, 30–43.
MacArthur, C. A. (in press). Evaluation and revision processes in writing. In V. W. Berninger, (Ed.), Past,present, and future contributions of cognitive writing research to cognitive psychology. Psychology
Press.
MacArthur, C. A., Ferretti, R. P., & Okolo, C. M. (2002). On defending controversial viewpoints: Debates
of sixth graders about the desirability of early 20th-century American immigration. LearningDisabilities Research and Practice, 17, 160–172.
MacArthur, C. A., Schwartz, S. S., & Graham, S. (1991). Effects of a reciprocal peer revision strategy in
special education classrooms. Learning Disabilities Research and Practice, 6, 201–210.
Midgette, E., Haria, P., & MacArthur, C. A. (2005). The effects of content and audience awareness goals
for revision on the persuasive essays of fifth- and eighth-grade students. Reading and Writing: AnInterdisciplinary Journal, 21, 131–151.
Moore, N., & MacArthur, C. A. (2008, February). The role of automated essay scoring in writingclassrooms. Paper presented at Writing Research Across Borders Conference, Santa Barbara, CA.
National Commission on Writing. (2003). The neglected R: The need for a writing revolution. Retrieved
May 5, 2008 from http://writingcommission.org/report.html.
National Commission on Writing. (2004). Writing: A ticket to work…or a ticket out: A survey of businessleaders. Retrieved May 5, 2008 from http://www.writingcommission.org/report/html.
National Commission on Writing (2005). Writing: A powerful message from state government. Retrieved
May 5, 2008 from http://www.writingcommission.org/report/html.
Nystrand, M. (2006). The social and historical context for writing research. In C. A. MacArthur, S.
Graham, & J. Fitzgerald (Eds.), Handbook of writing research (pp. 11–27). New York, New York:
Guilford Press.
Oostdam, R. (2004). Assessment of argumentative writing. In G. Rijlaarsdam, H. Van den Bergh, & M.
Couzijn (Eds.), Effective learning and teaching of writing (2nd ed., pp. 427–442). New York: NY:
Kluwer.
Persky, H. R., Daane, M. C., & Jin, Y. (2003). The nation’s report card: Writing 2002. (NCES 2003-529).
U.S. Department of Education. Institute of Education Sciences. National Center for Education
Statistics. Washington, DC: Government Printing Office.
Pontecorvo, C., & Girardet, H. (1993). Arguing and reasoning in understanding historical topics.
Cognition and Instruction, 11, 365–395.
Prior, P. (2004). Tracing process: How texts come into being. In C. Bazerman & P. Prior (Eds.), Whatwriting does and how it does it: An introduction to analyzing texts and textual practices. Mahwah,
NJ: Erlbaum.
Prior, P. (2006). A sociocultural theory of writing. In C. A. MacArthur, S. Graham, & J. Fitzgerald (Eds.),
Handbook of writing research (pp. 54–66). New York, NY: Guilford Press.
Pritchard, R. J., & Honeycutt, R. L. (2006). The process approach to writing instruction: Examining its
effectiveness. In C. A. MacArthur, S. Graham, & J. Fitzgerald (Eds.), Handbook of writing research(pp. 275–290). New York: Guilford.
Rijlaarsdam, G., Braaksma, M., Couzijn, M., Janssen, T., Raedts, M., Van Steendam, E., et al. (2008).
Observation of peers in learning to write, practice and research. Journal of Writing Research, 1,
53–83.
Rijlaarsdam, G., & Couzijn, M. (2000). Writing and learning-to-write. A double challenge. In R. Simons,
J. Van der Linden, & T. Duffy (Eds.), New learning (pp. 157–190). Dordrecht, Netherlands: Kluwer.
Rijlaarsdam, G., Couzijn, M., & Van den Bergh, H. (2004). The study of revision as a writing process and
as a learning-to-write process. In L. Allal, L. Chanquoy, & P. Largy (Eds.), Revision: Cognitive andinstructional processes (pp. 1–8). Norwell, MA: Kluwer.
Rijlaarsdam, G., & Van den Bergh, H. (2006). Writing process theory: A functional dynamic approach. In
C. A. MacArthur, S. Graham, & J. Fitzgerald (Eds.), Handbook of writing research (pp. 41–53).
New York, NY: The Guilford Press.
Rossman, G., & Rallis, S. F. (2003). Learning in the field: An introduction to qualitative research.
Thousand Oaks, CA: Sage.
The effects of being readers and observing readers 1477
123
Schriver, K. A. (1992). Teaching writers to anticipate readers’ needs: A classroom-evaluated pedagogy.
Written Communication, 9, 179–208.
Shanahan, T. (2004). Overcoming the dominance of communication: Writing to think and to learn. In T.
L. Jetton & J. A. Dole (Eds.), Adolescent literacy research and practice (pp. 59–73). New York:
Guilford.
Sonnenschein, S., & Whitehurst, G. J. (1984). Developing referential communication: A hierarchy of
skills. Child Development, 55, 1936–1945.
Spandel, V. (2005). Creating writers through 6-trait writing: Assessment and instruction (5th ed.). New
York, NY: Pearson.
Sperling, M., & Freedman, S. W. (2001). Review of writing research. In V. Richardson (Ed.), Handbookof research on teaching (4th ed., pp. 370–389). Washington, DC: American Educational Research
Association.
Traxler, M., & Gernsbacher, M. (1992). Improving written communication through minimal feedback.
Language and cognitive process, 7, 1–22.
Traxler, M., & Gernsbacher, M. (1993). Improving written communication through perspective taking.
Language and cognitive process, 8, 311–334.
1478 N. S. Moore, C. A. MacArthur
123
top related