the effects of being a reader and of observing readers on fifth-grade students’ argumentative...

The effects of being a reader and of observing readerson fifth-grade students’ argumentative writingand revising

Noreen S. Moore • Charles A. MacArthur

Published online: 11 June 2011

� Springer Science+Business Media B.V. 2011

Abstract Opportunities to read and analyze others’ writing or to observe readers

as they analyze writing might enhance one’s own sense of audience and improve

one’s own writing. This mixed-methods study investigated whether reader and

observer activities in comparison to writing practice activities affected fifth-grade

students’ persuasive writing and revising. After writing a first draft of a persuasive

letter, 87 fifth-grade students were randomly assigned to one of three conditions:

being a reader, observing readers, or practicing writing. The reader group read and

discussed three persuasive letters, considering whether they were persuasive and

why and selecting the most persuasive. The observer group listened to the reader

group’s discussions and took notes; then they had their own discussion to generate a

list of criteria for what made the letters persuasive. The practice-writing control

group practiced writing persuasive letters. Afterwards, all groups revised their first

drafts. The reader group produced second drafts that were of better quality and

contained more evidence of audience awareness than the control group. The

observer group did not differ from either group. The groups did not differ on a

transfer task occurring 1-week later. The authors discuss implications for designing

writing curriculums that utilize reader and observer activities.

Keywords Adolescents � Observational learning � Persuasive writing �Reading-writing � Revising

N. S. Moore (&)

Department of Special Education, Language and Literacy, The College of New Jersey,

Ewing, NJ 08628, USA

e-mail: [email protected]

C. A. MacArthur

School of Education, University of Delaware, Newark, DE, USA

e-mail: [email protected]

123

Read Writ (2012) 25:1449–1478

DOI 10.1007/s11145-011-9327-6

Introduction

Cognitive and social perspectives on writing emphasize the importance of audience

awareness (Flower & Hayes, 1980, 1981; Hayes, 1996; Nystrand, 2006; Prior,

2006). Proficient writers attend to audience while composing and understand that

writing is an interactive meaning-making process among readers and writers.

Argumentative writing, in particular, demands audience awareness because the

writer must consider the potential alternative perspectives of an audience to achieve

the compositional goal of convincing an audience to adopt a certain position on an

issue. Instructional activities in which writers take the role of readers or observe

readers may help them develop audience awareness.

Review of literature

The complexities of argumentative writing

In a report from the Carnegie Corporation of New York entitled, Writing Next:Effective strategies to improve writing of adolescents in middle and high schools,

the authors assert ‘‘writing well is not just an option for young people—it is a

necessity’’ (Graham & Perin, 2007, p. 3). The ability to write well is important for

school success; it is a tool for thinking and learning (Shanahan, 2004; Sperling &

Freedman, 2001). The ability to write persuasively is necessary because it is

practiced in many school subject areas, including science and social studies

(Chambliss, 2001; MacArthur, Ferretti, & Okolo 2002). Furthermore, persuasive

writing skills are required for success in higher education (National Commission on

Writing, 2003, 2004), the workplace (National Commission on Writing, 2004,

2005), and for participation in a democratic society and a global economy

(Crowhurst, 1990).

However, national assessments indicate that elementary, middle, and high

school students have considerable difficulty writing argumentative texts (Persky,

Daane, & Jin, 2003). In particular, research has shown that school-age writers

often fail to consider audience in their persuasive writing, as evidenced by their

lack of inclusion of counterarguments or rebuttals (Felton & Kuhn, 2001;

Knudson, 1994; Pontecorvo & Girardet, 1993). Consideration of audience when

writing persuasively is a complex task that demands perspective taking, or the

ability to decenter one’s self, in order to adopt another’s point of view or way of

thinking even though it may differ from one’s own beliefs (Clark & Delia, 1977;

Golder & Coirier, 1996). Holding alternative perspectives in mind in order to

construct arguments to support a proposition and counterarguments against

opposing propositions adds to the cognitive complexities of the writing process.

Argumentative writing poses particular social (e.g., consideration of audience

view points) and cognitive challenges (e.g., orchestrating declarative, procedural,

and conditional knowledge) for writers (Coirier, Andriessen, & Chanquoy, 1999;

Oostdam, 2004).

1450 N. S. Moore, C. A. MacArthur

123

Despite the difficulties, intervention studies show that with appropriate support

elementary and middle school students are able to include elements of audience

awareness in their persuasive writing. For example, Auriac-Peyronnet (2001) found

that 10- and 11-year old students who received oral training produced longer texts,

more argumentative and fewer non-argumentative utterances, and less use of the

personal ‘‘I’’ and more use of ‘‘you’’, which the author interpreted as a distancing

technique crucial to persuasive discourse. In another study, Midgette, Haria, and

MacArthur (2005) found that when fifth- and eighth-grade students were given

specific content and audience awareness goals for revising a persuasive essay, they

produced better quality essays than students given only content goals or only a

general goal to improve their essays. These studies suggest that with instructional

support, it is possible for students as young as fifth grade to demonstrate audience

awareness in ways that improve the quality of their writing.

Developing audience awareness

Peer review

One common way to develop audience awareness is through frequent response and

review from teachers and peers, which is a key aspect of process approaches to

writing instruction (Pritchard & Honeycutt, 2006). In particular, peer review

provides writers an opportunity to find out how well their writing communicates to

readers, as well as an opportunity to learn by reading and evaluating the writing of

their peers. Thus, they may learn both from receiving feedback from readers and

from giving feedback (MacArthur, in press).

A recent meta-analysis of research on writing instruction for adolescents

(Graham & Perin, 2007) reported strong effects (d = 0.75) for peer collaboration.

Peer collaboration was defined to include assistance at any stage of the process, but

all of the studies reviewed included peer review for revision. For example, Boscolo

and Ascorti (2004) studied peer review in grades 4, 6, and 8 in Italian schools. The

readers were instructed to read their partner’s text aloud, stop at a point where

something was unclear, and provide a suggestion for making this part of the text

clearer. Compared to children who only received teacher feedback, children who

received peer feedback had fewer information gaps and problems with clarity in

their final drafts.

Typical peer review involves both giving and receiving feedback. Two recent

studies isolated the effects of giving feedback, that is, the effects of reading and

evaluating others’ writing. Lundstrom and Baker (2009) randomly assigned

students in college L2 writing classes to either give feedback to peers or to

receive it for a full semester. The students who gave feedback made greater gains

in writing. Cho and MacArthur (2011) assigned undergraduate college students to

reviewer, reader, or no-treatment control conditions. The reviewers rated and

provided comments on three different peer papers of different qualities. The

readers read the same papers without evaluating or commenting on them.

Afterwards, all students wrote papers in the same genre, but on a different topic.

The reviewers wrote significantly better quality papers than the readers and the

The effects of being readers and observing readers 1451

123

no-treatment control group. These studies illustrate that reading and reviewing

writing can impact one’s writing quality.

Reading to act

Other research has investigated the effects on writing of another type of reading

activity—reading to comprehend and perform some specific action (Holliway &

McCutchen, 2004; Traxler & Gernsbacher, 1992, 1993). Holliway and McCutchen

(2004) had fifth-grade students write descriptions of tangrams (abstract figures

composed of geometric shapes). The students were then assigned to three groups:

(a) feedback only, (b) feedback and rate, (c) feedback and read-as-a-reader. All

students received simple feedback indicating whether a reader could successfully

match their description to the correct tangram picture. In the feedback and rate

condition, students also read and rated three descriptions of tangrams by other

students for general informational adequacy. Finally, in the feedback and read-as-a-

reader condition, writers received the same three descriptions of tangrams written

by other students, but instead of rating them, they had to match the descriptions to

the correct tangram picture. Students then revised their original descriptive writing.

Analyses indicated that the read-as-a-reader group made the most significant gains

in writing quality as measured by the number of correct description-to-tangram

matches by multiple readers.

In this study (Holliway & McCutchen, 2004), reading to perform an action was

more effective than reading to evaluate in helping students to improve their own

writing. However, the study used an atypical writing task with a highly concrete

goal, so it is not clear that the results would apply to more typical academic writing

tasks for which more general evaluation criteria are relevant. In addition, the

features of the writing and revisions made were not studied; nor were writers

thought processes as they made the revisions.

Observing readers

Another way students can learn about audience is through observing readers explain

their thinking processes or try to carry out a task based on reading. Observational

learning gives learners the opportunity to reflect upon and gain metacognition about

a task or skill they are learning, which ultimately helps them successfully carry out

this new task or skill (Sonnenschein & Whitehurst, 1984). Observational learning

through modeling consists of four processes: attention, retention, production, and

motivation (Bandura, 1986). In learning through observation, a learner focuses

attention on a model, retains information given by the model, translates the

information into a behavior, and is ultimately motivated to enact the behavior or not

depending on the success of the model. Observation may be more effective than

practice writing in some circumstances because it provides information on the task

and the opportunity for reflection while freeing the learner of the cognitive burdens

of doing the writing task (Rijlaarsdam & Couzijn, 2000; Rijlaarsdam & Van den

Bergh, 2006).


123

Several studies have examined the extent to which a combination of reading

activities (i.e., being a reader, reading a reader’s think aloud transcript, and listening

to a reader think aloud as he reads a text) affects student writing quality (Crasnich &

Lumbelli, 2005, Lumbelli, Paoletti, & Frausin, 1999; Schriver, 1992). These studies

suggest that observation of readers in combination with reading leads middle school

and junior high school writers to improve the clarity of their texts so that readers can

better access them.

Other studies have isolated the effects of observational learning (Couzijn, 1999;

Couzijn & Rijlaarsdam, 2004b). In general, the results show that observation of

readers and/or writers is a more effective instructional method than practicing

writing for improving one’s overall writing quality. In one study (Couzijn &

Rijlaarsdam, 2004a), high school students wrote instructions for a science

experiment, which they revised after participating in one of several experimental

conditions. After writing a first draft of the instructions, some students observed

readers thinking aloud while following their instructions or another students’

instructions; prior to writing instructions for the experiment, other students read the

instructions themselves while thinking aloud; and others self-evaluated their own

writing. Students who observed readers attempt to follow the instructions made

greater gains in the quality of their written directions, and it did not matter whether

the text being read was the writer’s own or another student’s. They also did a better

job providing written advice to other students about how to write science

instructions. This study suggests that observing readers think aloud as they try to

comprehend texts is an effective way for writers to reflect on audience needs and

learn about the characteristics of texts that pose difficulties for readers. However,

like the tangram studies (Holliway & McCutchen, 2004; Traxler & Gernsbacher,

1992, 1994), the writing task asked for concrete responses from readers.

A recent study (Rijlaarsdam & Braaksma, as cited in Rijlaarsdam et al., 2008)

investigated the effects both of experience as a reader and of observing readers using a

more common persuasive writing task. In this study, junior high students wrote

persuasive letters to the Yummy Yummy Candy Company about a special promotion.

A group of students simulating a review board at the company (readers) read several

letters and discussed which one was the most persuasive. Another group simulating

researchers (observers) took notes, discussed the evaluation criteria used by the board,

and presented their findings to the class. Then all students revised their own letters. The

board discussions and the poster presentations clearly demonstrated students’

audience awareness knowledge. Moreover, the results indicated that all letters

showed improvement in overall quality; however, the researchers (observers)

improved more than the review board (readers). It is important to note that the

readers did not just read and the observers did not just observe. The readers were

charged with making a decision based on reasons, and the observers were charged with

analyzing those reasons and presenting them to the class.

The current study

In this study, groups of readers read, discussed, and ranked three persuasive letters

according to overall persuasiveness. The observer groups listened to the reader


123

groups’ discussions and took notes; then they had their own discussion to generate a

list of criteria for what made the letters persuasive. The practice-writing control

group practiced writing persuasive letters. Then all groups revised a letter that they

had written previously on the same topic.

The current study builds on previous research on peer review, being a reader, and

observing readers. Like research on the effects of giving feedback, this study asked

students to analyze and comment on the writing of other students. The study also

separated the effects of being a reader from being an observer and compared both to

a practice writing control condition. The study extends prior research in several

ways. First, the procedures used in this study could be applied to any writing task.

They are not limited to tasks with concrete goals, like the tangram and science

instruction studies, or to a particular story line like the Yummy Yummy study. In

addition, this study not only investigated the immediate effects on revision of the

papers written before the intervention, but also checked for transfer to a similar

writing task the following week. Although Couzijn and Rijlaarsdam’s (2004a, b)

study on instructions for a science experiment included a far transfer task (a letter of

advice about the task), it did not include a transfer to a similar task. In addition, the

study not only documented the overall quality of the writing under various

conditions but also examined the types of revisions made following the intervention.

Furthermore, qualitative data were collected to describe the students’ thinking

during revision and to capture the nature of interactions in the group discussions.

Finally, the study investigated students’ perceptions about the various activities.

The current study used mixed-methods to investigate the extent to which being a

reader and observing readers affected the revisions made by fifth-grade students on

argumentative letters and the overall quality of their writing. We hypothesized that

readers and observers, in comparison to controls who practiced writing, would make

more revisions, particularly revisions to argumentative elements that demand

audience awareness (e.g., counterarguments), and increase the overall quality of

students’ writing. We also examined transfer to a similar writing task a week later

and gathered qualitative data to describe students’ reasons for revision, group

interactions, and perceptions about the activities.

Method

Participants

Eighty-seven fifth-grade students (47 female and 40 male) from five classrooms in

two elementary schools located in the same suburban school district in the

northeastern part of the United States participated in this study. The primary

approach to writing instruction in both schools was writers’ workshop (Calkins,

1986); however, one school’s instruction was also influenced by the six-traits of

writing (Spandel, 2005). Persuasive writing was part of the fifth-grade English

language arts curriculum in both schools. However, none of the students had

experienced persuasive writing instruction in fifth grade prior to the study.


123

The average age of the participants was 10.7 years (see Table 1). The ethnic

backgrounds were as follows: 80.5% Caucasian, 12.6% African-American, 3.4%

Hispanic, and 3.4% Asian-American/Pacific Islander. Socioeconomic status (SES)

data was available at the school and district levels: 42.7 % of fifth-grade students

were identified as low income at the first school and 2.9% at the second school; at

the district level, 46.9% of all fifth graders were identified as low income. Students

with disabilities were excluded from the study. The results of the 2008 statewide

annual testing program were used to determine students’ literacy achievement levels

in reading and writing. The scores on the reading and writing subtests range from 0

to 5; 2.5 represents a passing score on the state standards (Delaware Student Testing

Program, 2008). The average scores were as follows: reading, M = 3.39,

SD = 1.00; writing, M = 2.81; SD = 0.61; scores for three students were not

available.

Experimental design and procedures

The participants were randomly assigned individually within classroom to the

reader, observer, and control groups. Prior to the random assignment, classroom

teachers identified 3 or 6 strong leaders (depending on the class size) in the class

to act as discussion leaders for the groups. These leaders were randomly assigned

to the reader, observer, and control conditions. Next, the remaining students were

randomly assigned so that there was at least one group each of four readers, four

observers, and four control students. Three of the classes were large enough that

two groups of readers, observers, and control students were selected. Twelve

students were randomly selected, one per condition per classroom except for the

smallest class, to participate in think alouds and semi-structured follow-up

interviews.

Table 1 Descriptive data on participants

Variable Group

Reader Observer Control

Gender

Female (N = 47) 16 (51.6%) 19 (67.9%) 12 (42.9%)

Male (N = 40) 15 (48.4%) 9 (32.1%) 16 (57.1%)

Ethnicity

Caucasian (N = 70) 27 (87.1%) 21 (75%) 22 (78.6%)

American Indian/Alaskan Native,

Asian American/Pacific Islander,

Hispanic, Black (N = 17)

4 (12.9%) 7 (25%) 6 (21.4%)

Age M (SD) N = 87 10.6 (0.4) 10.7 (0.65) 10.8 (0.34)

DSTP reading M (SD) N = 84 3.52 (1.06) 3.37 (1.08) 3.29 (0.9)

DSTP writing M (SD) N = 84 2.93 (0.46) 2.78 (0.7) 2.71 (0.66)

The reading and writing test scores were taken from the Delaware State Testing Program (2008)


123

Originally, 32 students were assigned to each condition. Nine students were lost

to the study due to absenteeism for one or more sessions. Complete data were

available for 28 participants in the control group, 28 in the observer group, and 31 in

the reader group. No group of four students lost more than one member and no

student selected for participation in the think-alouds and semi-structured interviews

was lost. Students in the three conditions were compared on reading and writing

scores, age, gender, and demographic data with a p value set at .20; no statistically

significant differences were found.

The research study was carried out over six 45-minute sessions over 2 weeks.

The first four sessions were completed in 1 week. During session I, all students

planned and wrote a persuasive letter in response to prompt A. During session II,

participants in the reader and observer groups met in discussion groups of four, with

an observer group observing each reader group. The reader and observer groups

practiced the experimental activities using narrative texts. Participants in the control

group practiced writing to a narrative prompt. The decision to use narrative texts

during this session was because the primary purpose of this session was to teach

students about the procedures of the study activities (i.e., how to be a reader,

observer, control group participant) rather than to teach students about persuasive

writing. During session III, participants in the reader and observer groups engaged

in the experimental activities using persuasive texts. Participants in the control

group practiced writing to a persuasive prompt. During session IV, all participants

revised the first draft they wrote to prompt A. At this time, think alouds and

retrospective semi-structured interviews were collected from four students in each

condition. During session V, all participants planned and wrote a persuasive letter in

response to Prompt B, which they revised in session VI. Semi-structured interviews

were collected from the same sample of students who participated in the think

alouds and interviews during session IV. The writing and revising activities in

sessions I and IV-VI and the practice writing by control students were conducted in

students’ classrooms. The discussion groups in sessions II and III and the think-

aloud interviews were held in other spaces in the school.

The investigator, assisted by three trained research assistants, conducted the

study. The lead research assistant helped the investigator conduct the experimental

activities for the reader and observer groups and conducted think alouds with

selected students. The remaining two research assistants conducted think alouds

with selected students. All research assistants attended one 45-minute training

session on how to conduct think alouds. The lead research assistant attended an

additional 60-minute training session that gave an overview of the reader and

observer group procedures; she also observed the investigator facilitate the reader

and observer activities during the first week of the study and prior to doing so

independently. All procedures for think alouds and experimental activity facilitation

were scripted.

Session II: Practice experimental activity

The primary purpose of session II was to familiarize the participants in the reader

and observer groups with their roles and the activity. The investigator or lead


123

research assistant introduced the experimental activities, and the reader and

observer groups practiced with narrative texts. The reader groups responded to two

or three narratives written by students similar in age. A discussion guide consisting

of five questions focused the discussion. The observer groups observed the readers’

discussion and used a graphic organizer to take notes about what they heard the

readers say about each narrative. Following the readers’ discussion, the observers

had their own discussion in which they devised a list of criteria about what

characterizes an effective narrative. Throughout the activity, the investigator and

lead research assistant facilitated and monitored discussions to make sure

participants understood their roles in the activity. The control group practiced

writing to a narrative prompt during this session; the investigator-created narrative

writing prompt asked students to write about their favorite season.

Session III: Participation in the experimental activity

During session III, the reader and observer groups participated in the experimental

activities with persuasive texts. The reader group read persuasive letters written by a

group of fifth-grade students in response to prompt A; the students whose letters

were discussed were not participants in the study (see ‘‘Materials’’).

The reader groups were directed to discuss whether each letter was persuasive

and why; rate them as persuasive, somewhat persuasive, or not persuasive; and

choose the most persuasive letter. Each letter was read aloud while students

followed on their own copies. Readers used a discussion guide that contained five

guiding questions to guide their discussion and all discussions were audio-recorded.

The following questions were included on the guide in order to facilitate and focus

discussion; however, students were not limited to discussing these questions:

• Do you think this letter will persuade the teacher? Why or why not?

• Does the writer state his opinion in a strong and clear way? Does this make the

letter better or worse?

• Did the writer give good reasons for his or her opinion? Why are the reasons

good or not so good?

• Did the writer think about reasons why someone may disagree with him or her?

Does this make the letter better or worse?

• Did the writer write to the teacher with an appropriate attitude? Why or why

not?

The observer groups were instructed to listen to the reader groups’ discussions

and figure out what things the readers thought were good or bad about the letters and

what made them persuasive. To this end, the observers listened to the discussions

and took notes. Following their observation, they met to discuss what they observed

and to create a list of the characteristics the readers identified that make writing

more persuasive. To take notes, they used a graphic organizer with space for notes

on each letter and a numbered list of ways to make writing more persuasive. All

observer discussions were audio recorded.

Participants in the control group practiced writing to another persuasive prompt

on a different topic for the duration of the treatment group activities. The persuasive


123

writing prompt was taken from the state testing program: ‘‘Your school cafeteria

manager is thinking about taking ice cream and other sweets off the menu. Write a

letter to the cafeteria manager telling why you agree or disagree with this idea.’’

Materials

Writing prompts

Prompt A and B were selected to require little specific background knowledge and

to represent similar adult audiences (i.e., a teacher and a principal). Both prompts

were adapted from the state testing program.

Prompt A was, ‘‘Your teacher is thinking about giving students more homework

to get them ready for the next grade. Do you think this is a good idea? Write a letter

persuading your teacher to agree with what you think about giving students more

homework.’’

Prompt B was, ‘‘In order for students to have more time in the library, your

principal is thinking of having students attend gym class only once a month. Is this a

good idea? Write a letter convincing your principal to agree with your opinion about

students having gym class only once a month to have more library time.’’

Letters for discussion

The reader group read persuasive letters written by a group of similar fifth-grade

students from a pilot study. The investigator selected and made minor modifications

to the texts to represent a range of writing abilities, argument elements, and tones.

The texts also represented different points of view. The same set of three persuasive

letters was used in all classes.

Measures, scoring procedures, and inter-rater reliability

First and second drafts of the prompt A and B essays were typed double-spaced with

names replaced with codes. A trained rater unfamiliar with the purpose of the study

checked 20% (n = 70) of all of the writing for accuracy against participants’

original handwritten writing. The only errors were five typos that affected spelling

but not word meaning. The essays were analyzed for: (a) overall persuasive quality,

(b) revisions, (c) and inclusion of argumentative elements.

Overall persuasiveness

Overall persuasiveness of the first and second drafts of the Prompt A and B essays

was measured using a 7-point primary trait scoring rubric (Ferretti, MacArthur, &

Dowdy, 2000). The rubric defines persuasive writing as writing that ‘‘attempts to

influence readers to change their thinking or behavior’’ (Ferretti et al., 2000, p. 702).

Two trained raters unfamiliar with the purpose of the study scored all of the papers

independently. Training included discussion of the rubric and two anchor papers for

each score point and practice scoring of sample papers from a pilot study. Training


123

continued to a criterion of 70% exact agreement on a sample of ten papers. Inter-

rater reliability was calculated both as Pearson product moment correlation and

percent agreement within one point. The correlation between raters was r = .70.

Percent agreement within one point was 94%. The scores of the two raters were

averaged and the mean scores were used in the analysis.

Revisions

First and second drafts of essays written to Prompt A were compared and revisions

were coded in the following categories: (a) type: word, phrase, sentence;

(b) operation: add, delete, substitute; (c) purpose: give new information, clarify,

elaborate, organize, emphasize, delete, change tone, no purpose; (d) argument

element: introduction, proposition, reason for proposition, elaboration of proposi-

tion or reason, alternative proposition, reason for alternative proposition, rebuttal,

conclusions; and (e) quality: successful or no change (MacArthur, Schwartz, &

Graham, 1991; Moore & MacArthur, 2008). Revisions were identified using the

compare documents feature in Microsoft Word. Raters had to decide whether a

revision was non-surface or surface and then continue coding all non-surface

revisions for the remaining categories. Three raters unfamiliar with the purpose of

the study attended a 2-h training session that included discussion of codes and

practice scoring papers from the pilot study. Training continued until raters reached

at least 70% agreement in each category when papers were scored independently.

After the training, each rater scored one-third of the sample independently. In

addition, 20% of each rater’s scored papers were also scored by the investigator to

calculate interrater reliability. However, several of the codes for purpose and

argument element were used too infrequently to permit meaningful calculation

of interrater reliability. Low frequencies would also limit analysis. Consequently,

the codes for purpose were collapsed into two codes: information (give new

information, clarify, and elaborate) and other purpose (organize, emphasize, delete,

change tone, no purpose). The codes for argument elements were collapsed into two

codes: proposition elements (proposition, reason for proposition, and elaboration of

proposition or reason) and alternative proposition elements (alternative proposition,

reason(s) for alternative proposition, and rebuttal). Interrater reliabilities (agreement

on occurrences/ [agreements ? disagreements]) were 91% for purpose and 86% for

argument element.

Argument elements

The presence of argument elements in second drafts of both prompts A and B was

measured using a coding scheme developed in previous research (Ferretti et al.,

2000). Papers were rated for the following elements: proposition about the topic,

reason(s) for the proposition, elaboration of the proposition or reason(s), alternative

proposition about the topic, reasons for the alternative proposition, elaborations of

the alternative proposition or reason(s), rebuttals of the reason(s) for the alternative

proposition, conclusions, and non-functional.


123

Three trained raters unfamiliar with the purpose of the study independently coded

the papers after attending three training sessions. During training, the raters

reviewed the codebook and examples and practiced scoring pilot-study papers until

they reached at least 70% exact agreement. After training, each rater independently

scored one-third of the total number of papers. The investigator initially scored 20%

of each rater’s batch of papers. However, the low frequency of units coded for

alternative propositions, reasons for alternative propositions, and rebuttals prevented

reasonable calculation of interrater reliability. It also limited analysis because of the

high frequency of zeros. Therefore, codes were collapsed to form three categories

for analysis: proposition elements (including propositions, reasons for propositions,

and elaboration of propositions), alternative proposition elements (including

alternative propositions, elaborations of alternative propositions, and rebuttals),

and conclusions. All drafts were re-scored by two trained raters for alternative

proposition elements. The interrater reliabilities, calculated as percents exact

agreement were: proposition elements, 88%; alternative proposition elements, 71%;

conclusion, 87%.

Think alouds and semi-structured interview data

All think aloud and semi-structured interview data were transcribed using a format

that designated the difference between writing, thinking, and composing out loud

(Prior, 2004). Triangulation of the several data sources occurred to ensure validity

of the data analysis: the investigator and a trained rater read and coded the think

aloud transcripts, semi-structured interviews, and first and second drafts of writing

in order to confirm observations and draw conclusions.

All data were analyzed using constant comparative analysis (Rossman & Rallis,

2003). The unit of analysis was an idea unit, which consisted of a proposition and

everything related to it. The analysis occurred over several steps. First, the

investigator and a trained rater read through all of the transcripts and papers

associated with the transcripts and noted themes related to the research questions.

The investigator and trained rater discussed their observations as they did the initial

read through of the transcripts. Secondly, the investigator and trained rater began

coding the transcripts for the repeating ideas they noticed. When a code appeared

with great frequency, they coded all remaining transcripts for that code only. They

repeated steps two and three until saturation was reached. Third, another rater was

trained on the definitions of the final codes and given examples of each from the

transcripts. She coded 25% of the think alouds and semi-structured interviews for

prompt A (n = 3) and 25% of the semi-structured interviews for prompt B (n = 3).

The percent exact agreement with the codes was 87% for think alouds and

interviews associated with prompt A and 85% for interviews associated with prompt

B. Finally, the investigator grouped the codes into categories and then into themes

to describe the discourse processes, strategies, thoughts, and understanding of

students in each of the three groups in the study. Groups were compared and

contrasted.


123

Discussions

Reader and observer discussions during the experimental activities were audio-

recorded, transcribed, and analyzed for topics discussed and type of discussion

move. The same procedures for constant comparative analysis used to analyze the

think-aloud data were followed for the discussion analysis.

Results

Overall persuasiveness on prompt A

The quality of the final draft of prompt A was compared across groups using

analysis of covariance with quality of the first draft as a covariate (see Table 2 for

the quality data and the ‘‘Appendix’’ for correlations with other variables). A

significant difference by group was found (F = 5.70, df[2,86], p \ .01). Pairwise

comparisons, using the Bonferroni adjustment, indicated that the reader group had

greater overall quality scores than the control group (effect size, d = .625,

calculated as difference in mean posttest score divided by the pooled standard

deviation). The observer group did not differ significantly from either the reader

group or the control group.

Revision type on prompt A

The three groups were compared on four revision variables including two purposes

(information and other) and two types of argument elements (proposition and

alterative proposition; see Table 3 and the ‘‘Appendix’’). A violation of homoge-

neity of variance prevented the use of multivariate analysis of variance (MANO-

VA). Therefore, separate analyses were conducted for the four revision variables.

The Bonferroni procedure for four simultaneous tests was applied; alpha was set at

.012 (i.e., .05/4).

Purpose for revision

Separate one-way ANOVAs were conducted for the two purpose codes of

information and other purpose. For information, the main ANOVA revealed no

significant differences among groups (F = 1.34, df[2,84], p = .269).

Table 2 Overall persuasive quality ratings for prompt A

Group First draft Second draft Adjusted

M (SD) M (SD) M (SE)

Reader 3.26 (1.0) 3.89 (1.50) 3.74 (0.12)a

Observer 2.96 (0.68) 3.20 (0.60) 3.32 (0.13)a,b

Control 3.05 (0.94) 3.11 (0.89) 3.15 (0.13)b

Adjusted means that do not share subscripts differ significantly at p \ .05


123

For other purpose, the homogeneity assumption underlying ANOVA was not met

and scores were significantly skewed due to a high frequency of zeros. Therefore, as

recommended by Field (2005), the Kruskal-Wallis H test, the nonparametric

equivalent of the one-way ANOVA, was used to assess group differences. There

was no significant difference among groups on revising for other purpose (p = .738).

Revisions to argument elements

Separate analyses were conducted for the two argument elements of proposition and

alternative proposition. For proposition elements, the ANOVA revealed no

significant differences among groups (F = .404, df[2,84], p = .669).

For alternative proposition elements, the homogeneity assumption was not met

and scores were significantly skewed due to a high frequency of zeros, so the non-

parametric Kruskal-Wallis H test was used. There was a significant difference

among groups on revision to alternative propositions (H(2) = 8.52, p \ .01). As

recommended by Field (2005), Mann-Whitney tests were used for follow-up

comparisons between groups; a Bonferroni correction was applied. The reader

group made significantly more revisions affecting alternative propositions than the

control group (p \ .01; effect size, d = .57). The observer group did not differ

significantly from either the reader or the control group.

Argument elements on prompt A

Separate ANOVAs were conducted to assess group differences in inclusion of

argument elements coded as proposition, alternative proposition, and conclusion in

the second draft of prompt A (see Table 4 and the ‘‘Appendix’’). A violation of the

homogeneity assumption prevented the use of MANOVA, so separate analyses were

conducted for the three variables using the Bonferroni correction with alpha set at

.02 (i.e., .05/3).

For proposition elements, the ANOVA revealed no significant difference among

groups, F = 1.30, df[2,84], p = .279.

For alternative proposition elements, the assumption of homogeneity was not met

and the scores were significantly skewed due to a high frequency of zeroes.

Table 3 Number of revisions by purpose and argument elements on prompt A


M (SD) M (SD) M (SD)

Purpose for revising

Information 5.35 (3.66) 4.04 (2.53) 4.86 (2.99)

Other 2.03 (1.87) 1.74 (1.88) 2.39 (3.14)

Argument element revised

Proposition elements 3.74 (2.65) 3.29 (2.32) 3.86 (2.58)

Alternative proposition elements .87 (1.91)a .11 (.31)a,b .07 (.26)b

Means in the same row that do not share subscripts differ significantly at p \ .01


123

Therefore, the Kruskal-Wallis H test was used. There was a significant difference

among groups on inclusion of alternative propositions, (H(2) = 14.06, p \ .001).

Mann-Whitney tests with Bonferroni correction were used to follow up this finding.

The reader group included more alternative proposition elements than the control

group (p \ .001; effect size, d = .67) and more than the observer group (p \ .01;

effect size, d = .63).

For conclusions, the ANOVA was not significant, indicating no difference among

groups on this element, F = 1.5, df (2,84), p = .231.

Effects of treatment on prompt B, transfer task

Both the first and second draft for prompt B were written after the treatment. Thus,

treatment may have affected the quality of the first draft or the improvement in

quality due to revision. For the first drafts, we used an ANCOVA with the first draft

from prompt A as the covariate. No significant difference was found (F = .868,

df[2,86], p = .424). For the final draft, we also used ANCOVA with quality of the

first draft as a covariate (see Table 5). No significant difference was found

(F = .574, df[2,86], p = .565). In addition, analyses of variance showed no

significant differences among groups regarding the presence of argument elements

on the final draft of prompt B. The effects of the treatment did not transfer to a new

persuasive writing task 1 week following instruction.

Think alouds and semi-structured interviews

The think alouds and semi-structured interviews (12 students, 4 from each

condition) were coded for the types of revisions participants made and the rationale

that the participants gave for their revisions. The results support and extend the

Table 4 Argument elements on the final draft of prompt A


M (SD) M (SD) M (SD)

Proposition elements 9.35 (5.14) 7.79 (2.74) 7.96 (4.0)

Alternative proposition elements .87 (1.71)a .07 (.38)b .04 (.19)b,c

Conclusion .45 (.68) .25 (.52) .50 (.51)

Means with the same subscript do not differ significantly. Means with different subscripts differ as

follows: a and b differ at p \ .01; a and c differ at p \ .001

Table 5 Overall persuasive quality ratings for prompt B

Group First draft Second draft Second adjusted

M (SD) M (SD) M (SE)

Reader 3.31 (0.99) 3.35 (1.01) 3.24 (0.09)

Observer 3.20 (0.98) 3.13 (0.95) 3.10 (0.10)

Control 2.98 (0.82) 3.05 (0.82) 3.21 (0.10)


123

quantitative results related to group differences in revision type. Two major themes

emerged from the analysis: there were group differences in audience awareness and

there were group differences in the perception of the activities.

Group differences in audience awareness

Audience awareness was made apparent in several ways. First, students made

revisions or comments to show consideration of how an audience may also benefit

by sharing the same opinion on the topic (e.g., students gave a reason why giving

less homework may benefit the teacher, not just the student). Secondly, students

made revisions or comments about making the tone of the letter more appropriate

for the audience. Finally, students made revisions or comments to show awareness

that an outside audience may have a different point of view (i.e., a

counterargument).

The participants in the reader group showed the most evidence of audience

awareness during the revision phase (i.e., 33 instances). In addition, the evidence of

audience awareness for this group was the most rich and varied. Furthermore, all

readers interviewed showed evidence of thinking about audience in some way.

For example, one reader, John (all names are pseudonyms), was concerned that

his letter might have an inappropriate tone. Therefore, one of his major revisions

was to delete a phrase that he felt sounded like he was yelling at the teacher:

John: [reading]…now I’m gonna change this because I put: [rereading what he

wrote] ‘‘You know what.’’ And that sounds like I’m kinda yelling at him.

It was clear that John became more aware of tone through his participation in the

experimental activity. During the semi-structured interview, which followed the

think aloud, John was asked about what he learned through his participation in the

study activity. He responded:

John: We were talking about persuasive writing. We were talking about what

should be changed in there and this one letter, it was saying [quoting letter]

‘‘[if] you give us more homework [and] stupid’’ and ‘‘if you give us more

homework we will hate school.’’ Everybody said that shouldn’t be persuasive.

I mean it might persuade the teacher to yell at him [the writer] or give him

detention.

Another reader, Nicole, decided that one of her main revisions would be to

include a reason about why less homework would benefit the teacher. Prior to her

participation in the activity, Nicole only included reasons about why less homework

would be beneficial to her or why more homework would be harmful to her.

Nicole: [I’m] adding something: [talking aloud as she is adding a phrase to her

rough draft] ‘‘and the teacher might not get to grade it or will stay at school for

a couple hours.’’

In the semi-structured interview that followed her think aloud, Nicole was asked

about why her letter may persuade the teacher. She responded:


123

Nicole: It [has] a lot of detail for why it’s bad if you give us more homework

and what it could do to affect the students and the teacher.

Through her revision activity and her reflection on her revisions, it is clear that

Nicole is beginning to think about how her opinion may benefit her audience in

addition to herself.

Finally, another reader, Maura, thought about her audience in a different way.

She decided to add a counterargument and a rebuttal to the first draft of her letter.

She began her think aloud by saying:

Maura: Well, somewhere at the end I think. [scanning her writing] Here we go.

I think I might add something about how people [who] don’t agree with me

would feel. And, I said, [summarizing what she already wrote] yes, because it

would prepare us for middle school. [thinking about what she will write] And I

think some…a lot of people wouldn’t agree. So, it will probably be good to

share that point of view because yesterday there were one of the letters didn’t

share the other point of view…and it was okay, but I think if it shared the other

point of view it would have been a little better. So, I think if I added that other

point of view to mine it might be a little better. I’m just gonna write: [writing]

‘‘Even though people might not agree it will help prepare them and in the end

they’ll be glad.’’ I wrote [rereading what she wrote] ‘‘Even the students who

do not agree with me would be more prepared and in the sixth grade will be

glad you gave them more homework.’’

In summary, audience awareness was evident in the reader group’s thoughts in

multiple ways: they thought about tone, they thought about reasons to benefit their

audience, and they thought about counterarguments and rebuttals.

The observer group also showed evidence of audience awareness (i.e., a total of

22 instances), but it was not as rich and varied as the examples found from the

reader group participants. Of the four observers interviewed, only two showed

evidence of audience awareness. In addition, audience awareness for the observer

group focused on tone only. Their revision activity focused primarily on word

choice and clarifying or explaining what they already wrote in their first draft in

order to make their letters make more sense. For example, Timothy changed word

choice to make the tone more appropriate.

Timothy: Right here…where in the beginning where I say: [rereading what he

wrote] ‘‘I think it is a very bad idea to give us a lot more homework.’’ And I

think that…I don’t want to seem mean to the teacher. [summarizing] But like

saying it’s a very bad idea to give us more a lot more homework doesn’t sound

quite right to talk to a teacher with. So I will change it to: [writing] ‘‘I disagree

with the idea that we should have more homework.’’

Like John from the reader group, Timothy was concerned with sounding mean to

his teacher. However, Timothy spent the majority of his revision focusing on word

choice and clarifying his writing in order to make it sound right instead of on

concern for his audience. For example, a typical revision for Timothy sounded like

the following:


123

Timothy: I’m moving this sentence up here to: [rereading what he wrote]

‘‘Finally, I believe that we are being prepared for the next grade throughout

the school year.’’ But I want to change the wording around. I want to say:

[writing] ‘‘Finally I believe that we are being prepared throughout the school

year for the next grade.’’

Thus, this more typical revision illustrates Timothy rearranging two phrases in

the same sentence. Overall, Timothy’s transcript represents a blend of revisions

geared toward making tone more appropriate for the audience and surface revisions.

These are characteristic of the observer group’s revision practices. Whereas

audience awareness is apparent, it is not as strongly emphasized or evident in as

wide a variety as in the reader group revisions.

The control group only had two instances of audience awareness and they were

by the same participant. Instead, the control group transcripts were rife with changes

made because the word choice did not sound right to the writer. For example,

Brittany (who was the same participant that considered her audience twice) started

her think aloud by saying:

Brittany: Well, I already made this one change…’cause I said: [rereading what

she wrote] ‘‘Just give us more homework already.’’ And then I said: [rereading

what she wrote] ‘‘Why? I’m glad you asked.’’ It just…when I read it out loud

it didn’t sound right to me so I thought I’d change it so I put: [rereading what

she wrote] ‘‘I’ll tell you why.’’ Because it just didn’t sound right to me. And I

don’t know what else I’m gonna do yet…. probably read it out loud again to

see if anything sounds weird…’cause I wrote it on Monday. I’ll read it out

loud and see if anything sounds weird. [rereading silently] I don’t like that

sentence but I don’t know how to change it.

Brittany lacks strategies for revising, stating, ‘‘I don’t like that sentence but I

don’t know how to change it.’’ The lack of specific direction for revision apparent in

the control group transcripts may contribute to their tendency to make surface

revisions (i.e., rewording parts of their text) and revisions that do not alter the

meaning or persuasiveness of their letters.

Group differences in perception of activity

Follow-up semi-structured interviews conducted immediately after think alouds

confirmed and clarified students’ think alouds and allowed time for students to

discuss what they thought about the activity. Results show that students from the

reader group were much more enthusiastic about their experience than students in

the observer and control groups and also saw more value in the activity. For

example, Maura from the reader group found value in reading and evaluating

others’ writing:

Maura: When we were going over the writing, I realized that I didn’t have

some of those things [in the letters discussed] and that I should go back and

revise and make sure I added them or made sure that I had them. Yeah and I


123

think that later in other writing pieces, I’ll probably remember to do that other

stuff.

Nicole, another reader, thought the activity was fun because she had the chance

to offer advice to the authors of the letters. She stated: ‘‘It was just fun to see what

stories [letters] they had and how you could fix it.’’

Although the observers found their activity fun and valuable too, stating ideas like

‘‘I thought it was gonna help my writing first of all’’ and ‘‘It was kind of a fun thing to do

I thought,’’ the observers often asked the investigators to have a turn as a reader. In

their interviews, observers stated this again. For example, Owen said, ‘‘I would switch

jobs…so we could each have the experience [of being a reader].’’ In contrast, the

control group did not have much to say about their participation in practice writing

activities. Students in this group were silent or stated that they could not think of an

answer when asked about whether they liked the writing activity. One student did not

even perceive practice writing as an activity to help him learn to write.

Group discussions

Both the reader groups and the observer groups were asked to discuss the

characteristics of effective persuasive texts. The readers were asked to discuss three

persuasive letters, and the observers were asked to discuss the readers’ discussions.

Reader and observer discussions were coded for the evaluation criteria students

identified and the type of discussion move students utilized. Both the reader and

observer groups identified the same criteria with the exception of one criterion. On

the other hand, the reader and observer groups utilized different discussion moves

during their discussion tasks.

Group similarities regarding effective persuasive writing

Both the readers and the observers mentioned evaluation criteria for good writing in

general and for persuasive writing in particular. The evaluation criteria that

appeared with great frequency in the discussions included: considers the audience’s

perspective, written in an appropriate tone, contains details, has a clear focus,

includes an introduction, states a clear opinion, mentions an opposing view, has a

clear organization, includes reasons, includes a conclusion, and uses good word

choice. For example, in one reader group discussion several of these ideas are

discussed:

Jenna: Do you think this letter will persuade the teacher, why or why not?

Ethan: I think it will…well…semi-sort of because if they did…well…they

gave good reasons but they still should have put more details and stuff so…[included reasons]

Maura: I think it kind of would because it was talking about what it would

affect in their [the teacher] lives…not just what it would affect in our lives so

probably [it would] matter more to them [the teacher] then if it were just like

about him or her [audience consideration]


123

Caitlin: I think that it would because they’re…as well as persuading that it

would be good for the students, just like you said, they were telling the

teachers that it would be good for them because it would give them less papers

to grade and I think that the teachers might have not thought about that and

then they’re like, ‘‘Oh, that’s a good point. Maybe we should give less

homework.’’ So I think they have really persuasive reasons. [audience

consideration]

Jenna: I agree with Ethan and Caitlin and Maura…I agree with everyone but

like I think it …there are some things about it they could change and make

better…there are some things that are good about…there are some things that

aren’t good…like they put everything in one paragraph when it should be like

three paragraphs and they don’t have like a closing really or a lead…so they

don’t really like wrap up or anything and so it could persuade the teacher or it

could not persuade the teacher …yeah half and half. [organization]

Similarly, in the observer group evaluation criteria are identified:

Nicole: …let’s see… a little more detail? [includes details]

Owen: Yeah…Sara: lots more …Owen: More good…lots more detail…Sara: maybe less general [includes details]

Nicole: Yeah…Owen: maybe more good manners [written in an appropriate tone]

Despite similarities in criteria discussed, there was one point of differentiation

between the reader and observer group discussion. The readers identified

consideration of opposing viewpoints as one characteristic of strong persuasive

writing, but the observers did not mention this idea. Instead they identified other

criteria for consideration of audience such as providing reasons why the audience

may side with the opinion and using the appropriate tone. The excerpt below

illustrates how one reader group identified and evaluated the idea of including

opposing viewpoints to strengthen an argument:

Zoe: yeah it says [reading] ‘‘some people like you might disagree with me and

say that we need… might learn more if we had more homework…’’

Anna: yeah I think that’s a reason.

Despite this important difference between readers and observers regarding

discussion about counterarguments, readers and observer groups were both

successful in identifying many criteria for effective persuasive writing. Still, as

illustrated in the excerpts above from the reader and observer groups, the process by

which the groups discussed evaluation criteria was much different and will be

discussed in detail next.

Group differences in discussion moves and quality

Overall the readers’ discussions were longer, more elaborate and more specific than

the observers’ discussions. The readers’ discussions included a variety of discussion


123

moves such as critiquing written text, suggesting ways for the writer to improve the

text, and comparing texts with other texts. On the other hand, observers spent most

of their time just listing the evaluation criteria for persuasive writing.

The readers’ discussions were characterized by 12 dominant discussion moves,

which demanded reflection, evaluation, and analysis of the persuasive letters they

were comparing and contrasting. The discussion moves can be organized into

several cateogories: moves related to the discussion of the letters, moves related to

group management, and moves that did not serve a functional purpose. Discussion

moves related specifically to analysis of the letters included: critiquing the letters,

suggesting areas of improvement, comparing letters to other texts or objects,

elaborating an idea or comment, analyzing the effect of the letters on readers, and

analyzing the author’s purpose. Discussion moves related to group management

included: clarifying or explaining a comment made during discussion, correcting

peers’ analysis or evaluation, disagreeing with peers, and keeping peers on task.

Nonfunctional discussion moves, although infrequent, included not responding to or

answering a question because one did not have anything to say or did not

understand, and repeating what someone else already said or what the discussion

question asked because one did not have anything else to add or did not understand.

Several of these discussion moves are illustrated below in an excerpt from one

reader group small-group discussion:

Anna: I think the writer gave good reasons but one of them wasn’t so good,

like falling asleep in class, because I don’t really know much kids that would

fall asleep in class because they would probably get in trouble…I don’t think

that kids would fall asleep in class so I don’t even think that should be in

there…[critiquing]

Reanna: I think that the reasons are good, but some of them really aren’t that

good…[repeating]

Lloyd: well I disagree with what A said about how that…well they might fall

asleep in class …[disagreeing]

Anna: but it’s not very likely [talking over L] [clarifying]

Zoe: wait wait wait….Lloyd? [keeping on track]

Lloyd: I think it because, yeah, if they do stay up til 11:30 they might fall

asleep in class and that might be a problem [clarifying]

In contrast to the reader discussions, observer discussions contained one

discussion move: listing. Discussions typically began with one participant sharing

one of the evaluation criterion on his or her list of observational notes. Next, another

participant would share a different evaluation criteria extracted from the readers’

discussion. There was rarely more than one comment made on a single aspect, or

evaluation criteria, during observer discussion. In addition, there was not much

elaboration or specificity given to each evaluation criteria named. The excerpt

below is from one observer group small group discussion and illustrates the

discussion move evident in this group as well as the difference between the observer

and reader group discussions in terms of elaboration, variation, and specificity:.

Sam: vocabulary…


123

Max: yeah, vocabulary….

Paul: the last one held my interest…Max: you have to be honest…Amanda: you are supposed to be persuasive

Paul: you shouldn’t try to speed through your writing

Sam: good reasons…Paul: you could read them without stopping…Max: let’s do one more

Amanda: good examples. [listing]

In summary, although the readers’ and observers’ discussions focused on the

same topics, they differed in the way in which the topics were discussed.

Through rich discussions, readers grappled with understanding what aspects of

writing contributed to a text’s persuasiveness. In contrast, although observers

were asked to discuss what they had observed and generate a list of evaluation

criteria, they engaged in little discussion beyond listing or recounting what they

heard. This difference helps to explain the differences obtained from the

quantitative analysis of writing quality, revision type, and inclusion of argument

elements.

Discussion

The overall purpose of the current study was to investigate the effects of reader and

observational learning activities in comparison to practice writing activities on fifth-

grade students’ persuasive writing. Writing quality after revision was significantly

higher for students in the reader group than for students in the control group. The

observers did not differ significantly from either the readers or the control group.

However, on transfer papers written 1-week after the instructional activities, the

three groups did not differ on writing quality.

Analyses of revisions and argumentative elements included in final drafts found

differences between the reader and control groups in inclusion of persuasive text

elements that are markers of audience awareness. Readers made significantly more

revisions related to alternative proposition elements than control students. Also,

readers included significantly more alternative proposition elements in their revised

papers than observers and control students. On the other hand, no significant

differences among groups were found on purpose for revising or inclusion of

propositions.

Qualitative analyses shed additional light on the quantitative findings. Think-

aloud protocols during revision showed that students in the reader groups made

many of their revisions based on their goals to communicate with and convince the

audience. Few students in the observer or control groups commented on audience

issues as they were revising their drafts or during the semi-structured interviews that

followed the think alouds. Moreover, analyses of student semi-structured interviews

also illustrated that the reader group had a more positive perception of their


123

participation in the reader activities than the observer or control groups. Finally, an

analysis of reader and observer discussions illustrated that readers engaged in much

richer and more elaborated discussions than the observers and extracted one

additional criterion, use of counterarguments, which the observers missed.

The quantitative and qualitative analyses offer a rich, in-depth look at how

students’ writing and revising are affected by participation in reader and observer

activities as well as information on students’ cognitive processes during revision.

The combination of quantitative and qualitative analyses allows a comprehensive

evaluation of students’ writing performance; analyses illustrate differences in

students’ written products as well as differences in students’ writing processes.

Therefore, this study not only offers insight into what changes students make as the

result of instruction, but why and how they make these changes. The results of the

study provide important information for both practitioners and researchers in the

areas of theory, practice, and future study.

Implications for theory

The finding that readers were able to write better quality papers than the control

group is consistent with previous research (e.g., Holliway & McCutchen, 2004;

Traxler & Gernsbacher, 1992, 1993). As previous studies show, readers can take

many stances when reading. Therefore, it is important to note that the readers’ role

in this study was to rate or rank texts according to overall persuasiveness. Readers

were acting primarily as raters rather than as readers who would be persuaded to

take some action or change their opinions. Skilled writers evaluate their own writing

in terms of evaluation criteria appropriate for the writing task (MacArthur, in press).

Through reading other students’ persuasive letters and discussing whether and why

they were persuasive, the readers in this study learned evaluation criteria that they

could use to revise and improve the overall quality of their writing.

The current study extends previous findings through quantitative analysis of the

types of revisions made and qualitative analysis of students’ explanations of the

reasons for their revisions. Analysis of revisions showed that readers made more

revisions than controls in argumentative elements specifically geared towards

accommodating the perspectives of the audience (i.e., counterarguments and

rebuttals). Analysis of the think-aloud data and interviews showed that readers

demonstrated greater audience awareness, as revealed in comments about reasons

that would appeal to the audience, appropriate tone for addressing the audience, and

potential opposing positions.

In addition, whereas the previous research on being a reader used a writing task

with a concrete goal (i.e., describe a tangram so that a reader can identify it;

Holliway & McCutchen, 2004; Traxler & Gernsbacher, 1992, 1993), the current

study used a common school writing task, a persuasive letter. Therefore, the findings

suggest that reader activities are an effective way for upper elementary school-age

students to develop knowledge about audience specifically and quality writing in

general, which can be applied during revision to improve the overall quality of their

writing.


123

The finding that the observer group did not improve significantly over the reader

or control groups on any of the measures is inconsistent with previous research on

observational learning (Rijlaarsdam et al., 2004; Sonnenschein & Whitehurst,

1984). One explanation for this finding is that previous research on observation of

readers as a way to learn about writing has been conducted with junior high school

students rather than upper elementary school students. Age may be a factor in

observational learning.

Another explanation may be that the observational learning activities in

previous research involved more analysis. For example, in a closely related

study, Rijlaarsdam et al. (2008) asked observers to listen as readers discussed a

set of persuasive letters, to analyze the evaluation criteria that readers used, and

to make a poster and presentation to their class about what they had learned. In

the current study, students listened as readers discussed papers, discussed their

observations, and created a list of criteria for effective persuasive letters, but

they did considerably less analysis of evaluation criteria than in the previous

study. In fact, the qualitative analyses of the reader and observer group

discussions shows that the readers were more actively engaged in discussing

evaluation criteria. Readers were asked to read and evaluate each piece of

writing, whereas observers were asked to discuss what they heard the readers

say. Readers’ discussions were elaborate and specific to the texts, whereas

observers’ discussions were list-like and general. The analysis of observers’

discussions showed that they did not extract counterarguments as an evaluation

criterion. Therefore, it is not surprising that they did not include this element in

their revisions. The effects of reader and observer activities may depend on the

cognitive activities involved.

It is also important to consider why there were no group differences in

writing quality or inclusion of argument elements on the transfer writing task.

The most probable explanation for this finding is the short duration of the

intervention. The reader and observer activities lasted only 45 min on one day

and students had no prior instruction in persuasive writing during fifth grade.

Although Couzijn and Rijlaarsdam (2004a, b) did find transfer effects in their

short intervention study, the transfer task in their study was much different than

the transfer task in the current study. Couzijn and Rijlaarsdam’s (2004a, b)

transfer task required students to write an advice letter for how to write an

effective instructional text; on the other hand, the transfer task in the current

study required students to apply what they learned in writing their own

persuasive texts. Future studies should implement reader and observer activities

for an extended period of time to determine whether time would lead to greater

transfer. In addition, future studies should examine the effects of instruction on

different types of transfer activities.

Implications for practice

The study was carried out using typical academic writing tasks in small groups that

would be feasible in regular classrooms. Past research on the effects of being a

reader or observing readers has been conducted in artificial settings (i.e., researcher


123

and participants work one-on-one) and with concrete writing tasks of limited

generalizability such as describing tangrams (Holliway & McCutchen, 2004) or

instructions for a science experiment (Couzijn, 1999; Couzijn & Rijlaarsdam,

2004b). The design of the current study was in line with Rijlaarsdam et al. (2008) in

its goal of finding ways to study reader and observer activities in a real classroom

setting. Although the readers did write more persuasive essays than both the

observer and control groups, it is important to note that the readers’ essays did not

score very highly on the overall quality rubric in general. The average score for a

revised persuasive essay was a 3.89 on a 7-point scale. In addition, it is important to

note that not all readers were able to include argumentative elements geared towards

audience awareness after revision. Thus, the results imply that some learners may

require more extended or more explicit instruction in using evaluation criteria

effectively.

Implications for future research

Reader and observer activities hold promise as effective instructional methods for

teaching writing. However, future research is needed to understand the benefits and

challenges of studying these types of activities and using these methods. Future

research should use larger samples to permit nested analysis and examination of the

role of group discussion. In addition, it is necessary that future research

acknowledge that reader and observer activities can vary on many dimensions.

One particularly important consideration in designing activities is the nature of the

cognitive and social requirements. Reading to perform a task, such as identifying an

object from a description or carrying out a science experiment from written

instructions, is different from reading to evaluate as in peer review. The nature of

the analytic task set for readers or observers is critical to the effects, as shown in the

contrast between the results of the current study and a related previous study

(Rijlaarsdam et al., 2008). The social interaction in groups is also important in

stimulating engagement and developing understanding.

In conclusion, the present study addressed the call for more research on

effective writing instructional practices for adolescents (Graham & Perin, 2007).

Instruction must address both the social and cognitive complexities in writing

and it must hold teachers and students to high standards. Despite the difficulties

inherent in designing effective writing instruction, literacy scholars are respon-

sible for meeting the challenge. Adolescents deserve carefully designed

instruction to prepare them to write clearly and logically for a range of

purposes and audiences so that they can be successful students, workers, and

citizens of a democratic society.

Appendix

See Table 6.


123

Tab

le6

Co

rrel

atio

nm

atri

x

Var

iab

le1

23

45

67

89

10

11

12

13

14

1.

Qu

alit

yA

11

2.

Qu

alit

yA

2.7

72*

*1

3.

Qu

alit

yB

1.5

33*

*.5

64*

*1

4.

Qu

alit

yB

2.5

32*

*.6

17*

*.8

38*

*1

5.

To

tal

rev

isio

ns

.02

1-

.08

7.1

60

.10

71

6.

Rev

ise

for

info

.-

.18

8-

.15

8.0

86

-.0

06

.74

4*

*1

7.

Rev

ise

for

oth

er.1

43

.03

7.2

35*

.14

7.7

45*

*.2

89*

*1

8.

Rev

ise

pro

p.

elem

.-

.07

6-

.15

1.0

57

.04

1.8

69*

*.8

57*

*.3

22*

*1

9.

Rev

ise

Alt

.p

rop

.

elem

.

.13

4-

.01

9-

.03

4.0

58

.43

0*

*.0

68

.14

8.4

86*

*1

10

.A

rgu

men

tp

rop

.

elem

.

.48

9*

*.6

24*

*.3

60*

*.4

39*

*.1

17

-.0

80

.12

0.0

57

.29

9*

*1

11

.A

rgu

men

tal

t.el

em.0

67

.28

8*

*.2

19*

.33

5*

*-

.02

5-

.00

7-

.03

3-

.00

4-

.05

5-

.00

11

12

.A

rgu

men

t

con

clu

sio

ns

.01

8.0

13

.17

9.1

50

.03

3.0

09

.06

7.0

00

-.0

40

.08

4-

.08

81

13

.D

ST

Pre

adin

g.1

03

.07

3.2

37*

.15

6.1

42

.05

8.1

72

.06

7.1

37

.07

4-

.08

8.0

84

1


123

Tab

le6

con

tin

ued

Var

iab

le1

23

45

67

89

10

11

12

13

14

14

.D

ST

Pw

riti

ng

-.0

75

-.0

32

.15

5.1

25

.26

0*

.10

2.2

46*

.17

6.1

61

-.0

54

.09

1.0

86

.45

7*

*1

n=

87

exce

pt

for

cell

sre

late

dto

the

DS

TP

sco

res

wh

ere

n=

84

**

Corr

elat

ion

issi

gn

ifica

nt

atth

e0

.01

lev

el(2

-tai

led

)

*C

orr

elat

ion

issi

gn

ifica

nt

atth

e0

.05

lev

el(2

-tai

led

)

Fu

llv

aria

ble

nam

esb

yn

um

ber

1.

Qu

alit

yra

ting

,p

rom

pt

A,

dra

ft1

2.

Qu

alit

yra

ting

,p

rom

pt

A,

dra

ft2

3.

Qu

alit

yra

ting

,p

rom

pt

B,

dra

ft1

4.

Qu

alit

yra

ting

,p

rom

pt

B,

dra

ft2

5.

To

tal

revis

ion

s

6.

Pu

rpo

sefo

rre

vis

ing

,p

rom

pt

A:

info

rmat

ion

7.

Pu

rpo

sefo

rre

vis

ing

,p

rom

pt

A:

oth

er

8.

Arg

um

ent

elem

ent

revis

ed,

pro

mp

tA

:p

rop

osi

tio

nel

emen

ts

9.

Arg

um

ent

elem

ent

revis

ed,

pro

mp

tA

:al

tern

ativ

ep

rop

osi

tio

nel

emen

ts

10.

Arg

um

ent

elem

ents

incl

uded

inpro

mpt

A:

pro

posi

tion

elem

ents

11.

Arg

um

ent

elem

ents

incl

uded

inpro

mpt

A:

alte

rnat

ive

pro

posi

tion

elem

ents

12.

Arg

um

ent

elem

ents

incl

uded

inpro

mpt

A:

concl

usi

ons

13

.D

elaw

are

stat

ete

stin

gp

rog

ram

(DS

TP

)re

adin

g

14

.D

elaw

are

stat

ete

stin

gp

rog

ram

(DS

TP

)w

riti

ng


123

References

Auriac-Peyronnet, E. (2001). The impact of oral training on argumentative texts produced by ten- and

eleven-year old children: Exploring the relation between narration and argumentation. EuropeanJournal of Psychology of Education, 16, 299–317.

Bandura, A. (1986). Social foundations of thought and action: A social cognitive theory. Englewood

Cliffs, NJ: Prentice-Hall.

Boscolo, P., & Ascorti, K. (2004). Effects of collaborative revision on children’s ability to write

understandable narrative texts. In L. Allal, L. Chanquoy, & P. Largy (Eds.), Revision: Cognitive andinstructional processes (pp. 157–170). Boston, MA: Kluwer.

Calkins, L. M. (1986). The art of teaching writing. Portsmouth, NH: Heinemann.

Chambliss, M. J. (2001). Analyzing science textbooks materials to determine how ‘‘persuasive’’ they are.

Theory into Practice, 40, 255–264.

Cho, K., & MacArthur, C. (2011). Learning by reviewing. Journal of Educational Psychology, 103, 73–84.

Clark, R. A., & Delia, J. G. (1977). Cognitive complexity, social perspective-taking and functional

persuasive skills in second- to ninth-grade children. Human Communication Research, 3, 128–134.

Coirier, P., Andriessen, J., & Chanquoy, L. (1999). From planning to translating: The specificity of

argumentative writing. In J. Andriessen & P. Coirier (Eds.), Foundation of argumentative textprocessing (pp. 1–28). Amsterdam, Netherlands: Amsterdam University Press.

Couzijn, M. (1999). Learning to write by observation of writing and reading processes: Effects on

learning and transfer. Learning and Instruction, 9, 109–142.

Couzijn, M., & Rijlaarsdam, G. (2004a). Learning to read and write argumentative text by observation of

peer learners. In G. Rijlaarsdam, H. Van den Bergh, & M. Couzijn (Eds.), Studies in writing: Vol.14. Effective learning and teaching of writing. A handbook of writing in education (2nd ed.,

pp. 241–258). Boston, MA: Kluwer.

Couzijn, M., & Rijlaarsdam, G. (2004b). Learning to write instructive texts by reader observation and

written feedback. In G. Rijlaarsdam, H. Van den Bergh, & M. Couzijn (Eds.), Studies in writing:Vol. 14. Effective learning and teaching of writing. A handbook of writing in education (2nd ed.,

pp. 209–240). Boston, MA: Kluwer Academic Publishers.

Crasnich, S., & Lumbelli, L. (2005). Improving argumentative writing by fostering argumentative speech.

In G. Rijlaarsdam, H. Van den Bergh, & M. Couzijn (Eds.), Studies in writing: Vol. 14. Effectivelearning and teaching of writing (2nd ed., pp. 181–197). Boston, MA: Kluwer.

Crowhurst, M. (1990). Teaching and learning the writing of persuasive/argumentative discourse.

Canadian Journal of Education, 15, 348–359.

Delaware State Testing Program (2008). Delaware comprehensive assessment portal. Retrieved from

http://www.doe.k12.de.us/aab/default.shtml.

Felton, M., & Kuhn, D. (2001). The development of argumentative discourse skill. Discourse Processes,32, 135–153.

Ferretti, R. P., MacArthur, C. A., & Dowdy, N. S. (2000). The effects of an elaborated goal on the

persuasive writing of students with learning disabilities and their normally achieving peers. Journalof Educational Psychology, 92, 694–702.

Field, A. (2005). Discovering statistics using SPSS (2nd ed.). London: Sage.

Flower, L. S., & Hayes, J. R. (1980). The dynamics of composing: Making plans and juggling constraints.

In L. Gregg & E. R. Steinberg (Eds.), Cognitive processes in writing (pp. 31–50). Hillsdale, NJ:

Lawrence Erlbaum.

Flower, L. S., & Hayes, J. R. (1981). Plans that guide the composing process. In C. H. Frederiksen & J.

F. Dominic (Eds.), Writing: Vol. 2. The nature, development, and teaching of written composition(pp. 39–58). Hillsdale, NJ: Erlbaum.

Golder, C., & Coirier, P. (1996). The production and recognition of typological argumentative text

markers. Argumentation, 10, 271–295.

Graham, S., & Perin, D. (2007). Writing next: Effective strategies to improve writing of adolescents inmiddle and high school. Washington, DC: Alliance for Excellent Education.

Hayes, J. R. (1996). A new framework for understanding cognition and affect in writing. In C. M. Levy &

S. Ransdell (Eds.), The science of writing (pp. 1–27). Mahwah, NJ: Erlbaum.

Holliway, R. D., & McCutchen, D. (2004). Audience perspective in young writers’ composing and

revising. In L. Allal, L. Chanquoy, & P. Largy (Eds.), Revision: Cognitive and instructionalprocesses (pp. 157–170). Boston, MA: Kluwer.


123

http://www.doe.k12.de.us/aab/default.shtml

Knudson, R. E. (1994). An analysis of persuasive discourse: Learning how to take a stand. DiscourseProcesses, 18, 211–230.

Lumbelli, L., Paoletti, G., & Frausin, T. (1999). Improving the ability to detect comprehension problems:

From revising to writing. Learning and Instruction, 9, 143–166.

Lundstrom, K., & Baker, W. (2009). To give is better than to receive: The benefits of peer review to the

reviewers’ own writing. Journal of Second Language Writing, 18, 30–43.

MacArthur, C. A. (in press). Evaluation and revision processes in writing. In V. W. Berninger, (Ed.), Past,present, and future contributions of cognitive writing research to cognitive psychology. Psychology

Press.

MacArthur, C. A., Ferretti, R. P., & Okolo, C. M. (2002). On defending controversial viewpoints: Debates

of sixth graders about the desirability of early 20th-century American immigration. LearningDisabilities Research and Practice, 17, 160–172.

MacArthur, C. A., Schwartz, S. S., & Graham, S. (1991). Effects of a reciprocal peer revision strategy in

special education classrooms. Learning Disabilities Research and Practice, 6, 201–210.

Midgette, E., Haria, P., & MacArthur, C. A. (2005). The effects of content and audience awareness goals

for revision on the persuasive essays of fifth- and eighth-grade students. Reading and Writing: AnInterdisciplinary Journal, 21, 131–151.

Moore, N., & MacArthur, C. A. (2008, February). The role of automated essay scoring in writingclassrooms. Paper presented at Writing Research Across Borders Conference, Santa Barbara, CA.

National Commission on Writing. (2003). The neglected R: The need for a writing revolution. Retrieved

May 5, 2008 from http://writingcommission.org/report.html.

National Commission on Writing. (2004). Writing: A ticket to work…or a ticket out: A survey of businessleaders. Retrieved May 5, 2008 from http://www.writingcommission.org/report/html.

National Commission on Writing (2005). Writing: A powerful message from state government. Retrieved

May 5, 2008 from http://www.writingcommission.org/report/html.

Nystrand, M. (2006). The social and historical context for writing research. In C. A. MacArthur, S.

Graham, & J. Fitzgerald (Eds.), Handbook of writing research (pp. 11–27). New York, New York:

Guilford Press.

Oostdam, R. (2004). Assessment of argumentative writing. In G. Rijlaarsdam, H. Van den Bergh, & M.

Couzijn (Eds.), Effective learning and teaching of writing (2nd ed., pp. 427–442). New York: NY:

Kluwer.

Persky, H. R., Daane, M. C., & Jin, Y. (2003). The nation’s report card: Writing 2002. (NCES 2003-529).

U.S. Department of Education. Institute of Education Sciences. National Center for Education

Statistics. Washington, DC: Government Printing Office.

Pontecorvo, C., & Girardet, H. (1993). Arguing and reasoning in understanding historical topics.

Cognition and Instruction, 11, 365–395.

Prior, P. (2004). Tracing process: How texts come into being. In C. Bazerman & P. Prior (Eds.), Whatwriting does and how it does it: An introduction to analyzing texts and textual practices. Mahwah,

NJ: Erlbaum.

Prior, P. (2006). A sociocultural theory of writing. In C. A. MacArthur, S. Graham, & J. Fitzgerald (Eds.),

Handbook of writing research (pp. 54–66). New York, NY: Guilford Press.

Pritchard, R. J., & Honeycutt, R. L. (2006). The process approach to writing instruction: Examining its

effectiveness. In C. A. MacArthur, S. Graham, & J. Fitzgerald (Eds.), Handbook of writing research(pp. 275–290). New York: Guilford.

Rijlaarsdam, G., Braaksma, M., Couzijn, M., Janssen, T., Raedts, M., Van Steendam, E., et al. (2008).

Observation of peers in learning to write, practice and research. Journal of Writing Research, 1,

53–83.

Rijlaarsdam, G., & Couzijn, M. (2000). Writing and learning-to-write. A double challenge. In R. Simons,

J. Van der Linden, & T. Duffy (Eds.), New learning (pp. 157–190). Dordrecht, Netherlands: Kluwer.

Rijlaarsdam, G., Couzijn, M., & Van den Bergh, H. (2004). The study of revision as a writing process and

as a learning-to-write process. In L. Allal, L. Chanquoy, & P. Largy (Eds.), Revision: Cognitive andinstructional processes (pp. 1–8). Norwell, MA: Kluwer.

Rijlaarsdam, G., & Van den Bergh, H. (2006). Writing process theory: A functional dynamic approach. In

C. A. MacArthur, S. Graham, & J. Fitzgerald (Eds.), Handbook of writing research (pp. 41–53).

New York, NY: The Guilford Press.

Rossman, G., & Rallis, S. F. (2003). Learning in the field: An introduction to qualitative research.

Thousand Oaks, CA: Sage.


123

http://writingcommission.org/report.html

http://www.writingcommission.org/report/html

http://www.writingcommission.org/report/html

Schriver, K. A. (1992). Teaching writers to anticipate readers’ needs: A classroom-evaluated pedagogy.

Written Communication, 9, 179–208.

Shanahan, T. (2004). Overcoming the dominance of communication: Writing to think and to learn. In T.

L. Jetton & J. A. Dole (Eds.), Adolescent literacy research and practice (pp. 59–73). New York:

Guilford.

Sonnenschein, S., & Whitehurst, G. J. (1984). Developing referential communication: A hierarchy of

skills. Child Development, 55, 1936–1945.

Spandel, V. (2005). Creating writers through 6-trait writing: Assessment and instruction (5th ed.). New

York, NY: Pearson.

Sperling, M., & Freedman, S. W. (2001). Review of writing research. In V. Richardson (Ed.), Handbookof research on teaching (4th ed., pp. 370–389). Washington, DC: American Educational Research

Association.

Traxler, M., & Gernsbacher, M. (1992). Improving written communication through minimal feedback.

Language and cognitive process, 7, 1–22.

Traxler, M., & Gernsbacher, M. (1993). Improving written communication through perspective taking.

Language and cognitive process, 8, 311–334.


123

the effects of being a reader and of observing readers on fifth-grade students’ argumentative...

Documents