Cognitive activities in complex science text and diagrams

Jennifer G. Cromley a,*, Lindsey E. Snyder-Hogan a, Ulana A. Luciw-Dubas b

a Temple University, 1301 Cecil B. Moore Ave., RA201, Philadelphia, PA 19122, United StatesbNational Board of Medical Examiners, 3750 Market St., Philadelphia, PA 19104, United States

a r t i c l e i n f o

Article history:Available online 31 October 2009

Keywords:ComprehensionDiagramsStrategy useKnowledge levelInference

a b s t r a c t

Ainsworth’s (2006) DeFT framework posits that different representations may lead learners to use differ-ent strategies. We wanted to investigate whether students use different strategies, and more broadly, dif-ferent cognitive activities in diagrams vs. in running text. In order to do so, we collected think-aloudprotocol and other measures from 91 beginning biology majors reading an 8-page passage from theirown textbook which included seven complex diagrams. We coded the protocols for a wide range of cog-nitive activities, including strategy use, inference, background knowledge, vocabulary, and word reading.Comparisons of verbalizations while reading running text vs. reading diagrams showed that high-levelcognitive activities—inferences and high-level strategy use—were used a higher proportion of the timewhen comprehending diagrams compared to when reading text. However, in running text vs. diagramsparticipants used a wider range of different individual cognitive activities (e.g., more different types ofinferences). Our results suggest that instructors might consider teaching students how to draw inferencesin both text and diagrams. They also show an interesting paradox that warrants further research—stu-dents often skipped over or superficially skimmed diagrams, but when they did read the diagrams theyengaged in more high-level cognitive activity.

! 2010 Elsevier Inc. All rights reserved.

1. Introduction

Science textbooks include a wide range of images, including linediagrams, naturalistic drawings, flow charts, chemical diagrams,and hybrid diagrams (e.g., a photograph with a schematic diagram;Pozzer & Roth, 2003). Comprehending these visual representationsis particularly important for student learning from science texts(Otero, León, & Graesser, 2002), but what are the cognitive pro-cesses involved in comprehending visual vs. textual representa-tions? Ainsworth’s (2006) DeFT (designs, functions, tasks)framework describes effective student learning from multiple rep-resentations such as running text and diagrams. The design aspectof her framework concerns the number and form of representa-tions in a text or learning environment. The pedagogical functionsaspect of her framework concerns the way that similarities and dif-ferences among representations can foster different types of cogni-tive processes. The tasks aspect concerns the demands thatmultiple representations make on learners to abstract from repre-sentation, transfer learning from one representation to others, andto relate between representations. In the current research, we spe-cifically investigate the functions aspect of Ainsworth’s framework

by analyzing think-aloud protocols collected from 91 beginningbiology majors reading from a chapter in their course textbookand coding for cognitive processes verbalized while reading run-ning text vs. cognitive processes verbalized while readingdiagrams.

2. Visual representations and learning from illustrated text

Despite the saying that ‘‘a picture is worth a thousand words,”research on whether text or diagrams are better for learning hasshown very mixed results—sometimes there is an advantage fordiagrams (Chi, Feltovich, & Glaser, 1981; Kriz & Hegarty, 2007),while other studies show that students have a great deal of diffi-culty comprehending diagrams (Bodemer, Ploetzner, Bruchmüller,& Häcker, 2005; Graesser, Lu, Olde, Cooper-Pye, & Whitten, 2005;Hannus & Hyönä, 1999; Hegarty & Just, 1993; Mayer, 2005; Paas,Renkl, & Sweller, 2003). Several recent studies suggest that bettercomprehension of the visuals in illustrated text is associated withbetter overall comprehension of the text. More proficient learnersspendmore time studying visual representations compared to less-proficient learners (Schwonke, Berthold, & Renkl, 2009). Eye track-ing studies suggest that better comprehension of diagrams is asso-ciated with more time spent in relevant regions of the diagram andless time spent in irrelevant regions, including seductive details(Canham & Hegarty, in press; Sanchez & Wiley, 2006). In general,however, most learners spend the majority of their time in running

0361-476X/$ - see front matter ! 2010 Elsevier Inc. All rights reserved.doi:10.1016/j.cedpsych.2009.10.002

* Corresponding author. Address: Department of Psychological Studies in Edu-cation, Temple University, Ritter Annex 201, 1301 Cecil B. Moore Avenue,Philadelphia, PA 19122-6091, United States. Fax: +1 (215) 204 60134.

E-mail address: jcromley@temple.edu (J.G. Cromley).

Contemporary Educational Psychology 35 (2010) 59–74

Contents lists available at ScienceDirect

Contemporary Educational Psychology

journal homepage: www.elsevier .com/locate /cedpsych

text and little time inspecting diagrams (Schmidt-Weigand,Kohnert, & Glowalla, in press; Schwonke et al., 2009). The currentliterature does not offer evidence about why this is the case,although we can speculate about a few reasons—learners mightbe appropriately adjusting their reading times based on their levelof prior knowledge or diagram comprehension skills, learnersmight be focusing on the representation with which they feel mostcomfortable, or learners may skim or skip over diagrams becausethey do not appreciate—or perhaps do not know about—the roleof and importance of diagrams for comprehending science texts(Schwonke et al., 2009).

3. Ainsworth’s DeFT framework: functions of multiplerepresentations

Ainsworth’s (2006) DeFT framework is a comprehensive synthe-sis of the literature on learning with multiple representations, withmuch of the research conducted in computer-based environments.This complex framework includes the effects of learning environ-ment design (e.g., written captions vs. narrated captions), functionsof representations (e.g., to complement each other, to constraininterpretations, or to foster deeper understanding), and the taskdemands of multiple representations (building abstractions,extending knowledge to new representations, and understandingrelations between representations). Multiple representations canhave many functions, and Ainsworth specifically hypothesizes that‘‘Different forms of representation can encourage learners to usemore or less effective strategies” (p. 188). For example, in her ownresearch, Ainsworth (Ainsworth & Loizou, 2003) found more moni-toringstatementsexpressingunderstanding in text than indiagramsandmore principle-based explanations in diagrams than in text.Weinterpret strategies to be consistent with Samuelstuen and Braten’s(2007) definition as ‘‘forms of procedural knowledge that studentsvoluntarily use for acquiring, organizing or transforming informa-tion, as well as for reflecting upon and guiding their own learning,in order to reduce a perceived discrepancy between a desiredoutcome and their current state of understanding” (p. 352).Although some authors have used the term strategies to encompassinferences,weseparate out these twodifferent cognitiveactivities inour analyses (for other definitions, see our coding schemeonpp. 17–18). In the context of the present study, Ainsworth’s (2006) frame-work would predict that the strategies used in running text woulddiffer from the strategies used in a diagram such as the one shownin Fig. 1. Based on the two studies cited by Ainsworth, running textvs. diagrams might lead to the use of qualitatively different sets ofstrategies, or might lead differentially to the use of strategies thatare more effective for learning.

In the present study, we wished to not only test Ainsworth’s(2006) claim about strategies, but also to expand it to cognitiveactivities generally. The range of cognitive activities seen inthink-aloud studies include inference, cognitive and metacognitivestrategies, verbalization of knowledge, vocabulary, and word-read-ing difficulties (Fox, 2009). We hypothesized that, based on thestudies cited by Ainsworth, diagrams vs. running text might leadto the use of qualitatively different sets of cognitive activitiesbroadly (e.g., different types of inference), or might lead differen-tially to the use of cognitive activities that are more effective forlearning (e.g., more use of elaborative inferences).

4. Diagrams in science text

The diagram shown in Fig. 1 is typical of undergraduate biologytext: it includes a stylized representation of microscopic struc-tures—cells and cell parts—and microscopic processes. It featuresa lengthy caption, labels that name different parts in the diagram,arrows, and explanations of various processes. Line diagrams in thetextbook used in the present study had a mean of 4.3 such featuresper image, with the most frequent features being captions, naminglabels, explanatory labels, arrows, and color coding. Comprehend-ing a diagram such as the one shown in Fig. 1 requires numerouscognitive activities, such as activating prior knowledge abouthow pathogens are handled by the body; inferential processes suchas noticing similarities between the two halves of the diagram(both show a T cell reacting to a pathogen and an MHC moleculebringing a foreign antigen to the cell surface) and noticing differ-ences between the two halves of the diagram (infected cell, cyto-toxic T cells, and Class I MHC molecules on the left andmacrophage, helper T cells, and Class II MHC molecules on theright); knowledge of diagrammatic conventions such as thecaption, use of color, arrows, abbreviations, and lettering and num-bering systems; knowledge of specialized scientific vocabulary;and integration of all of this knowledge and cognitive processes.

Previous research on students’ reasoning with representationshas used simplified images, typically with only one (arrows; Butch-er, 2006; naming labels; Berthold & Renkl, 2009; Grosse & Renkl,2006; Schnotz & Bannert, 2003; numbering; Bodemer & Faust,2006) or two (arrows and words; Löhner, van Joolingen, Savels-bergh, & van Hout-Wolters, 2005; caption and abbreviations; Carl-son, Chandler, & Sweller, 2003; caption and labels; Lewalter 2003;symbols and labels; Lowe, 2003) of these features. Studies byMayer and colleagues on multimedia representations have usedthe most complex representations, which include captions, arrows,and shading (Mayer, 2003; Mayer, Hegarty, Mayer, & Campbell,2005). In summary, studies have not been conducted on the kinds

Fig. 1. Sample figure (Campbell & Reece, 2001, p. 910 !2001).

60 J.G. Cromley et al. / Contemporary Educational Psychology 35 (2010) 59–74

of highly complex diagrams that students actually encounter asthey try to master science content at the undergraduate level.

In addition to the use of simplified representations in prior re-search, most studies have been focused on the process of integrat-ing or coordinating information presented in running text withvisual representations (Bartholomé & Bromme, 2009; Florax & Plo-etzner, in press; Hegarty, Kriz, & Cate, 2003). However, in order tounderstand how readers coordinate these different types of repre-sentations, it is critical to understand what cognitive processes aresimilar and different across those representations. As Ainsworth(2006) states ‘‘Learning to use [Multiple External Representations]requires learners to understand each individual representation.This is a complex process in its own right” (p. 187). To summarize,in the present study we are interested in the differences betweencognitive processes used in purely textual representations vs. cog-nitive processes used in these highly complex visual representa-tions which almost always include verbal information in theform of captions and/or labels.1

5. Cognitive processes in text and diagrams

A wide range of theory and research suggests that some cogni-tive processes contribute more to understanding text than others.There is much evidence that cognitive activities that go beyond aliteral sentence-by-sentence understanding and involve the readeractively grappling with the text are associated with better compre-hension. Bridging and elaborative inferences as well as strategiessuch as summarizing, generating and answering questions, anddrawing or constructing concept maps have variously been termedhigh-level strategies, deep strategies, deep-processing strategies,or knowledge-transforming activities (Fox, 2009). By contrast,other strategies that do not transform knowledge are less stronglyassociated with good comprehension. These have been variouslytermed low-level, surface, shallow, or text-based strategies and in-clude highlighting, re-reading, and single-sentence paraphrases(Alexander, Sperl, Buehl, Fives, & Chiu, 2004; Cerdan & Vidal-Abar-ca, 2008; Meece & Miller, 2001; Weinstein, Husman, & Dierking,2000).

In addition to strategies, Kintsch’s (1998) construction-integra-tion model posits a key role for inference in text comprehension. Inthe CI model, the reader must actively combine prior knowledgewith information from the text in order to move from a textbasemodel to build a more sophisticated situation model of the text.In addition to the critical role of inference, other theories such asGraesser’s (2007) constructionist theory posit a key role for bothinference and strategic activity—including strategies such as self-questioning and summarizing. In the constructionist theory, strat-egies vary depending on reader goals, and some goals may, forexample, foster a great deal of summarizing or ‘‘what-questions.”

High-level strategies and inference may also be important forstudents’ understanding of diagrams, which are a prominentfeature of scientific text (Otero et al., 2002). A handful of studiescomparing students’ cognitive processes in text vs. diagrams havefound that students make more inferences in diagrams than in run-ning text (Ainsworth & Loizou, 2003; Butcher, 2006; Moore & Sce-vak, 1997). These studies suggest that inference, and perhapssummarizing, occurs more often when students are reading dia-grams than when they read running text, even when all studentsare trained to self-explain. These studies of cognitive processeswhile reading diagrams have all, however, been conducted withnon-science majors reading researcher-developed text. Are high-

level strategies and inference important for comprehension of bothtext and diagrams for science majors reading authentic texts?

6. Studies comparing cognitive processes in text vs. diagrams

The superiority of text or diagrams for learning could be due toa number of factors, one of which is the cognitive processes thatthe representation (running text vs. diagram) encourages the lear-ner to use. In order to better understand the role of these cognitiveprocesses, it is important to gather data about cognitions duringthe learning process itself. For example, to the extent that diagramslead students to draw more inferences, students may learn betterfrom diagrams than from text. However, we were only able to lo-cate three studies that directly compared cognitions while learningfrom text vs. learning from diagrams (Ainsworth & Loizou, 2003;Butcher, 2006; Moore & Scevak, 1997).2

Ainsworth and Loizou (2003) trained undergraduate students toself-explain, and then assigned one-half of their participants totext-only and one-half to diagram-only conditions. Students inthe diagrams condition verbalized more self-explanations, and alsolearned more, compared to students in the text-only condition.Specifically, these students verbalized significantly more goal-dri-ven or causal self-explanations than text-only students. In addi-tion, whereas students in the diagrams condition verbalizedprinciple-based explanations, students in the text-only conditiondid not While the self-explanation effect is robust, two differenttypes of reasons have been put forth to explain its effectiveness:monitoring/correcting knowledge (e.g., Ainsworth & Burcham,2007; Griffin, Wiley, & Thiede, 2008; Kastens & Liben, 2007), andinference (e.g., Aleven & Koedinger, 2002; Chi, Bassok, Lewis,Reimann, & Glaser, 1989). Because Ainsworth and Loizou did notcode for monitoring, they could not address this question of whyself-explanation is effective.

Butcher (2006) trained undergraduate psychology students toself-explain while learning in one of three conditions: text-only,text-with-complex-diagram, or text-with-simplified-diagram con-dition. Learning was measured using change from pretest to post-test in scores on a mental models rubric based on Chi, de Leeuw,Chiu, and LaVancher (1994). Self-explanation verbalizations werecoded as inferences or other types of self-explanations (para-phrase, elaboration, or monitoring). Participants in the two dia-gram conditions verbalized significantly more inferences (36%and 39% of self-explanations, respectively) compared to those inthe text-only condition (22% of self-explanations).

Moore and Scevak (1997) asked middle school and high schoolstudents to think-aloud from illustrated history and science texts.Students verbalized a higher proportion of main ideas in diagramsthan in text for both the history (69% vs. 49%) and science (88% vs.54%) passages, and verbalized a smaller proportion of details intext for the history passage. Moore and Scevak did not code sepa-rately for inferences, therefore these codes may include inferencesabout details or inferences about main ideas.

In summary, across the three studies reviewed above, untrainedstudents verbalize more main ideas when reading diagrams com-pared to text, and trained students verbalize more inferences whenreading diagrams compared to text. All of these studies were con-ducted with non-science majors reading researcher-developedtexts.

The present study is designed to build our understanding of thesimilarities and differences in the cognitive processes used whilemaking sense of text vs. complex diagrams. In addition to collect-

1 Moreno and Valdez (2005) found dramatically lower retention and transfer scoresfor undergraduates who were presented with wordless diagrams, on the order of a33–68% decrement over diagrams with text.

2 Note that there is a multitude of studies that collect post-reading measures oflearning from text vs. diagrams or compare the amount of learning from informationpresented in different media. In this review, we focus only on studies that collecteddata about cognitive processes used while learning.

J.G. Cromley et al. / Contemporary Educational Psychology 35 (2010) 59–74 61

ing data from a relatively large sample, we have also coded thethink-aloud transcripts for a wider range of variables than didresearchers in the three studies reviewed above. A secondary goalis to gather more ecologically valid data about the processes in-volved in comprehension of both text and complex diagrams bystudying science majors reading from their own course textbook.We assumed that readers’ cognitive processes would include theinferences which are prominent in theories of reasoning withdiagrams (Ainsworth, 2006; Larkin & Simon, 1987; Narayanan &Hegarty, 1998), interactions with prior knowledge that are promi-nent in several theories (Kintsch, 1998; Sweller, 2005), and bothhigh-level and low-level strategies frequently recorded in thethink-aloud literature (Fox, 2009). In order to compare the cogni-tive processes involved in comprehension of complex science textand diagrams, we collected and analyzed think-aloud protocolsfrom 91 undergraduate students in a course for biology majors,who were asked to say everything they were thinking out loudwhile reading from a chapter in their own textbook and then askedto provide a verbal free recall of the content. Our primary researchquestions were, for beginning biology students thinking aloudfrom their own course textbook:

1. Are there differences in the number and type of cognitiveprocesses verbalized when reading text vs. diagrams?

2. Are there differences in the mean proportion of use of variouscognitive processes verbalized when reading text vs. diagrams?

3. If so, are differences in the use of various cognitive processesassociated with level of understanding of the text as a whole?

7. Method

7.1. Participants

Participants were 97 undergraduate students enrolled in anintroductory biology course for life sciences majors at a large, mod-erately-selective urban university in the mid-Atlantic who partici-pated in exchange for extra course credit. Because of equipmentfailure, we did not collect audiotape data for six participants; wetherefore report data for 91 participants below. Their mean agewas 19.8 (SD = 2.3); there were 62 women (68%), 28 men (31%),and one student who did not identify sex (1%). Most of the studentswere freshmen (37%) or sophomores (41%), with some juniors(18%), and two post-baccalaureate students (2%). Participants wereracially diverse (40% White, 23% Black, 31% Asian, and 6% mixedrace or other races). A substantial minority of participants werefirst-generation college students; 40% had neither parent with abachelor’s degree or higher. Twenty-three percent of participantshad taken AP Biology in high school but had not obtained a highenough score to waive the Bio 101 course requirement.3 Overall,this was a relatively high-achieving group of undergraduate stu-dents, but they were by no means homogeneous in their knowledgeabout the biology topic in the text we presented, or in their SATscores or undergraduate GPAs (see below).

7.2. Materials and measures

7.2.1. Student demographicsStudents completed a demographics form, reporting their sex,

age, race, parental education, their current GPA, SAT scores, educa-tional and vocational aspirations, number of course credits

currently taken, time spent studying for this biology course, hoursof paid work per week, and other major time commitments (e.g.,parenting, involvement in sports, fraternities, sororities or otherorganizations).

7.2.2. Background knowledgeStudents completed an untimed, open-ended measure of back-

ground knowledge about the immune system. The instructionswere: ‘‘Please write down below everything you know and canremember about the immune system. Be sure to explain the partsof the immune system, what purpose each part has, and how theparts work both separately and together.” We developed a codingsystem that captures the complexity of students’ mental models(see coding below).

7.2.3. Think-aloud protocolsStudents produced a 40-min think-aloud protocol while reading

from a passage about the vertebrate immune system from a not-yet-read chapter in their own Biology textbook (Campbell & Reece,2001; see Appendix A). The topic was covered in class 10 days afterdata were collected, during the 11th week of the spring semester.The passage was 3463 words long and had six figures, five of whichwere schematic diagrams, and one of which was a flow chart. Thefocus of the text was on the two branches of the immune systemwhich involve white blood cells. We shortened the introductorymaterial for the chapter, but ensured that definitions of terms usedin the white blood cell sections were retained. With permissionfrom the publisher, we re-typed the text and scanned the illustra-tions. In the resulting 8-page text, we positioned the illustrations atthe same point they had been in the original text (i.e., if the illus-tration preceded the relevant text in the original, we formatted thepassage so that the illustration still preceded the relevant text). Thepassage was formatted and printed to resemble the original asmuch as possible, including font and text size, page layout, bolding,italics, and color illustrations. Participants had access to pen, pen-cil, highlighter, and paper with which to take notes, and were per-mitted to write on the text if they chose to do so.

The diagrams in the text included five schematic line diagrams,one flow chart, and one photomicrograph (see Fig. 1 for a samplediagram; see Table 1 for detailed coding of the diagram features).With regard to features of the diagrams, every diagram had a cap-tion, and each figure had a median of 10 labels naming specificparts (range: 2–17), a median of four explanatory labels (range:0–6), and a median of three arrows (range: 0–22). Taken together,the caption, naming labels, and explanatory labels included a meanof 165 words in each figure. These features are typical of diagramsin the Campbell and Reece (2001) textbook.

7.2.4. Verbal free recallAfter students produced the think-aloud protocol, we removed

the text and any notes they had taken, and asked them to sayeverything they remembered from what they had just read.

7.3. Equipment

The 40-min think-aloud session and untimed verbal free recallsession were audiotaped on a cassette recorder using a clip-onmicrophone. The think-aloud session was also videotaped. In aneffort to ensure anonymity, the video camera was set up to captureonly the participants’ text and note-taking activities.

7.4. Procedure

Data were collected in individual sessions conducted by the firsttwo authors in spring, 2006. After we obtained participant in-formed consent, students first completed the demographics form

3 Participants who had taken AP Biology in high school had virtually identicalbackground knowledge scores compared to participants who had not taken APBiology (M = 3.57, SD = 1.88 and M = 3.53, SD = 1.82, respectively, F [1, 89] = .007,MSE = 3.377, p = .936).

62 J.G. Cromley et al. / Contemporary Educational Psychology 35 (2010) 59–74

and written background knowledge measure (untimed). Eachstudent then completed the think-aloud (40 min) and verbal freerecall (untimed) measures, and was debriefed.

For the think-aloud session, the following instructions to partic-ipants were displayed and read out loud (adapted from Azevedo &Cromley, 2004): ‘‘You are being presented with an abridged pas-sage from your own Biology 101 textbook. We are interested inlearning about how students learn from what they read. I wantyou to read this passage as if you were learning the material forBiology 101. You will have 40 min to learn as much of this materialas you can while studying at your usual pace. You have paper, apen and a highlighter for taking notes, if that is what you usuallydo when you are studying by yourself from this Biology 101 text-book. However, I will collect them when you are done reading. Inorder to understand how you learn from a textbook, I need youto think out loud while you are reading. Please say everythingyou are thinking out loud while you read the text. I will be herein case anything goes wrong with the tape recorder or video cam-era, but I cannot answer any questions about the reading or helpyou with it. Please remember that it is very important to say every-thing you are thinking while you are working on this task.” Partic-ipants were free to ask questions, which we answered, for example,if a participant asked ‘‘Should I read aloud?”, we answered ‘‘Yes.”While reading, participants were prompted to think out loud withone of three reminders: ‘‘Please say what you are thinking,” ‘‘Donot forget to read out loud,” or ‘‘Say what you are doing.”4 Usingthe criteria from Crain-Thoreson, Lippman, and McClendon-Magnu-son (1997), we gave these prompts until participants were verbaliz-ing thoughts at the rate of approximately three utterances per 100words read. In practice, this meant that we gave reminders every4–8 sentences if participants were not verbalizing their thoughts.For the few participants who finished reading the entire passage be-fore the 40 min time limit, we repeated the instructions: ‘‘You willhave 40 min to learn as much of this material as you can.” Weemphasized the instruction ‘‘while studying at your usual pace” inorder to de-emphasize ‘‘getting through” the passage. Other researchwith students from this same course suggests that they read at a rateof approximately 140 wpm, but these students read a mean of 1950words in 40 min. We conclude that our instructions did indeedencourage them to emphasize learning over ‘‘getting through” thepassage.

At the end of the 40 min, we removed the instructions, passage,and any notes taken. We then asked participants to verbally recallinformation from the text, with the instruction, ‘‘Please say backeverything you can remember about what you just read” (unti-med). When participants finished, they were prompted with thequestion, ‘‘Anything else?” If they added any more statements,they were prompted once more with the same question. Afterany further responses, the session was concluded. All of the verbalrecalls took less than 10 min.

7.5. Data analysis and scoring

Below, we describe the transcription and video coding proce-dures we applied before coding.

7.5.1. TranscriptionParticipants’ statements on the open-ended written background

knowledge measure were typed verbatim. Inspection of these writ-ten measures suggested that participants began the study knowinglittle about the immune system, which is not surprising given thatmany had probably last studied biology in 9th or 10th grade of highschool, three or four years previous to our data collection. Themeannumber of words per participant was 85.4 (SD = 51.2, with a rangefrom 8 to 219 words, and 68% in the 33–134 word range). Eachthink-aloud session was transcribed verbatim from the audiotape,segmented, and later coded (see below). This resulted in a total of1215 typed pages (M = 13.4 pages per participant, range 7–20); eachtranscriptwas then segmented into clauses including a subject and averb (as in Chi, 1997).We also counted the last word that the partic-ipant read out of the possible 3463 words that the participant couldhave readwithin the40 min time limit; onaverage, participants readabout two-thirds of the text (M = 2189 words, range = 822–3463).Verbal recall protocols were transcribed according to the sameconventions; there was a total of 28,924 words (M = 318 words perparticipant, range = 42–1169).

7.5.2. Video codingThe videotapes were then viewed along with the transcripts in

order to code for note-taking, reading notes, and Coordinating Infor-mational Sources (e.g., pointing from text to diagram and then backto text). We also viewed the time-stamped videotapes in order toidentify the time and the point in the transcript when the partici-pant switched from verbalizing about text alone to verbalizingabout a diagram or verbalizing from his/her notes. Following thecoding for time and representation, the first author then coded ver-balizations on all of the transcripts using the coding scheme de-scribed below. The third author further coded the transcripts tonote which features of which diagrams were verbally noted by par-ticipants. Some examples of coded behavior are: any verbalizationindicating viewing of any part of the diagram at all (e.g., ‘‘OK, look-ing at the chart”), reading the caption, reading three out of five la-bels, or verbalizing ‘‘goes to” for an arrow in a diagram.

7.5.3. Coding scheme for the think-aloud protocolsThe coding scheme for the think-aloud protocols was adapted

from the Self-Regulated Learning coding scheme developed byAzevedo and colleagues (Azevedo & Cromley, 2004; Cromley &Azevedo, 2006), and modified based on previous think-aloud read-ing studies (Fehrenbach, 1991; Laing & Kamhi, 2002; McNamara,2001; Neuman, 1990; Robertson, 1990; Zwaan & Brown, 1996).Major classes of codes in the coding scheme are strategy use, infer-ence, background knowledge, vocabulary, and word reading (seeAppendix B for definitions and examples from reading running textand diagrams).

Table 1Features of the diagrams in the think-aloud text.

Topic Figure number Type Caption Naming Labels Explan. Labels Arrows Words

Clonal selection 43.6 Line diagram 1 6 4 3 178The interaction of T cells with MHC 43.9 Line diagram 1 10 2 0 122An overview of the immune responses 43.10 Flow chart 1 12 4 22 149The central role of helper T cells 43.11 Line diagram 1 11 6 4 167The functioning of cytotoxic T cells 43.12 (a) Line diagram 1 9 3 3 162The functioning of cytotoxic T cells 43.12 (b) Photo-micrograph 1 2 0 0 209Humoral response to a T-dependent antigen 43.13 Line diagram 1 17 5 7 178

4 In the unusual case that students did not read aloud as soon as the session began,we asked them to ‘‘please read aloud.”

J.G. Cromley et al. / Contemporary Educational Psychology 35 (2010) 59–74 63

Strategy use was divided into productive strategies and low-levelstrategies based on the distinction in the literature between knowl-edge-transforming and surface-level processes (Pintrich, 2000;Weinstein et al., 2000). That is, we use productive strategies to re-fer to moves such as summarizing across several sentences, whichrequire the reader to restructure the information given in the text.Productive strategies were further subdivided into high-level, meta-cognitive, and judging quality of text classes.

High-level strategies are strategies that transform the informa-tion in the text and organize the reading session. These includesummarizing, self-questioning, coordinating informationalsources, help seeking, imagery, drawing, taking notes, organizingnotes, reading notes, setting learning goals, and time and effortplanning.

We use the term metacognitive strategies to describe activitiesrelated to monitoring the reader’s own level of understanding,consistent with what Muis (2008) terms regulation of cognition:‘‘processes of planning activities prior to engaging in a task, mon-itoring activities during learning, and checking outcomes againstset goals” (p. 181). These strategies include feeling of knowing,judgment of learning, monitoring use of strategies, planning, taskdifficulty and task ease. Participants frequently combined thesemetacognitive strategies with re-reading and other ‘‘fix-up”strategies.

Judging quality of text was defined as strategies that focus onthe quality of the text itself, such as, ‘‘This is not written well.”These include codes for adequacy of text, inadequacy of text, ade-quacy of diagrams, inadequacy of diagrams, and order of text anddiagrams.

Low-level strategies are defined as those that use little restruc-turing of information from text; they focus on the surface level ofthe text, and do not make connections between different sentencesof text—even adjacent sentences—or between prior knowledge andtext. These codes include paraphrasing, re-reading, highlighting,memorizing, mnemonics, course demands, find location, not think-ing, omission of figure, importance of information, and text struc-ture. Importance of information and text structure are included aslow-level strategies because these codes consisted of studentscommenting that bolded vocabulary words were important. Thiswas frequently followed by copying or highlighting the definitions,indicative of surface-level reading strategies.

We adopt a definition of inference from Edmonds and Pring(2006): ‘‘to fill in missing information by applying general knowl-edge, and to make links between different sections of text” (p. 338).Therefore, we coded as inference all deductions that the readermakes, but we did not code for anaphoric reference (e.g., statingwhat the word ‘‘it” in the current sentence refers to, when thatinformation was given in a previous sentence). Inferences arehighly effortful, and inference has strongly differentiated studentswith higher levels of comprehension from those with lowercomprehension (Cain, Oakhill, Barnes, & Bryant, 2001). These codesinclude deductions drawn from information entirely within textand codes that include both background knowledge and informa-tion from text. Vocabulary includes verbalizations related to eitherknowing or not knowing or remembering the meaning of words inthe text (as coded by Wade et al., 1990). Word-reading difficultiesinclude mis-pronouncing a word and immediately self-correctingthe pronunciation or not self-correcting (as coded by Schellings,Aarnoutse, & van Leeuwe, 2006).

7.5.4. Coding scheme for background knowledge and verbal free recallprotocols

We created a single mental model coding scheme for codingboth the open-ended written background knowledge measureand the verbal free recall protocols. We coded both measuresusing a 9-point mental model coding scheme that captured

increasingly-integrated knowledge. This coding scheme wasdeveloped with guidance from the course instructors, who areboth practicing biologists (see Appendix C for the full codingscheme). We coded a mean of 4.1 clauses per participant at pre-test and a mean of 7.5 clauses at post-test. We did not deduct anypoints for incorrect or irrelevant information, misspellings, ormis-pronunciations in these measures. With regard to misconcep-tions, 41 participants verbalized at least one misconception atpretest (a mean of 1.2 misconceptions per person), and 30 partic-ipants verbalized at least one misconception at post-test (a meanof 1.4 misconceptions per person), however these misconceptionswere not the same at pre- and post-test except for oneparticipant.5

7.5.5. Coding participant notesWe typed up participants’ notes, whether written on separate

notepaper or on the text itself. The mean number of words per par-ticipant was 183 (SD = 121, range 0–470). Twenty-six participantstook either no notes at all, or wrote less than 10 words of notes. Forthe 65 participants who did take notes, we counted strings of 4 ormore words in a row that were taken directly from the text as ver-batim note-taking. The mean proportion of verbatim notes per par-ticipant was 29% (SD = 22%, range 0–91%).

7.6. Inter-rater agreement

For all measures, the first author coded all of the data and thesecond author—a graduate student in school psychology—servedas the second coder. She was trained on data other than those usedto calculate inter-rater reliability and then re-coded 35% of eachcorpus. In all cases, both raters were blind to scores on all othermeasures; after re-coding all differences were resolved by discus-sion. For the prior knowledge measure, the two coders agreed on30 out of 32 mental models, yielding an inter-rater agreement of94%. For the think-aloud protocols, we agreed on 3628 out of the3891 final codes, yielding an inter-rater agreement of 93%. Forthe final mental model (verbal free recall protocol) coding, weagreed on the scores for 29 out of the 32 protocols, yielding an in-ter-rater agreement of 91%.

7.6.1. Data analysisFirst, we converted the raw number of verbalizations within

each representation to a proportion for each participant, in orderto control for differences in amounts verbalized (these ranged from26 to 235 coded verbalizations per participant in running text andfrom 0 to 60 coded verbalizations per participant in diagrams). Formis-readings of words, we used the last word read in the text asthe denominator for this proportion (i.e., a participant who mis-read seven words and read up to the 1473rd word mis-read .005of words read). We note that participants almost never skippedmore than one paragraph of the text, even though they frequentlyskipped diagrams; only two of the 91 participants used a skimmingapproach in text.

Proportion data for verbalizations are not typically normallydistributed, and include students who never verbalized thevariable of interest (e.g., 0 inferences). We therefore used theWilcoxon matched pairs signed ranks test for comparisons be-tween representations; this test is the non-parametric equivalentof a repeated-measures t test. We interpreted the results ofthese tests to be statistically significant at an alpha level ofp < .05.

5 Analyses using our original rubric score minus a correction for number ofmisconceptions yielded exactly the same pattern of results.

64 J.G. Cromley et al. / Contemporary Educational Psychology 35 (2010) 59–74

8. Results

8.1. Descriptive statistics

Below, we describe scores on the pre- and post-test measures,time spent in text vs. diagrams and diagram features verbalized,and raw counts and mean proportions of use for each think-aloudcode.

8.1.1. Scores on pre- and post-test measuresScores on the written background knowledge measure had a

mean of 3.54 (out of nine) and a standard deviation of 1.82; scoreson the final mental model had a mean of 5.64 (out of nine) and astandard deviation of 1.94. In order to rule out the possibility thathigher mental models might simply be due to reading faster orfarther in the text, we computed a correlation between how farstudents read in text and their final mental model scores; thiswas non-significant (r [89] = !.07, p = .26). A dependent-samplest-test showed a significant increase from the written backgroundknowledge measure score (M = 3.54, SD = 1.83) to the final mentalmodel score (M = 5.64, SD = 1.94), t (90) = 8.60, p < .001, d = 1.15,suggesting that participants did comprehend the text well enoughto learn from it.

8.1.2. Time spent in text vs. diagramsOn average, participants spent 32 min 32 s in text and 7 min 28

s in diagrams, but amount of time spent in diagrams was not

significantly correlated with post-test mental model scores. Withregard to viewing diagrams, participants on average verbalizedsomething about 78% of the possible images (that is, images theycould have viewed given how far they read in the excerpt). In theimages that were viewed, participants made verbalizations about52% of the captions, 36% of the naming labels, 61% of the explana-tory labels, 13% of the process arrows, and 7% of the symbols.

8.1.3. Think-aloud codesThe raw counts for each think-aloud code, as well as mean per-

centage of use for each code (the frequency of use for each code foreach participant divided by the total number of verbalizations forthat participant within each representation) are shown in Table 2.

8.1.4. Data to support the validity of the background knowledge andfree recall scores

We present two types of evidence to support the validity of thebackground knowledge and free recall scores: correlations of thesevariables with background knowledge verbalized during the think-alouds, and differences among students with and without priorbiology coursework. The non-parametric Spearman correlation be-tween scores on the background knowledge measure and propor-tion of verbalizations of background knowledge was rs [89] = .28,the correlation between scores on the background knowledgemeasure and final mental model scores was rs [89] = .21. Juniorsand post-baccalaureate students had significantly higher back-ground knowledge scores (F [3, 85] = 2.964, MSE = 3.040, p = .037)

Table 2Frequency and percentage of coded student verbalizations for each variable.

Coded variable In text In diagrams

Total number ofoccurrences

Mean percentage ofuse within rep. (%)

Total number ofoccurrences

Mean percentage ofuse within rep. (%)

Background knowledgePrior knowledge activation+ 192 2.0 62 2.7Prior knowledge activation- 62 0.6 33 1.4

InferenceInference+ 90 0.5 39 1.7Knowledge Elaboration+ 187 1.8 62 3.8Other (see text for codes) 92 1.0 29 1.3

Productive strategiesHigh-level strategiesCoordinating info. sources 73 0.7 119 6.6Organizing notes 182 1.5 11 0.6Reading notes 122 1.1 42 2.0Summarizing+ 563 5.7 328 16.8Taking notes 1361 14.3 97 5.6Other (see text for codes) 231 2.4 72 4.3

Metacognitive strategiesFeeling of knowing 397 4.1 95 0.1Judgment of Learning 395 4.0 124 6.1Monitoring use of strategy 123 1.2 16 0.9Other (see text for codes) 113 1.1 11 0.5

Judging textInadequacy of text 127 1.2 15 1.7Other (see text for codes) 114 1.2 121 4.0%

Low-level strategiesHighlights text 872 8.8 74 3.6%Importance of information 189 1.8 13 0.6%Omission of figure 84 1.0 8 1.5%Rereads text 1737 18.2 192 10.2%Other (see text for codes) 572 6.1 115 5.5%

VocabularyVocabulary 658 6.8 145 6.1%

Word readingSelf-corrects word 1073 12.4 90 5.6%

Note: codes are defined in Appendix B.

J.G. Cromley et al. / Contemporary Educational Psychology 35 (2010) 59–74 65

than freshman and sophomore students, and juniors and post-bac-calaureate students were more likely to have taken a prior college-level biology course (89% vs. 33%).

8.1.5. Research question 1: are there differences in the number andtype of cognitive processes verbalized when reading text vs. diagrams?

While reading text, participants tended to verbalize more over-all (9670 vs. 1913 coded utterances), consistent with the greateramount of time spent in text vs. diagrams. Across all classes ofcodes, a significantly larger proportion of participants verbalizedat least one coded variable from that class when reading text thanwhen reading diagrams (see Table 3 for details and results of sta-tistical tests). For example, in running text 67 students (74% of par-ticipants) verbalized at least one inference, but in diagrams 47students (53% of participants) verbalized at least one inference.All of these proportions are significantly different by a z test for dif-ference between two proportions.

In addition, the types of cognitive processes within a class weresignificantly more varied in text than in diagrams (see Table 3 fordetails and results of statistical tests). For example, in running textparticipants verbalized a mean of 4.20 different high-level strate-gies (out of a total of 10 coded types), but in diagrams they verbal-ized a mean of 2.67 different types. All of these means aresignificantly different by non-parametric repeated-measures tests(Wilcoxon matched pairs signed ranks test). Diagrams provokedsignificantly less frequent use of various cognitive processes anda significantly smaller number of cognitive processes per partici-pant compared to running text.

8.1.6. Research question 2: are there differences in the meanproportion of use of various cognitive processes verbalized whenreading text vs. diagrams?

We conducted a series of non-parametric repeated-measurestests (Wilcoxon matched pairs signed ranks test) on proportionof verbalization of each think-aloud variable with representationas the within-subjects variable. We show the mean percentage ofuse of each variable by representation in Table 4.

The Wilcoxon test showed a significant difference betweenrepresentations (text vs. diagrams) in proportion of inferences (T[N = 91] = 2.594, p < .001). Participants verbalized a significantlyhigher proportion of inferences in diagrams (7%) than they did intext (4%). There was a significant difference between representa-tions in proportion of high-level strategies (T [N = 91] = 2.977,p = .003). Participants verbalized a higher proportion of high-level

strategies in diagrams (34%) compared to running text (26%; seeFig. 2). There was a significant difference between representationsin proportion of judging text strategies (T [N = 91] = 5.419, p < .001;8% in diagrams vs. 3% in running text), but no difference in meta-cognitive strategies (T [N = 91] = 1.476, p = .140; 12% in diagramsvs. 10% in running text).

The Wilcoxon test showed a significant difference between rep-resentations in proportion of low-level strategies (T[N = 91] = 5.448, p < .001). Participants verbalized a significantlyhigher proportion of low-level strategies in text (36%) than theydid in diagrams (21%). There was no significant difference betweenrepresentations in proportion of verbalizations of backgroundknowledge (T [N = 91] = 1.668, p = .095). Participants verbalizedroughly the same proportion of verbalizations of backgroundknowledge in text (3%) as they did in diagrams (5%). There was asignificant difference between representations in proportion ofvocabulary difficulty (T [N = 91] = 2.197, p = .028). Participants ver-balized a significantly higher proportion of vocabulary difficulty intext (7%) than they did in diagrams (6%). There was also a signifi-cant difference between representations in proportion of word-reading difficulties (T [N = 91] = 5.681, p < .001; 5% in diagramsvs. 12% in running text).

8.1.7. Research question 3: are differences in the use of variouscognitive processes associated with level of understanding of the textas a whole?

The results above suggest that—compared to purely textual rep-resentations—diagrams seem to provoke the use of significantlyfewer and less varied cognitive activities, but a significantly highermean proportion of these activities are inferences and high-levelstrategies, while a significantly lower mean proportion of theseactivities are low-level strategies and vocabulary difficulty.

How are these differences in diagram vs. text processing relatedto passage comprehension, as reflected in mental model scores? Aseries of non-parametric correlations between mental modelscores and proportion of verbalization for each of the various clas-ses of cognitive processes in text showed a significant relationshipfor inferences (rs [89] = .28, p = .007) and for verbalizations of back-ground knowledge (rs [89] = .36, p = .002). Participants who verbal-ized a higher proportion of inferences and background knowledgewhen reading text tended to have higher mental model scores. Thesame analyses for diagrams showed a significant relationship forinferences (rs [87] = .22, p = .037). Participants who verbalized a

Table 3Descriptives on percentages by representation and results of non-parametric repeated-measures t tests on cognitive activities.

Representation T p

Text Diagrams

Percentage of inferences 3.6% 6.5% 2.594 <.001(3.6%) (12.0%)

Percentage of high-level strategies 26.3% 33.5% 2.977 .003(16.0%) (18.9%)

Percentage of judging text strategies 2.5% 7.7% 5.419 <.001(2.7) (12.6)

Percentage of metacognitive strategies 10.1% 12.4% 1.476 .140(6.6) (12.0)

Percentage of low-level strategies 35.9% 21.2% 5.448 <.001(16.4%) (20.1%)

Percentage of verbalization of background knowledge 2.6% 4.5% 1.668 .095(3.6%) (8.6%)

Percentage of verbalizations of vocabulary difficulty 6.8% 6.4% 2.197 .028(5.0%) (8.4%)

Percentage of verbalizations of word reading difficulty 12.4% 5.5% 5.681 <.001(10.7%) (8.9%)

Note: df for all analyses is 90.

66 J.G. Cromley et al. / Contemporary Educational Psychology 35 (2010) 59–74

higher proportion of inferences when reading diagrams tended tohave higher mental model scores.

With regard to variability in the number of cognitive activities,Pearson correlations between the number of cognitive processesused within each class of cognitive processes in text and post-testmental model scores showed that the wider the variety of infer-ence types verbalized in running text, the higher were mentalmodel scores (r [89] = .35, p < .001) and the wider the variety inactivation of background knowledge (i.e., activation of both accu-rate and inaccurate knowledge), the higher were mental model

scores (r [89] = .32, p < .001). The same analyses for diagramsshowed that the wider the variety of inference types verbalizedin diagrams, the higher were mental model scores (r [89] = .22,p < .001). Students who verbalized a larger proportion and greatervariety of inference types in both text and diagrams showed bettercomprehension.

9. Discussion

It appears from the results of our think-aloud protocol analysisthat when reading diagrams, students engage in significantly moreinference and high-level strategies and show significantly less useof low-level strategies and vocabulary difficulty, compared towhen reading running text. Results of the think-aloud protocolsand other measures also suggest that both variability in inferencetypes and greater use of inference is as prevalent in the processof comprehending diagrams as in comprehending scientific text.

Participants did learn from the passage—their mental modelscores increased significantly from pretest to post-test—but overalllow levels of inference in running text and diagrams were associ-ated with increases that were only of modest size, consistent withresults frommuch prior research with non-science undergraduateslearning from scientific text (Otero et al., 2002). Overall, the mostfrequently used cognitive processes included some that have beenfound to be effective in prior research—taking notes, summarizing,metacognitive monitoring—and some that have been found to be

Table 4Number of students verbalizing at least one cognitive process and number of different types of processes, by representation.

Representation z p

Text Diagrams

InferenceNumber (%) of students verbalizing at least one 67 47 2.92 .002

(74%) (53%)Mean (SD) number of different types per student 1.72 .90 5.49 <.001

(1.34) (1.03)

High-level strategyNumber (%) of students verbalizing at least one 89 82 2.06 .022

(99%) (93%)Mean (SD) number of different types 4.20 2.67 5.93 <.001

(1.89) (1.34)

Judging text strategyNumber (%) of students verbalizing at least one 65 59 0.73 .233

(72%) (67%)Mean (SD) number of different types 1.32 .97 2.81 <.001

(1.12) (.90)

Metacognitive strategyNumber (%) of students verbalizing at least one 86 69 3.60 <.001

(96%) (78%)Mean (SD) number of different types 2.70 1.38 6.26 <.001

(1.12) (1.00)

Low-level strategyNumber (%) of students verbalizing at least one 91 73 4.11 <.001

(100%) (83%)Mean (SD) number of different types 4.56 1.53 7.67 <.001

1.64) (1.17)

Fact from background knowledgeNumber (%) of students verbalizing at least one 62 40 3.25 <.001

(69%) (45%)Mean (SD) number of different types .97 .59 3.53 .001

(.77) (.74)

Vocabulary difficultyNumber (%) of students verbalizing at least once 72 51 3.19 <.001

(80%) (58%)

Word reading difficultyNumber (%) of students verbalizing at least once 88 46 7.15 <.001

(98%) (52%)

Note: df for all analyses is 89.

Fig. 2. Mean proportion of verbalizations by representation.

J.G. Cromley et al. / Contemporary Educational Psychology 35 (2010) 59–74 67

relatively ineffective—highlighting, re-reading, expressions ofvocabulary difficulty, and mis-readings of words (Pintrich, 2000).

Overall, there was a large negative correlation between propor-tion of low-level strategies and proportion of productive strategies(r [89] = !.85, p < .05). That is, there seemed to be a trade-off be-tween the low-level strategies and more knowledge-transformingstrategies such as summarizing, self-questioning, and coordinatingtext and diagrams. Together with our results for inference, theseresults are strongly consistent with a ‘‘knowledge transforming,”‘‘knowledge building,” or ‘‘generative learning” perspective (Bereit-er & Scardamalia, 2006; Linn, 2006; Wittrock, 1990). Readers needto internalize the material, connect it with what they know, sum-marize in their own words across sentences, and actively drawconclusions between information in order to effectively learn fromtext about complex topics such as this one. Being a ‘‘good strategyuser” (Pressley & Harris, 2006) is not enough for these undergrad-uate majors to learn from this challenging scientific text withdiagrams. We note, however, that researchers who categorizeinferences as a strategy—rather than coding these separately—might consider our results to show that a subset of strategies isimportant for learning from this text.

Contrary to prior research, time spent in text vs. diagrams wasnot related to final mental model scores (Schwonke et al., 2009).One difference between our study and other research that has re-lated time in representations to learning outcomes, is that we useda relatively long passage with multiple diagrams, a large amount ofrunning text, and a 40 min learning period. Participants mightmake different decisions about relative time in text and diagramsin this situation than in a short learning session with one diagramand a small amount of running text. Consistent with prior research,participants verbalized about relatively few features of the dia-grams (34% of possible features), and entirely skipped a mean of22% of the figures (Schmidt-Weigand et al., in press), even thoughonly two of the 91 participants skipped more than one paragraphof text. It is possible that students perceived there to be a trade-off between reading text and reading diagrams because of the40 min time limit imposed (despite our repeated instructions to‘‘study at your usual pace”). However, if students did perceivethere to be a tradeoff, they almost unanimously chose to skip dia-grams, and that this in and of itself is useful information about stu-dents’ approaches to reading scientific text. Because readingdiagrams is associated with significantly more inference andhigh-level strategy use, together with significantly less low-levelstrategy use and vocabulary difficulty, students may be deprivingthemselves of opportunities to learn science content when theyskip or merely skim diagrams in biology text.

With regard to the number and type of cognitive activities in textvs. diagrams, textmay foster use of awider variety of types of cogni-tive activities (e.g., more students who verbalized at least one infer-ence). Text may also foster a wider variety of specific activitieswithin types (e.g., verbalizing a larger number of different inferencecodes, such as accurate and inaccurate knowledge elaborations andbridging inferences). In our data,we found this association across alltypes of cognitive processes—inference, high-level strategies, judg-ing text strategies, metacognitive strategies, activating facts frombackground knowledge, vocabulary difficulties, and word-readingdifficulties. It seems that on average there is a reduced repertoirewhen students look at diagrams, even though all codes were usedby at least one student in diagrams as shown in Table 2 (that is, theseare not ‘‘nonexistent” codes in diagrams). With regard to Ains-worth’s (2006) framework, this finding suggests that diagrams donot lead to the use of qualitatively different sets of strategies. Wespeculate that students may not be as proficient at comprehendingdiagrams as they are at comprehending running text, so in diagramsthey do not use the cognitive activities that we know they possessbecause they used them in running text.

With regard to the relative use of various cognitive activities,diagram reading was associated with a significantly higher propor-tion of inferences (7% vs. 4% in running text), high-level strategies(34% vs. 26%), and judging text strategies (8% vs. 3%), while runningtext was associated with a higher proportion of low-level strategies(36% vs. 21% in diagrams), vocabulary difficulty (7% vs. 6%), andword reading difficulty (12% vs. 6%). There were no significant dif-ferences for metacognitive strategies or verbalization of back-ground knowledge.

These finding are consistent with prior findings regarding infer-ence from Ainsworth and Loizou (2003) and Butcher (2006) andwith prior findings for high-level strategies fromMoore and Scevak(1997). Our findings regarding judging text strategies, low-levelstrategies, metacognitive strategies, verbalization of backgroundknowledge, and vocabulary and word reading difficulty have notbeen researched previously. Specifically, we believe that the supe-rior learning from self-explanation in diagrams found by Ains-worth and Loizou (2003) might be explained by the largeramount of inference associated with looking at diagrams ratherthan by a metacognitive explanation, since we found non-signifi-cant differences for metacognitive strategies and significant differ-ences for inference.

How are these cognitive activities related to learning from thetext as a whole? Our correlational results suggest that when read-ing text, verbalizations of a higher proportion of use of inferenceand background knowledge is associated with better free recall(as is verbalization of a wider variety of inferences). When readingdiagrams, verbalizations of a higher proportion of use of inferenceand a wider variety of inferences is associated with better free re-call. These results are consistent with a large number of think-aloud (Fox, 2009) and experimental (National Reading Panel,2000) studies. Our study shows how diagram use is associatedwith a higher proportion of and greater variability in (when com-pared to running text) those very cognitive activities that are asso-ciated with increased comprehension. However, the finding thatproportions of use of productive and low-level strategies are unre-lated to free recall may come as a surprise. We speculate that theremay be distinct sub-types of readers who use different combina-tions of inference, low-, and high-level strategies within our sam-ple (Alexander et al., 2004), leading to overall non-significantresults for strategy use.

9.1. Limitations

This study had certain limitations that limit the generalizabilityof the findings. First, these results are correlational; better learningfrom diagrams could be due to either a bi-directional relationshipor an unmeasured variable that is associated with more reading ofdiagrams, rather than to the cognitive processes verbalized whenparticipants try to learn from diagrams.

Second, we took a highly domain-specific approach in thisstudy—we analyzed the relationship of biology background knowl-edge to comprehension of biology text rather than the relationshipof general background knowledge to general reading comprehen-sion. We thereby sacrificed some generalizability in order to tapdomain-specific processes. While think-aloud reports have beenextremely useful in reading research, participants may not verbal-ize everything they are thinking (e.g., verbalizing an understandingof vocabulary), even when prompted. Furthermore, students do not‘‘usually study” by reading aloud, so the requirement to readaloud—in order to understand what specific part of the passagestudents were verbalizing about and to track re-reading—makesthis study less ecologically valid.

Using a lengthy naturalistic text means that we were not able toanalyze the relationship between text segments and verbalizationsat a fine-grained level. Using open-ended written background

68 J.G. Cromley et al. / Contemporary Educational Psychology 35 (2010) 59–74

knowledge and verbal free recall measures probably resulted inweaker relationships than would have been shown by using thesame response format (oral or written) on both measures and alsousing closed-ended measures. In addition, we only used one, non-standardized measure of immediate comprehension, and we didnot include a filler task. We should confirm the results of our cor-relational analyses of think-aloud data with experimental studies.We only considered one university and one course (albeit one witha very diverse student population) and only one text. Furthermore,undergraduate biology majors are a self-selected group—they suc-ceeded in entering college, have ambitious career goals, had al-ready passed college-level chemistry and math courses, and wereconcurrently enrolled in second-semester chemistry and mathcourses as well as this biology course. Our results may thereforebe university-, course-, and/or text-specific.

10. Conclusion

With regard to representations, our findings for inference andproductive strategies with untrained students were consistentwith the handful of studies that have compared verbalizationsfor students trained to self-explain in text and in diagrams (Ains-worth & Loizou, 2003; Butcher, 2006). Overall, students used a sig-nificantly higher proportion of inferences and high-level strategiesand a significantly lower proportion of low-level strategies in dia-grams than in text. This results in an interesting paradox—on theone hand, students often skipped diagrams (as found bySchmidt-Weigand et al., in press), skimmed only a few elementsin the diagrams, or complained about how difficult or useless theywere, yet diagrams seemed to promote more high-level, integra-tive activity and seemed to discourage low-level superficial strate-gies. One possible explanation is that diagrams do encourage moreintegrative activity and better learning, but students simply do notenjoy being made to work hard and therefore skip or skim the dia-grams, even though they are very useful. Another possible explana-tion which we cannot test in the present study is that students skipor skim diagrams because they have a low self-efficacy for under-standing diagrams—they feel they cannot understand diagrams, sothey skip over them. A third possibility is either a bi-directionalrelationship between diagram reading and inference, or a thirdvariable such as print exposure that affects both diagram readingand inference.

Diagrams have not always been found to promote the mosteffective learning for all students (Mayer & Sims, 1994). Research-ers have found effects of learner characteristics (e.g., spatial abil-ity), diagram features (e.g., cross-sections vs. other types ofdiagrams), and learning tasks (e.g., effects on factual knowledgevs. conceptual knowledge)—as well as interactions among theselearner characteristics—on using diagrams (Sanchez & Wiley,2006). In our sample, with these diagrams, and the task to read‘‘as if you were learning the material for Biology 101,” it seems thatstudents were able to engage in some of these integrative activitiesand this was associated with better understanding of the topic.Perhaps previous studies that have found worse learning from dia-grams have studied participants with very little knowledge of thetopic and/or poor strategic knowledge. This might have led tolow levels of inference and productive strategy use in diagrams,and therefore worse learning from diagrams compared to text. An-other possibility that we cannot test in this study is that diagramsuse certain conventions (e.g., an arrow can symbolize motion,change, or enlargement; Heiser & Tversky, 2006; Sweller, 2005),and students need a certain minimum amount of knowledge ofthese conventions before diagrams will lead to higher levels ofinference and productive strategy use. A third possibility is thatthere is either a bi-directional relationship or an unmeasured var-

iable that is associated with both more inspection of diagrams andbetter learning, but that better learning is not due to cognitiveactivities used while learning from diagrams.

Our findings in the present study suggest that the processesfound with non-science majors reading both non-scientific and sci-entific text (Otero et al., 2002) hold for science majors reading sci-entific text (Ozuru, Dempsey, & McNamara, 2009), suggestingsome generality of reading comprehension processes. All readersneed to engage in a variety of inferential processes in order to com-prehend both text and diagrams. Even though inferences do notmake up a very large proportion of total verbalizations (7% of ver-balizations in diagrams), they are strongly associated with compre-hension of both text and diagrams.

10.1. Implications for theories of diagrammatic reasoning

Overall, our findings are consistent with the theoretical impor-tance of background knowledge for comprehension of diagrams.They are also consistent with Ainsworth’s (2006) proposal thatdifferent representations lead to differential use of cognitive activ-ities, but we have evidence to broaden her claim beyond high-levelstrategies to include inference and low-level strategies. In addition,our findings shed light on self-explanation as it relates to diagrams.Our findings support the inferential account of self-explanation(Aleven & Koedinger, 2002) over the metacognitive account (Griffinet al., 2008). That is, we believe that self-explanation is beneficialfor diagram comprehension because it encourages students todraw inferences.

10.2. Implications for instruction with illustrated texts

Our results suggest that instructors can and should encouragestudents to thoroughly read and process diagrams in scientific text,as a way of encouraging inference, high-level strategies, and acti-vation of background knowledge and thereby increase learning.Instructors may not be aware that many students simply skip overmany of the diagrams, and only superficially skim those diagramsthat they do inspect. Since so few studies have tested methods ofteaching students to understand diagrams, there is no basis for rec-ommending a specific type of intervention. Those designinginstruction might consider building on existing instructional inter-ventions to foster inference, high-level strategies, and activation ofbackground knowledge as part of diagram instruction. We are cur-rently conducting research on classroom interventions at the highschool level, specifically comparing instruction in conventions ofdiagrams to instruction in coordinating text and diagrams.

10.3. Implications for future research

In addition to further research on interventions to improve rea-soning with diagrams, we see the need for more basic research onthe processes underlying diagram comprehension. Future researchshould triangulate results by using more than one measure oflearning processes—such as eye tracking, log files or other tracedata, reaction time data, think-aloud protocols, and/or psycho-physiological data. Researchers could also compare reasoning withdiagrams in different domains—what are the differences betweentypical graphical representations in chemistry and biology, forexample? How do visual representations in different domainsmake different demands on learners—do representations in somedomains demand more or less spatial ability or working memoryskills? Are there subskills such as comprehending three-dimen-sional visualizations that are more necessary for some domains(e.g., cutaway diagrams in geoscience) than others (e.g., free bodydiagrams in physics)?

J.G. Cromley et al. / Contemporary Educational Psychology 35 (2010) 59–74 69

The process of coordinating textual information and diagram-matic information is also under-specified in research. Studies,including ours, tend to track the number of times learners movefrom textual to visual information and occasionally relate this toindividual differences, but rarely analyze the cognitive processesinvolved in such coordination. Which cognitive processes precedea switch from text to diagrams and vice versa? Which processesare used to connect textual and visual information—is this primar-ily a process of matching? of analogy? of inference? Are there dif-ferent patterns of activity between text and diagrams thatcharacterize participants with different scores on individual differ-ence variables such as knowledge, spatial ability, or workingmemory?

Acknowledgments

This study was partially funded by a Return on Indirect Re-search Incentive Grant from Temple University to the first author.A previous version of this manuscript was presented at the 2007annual meeting of the American Educational Research Association.The authors wish to thank Todd Mendelssohn for assistance withtranscribing and video coding, and James P. Byrnes for commentson an earlier version of the article.

Appendix A

A.1. Excerpt from the think-aloud text (adapted from Campbell &Reece, 2001, p. 906)

The selective proliferation and differentiation of lymphocytesthat occurs the first time the body is exposed to an antigen is theprimary immune response. In the primary response, about 10–17 days are required from the initial exposure to antigen for se-lected lymphocytes to generate the maximum effector cell re-sponse. During this period, selected B cells and T cells generateantibody-producing effector B cells, called plasma cells, and effec-tor T cells, respectively. Eventually, symptoms of illness diminishand disappear as antibodies and effector T cells clear the antigenfrom the body. If that individual is exposed to the same antigenat some later time, the response is faster (only 2–7 days), of greatermagnitude, and more prolonged. This is the secondary immune re-sponse. The immune system’s capacity to generate secondary im-mune responses is called immunological memory.

Appendix B

B.1. Classes, descriptions, and examples of the variables used to codethink-aloud protocols

Class code Definition Example (T = text, D = diagram)

Background knowledgePrior knowledge

activationaccurate

Recalls specific correct information from memory or from previoustext read during the TA session

‘‘So meaning that a single antibody will bind to a singleantiagent and not to many others.” (D)

Prior knowledgeactivationinaccurate

Recalls specific incorrect information from memory or fromprevious text read during the TA session

‘‘Secrete cytokines which are going to go and attempt to killbacteria or the virus or whatever the harmful this is inthere.” (T)

InferenceHypothesis Makes a prediction or hypothesis about how something works or

what will come up next in the text.‘‘I bet that’s why you never get the same cold twice” (T)

Inferenceaccurate

Accurately draws a conclusion from current text + current textwithin one paragraph

‘‘So it’s going to stimulate antibodies and it’s going tostimulate cytotoxic t cells.” (D)

Inferenceinaccurate

Inaccurately draws a conclusion from current text + current textwithin one paragraph

‘‘Do these MHC molecules bind to antigens? I’m guessingthey’re probably and they combat and produce antibodies.”(T)

Knowledgeelaborationaccurate

Accurately draws a conclusion from prior knowledge + current text ‘‘It shows the infected cell and the antigen fragments thatare left over.” (D)

Knowledgeelaborationinaccurate

Inaccurately draws a conclusion from prior knowledge + currenttext

‘‘And AIDS I think would be a cell mediated because it has todo with the t cells.” (D)

StrategiesHigh levelCoordinating

InformationalSources

Puts text together with diagram (e.g., reads text, then finds part ondiagram) or puts diagram together with text (e.g., flips back andforth between diagram and text)

Reads about antigen-presenting cells, then looks at diagram‘‘there it is” (T)

Drawing Makes a drawing or diagram/flow chart. Can be coded from video ‘‘I’m going to copy this diagram” (D)Help-seeking

strategyStates a help-seeking strategy that he/she would use, e.g., lookingup a word in a dictionary, asking a peer or instructor

‘‘I would look that up in a dictionary” (T)

Imagery Participant forms an internal mental image ‘‘You have my cell, you have my fragment, and how it goes-how it presents it really, it says it carries it to the cellsurface, so I am trying to picture it. . .” (D)

Organizingnotes

Talks about organizing notes, using outline, adding notes to anearlier section, using highlighting, asterisks/stars, drawing arrowfrom one section to another, making a table that contrasts twoconcepts

‘‘I’m writing this in sort of an outline form” (T)

70 J.G. Cromley et al. / Contemporary Educational Psychology 35 (2010) 59–74

Appendix B (continued)

Class code Definition Example (T = text, D = diagram)

Reads notes Reads own notes or shows evidence of having read (e.g., ‘‘Yes, I dohave that in my notes”)

‘‘What I usually do now is read over my notes” (T)

Self-questioning States a curiosity question that might or might not be answerableby later text

‘‘How do these antigens interact with the antibodies?” (T)

Summarizingaccurately

Accurately re-states what was read/diagram in own words acrosstwo or more sentences (not simply paraphrasing one sentence orre-reading). Note that a sentence can be split into accurate (SUM+)and inaccurate (SUM!) portions

‘‘And antigen first exposure, and its engulfed by the macro-phagz, and it stimulates helpers T cells, and then it stimulatesthe B and cytotoxic cell. . .” (D)

Takes notes Writes notes on paper or on the text directly ‘‘I have to go back and take notes. . . . large number of effectorcells” (D)

Time and effortplanning

Notices how much time is remaining and/or has passed ‘‘I’ve been working for 14 min; I have 26 min left” (T)

MetacognitiveFeeling of

knowingStates that he/she understands, or that information is familiar (i.e.,matching new information to information in memory)

‘‘OK, that makes sense” (T)

Judgment oflearning

States that he/she does not understand ‘‘. . .trying to figure out what’s going on here in thisdiagram. . .don’t understand,. . .” (D)

Monitor use ofstrategy

Comments on the usefulness (or uselessness) or effect on easinessof a strategy such as taking notes, COIS, etc

‘‘don’t need to write a definition because if I already knowwhat primary immune [sic] is” (D)

Planning Plans out how to approach learning, including skimming/flippingthrough text to plan the comprehension session. States any twostrategies to be enacted in an order

‘‘I’m just writing down, making the topic into questionsactually, then as I read I’m going to try to answer thesequestions” (T)

Task difficulty States that the task is difficult ‘‘uh, this figure is kind of hard to memorize” (D)Task ease

States that the task is easy, not difficult ‘‘It’s not as complicated as I thought it would be” (T)

Judging textAdequacy of

diagramStates that a diagram is helpful, useful for comprehension, or‘‘good” or states a preference for pictures

‘‘I like learning from pictures a lot better than reading” (T)

Adequacy oftext

States that text is informative or upcoming text will be informative ‘‘I do not know what an MHC molecule is, and I haven’t readthe passage yet so I’m sure it will be explain what it is” (D)

Inadequacy ofdiagram

States that a diagram is not helpful, not useful for comprehension,‘‘bad” or ‘‘disliked,” ‘‘confusing,” or participant ‘‘doesn’t learn fromdiagrams.”

‘‘I have a difficult time sometimes interpreting thesepictures” (T)

Inadequacy oftext

States that text is confusing, unclear, or could have been statedbetter

‘‘the previous passage, . . . was complicated for me.” (D)

Order of textand diagrams

Talks about the order in which a) text and diagrams appear in thetext or b) the order in which participant should read them, butwithout stating that one representation is ‘‘better” or ‘‘morehelpful”

‘‘I just wish that the diagram came after the explanation thatthey just gave me” (T)

Low-level strategiesCourse demands Mentions what is required for course tasks/assignments such as

quizzes, labs, or exams‘‘Um, I’m just looking at the pictures, and looking at what’slabeled where, just to get a visual, just in case this would beon a test.” (D)

Find location Finds the place where he/she was last reading ‘‘Um, trying to find my place.” (T)Highlights text Uses highlighter on text or underlines ‘‘I’m going to highlight where the B cell is” (D)Importance of

informationStates that certain information is important or key ‘‘Ok, ah, antigen-presentation is highlighted so it must be

important” (T)Memorize States that he/she is (trying to) memorize what was read. ‘‘the diagrams is what I need to memorize and understand it

better” (D)Mnemonic Creates a verbal or visual memory aid to help remember

something from the text‘‘meditation [mediation] takes a long time and this cell-meditated response if long-term” (T)

Not thinking States that he/she is not thinking ‘‘Right now I’m not thinking anything” (T)Rereads text Re-reads five or more words in a row (in text or diagram) A foreign molecule that elicits a specific response by

lymphocytes is called an antigen. . . . A foreign molecule thatelicits a specific response by lymphocytes (T)

Paraphrasingaccurately

Accurately re-states what was read within one sentence in ownwords (not simply re-reading). Note that a sentence can be splitinto accurate (PARA+) and inaccurate (PARA!) portions

‘‘So, there’s different kinds of B cells and antigens, they hookonto one of the B cells onto its antigen receptors,” (D)

Paraphrasinginaccurately

Inaccurately re-states what was read within one sentence in ownwords (not simply re-reading). A sentence can be split intoaccurate (PARA+) and inaccurate (PARA!) portions

‘‘So T-dependent antigens can only take place with the helpof T cells” (T)

Summarizinginaccurately

Inaccurately re-states what was read/diagram in own words acrosstwo or more sentences (not simply paraphrasing one sentence orre-reading). A sentence can be split into accurate (SUM+) andinaccurate (SUM!) portions

‘‘by. . .a white blood cells. . . no, just a regular cell” (D)

(continued on next page)

J.G. Cromley et al. / Contemporary Educational Psychology 35 (2010) 59–74 71

Appendix C

C.1. Mental models coding scheme

Appendix B (continued)

Text signaling Uses bolding, italics, paragraph headings to guide comprehension(e.g., to guide note-taking, re-reading)

‘‘I am also writing down the main concepts that are basicallybolded.” (D)

Class code Definition Example (T = text, D = diagram)

VocabularyVocabulary Accurately states or recalls a definition of a term OR states that he/

she does not remember what a previously-defined word means‘‘I don’t remember what an antibody is” (T)

Word readingSelf-Corrects

WordImmediately self-corrects a full mis-pronunciation or partial mis-pronunciation of a word

‘‘In this sem – S.E.M.” (D)

Word readingerror

Mis-reads/mis-pronounces a word in a way that affects meaningwhile reading/re-reading, and without self-correcting

‘‘pereforin” for perforin (D)

Level %Pre %Post

Level 1 basic 16% 0%A. Protects from/defends/fights against diseases/infection/keeps body healthyB. Diseases caused by any of the following pathogens: viruses, bacteria, fungi, etc. (see T for HIV)

Level 2 nonspecific immune 9% 1%C. Pathogens have foreign (proteins (called antigens))D. Mucus as a defenseE. Temperature up/fever as a defenseF. Skin as a defenseG. Histamine/inflammation as a defenseH. Mentions lymph nodes/lymphatic systemDD. A person can be immune to a disease = never get it againEE. Vaccination/immunization can make a person immuneFF. Pathogens can adapt/mutate/change

Level 3 = 1 part (no functions) 33% 7%Level 4 = 2 parts (no functions) 9% 34%

I. White blood cells/lymphocytes as part (not B cells [U] or T cells [M] specifically; code for NK)J. WBCs are in blood/lymphK. Macrophages as partM. T cells as part (not specific Helper T [N] or Cytotoxic T [Q]N. Helper T (T4) cells as partT. T cells (and/or macrophages and/or immune system) affected by HIVQ. Cytotoxic T cells/killer T cells as partU. B cells as partY. Antibodies as partMentions CD4 (without Class II MHC OR helper T)JJ. Mentions Class I MHC (without CD8 OR cytotoxic T)KK. Mentions Class II MHC (without CD4 OR helper T)MM. Mentions CD8 (without Class I MHC OR cytotoxic T)

Level 5 = level 3 + at least 1 function OR Level 4 + only 1 function 21% 13%Level 6 = level 4 + at least 2 functions 6% 10%

L. Macrophages phagocytize/engulf/attack/absorb pathogensO. Helper T cells recognize/‘‘locate” antigen/pathogensR. Cytotoxic T cells secrete enzyme (perforin)S. T cells/Tc lyses/destroys pathogens/infected cellsV. B cells recognize/mark antigen/pathogensW. B cells recognize/are activated by ‘‘markers” (i.e., antibodies) on infected cellsX. B cells make antibodiesGG. Immune system (or effector/plasma) cells proliferate in response to antigenQQ. Mentions IL-1 (without PP)RR. Mentions IL2 (without PP)UU. Immune system exhibits self-tolerance; if self-tolerance fails = autoimmune diseaseVV. APCs (or B cell/T cell, etc.) ‘‘present” antigen

Level 7 = Level 6 + at least 1 from any one of the following categories 4% 12%Level 8 = Level 6 + at least 1 from any two of the following categories 1% 12%Level 9 = Level 6 + at least 1 from all three of the following categories 1% 11%

AA. Antibodies are specific to particular pathogensHH. Antibodies attach to invadersNN. Any pair or triplet from among: CD4, Class II MHC, helper TOO. Any pair or triplet from among: CD8, Class I MHC, cytotoxic TSS. Tc—cell-mediated—infected cellsTT. B—humoral—free antigen

72 J.G. Cromley et al. / Contemporary Educational Psychology 35 (2010) 59–74

References

Ainsworth, S. (2006). DeFT: A conceptual framework for considering learning withmultiple representations. Learning and Instruction, 16(3), 183–198.

Ainsworth, S., & Burcham, S. (2007). The impact of text coherence on learning byself-explanation. Learning and Instruction, 17(3), 286–303.

Ainsworth, S., & Loizou, A. T. (2003). The effects of self-explaining when learningwith text or diagrams. Cognitive Science. A Multidisciplinary Journal, 27(4),669–681.

Aleven, V. W. M. M., & Koedinger, K. (2002). An effective metacognitive strategy:Learning by doing and explaining with a computer-based cognitive tutor.Cognitive Science, 26(2), 147–179.

Alexander, P. A., Sperl, C. T., Buehl, M. M., Fives, H., & Chiu, S. (2004). Modelingdomain learning: Profiles from the field of special education. Journal ofEducational Psychology, 96(3), 545–557.

Azevedo, R., & Cromley, J. (2004). Does training on self-regulated learning facilitatestudents’ learning with hypermedia? Journal of Educational Psychology, 96,523–535.

Bartholomé, T., & Bromme, R. (2009). Coherence formation when learning from textand pictures: What kind of support for whom? Journal of Educational Psychology,101(2), 282–293. doi:10.1037/a0014312.

Bereiter, C., & Scardamalia, M. (2006). Education for the knowledge age: Design-centered models of teaching and instruction. In P. A. Alexander & P. H. Winne(Eds.), Handbook of educational psychology (pp. 695–714). Mahwah, NJ:Lawrence Erlbaum.

Berthold, K., & Renkl, A. (2009). Instructional aids to support a conceptualunderstanding of multiple representations. Journal of Educational Psychology,101(1), 70–87.

Bodemer, D., & Faust, U. (2006). External and mental referencing of multiplerepresentations. Computers in Human Behavior, 22(1), 27–42.

Bodemer, D., Ploetzner, R., Bruchmüller, K., & Häcker, S. (2005). Supporting learningwith interactive multimedia through active integration of representations.Instructional Science, 33(1), 73–95.

Butcher, K. (2006). Learning from text with diagrams: Promoting mental modeldevelopment and inference generation. Journal of Educational Psychology, 98(1),182–197.

Cain, K., Oakhill, J. V., Barnes, M. A., & Bryant, P. E. (2001). Comprehension skill,inference making ability and their relation to knowledge.Memory and Cognition,29, 850–859.

Campbell, N. A., & Reece, J. B. (2001). Biology (6th ed.). San Francisco, CA: BenjaminCummings.

Canham, M., & Hegarty, M. (in press). Effects of knowledge and display design oncomprehension of complex graphics. Learning and Instruction, 10.1016/j.learninstruc.2009.02.014.

Carlson, R., Chandler, P., & Sweller, J. (2003). Learning and understanding scienceinstructional material. Journal of Educational Psychology, 95(3), 629–640.

Cerdan, R., & Vidal-Abarca, E. (2008). The effects of tasks on integrating informationfrom multiple documents. Journal of Educational Psychology, 100(1), 209.

Chi, M. T. H. (1997). Quantifying qualitative analyses of verbal data: A practicalguide. The Journal of the Learning Sciences, 6(3), 271–315.

Chi, M. T. H., Bassok, M., Lewis, M. W., Reimann, P., & Glaser, R. (1989). Self-explanations: How students study and use examples in learning to solveproblems. Cognitive Science, 13, 145–182.

Chi, M., Feltovich, P. J., & Glaser, R. (1981). Categorization and representations ofphysics problems by experts and novices. Cognitive Science, 5(2), 121–152.

Chi, M. T. H., deLeeuw, N., Chiu, M., & LaVancher, C. (1994). Eliciting self-explanations improves understanding. Cognitive Science, 18(3), 439–477.

Crain-Thoreson, C., Lippman, M., & McClendon-Magnuson, D. (1997). Windows oncomprehension: Reading comprehension processes as revealed by two think-aloud procedures. Journal of Educational Psychology, 89(4), 579–591.

Cromley, J., & Azevedo, R. (2006). Self-report of reading comprehension strategies:What are we measuring? Metacognition and Learning, 1(3), 229–247.

Edmonds, C. J., & Pring, L. (2006). Generating inferences from written and spokenlanguage: A comparison of children with visual impairment and children withsight. British Journal of Developmental Psychology, 24, 337–351.

Fehrenbach, C. R. (1991). Gifted/average readers: Do they use the same readingstrategies? Gifted Child Quarterly, 35(3), 125–127.

Florax, M., & Ploetzner, R. (in press). What contributes to the split-attention effect?The role of text segmentation, picture labeling, and spatial proximity. Learningand Instruction, 10.1016/j.learninstruc.2009.02.021.

Fox, E. (2009). The role of reader characteristics in processing and learning frominformational text. Review of Educational Research, 79(1), 197–261.

Graesser, A. C. (2007). An introduction to strategic reading comprehension. In D. S.McNamara (Ed.), Reading comprehension strategies: Theories, interventions, andtechnologies (pp. 3–26). Mahwah, NJ, US: Lawrence Erlbaum AssociatesPublishers.

Graesser, A., Lu, S., Olde, B., Cooper-Pye, E., & Whitten, S. (2005). Question askingand eye tracking during cognitive disequilibrium: Comprehending illustratedtexts on devices when the devices break down. Memory & Cognition, 33(7),1235–1247.

Griffin, T. D., Wiley, J., & Thiede, K. W. (2008). Individual differences, rereading, andself-explanation: Concurrent processing and cue validity as constraints onmetacomprehension accuracy. Memory & Cognition, 36(1), 93–103.

Grosse, C. S., & Renkl, A. (2006). Effects of multiple solution methods inmathematics learning. Learning and Instruction, 16(2), 122–138.

Hannus, M., & Hyönä, J. (1999). Utilization of illustrations during learning of sciencetextbook passages among low- and high-ability children. ContemporaryEducational Psychology, 24(2), 95–123.

Hegarty, M., & Just, M. (1993). Constructing mental models of machines from textand diagrams. Journal of Memory & Language, 32(6), 717–742.

Hegarty, M., Kriz, S., & Cate, C. (2003). The roles of mental animations and externalanimations in understanding mechanical systems. Cognition and Instruction,21(4), 325–360.

Heiser, J., & Tversky, B. (2006). Arrows in comprehending and producing mechanicaldiagrams. Cognitive Science, 30(3), 581–592.

Kastens, K., & Liben, L. (2007). Eliciting self-explanations improves children’sperformance on a field-based map skills task. Cognition and Instruction, 25(1),45–74.

Kintsch, W. (1998). Comprehension: A paradigm for cognition. New York, NY:Cambridge University Press.

Kriz, S., & Hegarty, M. (2007). Top-down and bottom-up influences on learningfrom animations. International Journal of Human–Computer Studies, 65(11),911–930.

Laing, S., & Kamhi, A. (2002). The use of think-aloud protocols to compareinferencing abilities in average and below-average readers. Journal of LearningDisabilities, 35(5), 436–447.

Larkin, J., & Simon, H. (1987). Why a diagram is (sometimes) worth ten thousandwords. Cognitive Science, 11(1), 65–100.

Lewalter, D. (2003). Cognitive strategies for learning from static and dynamicvisuals. Learning and Instruction, 13(2), 177–189.

Linn, M. C. (2006). The knowledge integration perspective on learning andinstruction. In R. K. Sawyer (Ed.), The Cambridge handbook of the learningsciences (pp. 243–264). NY: Cambridge University Press.

Löhner, S., van Joolingen, W. R., Savelsbergh, E. R., & van Hout-Wolters, B. (2005).Students’ reasoning during modeling in an inquiry learning environment.Computers in Human Behavior, 21(3), 441–461.

Lowe, R. K. (2003). Animation and learning: Selective processing of information indynamic graphics. Learning and Instruction, 13(2), 157–176.

Mayer, R. E. (2003). The promise of multimedia learning: Using the sameinstructional design methods across different media. Learning and Instruction,13(2), 125–139.

Mayer, R. E. (2005). Cognitive theory of multimedia learning. In R. E. Mayer (Ed.),The Cambridge handbook of multimedia learning (pp. 31–48). Cambridge:Cambridge University Press.

Mayer, R., Hegarty, M., Mayer, S., & Campbell, J. (2005). When static media promoteactive learning: Annotated illustrations versus narrated animations inmultimedia instruction. Journal of Experimental Psychology: Applied, 11(4), 256.

Mayer, R., & Sims, V. (1994). For whom is a picture worth a thousand words?Extensions of a dual-coding theory of multimedia learning. Journal ofEducational Psychology, 86(3), 389–401.

McNamara, D. S. (2001). Reading both high-coherence and low-coherence texts:Effects of text sequence and prior knowledge. Canadian Journal of ExperimentalPsychology, 55(1), 51–62.

Meece, J. L., & Miller, S. D. (2001). A longitudinal analysis of elementary schoolstudents’ achievement goals in literacy activities. Contemporary EducationalPsychology, 26(4), 454–480.

Moore, P. J., & Scevak, J. J. (1997). Learning from texts and visual aids: Adevelopmental perspective. Journal of Research in Reading, 20(3), 205–223.

Moreno, R., & Valdez, A. (2005). Cognitive load and learning effects of havingstudents organize pictures and words in multimedia environments: The role of

Appendix C (continued)

Level %Pre %Post

Interactions among lymphocytesP. Helper T cells organize/‘‘mobilize” other immune cells; mention t-dependent antigensZ. T cells signal B cellsCC. Antibodies enable immune system/other cells to kill pathogens [NOT antibodies kill pathogens]PP. cytokines specifically signal Th? B or Th? Tc or Macro? Th

SecondaryBB. Antibodies/memory cells are long-lasting/have memory/body is prepared for later infectionLL. Secondary response faster and stronger

J.G. Cromley et al. / Contemporary Educational Psychology 35 (2010) 59–74 73

student interactivity and feedback. Educational Technology Research &Development, 53(3), 1042–1629.

Muis, K. (2008). Epistemic profiles and self-regulated learning: Examining relationsin the context of mathematics problem solving. Contemporary EducationalPsychology, 33(2), 177–208.

Narayanan, N. H., & Hegarty, M. (1998). On designing comprehensible interactivehypermedia manuals. International Journal of Human–Computer Studies, 48(2),267–301.

Panel, National. Reading. (2000). Teaching children to read: An evidence-basedassessment of the scientific research literature on reading and its implications forreading instruction. Washington, DC: National Institute for Child Health andHuman Development.

Neuman, S. (1990). Assessing children’s inferencing strategies. In J. Zutell & J.McCormick (Eds.), Literacy theory and research: Analyses from multiple paradigmsthirty-ninth yearbook of the national reading conference (pp. 267–274). Chicago,IL: National Reading Conference.

Otero, J., León, J., & Graesser, A. (2002). The psychology of science text comprehension.Mahwah, NJ, US: Lawrence Erlbaum Associates Publishers.

Ozuru, Y., Dempsey, K., & McNamara, D. (2009). Prior knowledge, reading skill, andtext cohesion in the comprehension of science texts. Learning and Instruction,19(3), 228–242.

Paas, F., Renkl, A., & Sweller, J. (2003). Cognitive load theory and instructionaldesign: Recent developments. Educational Psychologist, 38(1), 1–4.

Pintrich, P. (2000). The role of goal orientation and self-regulated learning. In M.Boekaerts, P. Pintrich, & M. Zeidner (Eds.), Handbook of self-regulation(pp. 452–494). San Diego: Academic Press.

Pozzer, L., & Roth, W. (2003). Prevalence, function, and structure of photographs inhigh school biology textbooks. Journal of Research in Science Teaching, 40(10),1089–1114.

Pressley, M., & Harris, K. R. (2006). Cognitive strategies instruction: From basic research toclassroom instruction. In P. A. Alexander & P. H. Winne (Eds.), Handbook of educationalpsychology (2nd ed., pp. 265–286). Mahwah, NJ: Lawrence Erlbaum.

Robertson, W. C. (1990). Detection of cognitive structure with protocol data:Predicting performance on physics transfer problems. Cognitive Science. AMultidisciplinary Journal, 14(2), 253–280.

Samuelstuen, M., & Braten, I. (2007). Examining the validity of self-reports on scalesmeasuring students’ strategic processing. British Journal of EducationalPsychology, 77, 351–378.

Sanchez, C., & Wiley, J. (2006). An examination of the seductive details effect interms of working memory capacity. Memory & Cognition, 34(2), 344–355.

Schellings, G., Aarnoutse, C., & van Leeuwe, J. (2006). Third-grader’s think-aloudprotocols: Types of reading activities in reading an expository text. Learning andInstruction, 16(6), 549–568.

Schmidt-Weigand, F., Kohnert, A., & Glowalla, U. (in press). A closer look at splitvisual attention in multimedia learning. Learning and Instruction, 10.1016/j.learninstruc.2009.02.011.

Schnotz, W., & Bannert, M. (2003). Construction and interference in learning frommultiple representation. Learning & Instruction, 13(2), 141–156.

Schwonke, R., Berthold, K., & Renkl, A. (2009). Applied Cognitive Psychology, 23(9),1227–1243. doi:10.1002/acp.1526.

Sweller, J. (2005). Implications of cognitive load theory for multimedia learning. InR. E. Mayer (Ed.), The Cambridge handbook of multimedia learning (pp. 19–30).Cambridge: Cambridge University Press.

Wade, S. E., Trathen, W., & Schraw, G. (1990). An analysis of spontaneous studystrategies. Reading Research Quarterly, 25(2), 147–166.

Weinstein, C. E., Husman, J., & Dierking, D. R. (2000). Self-regulation interventionswith a focus on learning strategies. In M. Boekaerts, P. R. Pintrich, & M. Zeidner(Eds.), Handbook of self-regulation (pp. 727–747). San Diego, CA, US: AcademicPress.

Wittrock, M. C. (1990). Generative processes of comprehension. EducationalPsychologist, 24, 345–376.

Zwaan, R., & Brown, C. (1996). The influence of language proficiency andcomprehension skill on situation-model construction. Discourse Processes,21(3), 289–327.

74 J.G. Cromley et al. / Contemporary Educational Psychology 35 (2010) 59–74