a research strategy for text desigbers: the role of headings

99

Instructional Science 14 (1985) 99-155 Elsevier Science Publishers B.V., Amsterdam Printed in the Netherlands

A RESEARCH STRATEGY FOR TEXT DESIGNERS: THE ROLE OF HEADINGS

JAMES HARTLEY and MARK TRUEMAN Department of Psychology, University of Keele, Keele, Staffordshire, ST5 5BG, U.K.

ABSTRACT

This article describes some of the issues confronting research workers who want to carry out research on the effects of"access structures" - devices which enable people to gain access to a text (such as summaries, headings, and numbering systems). Although this paper focusses specifically on the role of headings, two general points can be made: (1) the literature on any access structure is diffuse and inconclusive and (2) some order may be gained from the chaos by carrying out a series of systematic studies which replicate and build upon themselves. A third, more contentious, point is that we need descriptive and a-theoretical studies first in order to clear the ground for later theoretical work.

This article presents the results from seventeen experiments on headings which illustrate these points. The first nine experiments centre on three main variables, (1) the position of headings (marginal or embedded); (2) the form of headings (statements or questions); and (3) the nature of the task (free recall, searching unfamiliar text, and retrieval from familiar text). The results showed that the position of the headings (marginal or embedded) had no effect, but that headings in either form aided the recall, search and retrieval of information from the text used in these experiments with 14-15 year old participants.

These experiments were then replicated using a different text in a further four experiments with 1 1-12 year old participants. These experiments showed that headings aided l 1-12 year old's search and retrieval but not their recall from this second text. However, an additional study with 14-15 year olds and the second text showed that headings again aided recall with this group - suggesting a developmental trend in children's capacity to use headings as recall devices. Finally, another three experiments tested whether headings in the form of questions helped lower-ability readers more than headings in the form of statements (a hypothesis which had emerged from both the preceding and other earlier studies conducted by the authors). In these three studies, however, no significant differences were found to result from the different forms of heading.

0020 4277/85/$ 03.30 © 1985 Elsevier Science Publishers B.V.

100

Contents

1.0 Introduction 2.0 Our experiments: an overview 3.0 Recall Studies 3.1 Experiment One 3.2 Experiment Two 4.0 Search and Retrieval Studies 5.0 Search Studies 5.1 Experiment Three 5.2 Experiment Four 6.0 Retrieval Studies 6.1 Experiment Five 6.2 Experiment Six 7.0 Integrating the Studies 8.0 Recall 8.1 Experiment Seven 9.0 Search 9.1 Experiment Eight

10.0 Retrieval 10.1 Experiment Nine 11.0 Replication Studies: Another Text, Another Population 12.0 Replicating the Recall Studies 12.1 Experiment Ten 12.2 Experiment Eleven 12.3 Experiment Twelve 13.0 Replicating the Search Studies 13.1 Experiment Thirteen 14.0 Replicating the Retrieval Studies 14.1 Experiment Fourteen 15.0 Headings as Questions versus Headings as Statements with Low

Ability Pupils 15.1 Experiment Fifteen 15.2 Experiment Sixteen 15.3 Experiment Seventeen 16.0 Overall Summary and Concluding Remarks

101

1.0 Introduction

The research literature on text design is fascinating for many reasons. Some of the issues go back many years (e.g., the effects of different typefaces, typesizes and typestyles). Some are much more recent (such as notions like "considerate text") and some issues get re-named, or re-considered from fresh theoretical points of view (e.g., "access structures").

The phrase "access structures" was coined by Waller (1979) to describe a variety of features of text design that help readers to find their way around or to gain access to complex texts. These devices comprise items such as contents pages, indexes, summaries, headings, subheadings, table and figure captions, and various numbering systems (ranging from page to paragraph numbers).

The research on access structures has been discussed in more detail elsewhere (e.g., Hartley, 1985). Here we may note that, whatever the issue, it is highly likely that the research on it will be scattered, diffuse, and difficult to summarise. Different investigators will have pursued the issue in different ways, they will have used different methodologies, different participants and different theoretical perspectives. And, in most cases, there will be insufficient studies to make a meaningful summary of their results. It is hardly surprising, therefore, that it is difficult for research workers in this field to offer many precise guidelines to practitioners.

Let us consider these general points in a more specific context. If, as teachers, researchers or writers, we were to ask about the effects of headings in text, a number of issues might be raised. We could for instance ask questions about: • the presence of headings versus their absence • the effects of headings on recall or retrieval (or both) • the position of headings • the frequency of headings • the kinds of headings (e.g., questions versus statements; short versus long) • the kinds of text in which headings might appear (e.g. high versus low

structure; technical versus semi-literary) • the typographic denotation of different levels of headings • the effects of headings on immediate and long-term recall • the effects of age and ability in using headings • the effects of constructing headings for text we are reading • and so on.

Unfortunately, if we examined the research literature on these topics we would find no clear answers to our questions. Panel 1 summarises the research on headings. It is apparent from Panel 1 that: • most studies address the topic of the effects of headings on the recall of

102

information after reading the text • rather fewer studies address any other issues. Some other points, not apparent from Panel l, are that few investigators have studied more than three of the issues listed above, and no-one has investigated any of them systematically. Indeed, many of the studies cited in Panel 1 may be characterised as "one-off ' ones.

So, because of the paucity of experimental studies and their varying findings, no firm conclusions can be drawn about the value of headings for instructional text. Different investigators have used different texts, different kinds of headings, different learner samples, and different ways of testing the effectiveness of headings. Clearly, no-one is yet in a position to proffer precise guidelines of how headings in text can best be used.

What seems to be required to solve this impasse is a research strategy that focusses on carrying out a series of studies, each one of which partly replicates and partly builds upon the previous one(s) - thus constructing a more solid set of findings. This article reports the results of carrying out such a research programme.

In this programme we chose to work with headings and the three most common questions that people seemed to ask about them at that time (around 1978) - Do they have any effect? Is their position important? Does it matter how headings are phrased? When we started this research it seemed that these were important questions although we did not anticipate at the time that they would lead to quite so many experiments.

2.0 Our Experiments: A General Overview

The research suggests that headings can be used to aid recall (in memory tasks) and to aid retrieval (in search tasks) but no-one to our knowledge has examined the effects of headings on both recall and retrieval from the same text. Similarly, no-one to our knowledge has distinguished between retrieval from unfamiliar text (i.e., a search task) with retrieval from familiar text (i.e., a retrieval task). Commentators have suggested that it is easier to retrieve from text that has marginal headings (as opposed to embedded ones), and previous research had suggested that headings in the form of questions help less-able readers (e.g., Hartley et al., 1980, 1981). We therefore planned our experiments to focus on: • recall and retrieval (from unfamiliar and familar text); • the position of headings (marginal versus text embedded); and • the kind of headings used (questions versus statements).

Figure 1 illustrates the overall - or master plan of our research. We planned six initial experiments. Experiments 1 and 2 would focus on recall,

103

PANEL 1

Experiments on Headings: a Survey of Issues and Studies

1.

2.

3.

4.

5.

6.

7.

Effects of headings on recall of information Robinson and Hall (1941) Christensen and Stordahl (1955)

* Klare et al. (1958) Snavely (1962) Landry (1967) Cole (1977) Eggan et al. (1978)

* Doctorow et al. (1978) * Dee-Lucas and DiVesta (1980) * Hartley et al. (1980)

Brooks et al. (1981) * Hartley et al. (1981) * Holley et al. (1981)

Jonassen et al. (1981) Nist and Hogrebe (1984)

Effects of headings on retrieval from familiar text Charrow and Redish (1980)

Effects of headings on searching unfamiliar text * Hartley and Burnhill (1976) * Jonassen and Falk (1980)

Jonassen (1983)

Effects of different frequencies of headings * Klare et al. (1958)

Effects of different kinds of headings Christensen and Stordahl (1955)

* Doctorow et al. (1978) Proger et al. (1970) Proger et al. (1973)

* Hartley et al. (1980) * Hartley et al. (1981) * Jonassen (1983)

Effects of heading on long-term recall Christensen and Stordahl (1955)

* Hartley et al. (1980) * Hartley et al. (1981) * Holley et al. (1981)

Effects of headings on readers of different ability * Klare et al. (1958) * Hartley et al. (1980)

104

8.

9.

10.

11.

Effects of instructing readers to use headings Cole (1977) Halley et al. (1981)

* Brooks et al. (1981) * Adams et al. (1982) * Taylor (1982)

Effects of instructing readers to generate headings * Doctorow et,al. (1978)

Eggan et al. (1978) * Dee-Lucas and DiVesta (1980)

Halley et al. (1981)

Effects of heading on preferences for text * Klare et al. (1958) * Charrow and Redish (1980)

Effects of the position of headings in text (no studies located)

Experiments 3 and 4 on search, and Experiments 5 and 6 on retrieval. The variables explored in Experiments 7, 8 and 9 were dependent upon the outcome of Experiments 1-6. Experiments 10-14 replicated these studies with a different passage and with a different age group (11-12 year olds as opposed to 14-15 year olds) and Experiments 15-17 looked in detail at the effects of different types of headings with low-ability pupils.

In Experiments 1-9 we used a four page typescript version of a piece of semi-technical prose as our text material. In Experiments 10-14 we carried out

Recall 1 Expt. I Headings as statements position varied

Expt. 2 Headings as questions position varied

Recalt Expt. 7 Headings as statements versus Expts. 10 ,11,12 Replication of

headings as questions, position recall studies controlled

(unfamiliar text) "] Search Expt. 3 Headings as statements |

position varied

Expt. 4 Headings as questions position varied

Search (unfamiliar text) Expt. 8 Headings as statements versus Expt. 13 Replication of

headings as questlons, position search studies controlled

Expts. 15 ,16,17 Effects of headings as questions vs headings as statements with less-able pupils

Retrieval (famih~r text) ~] Expt. 5 Headings as statements |

position varied I Retrieval (familiar text) Expt. 9 Headings as statements versus Expt. 14 Replication of

Expt. 6 Headings as questions headin~sasquestions, posit]on retrieval studies position varied controlled

Fig. 1. Aschematicdiagramoftheresearchstrategy. Experiments 7, 8and9followedtheoutcome of Experiments 1-6. Experiments 10-14 replicated these studies with a different passage and with a different age group. Experiments 15-17 looked in detail at the effects of different types of heading with low-ability pupils.

105

:Result significant.

~ X X

X'X'XX/XX X'X

0000 IX'X~X

'X'X'XX'XXXX

XXX"XZk'X~

XXXXXXXX

XXXXXXXX

XXXXXX'XX

x .xxXJc3o~orxxxx

0000

~XDCXXX'XXXXXX X'X~r,~XX:O~O~XX X X X X X ~ X X X ~ X

Marginal headings (either questions or statements)

XXXXXXXX

XXXXXXXX

XXXX~X

XXXXXXXX

XXXXXXXX

~ x " x

)~xxx~c,£k'x

x ~ x x

Text headings (either questions or statements)

x x x x x x x x x x x x x x x x x x x x x x x x x x

x ' x x x x x x x x x x x x x x x X x x x x x x x x x

Control for marginal Control for text headings headings (text width approx. 12 cms) (text width approx. 16 cms)

Fig. 2. A schematic representation of the conditions used in the studies reported in this article,

replication studies with a second, easier text of roughly the same length. In Experiments 15-17 we varied these two texts.

Generally speaking, in all of our experiments the participants were presented with the passages of text in the conditions shown schematically in Fig. 2. (The headings were either in the form of questions or statements, making possible six versions.) The amount of text was held constant on each page, and the only thing that varied was the presence or absence of the headings. The headings were devised in order to indicate the structure of the text (i.e., they indicated the topics of the subsequent paragraphs).

We attempted to make the semantic content of the statement and the question form of the headings equivalent but we believed (perhaps naively) that the question form might indicate better the ensuing content of the text (especially for less-able readers). The following examples of headings used in the first passage indicate the nature and the style of the headings used in these experiments:

t06

• Time spent watching television How much time do people spend watching television?

• The most popular programmes What are the most popular programmes?

• Attitudes to BBC programmes How do people react to BBC programmes?

• Peoples' suggestions for improving television How do people suggest that television can be improved?

A short-answer test was attached to the back of each of the six versions of the passage. These test questions did not repeat questions asked in the headings. As noted above, the text headings indicated the structure of the text (e.g., "How do people react to BBC programmes?") whereas the test questions were more specific (e.g., "What percentage of viewers were dissatisfied with BBC 1 programmes?").

In the recall studies the participants were asked to read their passage at their own speed, and then turn over and answer the test questions. (A recall score was then derived from their performance.)In the search studies (after a practice task) the participants were asked to search for the answers to the test questions in the passage, and to circle the answers on the passage. (The time taken to complete the task was recorded.) In the retrieval studies (after a practice task) the participants were asked first to read their passage for a set period of time and they were then asked to search for the answers to the test questions in the passage, and to circle the answers on the passage (and the time taken to complete the task was recorded). A more detailed account of each experiment now follows.

3.0 Recall Studies

3.1 EXPERIMENT ONE

Aims The aims of Experiment One were (1) to see if headings written in the form

of statements improved readers' recall of factual information, and (2) to see if the position of a heading (embedded in the text or placed in the margin) affected readers' recall of information.

Participants 170 fourth-year comprehensive school pupils (aged between 14 and 15

years) of mixed-ability took part in this enquiry. The pupils were divided by the school into three groups of ability at English - high, middle and low - and were taught in separate sets. Pupils in the remedial section of the English department did not participate.

107

Materials Booklets in each of the four conditions shown in Fig. 2 (with headings in the

form of statements) were distributed systematically to the participants in two separate classes by two experimenters on each of two separate days. Each booklet contained a cover sheet, one version of the passage, and a test-sheet on the back. The passage used contained approximately 1000 words and was about television viewing habits in the United Kingdom. It had a Flesch reading ease score of 55, i.e., it was "fairly difficult" or suitable for 15-17 year olds. The passage was subdivided into twelve paragraphs, and as a report of a questionnaire, it contained a large number of facts and figures. In the headings conditions there were six headings - approximately one every two paragraphs.

Procedure The participants were asked to read the passage through once carefully, and

then when they had completed their reading to turn the booklet over and answer twelve short-answer test questions on the back. They were instructed to leave the answers blank or to guess at the answers to questions they could not complete and to go on to the next question.

In order to reduce the possibility of cheating, the participants were told that four ways of presenting the same passage were being compared, and that the experiment was concerned with testing the effectiveness of the different versions. In addition the order of the questions asked on the test-sheet was varied for each of the four conditions.

The experimental procedure used was that of a four groups design with different participants in each group.

Results The results obtained from this enquiry are shown in Table la. Analyses of

sub-groups showed: • no significant position effects for the headings: participants with marginal

headings (2 = 7.6) performed as well as participants with embedded headings (~ = 7.5);

• no significant differences between the two control passages: participants with the longer line-length (2 = 6.8) performed as well as participants with the shorter one (2 = 6.8).

Accordingly, it was deemed legitimate to pool the results for the headings groups and for the control groups. When this had been done, a two-way analysis of variance (presence/absence of headings × sex) was carried out. The main results indicated: • an overall conditions effect: participants with headings (2 -- 7.6) did better

than participants without them (x = 6.8) ( F = 4.554, df 1,166, p = 0,034); • no significant sex effects: girls (2 = 7.4) did better than boys (~ = 6.9) but this

108

TABLE Ia

The Results of Experiment 1. (Average recall scores out of 12 with (below) N in each group. Headings in the form of statements)

Boys Girls Total Overall

Headings Text 7.5 7.4 7.5 23 21 44

7.6 86 Margin 7.2 8.0 7.6

23 19 42

Controls Text 6.3 7.4 6.8 24 18 42

Margin 6.6 7.0 6.8 21 21 42

6.8 84

TABLE Ib

The Results of Experiment 1 Expressed in Terms of Ability

Boys Girls Total

High- ability Headings 9.3 9.5 9.4 14 15 29

Controls 8.1 8.3 8.2 12 15 27

Middle-ability Headings 7.3 7.4 7.4 14 16 30

Controls 6.9 6.8 6.9 15 15 30

Low-ability Headings 6.0 4.9 5.6 18 9 27

Controls 5.0 5.8 5.3 18 9 27

difference was not significant, (F = 2.015, df 1,166, p = 0.158);

• no significant interactions between these variables, ( F = 0.455, df 1,166, p = 0.501).

In the light of the data on ability obtained by Hart ley et al. 0980) (which

109

suggested that low-ability pupils profit more from headings) the data were next examined for each of the ability streams in the English department. The data obtained are shown in Table 1 b.

A three-way analysis of variance (ability × presence/absence of headings × sex), followed by Scheff6 tests when appropriate, showed: • a significant ability effect: high-ability participants (2 = 8.8) did significantly

better than those of middle-ability (2 = 7.1) and middle ability participants did significantly better than those of low-ability (2 = 5.5), ( F = 45.94, df2,158, p < 0.001);

• a significant conditions effect: participants with headings did better than participants without them ( F = 6.25, df 1,158, p < 0.02). The gain for headings for the high-ability participants was 15%, for the middle-ability it was 7% and for the low-ability it was 6%. These differences were not significant: that is, there was no significant interaction between the levels of ability and the presence/absence of headings (F = 0.830, df 2,158, p = 0.438);

• no significant sex effects ( F = 0.03, df 1,158, p = 0.856). These results appear to contradict those found by Hartley et al. (1980) but it

must be remembered that the differential effect with ability found in the earlier study lay with headings written in the form of questions and not written in the form of statements (as in this present experiment). Indeed, Hartley et al. (1980) found that a l though headings written in the form of statements helped the recall of their three ability groups, the amount of help was not related to ability.

Conclusions The results of this experiment indicate a superiority for headings written in

the form of statements but they do not indicate any differential effect regarding their position. High-ability participants recalled more than low-ability ones, but there were no significant interactions between ability and the presence or absence of headings. These findings occurred when the headings were written in the form of statements: in Experiment 2 we examine the effects of headings written in the form of questions.

3.2 EXPERIMENT TWO

Aims The aims of Experiment 2 were (1) to see if headings written in the form of

questions improved reader's recall of factual information, and (2) to see if the position of a heading (embedded in the text or placed in the margin) affected reader's recall of information.

Materials Three versions of the passage on television viewing habits were employed in

1t0

this experiment. They were as follows: • Headings written in the form of questions, embedded in the text. • Headings written in the form of questions, placed in the margin. • A control text, without headings, the same width as the text with marginal

headings (approximately 12 eros). The same twelve item short-answer test used in Experiment 1 was also used in this enquiry.

Participants 155 fourth-year comprehensive school pupils (age between 14 and 15 years)

of mixed-ability from a different school took part in this enquiry. They were divided by the school into three groups of ability at English - high, middle and low - but taught in mixed-ability groups.

Procedure The procedure was the same as that used in Experiment One except that on

this occasion three conditions were compared.

Results The results obtained from this enquiry are shown in Table 2a. Inspection of

these data suggested that it would be legitimate to pool the results from the two headings conditions, and to compare them with the control group. When this had been done a two-way analysis of variance (presence/absence of headings × sex) was carried out. The main results indicated: • an overall conditions effect: participants in the headings conditions (2 = 6.5)

recalled significantly more than participants in the control group (2 = 5.8), ( F = 6.28, df 1,151, p = 0.013);

TABLE IIa

The Results of Experiment 2. (Average recall scores out of 12. Headings in the form of questions)


Headings Text 6.8 6.4 6.6 28 26 54 6.5

104 Margin 6.8 6.1 6.4

2! 29 50

Control Margin 6.2 5.5 5.8 5.8 22 29 51 51

TABLE IIb


111

Boys Girls Total

High- ability Headings 7.2 6.6 6.9 27 21 48

Control 6.9 6.8 6.8 8 10 18


Control 6.2 5.6 5.8 6 9 15


Control 5.4 4.0 4.6 8 10 18

• a significant sex effect: boys (2 = 6.6) recalled more than girls (2 = 6.0), (F = 5.642, df 1,151, p = 0.019);

• no significant interactions between these variables, (F = 0.054, df 1,151, p = 0.816).

The results were next examined in terms of ability data obtainable from the school. The recall scores of the three ability groups (high, middle and low) were calculated. The means obtained are shown in Table 2b.

These data were subjected to a three-way analysis of variance (ability × presence/absence of headings × sex) followed by Scheff6 tests when appropriate. The results indicated: • a significant ability effect: high-ability participants (2 = 6.9) did significantly

better than those of middle-ability (2 = 6.0) and middle-ability participants did significantly better than those of low-ability (2 = 5.5), ( F = 9.223, df2,143, p < 0.001);

• a significant conditions effect: participants with headings did better than participants without headings in each of the levels of ability (with the exception of high-ability girls), (F - - 4.876, df 1.143, p = 0.029);

• a significant sex effect: boys did significantly better than girls in each of the levels of ability (except for low-ability boys in the headings condition), ( F = 4.363, df 1,143, p = 0.039);

112

no significant interactions between these three variables: nonetheless the gain for the high-ability participants was 2%, for the middle-ability it was 5% and for the low-ability it was 33%.

Conclusions The results of this experiment indicate a superiority for headings written in

the form of questions, but they do not indicate any differential effect regarding their position. High-ability participants recalled more than low-ability ones but there were no clear interactions between ability and the presence of headings.

There were two main differences between the findings reported for this experiment and those for Experiment One. 1. In this experiment the overall scores (ave. 2 = 6.3) are lower than those

reported in Experiment 1 (ave. 2 = 7.2). It is likely that this result reflects differences between the school populations used.

2. In this experiment the percentage gain for headings written in the form of statements was 2% for high-ability pupils, 5% for middle-ability and 33% for low-ability. In Experiment One the figures were 15%, 7% and 6%, respectively.

Thus this experiment (whilst not producing significant interactions) does seem to support the findings of Hartley et al. (1980) and Hartley et al. (1981) concerning the greater effects of headings written in the form of questions with low-ability participants.

Accordingly, in planning the third experiment in this series we wished to compare directly the effects on recall of headings written in the form of statements with those written in the form of questions. In the light of the results obtained in Experiments 1 and 2, it seemed reasonable to pursue this issue with headings in one position - embedded in the text. However, before we could proceed in this way, we felt it was first necessary to examine the effects of the position of headings on readers' ease of search and retrieval from the passage.

4.0 Search and Retrieval Studies

Some Problems Two previous studies that examined the effectiveness of headings as aids to

search and retrieval (Hartley and Burnhill, 1976; Charrow and Redish, 1980) both confounded the presence or absence of headings with other typographical and text changes, and therefore cannot be counted as "pure" studies of headings. In addition, Hartley and Burnhill asked students to search for material in text they had not seen before, whereas Charrow and Redish instructed their participants to read through the passages under consideration before asking them to retrieve information from it. Accordingly, we decided to study the effects of headings on retrieval in both ways, and not to confound the issue with other variables.

113

In order to measure the effectiveness of headings on search and retrieval it is necessary to measure how long it takes participants to find/retrieve material from the text. There seem to be a number of strategies for doing this (such as working with individuals and timing them separately, or group methods involving self-timing and /o r display clocks). We decided that it was best to keep things simple in a classroom situation. We planned a situation where one of the experimenters would write a number on the blackboard, and then change this number at regular intervals (e.g., every 30 seconds). The participants in the experiment would be instructed to write down the number showing at the start of the experiment and the number showing when they had completed their search task. The numbers would be presented in random order to minimise cheating.

Pilot Studies We felt it necessary to try out the viability of this approach, so we carried

out two pilot studies, one using a search task with unfamiliar text and one using retrieval with familiar text.

In order to accustom participants to the situation we devised a short practice situation. A paragraph of information on the life of Florence Nightingale was used, together with three questions. In the first pilot study 20 participants (fourth-year pupils) first read a question, found the answer to the question in the paragraph, circled it, and then did the next question. In the second study 26 fourth-years first read the paragraph and then circled the answers to the questions in order. This practice paragraph was presented without headings.

After completing the practice task, the participants repeated the task with the main passage on television viewing habits - in one of the usual three conditions (text headings, marginal headings, and control). The participants were asked to record the time they started and the time they finished the search task in the first pilot study, and the time they started reading, finished reading and completed the retrieval task in the second study. In both studies the participants were asked to find the answers to six questions.

Results of the Pilot studies The method of timing worked well and presented no difficulties. The results

obtained, however, indicated no differences between the times taken to find information in the passages with or without heading in both of the pilot studies.

In view of these results (which we recognised were clearly limited by the small sample size in each study) we decided to make a number of changes to our procedure. These changes were informative in that they suggested - indirectly - some of the factors that might affect the effectiveness of headings as tools to aid retrieval. The changes were as follows:

114

• The practice sheet was re-written. It contained a new topic (spiders), it was longer and the number of practice questions was increased from 3 to 5.

• Three versions of the practice passage were prepared- one with text headings, one with marginal headings and one without headings, and these were attached to the appropriate passages of the main experiment.

• The headings in the practice passages were written to match closely the search questions asked. Thus, e.g., a heading was "The colour of spiders" and the question was "What colour are spiders that get trapped in the bath?"

• Similarly, each of the headings in the main passage was made to reflect more clearly the phraseology of the search questions. Thus, e.g., the heading "The favourite programmes" became "The most popular kind of programmes" for the search question "What is the most popular kind of programme?".

• The number of search questions for the main task was increased from 6 to 12 (i.e., we used the same questions as we had used in the recall studies).

• Finally, accuracy was stressed by example. In the practice passage one question asked how many legs has a spider? The passage reported that insects have six, but spiders have eight. Pupils who circled six were reminded that they needed to read the passage carefully in order to ensure they circled the correct information. Similarly, other practice questions demanded careful reading of the text to obtain the correct answer.

Following these changes, we then carried out the series of search and retrieval studies which, in this report, we shall call Experiments 3, 4, 5 and 6.

5.0 Search Studies

5.1 EXPERIMENT THREE

Aims The aims of Experiment Three were (I) to see if headings written in the form

of statements helped participants to find information in text they had not seen before, and (2) to see if the position of such headings (marginal or embedded in the text) affected the readers' search times.

Materials (i) Practice passage Three versions of a (one-page) practice passage containing four paragraphs

on the topic of spiders (based on Whitlock, 1974) were prepared as follows: • Headings written in the form of statements embedded in the text

(approximately one per paragraph). • Headings written in the form of statements placed in the margin. • A control text, without headings, the same width as the text with marginal

statements.

I15

A cover sheet for the practice passage explained the procedure to be used and listed five questions which were to be used in the practice search task.

Materials (ii) The experimental passage Three versions of the passage on television viewing habits were again used in

this experiment. In this enquiry the headings were in the form of statements and only one control passage was used (the one with the shorter line-length).

The Search Questions The twelve item short-answer test used in Experiments I and 2 was modified

for use in this experiment. Students were instructed to circle on the passage the answer to the questions.

Participants Approximately 170 fourth-year comprehensive school pupils (aged

between 14 and 15 years) of mixed-ability from a third, different, school took part in this enquiry. They were divided by the school into three groups of ability at English - high, middle and low - and were taught in separate sets.

Procedure Booklets containing the practice and the experimental passages were

distributed systematically to the participants in each of six separate classes. The participants first did the practice task as a class exercise, i.e., they took each practice question in turn, and looked for and circled the answer on the practice passage. The need for accuracy was pointed out and explained, and so too was the requirement to "circle just that bit of the text that gives the answer".

After completing the practice passage, the participants were told about the need for timing the experiment, and how this was to be done. Participants were requested to complete a "time-of-starting-box" on the front cover, and, when they had finished searching for the answers to the twelve questions, a "time-of- finishing-box" below it. The participants were asked to signal (by raising their hand) when they had finished in order that one of the experimenters could check that they had recorded correctly the time of finishing. The order of the twelve questions for the search task was varied for each of the three conditions to minimise cheating, and the participants were asked to search for the answer to each question in order.

Results The data were first examined for accuracy. 143 pupils were correct.

Nineteen pupils made one error, four made two errors, and three made more than two errors. With such high accuracy the results from all 169 participants were included in the analyses of the time-data. The results for the time-data are

116

TABLE IIIa

The Results of Experiment 3. (Average search time (min.) to find the answers to 12 questions about unfamiliar text. Headings in the form of statements)


Text 13.2 13.2 13.2 31 26 57

13.0 111

Headings

Margin 13.3 12.3 12.8 29 25 54

Control Margin 15.8 14.0 15.0 15.0 32 26 58 58

TABLE IIIb


Boys Girls Total

High-ability Headings 10.2 10.5 10.4 19 23 42

Control 11.2 12.3 11.8 11 12 23


Control 15.4 15.0 15.2 10 7 17


Control 20.8 15.9 18.9 11 7 18

shown in Table 3a. Inspec t ion of these da ta suggested that it would be legit imate to pool the

results f r o m the two headings condi t ions and to c o m p a r e them with the control

group. W h e n this had been done a two-way analysis of var iance (p resence /absence of headings X sex) was carr ied out. The ma in results

indicated:

117

• an overall conditions effect: participants in the headings conditions found the answers to the questions significantly faster (2 = 13.0 min.) than participants in the control group (2 = 15.0 min.), (F = 7.91, df 1,165 p < 0.006);

• no significant sex effects ( F = 1.95, df 1,165, p = 0.164); • no significant interactions between conditions and sex ( F = 0.867, df 1,165, p

= 0.353). The results were next examined in terms of the ability groups operating in

the school. The means obtained are shown in Table 3b. These data were subjected to a three-way analysis of variance (ability × presence/absence of headings × sex). The results indicated: • a significant ability effect: high-ability participants found information

significantly faster (2 = 10.9 min.) than middle-ability (2 = 13.9 rain.) and low-ability participants (2 = 16.9 min.) and middle-ability participants were significantly faster than the low-ability participants (F = 42.84, df 2,157, p < 0.001);

• a significant conditions effect (F = 12,96, df 1,157, p < 0.001); • no significant sex effect (F = 0.745, df 1,157, p = 0.389); • no significant interactions.

Conclusions The results of this experiment indicated a superiority for headings (written

in the form of statements) but they did not indicate any differential effect regarding their position. High-ability participants searched faster than low- ability ones, but there were no interactions between ability and the presence/absence of headings. The percentage gains for the three ability groups were as follows: high-ability 14%, middle-ability 15% and low-ability 19%. To test whether these findings would be replicated with headings in the form of questions we next carried out Experiment 4.

5.2 EXPERIMENT FOUR

Aim The aim of this experiment was to replicate Experiment 3 using headings

written in the form of questions instead of headings written in the form of statements.

Materials The same materials that were used in Experiment 3 were used in this

experiment except that the headings in the practice and experimental passages were written in the form of questions.

118

TABLE IVa

The Results of Experiment 4. (Average search time (min.) to find the answers to 12 questions about unfamiliar text. Headings in the form of questions)


Headings Text 11.4 11.9 11.7 16 19 35

11.9 69 Margin 12.8 11.0 12.0

18 16 34

Control Margin 13.2 13.6 13.4 13.4 16 18 34 34

TABLE IVb


Boys Girls Total


Control 10.0 12.3 11.4 7 11 18


Control 15.6 15.5 15.6 9 7 16

Participants Approx ima te ly 115 four th-year comprehensive school pupils as before (but

f rom a four th school) t ook par t in this enquiry. There were two high-abili ty classes, one middle, and two (small) low-abili ty ones. These pupils were grouped in terms of mathematical ability, and taught in separate sets.

Procedure The procedure was the same as that used in Exper iment 3. Par t ic ipants who

made more than three errors (N = 5) were excluded f rom the analyses of t ime-data.

119

Results The results f rom this enquiry are shown in Table 4a. Inspection of these

data suggested that it would be legitimate to pool the results f rom the two headings conditions and to compare them with the control group. When this had been done a two-way analysis of variance (presence/absence of headings × sex) was carried out. The main results indicated: • an overall conditions effect: participants in the headings conditions found

the answers to the questions significantly faster (~ = 11.8 min.) than participants in the control group (2 = 13.4 min.), ( F - - 4.04, df 1,99, p < 0.05);

• no significant sex effect ( F = 0.176, df 1,99, p = 0.676); • no significant interaction (F = 0.465, df 1,99, p = 0.497).

The results were next examined in terms of the mathematics ability grouping data provided by the school. The results from the two high-ability classes were combined and compared with those from the three lower-ability classes. The means obtained are shown in Table 4b.

These data were subjected to a three-way analysis of variance (ability × presence/absence of headings × sex). The results indicated: • a significant ability effect: the high-ability participants found information

significantly faster (~ --- 10.8 min.) than the low-ability participants (~ = 14.1 rain.), (F = 25.521, df 1,95, p < 0.001);

• a significant conditions effect ( F - - 4.833, df 1,95, p < 0.03); • no significant sex effect (F = 0.07, df 1,95, p = 0.792); • no significant interactions.

Conclusions The results reported in this experiment replicated almost exactly those

found in Experiment 3. There was a superiority for headings (written in the form of questions) but no differential effect regarding their position. High-ability participants searched faster than low-ability ones but there were no interactions between ability and the presence or absence of headings. Once again, however, the low-ability participants gained more from the headings (16%) than did the high-ability ones (9%).

6.0 Retrieval Studies

6.1 EXPERIMENT FIVE

Aims The aims of Experiment 5 were (1) to see if headings, written in the form of

statements, helped participants to retrieve information from a text which they

120

had just previously read and were thus, to some extent, familiar with; and (2) to see if the position of such headings (marginal or embedded in the text) affected the participants' speed of retrieval.

Materials The same materials that were used in Experiments 3 and 4 were also used in

this experiment with one or two slight modifications. (See procedure.) The number of search questions for the main passages was reduced from 12 to 10 to reduce the time taken to complete the experiment.

Participants Approximately 100 fourth-year comprehensive school pupils as before (but

from a fifth school) took part in this enquiry. There was one high-ability class, two middle-ability ones, and one of low-ability.

Procedure Booklets containing practice and experimental passages were distributed as

before. In this experiment, however, participants first read the practice passage and then found the answers for the practice questions in a class-group exercise. Following this they were then asked to read their respective experimental passages for a period of six minutes. They were informed after four minutes had elapsed that two minutes remained, and they were instructed that, if they finished in the time available, they should look over their passage.

When the reading period was completed, it was explained to the participants how searching for answers to the questions would be timed. The participants were asked to complete a "time-of-starting-box" (at the top of the search list) and to complete a "time-of-finishing-box, (at the bottom) when they had completed the task. They were asked to signal (by raising their hands) when they had finished so that one of the experimenters could check that they had done everything correctly. Again the participants were asked to search for the answers to the questions in order. These instructions took approximately three minutes to give, so that there was this period of delay between reading the text and searching it in order to find the answers to the questions. In this experiment the numbers on the blackboard were changed every 20 seconds (instead of every 30 seconds as in the previous experiments).

Again, participants who made more than three errors ( N = 2) were excluded from the analyses of the time-data.

Results The experimental results are summarised in Table 5a. As in previous studies

inspection of this table suggests that it would be legitimate to pool the data for the two headings conditions and to compare them with the control group. A

121

TABLE Va

The Results of Experiment 5. (Average retrieval time (min.) to find 10 questions about familiar text. Headings in the form of statements)

the answers to


Headings Text 8.1 8.1 8.1 19 14 33 8.0

67 Margin 8.0 7.4 7.8

20 14 34

Control Margin 9.9 10.5 10.2 10.2 19 14 33 33

TABLE Vb


Boys Girls Total

High-ability Headings 7.4 5.8 6.6 10 10 2O

Control 8.3 9.3 8.9 4 5 9


Control 9.5 9.2 9.4 11 7 18


Control 12.6 18.0 14.4 4 2 6

two-way analysis of var iance (p resence /absence of headings × sex) was

employed . The results indicated: • an overall condi t ions effect: par t ic ipants in the headings g roups retr ieved the

answers to the quest ions significantly faster (2 -- 8.0 min.) than those in the

cont ro l g roup (2 = 10.2 min.) ( F = 14.2, df 1,96, p. < 0.001); • no significant sex effect ( F = 0.00, df 1,96, p = 0.998); • no significant interact ion ( F = 0.462, df 1,96, p = 0.492).

122

The results were next examined in terms of the school-based ability groupings. As noted above there was one high-ability class, two middle-ability ones, and one low-ability one. The means obtained are shown in Table 5b.

The data were subjected to a three-way analysis of variance (ability × presence/absence of headings × sex). The results showed: • a significant ability effect (F = 31.4, df 2,88, p < 0.001): the high-ability

participants (2 = 7.3 min.) and the middle-ability participants (2 = 8.2 min.) both retrieved information significantly faster than the low-ability ones (2 = 12.0 rain.) (Scheff6 test p < 0.01);

• a significant conditions effect (F = 25.2, df 1,88, p < 0.001); • no significant sex effect (F = 0.34, df 1,88, p = 0.561); • a significant sex × ability interaction (F = 5.49, df 2,88, p < 0.006) (a

consequence of the poorer performance of the low-ability girls); • no other significant interactions between these variables.

Conclusions The results of this experiment showed a superiority for headings (written in

the form of statements) in a retrieval task, but there was no indication of any position effect. High and middle-ability participants searched faster than low- ability ones but there were no interactions between ability and the presence or absence of headings. The percentage gain for the three groups was as follows: high-ability 35%, middle-ability 25% and low-ability 32%.

6.2. EXPERIMENT SIX

Aim The aim of this experiment was to replicate Experiment 5 using headings

written in the form of questions instead of headings written in the form of statements.

Materials The same materials that were used in Experiment 5 were used in the

experiment, except that the headings in the practice and the experimental passages were written in the form of questions.

Participants Approximately 115 fourth-year comprehensive school pupils as before (but

from a sixth school) took part in this enquiry. There were two high-ability English classes, one middle-ability, and two low-ability ones.

Procedure The procedure was the same as that used in Experiment 5 except that the

TABLE Via

The Results of Experiment 6. (Average retrieval time (min.) to 10 questions about familiar text. Headings in the form of questions)

123

find the answers to


Headings Text 8.1 7.2 7.5 12 21 33

7.5 69 Margin 7.3 7.4 7.4

13 23 36

Controls Margin 8.8 9.0 8.9 8.9 15 19 34 34

TABLE VIb


Boys Girls Total


Control 7.7 6.5 7.0 7 9 16


Control 9.7 11.2 10.5 8 10 18

high-abili ty pupils were given five minutes to read the passage, the middle-abili ty six minutes, and the low-abili ty seven minutes. Par t ic ipants who made more than three errors in f inding the answers to the questions (N = 6) were excluded f rom the analyses of the t ime-data.

Results The results obta ined are summarised in Table 6a. As before, inspection of

these data suggested that it would be legitimate to pool the data for the two headings condi t ions and to compare them with the cont ro l group. A two-way analysis of variance (presence,/absence of headings X sex) was employed. The results showed: • an overall condit ions effect: part icipants in the headings groups retrieved the

124

answers to the questions significantly faster (2 = 7.5 rain.) than those in the control group (2 = 8.9 rain.), (F = 6.11, df 1,99, p < 0.015);

• no significant sex effect (F = 0.13, df 1,99 p < 0.717); • no significant interactions (F - - 0.21, df 1,99, p = 0.646).

The data were next examined in terms of the ability groupings used by the school. The results from the two higher-ability classes were combined and compared with those obtained from the three lower-ability ones. The means obtained are shown in Table 6b.

These data were subjected to a three-way analysis of variance (ability × presence/absence of headings × sex). The results indicated: • a significant conditions effect (F = 8.93, df 1,95, p < 0.004); • a significant ability effect: higher-ability participants retrieved information

more quickly (2 = 6.3 rain.), than lower-ability ones (2 = 9.1 min.), (F = 34.53, df 1,95, p < 0.001);

• no significant sex effect (F = 0.12, df 1,95, p = 0.724); • no significant interactions between these variables.

Conclusions The results of this experiment replicated almost exactly those of

Experiment 5. Again headings (written in the form of questions) aided retrieval, but there was no indication of a position effect. The gain for the high-ability participants was 17% and for the low-ability ones it was 24%, but this interaction was not significant.

7.0 Integrating the Studies

The results from the first six experiments were clearcut. Headings helped recall, search and retrieval from unfamiliar and familiar text. These findings occurred with headings written in the form of statements or questions, and with embedded or marginal headings. In addition there was some indication that headings written in the form of questions were more helpful for less-able participants.

Because the results in all six experiments showed that the position of the headings had no clear effect, the way was now clear for us to repeat our studies without considering this variable. This meant that we could proceed to compare headings as statements with headings as questions using either embedded or marginal headings. We chose to use embedded headings on the grounds of cost-effectiveness.

Experiments 7, 8 and 9 thus replicated the previous studies on recall, search and retrieval respectively, but in each case attention was now focussed on the effects of different kinds of headings. In addition, the effects of the ability of the participants was also monitored.

125

8.0 Recall

8.1 EXPERIMENT SEVEN

Aim The aim of this experiment was to compare the effects on recall of headings

written in the form of statements with headings written in the form of questions when both forms of the headings were embedded in the text.

Materials Three versions of the passage on television-viewing habits were employed in

this experiment. They were as follows: • Headings written in the form of statements, embedded in the text. • Headings written in the form of questions, embedded in the text. • A control text, without headings, the same width as the texts with the

embedded headings. The same twelve item short-answer recall test used in Experiments 1 and 2 was also used in this enquiry.

Participants 122 fourth-year comprehensive school pupils of mixed-ability took part in

this enquiry. These pupils were divided by the school into two high-ability, two middle-ability and two low-ability classes, and taught in separate groups.

Procedure The procedure was the same as that used in Experiments 1 and 2. The

participants were asked to read through the passages once carefully, and then, when they had completed their reading, to turn over the booklet and answer the test questions on the back. The order of the questions was varied for each of the three conditions in order to minimise cheating.

Results The results obtained from this enquiry are shown in Table 7a. A two-way

analysis of variance (conditions × sex) showed: • an overall conditions effect (F = 4.78, df 2,116, p < 0.01): participants with

headings in the form of questions (2 = 7.7) and participants with headings in the form of statements (2 = 7.3) both recalled significantly more information than did participants in the control group without headings (2 = 6.5). (Dunnett 's t test, p < 0.01 and p < 0.05 respectively.) The difference between the performance of participants with headings in the form of questions (2 = 7.7) and with headings in the form of statements (x = 7.3) was not significant (Scheff6 test);

126

TABLE VIIa

The Results of Experiment 7. (Average recall scores out of 12. Embedded headings)


Headings as statements 7.6 7.0 7.3 23 19 42

7.5 83

Headings as questions 7.9 7.5 7.7 22 19 41

Control (no headings) 6.5 6.5 6.5 6.5 21 18 39 39

TABLE VIIb

The Results of Experiment 7 in Terms of Ability

Boys Girls Total

High-ability Headings as statements 8.1 7.5 7.9 11 ? 18


Control 6.5 7.4 6.9 10 7 17

Middle-ability Headings as statements 7.3 7.6 7.5 7 7 14


Control 6.6 6.8 6.7 8 6 14

Low-ability Headings as statements 6.9 5.5 6.2 5 5 10


Control 6.3 4.9 5.4 3 5 8

127

• no significant sex effect (F = 1.075, df 1,116, p = 0.302); • no significant interaction between these variables (F = 0.36, df 2,116, p =

0.702). The results were next examined in terms of the ability groupings used by the

school. The recall scores of the three ability groups were calculated, and the results are shown in Table 7b. These data were subjected to a three-way analysis of variance (ability × conditions × sex), followed by Scheff6 tests when appropriate. The analysis indicated: • a significant ability effect ( F = 6.79, df2,104, p < 0.002): Scheff6 tests showed

that both the high-ability par t ic ipants (~ -- 7.6) and the middle-ability participants (x = 7.4) recalled significantly more information than the low- ability participants (2 = 6.2), but that there was no significant difference between the recall of the high and middle-ability participants;

• a significant conditions effect ( F = 5.59, df 2,104, p < 0.005); • no significant sex effect ( F = 0.33, df 1,104, p -- 0.569); • no significant interactions between these variables. The percentage gain f rom headings for the various ability groups for headings written in the form of statements was (high-ability) 15%, (middle-ability) 12% and (low-ability) 15%. For headings in the form of questions these percentages were 17%, 16% and 26%, respectively.

Conclusions The results of this experiment replicated those found in Experiments 1 and

2. Headings, whether written in the form of statements or questions, significantly improved recall. In this experiment these effects were shown to be equally effective for both types of heading. As in Experiments 1 and 2, there was no significant interaction between the different kinds of headings and the ability of the participants but, once again, there was some indication that the gain was larger for low-ability participants with headings written in the form of questions.

9.0 Search

9.1 EXPERIMENT EIGHT

Aim The aim of this experiment was to compare the effects on search time of

headings written in the form of statements with headings written in the form of questions when both forms of headings were embedded in unfamiliar text.

Materials The same practice and experimental passages used in Experiments 3 and 4

128

were utilised in this experiment. Three versions of the passages were prepared as follows: • Headings written in the form of statements, embedded in the text. • Headings written in the form of questions, embedded in the text. • A control text without headings, the same width as the above passages. The search task posing ten questions (used in Experiments 5 and 6) was again used in this experiment.


this enquiry. The pupils were taught in mixed-ability classes. Edinburgh Verbal Reasoning Scores (collected a year previously by the school) were available for 167 of these pupils.


participants first completed the practice task before doing the main one. Each individual then searched a passage (without reading it first) in order to find and circle the answers to the ten search questions. The order of the search questions was varied for each of the three different conditions in order to minimise cheating.

Results The data were first examined for accuracy. 100 pupils made no errors, 47

made one error, 21 made two errors, six made three errors, and eleven made more than three errors. The results of these last eleven pupils were discarded.

The results for the search times obtained in this enquiry are shown in Table 8a. A two-way analysis of variance (conditions × sex) showed: • an overall conditions effect ( F = 6.59, df2,168, p < 0.002): participants with

headings in the form of questions (2 = 10.5 min.) and participants with headings in the form of statements (2 = 10.8 min.) both searched the text significantly faster than participants without headings (2 = 12.6 min.). (Dunnett 's t test: p < 0.01 in both cases.) The difference between the performance of the participants with headings in the form of questions (2 = 10.5) and with headings in the form of statements (2 = 10.8) was not significant (Scheff6 test);

• no significant sex effect ( F = 2.04, df 1,168, p = 0.155); • no significant interaction between these variables (F = 0.35, df 2,168, p =

0.703). The results were next examined in terms of the ability data provided by the

school. The 167 pupils for whom the Edinburgh Verbal Reasoning Scores were available were divided into three roughly equal-sized ability groups: high (N =

TABLE VIIIa

The Results of Experiment 8. (Average search times 10 questions with unfamiliar text)

(rain.)

129

to retrieve the answers to



10.6 117

10.8 10.2 10.5 26 33 59

12.8 12.5 12.6 12.6 27 30 57 57

Headings as questions

Control (no headings)

TABLE VIIIb


Boys Girls Total

High-ability Headings as statements 8.2 7.8 8.0 8 12 20


Control 9.2 9.8 9.5 6 6 12

Middle-ability Headings as statements 10.5 12.2 11.6 5 9 14


Control 12.4 12.2 12.3 8 11 19



Control 14.4 13.9 14.2 12 11 23

130

53), middle (N = 54), and low (N = 60). The mean search times taken by these groups in each of the three experimental conditions were calculated, and these are shown in Table 8b.

The search times were analysed in a three-way analysis of variance (ability × conditions X sex), followed by Scheff6 tests where appropriate. The results indicated: • a significant ability effect (F = 39.67, df 2,149, p < 0.001): Scheff6 tests

showed that the high-ability participants searched significantly faster (2 = 8.5 min.) than the middle-ability participants (2 = 11.4 min.) who, in turn, searched significantly faster than the low-ability ones (2 = 13.3 rain.) (both tests, p < 0.01);

• a significant conditions effect (F = 4.47, df 2,149, p < 0.013); • no significant sex effect (F = 0.41, df 1,149, p = 0.522); • no significant interaction between these variables. The percentage gain for the

various ability groups for headings written in the form of statements was (high-ability) 19%, (middle-ability) 6%, and (low-ability) 15%. For headings in the form of questions these percentages were 12%, 16%, and 7% respectively.


4. Headings, whether written in the form of statements or questions, significantly improved search time. In this experiment these effects were shown to be equally effective for both types of heading. Again, as in Experiments 3 and 4, there were no significant interactions between the different kinds of heading and the ability of the participants. In this experiment, however, unlike any of the previous ones with headings in the form of questions, there were no data to suggest that the questions gave greater help to the less-able participants.

10.0 Retrieval

10.1 EXPERIMENT NINE

Aim The aim of this experiment was to compare the effect on retrieval time of

headings written in the form of statements with headings written in the form of questions when both forms of headings were embedded in familiar text.

Materials The same practice and experimental passages used in Experiments 5 and 6

were utilised in this experiment. Three versions of the passages were prepared as follows:

131

• Headings written in the form of statements, embedded in the text. • Headings written in the form of questions, embedded in the text. • A control text, without headings, the same width as the above passages. The same search task posing ten questions as used in Experiments 5, 6 and 8 was used in this enquiry.


this enquiry. These pupils were divided by the English Department of the school into two main ability bands, a high-ability and a low-ability one. There were three classes in each band, each taught in separate groups.


participants first completed the practice task before doing the main one. Each individual then searched a passage (after having first read it) in order to find and circle the answers to the ten search questions. In this experiment both ability groups were given six minutes to read the text before searching for the required information. The order of the search questions was varied for each of the three different conditions in order to minimise cheating.

Results The data were first examined for accuracy. Participants who had made

more than three errors in retrieving the answers to the questions ( N= 3) were excluded from the analysis of the time data.

The results for the retrieval times obtained from this enquiry are shown in Table 9a. A two-way analysis of variance (conditions × sex) showed:

TABLE IXa

The Results of Experiment 9. (Average retrieval time (min.) to find answers to questions with familiar text)



7.8 92


Control (no headings) 8.1 9.8 9.0 9.0 22 24 46 46

132

TABLE IXb


Boys Girls Total


High-ability


Controls 7.9 8.1 8.0 10 15 25



Controls 8.3 12.6 10.1 12 9 21

• an overall (non-significant) conditions effect (F = 2.90, df 2,132, p < 0.06). Participants in the control group took 9.0 rain. to retrieve the information compared with 8.1 and 7.5 min. respectively for the headings as questions and headings as statements groups;

• no significant sex effect (F = 0.897, df 1,132, p = 0.345); • no significant interaction between these variables (F = 1.273, df 2,132, p =

0.283). The results were next examined in terms of the ability groupings used by the school. The retrieval times of the three high-ability classes were combined and compared with those of the three low-ability classes in a three-way analysis of variance (ability × conditions × sex).

The results indicated: • a significant ability effect (F = 26.72, df 1,126, p < 0.001): the high-ability

participants retrieved information significantly faster (2 -- 7.2 min.) than the low-ability ones (2 -- 9.5 min.);

• a significant conditions effect ( F = 3.21, df2,126, p < 0.044): the headings as statements group retrieved information significantly faster than the control group (Dunnett 's t test p < 0.05) but there was no significant difference between the performance of the control and the headings as questions groups;

• a significant sex effect ( F = 3.92, df 1,126, p < 0.05): boys retrieved information significantly faster (2 = 7.9 rain.) than girls (2 = 8.4 min.);

133

a significant sex × ability interaction (F = 10.23, df 1,126, p < 0.002): a consequence of the poorer performance of the low-ability girls; no other significant interactions between these variables. The percentage gain for the ability groups with headings written in the form of statements was (high-ability) 21% and (low-ability) 15%. The percentage gain for the ability groups with headings written in the form of questions was (high-ability) 14% and (low-ability) 5%.

Conclusions The results of this experiment replicatect those found in Experiments 5 and

6, except that they were not so clearcut. Nonetheless, participants with headings (whether they were written in the form of statements or questions) did better than participants without them. Again, as in Experiments 5 and 6, there were no significant interactions between the different kinds of headings and the ability of the participants, but (unlike Experiment 6) there was no indication that headings written in the form of questions helped low-ability participants.

U.O Replication Studies - Another Text: Another Population

Summarizing once again, the results so far suggest that headings aid recall, search and retrieval with the passage and participants used in the enquiries so far. The position of the headings (marginal or embedded), the kinds of headings (questions or statements), and differences between the sexes have not shown themselves to be important.

Whilst these findings are clear, it may well be that they are restricted to the passage and the participants used to date. We next wished to see whether we could replicate our findings, using a different passage, and a different population.

To do this we adapted a second passage, entitled "The Life of Louis Braille" from a text provided by Schonell et al. (1967). The text was approximately the same length as the television passage (i.e., approximately 1,000 words or three and a half sides of one and a half spaced typescript). The Flesch Reading Ease score for Louis Braille was 84 (i.e., "fairly easy") and it was thus suitable for 11 year olds whereas the Flesch score for the television passage was 55 (i.e., "fairly difficult"). The Louis Braille passage contained more narrative and less facts than the television passage and is perhaps best described as a semi-literary text.

Six versions of the Louis Braille passage were prepared for use in subsequent experiments. These were: • one with marginal headings in the form of statements • one with marginal headings in the form of questions • one without headings, but with text the same width as the above

134

• one with embedded headings in the form of statements • one with embedded headings in the form of questions • one without headings, but with the text the same width as the above.

The passage was subdivided into 21 paragraphs, and there were ten headings. The headings were devised by the authors with the aid of two colleagues who were simply asked - "If you had to insert headings in this text, what would you write and where would you put them?"

A ten item short-answer factual recall test was constructed to follow each version of the passage. As with the television passage, the test items were specific ones (which assisted the search and retrieval tasks), and the questions were not the same as those used for the headings (in the question format) which were more general.

The passages and questions were tried out in a pilot study (with thirty I 0-11 year old pupils) to cheek for any ambiguities or difficulties and one or two minor modifications were made to the text and the wording of the test questions.

We thus set out to replicate our recall, search and retrieval studies using this new passage with 11-12 year old participants. In these studies, however, we did not analyse the results in terms of sex differences, nor did we always contrast headings in the margin with headings embedded in the text, or headings in the form of statements with headings in the form of questions.

12.0 Replicating the Recall Studies

12.1 EXPERIMENT TEN

Aim The aim of this experiment was to replicate the recall experiments described

above. The specific aim was to see if headings written in the form of statements would significantly improve the participants' recall of factual information.

Materials Three versions of the Louis Braille passage were used in this experiment.

They were as follows: • Headings written in the form of statements (embedded in the text) • Headings written in the form of statements (placed in the margin) • A control text, without headings, the same width as the text with embedded

headings(approximately 15 cms). In order to minimise cheating, the order of the questions in the ten-item factual recall test was varied for each of the experimental conditions, and the cover and end sheets of the booklets for the different conditions were presented in different colours. When the task was done, the participants each completed the GAP reading comprehension test (McLeod and Unwin, 1970).

TABLE Xa

The Effects of Headings in the Form of Statements, on the Factual Recall of 11 - 12 year old Pupils Reading the Louis Braille Pas- sage. (Average recall scores out of 10 with (below) N in each group: headings in the form of statements)

Overall

Headings Text 7.0 31

Margin 6.7 30

6.8 61

Controls Text 6.2 6.2 41 41

135

TABLE Xb


High-ability Headings 7.9 33

Controls 6.9 18

Low-ability Headings 5.5 28

Controls 5.7 23

Participants A p p r o x i m a t e l y 100 f i rs t -year comprehens ive school pupils (aged between

11 and 12) t o o k par t in this enquiry. The pupils were taught in mixed-abi l i ty

groups.

Procedure The p rocedure was the same as tha t used in Exper imen t s l , 2 and 7. The

par t ic ipants were asked to read t h rough the passages once carefully and then, when they had comple t ed their reading, to turn over the book le t and answer the

test Questions on the back.

136

Results The results obtained from this enquiry are shown in Table 10a. An inde-

pendent sample t-test between the headings and no headings groups showed no significant difference in their recall. The headings groups recalled an average of 6.8 items (sd + 2.13) whilst the no heading group recalled an average of 6.2 items (sd + 2.12) (t = 1.41, df 199, n.s.d.).

The results were next examined in terms of ability by sub-dividing the participants into two groups on the basis of their scores on the GAP reading comprehension test. A median split of the sample at + 28 points on the GAP produced one group of 51 relatively "high" ability pupils and another group of 51 relatively "low" ability pupils. The data obtained from these two groups (shown in Table 10b) were examined in a two-way ANOVA (ability × conditions). This analysis indicated: • A significant effect for ability (F = 21.64, df 1,98, p < 0.001). High-ability

pupils recalled an average of 7.6 items whilst the low-ability pupils recalled an average of 5.6 items.

• No significant difference between the amount remembered by the headings group and the no headings group (F = 1.28, df 1,98, n.s.d.).

• No significant interaction between these variables ( F = 2.34, df 1,98, n.s.d.).

Conclusions The results of this experiment suggested that with this passage and with

these participants headings did not significantly affect recall. It was of interest to observe, however, that the headings groups did perform better than the non- headings group (although not significantly). It may be that these findings reflect differences between the passages or the participants used in these experiments. Experiments 11 and 12 described below attempted to tease out further these variables.

12.2 EXPERIMENT ELEVEN

Aim The aim of this experiment was to replicate Experiment 10 (with a larger

sample size) in order to check whether or not the results of Experiment 10 were reliable.

Materials All six versions of the Louis Braille passage were used in this experiment. As

in Experiment 10 the test and cover sheets were printed in separate colours for each of the six conditions and the orders of the test questions were varied in an attempt to minimise possible cheating. In this experiment the number of test questions was increased from ten to twelve.

137

Participants Approximately 190 first-year pupils from two comprehensive schools took

part in this enquiry. The pupils were taught in streamed classes.

Procedure The procedure was the same as that used in Experiments 1, 2, 7 and 10. The

participants were asked to read through the passages once carefully and then, when they had completed their reading, to turn over the booklet and answer the test questions on the back. When all the pupils had completed the experiment, they next completed a reading comprehension test devised by the authors. (This test was a composite of the GAP and the G A P A D O L tests devised by McLeod and Unwin, 1970, and McLeod and Anderson, 1973: it used the last half of the GAP and the first half of the GAPADOL).

Results The results (analysed in terms of the kinds of headings) are shown in Table

1 l a. A one-way analysis of variance showed that there were no significant differences between the recall scores of the three groups (headings as questions, headings as statements, and controls) ( F = 0.51, df 2,183, n.s.d.).

The results were next examined in terms of ability by sub-dividing the participants into two groups on the basis of their reading comprehension test scores. A median split of the scores produced one group of 87 relatively "high" ability pupils and another group of 97 relatively "low" ability ones. The data obtained from these two groups (shown in Table 1 I b) were examined in a two-way analysis of variance (ability X conditions). This analysis indicated: • A significant effect for ability (F = 52.97, df 1,180, p < 0.001). High-ability

TABLE XIa

The Effects of Headings on the Factual Recall of 11-12 year old Pupils Reading the Louis Braille Passage. (Average recall scores out of 12 with (below) N in each group)

Headings Questions 8.6 62

Statements 8.2 62

Overall

8.4 124

Controls (No headings) 8.2 8.2 62 62

138

TABLE XIb


High-ability Headings as questions 9.6 29

Headings as statements 9.1 31

Controls 9.7 29

Low-ability Headings as questions 7.6 33

Headings as statements 7.4 31

Controls 6.9 33

pupils recalled an average of 9.4 items whilst the low-ability pupils recalled an average of 7.3 items.

• No significant difference between the amounts remembered by the two headings groups and the no heading group (F = 0.60, df 2,180, n.s.d.).


Conclusions The findings of this study replicated those of Experiment 10, with a larger

sample size. It would appear, therefore, that headings do not assist the recall of 11-12 year old pupils with this particular passage.

12.3 EXPERIMENT TWELVE

Aim The aim of this experiment was to replicate Experiment 10 with a group of

older participants in order to check whether or not the age of the participants would have an effect when the passage was the same for both groups.

Materials Four versions of the Louis Braille passage were used in this experiment.

These were as follows: • Headings written in the form of questions (embedded in the text). • Headings written in the form of questions (placed in the margin).

139

• Two control texts with text widths to match the two experimental texts. As in Experiment 10 the test and cover sheets were printed in separate colours for each of the four conditions, and the orders of the twelve test questions were varied in an attempt to minimise possible cheating.

Participants Approximately 120 fourth-year comprehensive school pupils (aged be-

tween 14 and 15) took part in this enquiry. The pupils were taught in different ability groups.

Procedure The procedure was the same as that used in Experiments l, 2, 7, 10 and 11.

The participants were asked to read through the passages once carefully and then, when they had completed their reading, to turn their booklet over and to answer the test questions on the back. When all the pupils had completed the experiment they next completed the composite reading comprehension cloze- type described in Experiment 11.

Results The results obtained for the (combined) headings groups versus the (com-

bined) control groups are shown in Table 12a. These results were compared by using an independent t test. It was found that the headings groups recalled significantly more (2 = 9.8, sd 1.53) than the no headings groups ( 2 = 9.0, sd 1.72) (t = 2.64, df 120, p < 0.01).

The results were next examined in terms of ability by sub-dividing the

TABLE XIIa

The Effects of Headings in the Form of Questions on the Factual Recall of 14-15 year old Pupils Reading the Louis Braille Passage. (Average recall scores out of 12 with (below) Nin each group)

Overall

Headings sd

9.8 1.53

60

Controls sd

9.0 1.72

62

140

TABLE XIIb


Headings Controls

High-ability 9.8 10.0 30 28

Low-ability 9.8 8.2 30 34

participants into two groups on the basis of their reading comprehension scores. A median split of the data produced the group of 58 relatively high-ability pupils and another group of 64 relatively low-ability ones. The data obtained from these two groups (shown in Table 12b) were analysed in a two-way analysis of variance (ability × conditions). This analysis indicated: • A significant effect for ability. High-ability pupils recalled more (~ = 9.9) than

low-ability ones (~ = 8.9) (F = 12.20, df 1,18, p < 0.01). • A significant effect for conditions. Pupils with headings recalled more than

pupils without them ( F = 6.31, df 1,118, p < 0.02). • A significant two-way interaction between ability and conditions ( F = 10.55,

df 1,118, p < 0.01). Further analysis of this interaction with Scheff~ tests showed that there was no significant difference between the recall of high ability pupils with headings (~ = 9.8) or without them (~ = 10.0), but that low ability pupils with headings (~ = 9.8) recalled significantly more than low-ability pupils without them (~ = 8.2) (p < 0.01). In addition there was no significant difference between the recall of the low ability (~ = 9.8) and the high ability (~ = 9.8) with headings, but without the headings the low-ability students (~ = 8.2) recalled significantly less than the high-ability (~ = 10.0) (p < 0.01).

Conclusions The findings from this study are clearer. Despite the fact that the scores had

a restricted range (as the easier passage was used with the older pupils), the presence of headings significantly affected recall, especially the recall of the least able pupils. These findings suggest, therefore, that the difference between the findings of the studies of recall so far reported are more likely to be a consequence of age and experience than a consequence of different passages. Older pupils have gained from headings with both the easy and the difficult passages. Younger pupils have not gained from the presence of headings in the easy passage. (It was not, of course, feasible to test the younger pupils with the difficult passage.) These findings then suggest that there might be a developmental trend in the ability of children to use headings to aid recall.

141

13.0 Replicating the Search Studies

13.1 EXPERIMENT THIRTEEN

Aim The aim of this experiment was to replicate the search experiments des-

cribed above. The specific aim was to see if headings written in the form of statements would significantly improve participants' search times.

Materials Four versions of the Louis Braille passage were employed in this experi-

ment. They were as follows: • Headings written in the form of statements (embedded in the text). • Headings written in the form of statements (placed in the margin). • A control text, without headings, the same width as the text with embedded

headings. • A control text, without headings, the same width as the text with marginal

headings. The practice passages (on Spiders) were also used in this experiment.

Participants 116 first-year comprehensive school pupils (aged between ! 1 and 12) took

part in this enquiry. The pupils were taught in streamed classes.

Procedure The procedure was basically the same as that used in Experiments 3, 4 and

8. The practice passages and procedures used in those experiments were repeated here. In the search task proper, each participant searched a version of the Louis Braille passage (without having first read it) in order to find and circle the answers to ten search questions. The order of the search questions was varied for each of the experimental conditions and the question sheets were appropriately colour coded in order to minimise cheating. The times of searching were displayed on the blackboard and recorded on the answer sheets as before. After completing the search task, the participants completed the composite GAP and GAPADOL reading comprehension test.

Results The data were first examined for accuracy and the results of pupils who

made three or more errors (N = 9) were discarded. The results for the search times obtained in this enquiry are shown in Table 13a. An independent t-test between the times from the participants with headings (2 - 10.3 min.) compared with those from the participants without headings (2 = 12.2 min.) showed the

142

TABLE XIIIa

The Results of Experiment 13. Average Search Times (min.) to Find and Circle the Answers to 10 Questions in an Unfamiliar Text.

Headings Marginal 10.1 27

Embedded 10.4 26

10.3 53

Controls Marginal 12.1 25

Embedded 12.2 29

12.2 54

TABLE XIIIb



Controls 10.5 24


Controls 13.6 30

difference between them to be significant (t = 2.82, df 105, p < 0.01, one-tailed). The results were next examined in terms of ability by sub-dividing the

participants into two groups on the basis of their reading test scores. The average search times of the main sub-groups were calculated and are shown in Table 13b. These data were subjected to a two-way analysis of variance (ability × conditions). The results indicated: • A significant ability effect (F - - 21.86, df 1,103, p < 0.001). The high-ability

participants searched significantly faster (~ = 9.7 min.) than the low-ability participants (~ = 12.9 rain.).

• A significant conditions effect ( F = 5.27, df 1,102, p < 0.02). • No significant interaction between these variables (F = 0.01, df 1,103, p <

O.9O).

143


8. Headings, in the form of embedded statements, significantly improved the time taken to search unfamiliar text. These results have been found now with both passages, two different age groups, and with high and low-ability pupils, and thus they have some generality.

14.0 Replicating the Retrieval Studies

14.1 EXPERIMENT FOURTEEN

Aim The aim of this experiment was to replicate the retrieval experiments

described above. The specific aim was to see if headings written in the form of statements would improve participants' retrieval times.

Materials Four versions of the Louis Braille passage were employed in this experi-

ment. They were as follows: • Headings written in the form of statements (embedded in the text). • Headings written in the form of statements (placed in the margin). • A control text, without headings, the same width as the text with embedded

headings. • A control text, without headings, the same width as the text with marginal

headings. The practice passages (on Spiders) were also used in this experiment.

Participants Some 88 first-year comprehensive school pupils (aged between 11 and 12)

took part in this enquiry. The pupils were taught in streamed classes.

Procedure The procedure was basically the same as that used in Experiments 5, 6 and

9. The practice passages and procedures used in those experiments were repeated here. In the retrieval task proper, each participant searched a version of the Louise Braille passage after first having read it (for a period of 6 min.) in order to find and circle the answers to ten search questions. The order of the search questions was varied for each of the experimental conditions and the question sheets were appropriately colour coded. The times of the search were displayed on the blackboard, and recorded on the answer sheets as before. After completing the retrieval task the participants completed the composite GAP/GA- PADOL reading comprehension test.

144

TABLE XIVa

The Results of Experiment 14. Average Retrieval Times (min.) to Find the Answers to 10 Questions in Familiar Text.

Headings Text 6.1 19

Marginal 7.1 21

Controls Text 7.6 21

Marginal 8.2 21

6.6 40

7.9 42

TABLE XIVb



Controls 7.0 20


Controls 8.8 22

Results The data were first examined for accuracy, and the results of pupils who

made three or more errors were discarded (N = 6). The results for the retrieval times obtained in this enquiry are shown in Table 14a. A t-test between the times from participants with the headings (2 = 6.6 min.) compared with those from the participants without headings (2 10= 7.9 min.) showed the differences between them to be significant (t = 2.21, df 80, p < 0.05, two-tailed).

The results were next examined in terms of ability by sub-dividing the participants into two groups on the basis of their scores on the reading comprehension test. The average retrieval times of the main sub-groups were calculated and are shown in Table 14b. These data were subjected to a two-way analysis of variance (ability × conditions). The results indicated:

145

• A significant ability effect (F = 20.09, df 1,78, p < 0.001). The high-ability participants retrieved information significantly faster (2 = 6.1 rain.) than did the low-ability ones (2 = 8.6 rain.).

• A significant conditions effect (F = 3.80, df 1,78, p < 0.05). • No significant interaction between these variables (F- - 0.84, df 1,78).

Conclusions The results from this enquiry replicate those found in Experiments 5, 6 and

9. Headings in the form of statements significantly improved the time taken to retrieve information from familiar text. These results have been found now with both passages, and two different age groups, and with high and low-ability pupils. They thus have some generality.

15.0 A Final Consideration: Headings as Questions and Headings as State- ments with Low-ability Pupils

Previous research carried out by the author and his colleagues (Hartley et al., 1980; Hartley, et al., 1981) showed that less-able pupils recalled significantly more from text with headings written in the form of questions than they did from text with headings written in the form of statements. These early studies were limited, however, in that they used short texts (about 350 words) and small sample sizes (no more than ten in each group).

Experiment 7 in this report likewise had small sample sizes, but it did use a longer text. The results, it will be recalled, showed no significant differences between the recall of low-ability pupils from text with headings in the form of questions or headings in the form of statements.

A major problem with the studies reported so far is that the definition of low-ability has been inconsistent and somewhat crude (largely because low-ability has been a side issue). In most of the studies reported above, school designa- tions of low-ability have been accepted, and these have varied from school to school.

We attempted to circumvent this difficulty in our later studies by using the GAP, the GAPADOL and our composite reading comprehension tests. The measures obtained on such tests allowed us to group children into high and low ability on the basis of the test scores, and in future experiments they will allow us to pool children from different schools. Furthermore, the use of the GAPADOL test (with its reading age norms) wil! allow us to use more refined measures of ability with older children (see below).

Because of the earlier findings, and the intrinsic interest of the question, we carried out three further enquiries into the effects of headings written either in the form of questions or of statements on the recall of low-ability pupils. These studies are the most substantial ones on this topic known to the authors.

146

TABLE XV

The Effects of Headings as Questions versus Headings as Statements with 1 1 - 12 year old Low-ability Children Reading the Louis Braille Passage

Headings in the form Headings in the form of questions of statements

Low-ability Group 1 (Reading test scores 28-37)

Low-ability Group 2 (Reading test scores 10-27)

8.4 8.0 17 19

6.6 6.5 16 12

15.1 EXPERIMENT FIFTEEN

Aim The aim of this experiment was to compare the effects of headings written in

the form of questions with headings written in the form of statements on the recall of low-ability children.

Procedure The experiment was conducted as part ot Experiment 11. The materials,

participants and procedures were as described earlier. 64 low-ability 11-12 year old participants were extracted from the total sample on the basis of their scores on the composite reading comprehension test, and these were further divided into higher and lower low-ability groups as shown in Table 15. For these two groups the recall scores were calculated for those participants who had read the text with headings in the form of questions and those who had read the text with headings in the form of statements.

Results The results of this procedure are shown in Table 15. A two-way analysis of

variance (ability × condition) indicated: • A significant effect for ability (F = 8.87, df 1,60, p < 0.001). • No significant effect for different types of heading ( F = 0.34, df 1,60, n.s.d.). • No significant interaction between these variables (F = 0.03, df 1,60, n.s.d.).

Conclusion Headings as questions did not prove to be superior to headings as state-

ments with either of the two low-ability groups reading the Louis Braille passage.

147

15.2 EXPERIMENT SIXTEEN

Aim The aim of this experiment was to replicate Experiment 15 with older

participants and to focus directly on low-ability pupils.

Materials Four versions of the Louis Braille passage were used. These were text with

marginal or embedded headings written either in the form of questions or statements.

Participants Approximately 110 14-15 year old pupils designated as low-ability in three

comprehensive schools took part in this enquiry.

Procedure The pupils were randomly allocated into one of the four experimental

conditions. After having read the text they completed the twelve item factual recall test. (The orders of the test questions were varied for each of the four conditions, and tests and cover sheets were coloured appropriately for each condition). Following this the pupils completed the composite GAP and G A P A D O L reading comprehension test. The sample was then sub-divided into two low-ability groups (on the basis of the median test score), and groups with marginal and embedded headings in each condition combined.

Results The results obtained in this experiment are shown in Table 16. A two-way

analysis of variance (ability × condition) indicated: • A significant ability effect ( F = 10.40, df 1,104, p < 0.002).

TABLE XVI

The Effects of Headings as Questions versus Headings as Statements with 14- 15 year old Low-ability Pupils Reading the Louis Braille Passage


8.3 8.8 26 31

Low-ability Group 1 (Reading test scores 31- 50)

Low-ability Group 2 (Reading test scores 11- 30)

7.3 6.9 28 23

148

• No significant effect for the different types of heading (F = 0.02, df 1,104, n.s.d.).


Conclusion The results from this experiment (with older pupils) confirm the results

from Experiment 15. There were no significant differences in recall from headings in the form of questions or headings in the form of statements with either of the two low-ability groups.

15.3 EXPERIMENT SEVENTEEN

Aim The aim of this experiment was to replicate part of Experiment 7 again

focusing directly on headings as questions or statements with low-ability pupils.

Materials Two versions of the television viewing passage were used. These were the

text with embedded headings in either the form of questions or statements.

Participants Approximately 140 14-15 year old pupils designated as low-ability in four

comprehensive schools took part in this enquiry.

Procedure The procedure was the same as that described for Experiment 15 (except

that these pupils read the television viewing passage). Following the test, the pupils then completed the GAPADOL reading comprehension test.

TABLE XVII

The Effects of Headings as Questions versus Heading as Statements with 1 4 - 15 year old Pupils Reading the Television Viewing Passage


Low-ability Group 1 (Reading age up to 39 months below average)

Low-ability Group 2 (Reading age beyond 39 months below average)

5.5 4.9 19 23

5.3 4.9 26 14

149

Results The G A P A D O L test provides reading ages for each participant (and is

more suitable for these older participants). The average reading age of the sample was 13 years I0 months (sd + 30.6 months). This indicates that the sample was generally oflow-ab!lity. However, some forty percent ( N = 58) of its members had a reading age which was equal to or in excess of their chronological age. As these participants were clearly not of "low-ability" they were omitted from further consideration. The average reading age of the remaining 82 pupils was 11 years 11 months. Once again we were able to split this sample into two sub-samples of low-ability pupils. The result obtained from this enquiry are shown in Table 17. A two-way analysis of variance indicated: • No significant effect due to ability (F = 0.02, df 1,78, n.s.d.). • No significant effect due to the type of heading (F = !,26, df 1,78, n.s.d.). • No interaction between these variables (F = 0.01, df 1,78 n.s.d.).

Conclusions The results from this experiment confirm those reported in Experiments 14

and 15. All three experiments have shown (with both young and older participants and with easy and difficult text) no differential effects on recall resulting from headings in the form of questions versus headings in the form of statements. The only crumb of comfort that can be seen is that groups with questions do better than groups with statements (although not significantly) in five out of six comparisons. Nonetheless, the differences are so small that it would seem wise to conclude that in these experiments differences in the kind of heading have had no real effect.

16.0 Overall Summary and Concluding Remarks

Headings have aided search, retrieval and recall. The findings for recall were significant with 14-15 year old participants reading a relatively difficult text and a relatively easy one: they were not significant for 11--12 year olds reading a relatively easy text. However, headings aided search and retrieval for both age groups. The main results obtained and their associated probability values are shown in Table 18. In addition, a meta-analysis (following Glass et al., 1981) shows that the mean effect-size is 0.43. This figure suggests that the effects of headings are to move the average participant in the control group to about the 66th percentile.

Nine experiments in the series reported compared marginal with text embedded headings. None of these studies produced any significant difference. Table 19 summarizes these results. Clearly, in these experiments with these

150

TABLE XVIII

Gains for Headings: A Summary of the Results from Fourteen Experiments

Mean Mean Probability level Effect* result for result for size headings controls

Expts. 1 7.6 6.8 F = 4.56 p<0.03 0.33 2 6.5 5.8 F = 6.28 p<0.01 0.41 7 Recall 7.5 6.5 F = 4.78 p<0.01 0.60

10 scores 6.8 6.2 t = 1.41 n.s.d. 0.28 11 8.4 8.2 F = 0.51 n.s.d. 0.09 12 9.8 9.0 t= 2.64 p<0.01 0.46

Expts. 3 13.0 15.0 F = 7.91 p<0.01 0.44 4 Search 11.0 13.4 F = 4.04 p<0.05 0.38 8 times 10.6 12.6 F = 6.55 p<0.002 0.59

13 10.3 12,2 t = 2.82 p<0.05 0.46

Expts. 5 8.0 10.2 F = 14.20 p<0.001 0.68 6 Retrieval 7.5 8.9 F = 6.11 p<0.02 0.46 9 times 7.8 9.0 F = 2.90 p<0.06 0.34

14 6.6 7.9 t = 2.21 p<0.05 0.46

* The effect-size is the difference between the mean of the experimental group and the mean of the control group divided by the standard deviation of the control group. The average effect-size is a statistic introduced by Glass and his colleagues (e.g. Glass et al., 1981) for describing the average size of the difference obtained when there is a large number of studies on a particular issue which vary in size, complexity, measures used, etc. A critique of meta- analysis is provided by Slavin, 1984.

TABLE XIX

The Effect of the Position of Headings: A summary of the Results from Nine Experiments

Mean result for Mean result for marginal headings embedded headings

Expts. 1 Recall 7.6 7.5 2 6.4 6.6

10 scores 6.7 7.0

Expts.

Expts.

3 12.8 13.2 Search 4 12.0 11.7

13 times 10.1 10.4

5 7.8 8.1 Retrieval 6 7.4 7.5 times 14 7.1 6.1

151

TABLE XX

The Effects of Headings Written in the Form of Questions Compared with Headings Written in the Form of Statements: A Summary of the Results from Seven Experiments

Mean result Mean result Effect for headings for headings size as questions as statements

Expt. 7 Recall scores 7.7 7.3 0.24

Expt. 11 Recall scores 8.6 8.2 0.17

Expt. 8 Search times 10.5 10.8 0.08

Expt. 9 Retrieval times 8.1 7.5 0.24

Expt. 15 Recall scores 8.4 8.0 Included in (low ability) 6.6 6.5 Expt. 11

Expt. 16 Recall scores 8.3 8.8 -0.26 (low ability) 7.3 6.9 0.17

Expt. 17 Recall scores 5.5 4.9 0.31 (low ability) 5.3 4.9 0.20

materials and these participants, the position of the headings had no effect upon recall, search and retrieval.

The effects of headings and ability were examined in the first fourteen studies (although ability was weakly defined). In all cases groups defined as "high-ability" did significantly better than groups defined as "low-ability', and in all cases except one there were no significant interactions between the presen- se/absence of headings and ability. Experiment 12, which provided the excep- tional case, involved the older pupils reading the easier text. In this case it could be argued that the high-ability groups had no room for improvement on their test scores (with or without headings) but that the headings clearly assisted the less-able readers.

Headings as questions were directly compared with headings as statements in Experiments 7, 8, 9, 11, 15, 16 and 17, and the last three of these used low-ability participants. None of these experiments, however, produced significant differences (see Table 20). Experiments 15, 16 and 17 which focused on the recall of low-ability participants failed to replicate earlier findings which suggested that low-ability pupils profited more from headings in the form of questions than from headings in the form of statements. A meta-analysis of the results of all the studies considered in this article also indicated that there was

152

little to choose between the effectiveness of headings written in the form of questions and headings written in the form of statements. The average effect-size was 0.14.

The findings reported in this paper result from an extended series of studies, and they thus have some generality. They are of course limited by the length of the texts used (and the texts themselves) as well as by the age and ability of the participants used. In some ways, although much has been achieved, there is a long way to go yet with research on headings (and by implication, with other structural devices that aid text presentation). To our knowledge no-one has yet examined different ways of denoting headings typographically in text, or indeed examined the effectiveness of different ways of denoting text structure with (for example) main, secondary and tertiary headings. Twyman (198 I) has provided some useful suggestions in this regard which indicate the complexity of the problem.

Also, of course, we need to consider here the possible relationships between the semantic content of the headings and the structure of the text. As noted at the start of this article, we attempted to keep our different forms of headings semantically equivalent. However, one can of course formulate questions and statements in different.ways and it is not necessarily correct to assume that questions may always be more informative than statements. We are indebted to one of the referees of this article who pointed out that the statement "factor X improves Y" is more informative than the question "How does factor X affect Y?". Thus the wording of headings may turn out to be a particularly important feature in future research. Hartley and Jonassen (1985) explore these issues further, drawing attention in particular to the effects of the readers' prior knowledge, the misleading nature of some headings (which do not reflect their text structure) and to the limitations of one or two-word headings. Hartley and Jonassen also explore the role of headings in the context of word-processing and electronic text.

In conclusion, it should be observed that the work described in this article has been conducted from an applied rather than from a theoretical position. The research has not been driven by any particular notions about text structure, or by any strong views about mathemagenics or cognitive psychology. Along with others (e.g., Broadbent, 1971; Baddeley, 1981; Wright, 1983) we believe that interesting theoretical issues arise from the results of trying to answer practical questions. In our view the experiments described in this article have both practical and theoretical implications, but, as a research policy, we like to see evidence from detailed and thorough studies - such as our own - before we start to make theoretical generalisations.

This article has described a general research strategy which has been applied to a particular case. The general research strategy (of conducting a series of experiments in order to build up a reliable picture) has been applied in the

153

context of research on headings. As far as we are aware such an approach has not been widely used in the field of text design, although the studies by Dwyer (! 978) on illustrations in text provide a parallel example. True enough, even with this strategy the results obtained will be specific to the texts and to the participants used. And, as such they may be open to criticism. Our texts have been relatively short, and relatively factual, but at least we have used more than one text, and we have replicated our findings several times. Our conclusions, therefore, in this context, are much firmer than those obtained in previous "one-off" studies.

Acknowledgements

The research reported in this paper has largely been made possible by Contracts Nos. DAJA37-81-C-0277 and DAJA45-83-C-0033 from the U.S. Army Research Institute for the Behavioral and Social Sciences through its European Liaison Office at the European Research Office of the U.S. Army, London, England. The opinions expressed are those of the authors and do not necessarily represent those of the U.S. Army. We are indebted to pupils and staff in the following schools for their generous assistance: The Regis School, Tetten- hall, Wolverhampton; Clough Hall Comprehensive School, Kidsgrove, Stoke- on-Trent; Maryhill Comprehensive School, Kidsgrove, Stoke-on-Trent; Blythe Bridge High School, Stoke-on-Trent; Alsager Comprehensive School, Alsager; Marshlands High School, Newcastle-under-Lyme; The Edward Orme School; Newcastle-under-Lyme.

References

Adams, A., Carnine, D. and Gersten, R. (1982). "Instructional strategies for studying content area texts in the intermediate grades," Reading Research Quarterly XVIII (1): 27-55.

Baddeley, A. (1981). "The cognitive psychology of everyday life," British Journal of Psychology 72 (2): 257-269.

Broadbent, D. ( 1971). "The relation between theory and application in psychology," in P. B. Warr (Ed.), Psychology at Work. Hardmondsworth: Penguin.

Brooks, L. W., Dansereau, D., Spurlin, J. E., and Holley, C. D. (1981). "Instructing students on the use of headings as aids in processing scientific text." Paper presented to the AERA Convention, Los Angeles, April, 1981. Paper available from the authors, Texas Christian University, U.S.A.

Charrow, V. R. and Redish, J. C. (1980). "A study of standardised headings for warranties." Technical Report No. 6, Document Design Project, 1055 Thomas Jefferson Street N.W., Washington DC 20007.

Christensen, C. M. and Stordaht, K. E. (1955). "The effects of organisational aids on comprehension and retention," Journal of Educational Psychology 46: 65-74.

Cole, N. J. (1977). "The effects of non-prose textual characteristics on retention of maj or concepts and supporting details with and without specific instruction in their use," in P. D. Pearson and

154

J. Hansen (Eds.), Reading: Theory, Research and Practice. (26th Yearbook of the National Reading Conference) Carolina: Clemson.

Dee-Lucas, D. and DiVesta, F. J. (1980). "Learner-generated organisational aids: effects on learning from text," Journal of Educational Psychology 72:304-311.

Doctorow, M., Wittrock, M. C. and Marks, C. (1978). "Generative processes in reading comprehension," Journal of Educational Psychology 70:109-118.

Dwyer, F. M. (1978). Strategies for lmproving VisualLearning. Learning Services, Box 784, State College, Pennsylvania.

Eggan, P. D., Kauchuk, D. and Kirk, S. (1978). "The effect of hierarchical cues on the learning concepts from prose materials," Journal of Experimental Education 46: 7-10.

Glass, G. V., McGaw, B. and Smith, M. L. (1981). Meta-analysis in Social Research. Beverly Hills: Sage.

Hartley, J. (1985). Designing Instructional Text. 2nd edition. London: Kogan Page. Hartley, J. and Burnhill, P. (1976). "Explorations in space: a critique of the typography of BPS

publications," Bulletin of the British Psychological Society 29: 97-107. Hartley, J., Kenely, J., Owen, G. and Trueman, M. (1980). "The effects of headings on children's

recall from prose text," British Journal of Educational Psychology 50: 304-307. Hartley, J. and Jonassen, D. H. (1985). "The role of headings in printed and electronic text," in D.

H. Jonassen (Ed.), The Technology of Text Vol. 2. Englewood Cliffs, N J: Educational Technology Publications.

Hartley, J., M orris, P. and Trueman, M. (1981). "Headings in text," Remedial Education 16: 5-7. Holley, C. D., Dansereau, D. F., Evans, S. H., Collins, K. W., Brooks, L. and Larson, D. (1981).

"Utilizing intact and embedded headings as processing aids with non-narrative text," Contem- porary Educational Psychology 6: 227-236.

Jonassen, D. H. (1983). "Blocking and type's of headings in text: Effects on recall and retrieval." Paper presented at the annual meeting of the American Educational Research Association, Montreal, Quebec, Canada, April, 1983. Paper available from the author, School of Educa- tion, University of North Carolina at Greensboro, North Carolina, 27412, U.S.A.

Jonassen, D. H. and Falk, L. M. (1980). "Mapping and programming textual materials," Pro- grammed Learning and Educational Technology 17 (1): 19-26.

Klare, G. R., Shuford, E. H. and Nichols, W. H. (1958). "The relation of format organisation to learning," Educational Research Bulletin 37: 39-45.

Landry, D. L. (1967). "The effects of organizational aids on reading comprehension," Dissertation Abstracts International 27 (10) p. 3229.

Nist, S. L. and Hogrebe, M. C. (1984). "The effects of headings and interspersed questions on the immediate and delayed recall of text." Paper presented to the AERA Annual Conference, New Orleans, April, 1984. Copies available from the authors, Department of Psychology, The University of Georgia, Athens, GA 30602, U.S.A.

Proger, B. B., Carter, C. E., Mann, L., Taylor, R. G. and Bayuk, R. J. (1973). "Advance concurrent organizers for detailed verbal passages used with elementary school pupils," Jour- nal of Educationa ! Research 66:451-456.

Proger, B. B., Taylor, R. G., Mann, L., and Bayuk, R. J. (1970). "Conceptual pre-structuring for detailed verbal passages," Journal of Educational Reserach 64: 28-33.

Robinson, F. P. (1961). Effective Study. New York: Harper and Row. Robinson, F. P. and Hall, P. (1941). "Studies of higher-level reading abilities," Journal of

Educational Psychology 32:241-252. Slavin, R. E. (1984). "Meta-analysis in education: How has it been used?" Educational Researcher

13 (8): 6-15. Snavely, A. E. (1962). "The effectiveness of purpose statements and marginal notes as aids to

155

reading comprehension," Dissertation Abstracts International 22 (8), p~ 2711. Taylor, B. M. (1982). "Text structure and childrens' comprehension and memory for expository

material," Journal of Educational Psychology 74: 323-340. Twyman, M. (1981). "Typography without words," Visible Language VXI: 5-12. Waller, R. H. W. (1979). "Typographic access structures for instructional text," in P. A. Kolers, M.

E. Wrolstad and H. Bouma (Eds.), Processing of Visible Language. New York: Plenum. Whitlock, R. (1974). Spiders. London: Priory Press. Wright, P. (1983). "Writing and reading technical information," in J. Nicholson and B, Foss

(Eds.), Psychology Survey No. 4. Leicester: British Psychological Society.

a research strategy for text desigbers: the role of headings

Documents