Text coherence and multimedia

comprehension

Lucia Lumbelli, Gisella Paoletti, Maurizio Boscarol

Research question

Can the comprehension of a multimedia with static pictures and written text be improved by only increasing the cohesion and coherence of the text component?

Premises (1): Text cohesion and comprehension• Studies about text comprehension highlighted the

importance of the quality of a text which is called cohesion or cohesiveness (Ozuru et al.2009).

• It refers to the extent to which ideas conveyed in a text are made explicit.

• It is considered as an objective text feature which determines the coherence as a subjective state of readers, as a feature of a text representation.

• Text cohesiveness and the corresponding coherence of text representation have been shown to significantly affect the comprehension process outcomes (Mc Namara et al.1996;Graesser, Mc Namara, Louwerse & Cai 2004; O’Reilly & Mc Namara, 2007; Ozuru et al. 2009).

Premises (2): Coherence in multimedia learning• The definition of the coherence of internal

representation as a consequence of text cohesiveness seems to complete the definition of coherence chiefly adopted in the research on multimedia learning.

• A coherent document only presents the information items necessary to build the causal chains to be learned: every detail which is not related to these chains has to be eliminated if learning has to be facilitated (Mayer 2001, 2005). Relationship between ideas or concepts is considered rather than between sentences.

Premises (2): Multimedia coherence and length• Comparisons between coherent versus

incoherent multimedia are thus comparisons between shorter versus longer documents.

• The outcome is that more coherent and shorter multimedia give rise to better learning than less coherent and longer ones (Mayer, Heiser & Lonn, 2001; Moreno & Mayer, 2001; Mayer& Moreno, 2002; Harp & Mayer, 1997, 1998).

Premises (3)

Two main features of text cohesiveness are assumed to contribute to an accurate definition of coherence as a subjective state or as an attribute of internal representation:

• all information items necessary to reconstruct the causal connections are made explicit; no inference from prior knowledge is required,

• connectives like conjunctions and anaphors which have the function of connecting two adjacent sentences are integrated with information; this frees readers from the need to draw bridging inference.

Hypothesis

• If learning as establishing connections between ideas can be assumed also as the outcome of text comprehension,

• and if text cohesion is assumed to improve text comprehension by helping readers construct coherent internal representation,

• then by manipulating the text component of a multimedia so as to make it as much cohesive as possible by making all information necessary to understand explicit,

• we will give rise to an easier and better comprehension of the text and consequently of the whole multimedia.

Text revision as germane load

• Our revision might increase the ‘germane cognitive load’ related to the comprehension process, so adjusting to text comprehension a concept which has been defined as regards the learning process:

• explicitation of information which readers have to necessarily infer is an addition which should facilitate text comprehension although it makes the text longer.

The experiment

• Thirty-four participants read on a computer screen a multimedia presentation on lightening formation already used in many investigations (Mayer, 2001).

•Two text versions were compared:- the original version with information items only partially explicit and some connectives requiring integration;- the revised version with all information items explicitly stated and no integration required.

Participants

Two groups of 17 participants matched on a standardisedreading comprehension test scores

• Mean age= 24• Females 30, Males 12.

• Low specific prior knowledge � no ‘expertise reversal effect’expected

• Good reading comprehension ( MT scoring : Mean= 16.47619; St. Dev.= 2.987224, range= 0-20) � no negative effect of the greater length of text expected

Material for the experiment

The experimental material has been already used in many published experiments (Mayer, 2001, 2005).

It included 14 static pictures depicting the main steps of the process of lightening formation, each integrated with a brief written description.

Warmed moist air near the earth’s surface rises rapidly.

Characteristics of manipulationsWe only added information which was

implicit in the original text and/or which had to be necessarily inferred from it in order to form a coherent representation.

• No redundancy in the sense of repeated words and sentences

• No unnecessary details• No learning measures (Effects are

expected on the online comprehension process rather than on learning).

First type of addition: improvementof cohesion

By facilitating the connection between two adjacent sentences either by making the use of a connective (e.g. anaphora) easier, more effortless, or by making explicit an information item which in the original text should be inferred in order to link the new sentence with the previous one.

In this case the text manipulation has effects on the readers’representation of text-base, while accepting the definition of text-base according to which it includes the information item which should be necessarily inferred from the explicit one.

An example of the first type of addition

Warmed moist air near the earth’s surface rises rapidly.

As this updraft cools...

Warmed moist air near the earth’s surface rises rapidly so forming an updraft.....As this updraft cools...

Comment on the example

In the original version we find an anaphora (‘in this updraft’) which has to be connected to the phrase ‘rises rapidly’ by means of the following bridging inference:

since the air rises rapidly an updraft is formed

�The completed text of the revised version:Warmed moist air near the earth’s surface rises

rapidly so forming an updraft

Second type: improvement of coherence

By explicitly stating a causal sequence which otherwise readers should infer from their prior knowledge in order to obtain a coherent representation of the text.

In this case the text manipulations should directly affect the readers’ representation of situation model.

An example of the second type of addition

Warmed moist air near the earth’s surface rises rapidly.

As the air in this updraft cools...

Warmed moist air near the earth’s surface rises rapidly so forming an updraft.This updraft is formed because the warmer air is lighter and therefore tends to rise.

As the air in this updraft cools.....

Comment on the example

This integration should help readers usetheir prior knownledge to build the relationship between:

becoming warmer and becoming higher

which explains why the updraft builds

Procedure • Participants were asked to pay a special attention to the

chain of cause-and-effects described in the presentation.• After reading the multimedia presentation, the

participants were asked to write what they remember about the process described there with a new instruction in which the invitation to pay attention to the link between the events described was repeated.

• Recall was coded through two systems of analysis:– the one described by Mayer (2001) with 8 isolated propositions,– the second one produced ad hoc for the experiment.

The scoring system

Since comprehending a text means establishing and maintaining coherence between sentences (Oakhill, Cain & Bryant, 2003; van den Broek1994), the scoring has to reflect the extent to which the consequential connections which are always explicit in the revised text and only partially explicit in the original one have been comprehended/written by participants.

An example of the scoring system

1. Cool air becomes warmer when it draws near to the ground

2. since the air becomes warmer, it becomes lighter

3. the air tends to rise because it is lighter4. since the air rises, it brings about an

updraft.

Near to the ground � warmer

Warmer � lighter

Lighter � tends to rise

Air rises � updraft

The participants’ protocols about this initial event of the process described in the multimedia were scored from zero to 4 (score 1 to each correct answer, score 0 to each incorrect one).

The example corresponds to the initial one of Mayer’s scoring items (‘air rises’):

The consequential connections into which the updraft formation (first moment of the lightening formation) is articulated:

Examples of scoring writtenprotocols

• N.1. When cool and moist air meets warmground, it tends to become warmer(1) and therefore becomes lighter(2) and tends to rise(3) forming an updraft(4).score: 4

• N.2. Cool and moist air becomes warmer bynearing to a warm surface(1).It tends to rise and so becomes cooler(updraft).score: 1

Instructions

Instruction 1

You will be presented with a text which describes the lightning formation. The text is accompanied by pictures about the same subject. You should carefully read the text and look at the pictures so as to understand how and why the successive events there described occur (take place, happen)

Instruction 2

Now you should write down what you can find into your mind about the text you read and the pictures you looked at. You should explain as exhaustively as possible all stages of lightning formation.

Results

• The recall performance was analyzed in a one-way ANOVA with text-version as between subject factor.

• A significant main effect of text revision on comprehension of consequential connections (F=1,32= 4,8346, p<0.05) was found in a direction consistent with our expectation, with a significant improvement of the performance for the participants who examined the revised version of the text (mean score=6,6; d.s.=2,45) vs. those who had the original version (mean score=4,6; d.s.=2,69).

Comparison between original vsrevised text comprehension

Results (2)

• No effect of the revision on the recall analysed by means of Mayer’s list of isolated propositions (Fig. 4) was found: the increase in the amount of items of information, which might be expected to be a source of extrinsic cognitive load, was counterbalanced by the enhancement of text coherence.

Comparison according to Mayer’s scoring system