Reasoning in lexical tasks

during reading: Effects on recognition memory and L2 development

Ana Martínez-FernándezDepartment of Spanish and Portuguese

Georgetown University

TBLT 2009

Introduction

• SLA research on incidental vocabulary learning• SLA research on task-based learning• Cognitive psychology research on recognition memory

L2 development

Deep / cognitive processing

Task demands

Long-term recognitionmemory?

Literature review I

• The Involvement Load Hypothesis (Laufer & Hulstijn, 2001) Tasks inducing higher involvement lead to higher

vocabulary retention ‘Need’ (N) ‘Search’ (S) ‘Evaluation’ (E)

Conflicting results Hulstijn & Laufer (2001) Martínez-Fernández (2008)

Glosses

[+N, -S, -E]

Fill-in-the-blanks

[+N, -S, +E]

Literature Review II

• No control for the distance between the targeted words to fill in the blanks and distracters

• Hulstijn & Laufer (2001) No process measures to ensure that tasks led to the

involvement predicted

• Martínez-Fernández (2008) Think-aloud protocols No control for reactivity

Literature review III

• The Cognition Hypothesis (Robinson, 2001, 2005, 2007) Task complexity is determined by cognitive factors

This prediction has not been tested in the area of incidental vocabulary learning and text comprehension

Complex tasks with features that direct the cognitive effort to linguistic forms

Noticing and long-term retention

[+/- Reasoning]

Literature review IV

• No studies investigating the effects of reasoning on memory Tulving (1983) distinguished two subsystems of recognition memory

• Studies show that deep processing enhances episodic memory, which may have a positive effect on learning (Gardiner, 2008)

Long-term declarative memory

Episodic memory

Semantic memory

Experience of

remembering

Experience of knowing

Research questions

• Does the extent of reasoning involved in a reading comprehension task plus fill-in task have a differential effect on recognition memory?

• Does the extent of reasoning involved in a reading comprehension task plus fill-in task have a differential effect on vocabulary development?

• Does the extent of reasoning involved in a reading comprehension task plus fill-in task have a differential effect on text comprehension?

Research Design

• Independent variables Between-subjects: Extent of reasoning

[+R] [-R] Control

Within-subject: Time: Immediate posttest, 1-week delayed posttest Type of word: Targeted words, alternate words

• Dependent variables Recognition memory Vocabulary development: production & recognition Text comprehension: local ideas & global ideas

[+R, +TA], [+R, -TA][-R, +TA], [-R, -TA]

Participants

• 71 English-speaking adults enrolled in college-level second-year Spanish language courses

• Original pool: N = 147; participants were excluded from the final sample for any of the following reasons:

They failed to attend all sessions They did not follow the instructions They had prior knowledge of one or more targeted words They had had outside exposure to the targeted words either

between the pretest and the treatment or between the treatment and the delayed posttest

Experimental Texts and Items

• Experimental text Fairy tale in Spanish (“El verdadero valor del anillo”

in Déjame que te cuente, by Jorge Bucay) 646 words

• Targeted items 12 unfamiliar nouns (after excluding 4)

6 targeted words (3 concrete and 3 abstract) 6 alternate words (3 concrete and 3 abstract)

Experimental Task I

[+ Reasoning] Participants read a text containing 8 blanks. To fill in

each blank they had to choose one of two glossed words. Only one was correct based on specific contextual clues.

The man looked into the distance and pointed at (1) ______________ with his hand. “You will find him there,” he said. When Fernando got there, he saw the wise man standing outside, wearing ripped pants and an old shirt. He looked quite poor.

(1) - un cortijo: an estate- una chabola: a shack

Experimental Task II

• [- Reasoning] Participants read the same text and filled in the blanks. One of the two options was clearly incorrect

because it had the opposite lexical feature [+/- concrete] to that selected by the previous word.

Both tasks were pilot-tested with two native speakers and three students of Spanish

The man looked into the distance and pointed at (1) ______________ with his hand. “You will find him there,” he said. When Fernando got there, he saw the wise man standing outside, wearing ripped pants and an old shirt. He looked quite poor.

(1) - la desidia: the apathy- una chabola: a shack

Treatment

Random assignment to groups

[+R, +TA] N = 15 [+R, -TA] N = 16 [-R, +TA] N = 13 [-R, -TA] N = 14 [Control, +TA] N = 13

Oral instructions to think aloud Information about subsequent comprehension questions Time on task was measured but not controlled

Assessment Tasks and Procedure I

Week 2: Treatment + Immediate

Posttest

Week 3:Delayed Posttest

Multiple-choice

meaning recognition

test(32 items)

Text comprehension test (12 open

questions: 6 global and 6 local)

Oral + written instructions for the

memory judgment task followed by the task

Memory

judgment task

Post-debriefing questionnaire

Week 1: Pretest

Vocabulary tests (32 items)

-Meaning production-Meaning recognition

Vocabulary tests

- Production- Recognition

Assessment Tasks and Procedure II

• Instructions for the memory judgment task:

Definitions and examples 4 participants were asked to think aloud Items: 6 targeted words, 6 alternate words, 6 distractors

• All tasks were timed• Items were presented one by one with a Power Point

presentation• Participants provided their answers in an answer sheet

YES / NOIF YES: R / K / G

Analysis and Results

• One-way ANOVA on time on task: F(4, 71) = 8.698, p = .00 [+TA] groups spent significantly more time on task than [-TA] groups

• One-way ANOVA on performance on the fill-in task: F(3, 58) = 10.854, p = .00 [-R] groups significantly outperformed [+R] groups

• Index of performance quality on the remember-know task Hits minus false alarms

• Reliability of tasks: Cronbach’s alpha Text comprehension = .71 Meaning production = .57 Meaning recognition = .39

Results: Recognition Memory

• Data were submitted to 5 x 2 x 3 Repeated measures ANOVA

ItemsEffects

Targeted words

Alternate words

Time .00* .01*

Response .00* .20

Group .44 .03*

Time x Group .66 .94

Response by Time .30 .14

Response x Group .24 .60

Time x Response x Group .02* .73

Results: Recognition Memory (Targeted Words)Results: Recognition Memory (Targeted Words)

-5%

0%

5%

10%

15%

20%

25%

Remember Know Guess

Immediate

1-week-delayed

Response by Time for Targeted Words (TW)

Results: Recognition Memory (Targeted Words)Results: Recognition Memory (Targeted Words)

0%

5%

10%

15%

20%

25%

30%

35%

[+R, +TA] [+R, -TA] [-R, +TA] [-R, -TA] [C, +TA]

R

K

G

Response by Group for TW on the immediate posttest

-10%

-5%

0%

5%

10%

15%

20%

25%

30%

35%

[+R, +TA] [+R, -TA] [-R, +TA] [-R, -TA] [C, +TA]

Response by Group for TW on the delayed posttest

Results: Recognition Memory (Alternate Words)Results: Recognition Memory (Alternate Words)

0%

5%

10%

15%

20%

25%

[+R, +TA] [+R, -TA] [-R, +TA] [-R, -TA] [C, +TA]

RKG

Response by Group for AW on the immediate posttest

-5%

0%

5%

10%

15%

20%

25%

[+R+TA] [+R -TA] [-R +TA] [-R -TA] [C +TA]

Response by Group for AW on the delayed posttest

Results: Vocabulary Development

TestsEffects

Meaning production

Meaning recognition

Time .00* .00*

Word type .00* .07

Group .00* .00*

Time x Group .00* .00*

Time x Word type .01* .20

Word type x Group .04* .14

Time x Word type x Group .00* .31

o Data on vocabulary were submitted to 5 x 2 x 3 Repeated Measures ANOVA

Results: Meaning Production (TW)

Results: Meaning Production (AW)

+R, +TA

+R, -TA

Results: Meaning Recognition (TW)

Results: Meaning Recognition (AW)

+R, +TA

+R, -TA

Results: Text Comprehension I

• Data on text comprehension were submitted to 5 x 2 Repeated measures ANOVA

TestsEffects

Text comprehension

Idea Type .00*

Group .00*

Idea Type x Group .00*

Results: Text Comprehension II

Results: Qualitative Data I

• [+R] conditions Considered both options given to fill in the blanks Continued reading to find clues in the context, and test their

hypotheses

“I think it’s un cortijo…[7 lines below] You know what? This man is very poor, he would never have an estate, he has a shack… chabola”

“clumsiness or weakness, both could work [continues reading]

Oh, he breaks everything. He’s clumsy, not weak” ”

Results: Qualitative Data II

• [-R] conditions Do not verbalize the alternate words Reasoning refers to common knowledge of the world

• Control condition Skipped targeted unfamiliar words Commented lack of knowledge Contextual guessing

“he is pointing at a shack, chabola”

“weakness doesn’t make sense, so it’s gonna

be horse, because he’s gonna take something

and the weakness is not something you can take”

Results: Qualitative Data III

• R: I remember because I remember reading it on a side, and also because I wasn’t sure if it was pronounced “real” or “rial” so I remember thinking about that

• K: I think it was… I don’t remember where but I’m pretty sure that it was in there; Yes, I don’t remember where I saw it, but I know it was in there

• G: It’s a pure guess, I don’t remember that word, period; Yes, I am guessing, maybe when Fernando was in the market

Results: Qualitative Data IV

• Some Know experiences may be interpreted as guessing: I feel like that was in there, I almost think that it was one of

the words in the margin but I am not very sure; Yes, but I’m guessing

• Some Remember experiences may be interpreted as knowing: Yes, it was in the text, I remember reading it, I remember

being unsure of what it meant, but I can’t remember where in the text it was. So I guess I know it was in the text, but I don’t remember where

Discussion I

• RQ 1: Effect of [+/-R] on recognition memory Overall, the [+R] groups remembered more TW than the

other groups Thinking aloud while performing a complex task may help maintain

episodic memories over a period of a week There was a significant decrease of R responses in the [-TA] group on

the delayed posttest, and a significant increase of K responses• Remember-to-know shift (Knowlton & Squire, 1995; Conway et al.,

1997)

The [+R] groups remembered significantly more AW than the other groups, and this effect was mantained over a period of a week No reactivity for thinking aloud on immediate or delayed posttests

Discussion II

• RQ 2: Effect of [+/-R] on vocabulary development

Only the [+R] groups significantly outperformed the control group, due to their higher performance on AW Even though performance on the fill-in task was significantly lower

in the [+R] conditions Cognitive complexity did not affect time on task Thinking aloud and time on task did not affect performance

on the posttests Support for previous SLA studies on reactivity (Leow & Morgan-

Short, 2004; Bowles & Leow, 2005; Bowles, 2008; Medina, 2008; Yoshida, 2008)

Discussion III

The [+R] effect was not retained after one week, in contrast to what the Cognition Hypothesis predicts Consistent with incidental vocabulary learning studies (Watanabe,

1997; Bowles, 2004) Studies where multiple encounters with the targeted words are

provided show significant differences between task conditions on delayed posttests (Rott, 2005; Martínez-Fernández, 2008)

The superiority of [+R] group cannot be explained by the Involvement Load Hypothesis, since both tasks have the same involvement load: [+N, -S, +E]

Discussion IV

• The [+R] groups: Remembered significantly more words, especially alternate words, than

the [-R] groups and the control group Produced and recognized the meaning of significantly more words,

especially alternate words, than the control group

• It may be hypothesized that the meaning of those words that received remember responses were produced and recognized to a greater extent than those that received know or guess responses

Discussion V

• RQ 3: Effect of [+/-R] on text comprehension All groups reached high comprehension of global ideas All experimental groups reached significantly higher

comprehension of local ideas than the control group Learners can perform this type of dual task (i.e., a lexical task

embedded in a reading comprehension task) with different degrees of cognitive complexity, without negatively affecting text comprehension

Limitations and future research

• Increase the validity of the remember-know task

• Refine vocabulary learning measures Low reliability

• Refine the design of fill-in tasks Think-aloud protocols showed that not all blanks in the fill-in task led to

reasoning and evaluating to the same degree in the complex condition • Future studies should

Investigate the interaction between cognitive complexity and amount of exposure

Compare the effect of fill-in tasks and glosses in reading comprehension tasks, controlling for the effect of cognitive complexity

Investigate the role of recognition memory in L2 development Examine the effects of thinking aloud on recognition memory and L2

learning

References

Bowles, M. A. (2004). L2 glossing: To CALL or not CALL. Hispania, 87(3), 541-552. Bowles, M. A. (2008). Task type and reactivity of verbal reports in SLA: A first look at a L2 task

other than reading. Studies in Second Language Acquisition, 30(3), 359-387. Bowles, M. A. & Leow, R. P. (2005). Reactivity and type of verbal report in SLA research

methodology. Studies in Second Language Acquisition, 27(3), 415-440. Conway, M.A., Gardiner, J. M., Perfect, T.J., Anderson, S.J., & Cohen, G.M. (1997). Changes

in Memory Awareness During Learning: The Acquisition of Knowledge by Psychology Undergraduates. Journal of Experimental Psychology: General, 126,393-413.

Gardiner, J.M. (2008). Remembering and Knowing. In Byrne, J. H. (Ed.), Learning and memory: A comprehensive reference, Vol. 2, pp. 285-305. UK: Academic Press.

Knowlton, B. J. & Squire, L. R. (1995). Remembering and Knowing: Two Different Expressionsof Declarative Memory. Journal of Experimental Psychology: Learning, Memory, and Cognition, 21(3), 699-710.

Laufer, B., & Hulstijn, J. H. (2001). Incidental vocabulary acquisition in a second language: the construct of task-induced involvement. Applied Linguistics, 22(1), 1-26.

Leow, R. P., & Morgan-Short, K. (2004). To think aloud or not to think aloud: The issue of reactivity in SLA research methodology. Studies in Second Language Acquisition, 26(1), 35-57.

• Martínez-Fernández, A. (2008). Revisiting the involvement load hypothesis: Awareness, type of task and type of item. In M. Bowles, R. Foote, S. Perpiñán, & B. Rakesh (eds.), Selected Proceedings of the 2007 Second Language Research Forum (pp. 210-228). Somerville (MA): Cascadilla Proceedings Press.

• Medina, A. (2008). Concurrent verbalization, task complexity, and working memory: effects on L2 learning in a computerized task. Unpublished dissertation, Georgetown University, Washington, DC.

• Robinson, P. (2001). Task complexity, cognitive resources, and syllabus design: A triadic frameworkfor examining task influences on SLA. In P. Robinson (Ed.), Cognition and Second Language Instruction, p. 237-318. Cambridge: Cambridge University Press.

• Robinson, P. (2005). Cognitive complexity and task sequencing: A review of studies in a Componential Framework for second language task design. International Review of Applied Linguistics, 25, 45-73.

• Robinson, P. (2007). Criteria for grading and sequencing pedagogic tasks. In M. P. García Mayo (Ed.), Investigating Tasks in Formal Language Learning (p. 7-27). Clevedon: Multilingual Matters.

• Rott, S. (2005). Processing glosses: A qualitative exploration of how form-meaning connections are established and strengthened. Reading in a Foreign Language, 17(2), 95-124.

• Tulving, E. (1983). Elements of episodic memory. Oxford: Oxford University Press.• Watanabe, Y. (1997). Input, intake, and retention: Effects of increased processing on incidental

learning of foreign language vocabulary. Studies in Second Language Acquisition, 19(3), 287-307.• Yoshida, M. (2008). Think-aloud protocols and type of reading task: The issue of reactivity in

L2 reading research. In M. Bowles, R. Foote, S. Perpiñán, & Bhatt, R. (Eds.) Selected Proceedingsof the 2007 Second Language Research Forum, (pp. 199-209). Somerville, MA: Cascadilla Proceedings Project.

Thank you!