optimizing computer-based second language vocabulary learning
TRANSCRIPT
Optimizing computer-based second language vocabulary learning
Tatsuya NakataKansai University, Japan
This research was supported in part by Grant-in-Aid for Young Scientists (A) #16H05943 and Grant-in-Aid for Research
Activity Start-up (#15H06746) from MEXT.
Abstract• This presentation will examine how we can optimize
computer-based vocabulary learning in a second or foreign language (L2). Specifically, we will discuss empirical studies examining the effects of the following factors on vocabulary learning: studying vs. testing, part vs. whole learning, massed vs. spaced learning, equal vs. expanding spacing, immediate vs. delayed feedback, and single vs. repeated retrievals. These studies can recommend to us the optimal way to learn vocabulary from computer software.
• Flashcard (word card) learning: L2 on one side and meaning (L1 translation or definition) on the other.
Initial presentation
Retrieval
Feedback (Correct answer)
Feedback (Correct answer)
Outline
1. Effects of studying and testing (Karpicke & Roediger, 2008)
2. Effects of equal and expanding spacing (Nakata, 2015)
3. Effects of part and whole learning (Nakata & Webb, 2016)
4. Effects of single and repeated retrievals (Nakata, in press)
5. Q&A
Outline
1. Effects of studying and testing (Karpicke & Roediger, 2008)
2. Effects of equal and expanding spacing (Nakata, 2015)
3. Effects of part and whole learning (Nakata & Webb, 2016)
4. Effects of single and repeated retrievals (Nakata, in press)
5. Q&A
Effects of studying and testing
Karpicke, J. D., & Roediger, H. L. (2008). The critical importance of XXX for learning. Science, 319, 966-968.
Method
• Participants: 40 American undergraduate students • Materials: 40 Swahili-English word pairs (e.g.,
chakura- food)• Independent variable: Number of study and test
episodes Study : chakura = food Test : chakura = ??? * No correct answer (e.g., food) was provided as
feedback after each test.
4 Conditions
* No correct answer (e.g., chakura = food) was provided as feedback after each test.
# of Study episodes
# of Test episodes
1. Control 1.94 1.942. Repeated study 4 2.083. Repeated test 1.92 44. Repeated study + Repeated test 4 4
Method
• Dependent variable: One-week delayed meaning recall test (e.g., chakura = ???)
• Please rank the following four conditions in order of effectiveness (meaning recall 1 week after learning):
# of Study episodes
# of Test episodes
1. Control 1.94 1.942. Repeated study 4 2.083. Repeated test 1.92 44. Repeated study + Repeated test 4 4
* No correct answer (e.g., chakura = food) was provided as feedback after each test.
Repeated study + Repeated test
ControlRepeated test Repeated study
Adapted from Roediger 2009
Discussion• Number of testing, not studying, affects long-term
retention.• Increasing the number of studying from two to four
does not significantly increase retention 1 week after learning ( 33 36%⇒ )
• Increasing the number of testing from two to four, in contrast, significantly increases retention 1 week after learning ( 33 81%⇒ )
• When the number of testing is four, increasing the number of studying from two to four does not affect retention ( 81% 81% ⇒ )
Discussion• Vocabulary learning is a function of the
number of testing, not studying.• People learn more from testing than studying
(testing effect).• Gates, A.I. (1917). Recitation as a factor in
memorizing. Archives of Psychology, 6(40).• Karpicke, J. D., & Roediger, H. L. (2008). The
critical importance of retrieval for learning. Science, 319, 966-968.
• NY Times Jan. 20, 2011
Pedagogical implications
• Vocabulary quizzes are good!• It doesn’t necessarily mean that we have to
give “tests” in its traditional sense.• Self-testing also counts as a test.
(e.g.,) activities involving self-testing such as flashcards, fill-in-the-blanks, matching exercises, or translation exercises are helpful.
• Computer-based flashcards are also useful.
Outline
1. Effects of studying and testing (Karpicke & Roediger, 2008)
2. Effects of equal and expanding spacing (Nakata, 2015)
3. Effects of part and whole learning (Nakata & Webb, 2016)
4. Effects of single and repeated retrievals (Nakata, in press)
5. Q&A
• Nakata, T. (2015). Effects of expanding and equal spacing on second language vocabulary learning: Does gradually increasing spacing increase vocabulary learning? Studies in Second Language Acquisition. 37, 677-711. doi:10.1017/S0272263114000825
Background
• Karpicke and Roediger (2008) showed that testing facilitates vocabulary learning.
• How should the tests be distributed to maximize learning?
• Two practice schedules: equal and expanding spacing
Equal spacing vs. Expanding spacing• Equal spacing: The intervals between
encounters of a given item are held constant.
Note. A = a given target word.
Equal spacing vs. Expanding spacing
• Expanding spacing: The intervals between encounters of a given item are gradually increased as learning proceeds.
Note. A = a given target word.
Massed schedule (no spacing)• Target items are encountered several
times in a row.
E.g.,)A A A A B B B B C C C C D D D D …
• Used as the control treatment.
• Which is more effective for vocabulary learning, equal or expanding spacing?
• Expanding spacing is often regarded as the most effective practice schedule (Baddeley, 1997; Ellis, 1995; Hulstijn, 2001; Landauer & Bjork, 1978; Nation, 2001; Pimsleur, 1967; Schmitt, 2000).
• A common feature among existing flashcard software (e.g., iKnow, Word Engine, SuperMemo; Nakata, 2011)
Expanding spacing is …
http://www.wordengine.jp/vflash
Theoretical background
1. Retrieval practice effect (Karpicke & Roediger, 2010; Logan & Balota, 2008)
2. Retrieval effort hypothesis (Karpicke & Roediger, 2010; Logan & Balota, 2008; Pyc & Rawson, 2009)
Empirical evidence• Expanding > Equal (7 studies)
Cull et al., 1996; Dobson, 2011; Karpicke & Roediger, 2007; Landauer & Bjork, 1978; Maddox et al., 2011, Exp 2; Logan & Balota, 2008; Storm, Bjork, & Storm, 2010
• Expanding = Equal (9 studies) Balota et al., 2006; Carpenter & DeLosh, 2005; Cull, 2000; Kang et al., 2014; Karpicke & Bauernschmidt, 2011; Karpicke & Roediger, 2010; Maddox et al., 2011, Exp 2; Pyc & Rawson, 2007; Shaughnessy & Zechmeister, 1992
• Expanding < Equal (3 studies)Cull, 2000; Karpicke & Roediger, 2007; Logan & Balota, 2008
Empirical evidence
• Meta-analysis by Cepeda et al.(2006): Mixed results
• Why inconsistent results?The effects of expanding spacing may be affected by at least four factors … (Balota et al., 2007; Dobson, 2012; Karpicke & Roediger, 2007; Logan & Balota, 2008; Maddox et al., 2011; Roediger & Karpicke, 2010; Storm, Bjork, & Storm, 2010)
Empirical evidence• Expanding spacing tends to be effective
when … (1) the task difficulty is high, (2) feedback is not provided after retrieval, (3) the treatment involves a large amount of
spacing, and / or (4) the retention interval is shorter than 24 hours (i.e., posttest is given within 24 hours after the treatment).
Limitations• None of the existing L2 vocabulary
studies manipulated these four factors.(Kang et al., 2014; Karpicke & Bauernschmidt, 2011; Pyc & Rawson, 2007)
• This study will consider the effects of (a) the amount of spacing and (b) retention interval (i.e., timing of posttest).
Research questions1. Do (a) the amount of spacing and (b)
retention interval influence the effects of equal and expanding spacing on L2 vocabulary learning?
2. Which has a larger effect on L2 vocabulary learning: the type of spacing (equal or expanding) or the amount of spacing?
METHOD
• Participants: 128 Japanese college students
• Target items: 20 low frequency English wordse.g.,) apparition, billow, cadge, citadel, and dally
Independent variables• 1) Amount of spacing (between-participant)
Massed (No spacing) AAAA
Short (1 minute) A_ _A_ _A_ _A
Medium (2 minutes) A_ _ _ _A_ _ _ _A_ _ _ _A
Long (6 minutes) A_ _ _ _ _ _ _ _ _ _ _ _A_ _ _ _ _ _ _ _ _ _ _ _A_ _ _ _ _ _ _ _ _ _ _ _A
Independent variables
• 2) Type of spacing (within-participant)
• 3) Retention interval (within-participant)Immediate posttest 1-week delayed posttest
Equal A_ _ _ _ _ _ _ _ _ _ _ _A _ _ _ _ _ _ _ _ _ _ _ _A_ _ _ _ _ _ _ _ _ _ _ _A
Expanding A_ _ _ _ _ _ A_ _ _ _ _ _ _ _ _ _ _ _ A_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _A
Amount of spacing
Type of spacing
Short Equal A_ _A_ _A_ _A
Expanding A_A_ _A_ _ _A
MediumEqual A_ _ _ _A_ _ _ _A_ _ _ _A
Expanding A_ _ A_ _ _ _A_ _ _ _ _ _A
Long Equal A_ _ _ _ _ _ _ _ _ _ _ _A_ _ _ _ _ _ _ _ _ _ _ _A_ _ _ _ _ _ _ _ _ _ _ _A
Expanding A_ _ _ _ _ _A_ _ _ _ _ _ _ _ _ _ _ _A_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _A
Massed (No spacing)
AAAA
7 Conditions
Procedure
Week 1
ExplanationPretest (1. Productive 2. Receptive)Treatment Filler taskImmediate posttest (1. Productive 2. Receptive)Questionnaire
Week 2 Delayed posttest (1. Productive 2. Receptive)
1st Encounter: Initial presentation (Studying)
2nd, 3rd, and 4th Encounters: Productive retrieval (Test)
Feedback (Correct response)
• (1) Productive test: Translate from Japanese (L1) to English (L2).
Pre / Posttests
• (2) Receptive test: Translate from English (L2) to Japanese (L1).
Pre / Posttests
RESULTS (1): Type of spacingIs there any difference between equal and expanding spacing?
Dependent variables:• 1) Learning phase performance• 2) Immediate posttest scores• 3) 1-week delayed posttest scores
Equal A_ _ _ _ _ _ _ _ _ _ _ _A _ _ _ _ _ _ _ _ _ _ _ _A_ _ _ _ _ _ _ _ _ _ _ _A
Expanding A_ _ _ _ _ _ A_ _ _ _ _ _ _ _ _ _ _ _ A_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _A
0.0
0.2
0.4
0.6
0.8
1.0
Learning phase performance: Short spacing group
Pro
porti
on o
f cor
rect
resp
onse
s
Retrieval 1 Retrieval 2 Retrieval 3
Equal
Note. Brackets enclose +1 SE.
0.0
0.2
0.4
0.6
0.8
1.0
Learning phase performance: Short spacing group
Pro
porti
on o
f cor
rect
resp
onse
s
Retrieval 1 Retrieval 2 Retrieval 3
Equal
Expanding
n.s. d < 0.07
(small effect)
Expanding > Equal p = .002, d = 0.41 (small effect)
Note. Brackets enclose +1 SE.
0.0
0.2
0.4
0.6
0.8
1.0
Learning phase performance: Medium spacing group
Pro
porti
on o
f cor
rect
resp
onse
s
Retrieval 1 Retrieval 2 Retrieval 3
Equal
Expanding
n.s. d < 0.23
(small effect)
Note. Brackets enclose +1 SE.
0.0
0.2
0.4
0.6
0.8
1.0
Learning phase performance: Long spacing group
Pro
porti
on o
f cor
rect
resp
onse
s
Retrieval 1 Retrieval 2 Retrieval 3
Equal
Expanding
n.s. d < 0.15
(small effect)
Note. Brackets enclose +1 SE.
Note. Brackets enclose +1 SE.
n.s. d < 0.12
(small effect)
Note. Brackets enclose +1 SE.
n.s. d < 0.21
(small effect)
Note. Brackets enclose +1 SE.
Note. Brackets enclose +1 SE.
Note. Brackets enclose +1 SE.
Summary (1): Type of spacing• 1) Learning phase: equal = expanding • 2) Immediate posttest: equal = expanding• 3) Delayed posttest: equal = expanding
• Previous studies: Expanding spacing may be effective depending on (a) the amount of spacing and (b) retention interval.
• Not supported in this study.
Massed (No spacing) AAAA
Short (1 minute) A_ _A_ _A_ _A
Medium (2 minutes) A_ _ _ _A_ _ _ _A_ _ _ _A
Long (6 minutes) A_ _ _ _ _ _ _ _ _ _ _ _A_ _ _ _ _ _ _ _ _ _ _ _A_ _ _ _ _ _ _ _ _ _ _ _A
RESULTS (2): Amount of spacing• Is there any difference among the massed,
short, medium, and long spacing groups?
Dependent variables• 1) Learning phase performance• 2) Immediate posttest scores• 3) 1-week delayed posttest scores
0.0
0.2
0.4
0.6
0.8
1.0
Learning phase performance
Pro
porti
on o
f cor
rect
resp
onse
s
Retrieval 1 Retrieval 2 Retrieval 3
Massed
Note. Brackets enclose +1 SE.
Note. Brackets enclose +1 SE.
0.0
0.2
0.4
0.6
0.8
1.0
Learning phase performance
Pro
porti
on o
f cor
rect
resp
onse
s
Retrieval 1 Retrieval 2 Retrieval 3
Massed
Short
Note. Brackets enclose +1 SE.
0.0
0.2
0.4
0.6
0.8
1.0
Learning phase performance
Pro
porti
on o
f cor
rect
resp
onse
s
Retrieval 1 Retrieval 2 Retrieval 3
Massed
Short
Medium
0.0
0.2
0.4
0.6
0.8
1.0
Learning phase performance
Pro
porti
on o
f cor
rect
resp
onse
s
Retrieval 1 Retrieval 2 Retrieval 3
Massed
Short
Medium
Long
Massed > Short = Medium > Longp < .004, 0.78 < d < 4.75 (medium to large effect)
Note. Brackets enclose +1 SE.
Mas
sed
Sho
rt
Med
ium
Long
Mas
sed
Sho
rt
Med
ium
Long
Productive posttest
Pro
porti
on o
f cor
rect
resp
onse
s
0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1.0Immediate Delayed
Short = Medium = Long > Massedp < .004, 0.88 < d < 1.40 (large effect)
Mas
sed
Sho
rt
Med
ium
Long
Mas
sed
Sho
rt
Med
ium
Long
Productive posttest
Pro
porti
on o
f cor
rect
resp
onse
s
0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1.0Immediate Delayed
Short = Massed n.s. d = 0.39 (small effect)
Medium = Long > Massedp < .014, 0.80 < d < 0.85 (large effect)
Short = Medium = Long > Massedp < .004, 0.88 < d < 1.40 (large effect)
Mas
sed
Sho
rt
Med
ium
Long
Mas
sed
Sho
rt
Med
ium
Long
Productive posttest
Pro
porti
on o
f cor
rect
resp
onse
s
0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1.0Immediate Delayed
Spacing effect
0.0
0.2
0.4
0.6
0.8
1.0
Learning phase performance
Pro
porti
on o
f cor
rect
resp
onse
s
Retrieval 1 Retrieval 2 Retrieval 3
Massed
Short
Medium
Long
Mas
sed
Sho
rt
Med
ium
Long
Mas
sed
Sho
rt
Med
ium
Long
Productive posttest
Pro
porti
on o
f cor
rect
resp
onse
s
0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1.0Immediate Delayed
Short > Massedp < .001, d = 1.07 (large effect)
Short = Massed n.s. d = 0.39 (small effect)
Spacing-retention interval
interaction
Mas
sed
Sho
rt
Med
ium
Long
Mas
sed
Sho
rt
Med
ium
Long
Receptive posttest
Pro
porti
on o
f cor
rect
resp
onse
s
0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1.0Immediate Delayed
SUMMARY
• 1) Very little difference between equal and expanding spacing.
• 2) Expanding spacing was no more effective than equal spacing regardless of (a) the amount of spacing or (b) retention interval.> Expanding spacing may not be the most effective practice schedule.
SUMMARY
• 3) The massed schedule (no spacing) was the least effective in both short and long term.> The finding underscores the importance of spacing in L2 vocabulary learning (spacing effect).
Summary (2): Amount of spacing• 4) Medium and long spacing were the most
effective in both short and long term.• 5) Short spacing was effective in the short term,
but not in the long term.> Introducing a large amount of spacing may be effective, especially for improving long-term retention (spacing-retention interval interaction).>> The amount of spacing may have a larger effect on learning than the type of spacing.
PEDAGOGICAL IMPLICATIONS• The desirable difficulty framework (e.g., Bjork,
1994, 1999)Slow and effortful learning = good retentionFast and easy learning = poor retention(e.g.,) Massed vs. medium / long spacing
• Do not discourage learners from making errors• Make a distinction between learning phase
performance and long-term retention • We should not be concerned too much about
learning phase performance
DISCUSSION• Why no difference between equal and
expanding spacing?1. Primary / long-term memory (Karpicke &
Roediger, 2007; Roediger & Karpicke, 2010)
2. The retrieval effort hypothesis (Karpicke & Bauernschmidt, 2011)
LIMITATIONS1. (a) Task difficulty and (b) feedback were
not manipulated. 2. Short duration of the treatment (approx
25 minutes). Kang, Lindsey, Mozer, & Pashler (2014): Treatment conducted over 4 weeks. No difference between equal and expanding spacing.
Outline
1. Effects of studying and testing (Karpicke & Roediger, 2008)
2. Effects of equal and expanding spacing (Nakata, 2015)
3. Effects of part and whole learning (Nakata & Webb, 2016)
4. Effects of single and repeated retrievals (Nakata, in press)
5. Q&A
• Nakata, T., & Webb, S. (2016). Does studying vocabulary in smaller sets increase learning? The effects of part and whole learning on second language vocabulary acquisition. Studies in Second Language Acquisition, 38, 523-552. doi: 10.1017/S0272263115000236
Part vs. whole learning
• Whole learning: The materials to be learnt are repeated in one large block.
• Part learning: The materials are divided into smaller blocks and repeated.
Part vs. whole learning
• When memorising a poem, would it be more effective to read the whole poem several times (whole learning) or would it be more effective to read part by part for the same number of times (part learning)?
• When learning to play music, would it be effective to practise part by part (part learning) before trying to play the whole tune (whole learning)?
Part vs. whole learning
• When learning 20 words… • Whole learning: 20 words are repeated in one
large block• Part learning: 20 words are divided into
smaller blocks and repeated
Whole learning: 20 items are repeated in one large block
Part learning: 20 items are repeated in four blocks of 5 items
Whole learning: 20 items are repeated in one large block
Part learning: 20 items are repeated in four blocks of 5 items
Whole learning: 20 items are repeated in one large block
Part learning: 20 items are repeated in four blocks of 5 items
Whole learning: 20 items are repeated in one large block
Part learning: 20 items are repeated in four blocks of 5 items
Whole learning: 20 items are repeated in one large block
Part learning: 20 items are repeated in four blocks of 5 items
Whole learning: 20 items are repeated in one large block
Part learning: 20 items are repeated in four blocks of 5 items
Whole learning: 20 items are repeated in one large block
Part learning: 20 items are repeated in four blocks of 5 items
Whole learning: 20 items are repeated in one large block
Part learning: 20 items are repeated in four blocks of 5 items
Whole learning: 20 items are repeated in one large block
Part learning: 20 items are repeated in four blocks of 5 items
Whole learning: 20 items are repeated in one large block
Part learning: 20 items are repeated in four blocks of 5 items
Question
• Which do you think is more effective for L2 vocabulary learning, whole learning or part learning?
Background
• Researchers, learners, teachers, and materials developers: part learning > whole learning (e.g., Kornell, 2009; Salisbury & Klein, 1988; Wissman, Rawson, & Pyc, 2012; Woodworth & Schlosberg, 1954).
• The retrieval practice effect (Baddeley, 1997; Ellis, 1995) and list-length effect (Gillund & Shiffrin, 1984; Van Bussel, 1994): part learning > whole learning
Empirical evidence
• Empirical studies: whole learning > part learning (Brown, 1924; Crothers & Suppes, 1967, Experiments 8 & 9; Kornell, 2009, Experiments 1-3; McGeoch, 1931, Experiments 1 & 3; Seibert, 1932).
Limitation of previous research• In existing studies, the part-whole distinction
and spacing have been confounded. (i.e., whole learning always had longer spacing than part learning).
• Spacing effect: Large spacing > short spacing (e.g., Bahrick & Phelps, 1987; Cepeda et al., 2008, 2009; Pashler et al., 2003)
• The results of the earlier studies: due to spacing or the part-whole distinction?
Whole learning
Whole learningSpacing of 19 other items
Part learning
Part learningSpacing of 4 other items
Whole learning: Spacing of 19 items⇒ Large spacingPart learning: Spacing of 4 items⇒ Short spacing
Limitation of previous research
• Spacing effect: Large spacing > short spacing.• The results of the earlier studies: due to
spacing or the part-whole distinction?
Present study
• Purpose: investigate the effects of part and whole learning on L2 vocabulary learning while isolating the effects of the part-whole distinction and spacing.
Experiment 1
• Purpose: To investigate the effects of part and whole learning that were matched in spacing.
• RQ: Is whole learning more effective than part learning for L2 vocabulary learning when spacing is equivalent?
METHOD
• Participants: 91 Japanese college students
• Target items: 20 low frequency English wordse.g.,) apparition, billow, cadge, citadel, dally
Independent variable
• Unlike previous studies, part and whole learning were matched in spacing (approximately 4 min).
Part / whole distinction Block size
(1) 4-item part Part 4(2) 10-item part Part 10(3) Whole Whole 20
Procedure
Week 1
ExplanationPretest (1. Productive 2. Receptive)Treatment Filler taskImmediate posttest (1. Productive 2. Receptive)Questionnaire
Week 2 Delayed posttest (1. Productive 2. Receptive)
1st Encounter: Initial presentation
2nd - 5th Encounters: Productive retrieval
Feedback (Correct response)
• (1) Productive test: Translate from Japanese (L1) to English (L2).
Pre / Posttests
• (2) Receptive test: Translate from English (L2) to Japanese (L1).
Pre / Posttests
RESULTS
F (2, 88) = 0.14, p = .867, ηp
2 < .01
F (2, 88) = 0.41, p = .664, ηp
2 = .01
Discussion• Previous studies: Whole > part learning• This study: Whole part learning≒• As long as spacing is equivalent, the part-
whole distinction has little effect on learning.• The results of the earlier studies may have
been due to spacing rather than the part-whole distinction?
Limitation of Experiment 1
• We cannot necessarily attribute the lack of statistical significance to the fact that part and whole learning were controlled for spacing (e.g., small sample size, Type II error).
Experiment 2• Purpose: Test the three hypotheses below:• H1: When spacing is equivalent, whole learning does
not outperform part learning• H2: When whole learning has longer spacing than part
learning, whole learning outperforms part learning.• H3: Part learning with longer spacing outperforms part
learning with shorter spacing. • If all three hypotheses are supported, the lack of
statistical significance in Experiment 1 was perhaps because part and whole learning had equivalent spacing.
METHOD
• Participants: 78 Japanese college students
• Target items: 20 low frequency English wordse.g.,) apparition, billow, cadge, citadel, dally
Independent variable
* (2) “4-item part” and (3) “Whole” learning will be collectively referred to as the experimental treatments.
Part / whole distinction Block size
Spacing(Approx.)
(1) Control Part 4 30 sec(2) 4-item part Part 4 3 min(3) Whole Whole 20 3 min
n.s. d < 0.49
RESULTSPart = Whole > Control
0.87 < d < 1.02
n.s. d < 0.49
Productive posttest
• Immediate: Experimental = Control • Delayed: Experimental > Control • Spacing-by-retention interval interaction:
- When the posttest is given after a short delay, short spacing tends to be effective.- When the posttest is given after a long delay, long spacing tends to be effective. (e.g., Cepeda et al., 2006, 2008, 2009; Pashler et al., 2007; Rohrer & Pashler, 2007)
n.s. d < 0.60
Receptive posttest
• Immediate: Experimental = Control • Delayed: Experimental = Control • At each test administration, the productive
test was given prior to the receptive test.• This perhaps affected performance on the
receptive test and reduced a potential difference among the three groups on the receptive posttest?
DISCUSSION• All three hypotheses were supported:• H1: When spacing is equivalent, whole learning
does not outperform part learning (whole 4-≒item part).
• H2: When whole learning has longer spacing than part learning, whole learning outperforms part learning (whole > control).
• H3: Part learning with longer spacing outperforms part learning with shorter spacing (4-item part > control).
Experiments 1 & 2
• As long as spacing is equivalent, the part-whole distinction has little effect on learning
• Spacing has a larger effect on learning than the part-whole distinction
Pedagogical implications
1. As long as the treatment involves long spacing, learners may study with either part or whole learning without significantly affecting learning outcomes.
2. It is useful to pay more attention to spacing rather than the part-whole distinction. The amount of spacing has a larger effect on learning than the part-whole distinction.
Limitations
• 1. Short duration of the treatment (< 20 minutes).
• 2. Learning was restricted to flashcard learning.
Outline
1. Effects of studying and testing (Karpicke & Roediger, 2008)
2. Effects of equal and expanding spacing (Nakata, 2015)
3. Effects of part and whole learning (Nakata & Webb, 2016)
4. Effects of single and repeated retrievals (Nakata, in press)
5. Q&A
• Nakata, T. (in press). Does repeated practice make perfect? The effects of within-session repeated retrieval on second language vocabulary learning. Studies in Second Language Acquisition.
Background• Repeated input increases L2 vocabulary learning
(e.g., Bisson et al., 2014; Chen & Truscott, 2010; Horst, Cobb, & Meara, 1998; Pigada & Schmitt, 2006; Rott, 1999, 2007; Webb, 2007; Waring & Takaki, 2003; Zahar et al., 2001).
• Retrieval > input (e.g., Barcroft, 2007, 2013; Karpicke & Roediger, 2008; Royer, 1973).
• Would repeated retrieval increase learning?(1) repeated retrieval within one learning session (“within-session repeated retrieval”)(2) repeated retrieval over multiple learning sessions
Background• Repeated input increases L2 vocabulary learning (e.g.,
Bisson et al., 2014; Chen & Truscott, 2010; Horst, Cobb, & Meara, 1998; Pigada & Schmitt, 2006; Rott, 1999, 2007; Webb, 2007; Waring & Takaki, 2003; Zahar et al., 2001).
• Retrieval (= test) > input (= study) (e.g., Barcroft, 2007, 2013; Karpicke & Roediger, 2008; Royer, 1973).
• Would repeated retrieval increase learning?(1) repeated retrieval within one learning session (“within-session repeated retrieval”)(2) repeated retrieval over multiple learning sessions
Arguments in favor of within-session repeated retrieval
• Usage-based models of language acquisition (e.g., Goldberg, 2006; Tomasello, 2003): frequency is one of the major factors affecting acquisition
• Within-session repeated retrieval increases learning.
Arguments against within-session repeated retrieval
• Spacing effect: long spacing between retrievals> short spacing between retrievals (= within-session repeated retrieval)
• The benefits of within-session repeated retrieval are short-lived? (Rohrer et al., 2005; van Zeeland & Schmitt, 2013)
Arguments against within-session repeated retrieval
• van Zeeland & Schmitt (2013)Immediately: repeated > less input2-week delay: repeated = less input* repeated input, not repeated retrieval
• Rohrer et al. (2005)1-week delay: repeated > fewer retrievals
3-9 week delay: repeated = fewer retrievals * non-L2 vocab
Arguments against within-session repeated retrieval
• Retrieval effort hypothesis: concerned with the issue of efficiency (i.e., whether repeated retrieval allows learners to learn a large number of words when time-on-task is controlled).
• Difficult and effortful retrievals > easy and effortless retrievals
• In within-session repeated retrieval, retrievals will be progressively easier (e.g.,) 1st < 2nd < 3rd
Retrieval effort hypothesis
• The retrieval effort hypothesis: easy retrievals < difficult retrievals (e.g.,) 1st < 2nd < 3rd
• Most learning occurs in the first few retrieval attempts, and practicing retrieval beyond a certain frequency level may not provide a benefit that justifies its cost (i.e., increased study time).
• Within-session repeated retrieval is not necessarily efficient?
Previous research
• Barcroft (2013); Folse (2006)immediately: repeated > fewer retrievals
• Peters (2012, 2014)immediately & 1-2 weeks: repeated > fewer retrievals
Present study
1. Examine both the short and long term effects of within-session repeated retrieval.Posttests were given not only immediately and 1 week, but also 4 weeks after the treatment (Rohrer et al., 2005).
2. Examine not only effectiveness (posttest scores) but also efficiency (posttest scores / study time).
Research questions
• RQ1: In L2 vocabulary learning, does increasing within-session retrieval frequency increase effectiveness (posttest scores)?
• RQ2: In L2 vocabulary learning, does increasing within-session retrieval frequency increase efficiency (posttest scores / study time)?
Hypotheses
• Hypothesis 1: Within-session repeated retrieval is more effective than fewer retrievals immediately and 1 week after learning, but not 4 weeks (spacing effect).
• Hypothesis 2: Within-session repeated retrieval is more efficient than fewer retrievals regardless of the posttest timing (retrieval effort hypothesis).
METHOD
• Participants: 98 Japanese college students
• Target items: 16 low frequency English wordse.g.,) apparition, billow, citadel, dally, gouge
METHOD
• Participants: 98 Japanese college students
• Target items: 16 low frequency English wordse.g.,) apparition, billow, citadel, dally, gouge
Independent variable
• (1) Retrieval frequency (between-participant): 1, 3, 5, or 7 retrievals
• (2) Posttest timing (within-participant): immediately, 1 week, and 4 weeks after learning
Procedure
Week 1Pretest (a. Productive b. Receptive)Treatment Filler taskImmediate posttest (a. Productive b. Receptive)
Week 2 Delayed posttest 1 (a. Productive b. Receptive)
Week 5 Delayed posttest 2 (a. Productive b. Receptive)
1st Encounter: Initial presentation
Retrieval opportunities (1, 3, 5 or 7 times)
Feedback (Correct answer)
• (1) Productive test: Translate from Japanese (L1) to English (L2).
Pre / Posttests
• (2) Receptive test: Translate from English (L2) to Japanese (L1).
Pre / Posttests
1 3 5 7 1 3 5 7 1 3 5 7
Effectiveness: Productive posttest
Pro
porti
on o
f cor
rect
resp
onse
s
0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1.0 Immediate 1-week Delayed 4-week Delayed
Retrieval 5 = 7 > 1 = 3(0.88 < d < 2.04)
1 3 5 7 1 3 5 7 1 3 5 7
Effectiveness: Receptive posttest
Pro
porti
on o
f cor
rect
resp
onse
s
0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1.0 Immediate 1-week Delayed 4-week Delayed
Retrieval 5 = 7 > 1 = 3(0.98 < d < 1.37)
Summary
• Effectiveness: Retrieval 1 = 3 < 5 = 7(regardless of posttest timing)
• Efficiency: Efficiency score = posttest score / study time (min.)
1 3 5 7 1 3 5 7 1 3 5 7
Efficiency: Productive posttest
Pro
porti
on o
f cor
rect
resp
onse
s
0.0
0.2
0.4
0.6
0.8
1.0
1.2
1.4
1.6
Immediate 1-week Delayed 4-week Delayed
Retrieval 1 > 3 = 5 = 7(0.36 < d < 1.88)
1 3 5 7 1 3 5 7 1 3 5 7
Efficiency: Receptive posttest
Pro
porti
on o
f cor
rect
resp
onse
s
0.0
0.2
0.4
0.6
0.8
1.0
1.2
1.4
1.6
1.8 Immediate 1-week Delayed 4-week Delayed
Retrieval 1 > 3 = 5 = 7(1.40 < d < 3.10)
Summary
• Effectiveness Retrieval 1 = 3 < 5 = 7(regardless of posttest timing)
• EfficiencyRetrieval 1 > 3 = 5 = 7(regardless of posttest timing)
Discussion: Effectiveness
• Hypothesis 1: benefits of within-session repeated retrievals are short-lived
• This study: repeated retrievals > fewer retrievals regardless of posttest timing
• The negative effects of massing practice were outweighed by the positive effects of increased frequency?
• Underscore the importance of repetition in L2 vocabulary learning.
Discussion: Efficiency
• Single retrieval was the most efficient regardless of the posttest timing, supporting Hypothesis 2.
• The findings are consistent with the retrieval effort hypothesis, according to which the positive effects of repeated retrieval decrease as a function of the retrieval frequency.
• Most learning occurred in the first retrieval attempt, and practicing retrieval more than once did not provide a benefit that justified its cost (i.e., increased study time).
Pedagogical implications
• If efficiency is not an issue, increasing retrieval frequency within one learning session is preferable.
• If learners do not have enough time, it may be desirable to practice retrieval only once.
Limitations
1. The posttest timing was manipulated within participants.
2. The treatment in this study was restricted to flashcard learning.
SUMMARY
• Karpicke and Roediger (2008)• People learn more from testing than studying
(testing effect).• How should the tests be distributed to
maximize learning?• Two practice schedules: equal and expanding
spacing
Equal and expanding spacing
• 1) Very little difference between equal and expanding spacing.
• 2) The massed schedule was the least effective in both short and long term (spacing effect).
• 3) Medium and long spacing were the most effective in both short and long term.
• 4) Short spacing was effective in the short term, but not in the long term (spacing-retention interval interaction).
Whole and Part Learning
• As long as spacing is equivalent, the part-whole distinction has little effect on learning.
• Spacing has a larger effect on learning than the part-whole distinction.
• It is useful to pay more attention to spacing rather than the part-whole distinction.
Repeated retrieval• If efficiency is not an issue, increasing retrieval
frequency within one learning session is preferable.
• If learners do not have enough time, it may be desirable to practice retrieval only once.
• I am very grateful to Stuart Webb, Paul Nation, Jan Hulstijn, Rod Ellis, Stuart McLean, Atsushi Mizumoto, and Tomohiro Tsuchiya.
• Tatsuya Nakata (Kansai University)[email protected]