how does (a lack of) knowledge of multiword expressions affect reading comprehension?

How does (a lack of) knowledge of multiword expressions affect

reading comprehension?Ron Martinez

AAAL, Dallas, TX - March 17th, 2013

(and Candy Palma)

Still putting together ‘the (formulaic language) puzzle’ • “the puzzle of nativelike selection” (Pawley & Syder,

1983: 194)

• estimates of over 50% written text (Erman & Warren, 2000)

• studies tend to focus on effect of formulaic language (FL) on L2 production or processing (e.g. Boers, et al, 2006; Conklin & Schmitt, 2008; Millar, 2011)

• still not a lot known about how FL affects comprehension (perhaps more relevant?)

Martinez & Murphy (2011)Let me tell you about my home. It’s on this little hill out in the country. But I’m not far from the city (I don’t like the city – do you?) – not much time to get here. I can’t wait to show you a photo… or you can call me to come over to see in person! 07786 237 679

I don’t get out much – it’s about time I do. I’m not from here – this country or city. (But I like this country.) I’m far from home. I’m a little over the hill, let me tell you, but you can’t tell! (I can show you my photo, or wait to come see me in person!) Call me on 07786 554 0978

exact same wordsall very frequent

words (top 2,000)

1. 2. 3. 4. 5. 6. 7.

!He wants to go out but has a problem with time. !He is foreign. !He lives in a remote area. !He wants to keep his location a secret. !He thinks he looks younger than his age. !He probably lives in an area with hills. !He lives on a hill, or near one, but not on top of it.

My comprehension of this text: 5% 25% 50% 75% 100%

I don’t get out much – it’s about time I do. I’m not from here – this country or city. (But I like this country.) I’m far from home. I’m a little over the hill, let me tell you, but you can’t tell! (I can show you my photo, or wait to come see me in person!) Call me on 07786 554 0978

The resultsMin. Max. Mean SD

Transpar. Text

18 28 24.09 2.44

Idiomatic Text

6 25 14.76 3.93

t = 24.10 (p ≤ 0.001), eta squared = 0.828

Reported Comprehension vs. Actual Comprehension

• No statistically significant difference for Part 1 (87.38% reported vs 86.03% actual).

• Reported comprehension significantly overestimated in Part 2 (t = 3.95, p≤ 0.001, eta squared = 0.07) – 60.29% reported vs 52.58% actual.

• "Deceptive transparency" (Laufer, 1989) extended.

‘on occasion’

INTERMEDIATE

HIGHER

Questions about ‘frequency’

1. Is there a testable difference between the high-frequency single-word vocabulary L2 learners know and multiword vocabulary of the same frequency range?

2. Does that difference have any impact on reading comprehension?

11

20

Vocabulary Size Test (Nation & Beglar, 2007)

A tricky question...

How can a test be devised that assesses knowledge of multiword expressions in the same or similar way as current widely-used vocabulary tests?

Ron Martinez

at all times at all costs at all

More compositional? Less compositional?

Meaning still retained when each lexical word replaced with its own definition (Grant & Bauer, 2004)

�14

transparent opaque

frequent

infrequent

Frequency-Transparency

Framework (FTF)

(Martinez, 2013)

at all

at all costs

at all times

38

PHRASE List (Martinez & Schmitt, 2012)

single word – multiword expression frequency matching

39

BEFORE AFTERintegratedwordlist

45

Phrasal Vocabulary Size Test (PVST)

45

Does knowledge of the PHRASE List make a difference?

• 108 (from 130+) adult Brazilian university students (major: English)

• all students took both Martinez & Murphy (2011) reading comprehension test and Nation & Beglar (2007) Vocabulary Size Test (VST) and Phrasal Vocabulary Size Test (PVST)

• 156 test items total (28 + 28 on Reading Comprehension, 50 + 50 on Vocabulary)

• Also administered questionnaire (e.g. age, AoO, motivation, strategic investment, etc.)

�18

How does PVST compare to VST?

• Candidates knew significantly (t = 7.67, p<.001)more items on VST, but tests highly correlate (r = .90, p<.001) and both reliable (Chronbach’s Alpha: VST = 0.92, PVST = 0.96, BOTH = 0.97)

�19

0.02.04.06.08.0

K1 K2 K3 K4 K5

VST PVST

25.026.528.029.531.0

Total

M SDVST 30.49 9.16

PVST 25.78 13.1

�20

number incorrect = 69.45%

How does PVST (and VST) knowledge relate to reading?

• Correlations all significant p<.001

• But...

• with Reported Comprehension (x2), effect observed only with VST x Idiomatic Text (F (2,92) = 33.19, p<0.05, eta squared 0.05)

• Participants who performed relatively well on single-word vocabulary tended to think they understood more than they did when texts contained opaque idiomaticity.

�21

Transp. Reading

Idiom. Reading

VST r = 0.68 r = 0.60

PVST r = 0.64 r = 0.63

What does PVST add?• PVST cut-off set at 25 (mean was 25.78) • If VST >25 and PVST ≤ 25 (n=36)

• If VST >25 and PVST >25 (n=49)

�22

M (28 max.) SD

Transp. Text 25.88 1.12

Idiom. Text 12.52 2.41

M (28 max.) SD

Transp. Text 26.16 1.07

Idiom. Text 16.22 4.22

• PVST seems to add some predictive validity.

What about the questionnaire?

�23

�24

• Among reported time (per week) reading, listening to, and viewing media in English...

• only reading significantly impacted both VST and PVST performance (Kruskal-Wallis: H(4) =22.02/19.04, p <0.05, respectively)

• and TV viewing only significantly interacted with PVST (H(4) =11.30, p<0.05)

• Length of Study seemed to matter for VST

(F(27,76) =1.80, p <0.05, eta squared 0.09), but not the PVST.

�25

Vespoor, M.H., de Bot, K., and van Rein, E. (2011). English as a foreign language: The role of out-of-school language input. In A. De Houwer and A. Wilton (Eds.),

English in Europe Today: Sociocultural and educational perspectives (pp. 147-166). Amsterdam: John Benjamins.

Discovering America?• Vocabulary size important.

• Vocabulary helps reading.

• Reading helps vocabulary.

• Measuring one kind of vocabulary perhaps provides an incomplete picture.

Still plenty of the puzzle missing...

• Working memory? • Time abroad? • Effect on production? • What about less "polarized" reading

measure? • Other L1s, more interview data, more

analysis, less back surgery...

�28

Thank you

ronmartinez@sfsu.edu