the lexical profile of diverse and sophisticated academic essays

The Lexical Profile of Diverse and Sophisticated Academic Essays

Melanie C. Gonzalez, Ph.D.

Symposium on Second Language Writing

Auckland University of Technology

20 November 2015

The intersection of writing and vocabulary

Writing draws heavily on authors’ lexical faculties

Even advanced multilingual (ML) writers exhibit large differences in word knowledge from their monolingual English-speaking (MES) peers

Writing proficiency rubrics often feature a distinct category that measures effective vocabulary deployment

Is it a question of teaching more words? If so, which words do we teach? What lexical features should we focus on?

SSLW 2015 - M.Gonzalez, Ph.D, [email protected]

The intersection of writing and vocabulary

Rubric Lexical Criteria

ESL Composition Rubric (Jacobs et al., 1981)

Sophisticated range; effective word choice; word form mastery; appropriate register

TOEFL Independent Writing

Variety and range of vocabulary, occasional noticeable minor errors in word form and use of idiomatic language; Appropriate word choice and idiomaticity, minor lexical errors

IELTS Tasks 1 and 2 Uses a wide range of vocabulary with very natural and sophisticated control of lexical features; rare minor errors occur only as ‘slips’; use of uncommon lexical items

Salem State’s ENL 110 FYC Writing Rubric

Appropriate, effective, and precise word choice for register and style


Precedent research

Lexical diversity tends to be a strong indicator of writing proficiency and sophisticated word use (Crossley & McNamara, 2009; Friginal, Li, & Weigle, 2014; Gonzalez, 2014; Johnson, Acevedo, & Mercado, 2013; Yu, 2009)

Lexical diversity - varied use of different words in writing (Laufer & Nation, 1995)

Interest in reexamining lexical frequency bands/word lists (Gardner & Davies, 2013; Schmitt & Schmitt, 2012)

Lexical frequency – “counts” of how often a word occurs in language (Cobb, n.d.)

Computational tools allow for current resurgence in profiling lexical aspects of texts (see Cobb, n.d.; Gardner & Davies, 2014; Graesser, McNamara, Louwerse, & Cai, 2004)


Present study: A pilot

How can we help our advanced ML writers to achieve lexical diversity in their texts? Is it a question of teaching more words? If so, which words do we teach?

RESEARCH QUESTION 1:

How do the lexical frequency profiles of advanced ML writers’ academic compositions compare to those of their MES peers?

RESEARCH QUESTION 2:

What frequency level(s) is a significant contributor to lexical diversity in academic compositions?


Methods

172 essays (rated for English writing fluency by three independent raters) Yielded 43 viable essays for analysis: 20 MES and 23 ML writing samples

Word count limit of 500 words (within 8 words +/- target)

Analytical essays composed for grade; Earned a score of 5 using the TOEFL rubric

ML writers’ L1s: Arabic (n=14), Spanish (n=3), Mandarin (n=5), Korean (n=1)

Used Schmitt and Schmitt’s (2012) lexical frequency categories: 1K-3K = high-frequency terms

4K-8K = mid-frequency terms

9K+ = low-frequency terms

Lexical frequency: BNC-COCA 25 (in Lextutor; Cobb, n.d.)

Lexical diversity: MTLD (in Coh Metrix; Graesser et al., 2004)


Table 1: Descriptives

MES (n=20) ML (n=23) M SD Min Max

High-freq. MES 468.55 8.26 452.00 483.00

ML 487.17 8.29 452.00 502.00

Mid-freq. MES 14.45 6.01 3 24

ML 7.09 6.34 0 22

Low-freq. MES 3.20 2.14 0 7

ML 2.40 3.76 0 18

MTLD MES 79.95 17.35 46.85 120.86

ML 69.54 12.90 44.74 99.70


Research question 1

ANOVA ANALYSIS

ML writers used more high-frequency terms (F1,41=54.13, p<.00) than MES writers

MES writers used more mid-frequency words (F1,41=15.12, p<.00) than ML writers.

No statistical difference found in terms of either groups’ use of low-frequency terms

MES writers texts exhibited greater lexical diversity (F1,41=5.06, p<.05) than their ML peers


Research question 2

MULTIPLE REGRESSION ANALYSIS

The total variance explained by the model as a whole was 26.8% (F3,39= 4.75, p < .05)

Mid-frequency vocabulary was the only significant predictor of lexical diversity

As lexical diversity increases, there also an increase in the use of mid-frequency terms (beta = .93, p < .05)


Table 2: Examples of two essays from the pilot sample

1

1

4

5

5

32

447

1

3

4

9

2

28

43

396

K-8

K-7

K-6

K-5

K-4

K-3

K-2

K-1

Comparison of frequency profiles

MES ML


Examples of two essays from the pilot sample

One effect of eating fast food many times during the day is gaining weight. This is because these meals are saturated with far, cholesterol, and carbohydrates. So, people gain weight more easily than others who eat healthy food. This means that eating fast food for long time will lead to obesity.

As a result of gaining weight, people cannot exercise. In fact, overweight people think that exercise activities are difficult for them because they cannot even walk and move easily. So, they prefer to stay away from fitness centers, Therefore, those people gain more and more weight.

Even worse, all the previous factors lead to health problems. Due to eating many fatty foods, people are likely to suffer from high levels of cholesterol. This causes heart diseases, such as heart attack and other health problems, like diabetes.

After having this experience, many gain a new perspective on Christians. I have learned that a lot people view Christians as flawless people like in the Ned Flanders example stated earlier. However, most Christians considered themselves indeed as flawed. When we held a small group discussion, we shared people’s stereotypical views about Christians. Many of the students did not like this sense that they were considered “perfect” by outsiders because they believe they came to god for help and to repent their sins. In the group, we used words such as legalistic, hypercritical, boring, bible-pusher, and perfect to describe the stereotypical Christian according to others. When observing this small group, I noticed that many of the members wanted people to understand them. The members were very warm and welcoming. After attending three meetings and two games they already accepted me into their discourse community

1 2


Examples of two essays from the pilot sample

One effect of eating fast food many times during the day is gaining weight. This is because these meals are saturated with far, cholesterol, and carbohydrates. So, people gain weight more easily than others who eat healthy food. This means that eating fast food for long time will lead to obesity.

As a result of gaining weight, people cannot exercise. In fact, overweight people think that exercise activities are difficult for them because they cannot even walk and move easily. So, they prefer to stay away from fitness centers, Therefore, those people gain more and more weight.

Even worse, all the previous factors lead to health problems. Due to eating many fatty foods, people are likely to suffer from high levels of cholesterol. This causes heart diseases, such as heart attack and other health problems, like diabetes.

After having this experience, many gain a new perspective on Christians. I have learned that a lot people view Christians as flawless people like in the Ned Flanders example stated earlier. However, most Christians considered themselves indeed as flawed. When we held a small group discussion, we shared people’s stereotypical views about Christians. Many of the students did not like this sense that they were considered “perfect” by outsiders because they believe they came to god for help and to repent their sins. In the group, we used words such as legalistic, hypercritical, boring, bible-pusher, and perfect to describe the stereotypical Christian according to others. When observing this small group, I noticed that many of the members wanted people to understand them. The members were very warm and welcoming. After attending three meetings and two games they already accepted me into their discourse community.

1 2


Discussion and implications

The finding that ML writers use more high-frequency terms than their MES peers and that there is no difference between their use of low-frequency items is in line with precedent research (see Crossley & McNamara, 2009; 2012)

MES writers employed double the number of mid-frequency terms than ML writers

Points to a possible gap in ML writers’ lexicons

Guide students on how to use vocabulary profilers like Lextutor

During editing and peer-editing exercises, have students identify overly repeated words and phrases

Target mid-frequency vocabulary terms (precise synonyms) and bundles that add variety to the text


Limitations

Pilot study (43* essays; 500 word count) – need more, longer essays for examination

Multiple regression does not like small sample sizes

Longer essays may change the profile of the texts

Genre was limited to analytical papers, but did not control for topic

Remove proper nouns from analysis


Further research

Larger scale version of pilot to commence Examine the lexical frequency and diversity profiles of various

genres; same participants over time

Address limitations

Further studies in the research line: Include a measure of lexical recycling (Booth, 2014)

Include a measure of lexical error

Qualitative component - raters’ focus solely on lexical criteria


Further research

If you currently teach advanced ML writers, please consider contributing student essays to the larger study to run between December 2015 and May 2016

Please leave your email address with me and visit my Padlet (http://padlet.com/ProfMelanie/lexicaldiversity) for more details


References

Booth, P. (2014). The variance of lexical diversity profiles and its relationship to learning style. International Review of Applied Linguistics in Language Teaching, 52(4), 357-375.

Cobb, T. (n.d.). Compleat Lexical Tutor [Computer Program]. Retrieved from http://www.lextutor.ca

Crossley, S.A., & McNamara, D.S. (2009). Computational assessment of lexical differences in L1 and L2 writing. Journal of Second Language Writing, 18(2), 119-135.

Friginal, E., Li, M., Weigle, S.C. (2014). Revisiting multiple profiles of learner compositions: A comparison of highly rated NS and NNS essays. Journal of Second Language Writing, 23(1), 1-16.

Gardner, D., & Davies, M. (2014). A new academic vocabulary list. Applied Linguistics, 35(3), 305-327. doi:10.1093/applin/amt015

Gonzalez, M.C. (2014, November). Lexical diversity, sophistication, and size in academic writing. Presentation at the 13th Annual Symposium on Second Language Writing.

Graesser, A. C., McNamara, D. S., Louwerse, M. M.,&Cai, Z. (2004). Coh-Metrix: Analysis of text on cohesion and language. Behavioral Research Methods, Instruments, and Computers, 36, 193–202.

Johnson, M., Acevedo, A., & Mercado, L. (2013). What vocabulary should we teach?: Lexical frequency profiles and lexical diversity in second language writing. Writing & Pedagogy, 5(1), 82-103. doi: 10.1558/wap.v4i5.1

Laufer, B., & Nation, P. (1995). Vocabulary size and use: Lexical richness in L2 written production. Applied Linguistics, 16(3), 307-322.

Schmitt, N., & Schmitt, D. (2012). A reassessment of frequency and vocabulary size in L2 vocabulary teaching. Language Teaching, 47(4), 484-503.

Yu, G. (2009). Lexical diversity in writing and speaking task performances. Applied Linguistics, 31(2), 236-259.


Thank you! Melanie Gonzalez [email protected]

Please visit my Padlet to download a copy of this presentation and others:

http://padlet.com/ProfMelanie/lexicaldiversity

This research and presentation was funded in part by Salem State University's School of Graduate Studies and the Emilio and Mary DiFelice Endowment for Research in Education.