learning all-purpose academic words all-purpose academic words ... language group and amount of...

82
Learning all-purpose academic words Catherine E. Snow Harvard Graduate School of Education CREATE, 6 September 2007

Upload: lamtram

Post on 11-Mar-2018

219 views

Category:

Documents


5 download

TRANSCRIPT

Page 1: Learning all-purpose academic words all-purpose academic words ... language group and amount of intervention 0 5 10 15 20 25 30 35 ... global warming,

Learning all-purpose academic words

Catherine E. Snow Harvard Graduate School of Education

CREATE, 6 September 2007

Page 2: Learning all-purpose academic words all-purpose academic words ... language group and amount of intervention 0 5 10 15 20 25 30 35 ... global warming,

Welcome & Housekeeping • Discussion/Interactive Format

– Quick Polling

– Type messages into chat area

– Break for responding to chat questions/comments

– Those on just the teleconference can email questions to: [email protected]

Page 3: Learning all-purpose academic words all-purpose academic words ... language group and amount of intervention 0 5 10 15 20 25 30 35 ... global warming,

Quick Poll: Who is Online?

• elementary school teacher • secondary school teacher • school, district, or state administrator • curriculum coordinator • staff development specialist/trainer • post-secondary educator or administrator • researcher • other

Your primary profession

Page 4: Learning all-purpose academic words all-purpose academic words ... language group and amount of intervention 0 5 10 15 20 25 30 35 ... global warming,

The ultimate goal

Page 5: Learning all-purpose academic words all-purpose academic words ... language group and amount of intervention 0 5 10 15 20 25 30 35 ... global warming,

Poll: What are the biggest challenges to comprehension of content area texts in

the middle grades? Pick top two:

•Word reading difficulties

•Fluency

•Motivation

•Vocabulary

•Background knowledge

•Syntax

•Text structure

Page 6: Learning all-purpose academic words all-purpose academic words ... language group and amount of intervention 0 5 10 15 20 25 30 35 ... global warming,

My focus is vocabulary

• A complicated domain • Relation to reading

– Reading comprehension, obviously – But also early reading, perhaps via Lexical

Restructuring • Large social class differences

– Related to density of word exposure – Related to quality of word exposure – Related transactionally to literacy experience

Page 7: Learning all-purpose academic words all-purpose academic words ... language group and amount of intervention 0 5 10 15 20 25 30 35 ... global warming,

How is vocabulary related to

• Word reading? • Fluency? • Syntax? • Background knowledge? • Comprehension?

Page 8: Learning all-purpose academic words all-purpose academic words ... language group and amount of intervention 0 5 10 15 20 25 30 35 ... global warming,

My focus is middle school

• SERP principle: start with the most urgent

problem as defined by the practitioners • Reading for learning becoming an urgent

challenge at this age • Social class differences becoming

alarming • Possibly the last good chance for

intervention with struggling students

Page 9: Learning all-purpose academic words all-purpose academic words ... language group and amount of intervention 0 5 10 15 20 25 30 35 ... global warming,

Linking vocabulary to school reform

• The structure of the U.S. middle school – Large – Departmentalized – Little coordination across years or content areas – Many plagued by low internal accountability

• The focus of U.S. education reform: primary grades, inoculation model

• Lack of preparation/willingness among content area teachers to teach content area reading

• SERP Principle: work on student learning, teacher learning, organizational learning together

Page 10: Learning all-purpose academic words all-purpose academic words ... language group and amount of intervention 0 5 10 15 20 25 30 35 ... global warming,

Our work in Boston • Teacher surveys and interviews surfaced

vocabulary as a problem • Assessments (DRA, GRADE) confirmed it • Classroom observations:

– Teachers teach vocabulary rarely – Disciplinary, not all-purpose academic, vocabulary

focused on – All-purpose words crucial for understanding texts,

especially glossaries – Texts to be read are difficult and unengaging – Lively classroom discussion is rare

• Thus, Word Generation

Page 11: Learning all-purpose academic words all-purpose academic words ... language group and amount of intervention 0 5 10 15 20 25 30 35 ... global warming,

Word Generation Program Goals:

• Student level: Build knowledge of high frequency academic words in various contexts

• Teacher level: Promote regular use of effective instructional strategies

• School level: Facilitate faculty collaboration across grades, across departments

Page 12: Learning all-purpose academic words all-purpose academic words ... language group and amount of intervention 0 5 10 15 20 25 30 35 ... global warming,

Research-based Principles of Vocabulary Instruction: From

Studies of Young Children • Establish and discuss joint attentional

focus • Ensure affective engagement • Engage children in using the words • Ensure recurrent exposures • Celebrate successes • Encourage experimentation

Page 13: Learning all-purpose academic words all-purpose academic words ... language group and amount of intervention 0 5 10 15 20 25 30 35 ... global warming,

Research-based Principles of Vocabulary Instruction: From

Studies of School-aged Children • Pick the right words • Present them in motivating ways (texts) • Provide learner-friendly definitions • Ensure recurrent exposures • Expand each word’s semantic mapping • Provide opportunities to use the words • Teach word-learning strategies

(morphology, inferring from context) • Motivate ‘word awareness’

Page 14: Learning all-purpose academic words all-purpose academic words ... language group and amount of intervention 0 5 10 15 20 25 30 35 ... global warming,

New Challenges in Middle Grades

• Most students know the easy words already – Basic object terms – Brief/monomorphemic forms – Really frequent words – Minimally polysemous words – Or most frequent meaning of polysemous words

• Much word exposure comes through reading • Students need content-area technical terms • Students also need all-purpose academic words

– Category labels – Words for thinking – Abstract, low imageability terms

Page 15: Learning all-purpose academic words all-purpose academic words ... language group and amount of intervention 0 5 10 15 20 25 30 35 ... global warming,

Reflective Question

• In your experience, what kinds of words do middle school struggling readers have trouble with?

Page 16: Learning all-purpose academic words all-purpose academic words ... language group and amount of intervention 0 5 10 15 20 25 30 35 ... global warming,
Page 17: Learning all-purpose academic words all-purpose academic words ... language group and amount of intervention 0 5 10 15 20 25 30 35 ... global warming,

All-purpose academic words, e.g.

• Words for thinking: hypothesize, evidence, criterion

• Words for classifying: vehicle, utensil, process • Words for communication: emphasize, affirm,

negotiate • Words for expressing relationships: dominate,

correspond, locate http://www.vuw.ac.nz/lals/research/awl/index.html

Page 18: Learning all-purpose academic words all-purpose academic words ... language group and amount of intervention 0 5 10 15 20 25 30 35 ... global warming,

Previous efforts along these lines

Page 19: Learning all-purpose academic words all-purpose academic words ... language group and amount of intervention 0 5 10 15 20 25 30 35 ... global warming,

Principles of VIP

• Present words in context • Pick topics that ensure word recurrence • Provide native language support • Teach explicitly about cognates,

morphology, polysemy, and inferring word meanings

• Teach spelling linked to word meaning

Page 20: Learning all-purpose academic words all-purpose academic words ... language group and amount of intervention 0 5 10 15 20 25 30 35 ... global warming,

Table 4. Average Performance on the Mastery test as a function of time of test, language group and amount of intervention

0

5

10

15

20

25

30

35

Fall Spring

Time of Test

Number

correc

t

ELL never ELL 4th gradeELL 5th grade ELL 4th & 5th gradeEO never EO 4th gradeEO 5th grade EO 4th & 5th grade

Page 21: Learning all-purpose academic words all-purpose academic words ... language group and amount of intervention 0 5 10 15 20 25 30 35 ... global warming,

Table 2. Average performance on the polysemy task as a function of time of test, language group and amount of exposure

0

2

4

6

8

10

12

14

16

Fall Spring

Time of Test

Numb

er Co

rrect

ELL never EO neverELL 4th grade EO 4th gradeELL 5th grade EO 5th gradeELL 4th & 5th grade EO 4th & 5th grade

Page 22: Learning all-purpose academic words all-purpose academic words ... language group and amount of intervention 0 5 10 15 20 25 30 35 ... global warming,

Table 1. Average performance on the morphology task as a function of time of test, language group, and amount of exposure to the intervention

0

20

40

60

80

100

120

Fall Spring

Time of Test

Numb

er Co

rrect

ELL never EO neverELL 4th grade EO 4th gradeELL 5th grade EO 5th gradeELL 4 h & h d EO 4 h & h d

Page 23: Learning all-purpose academic words all-purpose academic words ... language group and amount of intervention 0 5 10 15 20 25 30 35 ... global warming,

Table 3. Average Performance on the Word Association test as a function of time of test, language group, and amount of intervention

0

10

20

30

40

50

60

Fall Spring

Time of Test

Numb

er Co

rrect

ELL never EO neverELL 4th grade EO 4th gradeELL 5th grade EO 5th gradeELL 4th & 5th grade EO 4th & 5th grade

Page 24: Learning all-purpose academic words all-purpose academic words ... language group and amount of intervention 0 5 10 15 20 25 30 35 ... global warming,

Table 5. Average performance on the cloze task as a function of time of test, language group, and amount of exposure to the intervention

0

2

4

6

8

10

12

14

16

18

Fall SpringTime of Test

Numb

er C

orre

ct

ELL never EO neverELL 4th grade EO 4th gradeELL 5th grade EO 5th gradeELL 4th & 5th grade EO 4th & 5th grade

Page 25: Learning all-purpose academic words all-purpose academic words ... language group and amount of intervention 0 5 10 15 20 25 30 35 ... global warming,

So • We can teach vocabulary to 4th-6th graders • Both ELLs and EOs But • We didn’t shrink the gap • And, most disturbingly, the intervention

disappeared when we did

Page 26: Learning all-purpose academic words all-purpose academic words ... language group and amount of intervention 0 5 10 15 20 25 30 35 ... global warming,

Reflective Question

Why would teachers not continue to use

the VIP after the end of the study?

Page 27: Learning all-purpose academic words all-purpose academic words ... language group and amount of intervention 0 5 10 15 20 25 30 35 ... global warming,

The Word Generation Approach • Summer 2006: 20-week curriculum designed,

two schools (grades 6-8) plus one teacher (grade 5) recruited

• 2006-2007 Piloting – 20 weeks implemented in 5th grade classroom, 12

weeks in two entire schools – Pre- and post-testing – Writing samples collected, partially analyzed – Classroom observations, teacher interviews, student

surveys • Summer 2007: design year two of curriculum • 2007-2008: larger scale implementation • 2009-2010: clinical trial with randomization at

school level

Page 28: Learning all-purpose academic words all-purpose academic words ... language group and amount of intervention 0 5 10 15 20 25 30 35 ... global warming,

Word Generation Program Features

• Words selected from the Academic Word List (AWL)

• Materials designed for all content-area teachers

• Expectation of at least 15 instructional minutes a day

• Flexibility for school-specific implementation models

Page 29: Learning all-purpose academic words all-purpose academic words ... language group and amount of intervention 0 5 10 15 20 25 30 35 ... global warming,

Word Generation: Materials (year 1)

• 20 weeks, each focused on a set of 5 words • Controversies include: global warming,

censorship, dress codes and schools, steroids and sports, censorship and hip-hop, junk food and schools, the nature of American culture, etc. Monday

Paragraph introduces

words

Tuesday-Thursday Content-area word activities

Friday Writing with focus words

Page 30: Learning all-purpose academic words all-purpose academic words ... language group and amount of intervention 0 5 10 15 20 25 30 35 ... global warming,

WEEK 8 Global warming: What should be done?

Global climate statistics suggest that the average temperature of the earth's surface is increasing. For example, the warmest ten years of the 20th century were between 1985 and 2000. Another statistic indicates that surface temperatures have risen by about 1˚F since the late 1800’s. Though this change may seem small, it has raised the ocean level by 4-8 inches. This is because more snow and ice are melting into the sea. Many scientists support the hypothesis that global warming is linked to heavier storms, floods, and other extreme weather. They attribute these changing environmental conditions to human activities like driving cars that use a lot of gas. Scientists believe that people contribute to global warming through burning fossil fuels (coal, oil, and natural gas). Such activities increase certain gases that trap the sun’s energy inside the atmosphere and warm the earth. This is called the greenhouse effect. Scientists say that this warm period is not just part of the earth’s natural climate cycle. This trend does not fit the usual pattern of warm periods followed by cool periods. Scientists project that temperatures will keep rising if we continue to ignore the impact of our activities. Should people be allowed to drive SUVs? Should companies be allowed to make them? Should the government invest in exploring other energy sources? Who is responsible for preventing future destruction?

Page 31: Learning all-purpose academic words all-purpose academic words ... language group and amount of intervention 0 5 10 15 20 25 30 35 ... global warming,

Midweek activities

• Social studies: debates of various sorts, social-studies specific uses of the words

• Math: studying graphs, math problem of the week, math-specific uses of the words

• Science: science problems, sentence starters using target words, science-specific uses of the words

Page 32: Learning all-purpose academic words all-purpose academic words ... language group and amount of intervention 0 5 10 15 20 25 30 35 ... global warming,

Pause for clarification questions about VIP or

WG design

Page 33: Learning all-purpose academic words all-purpose academic words ... language group and amount of intervention 0 5 10 15 20 25 30 35 ... global warming,

Pilot Schools/Demographics

Westfield Middle School • 80 % Black • 16% Hispanic • 1.8 White • 1.6 Asian • 29% Special

Education • MCAS

Reilley Middle School • 62% Black • 18.1 % Hispanic • 9.3% White • 8.9 % Asian • 25% Special

Education • MCAS

Page 34: Learning all-purpose academic words all-purpose academic words ... language group and amount of intervention 0 5 10 15 20 25 30 35 ... global warming,

Massachusetts Comprehensive Assessment System (MCAS)

Results 2006 (ELA) • Westfield

• Reilley 6th 7th 8th A/P 56% 52% 66% NI 41% 44% 29% W/F 3% 4% 6%

6th 7th 8th A/P 26% 23% 39% NI 51% 48% 43% W/F 23% 29% 18%

Page 35: Learning all-purpose academic words all-purpose academic words ... language group and amount of intervention 0 5 10 15 20 25 30 35 ... global warming,

GRADE data/6th grade

Mean stanine score

% below 3rd stanine (> 1 SD below mean)

Westfield 3.6 49

Reilley 4.4 29

Page 36: Learning all-purpose academic words all-purpose academic words ... language group and amount of intervention 0 5 10 15 20 25 30 35 ... global warming,

Two Vocabulary Assessments at Pre- and Post-testing

Vocabulary Self-Check (VSC) – student gauges his/her own level of knowledge

about a word (40 items= 30 WG words and 10 non-words

Multiple Choice (Pre-WG) – 30 WG words chosen from 100 WG words to be

taught over 20-week intervention

“I do not “I have “I know something “I know it well know it” heard of it” about it” and can use it.”

Page 37: Learning all-purpose academic words all-purpose academic words ... language group and amount of intervention 0 5 10 15 20 25 30 35 ... global warming,

PRE-VSC Across Grades, from least to most well known

Target Word

% 1-3: not known well enough to use %_4 %_1

Orious 98% 2% 84%

Lachritude 98% 2% 83%

Eleemosynary 97% 3% 91%

Codufied 96% 4% 78%

Reavity 96% 4% 66%

Trandict 96% 4% 63%

Phagus 95% 5% 78%

Delapse 94% 6% 57%

Stistics 91% 9% 55%

Vocate 90% 10% 41%

Reluctant 82% 18% 29%

Bulk 76% 24% 26%

Diverse 66% 34% 17%

Sufficient 63% 37% 18%

Hypothesis 62% 38% 14%

Prohibit 62% 38% 13%

Interpret 62% 38% 18%

Acknowledge 54% 46% 14%

Decades 51% 49% 20%

Indicate 50% 50% 5%

Exhibit 49% 51% 9%

Analyze 44% 56% 7%

Contribute 43% 57% 7%

Impact 39% 61% 5%

Conflict 37% 63% 3%

Monitor 36% 64% 5%

Cycle 29% 71% 4%

Promote 28% 72% 4%

React 28% 72% 9%

Route 27% 73% 3%

Dramatic 27% 73% 5%

Reveal 26% 74% 4%

Collapse 24% 76% 7%

Suspend 24% 76% 4%

Transferred 23% 77% 7%

Abandon 20% 80% 7%

Substitute 18% 82% 3%

Release 13% 87% 3%

Enormous 12% 88% 2%

Project 8% 92% 2%

Page 38: Learning all-purpose academic words all-purpose academic words ... language group and amount of intervention 0 5 10 15 20 25 30 35 ... global warming,

Multiple Choice (Pre-WG)

Sample items 1. She indicated that she was hungry. □ a. denied □ b. thought □ c. showed □. d. indeed 2. He will analyze the information. □ a. ignore □ b. anchor

□ c. remember □ d. examine

Page 39: Learning all-purpose academic words all-purpose academic words ... language group and amount of intervention 0 5 10 15 20 25 30 35 ... global warming,

Analysis Across Grades for Multiple Choice Pretest

Reilley %ortho %sem %plausible %correct (valid)

Gr. 6 9.4% 11.6% 10.4% 70.5%

Gr. 7 8.6% 10.5% 10.1% 72.5%

Gr. 8 7.7% 10.4% 8.5% 74.1%

Page 40: Learning all-purpose academic words all-purpose academic words ... language group and amount of intervention 0 5 10 15 20 25 30 35 ... global warming,

WG measures correlate with each other, r = .47**

0.00 50.00 100.00 150.00

vsc_sum

10

20

30

prem

c_ttl

AAA A

A AAAA

AAA A

A AAA A

A AA A A AAA AAAA AA

A AA AAA A AAAA AAAA AAA AA AA A AA A

AA A A AAAAA AAA A

AA AAAA AAAA AA A A

AAA A AAAA AA AA AAA AA AAAAAAA AA AA

A AA AAAA AA

A

Page 41: Learning all-purpose academic words all-purpose academic words ... language group and amount of intervention 0 5 10 15 20 25 30 35 ... global warming,

WG measures correlate with the

GRADE Vocabulary subtest for 6th Graders

Multiple choice vocabulary test

Vocabulary self assessment

Grade vocabulary

r =.67** n =123

r =.56** n =123

Page 42: Learning all-purpose academic words all-purpose academic words ... language group and amount of intervention 0 5 10 15 20 25 30 35 ... global warming,

10 20 30

premc_ttl

73768083868891939698

100103105107109111113115117119121123130

Stan

dard

Sco

re

A

AA

A

A

A

AA

A

A

A

A

A

A

A

AA

AAAA

A

AA

A

A

A

A

A

AAA

A

A

A

A

A

A

A

A

AA A

A

AA

A

A

A

AAA

A

AA

A

A

A

A

A

AA

A

A

A

A

A

A

AA

A A

A

A

A

A

AAA

A

A

AA

A

A

A

A

AA

A

A

A

A

A

A

A

AA

A

A

A

A

A

A

A

A

A

AAA

AA

A

A

AA

A

AA

A

A

A A

0.00 50.00 100.00 150.00

vsc_sum

73768083868891939698

100103105107109111113115117119121123130

Stan

dard

Sco

re

A

AA

A

A

A

AA

A

A

A

A

A

A

A

AA

AA

AA

A

AA

A

A

A

A

A

A AA

A

A

A

A

A

A

A

A

AA A

A

AA

A

A

A

A AA

A

A A

A

A

A

A

A

A A

A

A

A

A

A

A

AA

AA

A

A

A

A

A AA

A

A

AA

A

A

A

A

A A

A

A

A

A

A

A

A

AA

A

A

A

A

A

A

A

A

A

AAA

AA

A

A

AA

A

AA

A

A

A A

WG measures correlate with the

GRADE Vocabulary subtest for 6th Graders

Page 43: Learning all-purpose academic words all-purpose academic words ... language group and amount of intervention 0 5 10 15 20 25 30 35 ... global warming,

WG measures correlate with the

state ELA accountability assessment

Multiple choice vocabulary test

Vocabulary self assessment

MCAS ELA r =.53** n = 349

r =.47** n = 349

Page 44: Learning all-purpose academic words all-purpose academic words ... language group and amount of intervention 0 5 10 15 20 25 30 35 ... global warming,

Pre-WG Measures & MCAS Reading

mcas_2: MCAS Reading score (Spring 2006)

vsc_sum: total correct on Pre-WG Vocabulary Self-Check (Spring 2007)

0.00 50.00 100.00 150.00

vsc_sum

220.00

240.00

260.00

mca

s_2

A

A

A

A

A

A

AA

A

A

A

A

A

A

A

A

A

A

A

A

A

A

A

A

A

A

A

AA

A

A

A

A

A

A

A

A

A

A

A

A

AA

A

A

A

A

A

A

AA

A

A

A

A

AA

A

A

A

A

A

A

AA

A

AA

A A

A

A

AA

A

A

A

A

A

A

A

A

A

A

A A

A

A

A

A

A

A

A

A

A

A

A

A

A

A

AAA

A

A

A

A

A

A A

A

A

A

A

A A

A

A

A

A

A

A

AAA

A

A

A

A

A

A

A

AA

A A

A

A

A

A

A

A

A

A

A

A

A

AA

A

A

A

A

A

A

A

AA

A

A

A

A A

A

A

A

A

A

A

A

A

AA

AA

A

A

A

A

A

A

A

A A

A

A

A

A

A

AA

A

A

A

AA

A

A

A

AA

A

A

AA

A

A

A

A

AA

AA

A

A

A

A

A

A

A

A

A

AA

A

A

AA

A

A

A

A A

A

A

A

A

A

A

A

A

A

AA

A

A

AA

A

A

A

A

A

A

A

A

AA

A

A

A

AA

A

A

A

A

A

A

A

A

AA

A

A

A

A

A

A

A

A

A

A A

A

A

A

A

A

A

A

A

A

A

A

A

A

A

A

A

AAA

A

A

AA

A

A

A

A

A

A

A AA

A

A

A

AA

A

A

A

A

A

A

A

A

A

A

A

A

A

A

A

A

A

A

A

A

A

A

A

A

A

A

A

A

0 25 50 75 100

mc_pct

220.00

240.00

260.00

mca

s_2

A

A

A

A

A

A

A

A

A

A

A

A

A

A

A AA

A

A

A

A

A

A

A

A

A A

AA

A

A

A

A

A

A

A

A

A

A

A

A

A

A

A

AA

A

A

A

A

A

A

A

A

A

A

A

A

A

A

A

A

A

AA

A

AAA

A

AA

A

A

A

A

A

A

A

A

A

A

A

A

A

A

A

A

A

A

A

A

A

A

A

A

A

A

A

AA

A

A

A

A

A

A

A

A

A

A

A

A

A

A

A

A

A

A

AA

A

A

A

A

AA

A

A

A

A

A

A

A

A

A

A

A

A

A

AA

A

A

A

A

A

A

A

A

A

A

A

A

A

A

A

A

A

AA

A A

A

A

A

A

A

A

A

A

A

AA

A

A

A

A

A

A

A

A

A A

A

A

A

A

A

A

A

A

A

A

A

A

A

A

A A

A

A

A

A

AA

A

A

A

A

A

A A

A

A

A

AA

A

A

A

A

A

A

A

A

A

A

AA

A

A

A

A

A

A

A

A

A

A

A

A

A

A

AA

A

A

A A

A

AA

A

AA

A

A

A

A

A

AAA

A

A

A

A

A

A A

A

A

A

A

A

A

A A

A

A

A

A

A

A

A

A

A

A

A

A

A

A

A A

A

A

AA

A

A

AA

A

A

AA

AA

A

AA

A

A

A

A

A

A

A

A

A

A

A

A

AA

A

A A

A

A

A

A

A

A

A A

A

A

A

A

A

A

A

A

A

A

A

A

AA

A

AA A

A

A A

A

A

A

A

A

A A

A

A

A

A

A

A

AA

A AA

A

A

A

A

A

A

A

A

A

A

A

A

mcas_2: MCAS Reading score (Spring 2006)

mc_pct: %correct on Pre-WG Multiple Choice Test

Scatterplot of MCAS Reading and Pre-WG MC (n=349)

Scatterplot of MCAS Reading and Pre-WG VSC (n=349)

Page 45: Learning all-purpose academic words all-purpose academic words ... language group and amount of intervention 0 5 10 15 20 25 30 35 ... global warming,

Students who know more than 85% of WG pretest words pass the MCAS

MC_PCT

979083777063575043372717

Perce

nt

70

60

50

40

30

20

10

0

MCAS_1

Warning

Needs Improvement

Proficient

Advanced

Page 46: Learning all-purpose academic words all-purpose academic words ... language group and amount of intervention 0 5 10 15 20 25 30 35 ... global warming,

Students need to know at least 80% of the WG words to do well on the MCAS

MCAS_1

WarningNeeds ImprovementProficientAdvanced

Mea

n M

C_PC

T100

90

80

70

60

50

40

GRADE

6

7

8

Page 47: Learning all-purpose academic words all-purpose academic words ... language group and amount of intervention 0 5 10 15 20 25 30 35 ... global warming,

So… • The tests work pretty well • And correlate with other important

outcomes But… • Does the intervention work?

Page 48: Learning all-purpose academic words all-purpose academic words ... language group and amount of intervention 0 5 10 15 20 25 30 35 ... global warming,

Break to Review Chat Questions

Page 49: Learning all-purpose academic words all-purpose academic words ... language group and amount of intervention 0 5 10 15 20 25 30 35 ... global warming,

Reilley Multiple Choice Pre- and Posttest Results

6.00 7.00 8.00

Grade

0.00

10.00

20.00

30.00

MC_

PRE

A

A

AA

AA

AAA

6.00 7.00 8.00

Grade

10

20

30

MC_

POST

A

A

A

A

A

AAA

A

Page 50: Learning all-purpose academic words all-purpose academic words ... language group and amount of intervention 0 5 10 15 20 25 30 35 ... global warming,

6.00 7.00 8.00

Grade

25.00

50.00

75.00

100.00

mcp

re_p

erce

nt

A

A

A

6.00 7.00 8.00

Grade

0

25

50

75

100

mcp

ost_

perc

ent

A

S

Westfield Multiple Choice Pre- and Posttest Results

Page 51: Learning all-purpose academic words all-purpose academic words ... language group and amount of intervention 0 5 10 15 20 25 30 35 ... global warming,

Multiple Choice Test Results

Mean percent correct

Pre Post

Paired T-tests

n t Westfield 6th

Westfield 7th

Westfield 8th

62.41 68.85 65.58 75.22 74.06 83.28

29 3.33** 46 6.03*** 64 6.41***

Reilley 6th

Reilley 7th

Reilley 8th

70.48 75.13 72.51 77.58 74.11 81.47

104 1.78~ 109 2.43** 120 3.64***

Page 52: Learning all-purpose academic words all-purpose academic words ... language group and amount of intervention 0 5 10 15 20 25 30 35 ... global warming,

Intervention Effect Sizes by Grade (using pooled SD)

Westfield Reilley • Grade 6: 0.45 0.25

• Grade 7: 0.57 0.33

• Grade 8: 0.71 0.45

Page 53: Learning all-purpose academic words all-purpose academic words ... language group and amount of intervention 0 5 10 15 20 25 30 35 ... global warming,

In other words

• Effects are significant both statistically and educationally

• Furthermore, teachers like the program and want to do it again

Page 54: Learning all-purpose academic words all-purpose academic words ... language group and amount of intervention 0 5 10 15 20 25 30 35 ... global warming,

Where are we with WG?

• Weeks 1-12 implemented in two middle

schools • Weeks 1-20 implemented in one 5th grade • Coding writing samples • Coding teacher feedback • Gauging effective implementation through

various data sources

Page 55: Learning all-purpose academic words all-purpose academic words ... language group and amount of intervention 0 5 10 15 20 25 30 35 ... global warming,

Single Classroom Study carried out by Sarah Meacham

• Self-contained 5th grade classroom, Mystic School • Video-recording and observation of Weeks 8 and 11 of WG • Transcription of talk and activity (+/- 200 m) into CA format • Collection of student writing assignments

Page 56: Learning all-purpose academic words all-purpose academic words ... language group and amount of intervention 0 5 10 15 20 25 30 35 ... global warming,

Analysis

•Counted instances of attempted and appropriate word use in student writing

•Counted target word use in class • Identified discourse structures and activities within which target words were embedded

Page 57: Learning all-purpose academic words all-purpose academic words ... language group and amount of intervention 0 5 10 15 20 25 30 35 ... global warming,

Implementation of Word Generation

at The Mystic School

• Ms. Ross’ 5th grade class of 24-26 students • 100% former English Language Learners • Ms. Ross teaches all five days = 100 minutes/week of instructional time devoted to Word Generation

Page 58: Learning all-purpose academic words all-purpose academic words ... language group and amount of intervention 0 5 10 15 20 25 30 35 ... global warming,

Monday/Tuesday: informal assessment, theories of word meaning, reading of passage, talk about topic, scanning of text/annotation

Tuesday/Wednesday: informal assessment, group work relating words to world

Wednesday/Thursday: debate

Friday: essay writing

Page 59: Learning all-purpose academic words all-purpose academic words ... language group and amount of intervention 0 5 10 15 20 25 30 35 ... global warming,

Writing Prompt WEEK 8

What should be done to prevent global warming? If you feel nothing should be done,

say that. Have information to back up what you believe in.

Page 60: Learning all-purpose academic words all-purpose academic words ... language group and amount of intervention 0 5 10 15 20 25 30 35 ... global warming,

Word Use Rating

Appropriate: “My hypothissis is that global warming will get worse if no one does anything about it.” Indeterminate: “The global warming has a green house and it goes in a cycle by little by little.” Inappropriate: “I attribute my friends to stop waisting elecrisity and fuel.”

Page 61: Learning all-purpose academic words all-purpose academic words ... language group and amount of intervention 0 5 10 15 20 25 30 35 ... global warming,

WEEK 8 WORDS appropriate use in student writing as a proportion of attempted use

statistics project hypothesis cycle attribute

9 9 9

7

3

8

7

5

3

Page 62: Learning all-purpose academic words all-purpose academic words ... language group and amount of intervention 0 5 10 15 20 25 30 35 ... global warming,

Appropriate use as a proportion of attempted use next to number of mentions during week 8

28 30

19

37

9

statistics project hypothesis cycle attribute

9 9

7

9

3

8 7

5

3

Page 63: Learning all-purpose academic words all-purpose academic words ... language group and amount of intervention 0 5 10 15 20 25 30 35 ... global warming,

WEEK 11 WORDS appropriate use in student writing as a proportion of attempted use

acknowledge incorporate transport incidence initiative

11 11

8

4 4

9 9

6

3

2

Page 64: Learning all-purpose academic words all-purpose academic words ... language group and amount of intervention 0 5 10 15 20 25 30 35 ... global warming,

acknowledge incorporate transport incidence initiative

Appropriate use as a proportion of attempted use next to number of mentions during week 11

11 11

8

4 4

9 9

6

3 2

17

22 21

17 18

Page 65: Learning all-purpose academic words all-purpose academic words ... language group and amount of intervention 0 5 10 15 20 25 30 35 ... global warming,

Explaining Attempted Uses in Ms. Ross’ Students’ Writing

Number of times the word is mentioned over the course of the week seems to have some association with the number of attempted uses of the word in students’ writing

Page 66: Learning all-purpose academic words all-purpose academic words ... language group and amount of intervention 0 5 10 15 20 25 30 35 ... global warming,

Multiple Choice Test Comparison (analyses carried out by Jeannette

Mancilla-Martinez)

Intervention Class • Pretest Performance:

= 20/30 words

• Gain over the 20 weeks: = 25/30 words

(or 6 words)

Comparison Class • Pretest Performance:

= 20/30 words

• Gain over the 20 weeks: = 21/30 words

(or 1 word)

Page 67: Learning all-purpose academic words all-purpose academic words ... language group and amount of intervention 0 5 10 15 20 25 30 35 ... global warming,

Words Students Learned

• Hypothesis: from 32% to 88% • Project: from 40% to 83% • Monitor: from 56% to 88% • Bulk: from 56% to 96% • Decade: from 64% to 96% • Sufficient: from 67% correct to 96%

Page 68: Learning all-purpose academic words all-purpose academic words ... language group and amount of intervention 0 5 10 15 20 25 30 35 ... global warming,

Becoming More Word Conscious

Treatment Comparison

Under- estimated

Over- estimated

On Target Under- estimated

Over- estimated

On Target

Pre 16 2 12 13 3 14

Post 4 2 24 9 8 13

Comparison of students’ actual performance on the Multiple Choice test to their reported knowledge of the words

Page 69: Learning all-purpose academic words all-purpose academic words ... language group and amount of intervention 0 5 10 15 20 25 30 35 ... global warming,

Scoring Writing with a Rubric

• 3 Sections: ideas, overall cohesion, and academic language

• Possible Points: each section ranged from 0-3 points for a total scale of 0-9 points

• Overall Reliability = .86 (note: anything above .80 is considered almost perfect)

Page 70: Learning all-purpose academic words all-purpose academic words ... language group and amount of intervention 0 5 10 15 20 25 30 35 ... global warming,

Representative Individual Trajectories

Initially Low Initially Average

Initially High Consistently Average

Page 71: Learning all-purpose academic words all-purpose academic words ... language group and amount of intervention 0 5 10 15 20 25 30 35 ... global warming,

Students’ Writing Quality Scores Over Time

Page 72: Learning all-purpose academic words all-purpose academic words ... language group and amount of intervention 0 5 10 15 20 25 30 35 ... global warming,

Writing Quality Results • Average quality score = 4.53 (SD = 1.05) • Average growth over the 20 weeks = .04*** per

week (or .70 points total)

• Interestingly… more growth occurred during the last 10 weeks of the intervention: – First 10 weeks = .03 per week (or .58 points) – Second 10 weeks = .04* per week (or .81 points)

Page 73: Learning all-purpose academic words all-purpose academic words ... language group and amount of intervention 0 5 10 15 20 25 30 35 ... global warming,

.

Average type/token ratio showed a moderate increase

y = 0.007x + 0.7198R2 = 0.4937

0.6

0.65

0.7

0.75

0.8

0.85

0.9

0.95

1 2 3 4 5 6 7 8 9 10

Week

Ave

rage

Typ

e/To

ken

Rat

io

Page 74: Learning all-purpose academic words all-purpose academic words ... language group and amount of intervention 0 5 10 15 20 25 30 35 ... global warming,

Students’ Used Previously

Taught WG Words 2 3 4 5 6 7 8 9 10

factor ( wk 1) 1 1 analyze (wk 1) 1 1 1 4 1 2 1 Interpret (wk 1) 2 1 1 Indicate (wk 2) 1 impact (wk 4) 23* 2 context (wk 2) 1 1 1 culture (wk 3) 1 sufficient (wk 5) 1 benefit (wk 2) 1 2 2 react (wk 6) 1 rely (wk 6) 1 resourceful (wk 3) 1 contribute (wk 5) 1 demonstrate (wk 5 ) 1

17 out of 26 students used at least one WG word learned in a previous week.

* Word used in essay prompt

Page 75: Learning all-purpose academic words all-purpose academic words ... language group and amount of intervention 0 5 10 15 20 25 30 35 ... global warming,

Vocabulary Average (five target words) Weeks 1-7

y = 0.5546x + 2.6714R2 = 0.2574

0

1

2

3

4

5

6

7

1 2 3 4 5 6 7

Weeks

Vo

cab

ula

ry T

ota

l

0=did not use the word

1=Used the word arbitrarily and inappropriately

2=Used the word in a context that made some sense, but not quite appropriately

3=Used the word appropriately

Page 76: Learning all-purpose academic words all-purpose academic words ... language group and amount of intervention 0 5 10 15 20 25 30 35 ... global warming,

Criteria for Effective

Implementation of WG

• Student Level

• Teacher Level

• School Level

Page 77: Learning all-purpose academic words all-purpose academic words ... language group and amount of intervention 0 5 10 15 20 25 30 35 ... global warming,

What have we learned about vocabulary development?

• The principles established in research with pre-school aged and younger school-aged children work

• But there is much more to learn about teaching ‘academic words’

• As always, implementation is a bigger challenge than developing curricula

• Vocabulary instruction carries broader academic language skills with it

Page 78: Learning all-purpose academic words all-purpose academic words ... language group and amount of intervention 0 5 10 15 20 25 30 35 ... global warming,

statisticsACKNOWLEDGEhypothesistransportPROJECTINCORPORATEATTRIBUTEinitiativescycleINCIDENCEstatisticsACKNOWLEDGEhypothesistransportPROJECTI

NCORPORATEATTRIBUTEinitiativescycleINCIDENCEstatisticsACKNOWLEDGEhyp

othesistransportPROJECTINCORPORATEATTRIBUTEinitiativescycleINCIDENCEstatisticsACKNOWLEDGEhypothesistransportPROJECTINCORPORATEATTRIBUTEinitiativescycleINCIDENCEstatisticsACKNOWLEDGEhypothesistransportPROJECTINCORP

ORATEATTRIBUTEinitiativescycleINCIDENCEstatisticsACKNOWLEDGEhypothesistransportPROJECTINCORPORATEATTRIBUTEinitiativescycleINCIDENCEstatisticsACKNOWLEDGEhypothesistransportPROJECTINCORPORATEATTRIBUTEinitiativescycleINCIDENCEstatisticsACKNOWLEDGEhypothesistransportPROJECTINCORPORATE

ATTRIBUTEinitiativescycleINCIDENCEstatisticsACKNOWLEDGEhypothesistransportPROJECTINCORPORATEATTRIBUTEinitiativescycleINCIDENCEstatisticsACKNOWLEDGEhypothesistransportPROJECTINCORPORATEATTRIBUTEinitiativescycleINCIDENCEstatisticsACKNOWLEDGEhypothesistransportPROJECTINCORPORATEATTRIBUTEinitiativescycleINCIDENCEstatisticsACKNOWLEDGEhypothesistransportPROJECTINCORPORATEATTRIBUTEinitiativescycleINCIDENCEstatisticsACKNOWLEDGEhypothesistransportPROJECTINCORPORATEATTRIBUTEinitiativescycleINCIDENCEstatisticsACKNOWLEDGEhypothesistransportPROJECTINCORPORATEATTRIBUTEinitiativescycleINCIDENCEstatisticsACKNOWLEDGEhypothesistransportPROJE

Thanks to:

Joanna Christodoulou Rachel Currie-Rubin

Christina Dobbs Richard Elmore Michelle Forman Lasse Isakson Michael Kieffer

Jeannette Mancilla-Martinez Sarah Meacham

Claire White Jen Zeuli

Spencer and Hewlett Foundations Council of Great City Schools

Page 79: Learning all-purpose academic words all-purpose academic words ... language group and amount of intervention 0 5 10 15 20 25 30 35 ... global warming,

Additional information

• Vocabulary Improvement Program, Brookes Publishing, Baltimore

• www.serpinstitute.org

Page 80: Learning all-purpose academic words all-purpose academic words ... language group and amount of intervention 0 5 10 15 20 25 30 35 ... global warming,

Final Questions/Discussion

Page 81: Learning all-purpose academic words all-purpose academic words ... language group and amount of intervention 0 5 10 15 20 25 30 35 ... global warming,

Supplementary material

Page 82: Learning all-purpose academic words all-purpose academic words ... language group and amount of intervention 0 5 10 15 20 25 30 35 ... global warming,

Next Steps • http://www.schoolsmovingup.net/events/academicwords

• http://www.schoolsmovingup.net/events/academicwords/

survey.htm