anybody listening? the role of listening in integrated listening-to … · 2015-12-02 · what...

Anybody listening? The role of listening in integrated listening-to-write and listening-to-speak tasks

Anchana Rukthong & Tineke Brunfaut LTF 2015, Oxford

Please cite as:

Rukthong, A. & Brunfaut, T. (2015). Anybody listening? The role of listening in integrated listening-to-write and listening-to- speak tasks. Paper presented at LTF 2015. Oxford, UK, November, 20-22.

Copyright © Rukthong, A. & Brunfaut, T., 2015

What?

Integrated-skills assessment

Information processing & transferring

Input text(s)

Listening Reading Speaking Writing

Task production

Why?

• Strengths

‒ More authentic test tasks (Asencion, 2004; Cumming et al., 2004)

‒ Positive washback (Weigle, 2004)

‒ Fairness (Cumming et al., 2004)

‒ Greater inter-rater reliability (Weir, 2004)

• Weaknesses

‒ Unclear test construct (Weir, 1990)

‒ Difficulty in score interpretations (Lewkowicz, 1997)


Why research integrated-listening tasks?

1. Research focus re integrated-skills tasks primarily on productive skills (E.g., Brown et al., 2005; Frost et al., 2011), not receptive skills

2. No consensus on role of comprehension in integrated-skills tasks (E.g., Gebril, 2004 vs Plakans, 2008)

3. Comparatively limited insights in listening (Field, 2013;

Lynch, 2009; Rost, 2011)


Listening comprehension processing

Cognitive processes

Adapted from Anderson (1985), Rost (2011), and Field (2013)

Adapted from Goh and Vandergrift (2012)

Comprehension processing

Text understanding

Cognitive strategies

Metacognitive strategies

Input

Adapted from Rost (2011), Goh and Vandergrift (2012)

Listening comprehension processing

Acoustic-phonetic processing Pragmatic processing

Parsing

Word recognition Semantic processing (global)

Semantic processing (local)

Cognitive processes

Reconstruction

Translation Prediction

Elaboration

Inferencing

Fixation Comprehension evaluation

Real-time assessment of input

Comprehension monitoring

Directed attention

Selective attention

Preparing for listening

Metacognitive strategies


What listening abilities are assessed by integrated listening-to-summarize tasks?

a) What listening processes and strategies do ESL test-takers engage in while performing integrated listening-to-summarize tasks?

b) Are there any differences in the processes and strategies when compared between task modalities and performance levels?

Research questions

Materials

‒ Integrated listening-to-summarize tasks

‒ 4 tasks adapted from the PTE Academic

‒ 2 versions per task

Methodology

Listening

Listening-to-speak task

Listening-to-write task

Speaking or Writing

Oral/written summary

Participants

• 12 Thai-L1 students from a UK university

• 6 female, 6 male

• Age range: 23-40y (M= 29)

• English proficiency (IELTS): ‒ Overall: 6.0-8.5

‒ Listening: 5.5-7.5

‒ Reading: 5.5-8.5

‒ Writing: 5.5-8.0

‒ Speaking: 5.5-8.5

Methodology

Procedures

Methodology

Task 1

Background Questionnaire

Task 2

Task 3

Task 4

Stimulated recall

Stimulated recall

Stimulated recall

Stimulated recall

Data collection

Methodology

Stimulated recalls

Participants’ notes

Written summary content

…his father told him that….

Oral summary content

Analyses

• Summaries marked by two experienced PTE Academic raters

• Written summary: content, grammar and vocabulary

• Oral summary: content, pronunciation, fluency

• Stimulated-recall data + participants’ notes + summary content

• 2 coders

• Comprehension processing model

• Agreement (Kappa): 0.77- 0.88

Methodology

Findings (RQ1)

0

2

4

6

8

10

12

14

Acoustic-phoneticprocessing

Word recognition Parsing Semanticprocessing (local)

Semanticprocessing

(global)

Pragmaticprocessing

Par

tici

pan

ts

Cognitive Processes

Listening A

Listening B

Listening C

Listening D

0

2

4

6

8

10

12

14

Acoustic-phoneticprocessing

Word recognition Parsing Semanticprocessing (local)

Semanticprocessing

(global)

Pragmaticprocessing

Par

tici

pan

ts

Cognitive Processes

Listening A

Listening B

Listening C

Listening D

Lower –level processes Higher –level processes

Findings

0

2

4

6

8

10

12

14

Prediction Fixation Inferencing Elaboration Reconstruction

Par

tici

pan

ts


Listening A

Listening B

Listening C

Listening D

Findings

0

2

4

6

8

10

12

Preparing forlistening

Selectiveattention

Directedattention

Comprehensionmonitoring

Real-timeassessment of

input

Comprehensionevaluation

Par

tici

pan

ts

Metacognitive Strategies

Listening A

Listening B

Listening C

Listening D

Interaction between processes and strategies


Fixation

Directed attention

Listening input

Acoustic-phonetic processing

Word recognition


Semantic processing (global)

Pragmatic processing

Parsing +

-

Low

er-l

ever

Interaction between processes and strategies

Inferencing

Elaboration

Listening input


Word recognition


Semantic processing (global)

Pragmatic processing

Parsing


H

igh

er-l

ever

Listening-to-speak vs. listening-to-write:

Findings (RQ2)

Cognitive processing

Listening-to-speak Listening-to-write

Aco

ust

ic-p

ho

ne

tic

Wo

rd r

eco

gnit

ion

Par

sin

g

Sem

anti

c (l

oca

l)

Sem

anti

c (g

lob

al)

Pra

gmat

ic

Aco

ust

ic-p

ho

ne

tic

Wo

rd r

eco

gnit

ion

Par

sin

g

Sem

anti

c (l

oca

l)

Sem

anti

c (g

lob

al)

Pra

gmat

ic

Hig

h

sco

rer Listening A

Listening B

Listening C

Ave

rage

sc

ore

r

Listening A

Listening B

Listening C

Listening D

Low

sc

or

er

Listening A

Listening D

er

Directed attention

Co

mp

rehen

sion

Mo

nito

ring


Parsing

Word recognition


Semantic processing (global)*

Pragmatic processing*

Infe

ren

cin

g

Elab

ora

tio

n

Co

mp

reh

ensi

on

Mo

nit

ori

ng

Fixa

tio

n

Rea

l-ti

me

asse

ssm

ent

of

inp

ut*

Inferen

cing

Elabo

ration

Directed attention

Fixation

Acoustic-phonetic processing + Word

recognition + Parsing

Pre-listening preparation

Prediction Prediction Selective attention

Listening Input

Low-scorer High-scorer

Task output

Comprehension evaluation

Reconstruction

Tow

ard

th

e en

d

At

the

begin

nin

g

Wh

ile

list

enin

g

Selective attention

• Complex interaction of comprehension processes and strategies during integrated-listening task performance

• Common processes and strategies shared between listeners, but with different levels of success

• ‘Successful listeners’:

‒ higher-level processes,

‒ real-time assessment of input

‒ more automatic and accurate lower-level processes,

‒ more accurate inferences,

‒ more effective comprehension monitoring

Conclusion

• Listening abilities play an important role in integrated-listening task performance

Conclusion

Listening should be recognised to be part of the construct tested in listening-to-speak and listening-to-write tasks

Anybody listened?

Anybody questions?

Thank you!

Anchana Rukthong & Tineke Brunfaut

We would like to thank Pearson for their help with materials

anybody listening? the role of listening in integrated listening-to … · 2015-12-02 · what...

Documents