taking ielts academic reading test - 英国文化教育协 subject 1 subject 1 similar verbal...

37
Xiangdong Gu Qin Lv Lu Zhao Chongqing University A Verbal Protocol Analysis of Test-takers’ Processes of Taking IELTS Academic Reading Test 雅思学术类阅读测试受试答题过程 有声思维研究 6 th Nov. 2013

Upload: lybao

Post on 20-Feb-2018

224 views

Category:

Documents


1 download

TRANSCRIPT

Page 1: Taking IELTS Academic Reading Test - 英国文化教育协 Subject 1 Subject 1 similar verbal protocols of the same task Reliability Coder 1 Coder 2 Coder 1 coding scheme accurately

Xiangdong Gu Qin Lv Lu Zhao

Chongqing University

A Verbal Protocol Analysis of Test-takers’ Processes of

Taking IELTS Academic Reading Test

雅思学术类阅读测试受试答题过程

有声思维研究

6th Nov. 2013

Page 2: Taking IELTS Academic Reading Test - 英国文化教育协 Subject 1 Subject 1 similar verbal protocols of the same task Reliability Coder 1 Coder 2 Coder 1 coding scheme accurately

Our Teams’ Studies Using Verbal Protocol Analysis

Gao. X & Gu. X (2008) An introspective study on test-taking process for banked cloze ,

Teaching English in China (4), 3-16.

Zheng. Y & Gu. X (2009) A study on the construct validity of long dialogue multiple-choice

listening comprehension tests: retrospection approach, Teaching English in China (6), 15-26.

Gu. X & He.Y (2011) An introspective study on testees’ process of taking English to Chinese

translation test in TEM 8, Foreign Languages and Translation (3), 67-73.

Gu. X & Shi. C (2012) A retrospective study on test-takers’ cognitive and metacognitive

processes in taking a compound dictation test, Chinese Journal of Applied Linguistics 35 (4),

400-420.

Gu. X & Wang. J (2012) An introspective study on test-takers’ processes of taking sentence

translation test in CET-4

Gu. X & Shen. Y (2012) An introspective study on test-takers’ processes of taking vocabulary

test in reading section of HSK-6

Gu, X, Lv. Q & Zhao. L (2013) A verbal protocol analysis of test-takers’ processes of taking

IELTS academic reading test

Page 3: Taking IELTS Academic Reading Test - 英国文化教育协 Subject 1 Subject 1 similar verbal protocols of the same task Reliability Coder 1 Coder 2 Coder 1 coding scheme accurately

• Vol 4 The Development of IELTS: A study of the effect of

background knowledge on reading comprehension

--Caroline Clapham (1996)

• Vol 19 IELTS Collected Papers: Research in speaking

and writing assessment

--Edited by Taylor & Falvey (2007)

• Vol 23 Assessing Academic English: Testing English

proficiency, 1950–1989 – the IELTS solution --Alan Davies (2008)

• Vol 24 Impact Theory and Practice: Studies of the IELTS

test and Progetto Lingue 2000 --Roger Hawkey (2006)

• Vol 25 IELTS Washback in Context: Preparation for

academic writing in higher education

--Anthony Green (2007)

• Vol 34 IELTS Collected Papers 2: Research in reading

and listening assessment --Edited by Lynda Taylor and Cyril J Weir (2012)

Examples of IELTS Studies

Page 4: Taking IELTS Academic Reading Test - 英国文化教育协 Subject 1 Subject 1 similar verbal protocols of the same task Reliability Coder 1 Coder 2 Coder 1 coding scheme accurately

Weir. C, Hawkey. R,

Green. A, Ünaldi. A &

Devi. S

The relationship between the academic reading construct

as measured by IELTS and the reading experience of

students in their first year of study at a British university

Moore. T, Morton. J &

Price. S

Construct validity in the IELTS Academic Reading test:

a comparison of reading requirement in IELTS test items

and in university study

Weir. C, Hawkey. R,

Green. A & Devi. S

The cognitive processes underlying the academic

reading construct as measured by IELTS

Green. A & Hawkey. R

An empirical investigation of the process of writing

Academic Reading test items for the International

English language

Academic Reading

Page 5: Taking IELTS Academic Reading Test - 英国文化教育协 Subject 1 Subject 1 similar verbal protocols of the same task Reliability Coder 1 Coder 2 Coder 1 coding scheme accurately

Definition of Verbal Protocols

Rich data sources containing individuals’ spoken thoughts that are associated

with working on a task. Subjects usually either think aloud as thoughts occur

to them while working on a particular task (concurrent introspection) or

verbalize their thoughts upon completion of the task (immediate

retrospection). The verbalizations are recorded and then coded according to

theory-driven and/or empirically driven categories. (Gale Encyclopedia of

Education: Research methods)

Source: http://www.answers.com/topic/research-methods-verbal-protocols

Page 6: Taking IELTS Academic Reading Test - 英国文化教育协 Subject 1 Subject 1 similar verbal protocols of the same task Reliability Coder 1 Coder 2 Coder 1 coding scheme accurately

Information captured

within verbal reports

Information

actually needed corresponds closely with

Validity

Subject 1 Subject 1

similar verbal protocols of the same task

Reliability

Coder 1

Coder 2

Coder 1

Coder 2 coding scheme accurately captures the behavior

Validity and Reliability of Verbal Protocols

Page 7: Taking IELTS Academic Reading Test - 英国文化教育协 Subject 1 Subject 1 similar verbal protocols of the same task Reliability Coder 1 Coder 2 Coder 1 coding scheme accurately

What types of strategies do test-takers employ when taking

IELTS academic reading test?

Are there any similarities and differences in the employment of

strategies between high-level group and low-level group of test-

takers?

Are there any similarities and differences about test-takers’

employment of strategies among different task types?

Research Questions

Page 8: Taking IELTS Academic Reading Test - 英国文化教育协 Subject 1 Subject 1 similar verbal protocols of the same task Reliability Coder 1 Coder 2 Coder 1 coding scheme accurately

All task types of IELTS Task types in selected materials

• Identifying information

(True/False/Not given)

• Matching information

• Sentence completion

• Diagram label completion

• Short-answer questions

• Identifying writer’s

views/claims

(Yes/No/Not given)

• Matching headings

• Matching features

• Matching sentence endings

• Multiple choice

• Summary/note/table/flow

chart completion

1 Summary/Note/Table/Flow-chart completion

2 Identifying information (True/False/Not given)

3 Matching headings

4 Summary/Note/Table/Flow-chart completion

5 Matching features

6 Matching headings

7 Multiple choice

8 Sentence completion

Task Types

Page 9: Taking IELTS Academic Reading Test - 英国文化教育协 Subject 1 Subject 1 similar verbal protocols of the same task Reliability Coder 1 Coder 2 Coder 1 coding scheme accurately

Passage 1 Passage 2 Passage 3

Topic Sheet glass

manufacture the float

process

The Little Ice Age The meaning and

power of smell

Section 1

Tasks

Summary/Note/Table/

Chart-flow completion

Matching headings Matching headings

Section 2

Tasks

Identifying

information

(True/False/Not given)

Summary/Note/Table/

Chart-flow completion

Multiple choice

Section 3

Tasks

Matching features Sentence completion

Item No. 13 13 14

Instrument: Test 2 in Cambridge IELTS Test 8 (2011)

Page 10: Taking IELTS Academic Reading Test - 英国文化教育协 Subject 1 Subject 1 similar verbal protocols of the same task Reliability Coder 1 Coder 2 Coder 1 coding scheme accurately

Selecting and Training subjects

motivated, skilled at verbalization, different proficiency levels, pilot, rehearsal

Transcribing verbal reports word by word, time markers, double check

Collecting verbal reports and supplementary data

simultaneous introspection, immediate retrospection; subject performance first, thinking aloud second; researcher out of subjects’ view; retrospective interview; first language; clear tape or video recording

Developing an encoding scheme Segmenting and encoding protocols

Calculating encoder reliability units division, intra- or/and inter- reliability

Analyzing data descriptive statistics, Chi-square Tests

Task identification and analysis Selecting an appropriate procedure

decision on whether to use verbal protocol analysis

Operational Procedures

Inter-rater Coding Agreement of this study is 0.78.

Page 11: Taking IELTS Academic Reading Test - 英国文化教育协 Subject 1 Subject 1 similar verbal protocols of the same task Reliability Coder 1 Coder 2 Coder 1 coding scheme accurately

Participants Gender Age Major Education

background

IELTS

score

SL1 Male 24 Mechanics PhD candidate 6.0

SL2 Male 26 Power engineering MS candidate 5.5

SL3 Female 22 Measurement and

control MS candidate 5.0

SH4 Male 25 Electrical engineer PhD candidate 8.0

SH5 Female 23 Chinese linguistics

and literature MA candidate 7.5

SH6 Female 24 Law MA candidate 7.0

Background Information of the Participants

SL: Subject of Low-level; SH: Subject of High-level

Page 12: Taking IELTS Academic Reading Test - 英国文化教育协 Subject 1 Subject 1 similar verbal protocols of the same task Reliability Coder 1 Coder 2 Coder 1 coding scheme accurately

Bottom-up strategies Definition

B1 Breaking lexical items into parts Breaking words into smaller units to promote

comprehension

B2 Scanning for explicit information

requested in the item

Scanning the text for specific details or explicit

stated information requested in the item

B3

Identifying a synonym or a paraphrase

of the literal meaning of a word, phrase,

or sentence

Identifying or formulating a synonym or a

paraphrase of the literal meaning of a word, phrase,

or sentence in the text to help answer the question

B4 Relating visual or verbal information to

accompanying visuals

Matching visual information in the item to answer

the question

B5 Matching key vocabulary in the item to

key vocabulary in the text

Matching visual information in the item to the words

in the text to answer the question

B6 Using knowledge of grammar or

punctuation

Using awareness of grammar, syntax, parts of

speech, or punctuation to help answer the question

B7 Using local context cues to interpret a

word or phrase

Using the words in a sentence that precede or follow

a specific word or phrase to understand a particular

word or phrase

Abbott’s Reading Strategy Classification (2010)

Page 13: Taking IELTS Academic Reading Test - 英国文化教育协 Subject 1 Subject 1 similar verbal protocols of the same task Reliability Coder 1 Coder 2 Coder 1 coding scheme accurately

Top-down strategies Definition

T1 Skimming for gist or identifying the main

idea, theme, or concept

Drawing on the major points of the passage to answer

the question; summarizing main concept

T2

Connecting or relating information

presented in different sentences or parts

of the text

Relating new information to previously stated

information to help answer the question; synthesizing

scattered information

T3 Drawing an inference based on

information presented in the text

Making an inference, drawing a conclusion, or

forming a hypothesis based on information not

explicitly stated in the text to answer the question

T4 Speculating beyond the text Using background information to speculate beyond

the text

T5 Recognizing discourse format

Using discourse format or text organization to answer

the question (eg., discriminates between fact and

opinion or cause and effect; or notes how the

information is presented)

Abbott’s Reading Strategy Classification (2010)

Page 14: Taking IELTS Academic Reading Test - 英国文化教育协 Subject 1 Subject 1 similar verbal protocols of the same task Reliability Coder 1 Coder 2 Coder 1 coding scheme accurately

Coding Schema of Bottom-up Strategy Use

Bottom-up strategies Examples

B1 Breaking lexical items into parts /

B2 Scanning for explicit information requested

in the item

这有一段话,让选词,weather during …in the distant are 什么,应该是 tree rings and ice cores, tree rings, ice cores,这部分在paragraph 3里面找的到,Records … tree rings and ice

cores(第三段的原文)。(Participant 1)

B3

Identifying a synonym or a paraphrase of the

literal meaning of a word, phrase, or

sentence

A段的标题,我觉得它大多数都在讲emo t ion…有一个feelings很像,应该算是这个emotion的近义词吧,所以选…。(Participant 5)

B4 Relating visual or verbal information to

accompanying visuals

先看这个图…这有两个轮子,刚好第二段讲到这个问题,估计就应该填roller,因为文中提到两个two hot rollers。(Participant 6)

B5 Matching key vocabulary in the item to key

vocabulary in the text

重新再看一下这个文章,找这几个关键词,第一个是discover a ll the lands…那就逐行看,找关键词吧。(Participant 1)

B6 Using knowledge of grammar or punctuation however, 它这个呢就是一个转折,前面是优点,后面肯定就是讲的一个缺点…所以就可以直接做题了。(Participant 1)

B7 Using local context cues to interpret a word

or phrase /

Page 15: Taking IELTS Academic Reading Test - 英国文化教育协 Subject 1 Subject 1 similar verbal protocols of the same task Reliability Coder 1 Coder 2 Coder 1 coding scheme accurately

Coding Schema of Top-down Strategy Use

Top-down strategies Examples

T1 Skimming for gist or identifying

the main idea, theme, or concept

(读文章)Global temperatures …这一段主要讲的是人类对温度的这样一个影响,恩,所以就是选这个四,human impact on the climate. (Participant 3)

T2

Connecting or relating information

presented in different sentences or

parts of the text

/

T3 Drawing an inference based on

information presented in the text

12题,the process… improved,有没有说提到improved 呢?

就 是 说 , ( 在 文 中 找 相 应 信 息 ) they finally

succeeded…15 years,就是说之前的时候它要停,而且要

校正,需要花费四个月,所以这个时候呢我们推断一下,应该是improved. (Participant 4)

T4 Speculating beyond the text 9到13是一个判断问题true or false的问题,按照雅思的一个规则,那它就不会在第一部分出现。(Participant 3)

T5 Recognizing discourse format 前面讲到了优点,再看,后边应该就是缺点了吧,(继续看文章),glass was marked, 恩,第五个空应该就是填marked. (Participant 6)

Page 16: Taking IELTS Academic Reading Test - 英国文化教育协 Subject 1 Subject 1 similar verbal protocols of the same task Reliability Coder 1 Coder 2 Coder 1 coding scheme accurately

Frequency of Participants’ Bottom-up Strategy Use

Subject B1 B2 B3 B4 B5 B6 B7

SL1 0 18 4 1 10 0 0

SL2 0 20 1 1 7 0 0

SL3 0 24 3 0 6 1 0

SH4 0 18 5 1 10 1 0

SH5 0 16 2 1 10 4 0

SH6 0 17 5 1 7 1 0

Page 17: Taking IELTS Academic Reading Test - 英国文化教育协 Subject 1 Subject 1 similar verbal protocols of the same task Reliability Coder 1 Coder 2 Coder 1 coding scheme accurately

Frequency of Participants’ Bottom-up Strategy Use

B2: Scanning for explicit information

requested in the item

Page 18: Taking IELTS Academic Reading Test - 英国文化教育协 Subject 1 Subject 1 similar verbal protocols of the same task Reliability Coder 1 Coder 2 Coder 1 coding scheme accurately

Frequency of Participants’ Top-down Strategy Use

Subject T1 T2 T3 T4 T5

SL1 10 0 1 0 0

SL2 5 0 5 0 0

SL3 9 0 4 0 2

SH4 15 0 11 2 3

SH5 10 0 12 0 5

SH6 20 0 5 1 2

Page 19: Taking IELTS Academic Reading Test - 英国文化教育协 Subject 1 Subject 1 similar verbal protocols of the same task Reliability Coder 1 Coder 2 Coder 1 coding scheme accurately

Frequency of Participants’ Top-down Strategy Use

T1: Skimming for gist or identifying the

main idea, theme, or concept

T3: Drawing an inference based on

information presented in the text

T5: Recognizing discourse format

Page 20: Taking IELTS Academic Reading Test - 英国文化教育协 Subject 1 Subject 1 similar verbal protocols of the same task Reliability Coder 1 Coder 2 Coder 1 coding scheme accurately

Frequency of All Participants’ Strategy Use

Chi-square tests: Sig. .001

Page 21: Taking IELTS Academic Reading Test - 英国文化教育协 Subject 1 Subject 1 similar verbal protocols of the same task Reliability Coder 1 Coder 2 Coder 1 coding scheme accurately

Frequency of Participants’ Bottom-up Strategy Use

in Each Task Type

Task Strategy B1 B2 B3 B4 B5 B6 B7

Task 1 Summary/Note/Table/Flow-chart

completion 0 18 4 5 14 3 0

Task 2 Identifying information

(True/False/Not Given) 0 18 0 0 1 1 0

Task 3 Matching headings 0 9 4 0 5 2 0

Task 4 Summary/Note/Table/Flow-chart

completion 0 16 1 0 12 0 0

Task 5 Matching features 0 11 2 0 12 0 0

Task 6 Matching headings 0 10 6 0 4 0 0

Task 7 Multiple choice 0 14 1 0 0 0 0

Task 8 Sentence completion 0 17 2 0 2 1 0

Page 22: Taking IELTS Academic Reading Test - 英国文化教育协 Subject 1 Subject 1 similar verbal protocols of the same task Reliability Coder 1 Coder 2 Coder 1 coding scheme accurately

Frequency of Participants’ Top-down Strategy Use

in Each Task Type

Task Strategy T1 T2 T3 T4 T5

Task 1 Summary/Note/Table/Flow-chart

completion 5 0 8 0 8

Task 2 Identifying information

(True/False/Not Given) 9 0 10 1 0

Task 3 Matching headings 16 0 9 0 0

Task 4 Summary/Note/Table/Flow-chart

completion 1 0 1 0 1

Task 5 Matching features 2 0 5 2 2

Task 6 Matching headings 24 0 2 0 1

Task 7 Multiple choice 11 0 3 0 0

Task 8 Sentence completion 1 0 0 0 0

Page 23: Taking IELTS Academic Reading Test - 英国文化教育协 Subject 1 Subject 1 similar verbal protocols of the same task Reliability Coder 1 Coder 2 Coder 1 coding scheme accurately

Bottom-up strategies more frequently used in Tasks 1, 4, 5 & 8

0

5

10

15

20

25

30

35

40

45

50

Task 1 Task 2 Task 3 Task 4 Task 5 Task 6 Task 7 Task 8

Bottom-up Strategies Top-down Strategies

Summary/Note/Table/Flow-chart completion

Matching

features Sentence

completion

Page 24: Taking IELTS Academic Reading Test - 英国文化教育协 Subject 1 Subject 1 similar verbal protocols of the same task Reliability Coder 1 Coder 2 Coder 1 coding scheme accurately

Top-down strategies more frequently used in Tasks 3 & 6

0

5

10

15

20

25

30

35

40

45

50

Task 1 Task 2 Task 3 Task 4 Task 5 Task 6 Task 7 Task 8

Bottom-up Strategies Top-down Strategies

Matching headings

Page 25: Taking IELTS Academic Reading Test - 英国文化教育协 Subject 1 Subject 1 similar verbal protocols of the same task Reliability Coder 1 Coder 2 Coder 1 coding scheme accurately

0

5

10

15

20

25

30

35

40

45

50

Task 1 Task 2 Task 3 Task 4 Task 5 Task 6 Task 7 Task 8

Bottom-up Strategies Top-down Strategies

Identifying information

(True/False/Not Given)

Multiple

choice

Bottom-up and top-down strategies almost equally used in Tasks 2 & 7

Page 26: Taking IELTS Academic Reading Test - 英国文化教育协 Subject 1 Subject 1 similar verbal protocols of the same task Reliability Coder 1 Coder 2 Coder 1 coding scheme accurately

Conclusions

Q1: What types of strategies do test-takers employ when taking

IELTS academic reading test?

Test-takers’ cognitive processes of taking IELTS academic

reading test are very complex. They employ both bottom-up

strategies and top-down strategies when taking the test.

However, some strategies (B2, B5, T1 & T3) are much more

frequently used while some strategies (B1, B7 & T2) are not

used at all.

Page 27: Taking IELTS Academic Reading Test - 英国文化教育协 Subject 1 Subject 1 similar verbal protocols of the same task Reliability Coder 1 Coder 2 Coder 1 coding scheme accurately

Conclusions

Both high-level and low-level groups of test-takers tend to use

bottom-up strategies more frequently and there is no

significant difference in the frequency of their bottom-up

strategy use. But high-level group use more top-down

strategies than low-level group with significant difference.

Q2: Are there any similarities and differences in the

employment of strategies between high-level group and

low-level group of test-takers?

Page 28: Taking IELTS Academic Reading Test - 英国文化教育协 Subject 1 Subject 1 similar verbal protocols of the same task Reliability Coder 1 Coder 2 Coder 1 coding scheme accurately

Conclusions

In different tasks, test-takers’ strategy use is different. Bottom-

up strategies are more frequently used in three task types:

Summary/Note/Table/Flow-chart completion, Matching features,

Sentence completion; Top-down strategies are more frequently

used in Matching features; Bottom-up and Top-down strategies

are almost equally used in tasks of Identifying information

(True/False/Not Given) and Multiple choice.

Q3: Are there any similarities and differences about test-takers’

employment of strategies among different task types?

Page 29: Taking IELTS Academic Reading Test - 英国文化教育协 Subject 1 Subject 1 similar verbal protocols of the same task Reliability Coder 1 Coder 2 Coder 1 coding scheme accurately

Implications

A wider range of strategies need to be covered.

The proportion of the tasks which measure test-takers’ top-down

strategies needs to be increased as they can differentiate high-level

group from low–level group.

More training and practice is needed for test-takers to improve

their top-down strategy use.

Page 30: Taking IELTS Academic Reading Test - 英国文化教育协 Subject 1 Subject 1 similar verbal protocols of the same task Reliability Coder 1 Coder 2 Coder 1 coding scheme accurately

Significance

Theoretically, provided evidence to confirm the rationality of the

construct framework.

Practically, put forward suggestions for language testers on test

development and test evaluation, and provided implications for

language teachers and learners on how to improve strategy use.

Methodologically, proved verbal protocol analysis as an effective

approach to examine strategy use employed by test-takers.

Page 31: Taking IELTS Academic Reading Test - 英国文化教育协 Subject 1 Subject 1 similar verbal protocols of the same task Reliability Coder 1 Coder 2 Coder 1 coding scheme accurately

Some subjects may be unrepresentative due to convenience

sampling.

The validity and reliability of the coding schemes need to be

improved.

The theoretical framework used for the analysis is inadequate,

e.g. test-wiseness strategies are not included in it.

The generalizability of the research findings remains to be

further testified by triangulation methodology.

Limitations

Page 32: Taking IELTS Academic Reading Test - 英国文化教育协 Subject 1 Subject 1 similar verbal protocols of the same task Reliability Coder 1 Coder 2 Coder 1 coding scheme accurately

References and Further Readings

1. Abbott, M. L. (2010). An introspective study of Arabic and Mandarin speaker reading comprehension strategies. TESL Canada

Journal, 28.

2. Anderson, N. C. (1991). Individual differences in strategy use in second language reading and testing. Modern Language Journal,

75(4): 460-472.

3. Alderson, J. C. (2000). Assessing Reading. Cambridge University Press, Cambridge.

4. Alderson, J. C., Clapham, C. & Wall, D. (2000). Language Test Construction and Evaluation. Beijing: Foreign Language Teaching

and Research Press.

5. Barkaoui, K. (2011). Think-aloud protocols in research on essay rating: An empirical study of their veridicality and reactivity.

Language Testing, 28(1), 51-75.

6. Carver, R. (1992). Reading Rate: theory, research and practical implications. Journal of Reading, 36(2), 84-95.

7. Cohen, A. (1984). On taking language tests: What the students report, Language Testing, 1, 70-81.

8. Cumming, A. (1990). Metalinguistic and ideational thinking in second language composing. Written Communication 7, 482-511.

9. Enright, M., Grabe, W., Koda, K., Mosenthal, P., Mulcany-Ernt, P. & Schedl, M. (2000). TOEFL 2000 reading framework: A

working paper. TOEFL Monograph Series 17 ETS, Princeton.

10. Ericsson, K. A. & Simon, H. (1993). Protocol Analysis: Verbal Reports as Data. Cambridge, Massachusetts: The MIT Press.

11. Faerch, C. & Kasper, G. (1987). Introspection in second language research. Clevedon, Philadelphia: Multilingual Matters.

12. Green, A. (1998). Verbal protocol analysis in language testing research: A handbook. Cambridge: Cambridge University Press.

13. Goodman, K. S. (1973). Psycholinguistic universals of the reading process, In E Smith (ed.), Psycholinguistics and Reading. New

York: Holt Rinehart and Winston.

14. Grabe, W. & Stoller, F. L. (2002). Teaching and Researching Reading. Harlow, England: Pearson Education.

15. Glaser, B. G. (1978). Theoretical sensitivity: Advances in the methodology of grounded theory. Mill Valley, CA: Sociology Press.

16. Glaser, B. G. (1992). Basics of grounded theory analysis: Emergence vs. forcing. Mill Valley, CA: Sociology Press.

17. Grellet, F. (1981). Developing Reading Skills. Cambridge: CUP.

18. Hamp-Lyons, L. (1990). Second Language writing: assessment issues. In Kroll, B. (Eds.), Second Language Writing: Research

Insights for the Classroom. Cambridge University Press, Cambridge, 69-87.

Page 33: Taking IELTS Academic Reading Test - 英国文化教育协 Subject 1 Subject 1 similar verbal protocols of the same task Reliability Coder 1 Coder 2 Coder 1 coding scheme accurately

References and Further Readings

19. Hammadou, J. (1991). Interrelationships among prior knowledge, inference, and language proficiency in foreign language reading.

Modern Language Journal, 75, 27-38.

20. Hawkey, R. (2006). Impact theory and practice: Studies of the IELTS test and Progetto Lingue 2000. Cambridge University Press,

Cambridge.

21. Hill, C., & Parry, K. (1992). The test at the gate: Models of literacy in reading assessment. TESOL Quarterly, 26, 433–461.

22. Hudson, T. & Park, S. (2002). Validity issues for selected versus constructed response Internet-based language test. Paper

presented at AAAL, Arlington, Virginia.

23. IELTS (1996). The IELTS handbook. Cambridge: UCLES. The British Council, IDP Education Australia.

24. IELTS (2007). IELTS official practice materials. Cambridge University Press, Cambridge.

25. IELTS Handbook. (1999). Cambridge: University of Cambridge Local Examinations Syndicate.

26. IELTS Handbook. (2007). Cambridge: University of Cambridge Local Examinations Syndicate.

27. Irwin, J. W. (1986). Teaching Reading Comprehension Process. New Jersey: Prentice Hall Inc.

28. Khalifa, H. & Weir, C. (2009). Examining reading: Research and practice in assessing second language reading. Studies in

Language Testing 29. Cambridge: UCLES & Cambridge University Press.

29. Lumley, T. (2002). Assessment criteria in a large-scale writing test: What do they really mean to the raters? Language Testing, 19,

246-276.

30. McDonough, S. (1995), Strategy and Skill in Learning a Foreign Language, Arnold.

31. McGinnis, D. J. & Smith, D. E. (1982). Analyzing and Treating Reading Problems. New York: Macmillan Publishing Co., Inc.

32. McNamara, T. F. (1999). Computer-adaptive testing: A view from outside. In Chaloub-Deville M. (Eds.), Issues in computer-

adaptive testing of reading proficiency. Cambridge University Press, Cambridge, 136-149.

33. Moore, T. & Morton, J. (1999). Authenticity in the IELTS academic module writing test: a comparative study of Task 2 items and

university assignments in Tulloh, R. (Eds.), IELTS research reports 2. IELTS Australia, Canberra, 64-106.

34. Munby, J. L. (1978). Communicative Syllabus Design. Cambridge: CUP.

35. Nevo, N. (1989). Test-taking strategies on a multiple-choice test of reading comprehension. Language Testing, 6, 199-215.

36. Nunan, D. (1995). Teaching methodology: a textbook for teachers. Phoenix ELT.

Page 34: Taking IELTS Academic Reading Test - 英国文化教育协 Subject 1 Subject 1 similar verbal protocols of the same task Reliability Coder 1 Coder 2 Coder 1 coding scheme accurately

References and Further Readings 37. Oxford, R. (1990). Language learning strategies. New York: Newbury House. 38. Orr, M. (2002). The FCE speaking test: Using rater reports to help interpret test scores. System, 30, 143-154. 39. Parry, K. (1996). Culture, literacy and L2 reading. TESOL Quarterly, 30, 665-692. 40. Phakiti, A. (2003). A closer look at the relationship of cognitive and metacognitive strategy use to EFL reading achievement test

performance. Language Testing,1, 26-56. 41. Purpura, J. (1997). An analysis of the relationships between test takers’ cognitive and metacognitive strategy use and second

language test performance. Language Learning, 47, 289–325. 42. Rennie, D. (1984). Clients’ tape-assisted recall of psychotherapy: A qualitative analysis. Paper presented at the annual meeting of

the Canadian Psychological Association, Ottawa. 43. Richards, J. C., P late, J. & Platte, H. (2000). Longman Dictionary of Language Teaching and Applied Linguistics. Beijing: Foreign

Language Teaching and Research Press. 44. Rumelhart, D. E. (1980). Schemata:The building blocks of cognition. In Spiro, R. J., Bruce, B. C. & Brewer, W. E. (Eds.),

Theoretical Issues in Reading Comprehension. Hillsdale, N. J. Lawrence Erlbaum. 45. Rupp, A. A., Ferne, T. & Choi, H. (2006). How assessing reading comprehension with multiple-choice questions shapes the

construct: a cognitive processing perspective. Language Testing, 4, 441-474. 46. Schueller, J. (2004). Gender and foreign language reading comprehension: The effects of strategy training. Southern Journal of

Linguistics, 27(1), 45–65. 47. Shi, C. Y. (2010). A Retrospective Study on Test-takers’ Processes of Taking Compound Dictation Test. Master of Arts dissertation,

Chongqing University. 48. Taylor, G. (2009). A student’s writing guide, Cambridge University Press, Cambridge.

49. Urquhart, A. & Weir, C. J. (1998). Reading in a second language: process, product and practice. Longman, London. 50. Vaughan, C. (1991). Holistic assessment: what goes on in the rater's mind? In Hamp-Lyons, L. (Eds.), Assessing Second Language

Writing in Academic Contexts, 111-25. Norwood, NJ: Ablex. 51. Wang, H. Z. (2009). Introspection in L2 speaking performance assessment research. 2009 (1):12-22/127 CELEA Journal

(Bimonthly). 52. Weir, C. J. (1983). Identifying the language needs of overseas students in tertiary education in the United Kingdom, PhD thesis,

Institute of Education, University of London.

Page 35: Taking IELTS Academic Reading Test - 英国文化教育协 Subject 1 Subject 1 similar verbal protocols of the same task Reliability Coder 1 Coder 2 Coder 1 coding scheme accurately

References and Further Readings 53. Weir, C. J., Yang, H. & Jin, Y. (2000). An empirical investigation of the componentiality of L2 reading in English for Academic

Purposes, Studies in Language Testing, 12, UCLES Cambridge University Press, Cambridge.

54. Weir, C. J., Hawkey, R., Green, A., Devi, S. & Unaldi, A. (2009). The relationship between the academic reading construct as

measured by IELTS and the reading experiences of students in the first year of their courses at a British university, IELTS

Research Project Report.

55. Weir, C.J., Hawkey, R., Green, A. & Devi, S. (2009). The cognitive processes underlying the academic reading construct as

measured by IELTS, in Thompson P. (Eds.), Research Reports Volume 9, British Council/IDP Australia, London.

56. Widdowson, H. G. (1978). Teaching Language as Communication. Oxford: OUP.

57. Wu, Y. A. (1998). What do tests of listening comprehension test?-A retrospection study of EFL test-takers performing a multiple-

choice task. Language Testing 15 (1), 21-44.

58. Xiang dong, Gu & Xiao ying, Gao. 2008. An introspective study on test-taking process for banked Cloze. CELEA Journal, 31(4),

3-16.

59. Yamashita, J. (2003). Processes of taking a gap-filling test: Comparison of skilled and less skilled EFL readers. Language Testing,

20, 267-293.

60. Yang, H. & Weir, C. (1998). Validation study of the national college English test. Shanghai: Shanghai Foreign Language Education

Press.

61. Zhang, Y. (2006). Validation Study of Translation Items in National Entrance Test of English for MA/MS Candidates. Master of

Arts dissertation, Wuhan University of Technology.

62. 陈晓扣, 李绍山. 2006. TEM-4 完型填空测试结构 效度研究——答题过程分析法 [J]. 现代外语(1): 71-77.

63. 金艳,吴江. 1998.以“内省法”检验CET阅读理解测试的效度[J].外语界(2).

64. 郭纯洁(2007).《有声思维法》.北京:外语教学与研究出版社.

65. 刘建达,杨满珍(2001).成段改错试题考了什么?.《现代外语》.第2期:170-180.

66. 苗菊(2005).有声思维——翻译内在过程探索[J],《外语与外语教学》.第6期:43-46.

67. 文军(2005).翻译课程模式研究——以发展翻译能力为中心的方法.北京:中国文史出版社.

68. 文军,孙三军(2006).论使用出声思维研究翻译过程.《外语学刊》.第3期:93-97.

69. 郑宇静,辜向东(2009).长对话多项选择听力测试题结构效度的追述法研究(英文).《中国英语教学》. 第6期:15-26.

Page 36: Taking IELTS Academic Reading Test - 英国文化教育协 Subject 1 Subject 1 similar verbal protocols of the same task Reliability Coder 1 Coder 2 Coder 1 coding scheme accurately

Organizers

My MA team

My family

Acknowledgements

Page 37: Taking IELTS Academic Reading Test - 英国文化教育协 Subject 1 Subject 1 similar verbal protocols of the same task Reliability Coder 1 Coder 2 Coder 1 coding scheme accurately

Thank you very much

for your attention!

Xiangdong Gu, [email protected], 13983732737

Qin Lv, [email protected], 18696567794

Lu Zhao, [email protected], 15826189568