by wen-shuenn wu ( 吳文舜 ) chung hua university (chu) [email protected] chu.tw/~wswu
DESCRIPTION
Developing a prototype of Online Adaptive Vocabulary Test: A preliminary report. by Wen-shuenn Wu ( 吳文舜 ) Chung Hua University (CHU) [email protected] http://www.chu.edu.tw/~wswu. The status quo of General English at CHU. - PowerPoint PPT PresentationTRANSCRIPT
112/04/19 2
The status quo of General The status quo of General English at CHUEnglish at CHU
• Online English linear placement test was first used in September, 2001.
• Advantages:– fast– technical support
• Disadvantages:– expensive– no control on questions
NT$150~180/one personone time1,500 studentsx NT$150 x 3 =NT$ 810,000
112/04/19 3
Original ideas (1)Original ideas (1)
• CAT (computer adaptive test)• “Thus, in a CAT, each examinee’s first
item is usually of a projective medium-difficulty level for the test population. If the examinee responds correctly, the next item received is more difficult. If, however, the examinee misses the item, an easier question is presented in turn.” (Patricia A. Dunkel, 1999)
112/04/19 4
Original ideas (2)Original ideas (2)
Level 3
Level 4
Level 2
Level 5
Level 1
15 minutes
15 minutes
15 minutes
15 minutes
15 minutes+3 itemsin a row
+3 itemsin a row
-3 itemsin a row
-3 itemsin a row
112/04/19 5
Online Adaptive Vocabulary Online Adaptive Vocabulary Test (OAVT)Test (OAVT)
• the first stage of a web-based adaptive English placement test system
• vocabulary grammar / sentence structure listening reading comprehension
112/04/19 6
Classification of word listClassification of word list
• Collins COBUILD English Dictionary for Advanced Learners (2001): frequency bands (◆◆◆◆◆ ◆)
• Macmillan English Dictionary for Advanced Learners of American English (2002): the most frequent words in English ( ★★★ ★)
• Longman Dictionary of Contemporary English (red entry)
112/04/19 7
Levels of Word List (1)Levels of Word List (1)• Level 0: basic and the most frequently used
words (e.g. and, I, my, the, etc.)
• Level 1: Collin COBUILD English Dictionary
frequency band 5 ◆◆◆◆◆ and frequency band 4 ◆◆◆◆
112/04/19 8
Levels of Word List (2)Levels of Word List (2)• Level 2: Longman Dictionary W1, S1, W2, S2, W3,
S3 (-- Collin COBUILD English Dictionary◆◆◆◆◆ and ◆◆◆◆)
• Level 3: (A) Collin COBUILD English Dictionary
frequency band 3 ◆◆◆ (-- Longman Dictionary S1, S2, S3, W1, W2, W3)
(B) Macmillan English Dictionary ★★ (and ★★★ if not in Level 1 and 2)
112/04/19 9
Levels of Word List (3)Levels of Word List (3)
• Level 4: (A) Macmillan English Dictionary ★(B) Collin COBUILD English Dictionary
frequency band 2 ◆◆
• Level 5:
• Collin COBUILD English Dictionary frequency band 1 ◆
112/04/19 10
Numbers of Word ListNumbers of Word List
level 0 1 2 3 4 5
stemstem 141
words
1,567
words
1,548
words
1,179
words
2,670
words
6,198
words
derivativesderivatives
(word family)(word family)
3,864
words
3,435
words
2,583
words
6,038
words
13,066
words
112/04/19 11
Definition of item codeDefinition of item code
V1262022N000001
Code 1: Type
V: vocabulary
G: grammar
L: listening
R: reading
Code 2~4: Level
-3 <= b <= +3
b: difficulty parameter
code 2: 1(+); 0(-)
Code 5: Specification
1: recognition
2: collocation
3: context clue
4: synonymous clue
5: ESP
6: spelling
7: unclassified
Code 6~7: Category
1: general
2: business
3: science
4: medicine
5: law
Code 8: part of speech
1: verb
2: noun
3: adjective
4: adverb
5: phrase
6: idiom
Code 9: N
Code 10~15: serial number
000001~999999
112/04/19 12
Test-stopping rulesTest-stopping rules• An adaptive test terminates when one o
r more of the following stopping rules is met: (Wainer, 1990)
(A) when a target measurement precision level has been attained
(B) when a pre-selected number of items has been given
(C) when a predetermined amount of time has elapsed
112/04/19 13
Interdisciplinary study of IRT-based Online Interdisciplinary study of IRT-based Online Adaptive English TestAdaptive English Test
Question Items
IRT logisticalmodelsProgramming
EnglishDept
ComputerScience
Dept
MathDept
User Interface
Test Security
ReliabilityValidity
ServerMaintenance
Statistical Analysis
112/04/19 14
112/04/19 15
112/04/19 16
112/04/19 17
112/04/19 18
112/04/19 19
112/04/19 20
112/04/19 21
112/04/19 22
112/04/19 23
112/04/19 24
112/04/19 25
112/04/19 26
112/04/19 27
112/04/19 28
112/04/19 29
112/04/19 30
112/04/19 31
112/04/19 32