by Wen-shuenn Wu ( 吳文舜 ) Chung Hua University (CHU) wswu@chu.tw chu.tw/~wswu

by Wen-shuenn Wu (吳文舜 )

Chung Hua University (CHU)wswu@chu.edu.tw

http://www.chu.edu.tw/~wswu

112/04/19 2

The status quo of General The status quo of General English at CHUEnglish at CHU

• Online English linear placement test was first used in September, 2001.

• Advantages:– fast– technical support

• Disadvantages:– expensive– no control on questions

NT$150~180/one personone time1,500 studentsx NT$150 x 3 =NT$ 810,000

112/04/19 3

Original ideas (1)Original ideas (1)

• CAT (computer adaptive test)• “Thus, in a CAT, each examinee’s first

item is usually of a projective medium-difficulty level for the test population. If the examinee responds correctly, the next item received is more difficult. If, however, the examinee misses the item, an easier question is presented in turn.” (Patricia A. Dunkel, 1999)

112/04/19 4

Original ideas (2)Original ideas (2)

Level 3

Level 4

Level 2

Level 5

Level 1

15 minutes

15 minutes

15 minutes

15 minutes

15 minutes+3 itemsin a row

+3 itemsin a row

-3 itemsin a row

-3 itemsin a row

112/04/19 5

Online Adaptive Vocabulary Online Adaptive Vocabulary Test (OAVT)Test (OAVT)

• the first stage of a web-based adaptive English placement test system

• vocabulary grammar / sentence structure listening reading comprehension

112/04/19 6

Classification of word listClassification of word list

• Collins COBUILD English Dictionary for Advanced Learners (2001): frequency bands (◆◆◆◆◆ ◆)

• Macmillan English Dictionary for Advanced Learners of American English (2002): the most frequent words in English ( ★★★ ★)

• Longman Dictionary of Contemporary English (red entry)

112/04/19 7

Levels of Word List (1)Levels of Word List (1)• Level 0: basic and the most frequently used

words (e.g. and, I, my, the, etc.)

• Level 1: Collin COBUILD English Dictionary

frequency band 5 ◆◆◆◆◆ and frequency band 4 ◆◆◆◆

112/04/19 8

Levels of Word List (2)Levels of Word List (2)• Level 2: Longman Dictionary W1, S1, W2, S2, W3,

S3 (-- Collin COBUILD English Dictionary◆◆◆◆◆ and ◆◆◆◆)

• Level 3: (A) Collin COBUILD English Dictionary

frequency band 3 ◆◆◆ (-- Longman Dictionary S1, S2, S3, W1, W2, W3)

(B) Macmillan English Dictionary ★★ (and ★★★ if not in Level 1 and 2)

112/04/19 9

Levels of Word List (3)Levels of Word List (3)

• Level 4: (A) Macmillan English Dictionary ★(B) Collin COBUILD English Dictionary

frequency band 2 ◆◆

• Level 5:

• Collin COBUILD English Dictionary frequency band 1 ◆

112/04/19 10

Numbers of Word ListNumbers of Word List

level 0 1 2 3 4 5

stemstem 141

words

1,567

words

1,548

words

1,179

words

2,670

words

6,198

words

derivativesderivatives

(word family)(word family)

3,864

words

3,435

words

2,583

words

6,038

words

13,066

words

112/04/19 11

Definition of item codeDefinition of item code

V1262022N000001

Code 1: Type

V: vocabulary

G: grammar

L: listening

R: reading

Code 2~4: Level

-3 <= b <= +3

b: difficulty parameter

code 2: 1(+); 0(-)

Code 5: Specification

1: recognition

2: collocation

3: context clue

4: synonymous clue

5: ESP

6: spelling

7: unclassified

Code 6~7: Category

1: general

2: business

3: science

4: medicine

5: law

Code 8: part of speech

1: verb

2: noun

3: adjective

4: adverb

5: phrase

6: idiom

Code 9: N

Code 10~15: serial number

000001~999999

112/04/19 12

Test-stopping rulesTest-stopping rules• An adaptive test terminates when one o

r more of the following stopping rules is met: (Wainer, 1990)

(A) when a target measurement precision level has been attained

(B) when a pre-selected number of items has been given

(C) when a predetermined amount of time has elapsed

112/04/19 13

Interdisciplinary study of IRT-based Online Interdisciplinary study of IRT-based Online Adaptive English TestAdaptive English Test

Question Items

IRT logisticalmodelsProgramming

EnglishDept

ComputerScience

Dept

MathDept

User Interface

Test Security

ReliabilityValidity

ServerMaintenance

Statistical Analysis

112/04/19 14

112/04/19 15

112/04/19 16

112/04/19 17

112/04/19 18

112/04/19 19

112/04/19 20

112/04/19 21

112/04/19 22

112/04/19 23

112/04/19 24

112/04/19 25

112/04/19 26

112/04/19 27

112/04/19 28

112/04/19 29

112/04/19 30

112/04/19 31

112/04/19 32