language testing_developing and improving test item

8/13/2019 Language Testing_developing and Improving Test Item

1/22

DEVELOPING AND

IMPROVING TEST ITEM


2/22

WHAT IS AN ITEM ?

Brown (1996: 49) defines an item is thebasic unit of language testing. The item

is sometimes difficult to define. Some

types of items, like multiple-choice or

true-false items, are relatively easy toidentify as the individual test questions

that anyone can recognize as discrete

units.


3/22

DevelopingNorm-

ReferencedLanguage

Test

Item Format Analysis

DistractorEfficiency Analysis

ItemDiscrimination

Analysis

Item Facility Analysis


4/22

Item Format Analysis

GeneralGuidelines

Receptive

ResponseItems

Productive

ResponseItems


5/22

GeneralGuidelines

Is the itemformat correct lymatched to thepurpose andcontent of the

item? Is there onlyone correctanswer?

Is the itemwrit ten at the

students’ levelof prof ic iency?

Haveambiguousterms andstatements

been avoided?

Have negativesand double

negatives beenavoided? Does the item

avoid giv ingclues that could

be used inanswering

other items?

Are al l parts ofthe item on thesame page?

Is only relevantin format ionpresented?

Have race,gender, and

national ly biasbeen avoided?

Has at least oneother col leaguelooked over the

item?


6/22

ReceptiveResponse

Items

True-FalseItems

Multiple-ChoiceItems

MatchingItems


7/22

True-False Items

Is the statementworded careful ly

enough that it can be judged w ithout

ambigui ty?

Have “absoluteness”clues been avo ided?


8/22

Multiple-ChoiceItems

Have allunintentionalc lues beenavoided?

Are al l of thedistractorsplausible?

Has needlessredundancy beenavoided in the

opt ions

Has the orderingof the opt ionsbeen careful lycons idered? Orare the correct

answers randomlyassigned?

Have distractorssuch as “none ofthe above” and “aand b only” been

avoided?


9/22

MatchingItems

Are theremore

opt ions thanpremises?

Are opt ionssho rter thanpremises to

reducereading?

Are theopt ion and

premise l is tsrelated toone c entral

theme?


10/22

ProductiveResponse

Items

Fill-in Items

Short-response

Items

Task Items


11/22


12/22

Short-responseItems

Is th e i temformatted so thaton ly one relativelyconc ise answer is

poss ib le?

Is the item framedas a clear and

direct quest ion?


13/22

TaskItems

Is the students’task clear lydefined?

Is the tasksuf f ic ient ly narrow(and/or broad) forthe t ime avai lable?

Have scoringprocedures beenworked out in

advance with regardto the approach that

wil l be used?

Have scoringprocedures beenworked out in

advance with regardto the categories oflanguage that wil l be

rated?

Have scoringprocedures beenclear ly def ined interms of what eachscore within eachcategory means?

Is scoring to beas anonymousas possible?


14/22

ITEM F CILITY N LYSIS Brown (1996: 64) states that item facility (IF),

also called i tem di f f icul ty or i tem easiness , is astatistical index used to examine the

percentage of students who correctly answer a

given item.


15/22

ITEM F CILITY N LYSIS

=

Where

Ncorrect = number of students answering correctly

Ntotal = number of students taking the test


16/22

ITEM DISCRIMIN TION N LYSIS Item discrimination (ID) indicates the degree to which an

item separates the students who performed well fromthose who performed poorly. These two groups are

sometimes referred to as the high and low scorers or

upper and lower-proficiency students. The reason for

identifying these two groups is that ID allows teachers to

contrast the performance of the upper group students onthe test with that of the lower-group students (Brown,

1996: 66-67).


17/22

ITEM DISCRIMIN TION N LYSIS

= −

where

ID = item discrimination for an individual item

IFupper = item facility for the upper group on the

whole test

IFlower = item facility for the lower group on the

whole test


18/22

DISTR CTOR EFFICIENCY N LYSIS Brown (1996: 71) states the primary goal of

distractor efficiency analysis is to examine thedegree to which the distractors are attracting

students who do not know the correct answer.

To do this for an item, the percentages of

students who chose each option are analyzed.


19/22

NRT Development and ImprovementProjects

Pilot a relatively large number of test items on a group of studentssimilar to the group that will ultimately be assessed with the test.

Analyze the items using format analysis and statisticaltechniques.

Select the best items to make up a shorter, more effective revisedversion of the test.


20/22

Developing

Criterion-Referenced

Language Test

Item QualityAnalysis

Difference Index

The B-index

CRT ItemSelection


21/22

CRT DEVELOPMENT ND IMPROVEMENTPROJECTS

Brown (1996: 79) states that a central purpose

of a CRT is to assess how much of an objectiveor set of objectives has been learned by each

student, CRT assessment has to occur before

and after instruction in the concepts or skills

being taught in order to determine whetherthere was any gain in scores.


22/22

CONCLUSION

In testing, one way to improve and

develop a test is to examine the

individual items which are called item

analysis. This is usually done for

purposes of selecting the best items thatwill remain on a revised and improved

version of the test.

language testing_developing and improving test item

Documents