08 - agosto 17 2013 - material for test construction.pdf

7/25/2019 08 - agosto 17 2013 - material for test construction.pdf

1/20

Brown, H. D. (2001). Teaching by principles: An interactive approach to languagepedagogy. NY: Pearson Longman.

FOR EDUCATIONAL PURPOSES ONLY


2/20


3/20


4/20


5/20


6/20


7/20


8/20


9/20

39

C H P T ~ R

Language ssessment

I

Basic Concepts in Test Development

ency o r pronuncia t ion of a par t icular subset of phonology, and c an take th e form of

imitation, struc ture d respo nses, or free resp onses. Similarly, l istening co mp rehe n-

sion tests can conccn tra te o n a par t icular fca turc of language or o n overal l l is tening

for genera l meaning. Tests of reading can cov er the range of language uni ts and can

aim to test com prehens ion of long or shor t passages, s ingle sentence s, o r even

phrases and words . Wr i ting tc st s can take on an o pc ne nd cd form wi th f ree com-

posi t ion, o r be s tructured to e lici t anything f rom correct spel l ing to discoursc- level

compe tencc .

HISTORIC L DEVELOPMENTS IN L NGU GE TESTING

Historically, language-testing trend s and practices have follow ed th e ch anging w inds

and shifting sands of methodology described earlier in this book (Chapter 2 . For

exam ple , in th e 1950s, an era of behavior ism and specia l at tent ion to contrast ive

analysis , tcs t ing focuscd on specif ic language e lemen ts such as th e phonological ,

grammatica l, and lexica l contrasts beh vce n tw o languages. In th e 1970s and '80s,

communicat ive theor ies of language brought on more of an integra t ive view of

test ing in which tcst ing specia lis ts c la imed that the w ho le of th e comm unicat ive

event wa s considcrably greater than th e su m of i ts l inguistic elements (Clark 1983:

432). Today, test design ers are still challenged in the ir ques t for more auth entic ,

content-valid instruments that simulate real-world interaction while stil l rnceting

reliability an d p racticality criteria.

This his tor ica l perspcct ive underscores two major approaches to languagc

testing that still prevail, even if in m utated form, today: th c cho ice b etw ecn discrete

point a nd integrative tcs t ing methods. Discre te-point tcs ts werc constructed on

the a ssumpt ion tha t Ianguagc can be broken do w n in to it s com pone nt pa r t s and

tho se p ar ts adequate ly tested. Thos c com po ncn ts are basic;~lly the skil ls of lis-

tening, speaking, reading, writing, the various hierarchical units of languagc

(phonology/graphology, m orphology, lexicon, syntax, discourse) within cach skill ,

and sub categories with in tho se units. So, for exam ple, i t wa s claimcd th at a typical

proficiency test with its sets of multiple-choice questions dividcd into grammar,

vocabulary, reading, and the l ikc , with som e i tem s a t tending to smaller uni ts and

oth crs to largcr units , can measure thes e discre te points of language and, by ade-

qua te sam pling of thcse u nits, can achieve validity. Such a rationale is no t unrea-

sonable

if

on e cons idc r s types of tc st ing theo ry in w hich ce r ta in cons t ruc ts a re

measured by break ing d ow n the i r compo nent pa r ts .

Th e d iscre tc -po in t approach m et w i th so m e cr it ic ism as w e emerged in to an

era of em phasiz ing comm unicat ion, authentici ty, and context . T he earl iest c r it icism

(Ollcr 1979) argued that languagc c orn pcte nce is a unified set of interacting abilities

that can not b e tested separately. Th e cla im was, in shor t , tha t comm unicat ive com-

pete nce is so global and requires such integra tion (hence th e term integrat iven

test ing) tha t i t cannot be captured in addit ive tests of grammar and reading and


10/20

4 8 CHAPTER Language Assessment 11: Practical Classroom Applications

Table 22 2 Traditional and alternative assessment (adapted from Armstrong 1994 and Bailey

1998:207)

Traditional Assessment Alternative Assessment

One-shot, standardized exams

Timed, multiple-choice format

Decontextualized test items

Scores suffice for feedback

Norm-referenced scores

Focus on the right answer

Summative

Oriented to product

Non-interactive performance

Fosters extrinsic motivation

Continuous long-term assessment

Untimed, free-response format

Contextualized communicative tasks

Formative, interactive feedback

Criterion-referenced scores

Open-ended, creative answers

Formative

Oriented to process

Interactive performance

Fosters intrinsic motivation

It should be noted here that traditional assessment offers significantly higher

levels of practicality. Considerably more time and higher institutional budgets are

required to administer and evaluate assessments that presuppose more subjective

evaluation, more individualization, and more interaction in the process of offering

feedback. The payoff for the latter, however, comes with more useful feedback to

students, better possibilities for intrinsic motivation, and ultimately greater validity.

PRINCIPLES FOR DESIGNING EFFECTIVE CLASSROOM TESTS

For many language learners, the mention of the word

test

evokes images of walking

into a classroom after a sleepless night, of anxiously sitting hunched over a test page

while a clock ticks ominously, and of a mind suddenly gone empty

as

they vainly

attempt to multiple guessn heir way through the ordeal.

How can you, as a classroom teacher and designer of your own tests, correct

this image? Consider the following four principles for converting what might be

ordinary, traditional tests into authentic, intrinsically motivating learning opportuni-

ties designed for learners' best performance and for optimal feedback.

1.

Strategies

for

test takers

The first principle is to offer your learners appropriate, useful strategies for

taking the test. With some preparation in test-taking strategies, learners can allay

some of their fears and put their best foot forward during a test. Through strategies-

based test-taking, hey can avoid miscues due to the format of the test alone. They

should also be able to demonstrate their competence through an optimal level of

performance, or what Swain (1984) referred to as bias for best. Consider the

before-,during-,and after-test options (Table

22.3).


11/20


12/20


13/20


14/20


15/20


16/20


17/20

CHAPTER Language Assessment

:

Practicai Classroom App icalion s

4

7. Work for washback.

As you evaluate the test and return it to your students, your feedback should

reflect the principles of washback discussed earlier. Use the information from the

test performance as a springboard for rcview and/or for moving on to the nest unit.

LTERN TIVE ASSESSMENT OPTIONS

So far in this chapter, the focus has been on thc administration of formal tests

in

the

classroon~. t was noted earlier that assessmentn s a broad tcrm covering any con-

scious effort on the part of a teacher or student to draw some conclusions on the

basis of performance. Tcsts are a special subset of the range of possibilities within

assessment; of course they constitutc a vcry salient subset, but not all assessment

consists of tests.

In recent years language teachcrs have stepped up efforts to develop non-test

assessment options that are nevenhclcss carefully designed and that adhere to the

criteria for adequate assessment. Sometimes such innovations are referred to as

alternative assessment f

only to distinguish them from traditional formal tests.

Several alternative assessment options will be briefly discussed here: self- and peer-

assessments, ournals, conferences, portfolios, and cooperative test construction.

1. Self- and peer-assessments

conventional view of language pedagogy might consider self- and peer-assess-

ment to be an absurd reversal of the teaching-learning process. After all, how could

learners who are still in the process of acquisition, especially the early processes, be

capable of rendering an accurate assessment of their own performance? But a closer

look at the acquisition of any skill reveals the importance, if not the necessity, of self-

assessment and the benefit of pcer-assessment. What successful learner has not

developed the ability to monitor his or her own performance and to use the data

gathered for adjustments and corrections? Successfi~l earners extend the learning

process wcll bcyond the classroom and the presence of a tcachcr or tutor,

autonomously mastering the art of self-assessment. And whcrc peers are available to

render assessments,wlly not take advantage of such additional input?

Rescarch has shown (Brown

S

Hudson 1998) a number of advantages of self-

and pccr-assessment: speed, direct involvement of students, the encouragement of

autonomy, and increased motivation because of self-involvcrnent in thc process of

learning. Of course, the disadvantage of subjectivity looms large, and must be con-

sidered whenever you propose to involve students in self- and peer-assessment.

Following are some ways in which self- and peer-assessment can be imple-

mented in language classrooms.

Oral production: student self-checklists;peer checklists; offering and

receiving a holistic rating of an oral presentation; listening to tape-recorded

oral production to detect pronunciation or grammar errors; in natural


18/20

that entices the Mgh

group but 1s over h e head

of he low group,and

therefore

h e

atter

students

don t

even consider

it.

The

other two disiu;lcto~


19/20


20/20

n ~TFR

3

esigning < /asmorn

Language

~er; 63

chssroom

tars-consider the multitude

of uptions.Yt~umight

choose to return

the

tan to the

student with

one of or

a

mrnbioarion of, ny of the

possfbilltiesbelow:

I

a l e t t e r p d e

2.

a

tataI

score

3. Four subscores (spaking, listening,reading, writing)

4 for

the

listening and

reading

sections

a.

an

indication of carrect/incomcr

responses

b,

marginalcomments

5 for the

oral

interview

a. scores for each element

being

rated

b.

a

checklist af

areas

needing work

c oral feedhack

after

the

interview

cl a post-interviewconference

to go

over the results

6

an

the

essay

a scores for each

dement bring med

b. a checklist of areas needing work

c.

marginal and end-of-essay

comments

suggesfions

d,

a post-test canfer~rnceo

go

over work

e.

a

self-xssessment

7.

on aLl w

selected parts of

the

test peer checking of results

8. a wholedass

discussion

of

resulrs

of

she

cesr

9

individual conferences with each student ro r e v im the

whole

test

OMnusS.

options

f

and

2

giv

virtu lly

no

eedback-They

offer

the

student

only

a modest

sense

orwhere that studem

stands

and vague idea of ovcran

perfnrmance

hut

tile feedback they

present

does

not

become washhack.Washbacli

is

chieved

when students can, hrough the testing arperience, identify

rheir

a m

f success

and

d~allenge.When test becomes

a learning

experience, t achieves washhack.

Option 3

gives

a student a chmce

to see

the relative strer~gth f each skill

a m

and

W hecomes

minim~ll -

usef i~ l .Clptions

4 S

and

6 represent the

kind of

rrsponnse

tachw can

give

induding stimulatinga

student self-assessment)

that

approaches

m ximum

w s l ~ b c k tudents are provided wirh individualized Feedback

ckat has

good potential r

"washing

hackn

nto their

sul~seguent

erformance,

Of course

time

and

the

logistics

of

large

classes

r n q

nor

permit

Sd

and 6d,

which

for

many

tmchers

m ry be

going above and

h q n d

expectations tbr a test like tl .Likewise

option 9

mxy

be

impmctical.

Options

6m

7,110weware clearIyviable possibilities that solve

some

of the pracridity issues

that

are so important in reachers bus).

schedules,

h isd~apter,

uidelines

and

tools

were

provided ro enable

p u

o address th r

five questions posed at the outset: 1)

how

o determine the purpose or

crircriou

of

the

test.

21 how to

stare objectives,

3) how

to design

specficntims, 4) how to

08 - agosto 17 2013 - material for test construction.pdf

Documents