language assessment principles and classroom pratices

8/18/2019 Language Assessment Principles and Classroom Pratices

http://slidepdf.com/reader/full/language-assessment-principles-and-classroom-pratices 1/335

longman com



ONTENTS

Preface

Text Credits

Testing

Assessing

and

Teaching

What

Is a Test?, 3

Assessment

and

Teaching, 4

Informal

and

Formal Assessment, 5

Formative

and

Summative Assessment, 6

Norm-Referenced

and

Criterion-Referenced Tests, 7

Approaches to

Language Testing: A Brief History, 7

Discrete-Point

and

Integrative Testing, 8

Communicative Language Testing, 10

Performance-Based Assessment, 10

Current

Issues in Classroom Testing,

New

Views on Intelligence,

Traditional

and

Alternative Assessment, 13

Computer-Based Testing, 14

Exercises 16

For

Your

Further Reading 18

2 Principles

of Language

Assessment

Practicality, 19

Reliability, 20

Student-Related Reliability,

2

Rater Reliability,

2

Test Administration Reliability,

2

Test Reliability, 22

Validity, 22

Content-Related Evidence, 22

Criterion-Related Evidence, 24

Construct-Related Evidence, 25

Consequential Validity,

6

x

xii

1

9



v ONTENTS

Face Validity, 26

Authenticity, 28

Washback, 28

Applying Principles to the Evaluation of Classroom Tests, 30

1

Are

the

test

procedures

practical? 31

2. Is the test reliable? 31

3

Does

the procedure

demonstrate

content

validity? 32

4 Is the procedure face valid and biased for best ? 33

5 Are

the

test tasks as authentic as possible? 35

6 Does the test offer beneficial washback to the learner? 37

Exercises 38

For Your Further Reading

4

3 Designing

lassroom

Language Tests

Test Types, 43

Language

Aptitude

Tests, 43

Proficiency Tests, 44

Placement Tests, 45

Diagnostic Tests,

46

Achievement Tests, 47

Some

Practical Steps to Test Construction 48

Assessing Clear, Unambiguous Objectives, 49

Drawing Up Test Specifications, 50

Devising Test Tasks, 52

Designing Multiple-Choice Test Items, 55

1 Design each item to measure a specific objective, 56

2 State both

stem

and options as simply and directly as pOSSible, 57

4

3. Make certain

that

the intended answer is clearly the only correct

one

58

4. Use

item

indices to accept discard, or revise items, 58

Scoring, Grading, and Giving Feedback, 61

Scoring, 61

Grading,

62

Giving Feedback,

62

Exercises 64

For Your Further Reading 65

4 Standardized Testing

What Is Standardization?, 67

Advantages and Disadvantages

of

Standardized Tests, 68

Developing a Standardized Test,

69

1 Determine the purpose and objectives of the test, 70

2 Design test specifications,

70

3. Design, select, and arrange test tasks/items,

74

4

Make appropriate evaluations of different kinds of items, 78



5. Specify scoring

procedures

and

reporting

formats,

79

6. Perform ongoing construct validation studies, 81

Standardized Language Proficiency Testing, 82

Four Standardized Language Proficiency Tests, 83

Test

of

English as a Foreign Language TOEFL@),84

Michigan English Language Assessment Battery MEIAB), 85

International English Language Testing System

lELTS),

85

Test of English for International Communication TOEIC@), 86

Exercises 87


Appendix to Chapter 4:

Commercial Proficiency Tests:

Sample

Items and Tasks,

88

Test of English as a Foreign Language TOEFL , 88

Michigan English Language Assessment Battery MEIAB), 93

International English Language Testing System

lELTS),

96

Test of English for International Communication T O E I C ~ , 100

5 Standards Based

Assessment

ELD

Standards, 105

ELD

Assessment, 106

CASAS

and SCANS, 108

Teacher Standards, 109

The Consequences of

Standards-Based

and

Standardized Testing, 110

Test Bias, 111

Test-Driven Learning and Teaching, 112

Ethical Issues: Critical Language Testing, 113

Exercises 115

For Your Fur ther Reading 115

6

Assessing

Listening

Observing

the

Performance

of the

Four Skills, 117

The Importance

of Listening, 119

Basic Types of Listening, 119

Micro- and Macroskills of Listening, 121

Designing Assessment Tasks: Intensive Listening, 122

Recognizing Phonological and Morphological Elements, 123

Paraphrase Recognition, 124

Designing Assessment Tasks: Responsive Listening, 125

Designing Assessment Tasks: Selective Listening, 125

Listening Cloze, 125

Information Transfer, 127

Sentence

Repetition, 130

Designing Assessment Tasks: Extensive Listening, 130

O

T NTS V

104

116



v CONTENTS

Dictation, 131

Communicative Stimulus-Response Tasks, 132

Authentic Listening Tasks, 135

Exercises 138

For Your

Further

Reading 139

Assessing Speaking

Basic Types

of

Speaking, 141

Micro- and Macroskills

of

Speaking, 142

Designing Assessment Tasks: Imitative Speaking, 144

PhonePass® Test, 145

Designing Assessment Tasks: Intensive Speaking, 147

Directed Response

Tasks, 147

Read-Aloud Tasks, 147

Sentence Dialogue

Completion

Tasks

and

Oral Questionnaires, 149

Picture-Cued Tasks, 151

Translation of Limited Stretches

of

Discourse), 159

Designing Assessment Tasks: Responsive Speaking, 159

Question

and

Answer, 159

Giving Instructions

and

Directions, 161

Paraphrasing, 161

Test

of

Spoken English TSE®), 162

Designing Assessment Tasks: Interactive Speaking,

167

Interview, 167

Role Play, 174

Discussions and Conversations, 175

Games, 175

Oral Proficiency Interview (OPD, 176

Designing Assessment: Extensive Speaking, 179

Oral Presentations, 179

Picture-Cued Story-Telling, 180

Retelling a Story,

News

Event, 182

Translation of

Extended

Prose), 182

Exercises 183


8 Assessing Reading

Types (Genres)

of

Reading, 186

Microskills, Macroskills, and Strategies for Reading,

187

Types

of

Reading, 189

Designing Assessment Tasks: Perceptive Reading, 190

Reading Aloud, 190

Written

Response, 191

14

185



Multiple-Choice, 191

Picture-Cued Items, 191

Designing Assessment Tasks: Selective Reading, 194

Multiple-Choice for Form-Focused Criteria), 194

Matching Tasks, 197

Editing Tasks, 198


Gap-Filling Tasks, 200

Designing Assessment Tasks: Interactive Reading, 201

Cloze Tasks, 201

Impromptu

Reading Plus

Comprehension

Questions, 204

Short-Answer Tasks , 206

Editing Longer Texts), 207

Scanning, 209

Ordering Tasks, 209

Information Transfer: Reading Charts, Maps, Graphs, Diagrams, 210

Designing Assessment Tasks: Extensive Reading, 212

Skimming Tasks, 213

Summarizing and Responding, 213

Note-Taking

and

Outlining, 215

Exercises 216

For Your Further

Reading 217

9 Assessing riting

Genres of

Written

Language, 219

Types

of

Writing Performance, 220

Micro- and Macroskills

of

Writing, 220

Designing Assessment Tasks: Imitative Writing, 221

Tasks in [Hand] Writing Letters,Words, and Punctuation, 221

ONTENTS

V

2 8

Spelling Tasks and Detecting Phoneme-Grapheme

Correspondences

223

Designing Assessment Tasks: Intensive Controlled) Writing, 225

Dictation and Dicto-Comp, 225

Grammatical Transformation Tasks, 226


Vocabulary Assessment Tasks, 229

Ordering Tasks, 230

Short-Answer and

Sentence

Completion Tasks, 230

Issues

in

Assessing Responsive and Extensive Writing, 231

Designing Assessment Tasks: Responsive

and

Extensive Writing, 233

Paraphrasing, 234

Guided Question and Answer, 234

Paragraph

Construction

Tasks, 235

Strategic Options, 236

Test

ofWritten

English TWE@), 237



v CONTENTS

Scoring Methods for Responsive and Extensive Writing, 241

Holistic Scoring, 242

Primary Trait Scoring, 242

Analytic Scoring, 243

Beyond Scoring: Responding to Extensive Writing, 246

Assessing Initial Stages of the Process of Composing, 247

Assessing Later Stages of the Process of Composing, 247

Exercises 249


1 Beyond Tests: Alternatives in Assessment

The Dilemma

of

Maximizing Both Practicality and Washback, 252

Performance-Based Assessment, 254

Portfolios, 256

Journals,

26

Conferences and Interviews, 264

Observations, 266

Self- and Peer-Assessments,

27

Types of

elf and

Peer-Assessment, 271

Guidelines for Self- and Peer-Assessment, 276

A Taxonomy of Self- and Peer-Assessment Tasks, 277

Exercises 279


11

Grading

and

Student Evaluation

Philosophy of Grading: What Should Grades Reflect? 282

Guidelines for Selecting Grading Criteria, 284

Calculating Grades: Absolute and Relative Grading, 285

Teachers'

Perceptions of Appropriate

Grade Distributions,

289

Institutional Expectations and Constraints, 291

Cross-Cultural Factors

and

the Question

of Difficulty, 292

What Do Letter Grades Mean ?, 293

Alternatives to Letter Grading, 294

Some Principles and Guidelines for Grading and Evaluation, 299

Exercises 300


Bibliography

Name

Index

Subject

Index

251

281

3 3

313

315



PREF CE

The

field

of second

language acquisition and

pedagogy

has enjoyed a half

century

of academic prosperity,

with

exponentially increasing

numbers

of books, journals,

articles, and dissertations

now

constituting our stockpile

of

knowledge. Surveys

of

even

a subdiscipline

within

this growing field

now

require

hundreds of

biblio

graphic entries to

document the

state of

the

art.

In

this melange

of

topics and issues,

assessment remains

an

area of intense fascination. What

is

the best way to assess

learners ability?

What

are

the most

practical assessment

instruments

available? Are

current standardized tests of language proficiency accurate

and

reliable? In an

era

of

communicative language teaching,

do

our classroom tests measure up to standards

of authenticity and meaningfulness?

How can

a

teacher

design tests

that

serve

as

motivating learning experiences

rather than

anxiety-provoking threats?

ll these and many

more

questions now being addressed by teachers,

researchers, and specialists

can be overwhelming

to

the

novice language teacher,

who

is

already baffled by linguistic and psychological paradigms and

by

a multitude

of methodological options. This

book

provides

the teacher

trainee

with

a clear,

reader-friendly presentation of

the

essential foundation stones of language assess

ment, with

ample practical examples to illustrate their applicat ion in language class

rooms. It

is

a

book

that simplifies the issues

without

oversimplifying. It

doesn t

dodge

complex

questions,

and

it treats

them

in ways

that

classroom teachers

can

comprehend.

Readers do not have to

become

testing

experts

to understand

and

apply

the concepts

in this book, nor

do they

have

to become

statisticians

adept

in

manipulating mathematical equations and advanced calculus.

PURPOSE

ND

UDIENCE

This

book is

designed to offer a

comprehensive

survey of essential principles

and

tools for

second

language assessment. t has been used in

pilot

forms for teacher

training courses in

teacher

certification and in Master

of

Arts in

TESOL

programs.

s

the

third in a trilogy

of teacher

education textbooks, it is designed to follow my

other two books, Principles of anguage earning nd Teaching (Fourth Edition,

x



PREF CE

Pearson Education, 2000) and Teaching by Principles (Second Edition, Pearson

Education, 2001). References to those

two

books are sprinkled throughout

the

cur

rent

book. In keeping

with the tone set

in the previous

two

books, this

one

features

uncomplicated

prose

and a systematic, spiraling organization. Concepts are intro

duced with

a maximum

of

practical exemplification and a minimum

of

weighty def

inition. Supportive research is acknowledged and succinctly explained without

burdening

the

reader

with ponderous

debate over minutiae.

The

testing discipline sometimes possesses an aura

of

sanctity that

can

cause

teachers to feel inadequate as they approach the task of mastering principles and

designing effective instruments. Some testing manuals ,

with

their heavy emphasis

on jargon and mathematical equations, don t help to dissipate that mystique. y the

end of anguage

Assessment: Principles

and

Classroom Practices readers will have

gained access to this not-so-frightening field. They will have a working knowledge

of

a number

of

useful fundamental principles

of

assessment and will have applied

those

principles to practical classroom contexts. They will have acquired a store

house of

useful, comprehensible tools for evaluating and designing practical, effec

tive assessment techniques for their classrooms.

PRINCIP L FE TURES

Notable features

of

this book include the following:

• clearly framed fundamental principles for evaluating and designing assess

ment procedures of

all kinds

• focus

on the most

common

pedagogical challenge: classroom-based assess

ment

• many practical examples to illustrate principles and guidelines

• concise

but

comprehensive treatment

of

assessing all four skills (listening,

speaking, reading, writing)

• in each skill, classification

of

assessment techniques that range from

con

trolled to open-ended

item

types

on

a specified

continuum of micro

and

macroskills of language

•

thorough

discussion

of

large-scale standardized tests: their

purpose,

design,

validity, and utility

• a look at testing language proficiency.

or

ability

• explanation

of what

standards-based assessment is,

why

it

is

so popular, and

what its pros and cons are

• consideration

of the

ethics

of

testing in an educational and commercial

world driven by tests

• a

comprehensive presentation of

alternatives in assessment, namely, portfo

lios, journals, conferences, observations, interviews,

and self- and peer

assessment



PREF CE X

• systematic discussion

of

letter grading

and

overall evaluat ion

of

student

per

formance

in

a

course

• end-of-chapter exercises that suggest whole-class discussion

and

individual

pair

and

group

work

for

the teacher

education classroom

• a few suggested additional readings at

the end of

each

chapter

WORDS OF TH NKS

anguage Assessment Principles

and

Classroom Practices is

the product of

many

years

of

teaching language testing

and

assessment in

my

own classrooms.My students

have collectively taught

me

more

than

I have taught

them, which

prompts

me

to

thank them

all everywhere for these gifts of knowledge. I

am further indebted to

teachers in many countries around the world where I have offered occasional work

shops and seminars on language assessment. I have memorable impressions of such

sessions in Brazil the Dominican Republic Egypt Japan Peru Thailand Turkey

and

Yugoslavia

where

cross-cultural issues in assessment have

been

especially stimulating.

I am also grateful to my graduate assistant Amy Shipley for tracking down

research

studies

and

practical

examples

of tests

and

for

preparing

artwork for

some

of

the

figures

in

this book. I offer an appreciative

thank

you

to my

friend Maryruth

Farnsworth who

read the manuscript with an editor s eye and

artfully

pointed out

some idiosyncrasies n my writing. My gratitude extends to my staff at the American

Language Institute at San Francisco State University especially Kathy Sherak Nicole

Frantz

and

Nadya McCann who carried the ball administratively while I

completed

the bulk of writing on this project. And thanks to my colleague Pat Porter for

reading

and commenting on

an

earlier draft

of

this book.

As

always

the

embracing

support

of

faculty

and

graduate students at San Francisco State University is a con

stant source of stimulation and affirmation.

H Douglas Brown

San Francisco California

September 2003



TEXT REDITS

x

Grateful acknowledgment is made to the following publishers and authors for per-

mission to reprint

copyrighted

material.

American Council on Teaching Foreign Languages (ACTFL), for material from

ACTFL Proficiency Guidelines:

Speaking (1986);

Oral ProfiCiency

Inventory

OPI):

Summary

Highlights.

Blackwell Publishers, for material from Brown, James

Dean

& Bailey, Kathleen M.

(1984). A categorical

instrument

for scoring second language writing skills. Language

Learning, 34,

21-42.

California Department of Education, for material from California English

Language Development

ELD) Standards:

Listening

and Speaking.

Chauncey Group International (a subsidiary

of

ETS , for material from

Test of

English

for International Communication

TOEIC®).

Educational Testing Service (ETS), for material from Test

of

English as a Foreign

Language

TOEFL®);

Test

of

Spoken

English TSE®); Test

of

Writ ten English TWE®).

English Language Institute, University of Michigan, for material from Michigan

English Language ssessment Battery MELAB).

Ordinate Corporation, for material from

PhonePass®.

Pearson Longman ESL, and Deborah Phillips, for material from Phillips, Deborah.

2001). Longman Introductory Course

for

the TOEFL®

Test. White

Plains, NY:

Pearson

Education.

Second Language Testing, Inc.

SLm,

for material from

Modern

Language ptitude

Test.

University

of

Cambridge Local Examinations Syndicate (UCLES), for material from

International

English

Language

Testing System.

Yasuhiro Imao, Roshan Khan, Eric Phillips, and Sheila Viotti, for unpublished material.



CHAPTER

TESTING, ASSESSING,

AND TEACHING

If

you hear the

word

test in any classroom sening ,your th

oughts

arc nOt likely to be

positive, plcas,lllt , or :lffirming.

The

anticipation

of

a leSt is al

most

always

accompa-

nied

by

feelings

of

an

xiety

and self-doub

t d

ong

with II

fervent hope

th t

you

w ill

co

me

out

of

it

l

ve. Tests

seem

as

unavoidable as tOmorrow s sunri

se

in virrually

every

kjnd

of educational setting. Courses of study

in

every

diSCipline

are marked

by periodic (csts

-milcstom::s

of progress

or

inadequacy)-and you intense ly wish

for a mil lcuious

exempt

ion from

these

ordeals.

We

li

ve

by

tests and

sometimes

(mcl aphoricall y) die by lhem .

For a quick revisiting

of

how tests affect many learners, take the following

vocabu

lary quiz.

All tJlt:

words

are

found in standard English dictionaries,

SO

r ll

should

be

able

to

answer aU six items correctly, right?Okay, take the

quiz

and

circle

the correct definition for each word .

Circle the correct answer.

ou

have

3

minutes to complete this examination

1. polygene a.

b.

c.

d.

2.

cy

nosure

a

b

c.

d

the first stratum of lower-order protozoa containing multiple genes

a combination

of

two or more plastics to produce a highly durable

material

one of a set of cooperating genes, each producing a small

quantitative effect

any of

a number of multicellular chromosomes

an object that serves as a focal point of attention and admiration; a

center of interest or attention

a narrow opening caused by a break or fault in limestone caves

the cleavage in rock caused by glacial activity

one of a group of electrical Impulses capable of passing through

metals

1



2 Cli PTfR

1

Testing N5eSsing and e.1c hing

3. gudgeon

4. hippogriff

5. reglet

6 Hctile

a.

b.

c.

d.

a.

b.

c.

a jail for commoners during the Middle Ages, located in the villages

of Germany and France

a strip of metal used to reinforce beams and girders in building

construction

a tool used by Alaskan Indians to carve totem poles

a small Eurasian freshwater fish

a term used in children s literature to denote colorful and descriptive

phraseology

a mythological monster having the wings, claws, and head o a

griffin and the body of a horse

ancient Egyptian cuneiform writing commonly found on the walls of

tombs

d. a skin transplant from the leg or foot to the hip

a. a narrow, flat molding

b a musical composition of regular beat and harmonic intonation

c.

an Aus tralian bird

of

the eagle family

d. a short sleeve found on women s dresses in Victorian

England

a. a short. oblong-shaped projectile used in early eighteenth-century

cannons

b. an Old English word for the leading character of a fictional

novel

c. moldable plastic; formed of a moldable substance such as clay or

earth

d. pertaining to the tendency

o

certain lower mammals to lose visual

depth perception with increasing age

Now,

how

did that make you feel? Probably just t

he

same as many learners

fee l w hen they take many multiple-choice or shall we

s y

multiple·guess?).

ti

med

, Rricky·

tests

. To add

to th

e

to

rme,llt, if this

were

a

co

m

merc

ially a

dm

ini s·

te red standardized rest ,

you

mig

ht

have

to

wait weeks before learning yo

ur

resul tS .Yo u

can

check you,. an

swers

on this qu iz now by furning to page 16. If

yO ll

correctl

y idcn

ti fied

three

or

more items , congratulations YOli jllst

excee

ded

th

e average.

Of course, this little pop quiz on obscu

re

vocabulary is not :m appropriate

example

of

classroom·based achievement tcsting,nor is it intended

to

be. It's simply

an

illustration

of how

tests make us

[eel

much of the

ti

mc. can tests be positive

expe

riences?

am

they build a

person

's confidence and

become

learn ing experi

ences?

C 1

n they bring OUi tbe best in studen

ts

The answer is a resounding yes

Tests need not be degrading, artifiCial, anxiety·provoking experiences. And

that

's

part

ly what

th is

book is

a11

about: helping

YOll

to create m

ore aut

he

nt

ic, i

nt

rinsica llr



CHM'TfR I

Testing

Assessing, and Te

ching 3

motivating assessment

procedures

that are approp riate for their context and

designed

to

offcr constnlctive feedback

to

your

students

.

Before we look at tests and

(CSt

design in

second

language education,

we

need

to

understand three basic interrelated concepts: testing, assessment, and teaching.

Notice that

the

title

of

this book

is

Langtwge Assessmenl,

not

Language Testing.

Thcre

are

impon

'am differences

between these tWO

constructs. and an even more

important relationShip among testing, assessing, and tcaching.

WH T IS A TEST?

A test, in simple terms, is a

melhod

o

measuring a person s abillt;l knowledge, or

performance

in

a given domain.

Let 's look at

the compone

nts

of

this definition. A

test is first a method.

It

is an instrume

nt a

set of techniques, procedures,

or i lems-

th:lt reqUires performance on the part

of

the test-taker.

10

qualify as a test, the method

must be exp licit and stnlCtured: multiple-choice questions with prescribed correct

answers; a writing prompt with a searing mbric; an oral interv

i

w based on a ques

tion script and a dlecklist

of

expected n:sponses to be filled in by the administrator.

Second, a test must measure. Some tests measure

gener..

ability. while

olhers

focus on very

spec

ific

competencies or

objectives. A muJti-skili proficiency test

dctermines a

gene "AI

ability level; a quiz on recognizing

correc

t use

of

definite arti

cles measures specific knowledge. TIle

W dY the fCSuJtS or

measurements are com

mUll.icated may vary. Some tests. SUdl as a classroom-based shon-an

swer

essay test,

may earn

the

test-take r a letter grade accompanied by the instructor'S marginal com

m

ents

.

Others

, particuJarly large-scaJe standardized tests, provide a

tO tal

numerical

score, a percenlile rank, and

perhaps

some

subscores. If an instrument

does not

specify a form of reporting measurement a means for offering the test-taker some

kind of

result then

that technique cannot appropriately be defmed as a test.

Ncxt,a test measures an individual's ability, knowledge,

or

performance.Testers

need

to

understand who the test-takers are.What

is their

previous experiencc and

background? Is the test appropriately

matdl

ed

to their

abilities? How should test

takers interpret lheir scores?

A leSt measures

performance, but the

results imply

the

test·raker's ability,

or

, to

use a

concept co

mmon in the field

of

linguistic

s,

competence . Most language tests

measufC one 's ability

to

perform language. that is , to speak, write, read,

or li

sten

to

a

s

ub

set

of

language.

On

the

other

hand, it

is

not

uncommon

to

find

teStS

designed

to

tap into a test-taker's knowledge

i h2 . langlL1

ge:dcftning a vocabulary item, reciting

a granunatical rule,

or

identifying a rhetorical feature in written discourse.

Performance-based tests sample the test-taker 's actual use

of

language, but from

those samples the test administrator infers generd l competence. A test of reading

comprehension, for example, may consist of several short reading passages each fol-

lowed by a limited number

of

comprehension

questions a

small sample of a

second

language learner's

total

reading behavior. But from the results

of

that test, the

examiner may infer a certain level

of

general reading ability.



4 w

CR

I

Tf S,ing,

Assessing,

lIId

Teaching

Finally. a

1 eSI

measures a given domain. In

the

case

of

a proficiency

CSt

,

even

though

the

actual

perfonnance

on me test involves only a sampling of skills,

that

domain

is overnU

proficiency

in

a

language-general

competence in a

ll

skills

of

a

language.

Olher

tests may have more specific criteria. A test of pronunciation might

well

e

a

tCSt

of on

ly a limited

set of phonemic

minimal pairs.A vocabulary lesl may

focus on on ly the set of words covered in a particular lesson or unit. One of the

biggest

obs

tacles

to

overcome

in

conslructing adequate tests

is

to measure the

desired criterion and

nOt

include other factors inadvertcmiy, an issue that is

addressed

in

Chapters 2 and 3.

A well-constru

cte

d test

is

an instnlment that provides an

:ICC

urate measure

of

the test-laker's ability within a particular domain. 11lt:: definition

sou

nds

fdirly

simple,

bur in

fdCt

,

constr

ucting a good te

st is

a

comp

lex task involving both science and

art .

ASSESSMENT AND TEACHING

Assessment is :1popular :lnd sometimes misunde rstood I

crm

in curren t educational

practice.

You

might e

tempted

to

t ink of

testing and assessing as synonymous

terms, but they are

nOL

Tests are prepared administr:nivc proccdu.fCS that

occu

r

at

idenrifiable times in a

curriculum when

learners

mu

ste r aJl

their

fuculti

es

to offer

peak perfornlance, knowing that

their

responses arc being measured and evaluated.

Assessment. on

the

other hand, is an ongoing process

that

encompasses a

much

wider

domain . Whenever a

student

responds

to

a questiOn, offers a

comment

,

or

tries O

ut

a new word or SlruCfure,

the

teacher subconsciously makes an assessmem

of the student

's performance.Written

work-from

a joued-down phrase to a formal

essay-is

performance that u1timatt:1)'

is

assessed by self, teacher, and possibly oth

er

.

st

ud

ents

. Reading and

li

stening activities

lI

suaUy require some

SOrt of

productive

performance that

the

teacher implicitly judges, however peripheral that judgment

may

be.

A good

teacher

never ceases

to

assess students, wh

et

her th ose assessmcnts

are incidental

or

intend cd.

Tests, then, arc a s

ub

set of assessment; they are certainly not the on ly form of

a'i$Cssment tha t a teacher can make. Tests can

be

u

seful

devices,but they are only one

among many procedures and t:lsks that teachers can ultimately

uSt:

to asscss students.

But now

, you might be thinking, if you make assessmen ts every time you

teach

so

mething in 1.11C

dassroom

,

does

all teaching involve assessment?Are teachers con·

stantly assesSing stud

ents

wi

th no

interaction that

is

assessme

nt

-free?

'nlC w r

depends

on your perspective. For optim:.l learning to take place stll

dent'i in the classroom must have

the

freedom

to expe

riment,

to tryout

their

ow

n

hypotheses about language without feeling that their ove.ra U

compe

t

ence is

being

judged in temlS of those trials and errors.

I.n

the same way that tournament tennis

pbye

rs must, before a tournament, have the freedom

to pra<.

:

tice

their skiUs with

no

implications for their final placement on th:11 day of days,

so

al

so

must learners have

3ID pIe op

portunities

to

play with language in a classroom Without being formally



cw PTER 1 Tesling. Assessing.

and

Teaching 5

graded, Teaching sets

up

the practict: games of language learning: the opportunities

for learners to listen. think, take risks, set goals, and process feedback from the

coach- and then

recyde

through the skills that they are trying to m:lSter, (A diagram

of

the

rel:uionsh ip among testing, teaching, and assessme

nt

is found

in

Fig

ure

1.1.)

E:v

ASSESSMENT

TEACHING

Figure 1. 1. Tests assessment and teaching

At the same time,during these practice activities, teachers (and tennis coaches)

are indeed observing

snldents

' performance and making varions evaluations of cadI

learner: H

ow

did the performance compare to previolls performance? Which

aspects of the performance were better than others? Is tlle learner pedorming up

to an

c.. Cpccred

potential? H

ow

does

the performance

comp

are

to

that

of

ot

h

ers

in

the same learning communi ty? In the ideal dassroom , all these obscrv.

lIi

ons feed

intO tlle way the teacher provides instruction to each student.

Informal

and

Formal ssessment

One way to begin untangling the lex ical co

nundnml

c reated by distinguishing

among tests, assessment,and teaching is to distinguish between informal and formal

assessment.

Informal

assessment can

(:Ike

a number of forms, star ting with inci

demal , unplanned

com

ments and responses, along with

coac

hing and

ot

her

impromptu

feed

back to the student. Examples include saying kGood

you say c n or c n t? 4l think you meant to say you broke the glass,

n

ot

you

bre k

the glass

,

or putting a

on

some homework.

Informal assessment does not SLOp there. A good deal of a reacher 's informal

assessment is

embedded

in

dassroom

tasks designed to elicit performance witho

ut

recording results and making fLxed judgments about a student 's

comp

etence.

E.xamples at this end of the

continuum

are marginal comments on papers,

responding

to

a drAft of all essay, advice about

how to

bener pronounce a

wo

rd, a



C ~ f T E l Testing Assessing mel Teaching

suggestion for a strategy for compensating for a reading difficulty, and showing how

to modify a student's note-taking to bener remember the coment

of

a lecture.

On the

other

hand,

formal assessments

arc exercises or procedures specifi

ca lly designed to tap into a storehouse

of

skills and knowledge.They are systematic,

planned sampling

tedutiqucs

constructed to give teacher and student an appraisal

of studem achievement.

To

extend the tennis analogy, formal assessments are tbe

tournament games that occur periodically in the course

of

a regimen

of

practice.

Is fonnal assessment the same as a test?

We

can say that aU tests arc form3

assessments, but not

all

fonnal assessment is testing. For example, you might use a

student's journal or portfoliO of materi31s as a formal assessment of the allainment

of certain course objectives, but it is problematic to call those two procedures

"tests M A systematic set

of

observations

of

a student's frequen<:y

of

oral participation

in class is certai

nl

y a formal assessment, but it too

is

hardly what anyone would

ca

ll

a test. Tests arc usually rel:ltively tinH -constrained (usually spa nning a class period

or

at mO

St

several hours) and draw on a limiled sample

of

behavior.

Formative and Summative Assessment

Another useful distinction to bear in mind is the func ti

on

of an assessment: How is

the procedure to

be

used? 1\vo functions are commonly identified in the literature:

formative and summative assessment. Most

of

our dassroom assessment

is forma-

tive assessment: evaluating students in the process of o n n i n tbeir competen

cies and skills with the goal of helping them to continue that growth process. The

key 0 such formation

is

lhe del ivery (by the teacher) and int

ermLtizaUon

(by the stu

dent) of appropriatc feedback

on

performance, w ith an eye toward the future can·

tinuation (o r formation)

of

learning.

For 3 pr.1clical purposes, virtually all kinds of informal assessment are (or

should

be

formative. Tbey have as their primary focus tbe ongoing development of

the learner's language.

So

wben you give a student a comment

or

a suggestion,

or

ca ll attention to an error, that feedback is offered in order to improve tbe leamer's

language ability.

Summative assessment aims to measure, or summarize, what a student has

grasped, lind typically occurs at the end of a course or unit of in str uction. A sum

mation of w hat a student has learned implies looking back :

lOd

taking stock

of

b

ow

we

ll that studem h

as

accomplished objectives, but does not necessarily point the

way

to

future progress. Final exams

in

a course and general proficiency exams arc

examplcs

of

summative assessment.

One

of the problems with prevailing attitudes toward testing is the vi w that

l l tests (quizzes, periodic review tests, midterm exams, etc.) arc summative.At var-

iOlIS points

in

your past educational c.xperiences, no doubt you've considered such

tests as summ:J.tive.You may have tbought."Whew I'm glad that 's over. Now I don 't

ha e to remember thar stuff anymore "A challenge to you as a teacher

is

to change

thai attitude among your students: Can you instill a more formative quality to what



F

01APJ ER 1

Tes

tin

g, A55eS5iog.

and T

ea

chin8

yOllr

students

might

otherwise

vi

ew

as a summalivc test? Can you offer yO llr Stu-

dents ao opportunity to con

vert testS

into learning experiences· ? We will take lip

that

dl3.lIenge in s

ub

sequent chapters in th is book.

Norm·Referenced

and

Criterion-Referenced

Tests

Anotber dichotomy that is

important

to clarify h

ere and

th

at

aids in sorting out

ommon

terminology

in ssessment is tbe

distin tion

between norm·referenced

and criterion-referenced testing.

In

norm-referenced

tests

, each tcst-taker's score

is interpreted in relation to a mean (average score) , median (middle score) , standard

deviation

(extent of

variance in scores), and/or percentile rank.The

purpose

i.o such

tests

is

to pla

ce

tcst·takers along a mathematical

cont

inuum

in

rank order. Scores are

usually reponed back to the test-taker in the form of a numerical

score

(for

examp

le,

230

o ul

of

300) and a

percentil

e rank (such

IS

84 percent , w hi

ch

means

that

the

test-taker's

score

was

higher

than

84 percent of the

total number

of

test

lakers,but lower than 16 percent in

that

administration). Typical of norm-referen

ce

d

tests are standardized testS like the Scholastic Aptitude Test SAl

4J

or

the

Test of

English as a Foreign Language (TOEFL ), imended to be administered to large audi

ences, with resuils efficiently disseminated to tesHakers.SUdl tes ts mUSt bave fixed,

predetermined responses in a format that can be scored quickJy at minimum

expense. Money and cfiiciency are primary concernS in these tests.

Criterion-referenced tcsts , on

the

otber hand ,

arc

designed to give test-takers

feedback, usuaUy in the fonn of grades, on specific course or lesson objectives_

C

la

ssroom tests involving

lhe

srudents in only

one

class, and

connec

ted to a c

ur

ri

culum,

are

typical of criterion-referenced testing. H

ere

, mu ch lime

and

effort on the

part

of

the tea

cher

(test admin

i

stra

t

Or)

are

sometimes

required in

order

to deliver

useful,

appropriate

feedback to

studems

,

or what

O

iJ

er ( 1979. p. 52)

c lled

s t r u

lion

al

In

a c

ri

terion-referenced tcs t, the distribution

of

students ' scores lCroSS

a continuum may be of little con

ce

rn as long as

the

instrument assesses

appropriate

ohjectives. In

tm

guage

Assessme1lf

with an ludience of classroom language

teachers and teachers in training, and with its emphasis on classroom-based assess

ment

(as opposed to standardized, large-scale tcsting), c riterion-referenced testing is

of more prominent interest than norm-referenced testing.

APPROACHES

TO

LANGUAGE TESTING: A BRIEF HISTORY

Now that yOll

ha

ve a reasonably

clear

grasp of some common assessment terms,we

now

rum to one of the primary

concerns

of this book:

the

creation and use o f tests,

particularly classroom tests. A brief

history

of language testing over the past half

cent

ury w ill serve as a backdrop to an

understanding

of classroom-based t'esting.

HistOrically, language-testing trends and practices have followed

the

shifting

sa nds of reaching me thodology (for a description of these trends, see Brown,



8 H PTfR Testing Assessing and Tei/ching

Tetlcbing by

Prlndples [hereinafter TBP Chapter

2) . For

example

, in

the

1950s,

an

era of behaviorism and special anention to contrastive analysis. testing focused on

specific language elements such as the phonological , grammatical , and lexical con·

trasts between

two

languages. In the 1970s and 1980s,

communicative

theories

of

language

brought

with them

a

more

integrative vi

ew

of

testing in

whlch

specialists

daimed that

whole

of the communicative event was

considerably greater than

the sum of

its linguistic elements& (Clark, 1983, p .

432)

. Today, tcst

designers

are still

challenged in their

quest

for more authentic, valid instruments that simulate real·

world interaction.

Discrete·Point

and

Integrative

Testing

11tis hi

sto

ri

ca

l perspective underscores

(\ 10

major approaches to language testing

that were debat.ed in the 1970s

and

early I980s. TIlcse approaches stiU prevail today,

eve

n

if

i.n

mutated

form : the

dlOice

between

di

scre

te·poi

nt

and

integr.uive testing

methods (OUCI'

, 1979).

Discrete·point tests

are

co

n

st r

u

cted

o n the

assump

tio n that

langlt:lge

can be

broken

down

into its

compone

m

parts and that those parts can be

t

ested

successfully. These components an the sk ills of UStening, speaking, reading,

and writing , and various unilS

of

language discrete points) of phonologYI

graphology, morphology, lexicon, syntax,

and

discourse . It was claimed that an

overall language

proficiency test

, then , should sample all four skills

and as

many lin

guistic

discrete points

as

pos >ible

.

Such an approach demanded a decontextuali ..a tion that often confused the

test-taker. So , as the profession emerged into an

era

of emphasizing communication,

authentiCity,

and

context, new approach

es

were sought. Oller 1979) argued that

language

competence

is a unified

set

of

interJ.Cting abilities that

canno

t

be

tested

separate

ly.

His

claim was

that communicative

competcnce is

so

global

and

requires

such

integration

hencc the term int

egrative-

testing

)

that

it

canno

t be

captured in

additive tes ts of grammar, reading, vocabulary.

and

other discrete points of language.

Othe rs among them C

. ko

, 1982. and Savignon, 1982) soon followed

in

their sup

pOrt for integrative tes ting .

What

docs

an

integrative test

look like?

Two

types

of te

sts have hi

sto

rically

been claimed

to

be examples of

integrative tests:

doze

teSls and

dictations

.

A doze

test

is

a

reading

pa

ssage

pe rhap

s

150

to

300 words) i.ll

which roughly

every sixth

0

1

seventh word has bet:n deleted; the test-taker is required to supply

words

that fit

into

those

blanks. (See

Chapter

8

for

a full di

scuss

io n

of doze

testing

.) O

ll

er

1979)

I

Frequent references

are

made

in

this book

1

companion vol um

es

by

Lhe

author.

Prin

ciples

o umguage eaming and

TeachiflE pUj )

Founh Edition, 2000) is a

basic teacher reference book

on

essential foundations

of

second language acquisition

on which pedagogical practices arc based. Teachf E

by

Pri1lclples (TEP) (Second

Edition. 200 I spells out that pedagogy in practical terms for the language teacher.



OW TER Testing. ssessinS and Teaching 9

claimed that doze test results are good measures

of

overall proficiency. According

to

theoretical

cOnStruCLS

underlying this claim,

the

ability

to

supply

appropriate

words in blanks requires a number of abilities that lie at the bean of competence in

a language: knowledge of vocabuJary, grammatical strucrure, discourse st ructure,

reading skills and strategies,

and

an internalized

expectancy

grammar

e

nabling

onc

to predkl

an item lhat will come

next

in a seque.nce). t

was

argued that suc

cessful comple.tion of doze items taps into

ll

of

those

abilities, wl:tich were said

to

be

the

essence

of gl

oba

l language proficiency.

Dictation is a familiar language-teaching tt.'dlnique that evo lved into a testing

tec

hnique. Esscmially, learners listen to a passage of 100

to

150

words

read aloud by

an admjnistr'J.tor

or

audi

otape)

and

write

what they bear, usi

ng correct

spelli

ng

.TIle

listening portion usually has three stages: an ornl reading without' pauses; an oral

reuding witJ, long pauscs between every phrase (to give the learner

ti

me to write

down what is heard);and a third reading at normal speed to give test- takers a chance

to ch

eck

what

th

ey

wrote

. (Sec Chapt

er

6 for more discussion

of

dictation as an

assessment device.)

Sup

porte

rs arguc that dictation is an integralive test because it lapS into gram

matical and discourse

competencies

required for other modes of performance in a

language. Success

on a dictation requires careful

listening

, reproduction in writing

of what is heard, efficient shon-ter:m memory, and , to an extent,

so

me expectancy

rules

to

aid the short-term memory. Funher, dictation test resul

t s

tend to correlate

strongly

with

other tesLS of profiCiency. Dictation testing is usually classroom

centered since large-scale administration of dictations is quite impractical from a

scoring standpoint. Reliability

of

scoring criteria for dictation tests can be improved

by designing mUltiple-choice or exact-word cloze test scoring.

Iwponents of integrative test

methods soon centered their

arguments on what

became known as the unitary trait hypothesis, which suggestt':d an indivisible

view

of

language proficiency: that vocabulary, grammar, phonology, the o u r s k i l l s

and other discrete

points

of language couJd

not

be disemangJcd from each other in

language performance . The unitary trait hypothcsis contended that there is a gen

era l factor of language proficiency such that all the discrete pOillLS do ot add up

to

that whole.

Ot hers argued strongly against the

un

itary trait pOSition. In a study of

st

udents

in Braz

il

and

the

Philippines, Farhady (1982) found signlfic:

mt

and widely varying

differences in performance on an ESt proficiency test, dcpt':nding on subjects' native

countr

y, maj

or

field

of

study,and graduate versus undergraduate statu

s.

For

examp

le,

Brazilians scored very low in listening comprehensi

on

and reillti\'ely high in reading

comprehension. Filipinos, whose scores on five of the six componenLS of the test

were conS

id

erably higher than Bra:dlians' scores,

were

actual ly l

ower

than Brazilians

in

re3ding

comprehension

scores. Farhady's

contentions

were

supported

in

other

research that seriously

questioned

the unitary trait hypothesis. Finally, in the face of

the

evidence, O

ller

retreated from his earlier stand and admitted that

the

unit3ry

tro.it

hypothesiS was w r o (1983 , p . 352).



10 CWoPTfR Testing. A 5 ~ ; n g .

and eaching

Communicative Language Testing

By lhe mid-1980s, the language-testing field had aba nd

oned

argumentS abom the

unilary trait h

ypo

thesis and had begun to focus on designing

co

mmunicative

languagNesting tasks. Bachman and Palmer (1996, p .

9)

include among Kunda·

menia l- principles of language testing the need for a correspondence between lan·

guage leSt performance and language use: ln order for a particular language test to

be useful

fo

r itS intended purposes , test performance must correspo

nd

in demon

stl 'Jb

le ways

to

language use in non-t

eS

t situation

s.

The

prob

lem that language

assessment expens faced was that tasks tended to

be

artificial, contrived, and

unlike ly to mirror language use in real life. As Wei r ( 1990, p. 6) noted, Integrative

tests such ;IS doze only tell us about a candidate's linguistic competence. llley do

not te

ll

us anything directl y about a student 's performance ability.

And

so

a qu

es

t for aUlhenticity was launched , as test d

eS

igne rs

centere

d on

comm

unicative

pe

rforman

ce

. FoU

ow

ing Canale and Swain 's ( 1

98

0) model

of

com·

lUunicative

co

mp

ete

nce . Baclunan ( I O proposed a model of language compe

tence

co

nsistin g of organizational and pragmatic

compete

nce, respectively

subdiv

id

ed into grammatical and textual components, and into

iIl

ocu tionary and

sociolinguistic components , (Further discussion of both Cam le and Swain's and

Bachman's mode ls can be found in T Chapte r 9.) Bachman and Palmer 1996.

pp.700 also e

mph

asized the importance

of

strategic competence the ab ility to

emplo)'

co

mmunicative strategies to compensa te for breakdowns as well as [

enh

ance

the rhetorical eff

ect

of utterances) in the

pr

ocess of communi

ca

tion. All

element

s of the model , especia lly pr

.l

gmatic and strategic abilities,

needed to

be

included in the

co

nstructs of language te

st

ing and in the ac tual perf

or

mance

required of test·rakers.

Communicative testing presented challenges to test deSigners, as we will see in

subsequent chapters of this book. Test constructors began to identify the kinds of

real·world tasks that language learners were caJled

upon

to perform. t

was

dear that

the contextS for those tasks were extraordimlrily w idely varied and that

the

sa

m-

pling

of

tasks for anyone assessment procedure needed to be validated by what lan

guage users actually do wi th language .Weir (1990, p. II ) reminded his readers that

measure language proficiency

. . .

account must now be taken of: where, when ,

how, with who

lll

,and why language is

10

be used ,and on what IOp ics ,and with what

eff

ec

t . And the assessment ficJd became more and more concerned with the

:

tlI

the

nt

idty of

tasks and the genuineness

of

l

ex

ts. (See Skehan , 1988. 1989, for a

survey

of co

mmunicative testing research.)

Performance-Based Assessment

In language

co

urses and progr.tms around the wo rld, test deSigners are now tackling

this new and more s[udent-centercd agenda (Ald

erso

n. 200 1, 2 02). Instead of just

offering p:aper-and-pencil selective response test's of a pletho ra of separate items,

perfomlance·ba.sed assessment of language typically involves o ral production,



CHAf'TCR1 Tesfing,

A5S1 S5ing.

and Teachin 11

written

produ

ction, open-ended responses, integrat

ed

performance (across

sk ill

areas), group perfonllance, and o

ther

interactive tasks.

To be

sure, such assessment

is ti.me-coosuming and therefore expensive,hut th

ose

extra efforts are paying off in

the form ofmore direct testing because st

uden

ts are assessed as th

ey

perform actual

or simulat

ed

real-world tasks. n technical terms, higher

co

mcm validity (see

Chapter 2 for an explana

ti

on)

is

achieved btcausc:: learne rs arc measured in the

process of perfornling the targeted linguistic acts.

to an English language-teaching

co

ntext, performance-based assessmem means

that you m

ay

have a difficu

lt

time diStingu

is

hing between formal and informal

assessment .

f

you rely a little less

on

forma

ll

y

st

ru

ct

ured tests and a little more on

evaluation w hile students are performing va

ri

ous tasks , you will be taking

so

me

steps

tow

ard m

eetin

g the goals of performanc

e-

based testing. (Sec C

hapt

er 10 for a

funher

di

sc

ussion of pe

rfor

mance·based assessme

nt

.)

A dl3racteristi - of many but not all) performance-based language assessments

is

lhe

pre

sen

ce

of interactive tasks.

In

such cases,

th

e assessments involve l

ea

rne rs

in acttla

ll

y performing the behavior that we want

to

me:lSure. In interac

ti

ve tasks,

test

-ta

kers arc measured in the act of speaking, requesting, responding, or in co m

bining listening and speaking, and in integrating reading and writing. Paper-and

pencil tests certainly do nOI e

li

cit such communicative perform'

lO

ce .

A prime

examp

lc of an

int

e ractive language assessmclll procedure is an oral

intcn 'iew.

TI

le test-taker

is

required to

li

sten accurately

1

someone else and

to

respond appropriately. f care is taken in the test design process, language elicited

and volunteered by the st'Udent

can

be persona

li

zed and mC'J.ningful , and tasks can

approadl

the a

uth

e

nt

icity

of

real-life language use (see Chapter 7) .

CURRENT ISSUES

IN CIASS

RooM TESTING

The design of communicative, performance-based assessment rubrics continues to

challe

ng

e bo

th

assessment experts and classroom teachers. Such efforts to improve

variolls facets of classroom testing are accompanied by some

st

imulating issucs , aU

of which are helping to shape

our

current understand ing

of

effec ti ve assessment.

t et's look at three such issue

s:

the effect of new theo

ri

es of inte

lli

gence on the

testing industry;

th

e advent of what has come to be

ca

lled alternativ assessment;

and the increasing popularity

of comput

er-based tes tin

g.

New

Views o n Intelligence

in te lligence was

once

viewed stric

tl

y as

the

ability

to

perform (a) linguistic and (b)

logical·mathema tica l problem solving. This &Q (inte

lli

gence quo ti

ent)

concept of

intelligence: bas permeated the Western wo rld and

it

s way

of

testing f

or

almost a

cen

tury. Since MmarLness·

in

general

is

measured by timed, discrete-point teSts

co

n

Sisting of a h ierMChy of separate items,why shou ldn't evCJ )' fie ld of sttldy be so mea

sured? For many we have

li

ved in a world of standardized, norm-referenced



2 cw.rnlf

Tes/ing. Assessing.

and eaching

tests that are timed in a multip le-ch

oice

fonnat consiSting of a multiplicity of logic

constr.tined items, many of which are inauthentic .

However, research on intelligence by p t)'chologists like Howard Gardner.

Robert Sternberg, and Daniel Goleman has begun

to

[urn

the

psychometric wo rld

upside

down

. Gardner

1983

, 1999),

fo

r

example

,

extended

the traditional view

of

intelligence to seven different components.

He

accepted

the u-aditional

co

ncep tu

alizations

of

linguistic intelligence and logical·mathematical intelligence on which

standardized IQ tests are based, but he induded five o

ther

"frames of

m i n

in his

theory

of multiple intelligences:

• spatial intelligence the ability to fLOd your way around an enVironment, to

form menta l images of reality)

• musical intclligence

t

he ability to perceive and create pitch and rhythmiC

patterns)

bodily

k

inesrhetic intelligence (fine mot

or

move

ment

, athletic prowess)

• inte rperson:1I intelligence

the

ability to understand others and

how

they

feel ,

and

to

interac t effectively

with them)

• intrapersonal intelligence

the

ability to

lUl

de rsland oneself and

to

develop a

sense of self-identity)

R

aben

Ste

rnb

erg

0988

, 1997)

also charted

new territ

ory

in intelligence re

search in recognizing creative thinking and manipulative su-ategies as

pan

of intel·

Iigen

ce

. All · sman- peop le aren 't necessarily

adept

at fast, reactive thinking. They

may be vcry innovative in being able

to

think beyond

the

normal limits imposed by

existing tests, but they may

need

a good deal of processing time to enact this cre·

ativity. Other fo rms

of

smartness are found in those who know how to manipulate

their environme

nt

, namely,

other people

. Debaters , politicians, successful salesper·

sons, smooth talke rs, and co n artists are all

smart

in their manipulative ability to per

suade others to think

their

way, vote for them , make a purchase, or do something

they might not otherwise do .

More recently, Daniel Goleman'S (1995) co n

ce

pt (emotional quotient)

bas

sp

urred us

to underscore the

importance

of the emotion

s in our cognitive pro

cessing.

Those

who manage their emotions-especially emotions

that

can be detri

mental

t

nd to be more capable of fu lly intelligent processing . Anger, grief,

resentment, se

lf d

oubt , and

other

feel ings can easily impair peak performance in

everyday tasks as well as highcr-order

prob

l

em so

lvi

ng

.

These new conceptualizations of intelligence have

not

been univ

ers

ally

accepted

by the academic co mmunity (see White, 1998, for example). Nevertheless,

th eir intuitive appeal infused the decade of the 1990s with a sense of both freedom

and responsibility

in

our testing agenda. Coupled with parallel educational reforms

at

the

time

Armsuong

, 1994),

the

y

helped to

free us from relying exclusively on

•

I

For a summary of Gardner's theory of intelligence, see Brown 2000

.

pp .

100-102)

.



O«PTER

I

Testing, 'S5eS5inS.

and

Tea

chin

3

timed , discrete-point, analytical tests in measuring language. We were prodded to

cautiously combat the potential tyranny of objectivity"and its accompanying imper

SOllal approach. But

we

also assumed the responsibility for tapping into whole lan

guage skiUs, learning processes. and the ab

ili

ty to negotiate mean ing. Our dlallenge

was

to test interpersonal. creative.

comm

unicative, interactive sk

ill

s,

and

in

do

ing

so

to place

some

trust in our subjectivity

and

intuition.

Traditional

and Alternative

" Assessment

lmpli

ed

in

some of

the

ear

lier description

of

performance-based

dassroom

assess

ment is a trend to supplement traditional test deSigns willi alternatives that are more

a

mb

eoric in th

eir

elicitation

of

meaningful

com

munica

tion

. Table 1. I highlights dif·

f

erences between

the

two approac

hes (adapt

ed

from Armstrong, 1994,

and

Bailey,

1998,

p

207).

Two caveats need to be stated here. First,

the concep

ts in Table

1.1

repre

se

nt

some

ove rgeneralizations

and

should therefore

be

considered with

ca

ution.

It is dif-

ficult, in fact,

0

dr.tw a clear line

of

distincti

on

between what Armstrong ( 1

994) and

BaiJcy (1998) have ca

U<. d

traditional and alternative asseSSmenl. Many forms

of

assessment fa

ll n betwec

n the

two

,

and some

combine the

best of bo

th .

Second, it

is obv

ious that

the

table shows a bias toward alternativc assessm

ent

,

and

one

should not

be

mi sled into thinking that everything

on the

left-h

and

si

de is

tainted while

the

list on the righr-hand side offers salvation to

lhe fi

eld

of

language

assessment

As

Brown

and

Hudson (1998) ap tly

po

inted

Ollt

.

the

assessment tradi

ti

ons

av:

liI

able to us should

be

valued

and

utilized for the fu nctions that th

ey

pro

vide.

At

the same time,

we

might all

be

stimulated to l

oo

k at

the

right-

hand

list

and

ask

ourse

lves if,

among

th

ose

concepts. there

are

alternatives

to

assessment that

we

can

construct

ively use n our classrooms.

It should be n

oted

here thal

cons

id

erab

ly more time and higher institutional

budgelS

are

required to administer and

score

assessments Illat

pre

supp

ose

more

Table 1.

, . Traditional

and alternative assessment

Traditiona l Assessment

One-shot , standardized exams

Timed, multiple-choi ce form al

Decontextualized test items

Scores suffice for feedba ck

Norm-referenced scores

focus on the "right answer

Su

mmative

Oriented to product

Non-interactive performance

Fosters extrinsic motivation

Alternative Assessment

Co

ntinuous ong-term assessment

Untimed, free-respon

se

fo rmat

Contextu

ali

zed commun icative

ta

sks

Ind ivi

dualized feedback and washback

Criterion-referenced scores

Open-ended, crea

ti

ve answers

Formative

Orie

nt

ed

to

process

Interactive performance

Fosters

int rinsic motivat ion



4 CH PTffl I T

esting

Assessing and Te

ching

subjective evaluation,more individualization, and morc interaction

in

the process of

offering feedback . The: payoff for the latter.

how

ever. co mes

with

more useful feed

back to students , the potcntial Jor intrinsic motivation, and ultimately a more

complete descript

i

on of

a

student

's abil ity. See Chapler 10 or a

complete

[fCatment

of

alternatives

in

assessment.

ore

and

more

educators :lnd adVOC ttes

for

educa-

tional reform arc

arguing

for a de-emphasis on large-scale

standardized

te stS in favor

of buildi

ng

budgets that will offer the kind of comcxtlmlizcd, communicative

performance-based assessment that

w

ill bener facitit:

ue

learning in our schools. On

Chapter 4 , i

ssues

surrounding s t

andard

i

zed testing

are

addressed

at

length

.)

compute r-Ba

se

d

Testing

Recen t years have seen a burgeoning of assess

mcm

in wh ich the lCS t-t

ake

r performs

respon

ses

on a

co

m puter. Some

co

mp ut er-bascd rests (a l

so

know

n as "

co

mp Ul er

ass i

sted

" Or

w e b b s e d

tests) are sma

ll

·

scale

"

home

-

grow

n"

tes

ts ava

il

lible

on

we

I -

si tes . Others arc

standardjzed

, largc-scale

tests

in which thousands

Or

even t

ens

of

thousands

of

test-t'akers are invo h'

cd

. S

tudents

receive pro

mpts or

probes ,

as they

are sometimes

referred

to) in thc form of

spoken

or wriHen stimu li from ti

le

com

put

e rized test

and

are requ ired to

type

or in some

cases

, spcak) their responses .

A1m

os t a ll

comp

uter-based tcst items have fIXed , closed-end

ed

responscs;

however

,

tests like the Test of English as a Foreign Language

fOEFL offer

a written essay

sec tion

that

must be scored

by

hUUlans

as

opposed to

automatic

, e lectronic, o r

machine scoring).

As

this

book goes to press , the deSigners of the TOEFL are on the

verge of offering a spoken Eng li

sh

section .

A

specific

type

of

computer-based tcst , a com

pu

ter -ada pti

ve

test, has been

available for

man

y yea

rs but has

recently

gainel1

momentum

.

In

a

computer-adaptive

lest CAn ,

each test-taker

receives

a

se

t of

questions

that

meet the test

specifica·

tions

and

that arc gencraJJy

appropriate

for h is

or her

performance levd . Th e CAT

starts

with questions o f moderate difficul

ry

.As test-

taker

s answer

eadl

question, the

computer

scores

th e

questi

o n and u

ses

that

inf

ormation , as we

ll

as tllC

responses

to

previo

us questions,

to

d

ete

rmine w hidl question will be presented next. As long as

ex.1minees respond correctly, the compu t

er

typica

ll

y

se

lectS quest jons of grea te r or

equal d ifficulty. Incor rect :m

swers,

h

owever

, lypically bring questions

of

l

esser or

equa l difficul ty. Tht comp uter is programmed to fu

lfi

ll th e tcst

desig

n as it co

ntinu

ously adjusts to fi nd quest ions of approp riate d ifficulty for test-takers

at

a

ll perfo

r

man

ce

levels. In CATs,

the

test-take.r

sees

on

ly

one ques

tion

at a

time

,

an

d

the

computer scores e

ach qu es

tion

before

selecti ng

(h

e next one . As a result , tes t-takers

cannot sk ip questions,

and

once tllCY have

entered and

confirmed thei r answers,

they

cannot rerum to questions or

to

any

earJjer part of the

test

.

Computer-based esting,with

or

witlIom CAT te

dlllology

,o

fft tS these

advantages:

cI:lSsroom·bascd

te

s ting

se

lf ·directed t

esting

o n V". .rious aspects of a l

anguage

(vocabu l

ary

,

gramma

r,

discourse , onc or all of the four skills,

et c

.)



cm.I TlIl

I Tesfing,

Asse55ing

nd

Teil

ch i

ng 5

• pr.tcticc for upcoming high-stakcs standardi

zed

tests

•

some

ind

iv

idualizati

on

, in the

case of

CATs

• large-scale stan

dard

i

zed

tests thai can be

administered

casil)' to thousands of

tcs t-takers

at

many different stations, then scored electronica lly for rapid

re po

nin

g

of

results

Of

co

urse , some disad\rantagcs are presem in

our

currenl predile(:lion for com

pu t

erizing test

ing_Among th

em:

• Lack

of se

c urity

and

the

poss

ib

ili

ty of cheating are inhe rent

in

c1a

ssroo

m

based,

unsupervised

compute ri

ze

d tests.

Occasional

o m · r o w Qui

zzes

that appear on

unofficial

websites

may

be

mistaken for valida ted assessmen ts.

• Th e mu ltiple-choi ce forma t p referred for most computer-based

tests

comains

the usual

po t

ential fo r flawed it

em

de

sign

see

Ch

ap

t

er

3).

•

Ope

n e

nd

ed responses are less likely to appear because

of

lhe need for

human scorers, with all the attendant issues of

COS

t, reliability, lind tu

rn

a

round

time.

• ' c human interactive clement

es

pecia

ll

y in oral production) is

abse

nt _

More is said

abo

ut computer-based t

esting

in sub

sequent

ch

aplers

, especially

Chap ter 4, in a discussion of large·scale standardized testin g. In add itio n, the fol-

low

ing webs i[t"s

provide

further informatio n

and

examples o f computer·based tests:

Educational Testing Service

T

es

t

of

English

as

3.

Foreign

Language

Tcst of English for International Communication

lntem

:nio

naJ

English Language T

est

ing System

Dave'S ESL Cafe (computerized quizzes)

www .ets.org

www .tocfl.org

www

.todc .com

www .ie1ts.o rg

www.e slcafe

.co

m

Some argue th at computer-based testing, pushed to its ul timate level,

might

mil

i

gate

agai nst recent c fforts to rerurn t

csting

0 its artful form of be ing tailored

by

teach

ers

f

or

th

ei

r dassrooms,

of

being

de

signed [ be

perfor

mance-ba

se

d ,

and

of

a

llOW

ing a teache r- s

tud

ent d ialogue to f

or

m th e basis o f assessmen

t.

This ne

ed

no t

be the C

ll

se. Co mplllcr tcchnolo .')' am bc a boon to co mmun icative language

testing.Tead1ers

and

test-m

ake

rs

of

the

fu ture w

ilJ

have

access

to an

ever

-i

ncreasing

rdnge of too ls

10

safeguard against impcrson

aJ

, stamped-om fo

rmul

as fo r aSsessment.

By using t

ed

lOoJogical innovations creatively, testers will be able

to en

hance authen·

ticity,

10

in

cre

ase interactive c.'Xchange,

and

to promote

aut

o no my.

As rou TCad this book , I hope you

\ \

'

ilI

do so w

ith

an

appreciat

ion for the place

of testing in assessment , a

nd

w

ith

a s

ense of

the

in t

er

conn

ection

of

assessment

and



16

H m

ll

I

Testing

Assessing C nd T

eClching

teaching

. Assessment is an

int

egral part of

the teaching-learning cyde

. In an inter

active,

co

mmunicative c urriculum, assessment is alm ost constant. Tests, w hich are a

subset of assessment , can provide authenticity, motiV"dtion, and feedback to

the

l

earner

. Tests are essential

components

of a successful

cu

rriculum and one of sev

eral

partners

in

the

learning process. Keep

in

mind th

ese

basic p rinciples:

1. o d i assessments, both formal and informal, can in

crease

motivation by

se rving as milestones of srudent progress.

2.

Appropr

iate assessments aid in [he reinforcement and retention

of

informa

lion .

3. Assessments

can

co nfirm areas of strength and

pinpoint

areas n

eed

ing further

wo

rk.

4. Assessments can provide

a

sense

of

periodiC closure to modules

with

in

a

cur

ri

c ulum .

5

Assessments

ca

n

promote

st

udent

au

tono

my

by

en

co

ura ging s

tudents

' se lf

evruu3tion of their progre

ss

.

6 Assessments can spur learners to set goals fo r themselves,

7. Assessments can aid in evaluating

teadling

effectiveness.

Answers to the vocabulary quiz

on

pages 1 and 2: l c, 2a, 3d,

4b

, Sa, 6c.

EXERCISES

[Note: I) lndividual

work

; (G) Group or

pair work

; C) Whole-class d iscussion .)

1. G) In

a

smaU group , look at Figure 1.1 on page 5 that shows t

es

ts as a subset

of

assessment

and

the

laner as a s

ubset

of teaching.

Do yOll agree with

this

diagrammatic dep iction of th e

three

terms? Co

nsider

the following classroom

tcaching techniques : choral drill,

pair

pronunciation practice, reading aloud ,

infomlation gap task, Singing songs in English , wri ting a description of ule

wee ken d 's activities.What proportion of ellch has an assessment face t to il?

Share your

co

nclusions with the resl

of

the class.

2. G) TIle c

hart

below shows a hypo thetical tine of distinction between fonna

tivc

and summa

tive assessme

nt

.

and betwecn

informal and forma assessm

ent.

As a group , place

the

foUowing techniques

/proced

ures into one of the four

ce

Us and justify your decision. Sh

are

your results with othe r groups and

dis

Cll

SS any differences

of opin

i

on

.

Pl

ace

ment tests

Di

agnostic

te

st

Period ic achievemem tests

Short po p qujzzes



Standanlized proficiency

lestS

Final exams

Portfolios

CWol ml T

es

t

ing

Mres5ing and Teiiching

7

Journals

Speeches prepared

and rehearsed)

Oral presentatiOns

prepared

, but not rehearsed)

Impromptu student responses to teacher s questions

Student-written response one paragrnph) to a reading assignment

Drafting

and

revising writing

Final essays (after

severa

l

drafts)

Student oral responses

to tcache

r questions after a videotaped lecture

hole

class open ended discussion

of

a

topi

c

formative Summative

Informal

Form

al

3.

l ie)

Review

th

e distinction

be twee

n

norm

·referenced and cril crion

referenced testing. If norm-referenced tests typica lly yie

ld

a distribution

of

scores that resemble a beU

s

h

aped

curve,

what

ki

nds

of

distributions are

typical

of

classroom ac

ru

evement tests in your experience?

4. l ie) Restate i.n

your

own words lhe argumcill betwecn unilary trai t propo

nems

and discrete-point testing advocates.Why did O

Uer

back down from

the

unitary trait hypothesis?

5. eve Why are doze and dictation considered to be integra live tests?

6.

G) Look

at the

lisl

of

Gardner s seven intelligenc

es

. Take

one or

t

wo

intelli

gences, as assigned

to

your group, and brainstorm some teaching activities

that foster

thai

type of intelligence.

Then

, bminstorm some assessment tasks



8

HIoPTrR 1 Testing. Assessing and Teach ing

that may

presuppose the

sa

me

intelligence

n order

to pe

rf

o rm

we

ll . Sha

re

yo

ur

resulls

with

othe r groupS.

7. C) As

a

wh

ole-c.lass discussi

on

, brain

storm

a variery of test tasks

that

class

members

have e.xperien

ce

d

n

learning a foreign language.

Then decide

whi<:h of

those

tasks are performance-based ,

which are

nO

t, and

which

ones

fa n between

.

8. G)

Table

1.1

lists traditional and

alternat

ive assessment tasks a

nd

characteris

tics. ln pair

s

quickly review

the

advantages and disad\ antages of

cad

}, o n

both

sides of the dun Share

your

conclusi

ons

with

the

res t

of the

class.

9. C)

As

k

class

members

to share any

expe

ri

ences with

co

mputer

-based t

es

ting

and eva luate the adv-dntages and disadvantages

of

th

ose experie

nces.

FOR YOUR FURTHER READING

McNamara,

Tim.

(2000). Latlgllage testi

ng.

Oxford:

Oxf

ord

Un

iversity Press.

One of a number

of

Oxford University Press s brief introductions to various

of language study this 140-page primer

on

testing offers definitions

of

basic t

er

ms

in

language testing with brief explanations

of

fundamental

concepts. It

i s a

useful

little

reference

boo

k to check your understand ing

of

testing jargon and issues

in

the field.

Mousavi, Seyyed Abbas. 2002). An

e lcyclopedlc dictlo

1la

ry of language testing.

111ird Edition. Taipei: Tung Bua Book Co

mpany.

111

i5 publica

li

on may

e

d ifficult to find in loca l lx>okslores, but

it

is a

highly useful

co

mpilation of Virtually every term in the field

of

language

testi ng, w i

th

definitions, background

hi

story. and research references. It

provides comprehensive explanations

of

theories, principles, issues, tools,

and

ta

sk

s. It

s exhaustive BS-page b

ib

liography is al

so

down loadable at

http:/ /www.abbas

·

mousavi.com

. A shorter

ve

rsion of this 942-page lome

may

e fOUlld in

the previous version, Mousavi s (1999) Di

ctiollary

of

lan-

g age les ing

(Tehran: Rahnama Publicat

ions)

language assessment principles and classroom pratices

Documents