toefl communicative competence via reading.pdf

8/14/2019 TOEFL Communicative Competence via Reading.pdf

http://slidepdf.com/reader/full/toefl-communicative-competence-via-readingpdf 1/25

Lonograph

Series s M E S p ~ r~ M E R 1 9 9 6

s s e s s in g S e c o n d L a n g u a g e

A c a d e m ic R e a d in g f ro m a

C o m m u n ic a t iv e C o m p e t e nc e

P e r s p e c t iv e : Re le v a n c e f o r

TOEFL 2

Thom udson

d u c a t i o n a l

Testing ervice



ssess i n g Seco n d L an g u ag e cad em i c Read in g f ro m a

C o m m u n i c a t i v e C o m p e t e n c e P e r s p e c t i v e : R e l e v a n c e fo r T O EF L 2

Thom Hudson

Educational Testing ServicePr inceton New Jersey

RM-96-6



Educa tional Tes t ing Service is an Equal O pportunity /Aff irmative Act ion Employer

Cop yright © 1996 by Educational Testing Service. All rights reserved.

No part of this report may be reproduced or transmitted in any form or by any means

electronic or mechan ical including photocopy recording or any information storage

and retrieval system without permission in writing from the publisher. V iolators will

be prosecuted in accordance with both U.S. and international copyright laws.

EDUC ATION AL TESTIN G SERVICE ETS the ETS logo TOEFL the TOEFL logo

and TSE are registered trademarks of Educational Testing Service.



bs t r ac tll

This paper examines i ssues involved in the assessment of academic reading f rom a com mun ica t ive

prof ic iency perspec t ive , par t icula r ly these i ssues as they a re involved in the co ntext of the Test of Engl ish

as a Fore ign Language TO EFL 2000 projec t . The f i r s t par t of the paper presents the a reas tha t concernthe assessm ent of academic reading abi l i ty by br ie f ly examining not ions re levant to comm unica t ive

comp etence and how these not ions migh t re la te to academic reading. This discusses the ways in which

m ode l s o f c om m un ic a ti ve c om pe te nc e ha ve b r oa de ne d the v i e w o f wha t knowle dge s a r e ne c e ssa r y i n

order to use a language and language ski l l a reas such as academic reading. The paper notes tha t the

f ramewo rks tha t have evolved have inc luded such a reas as grammat ica l competence , organiza t iona l

competence , i l locut ionary comp etence , and pragmat ic competence . Such views of competence and

per formance a re impo r tant in language assessment in tha t in addi t ion to broadening views o f language

and language abi l ity , they of fe r some mean s for expla ining the extent to wh ich a person mig ht vary in

language per formance across tasks or contexts . The s t ress in commun ica t ive competence perspec t ives on

language use re f lec ts an emphasis on the impo r tance of viewing language in context .

' Ihe pape r employs a broad view of academic reading tha t inc ludes key aspec ts of

(a ) autom at ic i ty in word and sentence process ing, (b) content and formal background kno wledge ,

(c ) s t ra tegic and metacogni t ive ski l l appl ica t ion, and (d) reading purpose and context . By ack nowledg ing

the imp or tant role played by each o f these aspec ts , the paper proposes tha t ~ tu re assessment take the

ecology of academic reading into account . Insh or t , r eading s imul taneou sly involves both psycho logica l

and soc iologica l processes .

The pa per conc ludes wi th im pl ica t ions for academic reading assessm ent , paying par t icula r a t tent ion

to the four va l idi ty compo nents of (a ) const ruc t va lidi ty , (b) va lue im pl ica t ions , ( c ) re levance /ut il i ty , and

(d) soc ia l consequences . The im pl ica t ions involve the potent ia l needs to (a ) expand beyon d the mul t iple -

choice or o ther se lec ted- response formats , (b) incorpora te so me m ul t iple -choice or other se lec ted-

response form ats in order to ba lance genera l and context - speci f ic tasks , ( c ) view reading assessment as , a tleas t in par t, t ask based, (d) involve them at ica l ly based texts and dev e lop ad apt ive types o f tes t s , e i ther

via com puter or through se lec t ion o f texts and tasks by the examinee , ( e ) view score repor ting and

interpretat ion as ref lecting real-w orld acad emic read ing tasks, and (If) integrate readin g asses sme nt with

other language ski l ls based on the l i te racy task tha t is be ing encountered.



T a b l e o f o n t e n t s

age

Introduction 1

Communicative Competence and Reading 2

Review of the Literature on the Construct of Academic Reading 3

Models of the L1 and L2 Reading Processes 4

Implications for the Assess ment of Academic Reading 9

Conclusion 13

References 14



n t r oduc t i on

This paper focuses on i ssues involved in the assessm ent of academic reading f rom a comm unica t ive

prof ic iency perspec t ive in the context of la rge-sca le assessment for academic se lect ion. Spec i f ical ly , i t

addresses these i ssues for the context of the TO EFL 2000 projec t. The f i r s t par t of this paper wi l l presentthe a reas tha t a re of concern in the assessm ent of academic reading abi l i ty . This wi l l involve a br ie f

examina t ion o f ideas re levant to com munica t ive com petence and a discuss ion of how these i ssues m ight

re la te to academ ic reading. I ssues re levant to def ining the co nst ruc t of academic reading wi l l then be

discussed, concent ra ting on aspec ts involved in com mun ica t ive ly assess ing academic reading. Then, the

impl ica t ions o f reading ass essmen t discuss ion wi l l be explored, and the po ss ibi l it ies for integra ting the

var ious language ski l ls wi l l be discussed.

I~e approach taken here invo lves broadening the view of academic reading to inc lude key aspec ts of

(a ) automat ic i ty in word and sentence process ing, (b) content and formal background know ledge ,

(c ) s t ra tegic and metacogni t ive ski l l appl ica t ion, and (d) reading purpose context . By acknow ledging the

impor tant role played by each o f these aspec ts , the pape r proposes tha t future assessm ent take the eco logy

of academic reading into account . In short , r eading s imul taneously involves both psychologica l and

soc iologica l processes . In ord er for assessment to mo re f i f lly re f lec t the co nst ruc t of academic reading, i t

wi l l need to address the ful l r ange o f appl ied reading processes .



omm unicative om petence and Reading

Over the pas t severa l years , t he l anguage ana lys i s and as sessment com mu ni ty has increas ing ly

emphas ized the ro l e o f comm unic t ive competence and the appropr i a te me ans fo r it s assessment . Th i semp has i s has deve loped f rom theor i es o f l anguage tha t v i ew l anguage use a s a p r ima r y c o mp o n e n t t h a t

mu s t be addressed in s tud ies o f l anguage ana lys i s , acqu i s it i on , and educa t ion . C omm unica t ive

competence was p roposed by Hymes (1972) and Campbel l and Wales (1970) as an a l t emat ive to the

s t rong v iew pos i t ed by C hom sky tha t l ingu i s t ic com petence was l imi t ed to knowledge o f g rammat ica l

ru les . Com mun ica t ive com petence perspec t ives , i n con tras t , v i ew the ro l e p l ayed by con tex t o f d iscourse

beyon d sen ten ti a l cons t ruc tions as es sen ti a l t o any unders t and ing o f competence and per fo rmance . Thus ,

mo d e l s o f c o mmu n i c a t i v e c o m p e t e n c e h a v e b r o a d e ne d t h e v i e w o f w h a t k n o w l e d g e s a r e n e c e s s a ry i n

order to use a l anguage (Bachm an Sav ignon , 1986). Curren t mod el s inc lude know ledge o f l anguage

funct ions and know ledge o f l anguage con tex t s , as wel l as know ledge o f g rammar . The f ramew orks tha t

have evo lved have inc luded g rammat ica l competence , soc io l ingu i s t i c competence , and s t ra t eg ic

competence (Canale Swain , 1980), o r g ramm at ica l competence and t ex tua l competence , o rgan izedunder a rubri c o f o rgan iza t iona l compe tence w i th iUocu t ionary competence and soc io l ingu i s ti c

c o mp e t e n c e u n d e r p r a g ma t i c c o mp e t e n c e ( Ba c h ma n , 1 99 0) . T h e Ba c h m a n v i e w o f c omp e t e n c e a n d

per fo rmance i s impor tan t in l anguage as sessment in tha t i t o f fe rs some mech an i sm fo r exp la in ing the

ex ten t to which an ind iv id ua l ' s l anguage per fo rmance mig h t vary across t asks . In add i tion to the t ra it

fac to rs o f l i ngu is t i c competence , p ragm at i c competence , and s t ra t eg ic competence , t he Bach ma n mode l

adds sk i l l fac to rs ( such as psychophys io log ica l m echan i sms , mo de , and channel ) as wel l as method

fac to rs ( such as the l anguag e-use s i tua t ion , t he am ount o f con tex t , the d i s t ribu t ion o f in fo rmat ion , t he

type o f in fo rmat ion , and the response mode) . Skehen (1991) po in t s ou t tha t i nc lud ing these fac to rs m ay

help address the competence versus per fo rmance i s sue . Bas ic competences a re concerned wi th genera l

language abi l i t ies , whi le ski l l and method relate the general abi l i t ies to real -world language performance.

The s t res s in com mu nica t ive competence perspec t ives o n l anguage use re f l ec t s an emphas i s on theimpor tance o f v i ewing l anguage in con tex t . As S av ignon (1991) has no ted , comm unica t ive l anguage

teach ing embraces bo th the goa l s and the p rocesses o f l earn ing , t hus p rov id ing a focus on soc ia l

in t e rac tion and language acqu i s it i on . Th i s v iew i s cons i st en t wi th Gre en 's (1987) observa t ions abou t

how read ing has t rad i t iona l ly been addressed . She po in t s ou t tha t mos t p r io r v i ews o f read ing have e i ther

seen reading as an objec t o f ins t ruc t ion o r as a med iu m fo r ins t ruct ion in other curricular areas . Such a

dichotomizat ion has led to a focus in reading ins t ruct ion and assessment on the ski l l s , s t rategies , and

processes ind iv idua l readers acqu i re and has t ended to decon tex tua l i ze the ac t iv i t ies o f read ing and

l i te racy f rom the soc ia l and ins truc t iona l con tex t in which they a re embed ded . Th i s perspec tive o f

read ing and com prehens ion as ac t ive and in te rac t ive i s cons i s t en t wi th the no t ion tha t com prehens ion

m ode ls need to incorporate bo th rules and s trategies (van Di jk , 1985). St rategies are appl ied to rules in

o rder to reach an adequate t ex t i n t e rp re ta t ion , and s t ra teg ies a re def ined , i n par t , i n te rms o f the reader ' s

purpose and de te rmined nee d . Thus , t he end goa l o f the read ing p rocess wi l l de t e rmine wha t s t ra t eg ies

are app l ied to w hich ru les in o rder to de te rmine such re f l ec tions o f ab i li t y as dep th o f p rocess ing o r t ime

spen t in read ing . Curren t research has com e to v i ew ab i l i ty as var iab le and h igh ly dependen t up on

con tex t and purpose (Sav ignon , 1991).



These are imp or tant is sues for l anguage assessment . As Canale and Sw ain (1980) noted ,

com m un i ca t ive t e st i ng . . . m us t be devo t ed no t on l y t o wha t t he lea rne r know s abou t t he s econdlanguage and abo ut how to use i t [com petence] but a l so to wh at ex tent the l earner i s ab le to actual ly

demo ns t ra te th i s know ledge in a me aningf id com mun icat ive s ituat ion (p . 32) . ' Ihus , in expanding the

view s of reading to incorporate such concepts as process , goal , context, and v ar iab i l i ty of ab il i ty , i t ma y

be that we expand the appl icat ion of reading assessm ent to l i teracy in academ ic se t t ings , ra ther than

res t r ic t ing reading assessmen t to t rad i t ional concerns wi th au tom at ic i ty of reading sk i ll s . This v iew of

reading as academ ic l i teracy may prov ide valuable informat ion for se lection deci s ions a t academic

ins t itu t ions , where w hat a potent ia l candidate can do wi th the l anguage i s as impo r tant as that candidates '

unde r ly ing competence. Likew ise , f rom a com mun icat ive competence f ramew ork , it i s impor tant to

recognize that a candidate should be a l low ed to dem ons t ra te the ab i l i ty to a pply reading sk i ll s to a t ask in

purposefu l sociocul tural context .

R e v i e w o f t h e L i te r a tu r e on t h e C o n s t ru c t o f c a d e m i c R e a d i n ~

Defin ing prof ic ien t reading in the academ ic context is a d i ff i cu l t endeavor . One o f the problems w e

have w i th reading research and model bui ld ing i s that s ince the com ponen t processes are unobservable ,

they are pos tu la ted cons t ruct s . These processes are inferred f rom the resu l t s of behaviors . An a ddi t ional

problem wi th bui ld ing reading mo dels i s that the mo dels that emerge are a lways products o f h i s tory ,

l imi ted by the theor ies and ins t rumen ts avai lab le . Given th i s h i s tor ical backgrou nd in which models

operate , f ramew orks chang e re la t ively rap id ly , so i t i s impor tant to main ta in a heal thy a theoret ic

scept ic ism. Ov er several decades , how ever , a general consensus ( though cer ta in ly not unanimi ty) about

several is sues surround ing reading has em erged.

There i s general consensus th at reading invo lves the in teract ion o f a vas t ar ray of processes ,know ledges , and abi l it i es. These include bas ic decoding processes such as graphe me recogni t ion , lex ical

access , phonological representa t ion , and l inguis t i c s t ructure process ing , as w el l as h igher order cogn i t ive

processes su ch as the app l icat ion of back ground kn owled ge, process ing s t ra teg ies , t ex t s t ructure

unders tanding , and som e aspect s of vocab ulary knowledge. Reading a l so involves in teractional

processes such as the appl icat ion of evaluat ive sk i l l s , use o f metaco gni t ive know ledge, and sel f -

mon i tor ing . How ever , a majo r i ssue in reading l it era ture re la tes to the speci f ic re la t ionships betwee n

these comp onents . As w i l l be explained in mo re deta i l, some researchers have advocated a bot tom-up

approach , o thers a top-do wn approach , and o thers an in teract ive approach to explanat ions o f reading

success and fa i lure . Ear ly theor ies v iew ed reading as a bot tom-u p process in which the reader cons t ructs

me aning in a sequent ia l ma nne r f rorn le t ter , words , and sentences (Gough, 1972; LaB erge & Samuels ,

1974). Other theor ies of reading s t ressed that the ef f ic ien t reader ma kes the fe wes t t ex t process ings ,

s ince t ha t r eade r p red ic t s t he m ean i ng o f t he t ex t by app l y i ng know l edge o f the wor l d and l anguage

(Good man, 1967; Smi th , 1971) . Mo re recent v iew s acknowledge a st rong in terp lay betwe en both of

these processes .

Current theor ies em phas ize the in teract ive nature of reading and accept the fact that good readers

mak e extens ive use of pr in ted informat ion . In the in teract ive approach , som e have argued in favor of an

in teract ive sys tem s t rongly cons t ra ined by bot tom-up processes , and have consequent ly emphas ized

l inguist ic proc essing (Grabe , 1991; Perfet t i , 1992; Ra yn er & Pollatsek, 198 9; Stanov ich, 1990). Othe rs



have t ended to p lace l es s emphas i s on the bo t tom-up l ingu i st i c compon en t s and m ore em phas i s on the

ro le o f top -dow n process ing (Henk & Hel fe ld t, 1987 ; Hi l l & Par ry , 1992 ; Hud son , 1991 ; Johns ton , 1984 ;Rum elhar t , 1977 ; Valen c ia & Sta l lman , 1989) . W hich em phas i s is g iven seems o f t en to be in f luenced by

how the researchers address ~ exp l i c i tly o r impl i c i t ly ~ the four i s sues o f (a ) how un i fo rm the read ing

process i s wi th in any par t i cu la r ind iv idua l read ing across con tex t s ; (b ) the impo r tance o f background and

cu l tu re in read ing and l earn ing to read ; (c ) the ex ten t to w hich read ing sk i ll s a re impl i ca t iona l ly o rdered

in the i r acqu i s i tion and a pp l i ca t ion as opposed to be ing b road ly over l app ing an d com pensa to ry in na tu re ;

and (d ) the ex ten t to w hich a n ind iv id tml may avo id c lose l ingu i st i c p rocess ing and s t il l comprehend a

message . These i s sues wi l l con t inua l ly a r i se in the d i scuss ion tha t fo l lows . Essen t i a l ly , t he d i f fe rences

in emphas i s re l at e to wheth er the par t i cu la r researchers a dop t a p r imar i ly psycho l ingu i s t i c o r a p r imar i ly

sociol inguis t ic orientat ion. Add i t ional ly , as Grabe (199 1) points out , not al l d iscussion s of interactive

approaches use the t e rm in the same way . Som e use the t e rm to rep resen t an in te rac t ion be tween the

reader and the t ex t , wh i l e o thers use the t e rm to rep resen t an in te rac t ion be tween the d i f fe ren t sk il l s, t ha t

i s , as the in te rac t ion o f the en t i re w eb o f cogn it ive as se t s the reader has a t h i s o r her d i sposa l. Th i s paper ,

in tak ing a v iew tha t con tex t i s ex t rem ely impor tan t , u ses the t e rm interaction t o inc lude bo th o f the

above uses .

In genera l , then , success o r fa i lu re in read ing per fo rm ance can be addressed in t e rms o f the

in te rac tions be tween the rea der ' s (a ) automaticity , the ex ten t to which the per fo rman ce o f p rocedures no

longer requ i res l a rge am ount s o f a t t en tion ; (b ) content and form al schemata, the reader ' s men ta l

representat ions of facts and ski l ls ; (c) strategies an d metacog nitive skills , the reade r ' s s t ra t eg ies fo r

mon i to r ing the se lec t ion and app l i ca t ion o f ac t ions ; (d ) purpose , the goal s t r iven fo r by the reader ; and

e) context, the i n t e rac tiona l env i ronm ent in which the read ing ac t iv i ty t akes p lace . Thus , regard less o f

the approach to read ing per fo rman ce , there i s a need to ind ica te ho w the in te rac t ive p rocesses in read ing

invo lve bo th the under ly ing cogn i t ive p rocess ing and the purpose o r con tex tua l aspec t s o f read ing . In thef i r s t p rocess , t he reader in t e rac t s d i rec t ly wi th the t ex t to ga in m ean ing ; in the second , the read er p roduces

some ar t i fac t . These a re no t separa te en t i ti es in tha t they a re bo th opera t ing a t the sam e time. ' Ihe y

s imply re f l ec t d i ffe ren t perspec t ives emp loyed to exp la in the ac t iv i ty o f what an ind iv idua l does in the

read ing p rocess . One perspec t ive focuses on sk i l l app l i ca t ion whi l e the o ther focuses on the produc t o f

the apph ca t ion o f the sk i l l . I t is the in te rac t ion be twee n these two perspec t ives tha t p rov ides par t o f the

bas i s fo r a change in as sessme nt p rocedures . A recogn i t ion tha t read ing assessmen t invo lves a ba lance

between the tw o perspec t ives h as l ed , in part , to the in te res t in comm unica t ive ly o r i en ted assessment .

Models of the L and L2 Rea dina Processes

As no ted above , read ing mode l s have changed over the years . Curren t ly , however , researchers

genera l ly ag ree tha t read ing i s in t e rac t ive and compe nsa to ry (Stanov ich , 1980). As w i th language in

genera l , read ing i s no t the sum of sk i l ls , t ha t a re iden t i fi ed w i th read ing , such as sk im ming , scann ing ,

vocabu lary iden t i f ica t ion , and read ing fo r main idea . W hat seems reasonab ly c l ear f rom the research i s

tha t sk i l ls a re d if f i cu lt t o def ine in p rac ti ce (Snow & L ohm an , 1993 ; Alderson , 1990), and when they a re

iden t if i ed they a re no t o rdered imphcat iona l ly (Hudson , 1993). Rather , sk i l ls appear to cover wide bands

of over l app ing ab i l it ies . These ab i l it i es range f rom loca l t ex t recogn i t ion and p rocess ing to b roade r t ex t

interpretat ion and use s t rategies . The fol low ing discussion w il l presen t the abi l i t ies in f i rs t - and second -

language read ing tha t migh t be o f impor tance in as sess ing academic l anguage p ro f i c i ency .



Autom at ic i ty in W ord and Sen tence Recognit ion Current research em phasizes reading

com prehens ion's dependence on eff icient lexical access (Pollatsek & Rayner, 1990). Wo rd recognitionand decoding abilit ies have been shown to have strong relationships to f irst- language reading

comprehen sion (Ctmningham, Stanovich, & Wilson, 1990). Much o f this developm ent in the various

reading models has come from research that has show n that words, sentences, and other strings of words

that are processed in isolation are processed more slow ly and remembered w ith less accuracy than those

processed in context. The research has show n that, in general , context of reading and processing plays an

important role in reading comprehension and mem ory. Balota (1990), for example, has reported that

readers recognize words and str ings that are primed b y related words faster than th ey recognize words

primed by unrelated words. For example, when subjects were presented with the pairs of words dog -

cat versus pen - cat, cat wa s recognized faster in the first instance than in the second. Add itionally,

some of the features of f luent reading are manifested in the t ime readers spend looking at , or f ixating on,

particular words (Beck & C arpenter, 1986; Just & Carpenter , 1987). M uch o f this research observed thefrequency and duration of eye f ixations during reading. 'Ibis research showed that a large proportion of

the wo rds in a text are agended to by the reader, a much larger proportion than would be predicted by

strong top-dow n advocates (Just & Carpenter, 1987; Pollatsek & Rayner, 1990). I t is clear, then, that

automatic and eff icient word recognition is required for most skil led reading (Beck & Carpenter, 1986).

In academ ic contexts where large amounts o f text mu st be processed, this eff iciency in reading mus t be

considered to be o f extreme importance. Likewise, eff icient syntactic processing and vocabu lary are

important for eff icient academic reading. Rayn er (1990) points out the imp ortant interaction between

syntax and lexicon in reading, and Cunningham, Stanovieh, and Wilson (1990) demonstrate high

correlations between reading and vocabulary. Hayn es and C arr (1990) sh ow a strong relationship

between gram matical proficiency and reading speed. They claim that this indicates a large syntactic

proce ssing com pone nt in reading. Thus, the lower-level linguistic skills are essentially enabling abilities

for much of academic reading.

Given the important role o f eff icient lexical and syntactic processing in reading comprehension, i t is

important for method s o f assessing academ ic reading to represent lexical and syntactic levels appropriate

to academic texts. Further, readers who lack sufficient vocabulary and/or eff icient automatic text

processing abil i t ies are l ikely to have po orer chances for academic success than those with a high degree

of ability.

Co n tex t a n d F o rma l S ch ema Th e role of background knowledge in successful academic reading

involves the application of con tent schem ata related to the topic o f the passag e as well as the application

of formal knowledge of how texts are organized.

Co n ten t S ch ema Th e effect of prior knowledge on comprehension involves the theoretical notion of

schema As noted by M arshall (1993), schem a theories have tended to focus either on schem ata as

abstract mental structures that serve as framew orks with slots that ma y be f i lled by incoming text

information (Beck & McK eown, 1989), or as the actual knowledge o f a concept that is contained in a

schem a (Crick & Ho lyoak, 1983 ). How ever, Marshall notes the need to incorporate both an architectural

point of view o f schema as well as a content point of view. Sharkey (1990) approaches schemata as

mech anisms for collecting and representing norms, and thus presents an unm arked script that is helpful

but not sufficient for comprehension. Specif ically, he notes that scripts may be underspecif ied. I t is clear



I I I I I I I I

t ha t , regard les s o f how the t e rm i s used , read ing i s no t accura te ly rep resen ted in the t rad i tiona l

encod ing /decod ing m odel o f com mun ica t ion rep resen ted be low (Sperber & W i l son , 1986) :

me s s a g e - - > e n c o d e r -- > c h a n n e l -- > d e c o d e r - - > me s s a g e

Thi s v i ew assumed tha t t he reader success fu l ly decoded a message tha t o r ig ina ted f rom some encoder

wi th a p rec i se message . Th i s i s a s trong v iew en t renched in W es tern though t . How ever , the v i ew

presen t s mean ing as some th ing con ta ined wi th in a tex t ra ther than the resu l t o f t he reader de te rmin ing

how to make a rep resen ta t ion o f mea n ing (Zam el , 1992). No t every person can decode every p roper ly

cons t ruc ted l ingu i s t i c message , no r wi l l every person der ive the same m enta l rep resen ta t ion f rom

decod ing such a p roper ly cons t ruc ted message . Fo r example , i f someo ne had no knowledge o r

f rame work fo r cook ing , t ha t person cou ld no t in t e rp re t t he fo l lowing sen tence :

Beat the eggs un t i l f i rm.

I t i s t he reader ' s ab i l it y to app ly the appropr ia t e mea n ings o f be a t and f i rm tha t a l lows

comprehens ion . Readers wi th d i f fe r ing fami l i a r ity wi th cook ing wi l l l ike ly have d i f fe ring rep resen ta t ions

of wha t bea t mean s and wha t cons t itu t es f i rm.

The in t e rac t ion o f schema ta wi th t ex t is es sen ti a l fo r unders t and ing read ing com prehens ion and the

read ing p rocess. In the i r d i scuss ion o f f i r s t- l anguage read ing , Anderson and Pearson (1988) no te tha t

(a) com prehens ion me ans the in t e rac tion o f new in fo rmat ion wi th o ld knowledge ; (b) in fo rmat ion f rom

a new s to ry tha t f i ts wi th the reader ' s p r io r knowledge i s reca ll ed whi l e o ther de ta i ls a re om i t t ed o r

ra t iona l i zed wi th m ean ing no t in the o r ig ina l; and (c ) the au thor o f a s to ry sugges t s in fo rmat ion to the

reader and , i f i t i s a new or unfam i l i a r concep t, com prehens ion resembles a genera l p rob lem-so lv ingac t iv i ty . Fur ther , Sym ons and Press l ey (1993) have sho wn tha t p r io r know ledge a f fec t s the way in which

a reader searches th rough a t ex t fo r in fo rmat ion . Ande rson and P earson (1988) c l a im tha t an adequate

accoun t o f the s t ruc tu re o f the ro l e o f p r io r know ledge , o r schemata , w i l l inc lude : (a ) i n fo rmat ion abo u t

the re l a t ionsh ips am ong the comp onen t s ; (b ) a majo r ro l e fo r in ference ; (c ) accep tance tha t du r ing

language com prehens ion , peo p le p robab ly re ly on knowled ge o f par t i cu la r cases as wel l as abs trac t and

genera l schemata . M oreover , w ork in cu l tu ra l background , con ten t knowledge , l anguage sk i ll s , and

read ing com prehens ion has show n tha t schemata a re impo r tan t fo r read ing (Barne t t, 1986 ; Car re l l, 1988;

Reyn o lds , Tay lo r , S te f fensen , Sh i rey , & Anderson , 1982 ; S te f fensen , 1987). Bow er , Black , and Turner

(1979) examined ho w kno wled ge o f a top ic a ids in unders t and ing . In th i s s tudy , studen t s were asked to

read a s to ry abou t a v i s i t t o a d oc to r ' s o f f ice , and were then asked to reca l l t he mater i a l. In 20% of the

reca l ls , s tuden ts added in fo rm at ion tha t was no t in the passage . The au thors as ser t ed tha t because a

doc to r ' s v i s i t is a fami l i a r even t ( sc rip t) , t he tex t s were e l abora ted wi th backgrou nd kno wledge .

Al thoug h i t is c l ear tha t back ground k nowled ge a f fec t s read ing com prehens ion s t rong ly , i t is a l so

ev iden t tha t a s t rong in t e rac t ion be tw een t ex t p rocess ing and the app l i ca t ion o f backgroun d kn owled ge

takes p l ace dur ing the read ing p rocess . An ex ample o f the s t rong in te rac t ion be tween t ex t p rocess ing a t

the lowes t l eve l s and background knowledge can be seen in the way a reader changes in t emal i zed images

when read ing the fo l lowing example .



Imag ine you rse l f walk ing in to a room; i t i s t he mas te r bedroo m of a qu ie t Vic to r i an

house , i n a s lum o f Bo mb ay , w hich has jus t had a f i re and been rebu i l t i n m ode m s ty le ,excep t fo r the mas te r bedroom which i s on ly ha l f remodeled hav ing i t s decora t ive

paneUi'ng intact but barely vis ible because o f the thick smoke. (Feldm an, 1975, p . 93,

p resen ted in Jus t and Carpen ter , 1987)

In this ins tance, close interact ion wi th the iexical and syntact ic s t r ings in the text causes the reader to

con t inua l ly change the con ten t schema be ing used to p roduce the men ta l im age . Such in t e rac tion o f p r io r

know ledge and t ex t p rocess ing i s an impor tan t var i ab le in read ing as sessment .

Form a l Sc he m a T he ro l e o f fo rmal schem a invo lves a reader ' s know ledge o f such t ex tua l fea tu res as

coherence , cohes ion , and t ex t s truc tu re . The focus on these concerns com es f rom an as sumpt ion tha t

there a re su r face-l eve l pheno me na and an in t e rna l s t ruc tu re to a t ex t t ha t wi l l he lp (o r h inder ) the reader.

The su r face- l eve l pheno me na rep resen t cohes ive re l a t ions whi l e the in t e rna l s t ruc tu res rep resen t the

internal s t ructure o f the text as a whole.

Cohes ive re l a tions a re def ined as those re l a t ions tha t l i nk one sen tence to ano ther w i thou t re fe rence

to a h igher l eve l o f ana lys is . Coh es ion occurs wh en the in te rp re t a tion o f some e l ement in the d i scourse i s

dependen t on tha t o f ano ther . Gr ime s (1975 ) d iscusses cohes ion as the re l a t ionsh ip o f the new

info rma t ion in the t ex t t o in fo rm at ion tha t is a l ready ava il ab le . Hal l iday and H assan (1976) def ine

cohes ive t i es as ins t ances in wh ich two words a re l inked by one o f f ive types o f re l a tionsh ips :

(a) referent ial ; (b) subst i tu t ion o f one w ord w i th another; (c) el lips is ; (d) conjunct ion, addi t ive,

adversat ive, cau sal , and tem pora l ; and (e) lexieal , including rei terat ions and col locat ion. Irwin (1986)

po in t s ou t tha t re fe ren ti a l re l a tionsh ips a id t ex t com prehens ion and tha t cohes ive dev ices such as

con junc t ions and connec to rs he lp com prehens ion and read ing speed . How ever , cohes ion i s d i f fe ren tf rom coherence . Cohes ion he lps coherence. Van D i jk (1977) asser ts t ha t m any aspec t o f l anguage

s imul t aneous ly con t r ibu te to coherence . Connec t ives , impl i ca t ions , verb f rames , p roper ty re l at ions,

cond i t ion-consequence re l a tions , genera l -par t i cu la r re l a tions , and o ther sem ant i c re l a t ions tha t l ink

sen tences a re a l l s a id to add to the coherence o f a tex t . No t su rp r i s ing ly , s tud ies have shown tha t readers

p resen ted wi th m ore and l es s coheren t vers ions o f t ex t show be t t e r comprehe ns ion in the more coheren t

vers ion s 0VIeKeown, Bec k, Sinatra, Losterrnan , 1992). Several areas associated wi th coherence have

shown negat ive affects on comprehension: (a) ambiguous, d is tant , or indirect references; (b) indis t inct

re l a t ionsh ips be tween even t s ; and (e) the inc lus ion o f i r re levan t even t s o r ideas (Beck M eKew on ,

1989). As such , these a reas shou ld be t aken in to accoun t when se l ec ting t ex t s fo r as sessment and wh en

des ign ing i t em con ten t.

I n a d d i t i o n t o c o h e s i v e r e l a ti o n s hi p s , e x t s tr u c t u re a n d o r g a n i z a t i o n a l p a t t e r n p l a y a l a r g e r o l e i n

c o m p r e h e n s i o n . R e s e a r c h h a s s h o w n t h a t k n o w l e d g e o f t e x t s t r u c t u r e n te ra c t s i t h c o m p r e h e n s i o n

C a rr e ll , 1 9 8 5 ) . F o r e x a m p l e , t e x t s t r u c tu r e s i t h a s t r o n g a n d c o n s i s t e n t i n t e r na l l o g i c a r e m o r e

c o n d u c i v e t o p r o c e s s i n g t h a n a r e t h o s e w i t h w e a k i n te r na l o g i c C a rr o ll , 1 9 8 4 ) . N a r r a t i v e s h a v e a

h i er a rc h ic a l s c h e m a t i c s t ru c t ur e a n d w h e n t h a t s t ru c tu r e s u s e d t o g u i d e c o m p r e h e n s i o n a n d r ec al l, o t h

a r e f a ci l it a te d . S i m i l a r l y , e s e a r c h w i t h d i f f e r en t t r u c tu r e s n e x p o s i t o r y p r o s e h a s s h o w n e f f e ct s n

r e a d i n g c o m p r e h e n s i o n d u e t o t h e l o c a t i o n o f i n f o r m a t i on w i t h i n t h e s t ru c tu r es . S p ec i fi c al l y, n f o r m a t i o n

l o c a t e d h i g h , o r a t t o p l e v el s , n t h e h i e r a r ch i c a l o r g a n i z a t i o n o f a p a s s a g e i s r e c a l l e d b e t t e r h a n



in format ion a t lower l evel s. Tight ly organized pat tems of compar i son , causat ion , and problem /solu t ion

faci li t a te the recal l o f sp eci f ic ideas f rom a t ex t bet ter than do less cons t ra ined tex t s t ructures such asdescr ip t ion (Carre l l, 1984) . Add i t ional ly , Rol ler (1990) pos i ts an in teract ion between know ledge and tex t

s t ructure . This l ine of research has sho wn that t ex t s t ructure p lays a s t ronger ro le in com prehens ion whe n

the content i s mo derate ly unfam i l iar than whe n the mater ia l i s fami l iar or com plete ly unfami l iar . Given

the resu l t s of the research o n tex t s t ructure , cons iderat ion o f the t ex tual organ izat ion and the d i s t r ibut ion

of d i f fer ing s t ructures across a t es t should be cons idered in the t es t des ign and cons t ruct ion process .

Strategies andM etacognitive S kills Readers em ploy reading s t ra teg ies and mon i tor thei r

performan ce througho ut each reading act iv i ty . Carre l l (1989) asser t s that reading s t ra teg ies are impor tant

fo r wha t t hey r evea l abou t t he wa y r eade rs m anage t he i r i n te rac ti on wi t h w r i tt en t ex t and how t hes e

s t ra teg ies are re la ted to t ex t com prehens ion . These s t ra teg ies are employe d as the reader regulates

com prehens ion and em ploys repa i r s tra teg ies . A l i s t o f s tra teg ies wo uld include such d iverse sk i l ls as

sk imming, scanning , guess ing meaning in context , sk ipping unknown words , to lera t ing ambigui ty ,

reading for mean ing , reading cr i ti ca l ly , making inferences , apply ing appropr ia te backgroun d know ledge,

and recognizing tex t s t ructure (Carre l l, 1991) . Barnet t (1988) def ines two level s of s t ra tegy categor ies ,

the t ex t l evel and the w ord level. Text - level s t ra teg ies are those s t ra teg ies re la ted to the reading passage

as a whole or to large par t s of the passage; they include co ns ider ing backgrou nd know ledge, predic t ing ,

us ing r if l es and i l lus t ra t ions , reading w i th a purpose, sk imm ing, and scanning . W ord- level s t ra teg ies

include such s t ra teg ies as us ing context to guess wo rd meanings , iden t i fy ing the gram mat ical category o f

words , fo l lowing reference words , and recognizing me anings through wo rd fami l ies and format ion . In

terms o f com prehens ion m oni tor ing , Casnave (1988) em phas izes the reader ' s ab i l i ty to evaluate the

cu r rent leve l o f under s t and ing and t o p lan how t o r em edy a com prehens i on p rob l em . B l ock (1986 )

ident i f ies main-m eaning l ine s t ra teg ies and word-so lv ing s t ra teg ies, and asser t s that good readers are

more ab le to mo ni tor thei r comprehen s ion , are more aw are of the s t ra teg ies they need and use , and uses t ra teg ies more f l ex ib ly than less success fu l readers . Speci f ically , good readers ad jus t thei r s t ra teg ies to

the type o f t ex t they are reading and to thei r purpose. In the a ssessment o f reading , readers need to be

given the opp or tuni ty to apply thei r word - level and tex t - level s t ra teg ies in dem ons t ra t ing thei r reading

a b i l i t y

Purpose and Context From the communicat ive competence perspect ive , i t i s impor tant to address

academ ic reading as a mea ning-ba sed act iv i ty that is purpose and com prehens ion dr iven . Read ing i s

mot ivated b y the reade r ' s par t i cu lar purpose and i s propel led by increas ing compreh ens ion o f the t ex t s.

Fur ther , g iven that l anguage i s em bedded in context (s ) and tasks , academ ic reading assessment should

focus on contexts and purposes and address narrow language abi l it i es as they em erge f rom context .

W hethe r we are t a lk ing a bout ch i ldren reading in school or adul t s reading univers i ty- level course

mater ia l , i t seems clear that m os t com prehens ion i s l inked to purpose, and i t is thus im por tant to ex amine

reading wi th in the con text of that purpose.

In address ing the context and pu rpose i s sues surrounding reading , Hi l l and Parry (1992) argue agains t

an autonomo us mode l of reading that t reats reading as an au tonom ous sk i l l that is independent of o ther

factors and t ransferable across a l l k inds of t ex t s (p . 444). Rather , they emphas ize the socia l nature of

reading in whic h reader and wri ter are involved in an exchange. They assum e that readers impu te



character ist ics to an author, construct some social identi ty for themselves, and in this process view the

tex t as a fo rm o f communica t ion wi th some soc ia l purpose. In th is soc ia l con tex t, any par t icu lar tex t tha ti s read was wr i t ten fo r some purpose , and the reader i s l ikewise read ing fo r som e purpose . Consequent ly ,

read ing is no t comple te ly au tonomou s, bu t ra ther i s h igh ly con tex tua l ized in te rms of how and why a

reader p rocesses tex tua l in format ion . As F r ie re (1991) s ta tes, Language and rea l i ty a re dynamica l ly

in terconnected . The unders tand ing a t ta ined by cr i t ica l read ing o f a tex t impl ies perce iv ing the

relationship between text and conte xt (p. 139).

Bennet t (1993) has ind ica ted tha t research on the domain-spec i f ic na ture o f ab i l i ty has caused

cogni t ive psycholog y to red i rec t it s o r ien ta tion in some ways away f rom genera l ab i l i t ies to s i tua ted

cogni t ion . A reader ' s ab i l i ty i s no t a un i fonn const ruc t in tha t read ing per formance w i l l vary depending

on such fac tors as task demand , purpose , and know ledge . Bennet t (1993) no tes a concomitan t need to

balance genera l and con tex t - spec i f ic v iews o f ab i l ity g iven tha t genera l and spec ia l ized knowledge

opera te together . Whereas genera l sk i ll s in the absence of a r ich domain-spec i f ic knowledge base a re

inef fec tua l, a domain-spec i f ic know ledge base wi thout genera l sk i l ls m ay on ly f imct ion ef fec t ive ly wi th

formula ic p rob lem s (p. 7 -8). Snow & Loh man (1993) ind icate tha t there a re sources o f var ia tion in bo th

abili ty and task performa nce in addition to skil l and strategy differences. The y note that attentional

requ irements o f tasks , and the degree to which consis ten t p rocessing becomes au tomat ic wi th p rac tice ,

ma y change the am ount o f a tten t ion the examinee has to devote to the ac t ivi ty . That i s , the amount o f

exposure to the par t icu lar task , g iven the par t icu lar purpose , can i t se l f change the exam inee ' s

per formance . For example , i t i s l ike ly tha t in ma ny con tex ts a reader ' s comprehe nsion is h igher a f te r

pro longed exposure to a tex t o r se t o f tex ts than i t was a t the beg inn ing of the tex t . Thus, assessment

needs to account fo r the b roader con tex t o f the read ing task .

Im p l ica tions f o r t he sses sm ent o f cad em ic R ea d ina

Severa l impl ica t ions fo r assessment emerge f rom the p rev ious d iscussion o f communica t ive

competence and the read ing process . These wi l l be d iscussed be low. I t should be no ted fi r st , however ,

tha t in a t tempt ing to es tab l ish the parameters re levan t fo r developmen t o f TOE FL 2000 , the TO EFL

Com mit tee o f Examiners a t i ts sp r ing 1993 meet ing , ou t l ined four components o f va l id i ty tha t would

p lay cen t ra l ro les in tes t design . These four components w ere iden t if ied by M essick (1989) as

represen ting a un i f ied va l id i ty f ramework tha t inc luded the face ts o f jus t i f ica t ion o f tes t ing and f imct ion

or ou tcome of tes t ing . These c omponen ts a re const ruc t va lid i ty, va lue impl ica t ions , re levance/u t i li ty , and

socia l consequences . The fo l lowing d iscussion of impl ica t ions fo r read ing assessm ent f rom a perspec tive

of assess ing com munica t ive com petence wi l l re la te impl ica tions f rom the p rev ious d iscussion on read ing

to these four compo nents whe n app l icab le.

Implicat ion 1. In te rms of const ruc t va l id i ty , the c leares t impl ica t ion f rom the p rev ious d iscussion

is the need fo r expanding beyond the m ul t ip le-cho ice o r o ther se lec ted- response fo rmats . Th is

impl ica t ion i s due to the concerns iden t i f ied regard ing con tex t and purpose , as wel l as the na ture o f

reading skil l application. First , as M islev y (1993) indicates, tests made up o f tradit ional selected-

response i tems presen ts tasks to the reader tha t a re more d ear ly def ined and const ra ined than most rea l -

wor ld p rob lems. Ma ny rea l -wor ld tasks a re i l l defined and a l low the reader to m ake cho ices in p rec ise ly

how a p rob lem w i l l be so lved . Thus, in o rder to avo id under represen ta tion of the const ruc t o f academic



read ing , const ruc ted- response fo rmats wi l l need to be u t i l ized . Given the d iscussion o f con tex t and

purpose, an overreliance on selected-response formats clearly restr icts the view of academic reading to anunder represen ted const ruct . For example , the ab i l i ty to res t ruc ture in format ion and app ly knowledge is

the essence of read ing and learn ing f rom tex t. Given tha t , examinees need to be requ ired to app ly

knowledge and to res t ruc ture the in format ion they conf ron t in a tex t . I t is no t possib le to do th is so le ly

wi th se lec ted- response fo rmats . A second reason for expanding beyond se lec ted- response fo rmats re la tes

to B ennet t ' s (1993) observa t ion tha t in some w ays tes ts compr ised o f se lected- response i tems appear to

assume tha t some sk i l l s can be decomposed and iso la ted from appl ied con tex ts . They thus suppor t a

v iew tha t sk i l l s a re incrementa l ly acquired . How ever , the above d iscussion of the na ture o f sk i ll s po in ts

out that skil ls overlap and are applied in particular contexts differentially depending on, among other

things, reading purpose.

The inc lusion o f const ruc ted- response sec t ions in the tes t wi l l address som e o f the negat ive va lue

impl ica t ions and soc ia l consequences tha t have become assoc ia ted wi th mul t ip le-cho ice tes ts . Given tha t

there may be washback ef fects f rom assessm ent on to ins truc t ion , the inc lusion of more per formance- type

sec t ions wi l l p rov ide an impetus fo r teachers and s tuden ts to focus m ore on language product ion and

appl ica tion than on t ra in ing fo r the test . Such changes ma y enhance the re la t ionsh ips be tween read ing

and wr i t ing as mea ning-ma king ac t iv i ties (Zamel , 1992). How ever , care wi l l have to be taken to ensure

that different subgroup s are not adv ersely affected by the types o f tasks and texts that are uti l ized.

Implication 2. Notw ithstand ing impl ica t ion # 1 , there ma y s t il l be a need to incorpora te mul t ip le-

choice or other selected-respo nse formats in order to balance general and context-specif ic tasks. This

inc lusion of se lec ted-response i tems w i l l a l low read ing assessment to incorpora te the w ay in w hich bo th

genera l and spec i f ic knowledge in terac t (Bennet t , 1993). Given the im po r t~ ce o f au tomat ic i ty in lex ica l

and syn tac t ic p rocessing , there i s a c lear need to m easure the under ly ing read ing com petence o f theexaminees . I t is imp or tan t to assess bo th comprehension ab i l i ty and the app l ica t ion of tha t

comprehension . In th is wa y , t rad it ional tes ts composed of se lec ted- response i tems may be v iew ed as

contr ibu t ing in format ion abou t the more genera l c ross-con tex tua l components o f academic read ing .

Thus, the tes t ma y incorpora te a combinat ion of const ruc ted- response and se lec ted- response i tems. As

Messick (1993) po in ts ou t, . . . the quest ion i s no t s imply which fonnat i s be tte r fo r a par ticu lar purpose ,

bu t a lso what co mbinat ion o f fo rmats serves tha t purpose be t te r wi th the few est measureme nt

disadva ntages (p. 62) . Selected-respon se formats are eff icient me chan isms for establishing the existence

and level of underlying e nabling abil i t ies.

Implication 3. As no ted in the d iscussion of academic read ing , comprehension does no t take p lace

in a vacuum. Com prehension in academic se t t ings i s t ransla ted in to some product . The crea t ion of th is

produ ct involves the application of multiple skil ls and abil i t ies in carrying out a task or tasks. Thus,

reading assessm ent should be, at lease in part , task based (Long & C rookes, 1992). The incorporation of

a task basis fo r assessm ent w i l l a l low the app l ica t ion of many sk i ll s on the par t o f the reader , essen t ia l ly

represen t ing the eco logy of read ing .

The academ ic tasks tha t a re used in assessment should encourage the readers to ex tend themselves

and app ly read ing sk i l ls fo r au then t ic purposes . Authen t ic academic tasks , such as answer ing essay tes t

questions, reading textbo oks or journa l ar t icles, and taking notes sho uld be identif ied and incorporated

10



¸

in to the assessment . A represen tat ive se t o f academic read ing tasks should be p resen ted to each

examinee . Th is task-based approach recognizes tha t , genera l ly, the ob jec t o f comprehen sion is a por t ionof a tex t ra ther than the com ple te physica l tex t, and the task be ing per formed genera l ly de termines the

appropr ia te por t ion of tex t . Fur ther , a focus on tasks w i l l increase the nu mb er o f con texts tha t requ ire the

reader to app ly metacogni t ive p rocess in moni to r ing success . By focusing on tasks , then , the tes t re flec ts

the man ner in which read ing usual ly takes p lace . L ikewise , au then tic tex ts should be used , g iven the

impor tance o f tex t s t ruc ture and cohesive e lements . In o rder to p rov ide suf f ic ien t con tex t fo r the

examinee , the au then tic tex ts should be long enough to p rov ide the fu l l con tex t o f the m ater ia l o f in te res t

(Pierce, 1992). How ever, texts will need to be evaluated for coherence, rhetorical organization,

proposi t ional densi ty, and l ingu is tic com plex i ty and so on in o rder to ensure tha t the tex ts examinees

confront are cooperative texts. Finally, a focus on authentic tasks and texts will l ikely have posit ive

value impl ica t ions and socia l consequences by emphasiz ing language in academic con tex t . In te rms o f

va lue impl ica t ions , any tes t -based ins t ruc t ion wi l l need to address the ta rge t s i tua t ions , and in te rm s of

soc ial consequences , the tes t i tem types wi l l hopef i fl ly no t favor any par t icu lar g roup s ince the tasks a re

d i rec t ly re f lec t ive o f the academ ic domain . Thus, the tes ts would re f lec t tasks tha t a re represen ta tive o f

the w ays in w hich ab i l i ty i s app l ied in au then t ic academic con tex ts , and the ex ten t to which tes t metho d

is a facet in performance is a direct ref lection of academic demands.

Implication 4. As ind ica ted above , background knowledge and con tex t p lay impor tan t ro les in

read ing . Tw o possib le conclusions can be d rawn f rom these observa t ions , g iven the emphasis tha t has

thus fa r been p laced on purposef td read ing . Fi r s t , a lthough i t i s impor tan t to p rov ide exam inees wi th a

var ie ty o f tex ts and tex t types in o rder to genera lize the i r ab i l it ies beyond a s ing le s i tua t ion-spec i fic

con tex t, i t i s a lso impor tan t to p rov ide a tes t tha t i s coheren t in te rms of i t s con tent . Th is means tha t the

examinat ion m ay involve themat ica l ly based tex ts in te rms of con ten t . Such an organ iza t ion ref lec ts the

large amo unt o f rea l-wor ld academic read ing ac t iv i ty tha t i s o rgan ized around par t icu lar courses o rresearch ac tiv i ties . The use o f themat ica l ly re la ted tex ts addresses Hi l l and Pa r ry s (1992) observa tions

tha t in an a t tempt to p resen t man y d i f fe ren t k inds o f tex t to an exam inee , t rad i t ional read ing tes ts

typ ica l ly inc lude severa l passages on d i f fe ren t top ics . Thus, each passage i s embedd ed in i t s own

separate tex tua l wor ld , causing a reader to read in a m anner mu ch d i f fe ren t f rom au then t ic read ing

interactions.

Second , i t may be appropr ia te to develop m ore adap t ive types o f tes ts , e i ther v ia com puter o r th rough

se lec t ion of tex ts and tasks by the examinee . Th is re f lec ts the need fo r the exam inat ion to be coheren t in

te rms of con ten t and f rom the recogni t ion tha t mu ch knowledge has a s i tua ted basis (Gi tomer , 1993).

Tests could ref lect different content domains, such as natural science, social science, or humanities.

Exam inees w ould se lect the a rea o f in te res t.

Implications 5. Given tha t the TO EFL 2000 wi l l be used fo r academic se lec tion , i t i s impor tan t to

consider the re levance and u t i l i ty components o f Messick s (1989) va l id i ty model w i th re ference to the

new tes t . A pr im ary e lement here i s the ex ten t to which score in terpre ta tion and use may more c lose ly

reflect real-wo rld academic reading tasks. Thus, in order to ref lect the variab il i ty in solving real-world

academic tasks , assessm ent may need to go beyond prov id ing a read ing score and presen t a l i te racy score

tha t i s der ived f rom per formance on wel l def ined tasks . Adm issions o f f icers ma y take the app l ican t s

11



abili ty to ap ply language abil i t ies into account. In order to facil itate this, a descriptive scale related to

academic l i te racy wi l l need to be developed . Fur thermore , i f score repor ting becomes more descr ip tiveand cr i ter ion referenced to partic ular tasks and task d om ains, rather than str ictly norm referenced, the use

of the tes t m ay c onceivab ly be ex tended to p lacem ent o r d iagnost ic dec isions .

Implication 6. From the p r io r d iscussion and the im pl ica t ions d rawn above , add i t ional genera l

recom men dations abo ut test desig n seem warranted. First , in plrincipal, reading could be integrated with

any of the o ther language sk i l ls . How ever , th is should be based on the l i te racy task be ing encountered .

The tes t fo rmat cou ld be such tha t the examinee acquires in format ion on a par t icu lar top ic f rom read ing

and/or l is ten ing , and then producccs some w r i t ten o r o ra l p roduct . For exam ple , the exam inee migh t mad

a passage and sum mar ize the in format ion . Sum mary tasks can represen t what the examinee unders tands

about a tex t as he o r she reads . A w r i t ten sum mary can ind ica te the g is t in the exam inee ' s m ind , and th is

ma y represen t overa l l comp rehension of a passage (Kin tsch & van Di jk , 1978). The wr i t ing summ ary

migh t be fo l lowed by a l i s ten ing passage on a s imi lar top ic . The examinee migh t then answ er se lec ted-

response i tems re la ted to in format ion presen ted in the passage and then wr i te an eva lua t ive essay . Score

reporting could then indica te both a reading score and a f imctional academ ic l i teracy score.

12



onc lus ions

T h e p r e c e d in g d i s c u s s io n p r e s e n t s a n i n i ti a l v i e w o f h o w to c o m mu n ic a t i v e ly a s s e s s a c a d e mic

r e a d in g a b i li t y . T h e a p p r o a c h d e s c r i b e d v i e w in g r e a d in g i n a li t e ra c y c o n t ex t . I t st r es s e s mo v in g b e y o n d

s e e in g r e a d in g a s a n a u to n o m o u s a b i l i t y t h a t i s a s s u m e d t o b e r e l a t iv e ly s t a t ic a c r o s s t e x ts a n d t a s k s .S u c h a v i e w in v o lv e s r e g a r d in g r e a d in g a s a p u r p o s e f u l a c t i v i ty w i th in a c o n te x t . A d d i t i o n a l e m p h a s i s i s

g iv e n t o t h e n e e d t o i n c o r p o r a t e c o n s t r u c t e d - re s p o n s e f o r ma t s i n to t h e a s s e s s m e n t i n o r d e r to a d d r e s s t h e

c o n s t r u c t i v e a n d re c o n s t r u c ti v e a s p e c t s o f r e a d in g i n co n t e x t. F in a l l y t h i s d i s c u s s io n r e c o mm e n d s t h a t

r e a d in g a s s e s s m e n t b e i n t e g r a t e d w i th o th e r l a n g u a g e s k i l l s.

In prop os ing the above the re i s no a sse r t ion tha t th is wi l l be easy . Seve ra l i s sues r emain to be

reso lved . Som e of these i ssues are :

1 . H o w w i l l s co r e s b e r e p o r te d ? W i l l t h e y p r e s e n t a c o n t i n u o u s s c a le o r r e p r e s e n t t h r e s h o ld

b e h a v io r s ?

2 . H o w w i l l t e s t s e c u r i ty b e ma in t a in e d ?

3 . H o w w i l l e q u iv a l e n c e b e en s u r e d ?

4 . H o w w i l l t a s k f a i rn e s s b e a s s u re d ?

5 . H o w s y s t e ma t i c w i l l t h e i n t e g r a t i o n o f s k i ll a re a s h a v e t o b e a cr o s s a d min i s t r a ti o n s ?

6 . H o w w i l l r e l i ab i l i ty b e e n s u r e d ?

7 . H o w w i l l a n y n e g a t i v e e ff e c ts d u e t o s u b j e c t i v i ty o f e v a lu a t i o n b e min imiz e d ?

T h i s i s c e r ta in ly n o t a n e x h a u s t i v e l i s t o f q u e s t i o n s th a t r e ma in t o b e a n s w e r e d . I t in d i c a te s h o w e v e r

m a n y o f t h e i s s u e s t h a t w i l l a r is e a s t h e T O E F L 2 0 0 0 i s d e v e lo p e d to r e f l e c t t h e c o m mu n ic a t i v e n a tu r e o f

a c a d e mic r e a d in g .

13



efe rences

Alderson, J . C. (1990) . Tes t ing reading com prehens ion ski ll s (pa r t one) . Reading in a Fore ign Language .

6:425-438.

Anderson, R. C. , Pearson, P . D . (1988). A schema-theore t ic v iew of bas ic processes in reading

comprehens ion. In P . Carre l l, J . Devine , D . Eskey (Eds .) , Interact ive Approaches to Second

Language Re ad ing (pp. 37-55) . Cambridge : C amb ridge Univers i ty Press .

Bachman, L. (1990) . Fundamental considerations in language testing. Oxford: O xford Un ivers i ty Press .

Bachm an, L., Savignon , S . (1986) . The eva lua t ion of comm unica t ive language prof ic iency: A c r it ique

of the AC TF L o ra l in te rview. The Mo dern Language Journa l 70, 380-390.

Ba lota , D . (1990). The ro le of mean ing in word recogni t ion . In D. Ba lota , G . d 'Arc ias , K . Rayner

(Eds.), Com prehens ion processes in reading (pp. 9-32). Hil lsdale , NJ: Lawrence Erlbaum Associates.

Barne t t , M. A. (1986) . Syntac t ic and lexica l/ semant ic ski l l in fore ign language reading: Im portance and

interact ion. The M od ern Lang uage Journal, 70, 343-349.

Barne t t , M. A. (1988) . Read ing throug h context: Ho w rea l and perceived s t ra tegy use a ffec ts L2

comprehens ion. The Mo dern Language Journal 72, 150-162.

Beck, I. L. , Carpente r , P . A . (1986) . Cogni t ive approaches to unders tanding reading: Impl ica tions for

instructional pract ice . American Psychologist, 41 10), 1098-1105.

Beck, I. , M cKeow n, M. (1989) . Exposi tory text for youn g readers : The i ssue of coherence . In L.

Resnick (Ed. ) , Kno wing , learning, and instruction (pp. 47-65) . Hi ll sda le , NJ: L awrence E rlbaumAssociates.

Benne t , R. (1993) . On the meanings of cons t ruc ted response . In R. Benne t t W . W ard (Eds . ),

Construction versus choice in cognitive measurement: lssues in constructed response, performance

tes ting, an d por t fol io assessment (pp. 1-27) . Hi l l sda le , NJ: L awrence E dba um Assoc ia tes.

Block, E. (1986) . The com prehens ion s t rategies of second language readers . TESO L Q uarterly .

20:463-494.

Bow er, G . , Black, J . , Turner , T. (1979) . Sc r ip ts in me mo ry for text . Cognitive Psychology, 11,

177-220.

Cam pbe l l , R. , W ales , R. (1970) . The s tudy of language acquis i tion . In J . Lyo ns (Ed. ), New hor izons in

linguistics. H a rmondsw or t h : P e ngu i n Books .

Cana le , M . , Swain , M. (1980) . Theore t ica l bases of com mun ica t ive approaches to second language

teaching and tes t ing. Ap plied Linguistics, 1, 147.

14



Carrel l , P. (1984). The effects of rhetorical organizat ion on ES L readers. TESOL Quarterly 17 441-469.

Carrel l , P. L. (1985). Facil i ta t ing ESL reading by teaching text structure . TESOL Quarterly 19 727-752.

Carre l l, P . L. (1988) . Some causes o f text -bou nded ness and schem a infe rence in ESL reading. In P .

Carrel l , J . Devine, D. Esk ey (Eds.), Interactive approaches to second language reading (pp. 101-

113) . Cambridge : Cam bridge U nivers i ty Press .

Carre ll , P . L. (1989) . M etacogni t ive awareness and second language reading. The Modern Language

Journal 73 121-134.

Carre l l, P . L. (1991) . S t ra tegic reading. In : George town Univers i ty Rou nd T able on Languag es and

Lingu is t ic s (pp. 167-178) . W ashington, D C: George tow n Univers i ty Press .

Casnave , C. P . (1988). C om prehens ion m oni tor ing in ESL reading: A neglec ted essentia l. TESOL

Quarterly 22(2) , 283-302.

Cun ningham , A. , S tanovich, K . , W i lson, M. (1990) . Cogni t ive va r ia t ion in adul t col lege s tudents

di ffe r ing in reading abi l ity . In T. Carr B. Levy (Eds . ) , Read ing and i ts development: Com ponent

skills approaches (pp. 129-159) . S an Diego, CA: Academ ic Press .

Fe ldman , J . (1975). Bad-m outhing frames . In R. Schank B. L. Nash -W ebber (Eds .) , Tinlap conj~rence

proceedings. Ca mbr i dge MA : The M IT P re ss .

Fr ie re , P . (1991). The importance of the ac t of reading. In C. Mi tche l l K . W ei le r (Eds . ), Rewri t ingliteracy: Culture and the discourse of the other (pp. 139-145) . Ne w York: Bergin Garvey

Publishers.

Gick, M. , Holyoak, K. (1983) . Schema induc t ion and ana logica l t ransfer . Cognitive Psychology 15

1-38.

Gi tomer, D . (1993) . Pe rformance assessment and educa t iona l measurem ent . In R. Benne t W . W ard

(Eds.), Construction versus choice in cognitive measurement: Issues in constructed response

perform ance test ing a nd portfol io assessment (pp. 241-263) . HiUsda le , NJ: Law rence Erlbaum

Associates.

Goodman, K. (1967) . Reading: A psychol inguis t ic guess ing game. Journ al of the Reading Specialist 6

126-135.

Goug h, P . B. (1972) . One second of reading. In J . F . Kavan augh I . G . Mat t ingly (Ed.) , Language by

eye and ear (pp. 331-358) . C ambridge : M IT Press .

Grabe , W. (1991) . Current deve lopm ents in second language reading research. TESOL Quarerly 25

375-406.

15



Green, J . (1987) . In sea rch o f meaning: A soc iol inguis tic pe rspec t ive on lesson cons t ruc t ion and reading.

In D. Bloome (Ed. ) , Literacy and Schooling (pp. 4-34) . Norwood, NJ: A blex Publ i shing C orp.

Grimes, J . (1975). The T hread o f Discourse. The Hague , Ne ther lands : Mouton.

Hal l iday, M. , Hassan, R. (1976) . Cohesion in English. London : Longma n .

Haynes , M. , C ar , T. (1990) . W ri t ing sys tem backgrou nd and second language reading: A component

ski ll s ana lys is of Eng l i sh reading by na t ive speaker-readers of Chinese . In T. Carr B. Levy (Eds . ) ,

Reading a nd its development: C ompo nent skills approaches (pp. 375-421). San Diego, CA:

Academ ic Press .

Henk, W. , He l fe ld t , J . (1987) . In-process measures of ambiguous text in te rpre t ion: Ano ther look a t the

inf luence of pr ior know ledge . In J . Readence R. Ba ldw in (Eds . ) , Research in literacy: Mergingperspectives: 3 6th Yearbook o f the National Reading ConJbrence (pp. 143-149). Rochester: National

Reading Confe rence , Inc .

Hil l , C. , Parry, K. (1992). The test a t the gate: M odels of l i teracy in reading assessm ent. TESOL

Quarterly 26 433-461.

Hudson , T. (1991) . A content -comp rehens ion approach to reading Engl i sh for sc ience and technology.

TESOL Quarterly 25 77-104.

Hudson , T. (1993). Tes t ing the spec i fic ity of ESP reading ski ll s. In D. Dou glas C. Chape l le (Eds . ), A

New Decade o f Language Testing Research (pp. 58-82). Alexandria , VA: TES OL.

Hym es, D . (1972) . On comm unica t ive competence . In J . Pr ide J . Holm es (Eds .) , Sociolinguistics.

H a rmondsw or t h : P e ngu i n .

I rwin, J . W . (1986) . C ohes ion and comprehens ion: A resea rch review. In J . W. I rwin (Ed. ) ,

Understanding and teaching cohesion comprehension (pp. 31-34). Newark, Delaware: International

Reading A ssoc ia t ion.

Johns ton, P . (1984) . Pr ior know ledge and reading com prehens ion tes t b ias . Reading R esearch Q uarterly

19 219-239.

Just , M ., Carpe nter, P. (1987 ). The psychology o f reading and language comprehension. Boston:

Al lyn Bacon.

Kintsch, W . , van Di jk , T. (1978) . Tow ard a mo de l of text comp rehens ion and produc t ion.

Psychological Review 85 363-394.

LaBerg¢, D. , S am u e ls , S . J . (1974) . Toward a theory o f automat ic informat ion process ing in reading.

Cognitive Psychology 6, 293-323.

16



Long , M., Crookes, G. (1992). Three approache s to task-bas ed syllabus design. TESOL Quarterly, 26,

27-56.

Marshall , S. (1993). Assessing schem a knowledge. In N. Frederiksen, R. Mislevy, I. Bejar (Eds.), Test

theory fo r a new g eneration o f tests (pp. 150-180). Hil lsdale, NJ: Law rence Ed baum Associates.

McK eown, M. G., Beck, I. L. , Sinatra, G. M., Losterman, J. A. (1992). The contribution of prior

know ledge and coherent text to comprehension. Reading Research Quarterly, 27, 79-93.

Messick, S. (1989). Validity. In R. Linn (Ed.), Educat ionalMeasurement 3rdEdi t ion) (pp. 13-103).

Ne w York: Am erican Council on Education ~ Macm illan Publishing Company.

Messick, S. (1993). Trait equivalence as construct validity of score interpretation across multiplemethod s of measurement. In R. Benne t W. Wa rd (Eds.), Construction versus choice in cognitive

measurement: lssu es in constructed response, performance test ing, portfol io assessment (pp. 61-73).

Hil lsdale, NJ: L awrence E rlbaum A ssociates.

M islevy, R. (1993). Foundations of a new test theory. In N. Frederiksen, R. Mislevy, I. Bejar (Eds.),

Test theory fo r a new generation o f tests (pp. 19 -39). Hillsdale, NJ: Law rence Erlbaum A ssociates.

Perfetti, C. (1992). The repres entation prob lem in reading acquisition. In P. Gough, L. Ehri, R.

Treiman (Eds.), Reading Acquis i tion (pp. 145-174). Hil lsdale, NJ: Lawrence Erlbaum A ssociates.

Peirce, B. (1992). Dem ystifying the TO EFL reading test. TESOL Quarterly, 26, 665-691.

Pollatsek, A., Rayner, K. (1990). Lexical access: Some comm ents on mode ls and metaphors. In D.

Balota, G. d 'Arcias, K. R ayner (Eds.), Comprehension processe s in reading. Hillsdale, NJ: Lawrence

Erlbaum Associates.

Rayner, K. (1990). Com prehension processes: Introduction. In D. Balota, G. d 'Arcias, K. Rayner

(Eds.), Comprehension processes in reading (pp. 1-6). Hillsdale, NJ: Law rence Erlbaum A ssociates.

Ray ner, K., Pollatsek, A. (1989). The psych ology o f reading. Englewood Cliffs: Prentice Hall.

Reynolds, R. E., Taylor, M. A., Steffensen, M. S., Shirey, L. L., Anderson, R. C. (1982). Cultural

schem ata and reading com prehension. Reading Research Quarterly, 17, 353-366.

Roller, C. (1990). Comm entary: The interaction of knowledge and s tru~ure variables in the processing of

expository prose. Rea ding Research Q uarterly, 25, 79-89.

Rum elhart , D. (1977). Toward an interactive model o f reading. In S. Dom ic (Ed.), Attention and

p e r f o r m a n c e / V (pp. 573-603). New Y ork: Academ ic Press.

Savigno n, S. (1991). Comm unicative lang uage teaching: State of the art. TESO L Q uarterly, 25, 261-277.

17



Sharkey, N. (1990). A connectionist model of text comprehension. In D. Balota, G. d 'Arcias, K.

Rayn er (Eds.), Comprehension Processes in Reading (pp. 487-514). H il lsdale, NJ: Law renceErlbaum Associates.

Skehen, P. (1991). Progre ss in language testing: The 199 0s. In J. Alde rson B. No rth (Eds.), Language

testing in the 1990s: The communicative legacy. London: M acmillan Publishers, Ltd.

Smith, F. (1971) Understanding Reading. New York: Holt , Rinehart , W inston.

Snow, R., Lohm an, D. (1993). Cognit ive psychology, new test design, and new test theory: An

introduction. In N. Frederikse n, R. M islevy, I. Bejar (Eds.), Test Theory For A New Generation o f

Tests (pp. 1-17). Hil lsdale, NJ: L awrence E rlbaum Associates.

Sperber, D., W ilson, D. (1986). Relevance: Communication and cognition. Cambridge, MA: Harvard

University Press.

Stanovich, K. (1980). Tow ard an interactive-compensatory model o f individual differences in the

development o f reading fluency. Rea ding Research Quarterly 16 32-71.

Steffensen, M . S. (1987). The effect of context and culture on children 's L2 reading: A review . In J.

Dev ine, P. Carrell, D. E. Esk ey (Eds.), Research in reading in English as a second language

(pp. 43-54). Washington: TE SOL.

Sym ons, S., Pressley , M. (1993). Prior know ledge affects text search success and extraction of

information. Reading Research Quarterly 28 251-259.

van Dijk, T. (1977). Text an d context. London: Longman.

van Dijk, T. (1985). Strategic discourse comprehension. In T. Ballmer (Ed.), Linguistic dynamics:

Discourses procedures a nd evolution (pp. 30-61). Berlin: W alter de Gru yter Co.

Valencia, S. W ., Stal lman, A. (1989): Multiple measures of prior knowledge: Com parative predict ive

validity. Yearbook of the National Reading Conference 38 427-436.

Zam el, V. (1992). Writ ing one 's wa y into reading. TESO L Quarterly 26 463-485.

18



C o v e r P r i n te d o n R e c y c l e d P a p e r

5 8 7 0 1 - 1 3 9 9 0 ° Y 9 6 M . 7 5 , , 2 5 37 C ,1 • P r l n l e d i n U , , A ,

toefl communicative competence via reading.pdf

Documents