language assessment (chap. 7)

7/30/2019 Language Assessment (Chap. 7)

1/43


2/43

Basic Types of Speaking Imitative. It is simply the ability to parrot

back a word or phrase or a sentence.

Intensive. It is the production of short

stretches of oral language. Examples

include directed response tasks, reading

aloud, sentence and dialogue completion,

limited picture-cued tasks.


3/43

Basic Types of Speaking

Responsive. The tasks include interaction

and test comprehension but at the limited

level of short conversations, standard

greetings, small talk, requests, and

comments.

Interactive. The length and complexity of

the interaction are more in interactive tasks

than in responsive ones. The task

sometimes includes multiple exchanges

and/or multiple participants.


4/43

Basic Types of Speaking Extensive. (monologue) The tasks include

speeches, oral presentations, and story-

telling. Oral interaction from listeners is

either highly limited or ruled out altogether.


5/43

Assessment Tasks: Imitative

Speaking

Word repetition task

Test-takers hear:

beat/bit bat/vat

I bought a boat yesterday.

The glow of the candle is growing. Test-takers repeat the stimulus.


6/43

Scoring scale for repetition tasks 2 acceptable pronunciation.

1 comprehensible, partially correct.

0 silence, seriously incorrect.


7/43

Phonepass Test It elicits computer-assisted oral production

over a telephone. Test-takers read aloud,

repeat sentences, say words, and answer

questions.

Part A: read aloud selected sentences.

Examples: Traffic is a huge problem in

Southern California.


8/43

Part B: repeat sentences dictated over thephone.

Example: Leave town on the next train.

Part C: Answer questions with a singleword or a short phrase.

Example: Would you get water from abottle or a newspaper?

Part D: hear three word groups in randomorder and link them in a correctly ordered S.Ex.: was reading/my mother/a magazine


9/43

Part E: have 30 seconds to talk about theiropinion about some topic that is dictated

over the phone. Topics center on family,

preferences, and choices. Scores are calculated by a computerized

scoring template and reported back to the

test-taker within minutes.


10/43

Assessment Tasks: Intensive

Speaking Directed Response Tasks

Directed response

Tell me he went home.

Tell me that you like rock music.

Tell me that you arent interested in tennis. Tell him to come to my office at noon.

Remind him what time it is.


11/43

Test of Spoken English Scoring

Scale (Read-Aloud Tasks) Pronunciation:

Points:

0.00.4 frequent errors and unintelligible.

0.51.4 occasionally unintelligible.

1.52.4 some errors but intelligible. 2.53.0 occasional errors but always

intelligible.


12/43

Fluency:

Points:

0.00.4 slow, hesitant, and unintelligible.

0.51.4 non-native pauses and flow that

interferes with intelligibility.

1.5--2.4 non-native pauses but the flow is intelligible.

2.53.0 smooth and effortless.


13/43

Variations on Read-Aloud tasks Reading a scripted dialogue.

Reading sentences containing minimal

pairs. Examples: Try not to heat/ hit the

pan too much.

Reading information from a table or chart.


14/43

Read-Aloud Tasks Advantages:

Comparisons between students are quite

simply.

Tests are easy to prepare and to administer.

Predictable output, practicality, andreliability in scoring.


15/43

Disadvantages: It is inauthentic, except in situations such

as parent reading to a child, sharing a story

with someone, giving a scripted oralpresentation.

It is not communicative in real contexts.


16/43

Sentence/Dialogue Completion

Tasks and Oral Questionnaires First, test-takers are given time to read

through the dialogue to get its gist, then the

tape/teacher produces one part orally andthe test-taker responds.

Example (p. 150) short dialogue (p. 151)

Advantage: more time to anticipate ananswer, no potential ambiguity created byaural misunderstanding (oral interview).


17/43

Picture-Cued Tasks A picture-cued stimulus requires a

description from the test-taker. It may elicit

a word, a phrase, a story, or incident.

Scoring scale for intensive tasks:

2 comprehensible; acceptable target form

1 comprehensible; partially correct

0 silence; or seriously incorrect


18/43

A Scale for Evaluating Interviews Grammar

Vocabulary

Comprehension

Fluency

Pronunciation Task (the objective of the elicited task)

Example (p. 158)


19/43

Translation Translation is a communicative device in

contexts where English is not a native lang.

English can be called on to be interpretedas a second language.

Conditions may vary from an instant

translation of a native word, phrase, orsentence to a translation of longer texts.

Advantages: the control of the output &easily specified scoring.


20/43

Responsive Speaking Question and Answer

Examples: 1. What is this called in English?

( to elicit a predetermined correct response)2. What are the steps governments shouldtake, if any, to stem the rate of de-

forestation in tropical countries? ( givenmore opportunity to produce meaningfullanguage in response)


21/43

Questions Eliciting Open-Ended

Responses 1. What do you think about the weather

today?

2. Why did you choose your academic

major?

3. a. Have you ever been to the U. S. before? b. What other countries have you visited?

c. Why did you go there? What did you

like best about it?


22/43

Giving Instructions & Directions Examples: how to operate an appliance,

how to put a bookshelf together, or how to

create a dish.

Scoring: based on (1) comprehensibility (2)

Specified grammatical/discourse categories.


23/43

Eliciting Instructions or Directions Test-takers hear:

Describe how to make a typical dish

Whats a good recipe for making _____?

How do you access email on a PC

computer? How do I get from ___ to ____ in your city?

Test-takers respond.


24/43

The task should require the test-taker to

produce at least 5 or 6 sentences.

Use familiar topics and test linguistic

competence.

Paraphrasing

Examples: paraphrasing a story and

paraphrasing a phone message (p. 162)


25/43

Considerations of Paraphrasing 1. elicit short stretches of output

2. the criterion being assessed:

a. Is it a listening task more than

production? b. Does it test short-term

memory rather than linguistic ability? c. How does the teacher determine

scoring of responses?


26/43

Test of Spoken English (TSE)

TSE is a 20-minute audiotaped test of orallanguage ability within an academic or

professional environment. TSE scores are used by many North American

institutions of higher education.

The tasks are designed to elicit oral production in

various discourse categories. (p. 163) Example: sample items in TOEFL (p. 164)

Scoring: a holistic score ranging from 20 to 60

(performance, function, appropriateness, andcoherence)


27/43

Interactive Speaking Oral Interview: a test administrator and a

test-taker sit down in a direct face-to-face

exchange and proceed through a protocolof questions and directives.

It varies in length from 5 to 45 minutes,

depending on purpose and context.Placement interviews may need only 5

minutes while Oral Proficiency Interview

(OPI) may require an hour.


28/43

A Framework for Oral Proficiency

Testing Four stages: Warm-up, Level check, Probe, and

Wind-down.

Warm-up: The interviewer directs mutualintroductions, helps the test-taker become

comfortable with the situation, apprises the

format, and reduces anxieties.

Level check: Through preplanned Qs, the test-

takers respond using expected forms and

functions. Linguistic target criteria are scored.


29/43

Probe: In this phase, test-takers go to theheights of their ability and extend beyondthe limits of the interviewers expectation.

Through probe questions, the interviewerdiscovers the test-takers proficiency. Atthe lower levels of proficiency, probe itemsmay demand a higher range of vocabulary

and grammar than predicted. At the higherlevels, probe items will ask the t-t to givean opinion, to recount a narrative or torespond to questions.


30/43

Wind-down: the interviewer encourages

the test-taker to relax with some easy

questions, sets the t-ts mind at ease, and

provides information about when andwhere to obtain the results of the interview.

This part is not scored.

Content specifications (p. 169) Sample questions (p. 169-170)


31/43

Sample Questions of an Oral

Interview 1. Warm-up:

How are you?/Whats your name?/What

country are you from?/Let me tell yourabout this interview.

2. Level check:

How long have you been in this city?/tell

me about your family./What is yourmajor?/How long have you been workingat your degree?/What are your hobbies orinterests?/Why do you like your hobby?


32/43

Continue What is your favorite food?/Tell me about your

exciting experience youve had.

3. Probe: What are your goals for learning English in this

program?/Describe your academic field to me.What do you like or dislike about it?/Describe

someone you greatly respect, and tell me why yourespect that person./If you were [president, primeminister] of your country, what would you like tochange about your country?


33/43

Continue 4. Wind-down:

Did you feel okay about this

interview?/Youll get your results from this

interview next week./Do you have any

question to ask?/It was interesting to talk

with you. Best wishes.


34/43

The Success of an Oral Interview Clear administrative procedures (practicality)

Focusing the questions and probes on the purpose

of the assessment (validity)

Biased for best performance

Creating a consistent, workable scoring system

(reliability) Descriptions of the Oral Proficiency Scoring

Categories (p. 172-173)


35/43

Role Play It is a popular pedagogical activity in

communicative language-teaching classes.

The test administrator must determine theassessment objectives of the role play, then

devise a scoring technique that pinpoints those

objectives.

Examples: Pretend that youre a tourist asking

me for directions, You are buying a necklace

from me in a flea market, and want a lower price.


36/43

Discussions & Conversations As informal techniques to assess learners, D & C

offer a level of authenticity and spontaneity that

other assessment techniques may not provide. (clarifying, questioning, paraphrasing, intonation

patterns, body language, eye contact, and othersociolinguistic factors)

Games Oral Proficiency Interview (OPI) guidelines (p.

177)


37/43

Designing Assessments:

Extensive Speaking Extensive speaking tasks are frequently

variations on monologues, usually with

minimal verbal interaction.

Oral Presentations:

Examples: presenting a report, a paper, a

marketing plan, a sales idea, a design of a

new product, or a method.


38/43

Rules for effective assessment: (a) specify the

criterion, (b) set appropriate tasks, (c) Elicit optimal output, and (d) establish practical,

reliable scoring procedures.

Oral presentation checklist

3 excellent 2 good 1 fair 0 poor

Content:

The purpose or objective of the presentation was

accomplished. The introduction was lively and got my attention.

The main idea or point was clearly stated towardthe beginning.


39/43

The supporting points were clearly expressed and

supported well by facts and argument. The conclusion restated the main idea or purpose.

Delivery

The speaker used gestures and body languagewell.

The speaker maintained eye contact with the

audience.

The speakers language was natural and fluent.

The volume of speech was appropriate.


40/43

The rate of speech was appropriate.

The pronunciation was clear andcomprehensible.

The grammar was correct and didnt

prevent understanding. Used visual aids, handouts, etc., effectively.

Showed enthusiasm and interest.

Responded to audience questions well.


41/43

Picture-Cued Story-Telling

At this level, a picture/a series of pictures is usedas a stimulus for a longer story or description.

The objective of eliciting narrative discourseneeds to be clear. (p. 181) (Tell & use the P. tense)

For example, are you testing for oral vocabulary,(girl, telephone, wet) for time relatives (before,

after, when), for sentence connectors (then, so),for past tense of irregular verbs (woke, drank,rang), or for fluency in general?

Criteria for scoring need to be clear.


42/43

Retelling a Story, News Event Test-takers hear /read a story or news event

that they are asked to retell.

It differs from the paraphrasing task

discussed above in that it is a longer stretch

of discourse and a different genre.


43/43

Translation (of Extended prose) Longer texts are presented for the test-taker to

read in the native language and then translate intoEnglish.

Texts vary in forms: dialogue, directions, play,movie, etc.

Advantages: the control of the content,

vocabulary, the grammatical and discoursefeatures.

Disadvantages: a highly specialized skill isneeded.

language assessment (chap. 7)

Documents