m1u6 lecture.ppt

8/10/2019 M1U6 Lecture.ppt

1/42

ASSESSING SPEAKING PURPOSES AND

TECHNIQUES

Prepared by Elena Onoprienko, Yulia Polshina, TatianaShkuratova

Based on material by Fumiyo Nakatsuhara


2/42

Outline

Key questions

Nature of speaking

Speaking as a skill Test purposes and types of test

Speaking test tasks

Scoring Washback effect


3/42

KEY QUESTIONS


4/42

Key questions

Construct Purpose

Task types Scoring criteria

How(score)?

Why assess

speaking?

How(test)?

What is

speaking?


5/42

THE NATURE OF SPEAKING


6/42

Nature of speaking:

spoken language;

speaking as interaction;

speaking as a social activity; speaking as a situation-based activity.


7/42

What is speaking?

A part of the shared social activity of talking

(Luoma, 2004: 29).

In comparison with writing, speakingisMore: transient

dynamic interpersonal

content dependent.

Less:

planned

complex formal

lexically dense.


8/42

Speaking vs Writing

The main differences are in two sets ofconditions - processingand reciprocity:

Processingis connected with time - speakingis going on under greater pressure of time.

Solution to this problem in spoken languagereciprocity. Speakers take turns and create a

text together.(Bygate,1987)


9/42

Spoken language

Pronunciation

Spoken grammar

Lexis


10/42

Pronunciation

Speech is judged on the basis of pronunciation.

What is standard? Native speaker vs non-native speaker.

Communicative effectiveness, which is based on

comprehensibility and probably guided by native speakerstandards but defined in terms of realistic learnerachievement, is a better standard for learner pronunciation.(Luoma, 2004).

What to include in assessment of pronunciation?

Pronunciationindividual sounds, pitch, volume, speed,pausing, stress and intonation.


11/42

Spoken grammar:

grammar is easy to judge because it is easy todetect in speech and writing;

speakers do not usually speak in sentences;

speech consists of idea units connected with and,or, but, or that;

planned vs unplanned speechcomplexstructures vs short idea units;

the internal structure of idea units - topicalisationand tailscreate an impression of naturalness.


12/42

Features of spoken lexis:

simple and ordinary words are common innormal spoken discourse and mark a highlyadvanced level of speaking skills (Luoma, 2004);

generic words (important for the naturalness oftalk);

vague words;

fixed conventional phrases; small words (the more the better perceived

fluency).


13/42

Slips and errors

Normal speech contains a fair number ofslips and errors such as mispronouncedwords, mixed sounds, and wrong words

due to inattention (Luoma, 2004).


14/42

SPEAKING AS A SKILL


15/42

Speaking as a skill

What is skillful speech?

task fulfillment/content;

fluency;accuracy;

vocabulary and grammar range;

interaction.


16/42

Speaking as meaningful interaction

Speaking is both personal and a part of theshared social activity of talking.

The openness of meanings is not only aconvenience in speech; it is also an effectivestrategy for speakers. (Luoma, 2004)

Chatting vs information-related talk.

The role of speaking situations. Roles, role relationships and politeness.


17/42

Why assess speaking?

No single answer:

different groups of language learners have different needs, suchas:

international travellers: language for travel, leisure;

migrants: survival skills, access to employment;

students: exams, academic communication, socialinteraction;

professionals: workplace communication, presentations.

different users have different purposes when they seekinformation from tests;

but most users of language do need to speak.


18/42

TEST PURPOSES AND TYPES OF TEST


19/42

Test purposes and types of test

Test purposes:

proficiency tests

achievement tests placement tests

diagnostic tests.


20/42

What do we need to decide beforegiving a speaking test?

what aspects of languagewe want to assess;

how to elicit ratable language samplesfrom test-takers suitable for the aspects of language.

We need to decide:

rating criteria[marking categories, levels,descriptors] [holistic scales vs. analytical scales];

elicitation techniques / test format(types ofquestions, task types).


21/42

Performance testing

Performance testing in second languageproficiency assessment is traditionally usedto describe the approach in which a

candidate produces a sample of spoken orwritten language that is observed andevaluated by an agreed judging process.

(McNamara, 1996)


22/42

What is performance testing?

sample of written or spokenlanguage;

simulates behaviour in the real world- not

like paper-and-pencil objective tests; observed and evaluated by an agreed

judging process.


23/42

Speaking tasks

A communicative task is a piece of classroom work

which involves learners in comprehending,

manipulating, producing or interacting in the target

language while their attention is principally focusedon meaning rather than form (Nunan 1993:59).

Speaking tasks can be seen as activities that involve

speakers in using language for the purpose of

achieving a particular goal or objective in a particular

speaking situation (Bachman and Palmer 2010).


24/42

Types of information-related talk

Factually-oriented talk:

description

narration instruction

comparison.

Evaluative talk:

explanation

justification prediction

decision.

(Bygate,1987)


25/42


26/42

Features of a speaking task:

input, or material used in the task;

roles of the participants;

settings, or classroom arrangements for paired or group

work; actions, or what is to happen in the task;

monitoring, or who is to select input, choose role or setting,alter actions;

outcomes as the goal of the task;

feedback given as evaluation to participants.

Candlin (1987) cited by Fulcher (2003)


27/42

Speaking test task formats

Individual

Paired

Group

Open-ended tasks

Structured tasks


28/42

Advantages and disadvantages ofan interview

+ testers control over interaction

+ opportunity for an examinee to show the

range of their speaking skills- it is costly in terms of testers time

- interviewers power over an examinee


29/42

Advantages and disadvantages ofpaired formats

+Capable of eliciting more symmetrical contributionto theinteraction from test-takers

+Capable of eliciting much richer and more varied languagefunctions

+ Positive reactionfrom test-takers (less anxious), a sign ofpositive washbackeffect

+ Practical: time-efficient, cost-effective, less burden and lesstraining for the examiners

-The amount of responsibilityon examinees who are nottrained in interview techniques


30/42

Advantages and disadvantages ofgroup formats

+ Well-received by learners

+ Support learning

- Difficult to administer and manage (sizeof the groups and mixture of learnersabilities)

- Difficult to monitor the progress of the

testing


31/42

SPEAKING TEST TASKS


32/42

Speaking test tasks:

oral presentation (verbal essay, prepared monologue);

information transfer (description of picture sequence,questions on a single picture, alternative visual stimuli);

interaction tasks (information gap: studentstudent,studentexaminer, open role play, guided role play);

interview (free, structured);

discussion (student-student, student-examiner).

(OSullivan, 2008: 10-11)


33/42

Framework for designing test tasks

Operations (activities/skills) - informational routines (e.g.telling a story) and improvisational skills (negotiation ofmeaning and management of interaction)

Conditionsunder which the tasks are performed (e.g. timeconstraints, the number of people involved and familiaritywith each other)

Quality of output, the expectedlevelof performance interms of various relevant criteria, e.g. accuracy, fluency or

intelligibility.(Weir, 1993: 30)


34/42


35/42

Developing criteria for assessingspeaking

The importance of double marking for reducing unreliabilityis undeniable.

These criteria need to reflect the features of spokenlanguage interaction the test task is designed to generate.

The criteria used would depend on the nature of the skillsbeing tested and the level of detail desired by the end users.The crucial question would be what the tester wants to findout about a students performance on appropriate spoken

interaction tasks.(Weir, 1993, p.30)


36/42

Rating criteria

Phonological control; Grammatical accuracy; Vocabulary range; Fluency(Council of Europe 2001)

Test format:interview formatwith the following structure:

1.Openings(1 minute).

2.Conversation on familiar topics (3 minutes)The interviewer asks thecandidate to talk about him/herself.

3.Picture Description (2 minutes)The interviewer asks the candidate todescribe a photo.

4.Conversation on topics from the given picture(5 minutes) The interviewer

asks the candidate questions linked to the picture (from general to extendedquestions).

5.Closings (1 minute). (Nakatsuhara, 2012)


37/42

Scoring

Holistic scale

e.g. Trinity College

Bands A, B, C, D

Analytic scale

e.g. IELTS

Fluency and coherence

Lexical resources

Grammatical range andaccuracy

Pronunciation


38/42

Holistic rating scales

Positive features:

practicality: fast;

rating holistically may be more naturalistic.

Disadvantages:

no useful diagnostic information: single score;

not always easy to interpret: raters not requiredto use same criteria to arrive at score.


39/42

Analytic rating scales

Positive features:

can provide diagnostic information if scores reportedseparately;

potentially clear, explicit and detailed; usually more reliable (multiple scores);

useful in training raters to focus on our construct;

potentially useful in guiding learners.

Disadvantages: time-consuming;

may overburden raters. (Green, 2012)


40/42

The role of an interviewer

Interrater/ intrarater reliability

The solutiontraining raters:

understanding criteria for assessment;

agreement with other raters;

consistency of performance.


41/42

WASHBACK EFFECT


42/42

Washback Effect:

The effect of testing on teaching and learning

Positive / negative washback:

positivetest stimulates classroomteaching of important skills;

negativenarrow focus on teaching just forthe test.

m1u6 lecture.ppt

Documents