ORAL ASSESSMENT and ROLE PLAY ACTIVITIESUNA (Universidad Nacional de costa Rica)

Lourdes Pomposo lpomposo@flog.uned.es

UNED: Universidad Nacional de Educación a Distancia.

Madrid

Facultad de Filologías Extranjeras y sus Lingüísticas

Contents

Oral skills in ESL.

Types of interaction in oral assessment.

Difficulty of oral assessment.

Types of rating scales.

What to assess.

The rater in the oral test.

Influence of the rater in the assessing process.

Content of an oral test: individual production and interaction.

Format of the oral test.

Workshop.

Feedback.

Bibliography.

Oral skills in ESL

Cognitive dimension

Social dimension

Knowledge planning:

vocabulary, syntax,

phonology, coherence.

Pragmatic knowledge

Functional: apologise, etc

Sociolinguistic: speech to children or business

managers…

Commmunication through

individual or interactive production

09/04/2023

Types of interaction in oral assessment

E-C: Examiner-Candidate: requires quick answer = pressure

E-C-C: Adaptability to the partner competence in L2. Unexpected answers.

E-C-C-C: Needs of different processes of competences and different points of view. Ability to follow a conversation.

Candidate’s personality: discourse and exam preference are different: Introvert candidate: structured and guided exam (closed ended questions)

ExtrovertCandidate: free speech (Berry, 2004)

To be assessed under pressure (rater, recording, observer…) Biased assessment (knowing or not knowing the candidate, physical

appearance, etc)

Difficulty of oral assessment

Types of rating scales

Analytic scales Holistic Scales

Assesses communication as a whole. Few ratings (3 or 5). “as many as

necessary, as few as posible” (Taylor, 2011) Quick assessment Not decisive for diagnostic

assessment but descriptive. It can have different interpretations. Holistic scales: ..\CURSO DE VERANO PONFERRADA_

14\Rúbricas holisticas\5Tabla32.jpg

..\CURSO DE VERANO PONFERRADA_14\Rúbricas holisticas\imagen-11.png

Identifies and assess the components of the oral production.

More reliable for decisive diagnosis. Useful for raters trainers. Useful for students’self asessment. Time-cosuming. Analitic scales:

..\CURSO DE VERANO PONFERRADA_14\Rúbricas analíticas\Marco Europeo.JPG

09/04/2023

What to assess

Analytic assessment

Fluency Accuracy Coherence Range of Language Body language Pronunciation Interaction …..

Holistic assessment TO GIVE THE STUDENT A LEVEL: Able to communicate in an effective

way Able to transmit ideas and to be

understandable Understandable pronunciation

despite foreign accent

The rater in the oral assessment process

Experienced rater and with teaching experience. To know the demands of levels and to know the purposes of

examinations. To know expectations related to Microabilities (phonemes, grammar,

vocabulary, fluency, effective interaction, etc) y Macroabilities (non-verbal language, communicative functions, suitable discourse for each situation, etc) (Brown, 2004)

Friendly and communicative personality Interaction facilitator.

Influence of rater in the oral test process

PERFORMANCE

RATERS SCORE

Influence of rater personality. Non-verbal rater’s communication. Respect candidate and not to interrupt to correct. Example in which rater is giving new vocabulary to candidate.

EN: What is she like in the class?CA: Well, she is very quiet.

EN: And you are noisier. Correct explanation of instructions: https://

www.youtube.com/watch?v=Iq0c9jFo89A Example of Cambridge rater: https://

www.youtube.com/watch?v=QwlBZc9MrJA“No one judgement may be said to be definitive” (Lumley and McNamara 1995:57)

Content of oral tests: individual production and interaction

Personal information or opinions. Descriptions (photographs, places, hobbies…) Global simulations of possible real situations:- Inmigrants- School students- Travellers- Professionals Improvised interaction.

Oral test format (human and ICTs)

Human-scored

Computer-scored: Automatic Speech recognition (ASR)

Human-deliveredComputer-delivered

Semi-directBulats online, TOEIC, TOEFL iBT

AutomáticPearson PTE Academic, TOEFL iBT practice tests

DirectCambridge

Examinations, Trinity

Examinations

ICTs in oral assessment Automatic evaluation: Automatic Speech Recognition (ASR): not very reliable

for assessing but good for self learning and improving pronunciation.http://www.iallt.org/iallt_journal/an_online_oral_practiceassessment_platform_speak_everywhere International Association for Language Learning Technology

Semi.direct evaluation: https://www.youtube.com/watch?v=QKmcrjQxDMohttps://www.youtube.com/watch?v=TUPTT659464

ICT tools for oral assessment Tools to practice and assess oral skills

Audacity http://audacity.sourceforge.net/ (Recorder and editor)

Chirbit: www.chirbit.com#sthash.RHC8JsMK.dpuf (Recorder and extract tracks of videos)

Voki: www.voki.com (It creates an avatar that speaks)

Voicethread: www.voicethread.com (Uploading multimedia content and other users can comment with text, audio or video. 

Voxopop: www.voxopop.com (audio recording among students and teachers)

UNED EORALS: http://eoral.uned.es

Advantages and limitations of ICTs in oral assessment

Advantages

Face to face examiner disappears. Recordings can be listened to more

times for assessment (examiner). The recording can be repeated

(candidate) Less embarrasing for shy candidates. Time saving. More thoughtful assessment.

Limitations

Face to face examiner disappears. Pressure of time and media . Skills in ICTs use. Recording quality. It may cause

misunderstanding. Technical or internet failures.

09/04/2023

WORKSHOP 1

Holistic assessment Business English Exam

(Bulats)Just give a CEFRL level (A1-C1) to the following speeches.

Speaking Markscheme.pdf

Holistic assessment

From 1-3 assess each of the oral skills of the following interaction:

Final Restaurant.MPG Interaction

restaurant scale.xlsx

09/04/2023

WORKSHOP 2

Design an interaction activity and a suitable way to assess the students. Specify:- Students profile- Level- Rater(s)- Time available for the activity

(classroom/home/preparation time)- How to assess the students- Objective of activity

References

Berry, V. (2004) A Study of the Interaction between Individual Personality Differences and Oral Performance Test Facets. Unpublished PhD Thesis. Kings College. The University of London.

Brown, H.D. (2004) Language Assessment: Principles and Classroom Practices, New York: Longman.

Fulcher, G. (2003) Testing Second Language Speaking. Harlow: Person Education Ltd.

Lumley, T. & McNamara, T.F. (1995) Rater characteristics and rater bias: implications for training, Language testing 12(1), 54-71.

Taylor, L. (Ed) (2011) Examining Speaking: Research and Practice in Assessing Second Language Speaking, Studies in Language Testing, volume 30, Cambridge: UCLES/Cambridge University Press.

09/04/2023

Recommended bibliography• Alarcón Pérez, C. (2007) “Evaluar la expresión oral: propuesta contra los viejos

inconvenientes. Logroño. Libro de Actas .pp.191-204.• Alderson, J.C., Clapham, C. y Wall, D (1995) Language test Construction and Evaluation.

Cambridge. Cambridge University Press. • ALTE (2001) Principles of Good Practice for ALTE examinations. www.alte.org• Calle, C. y Pomposo, L. (2010) “Estudio de la eficacia de actividades prácticas en el aula de inglés

de negocios para mejorar las destrezas orales”. Universidad de Vigo. Libro de actas de XXVIII Congreso de AESLA. Pp. 461-472. Vigo.

• Carroll, B. (1980) Testing Communicative Performance. An interim Study. Oxford. Pergamon Press• Douglas, D. (1997). “Testing speaking ability in academic contexts: theoretical considerations”.

TOEFL Monograph Series. Princeton, NJ: Educational Testing Services .• García-Debenc, C. (2010) “Evaluar lo oral”. Enunciación. Vol 15. Num 2/Julio-Diciembre. Bogotá.

Colombia. pp. 103-13• Luoma, S. (2004) Assessing Speaking. Cambridge. Cambridge University Press.• McNamara, T.F. (1996) Measuring Second Language Performance. London: Longman.• O’Malley, J.M. y Pierce, L. (1996) Authentic Assessment for English Language Learners: practical

approaches for teachers. New York: Addison Wesley.

Thanks