language ability & content knowledge by ivana vidakovic at iatefl besig teasig pce 2015

Language ability and content knowledge in an ESP

classroom: Two sides of the coin?

Ivana Vidaković BEASIG and TEASIG, IATEFL Manchester 2015

Overview

• Insights from English for Specific Purposes (ESP)

assessment – Key features of ESP tests – The effect of content knowledge on linguistic performance – Roles of content knowledge and language ability in

Cambridge English ESP tests

• Insights from ESP classrooms – Teachers’ voices: how much does a teacher need to know?

3 key features of ESP tests

Authenticity

Content knowledge Specificity

Authenticity has two faces

Situational authenticity

Interactional authenticity

The real life quality of

tasks: content,

language, context, etc.

Test takers interact with the task as in real life

Does authenticity matter?

• Any language test should have both kinds of authenticity • Authenticity allows one to generalise from test

performance to performance in real life • Full authenticity cannot be achieved

– Practicality constraints – Finding a right balance


Authenticity


What makes a test specific?

• What goes into a task and what comes out • Relevant and accessible texts, tasks and topics • Lexis: (semi-)specialised; quantity; explained or not? • Topic and the treatment of the topic • Text structure

• Problem-solution; structure of a legal letter of advice • Task purpose and audience

• E.g. Provide recommendations to the patient … • Channel of communication

• Visual – graphs, pictures; audio; verbal • Source of the text

• E.g. A business journal

Adapted from O’Sullivan (2006:178)

Generalisability

Increase in content knowledge required

Degree of specificity Very specific Non-specific

∞ ∞

A General English Test

e.g. PET

A test of English for Air Traffic

Controllers

Specificity varies on a continuum


Authenticity


Content knowledge in language tests

• Important not to test this in general English tests • Content knowledge is integral to ESP tests • ESP tests could assess:

– Content knowledge and language ability (e.g. PLAB) – Only language ability in a specific context

Teasing out language ability from content knowledge is a delicate task

Language ability

Content knowledge (business, legal)

Specific purpose language ability

Overview

• Insights from ESP assessment

– Key features of ESP tests – The effect of content knowledge on linguistic performance – Roles of content knowledge and language ability in



Content knowledge has a facilitating effect on ESP test performance

• Test-takers with the relevant content knowledge score

higher on discipline-specific tests of: – Reading (Alderson & Urquhart 1985, Krekeler 2006) – Listening (Jensen & Hansen 1995) – Speaking (Papajohn 1999) – Writing (Tedick 1990)

The influence of content knowledge is not always consistent

• Test-takers from a specific discipline do not always do

better on a text/task from their discipline than test-takers from a different discipline

• The influence varies with:

– Text/test specificity – Language proficiency

Language proficiency and content knowledge interact

• Test-takers with …

… intermediate language proficiency: strong effect of content knowledge … basic language proficiency: no effect of content knowledge … advanced language proficiency: enough linguistic proficiency to compensate for the lack of content knowledge – mixed findings

Overview





Degree of specificity (impression-based) Very specific Non-specific

∞ ∞

CE ESP tests on the specificity continuum

Increase in content knowledge required More specialised content knowledge

KET, PET

CAE IELTS Academic

BULATS, BEC

ILEC,

ICFE

OET

Variations in specificity: Speaking tasks

BEC What is important when selling a new product?

• Price/Quality/Advertising

What is important when choosing a training course? • Size of group, cost, times

of course

ILEC

Negotiable instruments • The meaning of… • The importance of a

promissory note • How a bill of exchange

works

The Internet. • The effect email has had

on the practice of law, how people in the legal profession make use of the internet, etc.

Test More specific Less specific

Variations in specificity: Writing tasks

012345

Purposetopic

subjectknowledge

lexis

grammarfunctional

dicourse mode

channel

writer-readerrelationship

BULATS (Task A)

012345

Purposetopic

subjectknowledge

lexis

grammarfunctional

dicoursemode

channel


IELTS (Task C)

012345

Purposetopic

subjectknowledge

lexis

grammarfunctional

dicoursemode

channel


Skills for Life (Task B)

012345

Purposetopic

subjectknowledge

lexis

grammarfunctional

dicourse mode

channel


ILEC (Task D)

The role of content knowledge: Views from ILEC test takers

• Familiarity with legal text structure, terminology and concepts helped in the ILEC Reading test

‘I found that I was familiar with phrases and concepts which helped me to answer the questions. I was also familiar with some of the cases.’ ‘Familiarity with the phrases makes it much easier to answer the questions … those unfamiliar with this terminology may struggle.’ • ILEC Reading test can be accessed by those with no legal

knowledge ‘Lawyers have the advantage of being familiar with the terminology but it’s not a necessary attribute. You don’t have to know every single word to answer the questions.’

The role of content knowledge in CE ESP tests

Content knowledge assessed?

Content knowledge

accessed?

Basis for answering comprehension questions

No In some R, L, S and W tasks to process text faster, to enrich meaning

Yes

Basis for assessing S and W performance

No To produce language in S and W tasks

Yes

Test understanding through language rather than fine points of technical expertise Ground questions in the text Avoid very specialised texts

Reading/ Listening

comprehension

Create linguistic assessment criteria: Coherence, cohesion, relevance Intelligibility Range (grammar, vocabulary) Appropriateness, etc.

How to separate language ability from content knowledge?

Speaking/ Writing ability

Insights from ESP assessment:

How specific should an ESP test be?

• Too specific: – Suitable for a select few test-takers – Inadequate performance due to deficiencies in language

ability or content knowledge? – Difficult for language professionals to identify suitable texts

and write questions – Difficult for language professionals to assess Speaking and

Writing • Decision to be guided by:

– Characteristics of the test takers: content knowledge, specialism, language ability

– Test purpose and the intended test uses – Practicality issues – Fit to the ESP course (teacher-designed tests)

Insights from ESP assessment: Research and collaborate

• Needs analysis (OET, ILEC, ICFE) – Challenges: Resources

• Collaboration with content specialists throughout the test development and production cycle (ILEC, ICFE) Challenges: – Accessible and affordable content specialists – Content specialists with a sense for language – Different perspectives and priorities

Overview





Participating teachers

• N=9 (7 female) teachers responded to an online survey • L1: English (n=7) and German (n=2) • Location:

– Germany (8/9) and Switzerland (1/9) – 5 cities in Germany, 8 different employers

• Degree: 8/9 with a postgraduate degree • Teaching qualifications: 5/9 have them

Teachers’ ESP/ESAP background

• Experienced teachers of ESP/ESAP: 4-15 years • Taught a wide range of ESP/ESAP areas:

BusinessLegalMedicalEngineering (mechanical, architecture, civil)FinancialITTaxChemicalHRProject ManagementPsychologyHuman biologyCaregivers for special needsTelecommunicationsMuseum professionalsGeographyOptometry

Teachers’ content knowledge

n=5 n=4

No qualification nor work experience in thediscipline/profession of own studentsWith qualification and/or work experience in thediscipline/profession of own students

Challenges for ESP/ESAP teachers

• The teacher’s insufficient knowledge of profession-specific… … subject matter … terminology … communicative tasks … speaking and writing conventions & practices

• Catering for non-homogenous groups of learners in an ESP or ESAP course

How teachers address the challenges

• Own research – The most frequent resource for addressing all the

challenges (7/9 teachers) • Input from students

– Students teach the teacher content and terminology (4/9 teachers)

• Input from experts – More frequently sought for communicative tasks,

conventions and practices (4/9 teachers)

Own research

• Read basic course books, refresher material, discipline-specific (research and professional) journals

• Use the Internet:

– Websites (e.g. webmd, nhs) – YouTube videos of authentic tasks, lectures, discussions – Online simulations – Company reports

• Attend discipline-specific conferences • Attend seminars and lectures to learn more about the subject

‘I research…as far as I can.’

Input from students

• Students teach the teacher by: – Explaining content and discipline-specific issues – Explaining terms and expressions to the teacher and the class – Identifying discipline-specific tasks they have to do – Identifying challenges with such tasks – Supply the teacher with material (emails, letters, reports)

• Teacher uses framework material, students supply the content

‘I make the students the experts who have to teach me about the subject matter…’ ‘...You can get deeper insights into the subject matter if you swap roles with your students and let them explain their professional knowledge in English. This is a win-win situation.’

Input from experts

The surveyed ESP/ESAP teachers … • Interact with professionals

– Interview them to identify the key tasks and their features – Volunteer for research experiments – Draw on personal experiences as clients or patients

• Observe professionals – Record conferences, lectures, discussions – Ask professionals to record conversations and other

communications – Observe professionals at work (meetings, phone calls)

• Collaborate with subject teachers ‘The best way is to collaborate with a subject teacher. I have done this and designed a class together with the subject expert.’

Challenge-specific insights

• Terminology – Build a terminology database – Exercise terminology with your students – Do not teach very specialised terminology – Focus on skills; students learn terminology mostly on their own

(for homework) • Content knowledge and terminology

– Know your limits: do not pretend to be an expert in the subject ‘I try to be honest about how far I can go with specialised terminology.’ ‘I teach tax professionals English but don't try to teach tax in English to tax students.’

Other challenges: Lack of resources

‘Being thrown in the deep end as an inexperienced teacher on low hourly rates without suitable material and being expected to write ESP courses. I just struggled through and focused on more general language skills.’ ‘A continuous challenge has been finding appropriate material to use for 1st/2nd semester students. Sometimes I actually know more than my students and they have criticised that I am asking too much of them.’ ‘I work as a free-lance teacher, never knowing from one semester to the next just what my course combination will be. This has been most unsatisfying, never having enough time to delve into a new subject area.’

Other challenges: Non-homogenous groups

Different specialisms and levels of content knowledge • Agree on focus to suit everyone • Mix students: those with more content knowledge helping

those with less • Pair work: students explain their specialisms to each other • Different materials and homework tasks • Individual feedback

Should ESP/ESAP teachers have specific subject-matter knowledge?

4

0

3

Yes No Depends

How much subject-matter knowledge should an ESP/ESAP teacher have?

4 3

1

The more the better Enough to… Basic

How much subject-matter knowledge should a teacher have?

The more the better:

‘As much as possible to deliver a meaningful course in context which is both exam preparation and professional life preparation.’ ‘A substantial amount, otherwise communication will break down.’ ‘It certainly adds to their authority when they can claim experience and can comfortably use the specific vocabulary.’ ‘A teacher does not have to be an expert, but should have some solid knowledge.’

How much subject-matter knowledge should a teacher have?

• Enough to…

‘…accomplish the learning task(s) required by the group.’ ‘… judge accuracy of vocabulary use and text style.’ ‘… follow what students are saying and ask intelligent questions.’

Key insights from ESP classrooms

• Do own research

• Seek input from students • Seek input from content specialists

More insights from ESP classrooms

• Be open ‘Explain you are a language teacher, not a biologist.’ ‘Be very open and flexible and prepared to accept students’ advice and opinions.’

• Exploit your lack of knowledge ‘By challenging the students to help [you] gain a greater understanding, they [will be] required to use and expand their repertoire of language and their communication skills.’

• Be willing to learn

‘The teacher has to be at least interested in the subject and be willing to learn from the students.’ ‘It is most important that a teacher is willing to learn continuously in the subject matter area.’

Overlap between ESP teaching and assessment issues

Your thoughts

Thank you!

[email protected]

language ability & content knowledge by ivana vidakovic at iatefl besig teasig pce 2015

Education