enlta european network for language testing and assessment

ENLTA

European Network for Language Testing

and Assessment

EALTA

European Association for

Language Testing and Assessment

ENLTA

• Funded by European Commission to create EALTA - a network of individual language testers

• Two years, December 2003 – December 2005

• 14 partner institutions

• 8 activities in the Work Plan

Activities

1. The organisation of EALTA

2. Analysis of training needs

3. 1st Annual Conference

4. Survey of assessment policy and practice

5. Develop and pilot a training model

6. Develop a code of good practice

7. 2nd Annual Conference

8. Identify a basic training resource

Presenters

• Activity 4: Survey of assessment policy and practice in Europe

• Gudrun Erickson

• Sauli Takala

• Dianne Wall

Presenters

• Activity 5: Develop and pilot a training model

• Ülle Türk

• Melanie Ellis

Presenters

• Activity 8: Identify a basic training resource

• Melanie Ellis

E N L T A – Activity 4 Survey of Policy and Practice

• Gudrun Erickson, Jan-Eric Gustafsson & Cecilia Nihlén, Sweden

• Sauli Takala, Finland

• Barbara Czarnecka-Cicha, Poland

• Tania Horak & Dianne Wall, UK

AIMS

• Links to, and short comments on previous studies of assessment cultures and practices (EALTA website)

• Links to, and short comments on official language testing sites in different European countries (EALTA website)

• Reports on [some] European students’ and teachers’ thoughts about language assessment practices (EALTA website)

Review of literature

The purpose is to provide a list of readings which deal with pupils’ views of assessment, tests and examinations.

We are mainly interested in empirical studies but also reviews, essays and position statements are of interest.

We have consulted Finnish (Linda) and Swedish (Libris) data bases as well as ERIC, Language and Linguistics Behavior Abstracts, and a couple of other sources.

•Kärkkäinen, K. & Takala, S. (1978) A feasibility study of incorporating a structures test in the matriculation examination. Reports from the Institute for Educational Research, University of Jyväskylä, 290/1978.

• IEA: Language Education Study (Ari Huhta et al.)

•Westhpal, P.B., Wacha, H. & Rhodes, P. (2002) Assessing oral performance in the secondary classroom. French Review, v. 75, n 3, 560-569.

• Kenyon, D. M. & Malabonga, V. (2001) Comparing examinee attitudes toward computer-assisted and other oral proficiency assessments. Language Learning and Technology, v 5, n2, 60-93.

• Barnes, A., Hunt, M. & Powell, B. (1999) Dictionary use in the teaching and examining of MFLs at GCSE. Language Learning Journal, 19, 19-27.

• Powell, B., Barnes, A. & Graham, S. (1996) Teachers´ views of target language testing. Language Learning Journal, 14, 3-9.

• Alderson, J.C. & Clapham, C. (1995) Assessing student performance in the ESL classroom. TESOL Quarterly, 29, 1, 184-187.

• Sjöberg, A. (2002) Functionality of language skills in occupational English: the point of view of language users, language training and language testing. PhD thesis, University of Oulu.

In order to be able to cover all the relevant literature we ask for your assistance.

Please send information about electronic data bases that we might search.

References following the model provided above are most welcome. To make the material more easily accessible, please provide a translation into English of the title (when needed). Also, whenever possible, provide a brief abstract or summary in English. If the whole publication is available, it can be sent to the following address.

[email protected]

Address: Luokotie 2

40950 Muurame

Finland

We will provide a register of the references to the EALTA membership and search for ways to analyse them an publish the results.

mailto:[email protected]

Links to official language testing resources in Europe

We have drawn up a list of sites we have found. We would be very grateful if EALTA members sent us details of any other sites they feel would be of interest and relevance. If your own organisation is missing from the list on the next pages, please send information to

[email protected]


International:

ETS Europe

International Baccalaureate Organisation

Belgium

Centre for Language and Migration

University of Antwerp Language Test Centre


Czech Republic

CERMAT (School-leaving examinations)

STANAG Examination Centre

Language School State Exams

Finland

Finnish Matriculation Examination Board


FranceFranc-Parler

GermanyDAADTestDaFThuringian Ministry of EducationUnicert


Greece

Greek State Certificate

Hellenic American Union

Hyphen


HungaryHungarian Accreditation Board for Foreign

Language ExaminationsHungarian Association of Language Examiners

and Measurement SpecialistsHungarian Exams Reform Teacher Support

ProjectInstitute of International Education, European

Office+ about 20 links to exams approved by OKI


ItalyTrinity College ItalyUniversity for Foreigners of SienaUniversity of Rome “Roma Tre”

LatviaCentre for Curriculum Development and

Examinations


Lithuania

National Examinations Centre

Public Service Language Centre

Luxembourg

Centre de Langues Luxembourg


The NetherlandsBureau ICECITO

NorwayAKSIS/ University of BergenDirectorate for Primary and Secondary

Schools


PolandCentral Examinations Commission

RussiaFederal Institute for Educational

MeasurementFederal Testing CentreUnified State Examinations


SloveniaCentre for Slovene as a Second/Foreign

LanguageNational Examinations Centre

SpainEscuela Oficial de Idiomas de AvilésInstituto Cervantes


Sweden

Göteborg University, Dept of Education

Turkey

OSYM – Student Selection and Development Centre


United KingdomCambridge ESOLCity and Guilds Pitman QualificationsDfES – Information on ESOL qualificationsDave’s ESL Café Assessment ForumFederation of Awarding BodiesInstitute of LinguistsLearning and Skills Council


United Kingdom (cont.)

QCA – Qualifications and Curriculum Authority

Scottish Qualifications Authority

Test of English for Educational Purposes

Trinity College London

Survey: Student Questionnaire

Piloted in 3 countries (Nov. 2004)

Administered in Dec. 2004 – March 2005 Referring to 1st Foreign Language

If possible, answers in English

Open-ended questions

What is good language assessment? What is bad language assessment? Why? Other comments?

Likert scales

Attitudes to and feelings about testing, assessment and achievement

Areas emphasized in testing and assessment

Distinction made between exams/formal tests andassessment during lessons

Students

1, 373 students in ten European countries

Bulgaria, Estonia, Finland, Hungary, Norway, Poland, Slovenia, Spain (Catalunya) , Sweden,

the UK

End of compulsory school

Survey: Teacher Questionnaire

Comments on

the whys, whats and hows of language testing and assessment

+

Likert scales

62 responses

Some preliminary results

• Agreement between students and teachers about what is most frequently tested / assessed:– grammar– words & phrases– reading comprehension– written production

• Agreement that assessment of oral proficiency is much less frequent, especially in formal testing situations.

Students’ perceptions

Examples of positive features in tests

• Variety

• Communication

• “Usefulness”

• Pedagogical potential

• Clarity

• Fairness

• Enough time

Students’ perceptions

Examples of negative features

• Too much focus on single aspects of language, in particular grammar

• Too little focus on communication, especially speaking

• Too little focus on active language use

• Stress

Examples of students’ comments

“A good language test is a test whiff a lot of diferent parts. Not just gramma, it is to much. The best test is an essaytest, becouse there you test a lot of different things at the same time.”

“A good test is the one which enables you to learn and you can see (find out) what you know (can). A bad test makes you nothing to learn. You do not learn anything by doing it.”

Examples of students’ comments“In my opinion, good language assessment is assessing students for all the time - during classes, not only during tests. For me, writing texts, grammar or vocabulary is the same importaint as speaking, reading or listening. A good language test is checking all language abilities. There are many people who are very good at grammer, but they have some problems with speaking; there are also people who are good at speaking but they are not good at grammer, so they can’t improve their english at such ”bad” tests. I think the best way to teach and to learn english is to encourage and to be encouraged, to interest and to be interested in language. Some tasks, projects and films(movies) make students encouraged to learn the language. ”


“I like language tests and assessments because they show how much I know about the language. – I think speaking tests are better than writing because if you listen to somebody speak you hear his/her pronounciation as well and also how he/she expresses himself/herself and of course how he/she knows grammar.”

“BAD:-fast, I mean not enough time, so generating stress-veeery narrow domain-only ONE answer is correct.”


“This ENLTA/student questionaire doesen’t like me either because probably it’s never read by someone other than me so basicly it wastes our lesson time and don’t do any good for me or my classmates”

“Thank you for taking time to lisen one my toughts.”

“This was nice questionnaire!”

“Good luck with the reasearch!”

ENLTA Activity 5

Survey of training needs:

Develop and pilot a model for training events and modes of delivery

The team

• Györgyi Együd (Hungary)*¤#• Melanie Ellis (Poland)*#• Neus Figueras (Spain)+• Sara Gysen (Belgium)• Angela Hasselgreen (Norway)*• Günter Nold (Germany)*• Ülle Türk (Estonia)*#• Norman Verhelst (the Netherlands) (Classical Test

Theory)#• * prepared materials• + reviewed materials• ¤ organised the training event• # delivered training

Tasks• Plan training events and modes of delivery• Design pilot training events and time schedule of

delivery• Deliver pilot training event• Report to 2nd conference

• Two face-to-face meetings:– Kranjska Gora (May 2004): initial planning– Frankfurt (September 2004): detailed planning

Initial decisions• Two separate events:

– Introduction to classical test theory – Norman Verhelst– Introduction to testing and assessment – the rest of the

team

• A three-day event to take place in Szeged, Hungary in March 2005

• The target group: foreign language teachers who have had no previous training in testing and assessment

• Classroom assessment, self-assessment and ELP included

Day One

• 09.30–11.30 Introduction to testing and other forms of assessment

• 12.00–13.30 Introduction to assessing receptive skills

• 14.30–16.00 Assessing receptive skills: Test formats, task types, principles of scoring, issues related to scoring

• 16.30–18.00 Classroom assessment including ELP – focus on listening

Day Two

• 09.00–10.30 Introduction to testing productive skills

• 11.00–12.30 Testing writing: focus on tasks

• 13.30–15.00 Testing writing: focus on assessment

• 15.30–17.00 Classroom assessment of writing

Day Three

• 09.00–10.30 Testing speaking: what and how to test

• 11.00–12.45 Testing speaking: assessment

• 13.30–15.00 Classroom assessment of speaking

Timeline

• First draft 30 November

• Feedback 10 January

• Second draft 25 January

• Feedback from Neus 10 February

• Final version 01 March

• Training event in Szeged 10–12 March

Lessons learnt• 22 hours too little:

– More time for reading and listening needed– No time for classroom assessment– ELP – a separate issue

• Possible solutions: – Exclude classroom assessment– Increase the number of hours

• A five-day event more realistic– Delivered as one event– Two modules:

(1)General principles and testing receptive skills(2)Testing productive skills

• Local needs– Core materials supplemented by local materials

Activity 8: Resources

• Melanie Ellis– Foreign Language Teacher Training College,

Zabrze, Poland

• Laurence Kane– University of Dortmund, Institute for English

and American Studies

• Milena Ivanova Grigorova– New Bulgarian University

Aims of the evaluation

• To evaluate the pilot materials in use

• To evaluate the training event

• To gather ideas for resources teachers need to support their learning about testing and assessment

How did we carry out the evaluation?

• Observation and Feedback sheets– timing, order of activities, choice of interaction– content evaluation: comments, suggestions,

questions– overall impressions: two perspectives- the

observer and the trainer

• Feedback sheets on each session for participants

• End of course feedback sheet

Qualitative evaluation

• Focus group discussions at end of each day

• Introductory session task and final reflection taskBy the end of the course I’d like to

know more about...

have learnt...

be able to...

End of course feedback

• I enjoyed the course5 (13) 4 ( 5)

• I found the course useful5 (10) 4 (8)

• I learned new things5 (6) 4 (7) 3 (5)

• The classes were well-organised5 (11) 4 (6) 3 (1)

ENLTA Activity 8

As part of Activity 8 we are preparing a bibliography for teachers who want to develop their knowledge of and skills in testing. The result will be available on the EALTA wesite. Please help us by listing any book, CD, tape, video, computer program or website you have used and would recommend.

Send your response [email protected]. and put 'Bibliography' in the Subject line

Thanks

enlta european network for language testing and assessment

Documents

language training

assessment slide

language users

target language testing

language learning journal

basic training resource

work plan slide

survey of assessment