marleen colpin & machteld verhelst

© 2005

Assessing the language proficiencyof young learners

in the Dutch classroom:developing and implementingtasked-based language tests

Marleen Colpin & Machteld Verhelst

© 2005

CONTENTS

PART I (Machteld Verhelst)

• Background & principles• Assessing very young children’s language

proficiency

PART II (Marleen Colpin)

• Developing and implementingtasked-based language tests in the classroom

• Conclusion

© 2005

• Its task: promoting equal educational chances for all learners

• Most important target group: learners at risk in their educational career (immigrants and learners from low social classes)

• Providing support to the field of primary, secondary and adult education

Centre for Language and Migration

© 2005

• Focus on successful language performance in order to function in relevant, communicative situations (school, society)

• Language as a device, not a goal in itself • Becoming proficient through carrying out

realistic, relevant and motivating tasks • Task-based approach for every learner

regardless of his/her language background

Task-based approach of CTM

© 2005

Task-based assessment

• Task is central concept, not construct of LP

• Not to elicit components of system• Interest in performance on task itself

© 2005

Principles

Three principles concerning development of language tests and assessment instruments:

1. Embedded in education 2. Implementation3. Impact

© 2005

Three interrelated basic questions to start with:

• WHY is a test needed? purpose• WHAT should be tested? content• HOW should language proficiency be

tested? format

© 2005

• To monitor the language development process and diagnose possible problems

• To provide feedback and specific support on the basis of diagnostic information perspective of learner: to avoid problems and/or to overtake these perspective of teacher: to adjust own education

• To take decisions• To inform parents• ...

WHY?

© 2005

Why do teachers evaluate?

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

© 2005

WHAT?

• Language performance in relation to Intermediate goals / Governmental attainment goals / Framework of Reference for young learners

• No global level system such as the CEFR but goals in terms of the language tasks that learners should be able to perform at a certain age within and outside the school context

© 2005

Which goals do teachers assess?

Kindergarten Primary School

© 2005

HOW?

• The most appropriate procedures/instruments to test/assess with regard to

- WHY you want to test- WHAT you want to test- but also the GROUP of PUPILS that will be

tested

• principles of good testing and assessment: validity, reliability, efficiency and practicality

© 2005

Assessing very young learners’ language proficiency: a broad view…

Social

Cognitive Language

© 2005

WHY?

Purpose:to detect children at risk in order to

create a more powerful language environment

© 2005

WHAT?

‘Attainment’ or ‘developmental’ goals

Language use

In the school domain

Comprehension

No correctness

© 2005

Examples of attainment goalsend kindergarten• Listening

A child can understand an oral instruction about a concrete physical action in a here-and-now situation, and is able to show its comprehension by reacting adequately.

• SpeakingA child can answer questions about its

own feelings, intentions and interests, in a communicatively adequate way.

© 2005

What do teachers test?

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

© 2005

HOW?

Specific demands concerning:

(1) Validity, reliability and bias• Tasks• Test format• Cultural/social bias

Avoiding typical culture-bound elementsThemes and contexts that all children know

© 2005

(2) Efficiency and practicality

Tests demand much of the concentration of young children

as short and as natural as possible

variation and motivation

© 2005

(3) Embedding in educational context

• Possibility to draw educational consequences?

• Education effects visible?• Evolution visible? • Enough differentiation?

© 2005

Example 1: KOBI-TV

• Age 4• Language proficiency in the school

domain • Integrated oral skills• 27 tasks: real language use situations• Administration within the natural class

environment of the children• Allows monitoring during the school year

© 2005

100

© 2005

TASK 7

© 2005

‘Take three green blocks.Put these blocks in a row.’

TASK 17

© 2005

TASK 24

© 2005

TASK 27

© 2005

The standards of KOBI-TV

December March June

11 16 1919 23 25

December March June

18 22 24

Immigrant children

Flemish children

All children

© 2005

Developing and implementingtasked-based language tests in the

classroom

Marleen Colpin

© 2005

VLOTVolgsysteem Lager onderwijs

Taalvaardigheid to monitor the language development of primary school pupils (8-12 year olds, L1

and L2 learners) a battery of 60 tests designed for teachers in relation to the official attainment goals

© 2005

Examples of attainment goalsfor the end of primary education

• ListeningThe pupils can understand and structure the information from an explanation or an instruction of the teacher.

• WritingThe pupils can write a letter to a familiar addressee to communicate a personal message or experience.

© 2005

• Relevant and authentic assessment tasks which relate to the official attainment goals

• Criteria/parameters for difficulty:

Text processing level Text type Public

Familiarity of topic and theme Vocabulary Reading difficulty level (AVI-level)

VLOT

© 2005

Unequal division between receptive and productive tests

Tasks with a communicative goalConfronting children with language they

encounter in real life (their world): motivation

60 tests: 1 test for each grade (5 grades) for the 4 main language skills

at 3 moments during the school year

VLOT

© 2005

Direct testing to enhance content validityPractical and efficient to be administeredTarget group: L1 and L2 learners

avoiding bias Comparable test results to monitor pupilsNeed for efficient rating scheme that

guarantees objective and reliable rating

VLOT

© 2005

Example test: Birthday partyOral instructions from the test administrator/teacher:“… You will now call X (name) on the phone to askhim/her whether he/she can come. You will tellhim/her everything he/she needs to know to come toyour party. So you will have to mention when andwhere the party will be, what time it begins and ends,if the children need to bring something, e.g. apresent, ….I will play X. I will first give you a little time to plan, sothat you can prepare what you are going to say.”

VLOT

© 2005

Comparable test resultsResults gathered from a group of more than 600 pupils in 32 schools representative for Flemish population 5 standard categories

A = goodB = average goodC = average weak D = weakE = very weak

VLOT

© 2005

VLOT

© 2005

Rating table Birthday Party task

1. Why? (the friend is invited) 1/0 1/02. Where? (the party takes place) 1/0 1/03. When? (the party takes place) 1/0

1/04. At what time? (the party starts) 1/0 1/05. What to bring along? 1/0 1/0

+ speaking fluency: 1/0

content form

VLOT

© 2005

Implementing task-based tests

Multidirectional relationships between tests,educational programme and learning goals

>< tradition of focus-on-formS language teaching methodology

>< control+ high impact and washback on views

and practice of education

© 2005

TASANTaalvaardigheidstoets Aanvang

Secundair Onderwijs Anderstalige Nieuwkomers

Test entrance secondary education non-native newcomers

Purpose: to yield information about the linguistic skills of a non native newcomer after one year preparatory class and to diagnose problematic aspects

© 2005

Parameters of task complexity Parameters concerning the world

represented in the task (e.g. ‘here-and-now’ versus ‘there-and-then’; amount of visual and linguistic support)

parameters with regard to the tasks’ cognitive processing demands

parameters with regard to linguistic complexity (vocabulary, syntactic structures, length of text)

TASAN

© 2005

Task Complexity IndicatorTASAN

© 2005

TASAN In-service trainings

Links between tasks in the test and attainment goals

Tasks in the test = examples of the tasks to be used in language education

Illustration of manipulation of complexity

© 2005

Teachers developing tests?

Difficulties:- finding or selecting proper source materials- avoiding test bias- establishing the difficulty of the test- constructing independent items- constructing objective and task-relevant

checklists for scoring productive skills

Arguments pro:+ Carefully consider attainment goals+ Elaborate professional proficiency with regard to testing

© 2005

Broad concept of evaluation

on a permanent base as well as at specific moments throughout the school year

on products the pupils produce as well as on their acquisition/learning process

with more traditional tests as well as more ‘alternative’ evaluation instruments, such as observation instruments, portfolio, ...

by the teacher as well as the pupils themselves or other pupils

© 2005

Teachers’ assessment skills

Insight in what relevant tasks are andwhat constitutes their complexity

more refined observation skillsbase assessment of pupils’ language

proficiency upon a rich and varied base of ‘real’ language use situations in the classroom

© 2005

ConclusionGive teachers insight in:• What attainment goals are claimed to be

evaluated?• How was the test constructed?• What aspects of learner performance are

assessed?

Task-based assessment as a powerful tool to promote more functional,

performance-based assessment, and task-based language education