evaluation: the hinge on which learner autonomy turns · evaluation: the hinge on which learner...
TRANSCRIPT
18 March 2017
University of Strasbourg, LANSAD Teachers
Evaluation: the hinge on which
learner autonomy turns
David Little
Trinity College Dublin
Ireland
18 March 2017
University of Strasbourg, LANSAD Teachers
Overview
• My answers to some of your questions
• Language learner autonomy and self-assessment
• The Common European Framework of Reference for
Languages, language learner autonomy and self-
assessment
• Constructive alignment of curriculum, teaching/learning
and evaluation/assessment
• Implementing constructive alignment in five steps
• Conclusion: some questions for you
18 March 2017
University of Strasbourg, LANSAD Teachers
My answers to some of your questions
18 March 2017
University of Strasbourg, LANSAD Teachers
• L’évaluation prend trop de temps aux enseignants, aux dépens de
l’accompagnement (modalités d’accompagnement non exploités faute
de temps : entretiens-conseil, ateliers méthodologiques, autres
ateliers, etc.)
– In my view evaluation/assessment should play a fundamental role in all
teaching and learning
• L’évaluation intermédiaire (de type validé / non validé) est sommative
(obligatoire) de fait mais est aussi formative (fait partie de
l’accompagnement) dans l’esprit des enseignants : l’étudiant le
comprend-t-il ainsi et en profite-t-il vraiment ? Devrait-on
obligatoirement scinder les moments d’évaluation formative et
sommative ?
– Self-assessment, peer assessment, teacher assessment and external/
official exams should exist on a continuum and be mutually supporting
18 March 2017
University of Strasbourg, LANSAD Teachers
• Y a-t-il une contradiction fondamentale entre la liberté (de choix)
nécessaire au développement de l’autonomie et la contrainte
institutionnelle de la notation ?
– For me, learner autonomy in formal educational contexts is always
framed by institutional considerations, including exams, but those
considerations need not be obstacles
• Peut-on évaluer l’autonomie ? Laquelle (autonomie langagière,
autonomie d’apprentissage) ? Est-ce éthique de le faire d’évaluer
l’autonomie ?
– In my view evaluation/assessment should focus on learning outcomes,
not on the degree of autonomy achieved by the learner
• Faut-il être un spécialiste pour évaluer ? Un spécialiste de la langue ?
Un spécialiste de l'enseignement/apprentissage ? Les deux ?
– Both, given the inseparability of teaching/learning and assessment
18 March 2017
University of Strasbourg, LANSAD Teachers
• L’évaluation fait-elle partie de l’apprentissage, est-elle un levier de
l’apprentissage ?
– Yes and yes!
• Quel est le meilleur moment d’évaluation pour l’apprenant ?
– In reflective learning evaluation is ever-present (self-assessment, peer
and teacher assessment), but summative assessment is probably most
useful if it comes at the end of a course/module
• Comment conjuguer apprentissage, évaluation et émulation ?
– That is the question I will now try to answer
18 March 2017
University of Strasbourg, LANSAD Teachers
Language learner autonomy
and self-assessment
Success in
formal learning
at any level and
in any domain
depends on
the learner’s
ACTIVE
ENGAGEMENT
Identity • Sense of self
• Unique complex
of knowledge
and experience
Evaluation • Reflection
evaluation
self-assessment
Reflection • Thinking about
what you are
doing: planning
and monitoring
Interaction • Engagement in
communication
• Learners share
initiative
Success in
formal learning
at any level and
in any domain
depends on
the learner’s
ACTIVE
ENGAGEMENT
Identity • Sense of self
• Unique complex
of knowledge
and experience
Evaluation • Reflection
evaluation
self-assessment
Reflection • Thinking about
what you are
doing: planning
and monitoring
Interaction • Engagement in
communication
• Learners share
initiative
LEARNER AUTONOMY an ever-expanding
dynamic capacity
Identity
Evaluation Reflection
Interaction
LEARNER AUTONOMY an ever-expanding
dynamic capacity
Identity
Evaluation Reflection
Interaction
LANGUAGE
LEARNER AUTONOMY
Target language as
channel of learner’s
AGENCY
Language learner autonomy as
discourse and agency
Target language use
(communicative, metacognitive, self-regulatory; dialogic, monologic)
Learner engagement (needs, investment,
identity)
Learner reflection (planning, monitoring
evaluation)
What are
we doing?
Why are we
doing it?
How are we
doing it? With what
results?
What next?
Stimulating
reflection: five questions
(Dam 1995)
What are
we doing?
Why are we
doing it?
How are we
doing it? With what
results?
What next?
Stimulating
reflection: five questions
(Dam 1995)
Learner logbooks −
record of individual
learning – Content of lessons
– Words etc. to be
memorized
– Plans for homework
– Evaluation of own
progress
– Especially in the early
stages, the texts they compose
– As far as possible in TL
Posters − stimulate,
guide and record
learning of class – Aims and objectives
– Plans (lessons, group
work)
– Ideas (useful activities)
– Things to remember
– Learners’ own
experience (e.g. definitions of
good group work)
– Help (useful
expressions)
Tools to support the exercise and
development of language learner autonomy (Dam 1995; Little, Dam & Legenhausen 2017)
18 March 2017
University of Strasbourg, LANSAD Teachers
The Common European Framework of
Reference for Languages, language learner
autonomy and self-assessment
“Language use, embracing language learning, comprises the
actions performed by persons who as individuals and as social
agents develop a range of competences, both general and in
particular communicative language competences. They draw
on the competences at their disposal in various contexts under
various conditions and under various constraints to engage in
language activities involving language processes to produce
and/or receive texts in relation to themes in specific domains,
activating those strategies which seem most appropriate for
carrying out the tasks to be accomplished. The monitoring of
these actions by the participants leads to the reinforcement or
modification of their competences” (Council of Europe 2001: 9)
The CEFR’s action-oriented approach
“Language use, , comprises the
actions performed by persons who as individuals and as social
agents develop a range of competences, both general and in
particular communicative language competences. They draw
on the competences at their disposal in various contexts under
various conditions and under various constraints to engage in
language activities involving language processes to produce
and/or receive texts in relation to themes in specific domains,
activating those strategies which seem most appropriate for
carrying out the tasks to be accomplished. The monitoring of
these actions by the participants leads to the reinforcement or
modification of their competences” (Council of Europe 2001: 9)
• L2 proficiency develops from sustained interaction
between the learner’s competences and the
communicative tasks whose performance requires
him or her to use the target language
• Language use is autonomous behaviour
• As a variety of language use, L2 learning should also
be rooted in autonomous behaviour
The CEFR’s action-oriented approach
“Language use, embracing language learning, comprises the
actions performed by persons who as individuals and as social
agents develop a range of competences, both general and in
particular communicative language competences. They draw
on the competences at their disposal in various contexts under
various conditions and under various constraints to engage in
language activities involving language processes to produce
and/or receive texts in relation to themes in specific domains,
activating those strategies which seem most appropriate for
carrying out the tasks to be accomplished. The monitoring of
these actions by the participants leads to the reinforcement or
modification of their competences” (Council of Europe 2001: 9)
The CEFR’s action-oriented approach
“Language use, embracing language learning, comprises the
actions performed by persons who as individuals and as social
agents develop a range of competences, both general and in
particular communicative language competences. They draw
on the competences at their disposal in various contexts under
various conditions and under various constraints to engage in
language activities involving language processes to produce
and/or receive texts in relation to themes in specific domains,
activating those strategies which seem most appropriate for
carrying out the tasks to be accomplished.
” (Council of Europe 2001:
9)
• In autonomous learning monitoring begins as a
conscious process of self-management
• But using the TL as the channel of that explicit
monitoring helps to develop the capacity for
involuntary and implicit monitoring that is fundamental
to spontaneous/autonomous language use
• In this way, self-assessment becomes a habit of mind
and a fully integrated part of L2 proficiency
The CEFR’s action-oriented approach
Example 1, B2 reading • I can quickly scan through long and complex texts on a variety of
topics in my field to locate relevant details
– What is “my field”?
– What is an appropriate “variety of topics”?
– In terms of field and topics, what counts as a “long and complex text”?
– What count as “relevant details”?
– Why do we need to answer these questions?
– What are our current reading skills?
– What do we need to add in order to achieve B2?
– How can we best go about developing those additional skills?
– How will we monitor our progress?
– Etc., etc.
CEFR-based self-assessment and
Dam’s five questions: two examples
Example 2, B2 writing • I can write clear detailed text on a wide range of subjects relating
to my personal, academic or professional interests
– What are “my academic and/or professional interests”?
– What is an appropriate “range of subjects”?
– How do we define “clear detailed text”?
– Why do we need clear answers to these questions?
– What are our current writing skills?
– What new elements do we need to master in order to achieve B2?
– How can we best master those elements?
– How will we know when we have mastered them?
– Etc., etc.
CEFR-based self-assessment and
Dam’s five questions: two examples
18 March 2017
University of Strasbourg, LANSAD Teachers
Constructive alignment of curriculum,
teaching/learning and evaluation/
assessment
18 March 2017
University of Strasbourg, LANSAD Teachers
Key features of constructive alignment (Biggs & Tang 2011)
• Intended learning outcomes (the competences students are
required to develop) defined at four levels
1. the best that can reasonably be expected
2. highly satisfactory
3. moderately satisfactory
4. minimally satisfactory
• Verbs define the competences at each level
1. hypothesise, reflect, relate to principle
2. solve expected problems, explain complex ideas, apply to professional
practice
3. solve basic problems, explain basic ideas, use standard procedures
4. inadequate but salvageable higher level attempts
• Objects of verbs = course content
18 March 2017
University of Strasbourg, LANSAD Teachers
Key features of constructive alignment (Biggs & Tang 2011)
• Teaching/learning activities – designed to generate or elicit
the activities defined by “competence verbs” – Large classes, small classes, group work, individual activities
– Teacher-managed, peer-managed, self-managed
• Assessment tasks – designed to elicit and display the
activities defined by “competence verbs”
• Assessment criteria – specified to distinguish clearly between
the different competence levels
• Three stages in assessing student performance – Setting the criteria
– Selecting the evidence
– Making a judgement
18 March 2017
University of Strasbourg, LANSAD Teachers
The status of self-assessment • According to Biggs & Tang (2011: 245)
1. Self- and peer-assessment give the students first-hand, active
involvement with the criteria for good learning
2. Students learn how to select good evidence
3. Judging whether a performance of product meets given criteria is vital
for effective professional action
• “Self-assessment, while commonly portrayed as a
technique to enhance learning, is more transformative,
elusive and confronting to conventional teaching than it
is normally expedient to recognise” (Boud 1995: 1)
• On the other hand, if our teaching is calculated to exploit
and develop learner autonomy, self-assessment will be
one of its foundational features and values
18 March 2017
University of Strasbourg, LANSAD Teachers
The CEFR as a tool of constructive alignment
• For the CEFR learning and assessment are inseparable
• Each of the CEFR’s “can do” descriptors can be used to
– specify a learning outcome
– provide a learning focus
– imply an assessment task
• Learners themselves can participate fully in this culture of
learning and assessment because from early childhood we
know what we can and cannot do behaviourally
• The European Language Portfolio was conceived as a way of
mediating the CEFR’s ethos and approach to learners and
enabling them to manage their own language learning on the
basis of reflection driven by self-assessment (Little 2005, 2009,
2011)
18 March 2017
University of Strasbourg, LANSAD Teachers
Implementing constructive alignment
in five steps
18 March 2017
University of Strasbourg, LANSAD Teachers
Step 1: Foundations • Define the programme in terms of
– Content: the body of knowledge that students are required to engage with
and master, defined in terms of discipline(s), sub-disciplines, concepts,
texts, procedures, etc.
– Competences: the skills that learners are required to develop in relation
to programme content
• Cognitive, metacognitive, interactive, collaborative (Biggs & Tang
2011)
• Language activities and communicative/metacognitive language
competences (CEFR; Council of Europe 2001)
• If the programme lasts more than one semester
– Divide content into appropriate segments
– Define competences for each segment
– Pay attention to the various kinds of progression that your division is likely
to imply: content, competences, language proficiency
18 March 2017
University of Strasbourg, LANSAD Teachers
Step 2: Applying the CEFR • Use the levels and scales of the CEFR to determine the
L2 capacities students should have (reception, interaction,
production) – at the beginning of the programme
– at the end of each segment
– at the end of the programme
• Be realistic! – B2 is an acceptable minimum level of proficiency at the end of a degree
programme provided it is appropriately focused:
• Can give clear, detailed descriptions and presentations on a wide range of
subjects related to his/her field of interest, expanding and supporting ideas with
subsidiary points and relevant examples (Council of Europe 2001: 58)
• Can write clear, detailed texts on a variety of subjects related to his/her field of
interest, synthesising and evaluating information and arguments from a number
of sources (ibid.: 61)
18 March 2017
University of Strasbourg, LANSAD Teachers
Step 3: Programme design • With the requirements of learner autonomy always in mind,
design a programme of teaching and learning that
– assigns content to large classes, small classes, group work, individual
activities
– explains the relation between these different modes of teaching/learning
– explains how students are expected to engage with each of them
receptively, interactively and productively
– provides additional activities (classes, self-access) to support students
who are in danger of falling short
• At each step of programme design, answer these questions:
– How will students explicitly accept responsibility for their learning?
– How will they be led to engage reflectively with the learning process?
– How will self-assessment and peer assessment be built into the
dynamic of teaching and learning?
18 March 2017
University of Strasbourg, LANSAD Teachers
Step 4: Supporting learner autonomy
• If we want learner autonomy to be central to students’
academic experience, we we must help them to – document the learning process
– manage their own learning
– engage in self-assessment and peer assessment using criteria that are
fully harmonious with the criteria applied in teacher and institutional
assessment
• Tools to support these procedures include – portfolios and e-portfolios whose structure reflects the structure of the
programme in question
– checklists of “I can” descriptors arranged according to activity and
CEFR level (use generic descriptors and interpret them with specific
reference to programme content and target competences)
18 March 2017
University of Strasbourg, LANSAD Teachers
Step 5: Assessment • Design assessment tasks that elicit and display the target
competences
• Design rating scales that are fully harmonious with the
“I can” checklists, so that there is a clear continuum from self-
assessment through peer and teacher assessment to
institutional assessment
• Incorporate self-assessment into the reporting of outcomes
• Note that
– It is not a matter of confirming the accuracy or otherwise of self-
assessment but of acknowledging the central role it plays in the
teaching/learning process
– If self-assessment and peer assessment are foundational to teaching and
learning, there is no reason why they should lack either validity or reliability
(see Dam & Legenhausen 2010)
18 March 2017
University of Strasbourg, LANSAD Teachers
Conclusion
18 March 2017
University of Strasbourg, LANSAD Teachers
The argument in a nutshell
• Evaluation/assessment should be inseparable from
learning and teaching
• In autonomous learning, self-assessment, peer
assessment, teacher assessment and external/
institutional exams should exist on a continuum and be
mutually supporting
• This implies a need for constructive alignment of
curriculum, teaching/learning and evaluation/assessment
18 March 2017
University of Strasbourg, LANSAD Teachers
Four questions for you
• To what extent do the courses you teach meet the
criteria of constructive alignment?
• What role does self-assessment play in your courses?
What is the relation between self-assessment and end-
of-course exams as regards criteria?
• What instruments do you require/expect your students to
use to manage their learning?
• Do your students manage their learning in and through
the target language? If not, why not?