future learning skills in a global context: … · future learning skills in a global context: from...
TRANSCRIPT
FUTURE LEARNING SKILLS IN A GLOBAL
CONTEXT: FROM DISCOURSE TO
PRACTICE
Stéphan Vincent-Lancrin, Ph.D.
Senior Analyst, Project Leader and Deputy Head of Division
Centre for Educational Research and Innovation,Directorate for Education and Skills
Helsinki, May 9th 2017
Skills that tertiary-educated professionals
report as very important in their job
Percentage of employees reporting the following skills as very important in their job
Source: Avvisati, Jacotin and Vincent-Lancrin (2014), based on REFLEX and HEGESCO data
22.7
30.6
40.2
40.3
40.4
40.4
40.5
41.8
46.5
48.0
50.0
53.4
54.2
55.0
56.5
56.9
58.6
60.8
61.7
0 10 20 30 40 50 60 70
knowledge of other fieldswrite and speak a foreign language
assert your authoritynegociate
alertness to opportunitiespresent ideas in audience
willingness to question ideasmobilize capacities of others
analytical thinkingcome with news ideas/solutions
write reports or documentsacquire new knowledge
master of your own fieldcoordinate activities
use computers and internetmake your meaning clear
work productively with othersperform under pressure
use time efficiently
Critical skills for the most innovative jobs
(according to tertiary-educated workers)
1,83
2,05
2,08
2,09
2,15
2,19
2,19
2,20
2,35
2,36
2,36
2,42
2,51
2,58
2,60
2,71
2,81
3,00
3,90
1,00
assert your authorityknowledge of other fields
negociateperform under pressure
use time efficientlywork productively with others
use computers and internetwrite and speak a foreign language
write reports or documentsmaster of your own field
make your meaning clearmobilize capacities of others
acquire new knowledgecoordinate activities
analytical thinkingalertness to opportunitiespresent ideas in audience
willingness to question ideascome with news ideas/solutions
Likelihood (odds ratios) of reporting the following skills: people in the most innovative jobs vs. least innovative jobs
Source: Avvisati, Jacotin and Vincent-Lancrin (2014), based on REFLEX and HEGESCO data
What individual skills should
education systems foster?
Technical skills (know-what and know-
how)
Creative and Critical Thinking
skills(Critical thinking,
observation, curiosity, ability to make connections,
imagination,...)
Social and Behavioural skills (Self-confidence, energy,
perseverance, passion, leadership, collaboration,
communication)
Some comments on these skill
categories
• They are domain-specific
– Skills are generally domain-specific: one is creative in a field, one knows how to behave/communicate in a specific context, one has problem-solving skills in a field, one has content knowledge in a field
• They can become « domain-generic »
– A skills becomes « domain-generic » when one has gained it in a number of domains or settings, so that it becomes a « habit of mind » (a disposition or a stabilised skill) that one can apply to new fields
• They overlap and may reinforceeach other
But
• They are different and cannotbe reduced to a single skill (or measure)
Technical skills
Creative and Critical Thinking
skills
Social and Behavioural skills
What the hell do we mean?
• « 21st century » skills are acknowledged and competence-based curricula are in place in virtually all OECD countries
• Teachers do not disagree withthem, but they don’t know whatthey actually mean in practice
• Curricula acknowledge them but remain at odds with them in mostcountries
• What does it mean to develop the creativity and critical thinking?
Innovation
Skills
Educationand
training
1. Can we articulate a common international language?
2. Can we develop an exemplary pedagogical toolkit to teach and assess creativity and critical thinking as part of countries’ (current) curriculum?
3. Can we identify a development (progression) scale for theseskills?
4. What are the key aspects of context that matter for theirdevelopment?
5. What are the effects of using the developed pedagogical toolkiton pedagogies, beliefs, social and behavioural skills, and standardised measures of creativity and academic achievement?
Ongoing project on fostering and assessing
students’ creativity and critical thinking
• Need for a common language, social representation and guidance about what some desired skills actually mean
• Skills that are not assessed are not taught consistently, but teachers also need to teach what they assess
• There is generally little space for students to develop and demonstrate creativity and critical thinking as part of theirusual disciplinary learning
• Start a process of change: pilot, prototype and developpedagogical resources as a proof of concept for otherteachers – before validation and possibly scale up (second phase)
Theory of action
Starting point: 5 creative habits of mind
(21st century skills)
Source: Lucas, Claxton and Spencer (2013)
• Intervention: Use of a (new) common rubric on creativity and critical thinking to develop pedagogicalactivities, assessments and define proficiency levels
• Domains of intervention (2 out of 3):– STEM (science and/or maths)
– Arts (music and visual arts)
– Interdisciplinary
• Minimum sample for intervention and for control group:– ≥ 5+5 schools (9 year olds / 14 year olds)
– ≥ 10+10 teachers (classes)
– ≥ 200+200 students
What we do: basics of the intervention
• Participation in 11 countries overall– Brazil, France (3), India, Hungary, Netherlands,
Russia, Slovakia, Spain, Thailand, UK (Wales), United States (3)
• 1st round (2015-16, 2016) in 9 countries– Brazil, France (3), India, Hungary, Netherlands,
Russia, Slovakia, Thailand, United States (3)
– At least 128 schools, 372 teachers, 9093 students
• 2nd round (2016-17) in 9 countries– Brazil, France, India, Hungary, Russia, Spain,
Thailand, UK (Wales), United States (2)
Participation in primary and secondary
education
Developing a pedagogical toolkit
A. The toolkitA1.
Rubric for assessment
Dimensions
Levels of progression
A2.
Pedagogical activities
Specific for each domain
Designed to test
dimensions and levels of progression
A3.
Set of exercises
Specific for each domain
Designed to prepare for the
assessment
A4.
Portfolio of student work
For each domain
As examples of different skill
levels
Contextual data collection
B1.
Subject-based assessment
Standardised assessment of
academic achievement
(maths and science; visual
arts and music)
B2.
Creativity assessment
Standardised test for
creativity
(EPoC)
(domain-specific)
B3.
Survey questionnaires
School principals
Teachers
Students
B4.
Interviews / focus groups
Teachers
Students
School-based intervention
School year
A A A A
Primary school group - Ages 8-9. STEM, Arts or interdisciplinary domain
Secondary school group - Ages 13-14. STEM, Arts or interdisciplinary domain
A
What factors influence the outcomes?
• Pre-tests and questionnaires at the beginning of the intervention:– Are differences related to baseline in achievement, creativity, to student
beliefs, to pedagogies, to socio-economic bacground, etc.?
• Observations and discussions within the network
B
B
School-based intervention
School year
A A A A
Primary school group - Ages 8-9. STEM, Arts or interdisciplinary domain
Secondary school group - Ages 13-14. STEM, Arts or interdisciplinary domain
A
What effects of the intervention?
• Measures after the intervention:– Post-tests and -questionnaires
– Qualitative observations of pedagogies
– Interviews, ESM, games to test executive functions
• Matched control group (with some kind of intervention as well)
Control group
Control group
B
B
B
B
• Domain-specific creativity tests: – Maths, Science, music, Graphic artistic, social
EPoC (Evaluation of Potential Creativity)
– All (except graphic) are still experimental
• Achievement tests:– Maths and Science: TIMSS (9 year olds) / PISA
(14 year olds)
– Visual Arts and Music: New pilot tests designedin-house for the project
Standardised assessments we use for
pre- and post-testing
Initial version of the rubric
CREATIVITY
(Coming up with ideas and solutions)
CRITICAL THINKING
(Questionning and evaluating ideas and solutions)
Progression
INQUIRE
Feel, empathise, observe, describe relevant experience and information
Explore, seek and generate ideas
Understand context/frame and boundaries of the problem
Review alternative theories and opinions and compare/find perspectives on the problem
Level 1 Level 2 Level 3 Level 4
IMAGINE
Make connections, integrate other disciplinary perspectives
Stretch and play with unusual/risky/radical ideas
Identify strengths and weaknesses of evidence, arguments, claims and beliefs
Challenge assumptions, check accuracy, analyse gaps in knowledge
Level 1 Level 2 Level 3 Level 4
DO / SHARE
Envision / Express / Produce / Prototype new product / solution / performance
Appreciate the novelty of solution and/or possible consequences
Appraise / Base / Justify opinion/products on logical, ethical or aesthetic criteria/reasoning
Acknowledge own bias (as perceived by others) and uncertainty/limits of endorsed opinion/solution
Level 1 Level 2 Level 3 Level 4
• Critical thinking is not (just):– Problem solving (= usually technical knowledge)
– Logical/rational thinking or understanding (weighing and justifying arguments)
• Most existing standardised tests of « critical thinking » assessunderstanding and logical thinking (ability to weigharguments within a given paradigm)
• It is about: – Challenging assumptions (the « core theory », not just auxiliary
hypotheses)
– Understanding the limitations of theories and conventions (live with ambiguity/uncertainty)
– Being able to consider other theories (perspective taking)
Some conceptual difficulties
• Creativity is not:
– Giftedness
– « New to the world » ideas
– Innovation (ideas/solutions with proven social/market value)
• Creativity is usually about:
– Originality + some level of usefulness/functionality
– Fluency + originality + elaboration
– Divergent-exploratory + convergent-integrative
• In our case
– Imagination, ideation
– Exploring unusual ideas
– Dare to be different
Some conceptual difficulties
• To develop new pedagogical activities
• To improve existing pedagogical activities
• To develop new rubrics (domain-specific, to assessstudennts, self-assessment, etc.)
• To assess student work
• To keep in mind the importance of thesecompetences (metacognition)
Uses of the rubric
Class-friendly rubric
CREATIVITY
(Coming up with ideas and solutions)
CRITICAL THINKING
(Questionning and evaluating ideas and solutions)
INQUIRING Play with unusual and radical ideas Challenge assumptions
IMAGINING• Generate ideas and make
connections• Find several perspectives on the problem
DOING Produce, perform or envision
something that is personally novel Propose own product/opinion justified
on logical, ethical or aesthetic criteria
REFLECTING
Assess the novelty of solution and of possible consequences
Acknowledge uncertainty/limits of chosen solution/position
Creative and critical thinking scaffolding
rubric (in progress)
Level 4 Level 3 Level 2 Level 1
Inquiring
Imagining
Reflecting
A pedagogical activity aligned with the creativity and critical thinking rubric should:
1. Create students’ need/interest to learn
2. Be challenging
3. Develop clear technical knowledge in one domain or more
4. Include the development of a product
What are the criteria for a good activity?
A pedagogical activity aligned with the creativity and critical thinking rubric should:
5. Have students co-design part of the product/solution or problem
6. Deal with problems that can be looked at from different
perspectives
7. Leave room for the unexpected
8. Include space and time for students to reflect and give/receive
feedback
What are the criteria for a good activity?
• Process and learning environment?– Montessori: Developmental Environmental Rating Scale (DERS)
– Developing a class observation tool to make sure we have a good teaching process?
• Student artefacts?– Should we notice in students’ outputs that they have more
creative and critical thinking potential?
– Could the process not be reflected in the actual output of students?
• Is there a possible contradiction between process and product?
– A creative educational process might not foster creative skills
– A directed educational process might foster creative skills
Some difficulties: how and what should
we assess?
25
30
35
40
45
Control Intervention
Pre Post
-4.7
+5.3
Primary education:
I have to use my imagination
45
50
55
60
65
Control Intervention
Pre Post
-3.4
+3.2
Primary education:
I have to make connections
between different school subjects
60
65
70
75
80
Control Intervention
Pre Post
-4
+5.4
Primary education:
I have to look for several explanations
45
50
55
60
65
Control Intervention
Pre Post
-0.8 +7.4
Primary education:
I do NOT only learn what I am interested in
Secondary education:
I have to use my imagination in this course
-1.2
+2.5
70
71
72
73
74
75
76
77
78
79
Control Intervention
Pre Post
Based on HUN, NLD, THA
Secondary education:
I have to solve problems that have more
than one possible solution
+0.8
+4.9
70
71
72
73
74
75
76
77
78
79
Control Intervention
Pre Post
Based on HUN, NLD, THA
Secondary education:
I have to produce or perform something on
my own
+2.2
+5.9
66
67
68
69
70
71
72
73
74
75
76
77
Control Intervention
Pre Post
Based on HUN, NLD, THA
Secondary education:
I have to explore different points of view on
a problem or topic in this course
-2.4
+3.4
68
69
70
71
72
73
74
75
Control Intervention
Pre Post
Based on HUN, NLD, THA
25
30
35
40
45
Control Intervention
Pre Post
-11.0
-0.1
Primary education:
I try to explore new things
40
45
50
55
60
Control Intervention
Pre Post
-6.5
+3.8
Primary education:
I am curious about many different things
50
55
60
65
70
Control Intervention
Pre Post
+5.0+10.9
Primary education:
Performance in maths and science tests
60
62
64
66
68
Control Intervention
Pre test VAM Post test VAM
+1.7
+3.47
Primary education:
Performance in visual art and music test
Secondary education:
Performance in maths and science tests
-0.3
+2.1
40
45
50
55
Intervention Control
Pre score Post score
Based on HUN, NLD, USA-Montessori
Secondary education:
Performance in visual arts test
-0.4
-2.8
55
57
59
61
63
65
Intervention Control
Pre score Post score
Based on NLD
50
55
60
65
70
75
80
85
Control Intervention
Pre Post
+12.5
-0.32
Share of teachers who agree that creativity
can effectively be taught in schools
0
2
4
6
8
10
12
14
16
18
20
Control Intervention
Pre Post
+4.75-3.33
Average number of times teacher ask students
to use everyday life examples during last month
0
2
4
6
8
10
12
14
16
18
20
Control Intervention
Pre Post
+2.87-2.18
Average number of times teacher ask
students to explain the reasoning behind
their answers during last month
62.5 62.5
50
0
10
20
30
40
50
60
70
Preparation of lessons Interaction with students Pedagogical activities
Share of (intervention) teachers who report
they have changed their…
75 75
68.75
0
10
20
30
40
50
60
70
80
90
Students' motivation Students' enjoyment Promoting creativity skills
Share of (intervention) teachers who report
perceived positive effect on…
• The innovative domain of PISA 2021 will be« creative thinking », building on the CERI projectand other sources
• To what extent do we need to change the usualassessment model – acknowledging that there willnot be more student time?
• International standardised assessments– Raise the profile of the objectives in policy making
– Have to remain connected to school teachingpractices
– Need to innovate in assesment format?
PISA 2021
• Teachers
– Common understanding through a multiplicity of artefacts and tools (slow and long process)
– Need for changes in teaching (including assessment) practice and professional development
– Professional learning communities?
• Students
– More relevance of and engagement in their learning
– Acquisition of broader skill set
– Development of stronger social and behavioural skills
Concluding remarks
THANK YOU
www.oecd.org/edu/innovation
www.oecd.org/edu/internationalisation
www.oecd.org/edu/universityfutures