From Evidence to Great TeachingRobert Coe, Durham UniversityASCL Annual Conference, 20 March 2015

∂

2

eef toolkitimproving educationWhat makes great teaching

∂

The argument

Successful implementation of evidence-based strategies requires deep understanding of the evidence

For most people this requires learning Even if you do ‘what works’ it may not work:

always evaluate

3

EvidenceCan’t we just bolt it on?

4

∂

True or false?1. Reducing class size is one of the most

effective ways to increase learning [evidence]

2. Differentiation and ‘personalised learning’ resources maximise learning [evidence]

3. Generous praise encourages learners and helps them persist with hard tasks [evidence]

4. Technology supports learning by engaging and motivating learners [evidence]

5. The best way to raise attainment is to enhance motivation and interest [evidence]

5

∂

Impact vs cost

Cost per pupil

Eff

ect

Siz

e (

mon

ths

gain

)

£00

8

£1000

Meta-cognitive

Peer tutoringEarly Years

1-1 tuitionHomework (Secondary)

After schoolTeaching

assistantsMentoring

Summer schools

AspirationsPerformance pay

Smaller classes

Ability grouping

Most promising for raising attainment

May be worth it

Small effects /

high cost

Feedback

Phonics

Homework (Primary)

CollaborativeSmall gp

tuition Parental involvement

Individualised learning

ICT

Behaviour

Social

www.educationendowmentfoundation.org.uk/toolkit

∂

Poor Proxies for Learning Students are busy: lots of work is done (especially written

work) Students are engaged, interested, motivated Students are getting attention: feedback, explanations Classroom is ordered, calm, under control Curriculum has been ‘covered’ (ie presented to students

in some form) (At least some) students have supplied correct answers,

even if they– Have not really understood them– Could not reproduce them independently– Will have forgotten it by next week (tomorrow?)– Already knew how to do this anyway

7

∂

Learning happens when people have to

think hard

A better proxy for learning?

What makes great teaching?(According to the evidence)

∂

10

∂

Dimensions of great teaching

1. (Pedagogical) content knowledge (PCK)

2. Quality of instruction

3. Classroom management / behaviour / control

4. Classroom climate / relationships / expectations

5. Beliefs (theory) about subject, learning & teaching

6. Wider professional elements: collegiality, PD, stakeholder relationships

11

∂

1. We do that already (don’t we?)

Reviewing previous learning Setting high expectations Using higher-order questions Giving feedback to learners Having deep subject knowledge Understanding student misconceptions Managing time and resources Building relationships of trust and challenge Dealing with disruption

12

∂

2. Do we always do that?

Challenging students to identify the reason why an activity is taking place in the lesson

Asking a large number of questions and checking the responses of all students

Raising different types of questions (i.e., process and product) at appropriate difficulty level

Giving time for students to respond to questions Spacing-out study or practice on a given topic, with

gaps in between for forgetting Making students take tests or generate answers, even

before they have been taught the material Engaging students in weekly and monthly review

13

∂

14

∂

3. We don’t do that (hopefully) Use praise lavishly Allow learners to discover

key ideas for themselves Group learners by ability Encourage re-reading and highlighting to memorise

key ideas Address issues of confidence and low aspirations

before you try to teach content Present information to learners in their preferred

learning style Ensure learners are always active, rather than

listening passively, if you want them to remember

15

If we know what it looks like, can we just do it?

∂

17

Improving Teaching

Teacher quality is what matters We need to focus on teacher learning Teachers learn just like other people

–Be clear what you want them to learn–Get good information about where

they are at–Give good feedback

∂

Just a check-list of techniques?

No! Great teaching involves– selecting, integrating, orchestrating, adapting,

monitoring, responding, etc,

and depends on – context, history, personalities, relationships, etc,

But without the skills, a teacher’s choices are more limited

Developing these skills & techniques takes dedicated, extended practice, with feedback

18

∂

What CPD benefits students? Promotes ‘great teaching’

– PCK, assessment, learning, high expectations, collective responsibility

– Focuses on student outcomes

Supported by– External input: challenge and expertise– Peer networks: communities of practice– School leaders must actively lead

Builds teacher understanding and skills– Challenges and engages teachers– Integrates theory and active skills practice – Enough learning time (monthly for min 6 months: 30hrs+)

19

Timperley et al 2007

When ‘what works’ doesn’t workThe case for evaluation

20

∂

Why monitor teaching quality? Good evidence of (potential) benefit from

– Performance feedback (Coe, 2002)– Target setting (Locke & Latham, 2006)– Accountability (Coe & Sahlgren, 2014)

Individual teachers matter most Teachers typically stop improving after 3-5

years Everyone can improve Assessment is an essential part of learning

(including teacher learning)

21

∂

Methods of identifying effectiveness

classroom observations by peers, principals or external evaluators

‘value-added’ models (assessing gains in student achievement)

student ratings principal (or headteacher) judgement teacher self-reports analysis of classroom artefacts and teacher

portfolios

22

∂

Do we know a good lesson when we see one?

23

∂

Lesson Observation

1. Two teachers observe the same lesson, one rates it ‘Inadequate’. What is the probability the other will agree?

a) 10% b) 40% c) 60% d) 80%

2. An observer judges a lesson ‘Outstanding’. What is the probability that pupils are really making sustained, outstanding progress?

a) 5% b) 30% c) 50% d) 70%

24

www.cem.org/blog

∂

Beware these traps

Overconfidence about knowledge of what is effective

Focus on teaching rather than learning Thinking that we are doing it already Overconfidence in assessments (even if

formative) of teaching quality Thinking that if we assess teaching we must

attach consequences to that (cf ‘assessment for learning’)

25

∂

Problems with assessment criteria If you know what it means, you know what it means (eg

from KS1 Performance Descriptors)– capital letters for some names of people, places and days of the

week (below)– capital letters for some proper nouns and for the personal

pronoun ‘I’ (towards)– capital letters for almost all proper nouns (at)– correctly punctuated (mastery)

Teaching by numbers (from KS2 2014 guidance)– a range of openings, e.g. adverbials (some time later, as we ran,

once we had arrived...), subject reference (they, the boys, our gang...), speech.

– Some variety in subordinating connectives, e.g. because, if, which– use of modals to express prediction, possibility, permission, e.g.

should, might, could.

26

∂

Bias in Teacher Assessment (vs standardised tests) Systematic bias against

– Pupils with SEN, EAL & FSM – Pupils with challenging behaviour

Reinforcing stereotypes– Eg boys perceived to be better at maths– ethnic minority / subject combinations

Pupil/teacher interaction– Bias against pupils whose personality is different from the teacher’s

∂

‘Improvement’ often isn’t School/college would have improved anyway

– Volunteers/enthusiasts improve: misattributed to intervention– Chance variation (esp. if start low)

Poor outcome measures– Perceptions of those who worked hard at it– No robust assessment of pupil learning

Poor evaluation designs– Weak evaluations more likely to show positive results – Improved intake mistaken for impact of intervention

Selective reporting– Dredging for anything positive (within a study)– Only success is publicised

(Coe, 2009, 2013)

∂

Clear, well defined, replicable intervention

Good assessment of appropriate outcomes

Well-matched comparison group

EEF DIY

Evaluatio

n Guide

Key elements of good evaluation

What could

you evaluate?

Simple, huh?

30

∂

“After 30 years of doing such work, I have concluded that classroom teaching…is perhaps the most complex, most challenging, and most demanding, subtle, nuanced, and frightening activity that our species has ever invented…The only time a physician could possibly encounter a situation of comparable complexity would be in the emergency room of a hospital during or after a natural disaster.”

Lee Shulman, The Wisdom of Practice

31