pisa oecd programme for international student assessment what students know and can do andreas...

11P

ISA

OE

CD

Pro

gram

me

for

Inte

rnat

iona

l Stu

dent

Ass

essm

ent

Wha

t stu

dent

s kn

ow a

nd c

an d

oA

ndre

as S

chle

iche

r7

Dec

embe

r 201

0 PISA 2009Evaluating systems to improve

education

Andreas SchleicherSpecial advisor to the Secretary-General on Education Policy

Head of the Indicators and Analysis Division, EDU

Programme for International Student Assessment

The yardstick for success is no longer improvement by national standards alone but the best performing

education systems

22P

ISA

OE

CD

Pro

gram

me

for

Inte

rnat

iona

l Stu

dent

Ass

essm

ent

Wha

t stu

dent

s kn

ow a

nd c

an d

oA

ndre

as S

chle

iche

r7

Dec

embe

r 201

0

1998PISA countries in 20002001200320062009

77%81%83%85%86%Coverage of world economy 87%

PISA 2009 in brief Over half a million students…

representing 28 million 15-year-olds in 74* countries/economies

… took an internationally agreed 2-hour test… Goes beyond testing whether students can

reproduce what they were taught……to assess students’ capacity to extrapolate from what

they know and creatively apply their knowledge in novel situations

…and responded to questions on… their personal background, their schools

and their engagement with learning and school Parents, principals and system leaders provided data

on… school policies, practices, resources and institutional

factors that help explain performance differences .* Data for Costa Rica, Georgia, India, Malaysia, Malta, Mauritius, Venezuela and Vietnam will be published in December

2011

33P

ISA

OE

CD

Pro

gram

me

for

Inte

rnat

iona

l Stu

dent

Ass

essm

ent

Wha

t stu

dent

s kn

ow a

nd c

an d

oA

ndre

as S

chle

iche

r7

Dec

embe

r 201

0



PISA 2009 in brief PISA seeks to…

… Support governments to prepare students…

… to deal with more rapid change than ever before…

… for jobs that have not yet been created…… using technologies that have not yet been

invented…… to solve problems that we don’t yet know will

arise… Provide a basis for policy dialogue and

global collaboration in defining and implementing educational goals, policies and practices– Show countries what achievements are possible– Help governments set policy targets in terms of

measurable goals achieved elsewhere– Gauge the pace of educational progress – Facilitate peer-learning on policy and practice .

44P

ISA

OE

CD

Pro

gram

me

for

Inte

rnat

iona

l Stu

dent

Ass

essm

ent

Wha

t stu

dent

s kn

ow a

nd c

an d

oA

ndre

as S

chle

iche

r7

Dec

embe

r 201

0



PISA 2009 in brief Key principles

‘Crowd sourcing’ and collaboration– PISA draws together leading expertise and institutions from

participating countries to develop instruments and methodologies…

… guided by governments on the basis of shared policy interests

Cross-national relevance and transferability of policy experiences

– Emphasis on validity across cultures, languages and systems– Frameworks built on well-structured conceptual

understandingof assessment areas and contextual factors

Triangulation across different stakeholder perspectives– Systematic integration of insights from students, parents,

school principals and system-leaders Advanced methods with different grain sizes

– A range of methods to adequately measure intended constructs with different grain sizes to serve different decision-making needs

– Productive feedback, at appropriate levels of detail, to fuel improvement at multiple levels .

55P

ISA

OE

CD

Pro

gram

me

for

Inte

rnat

iona

l Stu

dent

Ass

essm

ent

Wha

t stu

dent

s kn

ow a

nd c

an d

oA

ndre

as S

chle

iche

r7

Dec

embe

r 201

0

What 15-year-olds can do

66P

ISA

OE

CD

Pro

gram

me

for

Inte

rnat

iona

l Stu

dent

Ass

essm

ent

Wha

t stu

dent

s kn

ow a

nd c

an d

oA

ndre

as S

chle

iche

r7

Dec

embe

r 201

0Average performanceof 15-year-olds in reading – extrapolate and apply

High reading performance

Low reading performance … 17 countries perform below this line

1525354555440.000

460.000

480.000

500.000

520.000

540.000

560.000Shanghai-China

KoreaFinlandHong Kong-China

Singapore CanadaNew ZealandJapanAustralia

NetherlandsBelgium Norway, EstoniaSwitzerlandPoland, IcelandUnited States LiechtensteinSwedenGermany,IrelandFrance, Chinese TaipeiDenmarkUnited KingdomHungary,

PortugalMacao-China ItalyLatviaSlovenia GreeceSpainCzech RepublicSlovak Republic, Croatia

IsraelLuxembourg,Austria Lithuania

TurkeyDubai (UAE) Russian Federation

Chile

Serbia

NortheastMidwest

WestSouth

Urban schools

Suburban schools

Performance distribution in US18% do not reach baseline Level 2 (16% when excluding immigrants) (Finland 6%, Canada 9%)

Economic cost: 72 trillion $10% are top performers (Shanghai 20%)

77P

ISA

OE

CD

Pro

gram

me

for

Inte

rnat

iona

l Stu

dent

Ass

essm

ent

Wha

t stu

dent

s kn

ow a

nd c

an d

oA

ndre

as S

chle

iche

r7

Dec

embe

r 201

0Average performanceof 15-year-olds in science – extrapolate and apply

Low average performanceLarge socio-economic disparities

High average performanceLarge socio-economic disparities

Low average performanceHigh social equity

High average performanceHigh social equity

Strong socio-economic impact on

student performance

Socially equitable distribution of

learning opportunities


Low reading performance

88P

ISA

OE

CD

Pro

gram

me

for

Inte

rnat

iona

l Stu

dent

Ass

essm

ent

Wha

t stu

dent

s kn

ow a

nd c

an d

oA

ndre

as S

chle

iche

r7

Dec

embe

r 201

0Durchschnittliche Schülerleistungen im Bereich Mathematik






student performance





AustraliaBelgiumCanadaChileCzech RepDenmarkFinlandGermanyGreeceHungaryIcelandIrelandIsraelItalyJapanKoreaLuxembourgMexicoNetherlandsNew ZealandNorwayPolandPortugalSpainSwedenSwitzerlandUKUS

2009

1525354555

2009

99P

ISA

OE

CD

Pro

gram

me

for

Inte

rnat

iona

l Stu

dent

Ass

essm

ent

Wha

t stu

dent

s kn

ow a

nd c

an d

oA

ndre

as S

chle

iche

r7

Dec

embe

r 201







student performance





2009

1010P

ISA

OE

CD

Pro

gram

me

for

Inte

rnat

iona

l Stu

dent

Ass

essm

ent

Wha

t stu

dent

s kn

ow a

nd c

an d

oA

ndre

as S

chle

iche

r7

Dec

embe

r 201







student performance





2000

1111P

ISA

OE

CD

Pro

gram

me

for

Inte

rnat

iona

l Stu

dent

Ass

essm

ent

Wha

t stu

dent

s kn

ow a

nd c

an d

oA

ndre

as S

chle

iche

r7

Dec

embe

r 201







student performance





2000

Other rapid improvers in reading:Peru, Indonesia, Latvia, Israel and Brazil

Rapid improvers in mathematics:Mexico, Brazil, Turkey, Greece, Portugal, Italy

and GermanyRapid improvers in science:

Qatar, Turkey, Portugal, Korea, Brazil, Colombia, Italy, Norway, United States, Poland

1212P

ISA

OE

CD

Pro

gram

me

for

Inte

rnat

iona

l Stu

dent

Ass

essm

ent

Wha

t stu

dent

s kn

ow a

nd c

an d

oA

ndre

as S

chle

iche

r7

Dec

embe

r 201

0

Quality differences between schools

1313P

ISA

OE

CD

Pro

gram

me

for

Inte

rnat

iona

l Stu

dent

Ass

essm

ent

Wha

t stu

dent

s kn

ow a

nd c

an d

oA

ndre

as S

chle

iche

r7

Dec

embe

r 201

0

ArgentinaIta

ly

Turke

yIsr

ael

Germany

Hungary

Austria

Luxem

bourgJapan

Uruguay

Brazil

Sloven

ia

Croatia

United States

Singap

ore

Kyrgyzst

an

Montenegro

Albania

Slovak

Republic

Kazakh

stan

Ireland

Chinese Taipei

Switz

erland

New Zealand

Shanghai-

China

Russian Federation

Sweden

Indonesia

PolandLat

via

Thail

and

Norway

100

80

60

40

20

0

20

40

60

80

100

Variability in student performance between and within schools

Varia

nce

Performance variation of students within schools

Performance differences between schools

1414P

ISA

OE

CD

Pro

gram

me

for

Inte

rnat

iona

l Stu

dent

Ass

essm

ent

Wha

t stu

dent

s kn

ow a

nd c

an d

oA

ndre

as S

chle

iche

r7

Dec

embe

r 201

0

How do social background and learning outcomes interact?

1515P

ISA

OE

CD

Pro

gram

me

for

Inte

rnat

iona

l Stu

dent

Ass

essm

ent

Wha

t stu

dent

s kn

ow a

nd c

an d

oA

ndre

as S

chle

iche

r7

Dec

embe

r 201

0

-2 -1 0 1 2350

643

School performance and socio-economic background United States

Stud

ent p

erfo

rman

ce

AdvantagePISA Index of socio-economic background

Disadvantage

School performance and schools’ socio-economic backgroundStudent performance and students’ socio-economic background within schools

Private school Public school in rural area Public school in urban area

700

1616P

ISA

OE

CD

Pro

gram

me

for

Inte

rnat

iona

l Stu

dent

Ass

essm

ent

Wha

t stu

dent

s kn

ow a

nd c

an d

oA

ndre

as S

chle

iche

r7

Dec

embe

r 201

0

0

10

20

30

40

50

60

70

80

Percentage of resilient students among disadvantaged students%

More than 30% resilient students

among disadvantaged students

Between 15%-30% of resilient students among

disadvantaged students

Less than 15% resilient students among disadvantaged

students

Resilient student: Comes from the bottom quarter of the socially most disadvantaged

students but performs among the top quarter of students internationally (after

accounting for social background)

1717P

ISA

OE

CD

Pro

gram

me

for

Inte

rnat

iona

l Stu

dent

Ass

essm

ent

Wha

t stu

dent

s kn

ow a

nd c

an d

oA

ndre

as S

chle

iche

r7

Dec

embe

r 201

0

Does it all matter?

1818P

ISA

OE

CD

Pro

gram

me

for

Inte

rnat

iona

l Stu

dent

Ass

essm

ent

Wha

t stu

dent

s kn

ow a

nd c

an d

oA

ndre

as S

chle

iche

r7

Dec

embe

r 201

0

Age 19

Age 21

Age 21

048121620

Level 2Level 3

Level 4Level 5

Increased likelihood of postsec. particip. at age 19/21 associated with PISA reading proficiency at age 15

(Canada)after accounting for school engagement, gender, mother

tongue, place of residence, parental, education and family income (reference group PISA Level 1)Odds ratio

higher education entry

School marks at age 15

PISA performance at age

15

1919P

ISA

OE

CD

Pro

gram

me

for

Inte

rnat

iona

l Stu

dent

Ass

essm

ent

Wha

t stu

dent

s kn

ow a

nd c

an d

oA

ndre

as S

chle

iche

r7

Dec

embe

r 201

0

Policies and practicesLearning climateDisciplineTeacher behaviourParental pressureTeacher-student relationshipsDealing with heterogeneityGrade repetitionPrevalence of trackingExpulsionsAbility grouping

(all subjects)Standards /accountabilityNat. examinationStandardised tests

Policy

System

R

School

R

Equity

E

2020P

ISA

OE

CD

Pro

gram

me

for

Inte

rnat

iona

l Stu

dent

Ass

essm

ent

Wha

t stu

dent

s kn

ow a

nd c

an d

oA

ndre

as S

chle

iche

r7

Dec

embe

r 201

0

What does it all mean?

2121P

ISA

OE

CD

Pro

gram

me

for

Inte

rnat

iona

l Stu

dent

Ass

essm

ent

Wha

t stu

dent

s kn

ow a

nd c

an d

oA

ndre

as S

chle

iche

r7

Dec

embe

r 201

0

Commitment to universal achievement

Goals, gateways, instructional

systems

Capacity at point of delivery

Incentives and accountability

Resources where they yield most

A learning system

Coherence

Lessons from PISA on successful

education systems

A commitment to education and the belief that competencies can be learned and therefore all children can achieve

Universal educational standards and personalisation as the approach to heterogeneity in the student body…

…as opposed to a belief that students have different destinations to be met with different expectations, and selection/stratification as the approach to heterogeneity

Clear articulation who is responsible for ensuring student success and to whom

2222P

ISA

OE

CD

Pro

gram

me

for

Inte

rnat

iona

l Stu

dent

Ass

essm

ent

Wha

t stu

dent

s kn

ow a

nd c

an d

oA

ndre

as S

chle

iche

r7

Dec

embe

r 201







student performance





2009

Early selection and institutional differentiation

High degree of stratificationLow degree of stratification

2323P

ISA

OE

CD

Pro

gram

me

for

Inte

rnat

iona

l Stu

dent

Ass

essm

ent

Wha

t stu

dent

s kn

ow a

nd c

an d

oA

ndre

as S

chle

iche

r7

Dec

embe

r 201

0



systems




A learning system

Coherence


education systems

Clear ambitious goals that are shared across the system and aligned with high stakes gateways and instructional systems

Well established delivery chain through which curricular goals translate into instructional systems, instructional practices and student learning (intended, implemented and achieved)

High level of metacognitive content of instruction

2424P

ISA

OE

CD

Pro

gram

me

for

Inte

rnat

iona

l Stu

dent

Ass

essm

ent

Wha

t stu

dent

s kn

ow a

nd c

an d

oA

ndre

as S

chle

iche

r7

Dec

embe

r 201

0



systems




A learning system

Coherence


education systems

Capacity at the point of delivery Attracting, developing and retaining high

quality teachers and school leaders and a work organisation in which they can use their potential

Instructional leadership and human resource management in schools

Keeping teaching an attractive profession System-wide career development

2525P

ISA

OE

CD

Pro

gram

me

for

Inte

rnat

iona

l Stu

dent

Ass

essm

ent

Wha

t stu

dent

s kn

ow a

nd c

an d

oA

ndre

as S

chle

iche

r7

Dec

embe

r 201

0


education systems



systems




A learning system

Coherence Incentives, accountability, knowledge

management Aligned incentive structures

For students How gateways affect the strength, direction, clarity and nature

of the incentives operating on students at each stage of their education

Degree to which students have incentives to take tough courses and study hard

Opportunity costs for staying in school and performing wellFor teachers Make innovations in pedagogy and/or organisation Improve their own performance

and the performance of their colleagues Pursue professional development opportunities

that lead to stronger pedagogical practices A balance between vertical and lateral accountability Effective instruments to manage and share knowledge

and spread innovation – communication within the system and with stakeholders around it

A capable centre with authority and legitimacy to act

2626P

ISA

OE

CD

Pro

gram

me

for

Inte

rnat

iona

l Stu

dent

Ass

essm

ent

Wha

t stu

dent

s kn

ow a

nd c

an d

oA

ndre

as S

chle

iche

r7

Dec

embe

r 201

0

Systems with more accountability Systems with less

accountability

480

490

500

Schools with less autonomy

Schools with more autonomy

495

School autonomy in re-source allocation

System’s accountability arrangements

PISA score in reading

School autonomy, accountability and student performance

Impact of school autonomy on performance in systems with and without accountability arrangements

2727P

ISA

OE

CD

Pro

gram

me

for

Inte

rnat

iona

l Stu

dent

Ass

essm

ent

Wha

t stu

dent

s kn

ow a

nd c

an d

oA

ndre

as S

chle

iche

r7

Dec

embe

r 201

0Local responsibility

and system-level prescription

System-level prescription‘Tayloristic’ work organisation

Schools leading reformTeachers as ‘knowledge workers’

Schools todayThe industrial

model, detailed prescription of

what schools do

Schools tomorrow?

Building capacity

Finland todayEvery school an effective school

Trend in OECD countries

2828P

ISA

OE

CD

Pro

gram

me

for

Inte

rnat

iona

l Stu

dent

Ass

essm

ent

Wha

t stu

dent

s kn

ow a

nd c

an d

oA

ndre

as S

chle

iche

r7

Dec

embe

r 201

0Public and private schools

AustraliaAustriaCanada

ChileCzech Republic

DenmarkEstoniaFinland

GermanyGreece

HungaryIcelandIreland

IsraelItaly

JapanKorea

LuxembourgMexico

NetherlandsNew Zealand

NorwayPoland

PortugalSlovak Republic

SloveniaSpain

SwedenSwitzerland

TurkeyUnited Kingdom

United StatesArgentina

BrazilHong Kong-China

IndonesiaJordan

Russian FederationShanghai-China

SingaporeChinese Taipei

0 20 40 60 80 100

Government schoolsGovernment dependent private Government independent private

-150 -100 -50 0 50 100

Difference after accounting for socio-economic background of students and schoolsObserved performance difference

Private schools perform better

Public schools perform better

%

Score point difference

2929P

ISA

OE

CD

Pro

gram

me

for

Inte

rnat

iona

l Stu

dent

Ass

essm

ent

Wha

t stu

dent

s kn

ow a

nd c

an d

oA

ndre

as S

chle

iche

r7

Dec

embe

r 201

0



systems




A learning system

Coherence


education systems Investing resources where they can make

most of a difference Alignment of resources with key challenges

(e.g. attracting the most talented teachers to the most challenging classrooms)

Effective spending choices that prioritise high quality teachers over smaller classes

3030P

ISA

OE

CD

Pro

gram

me

for

Inte

rnat

iona

l Stu

dent

Ass

essm

ent

Wha

t stu

dent

s kn

ow a

nd c

an d

oA

ndre

as S

chle

iche

r7

Dec

embe

r 201

0



systems




A learning system

Coherence


education systems

A learning system An outward orientation of the system to

keep the system learning, international benchmarks as the ‘eyes’ and ‘ears’ of the system

Recognising challenges and potential future threats to current success, learning from them, designing responses and implementing these

3131P

ISA

OE

CD

Pro

gram

me

for

Inte

rnat

iona

l Stu

dent

Ass

essm

ent

Wha

t stu

dent

s kn

ow a

nd c

an d

oA

ndre

as S

chle

iche

r7

Dec

embe

r 201

0



systems




A learning system

Coherence


education systems

Coherence of policies and practices Alignment of policies

across all aspects of the system Coherence of policies

over sustained periods of time Consistency of implementation Fidelity of implementation

(without excessive control)

3232P

ISA

OE

CD

Pro

gram

me

for

Inte

rnat

iona

l Stu

dent

Ass

essm

ent

Wha

t stu

dent

s kn

ow a

nd c

an d

oA

ndre

as S

chle

iche

r7

Dec

embe

r 201

0

Beyond schooling

3333P

ISA

OE

CD

Pro

gram

me

for

Inte

rnat

iona

l Stu

dent

Ass

essm

ent

Wha

t stu

dent

s kn

ow a

nd c

an d

oA

ndre

as S

chle

iche

r7

Dec

embe

r 201

0

Lith

uani

a

Ger

man

y

Denm

ark

Croa

tia

Hong

Kon

g-C

hina

Kore

a

Mac

ao-C

hina

Port

ugal

Hung

ary

New

Zea

land

Chile

Italy

Pana

ma

Qat

ar

-100

102030405060

Scor

e po

int d

iffer

ence

Parental support at the beginning of primary school

Score point difference between students whose parents often do (weekly or daily) and those who do not:

"talk about what they had done"

3434P

ISA

OE

CD

Pro

gram

me

for

Inte

rnat

iona

l Stu

dent

Ass

essm

ent

Wha

t stu

dent

s kn

ow a

nd c

an d

oA

ndre

as S

chle

iche

r7

Dec

embe

r 201

0Performance difference between students who had attended pre-primary school for more than one year and those who did

not

Scor

e po

int d

iffer

ence

Israe

l

Belgium

Macao

-China

France

Switzerl

and

United K

ingdom

Dubai (U

AE)

Kyrgyz

stan

Argen

tina

German

y

New Zea

land

Slovak R

epublic

Brazil

Luxembourg

Thailan

d

Canad

a

Chinese T

aipei

Poland

Kazak

hstan

RomaniaJa

panPeru

Jordan

Norway

Azerbaij

an

ColombiaChile

Lithuan

iaSerb

ia

Netherl

ands

Slovenia

FinlandLatv

ia0

20

40

60

80

100

120

Beyond schooling

Observed performance advantage

Performance advantage after accounting for socio-economic

factors

3535P

ISA

OE

CD

Pro

gram

me

for

Inte

rnat

iona

l Stu

dent

Ass

essm

ent

Wha

t stu

dent

s kn

ow a

nd c

an d

oA

ndre

as S

chle

iche

r7

Dec

embe

r 201

0

Some students learn at high levels

All students need to learn at high levels

Student inclusion

Routine cognitive skills, rote learning

Learning to learn, complex ways of

thinking, ways of workingCurriculum, instruction and assessment

Few years more than secondary

High-level professional knowledge workers

Teacher quality

‘Tayloristic’, hierarchical

Flat, collegial

Work organisation

Primarily to authorities

Primarily to peers and stakeholders

Accountability

Education reform trajectoriesThe old bureaucratic system The modern enabling system

3636P

ISA

OE

CD

Pro

gram

me

for

Inte

rnat

iona

l Stu

dent

Ass

essm

ent

Wha

t stu

dent

s kn

ow a

nd c

an d

oA

ndre

as S

chle

iche

r7

Dec

embe

r 201

0

Thank you !

Find out more about PISA at… OECD www.pisa.oecd.org

– All national and international publications– The complete micro-level database

U.S. White House www.data.gov

Email: [email protected]

…and remember:Without data, you are just another person with an opinion

http://www.pisa.oecd.org/

http://www.data.gov/

3737P

ISA

OE

CD

Pro

gram

me

for

Inte

rnat

iona

l Stu

dent

Ass

essm

ent

Wha

t stu

dent

s kn

ow a

nd c

an d

oA

ndre

as S

chle

iche

r7

Dec

embe

r 201

0

Five volumes released on 7 December Volume I, What Students Know and can Do:

Student Performance in Reading, Mathematics and Science

Volume II, Overcoming Social Background: Equity in Learning Opportunities and Outcomes

Volume III, Learning to Learn: Student Engagement, Strategies and Practices

Volume IV, What Makes a School Successful? Resources, Policies and Practices

Volume V, Learning Trends: Changes in student Performance since 2000

One volume to be released in June 2011 Volume VI, Students On Line: Reading and Using

Digital Information

PISA 2009 results

pisa oecd programme for international student assessment what students know and can do andreas...

Documents