PGCE in Mathematics

International and Comparative Education

David Pepper, King’s College London

11 February 2014

Session structure

• What international and comparative education is

• The rise of international comparisons in education

• Which is ‘the best education system in the world’

I.C.E.

• InternationalInsider/outsider observer or audience; practical

• Comparative Comparison between units eg countries; academic

• EducationItself multidisciplinary; formal/informal education

Way of defining ICE?

• Not a cohesive disciplineDifferent methodologies, theories and expertise

• A blurred but diverse fieldMulti-disciplinary …but inter-disciplinary?

• A (porous) communityOrganisations & journals… beyond ICE?Influence of (inter)national agencies

Broadfoot, (1977, 2000); Manzon, 2011

ICE ‘stages’• Travellers’ tales (pre-history of ICE)

• Educational borrowing (18/19th Century)

• International cooperation (19/20th Century)

• Forces and factors (1900-1950)

• Social science explanation (1960s)

Noah & Eckstein (1969)

The rise and rise of ICE

• Long history of ICE (Noah & Eckstein’s ‘stages’)

• Formation of the nation state (Hobsbawm)

• Inter-/post-war supranational apparatus

• Hollowing out of the state (Rhodes) but not education (Green)

• ‘Globalisation’ of trade, labour and information

• Technocratic government and ‘evidence-based policy’

• From national censuses to international surveys

Global and regional players

• Global agencies: World Bank, UNESCO, UNICEF

• Regional/cross-sectional blocs: OECD, EU et al

• Policy borrowing, influence and imposition

• Cooperation in tension with competition

Some major issues

• Education For All (MDGs 2015) and Child-Friendly Schools (United Nations agencies and partners)

• Lifelong and life-wide learning (EU, OECD, World Bank)

• Early years education frameworks (OECD… World Bank)

• MST graduates (EU Lisbon Goal 2010)

• Curriculum convergence (basic skills v competences)

• Qualification equivalence (regional/global frameworks)

What would you use international and comparative education for?

Purposes for ICE?

• Responses to common issues or ‘global’ forces

• Explanation and understanding dynamics

• Mutual understanding or technical equivalence

• Universals and specifics of concepts/theories

• Self-awareness and reflection

• Challenge assumptions, discourses, power

Research, policy and practice Different people, contexts and purposes

International Comparisons in Mathematics Education

Participation in upper secondary mathematics

Vorderman (2011)

Which is the best education system in the world?

International surveys: history

• First International Mathematics Study (FIMS) 1960s

• Second International Mathematics Study (SIMS) 1970s/80s

• International Assessment of Educational Progress (IAEP) 1988, 1991

• Third International Mathematics and Science Study (TIMSS) 1995

International surveys: present

• IEA: Trends in International Mathematics and Science Study (TIMSS) 1999, 2003, 2007, 2011, 2015

• IEA: Progress in International Reading Literacy Study (PIRLS) 2001, 2006, 2011, 2016

• OECD: Programme for International Student Assessment (PISA) 2000, 2003, 2006, 2009, 2012

• Several others: IEA- TIMSS advanced, pre-PIRLS, ICCS; OECD- PIAAC; EU L2L, civic; South and East Africa- SACMEQ…

TIMSS/PIRLS and PISA

International Association for the Evaluation of Educational Achievement website particularly: About us; Studies/Completed studies/TIMMS and PIRLS

OECD Programme for International Student Assessment particularly: What PISA is; What PISA produces; Participating countries/economies; YouTube video

IEA on TIMSS and PIRLS

• System-level achievement

• Establishing goals and standards

• Stimulating reform

• Related studies (NB encyclopedia)

• Training in assessment and evaluation

OECD (2010) on PISA 2009

• International comparison of knowledge and skills• Reveals what level of performance is possible• Gauge the pace of educational progress• Set policy targets against measurable goals• To initiate research to identify policy levers• While PISA cannot identify cause-and-effect relationships

between inputs, processes and educational outcomes, it can highlight the key features in which education systems are similar and different, sharing those findings with educators, policy makers and the general public.

Which is the best education system in the world?

Which is the best education system in the world?

TIMSS 2007

TIMSS 2007

PISA 2009

Which is the best education system in the world?

IEA v OECD

IEA IEA OECD

PIRLS 2011 TIMSS 2011 PISA 2012

77 countries 58 countries 34 OECD, 31 others

Grade 4 Grades 4 & 8 Age 15

Reading Maths/science Reading/maths/science

Curriculum Curriculum Competence/literacy

Every 5 years Every 4 years Every 3 years

“the most important……man in English education“:

WHO DECIDES WHAT’S ‘BEST’?

Making headlines

Korean Education: Excellent Overall, but No Bright Sparks

Japanese students fare better in latest international tests, but alarm over declining standards persists

Finland Slips in PISA Ranking

See: Luzon & Torres (2011) in PISA under examination

Policy influence

• Germany’s PISA 2000 ‘shock’ (‘you stupid PISA!’)

• Japan’s PISA 2003 ‘crisis’ (6th Ma, 14th Sc, 2nd Re)

• England’s current proposed reforms (‘decline’/’stagnation’)

• ‘Selling’ the data to policy makers eg Hong KongPhillips & Ertl (2006)

Takayama (2008)Pons (2011)

Breakspear (2012)

‘The tyranny of the international horse race’

Problems:• Sampling (technical)• Reporting (technical)• Curriculum match (fundamental; even PISA)• Student motivation (trivial)• Taught time (fundamental)• Differentiation by attainment (fundamental)• Use of surveys (political)

Brown (1998)

‘World class schools’ – noble aspiration or globalised hokum?

• Surveys of learning outcomes (ie rather than attendance etc)

• Problems: Limited model of inputs/outputs/processesCultural reductionNarrow literature

• US NRC (2003) typology of international comparisons: Type 1- on outcomes for policy purposesType 2- on policies for policy purposesType 3- on general processes for academic purposes

• Discourse of international competition versus intercultural understanding and facing global issues (see Kandel on ICE)

Alexander (2010)

Further reading

Brown, M. (1998). The tyranny of the international horse race. In R. Slee, S. Tomlinson & G. Weiner (Eds.), School Effectiveness for Whom? London: Falmer Press.

Alexander, R. J. (2010). ‘World class schools’ – noble aspiration or globalised hokum? Compare: A Journal of Comparative and International Education, 40(6), 801-817.

Let’s look at some results

• PISA results

• TIMSS results

How are the results presented? How would you interpret the results?

TIMSS 2011 – mathematics

PISA 2009 – mathematics

Rankings

“The truth is, at the moment we are standing still while others race past. In the most recent OECD PISA survey in 2006 we fell from 4th in the world in the 2000 survey to 14th in science, 7th to 17th in literacy, and 8th to 24th in mathematics.”(Prime Minister and Deputy Prime Minister, Preface to the Schools White Paper, 2011)

“In the last ten years we have plummeted in the rankings: from 4th to 16th for science, 7th to 25th for literacy and 8th to 28th for maths.”(Secretary of State for Education after publication of PISA 2009; National Curriculum Review launch, 21/1/2011)

Further reading

Jerrim, J. (2011). England's "plummeting" PISA test scores between 2000 and 2009: Is the performance of our secondary school pupils really in relative decline.

• PISA maths ‘decline’ 2000 to 2009• Yet TIMSS maths improvement• This pattern not observed in others countries

England’s performance over time

• PISA ‘decline’ 2000 to 2009– Firstly, focus on score not rank (sample; clustering)– Year group changed (positive effect!)– Test month changed (negative effect)– Non-response bias (negative effect)– No substantive change in performance

• No such changes in TIMSS

Curriculum match

• ‘All we can safely say (we hope) is that students do experience different types of instructional arrangements cross-nationally and the influence of these arrangements generically appears weak relative to such matters as prior learning and the contents of learning opportunities during the course of study’ (Burstein, 1992, p. 278)

• The implemented curriculum (Askew et al, 2010)

What’s in a name?

TIMSS mathematics PISA mathematicsContent Number; algebra;

geometry; data and chance

Quantity; space and shape; change and relationships; uncertainty

Cognition/ competence

Knowing; applying; reasoning

Reproduction; connection; reflection

Contexts Personal; occupational; civic; scientific

Content balance and years of schooling account for 93% of the difference in countries performance in TIMSS and PISA (Wu, 2009)

TIMSS Grade 8 curriculum item

TIMSS Grade 8 problem-solving item

Test!

Which domains are each of the three released PISA items from (see hand out)?

a) Readingb) Mathematicsc) Scienced) Problem-solvinge) Financial literacy

‘Best’ …attitudes and attainment

• PISA, TIMSS and PIRLS also survey students’ attitudes, using questionnaires

• Validity is just as relevant to affective questionnaires as to cognitive tests

• What do they mean by ‘attitudes’?

Self-confidence

• Self-concept items (Marsh, Shavelson et al)

To what extent do you agree… I get good marks in mathematics

• Self-efficacy items (Bandura, Pajares, Schunk et al)How confident do you feel about… calculating how many square metres of tiles you need to cover a floor

Self-efficacy as self-regulation

• The extent to which you think you can succeed in a task affects whether you attempt the task and how you attempt it

• Models of self-regulated learning… (Winne & Hadwin, 1998 versus Zimmerman, 1989)

• …and formative assessment (Black and Wiliam, 2009)

Self-efficacy in mathematics

• Lack of longitudinal or experimental studies

• Lack of sufficient validation for assessments

Pepper (2014, forthcoming)

Assessment validity

…an integrated evaluative judgment of the degree to which empirical evidence and theoretical rationales support the adequacy and appropriateness of inferences and actions based on test scores or other modes of assessment

Messick (1989)

Is the PISA assessment of mathematical self-efficacy valid for its proposed

interpretations and use?

Threats to validity

• Construct-irrelevant variance

• Construct underrepresentation

Messick (1995)

Validity evidence

• Test content• Internal consistency• Relations with other variables• Response processes• Consequences of the test

Standards (1999) AERA/APA/NCME

Quant:PISA 2003 questionnaire41 countries10,274 schools276,165 students

Qual:Think aloud interviewsEN, NL, EE & HK2 schools per country5 students per school

Sources of evidence

Self-efficacy items

-2

-1

0

1

2

3

4

5

6 Students (x=1500)

-1.93

-1.62

-1.34

-1.07

-0.81

-0.55-0.3-0.05

0.21

0.47

0.73

1.01

1.3

1.61

1.94

2.31

2.74

3.29

4.11

5.41

Mea

sure

(log

its)

Timeta

ble

TV Tiles

Newsp

apers

Linea

rMap

sQuad

ratic

Petrol

-1

-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0.8

1

HKNLEWNI

Differential item functioning between Hong Kong, the Netherlands and, England/Wales/Northern Ireland

DIF

siz

e

Newspapers may be misleading

There are many factors

There may be delays

Hong Kong students

Petrol item

‘A lot of sources maybe needed and ...need to combine them with the formula. So maybe I will get it wrong and then I will just mix up with the formula and then the result will be wrong. So I think these kind of questions involve more thinking and I maybe usually get some wrong mistakes.’ [Petrol – Not very confident; HK Island school interview 3 – girl, moderate attainment]

Recent use of evidence

OECD (2011) report OECD (June, 2011) briefing

ANSWER: Self-efficacyIMPLICATION: Boost self-efficacy

Which is the best education system in the world?

‘Education system’

• Variation within systems

• ‘Shadow education’? (Bray, 1999, 2011)

• Relations between education and society

Dangers of “Cherry –picking”

• What “works” somewhere does not “work” elsewhere– “disorderly classrooms”: Korea - US – “disciplined environment”: Japan - Brazil – Technology: Brazil – Norway – Japan

• Finland: – Masters (78%) or high-status profession & competitive entry– BUT Pacific Rim …

• High-attaining countries are dissatisfied with their performance– Pacific Rim: application of mathematics– Japan: standards falling– Korea / Singapore: communication & creativity– Finland: more heterogeneity

In studying foreign systems of Education we should not forget that the things outside the schools matter even more than the things inside the schools, and govern and interpret the things inside. We cannot wander at pleasure among the educational systems of the world, like a child strolling through a garden, and pick off a flower from one bush and some leaves from another, and then expect that if we stick what we have gathered into the soil at home, we shall have a living plant (Michael Sadler, 1900).