travelling the road to expertise: a longitudinal study of learning

Travelling the road to expertise: A longitudinal study of learning

Kaye StaceyUniversity of Melbourne

Australia

A journey

with 3204 students

as they learn to understand decimal notation

over seven years (Grades 4 – 10)

2.71828

0.6

0.3repeating

4,08

Grades 4 – 6 “primary”; Grades 7 – 10 “secondary”

Our decimals work in summary

Understanding how students think about decimals

Tracing students’ progress in the longitudinal study

Looking at teaching interventions

Creating computer games using intelligent tutoring and AI (Bayesian nets)

Developing CD and website for teachers

Thanks to Vicki Steinle Liz Sonenberg Ann Nicholson Tali Boneh Sue Helme Nick Scott Australian Research

Council many U of M honours

students Dianne Chambers teachers and children

providing data

Why decimals?

Practical Importance Is my blood alcohol over 0.05% or not? Is my p value over 0.05? Links to metric measuring

Fundamental role of number in mathematics (e.g. understanding 0)

Known to be complex* with poor learning

A case study of students’ growth of understanding, which was able to start from a good research base (incl M.Swan)

* place value, fractions, density of real numbers etc

In preparation for the journey:

Who? 3204 students(12 schools, all SES, volunteer teachers)

Transport: ordinary teaching Territory and map – see later The destination – “understanding

decimals”

The destination: understanding decimal notation

3.145

27.483

Why is such a simple rule as rounding hard to remember?A convention carrying distributed intelligence

Sample Cohort study

aimed to follow as many students as possible for as long as possible

1079 students first tested in primary – nearly 60% followed to secondary school

over 600 students completed 5,6 or 7 tests (i.e. followed into a third or fourth year)

Quantitative analysis of longitudinal data conducted by Dr Vicki Steinle (PhD thesis)

Sample data Two tests per year – one per “semester” 9862 tests completed Students tracked for up to 7 semesters Tests average 8.3 months apart. Absentees not

chased. ID Grade 4 Grade 5 Grade 6 Grade 7 Grade 8 Grade 9 Grade 10 210403026 L1 A1 S3 S5 S1 300704112 L1 L4 L4 L2 L1 310401041 L2 L1 U1 U1 L4 U1 U1 390704012 L1 A1 U1 A1 S3 400704005 A1 A2 A1 A2 A1 410401088 L1 L1 L4 L1 L2 A1 A1 500703003 S1 S5 S3 S3 U1 500703030 S3 S5 S1 A2 600703029 A1 U1 A1 A1 A3

Interconnections between the map and the mapping tool

“It is only by asking the right, probing questions that we discover deep misconceptions, and only by knowing which misconceptions are likely do we know which questions are worth asking”, (Swan, 1983, p65).

The longitudinal study uses one type of question: which of two decimals is larger? e.g. 0. 8 or 0.75 (CSMS item, 1981)

4.8 or 4.63 (Resnick et al, 1989)

Similar items – different success

Select the largest number from0.625, 0.5, 0.375, 0.25, 0.125Correct: 61%

Select the smallest number from0.625, 0.5, 0.375, 0.25, 0.125Correct: 37%

Why such a large difference?Foxman et al (1985) Results of large scale “APU” monitoring UK. All sets given here as

largest to smallest; not as presented.

Common patterns in answers

0.625 0.5 0.375 0.25 0.125Largest Smallest

0.625 0.125 correct 0.625 0.5 well known error

“longer-is-larger”

0.5 0.625 identified 1980s“shorter-is-

larger”

Persistent patterns

Select the smallest number from0.625, 0.5, 0.375, 0.25, 0.125

Option TIMMS-R International

TIMMS-R Australia

Foxman et al. APU, age 15

Prediction (Grade 8)

0.125 46% 58% 37% 60%

0.25 4% 4% 3% 2%

0.375 2% 1% 2% 2%

0.5 24% 15% 22% 18%

0.625 24% 22% 34% 17%

predicted from other responses by our sample

Our sample is typical

A1expert

A2

A3L2L1

L4

S1S5

S3 U2

U1

Longer-is-larger

expert

Map prior to 1980

Early explorers

4.84.63

Comparison used by Brueckner 1928

A1expert

A2

A3L2L1

L4

S1S5

S3 U2

U1Map by 1990

Early explorers

Longer-is-larger

Shorter-is-largerexpert

4.84.63

5.7365.62

Use patterns of responses to sets of comparison items – Sackur, Resnick, Peled and others

A1expert

A2

A3L2L1

L4

S1S5

S3 U2

U1

Map for thislongitudinal study

Longer-is-larger (L)

Shorter-is-larger (S)expert (A1)

other AU

A1expert

A2

A3L2L1

L4

S1S5

S3 U2

U1

Decimal Comparison Test (DCT2)

We now have better version

set of similar items

set of similar items

Complex test, easy to complete

Several items of each type Codes require consistent responses Items within types VERY carefully matched

Comparison Item A1 A2 L1 L2 S1 S3 U2

4.8 4.63

5.736 5.62

4.7 4.08

4.4502 4.45

0.4 0.3

A1expert

A2

A3L2L1

L4

S1S5

S3 U2

U1

A1expert

A2

A3L2L1

L4

S1S5

S3 U2

U1

whole number analogy 4.8, 4.63

additional information – zero

makes small 4.71 4.082

“fat columns” 4.71 has 71 tenths, 4.082 has 82 hundredths

Longer-is-larger

A1expert

A2

A3L2L1

L4

S1S5

S3 U2

U1

“unclassified”: we don’t know

everything wrong –

quite smart!

A1expert

A2

A3L2L1

L4

S1S5

S3 U2

U1

expert on comparison task

truncation – no meaning for later dec pl;

failed algorithms – e.g. comparison of space and

0 4.45024.45

Only known to be OK on easy items on DCT2 – maybe not OK on harder items

4.77777 vs 4.7

0.600.00.00

A1expert

A2

A3L2L1

L4

S1S5

S3 U2

U1

Shorter–is-larger

analogies with fractions or negatives

(e.g. 0.4 < 0.3)

some place value considerations – all thousandths smaller than all hundredths

e.g. 5.62 greater than 5.736

Distinguish behaviour and way of thinking

A1expert

A2

A3L2L1

L4

S1S5

S3 U2

U1

Shorter–is-larger

analogies with fractions or negatives

(e.g. 0.4 < 0.3)

some place value considerations – all thousandths smaller than all hundredths

e.g. 5.62 greater than 5.736

Distinguish behaviour and way of thinking

Can subdivide these with improved DCT

Example of S thinking Courtney - a “text

book” case derived from research

interviews – not too obvious for

our student teachers

Hidden Numbers Making the biggest

and smallest numbers

Number Between

S behaviour: “through the looking glass”

5.736 < 5.62 (because larger whole number, so reverse)

The mirror is a powerful metaphor underpinning some everyday and mathematical concepts (Lakoff & Johnson)

Fractions (and hence decimals) and negative numbers as “mirror images” of whole numbers, so everything is reversed (Stacey et al, PME, 2001)

negative is an additive inverse reciprocal is a multiplicative inverse

Can even lead to getting all comparisons wrong (U2)

3110

1000

S behaviour: false number line analogies

Th H T U t h th

-3 -2 -1 0 1 2 3

Further right means smaller 5.736 “further right” than 5.62 Consequences of 0 and Units as “mirror position”

0 vs 0.6 Further confusion with 1 as “mirror position” for

fractionsSome S-like students get classified into A by DCT2

Characteristics of test

Reliable (56% of students in same code after one semester!)

Generally agrees with interviews Weakness is in diagnosing expertise with

consequence that all our estimates of expertise are overestimates some “experts” follow rules without understanding some “experts” cant do other tasks (e.g. could

reshelve books in library, but don’t know metric properties of decimals)

Test has been improved over life of study – now have improved versions – cycle of improvement

Examples of student journeys

ID Grade 4 Grade 5 Grade 6 Grade 7 Grade 8 Grade 9 Grade 10 210403026 L1 A1 S3 S5 S1 300704112 L1 L4 L4 L2 L1 310401041 L2 L1 U1 U1 L4 U1 U1 390704012 L1 A1 U1 A1 S3 400704005 A1 A2 A1 A2 A1 410401088 L1 L1 L4 L1 L2 A1 A1 500703003 S1 S5 S3 S3 U1 500703030 S3 S5 S1 A2 600703029 A1 U1 A1 A1 A3

A1expert

A2

A3L2L1

L4

S1S5

S3 U2

U1

210403026

A1expert

A2

A3L2L1

L4

S1S5

S3 U2

U1

310401041

A1expert

A2

A3L2L1

L4

S1S5

S3 U2

U1400704005

A1expert

A2

A3L2L1

L4

S1S5

S3 U2

U1

410401088

A1expert

A2

A3L2L1

L4

S1S5

S3 U2

U1500703030

A1expert

A2

A3L2L1

L4

S1S5

S3 U2

U1

500703030

A1expert

A2

A3L2L1

L4

S1S5

S3 U2

U1

600703029

A1expert

A2

A3L2L1

L4

S1S5

S3 U2

U1

390704012

A1expert

A2

A3L2L1

L4

S1S5

S3 U2

U1

Where are the students in each grade?

Prevalence of coarse codes by grade

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Gr 4 Gr 5 Gr 6 Gr 7 Gr 8 Gr 9 Gr 10

A

U

S

L

Test-focussed prevalence

Prevalence of L codes

0%

10%

20%

30%

40%

50%

60%

70%

80%

Gr4 Gr5 Gr6 Gr7 Gr8 Gr9 Gr10

L4

L2

L1

L drops exponentially (L = 440exp(-0.45*grade)) L2 about 5% in Grades 5-8: some just

accumulating facts, not changing concepts

“just a few little things still to learn”

Prevalence of A codes by grade

One quarter expert at Grade 5, one half in next 4 years, one quarter never

An issue for adult education e.g. “death by decimal”

Remember these are overestimates of expertise!

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%


A3

A2

A1

A1 = expert

Comparison Item

A1 A2 A3

4.8 4.63

5.736 5.62

4.7 4.08 -

4.4502 4.45 -

0.4 0.3 -

Prevalence of A codes by grade

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%


A3

A2

A1

Note:10% in the non-expert A category4.4502 / 4.45 is a difficult item – more about these later

A1 = expert

Prevalence of S codes

0%

5%

10%

15%

20%

25%


S5

S3

S1

•10-15%•Peaks in early secondary - probably curriculum effect

* This graph was incorrect in the original, and is corrected here.

Within the S region

Around 5% in S1 in all grades 0.6 < 0.7 0.5 <0.125 x

Around 10% in S3 in all grades (more Grade 8) 0.6 < 0.7 x 0.5 <0.125 x

Early studies did not ask the S3 question! S3 Possibilities:

analogy with fractions (one sixth, one seventh) analogy with negative numbers (-6, -7) doesn’t include any place value considerations

0%

5%

10%

15%

20%

25%


S5

S3

S1

Another look at prevalence

Which towns are most visited?

Another look at prevalence

Previously “test-focussedprevalence” - how many are at each town at eachtime?

Also “student-focussed prevalence” – how many students visit each town sometime on their journey?

S.F.P. > T.F.P. S.F.P. measured over time – in primary school

(Gr 4 - 6) and in secondary school (Gr 7 – 10) S. F. P. increases if you make more frequent

observations, so reports are under-estimates.

Student-focussed prevalence of codes amongst primary (left side) and secondary (right side) students

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

A1 L S A2, A3 U2 A1 L S A2, A3 U2

Student-focussed prevalence

SFP of A1 is 80%SFP of non-expert codes 15% - 30+%SFP primary different to SFP secondary


0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

A1 L S A2, A3 U2 A1 L S A2, A3 U2

Differences between TFP & SFP

TFP S < 25% SFP S = 35%

TFP S1 = 6% (most grades) SFP S1 (pri) = 17% SFP S1 (sec) = 10%

TFP estimates, SFP under-estimates


0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

A1 L S A2, A3 U2 A1 L S A2, A3 U2

PersistenceWhere do students stay the longest?

Questions about persistence

Persistence = percentage of students who retest in the same code after 1 semester

Persistence tells us: Which towns are the most attractive, in

the sense of hard to leave? Do students stay in one place for a long

time, or do they move around? Do experts stay as experts?

Persistence of A1

A1 is most persistent (Hurrah!) 90% retest as A1 next semester, 80% of A1 always stay A1 20% of “experts” don’t stay as expert – less than

lasting understanding Quick instruction before the test is better than

nothing! A1 students often go to A2

lucky guesses with truncation strategies which fail to give comparison decision (e.g. 4.4502 vs 4.45)

different repairs to faulty algorithms (Brown & vanLehn “bug migration” 1982) or faulty concepts

4.45024.45

Persistence over 1 semester Persistence of L1

around 30% Persistence of A2 and

S3 increases with age MERGA 2005: Sec S

about 40% more likely to remain in S than Pri S (stat. sig.)

Effect holds across schools

Conclusion: Naïve L ideas are challenged by secondary school but S and non-expert A ideas are supported by school practices (e.g. always rounding off) and new ideas in curriculum.

0%

10%

20%

30%

40%

50%

Gr 4/5 Gr 6 Gr 7 Gr 8 Gr 9/10

A2 L1 S3

Persistence over 1, 2, 3, 4 semesters

About 35% of L,S,U retest same after one semester

About 15% retest same after 2.5 years

Schooling is not impacting on ideas!

0%

10%

20%

30%

40%

50%

8 months 16 months 24 months 32 months

L S U

Proximity to Expertise

Which town is the best place to be?

From which non-A1 code is it most likely that a student who changes code* will become an expert on the next test?

* Proximity is independent of persistence


Proximity to A1 (secondary)

0 20 40 60 80

A2

A3

U1

S1

L2

S3

S5

L4

L1

Chance that next test is A1

Proximity to A1 (primary)

0 20 40 60 80

A2

A3

U1

S1

L2

S3

S5

L4

L1


Proximity to Expertise General hierarchy

A > U > S > L A2 is nearest (but

maybe VERY LITTLE place value understanding )

U1 (unclassified) is high – why is not having a definite misconception better?


0 20 40 60 80

A2

A3

U1

S1

L2

S3

S5

L4

L1



0 20 40 60 80

A2

A3

U1

S1

L2

S3

S5

L4

L1




S better than L in primary( as inferred from previous research)

L better than S in secondary – why?

S pri student has 60% more chance of moving to expertise than S sec

L pri student has 30% less chance of moving to expertise than L sec


0 20 40 60 80

A2

A3

U1

S1

L2

S3

S5

L4

L1



0 20 40 60 80

A2

A3

U1

S1

L2

S3

S5

L4

L1



A1expert

A2

A3L2L1

L4

S1S5

S3 U2

U1

Proximity to A1(primary)

A1expert

A2

A3L2L1

L4

S1S5

S3 U2

U1

Proximity to A1(secondary)

Lessons about learning (1)

Variation in age when expertise is attained Too many never make it Many with little understanding hide it (e.g.

A2) Schooling not IMPACTING on fundamental

ideas – 15% persistence over 4 semesters Harder to shake ideas of students with a

specific misconception – students need to be shown they have something to learn

Importance of looking at prevalence in both ways

Lessons about learning (2)

Different misconceptions have different causes and are impacted differently by the learning environment L1 - naïve, first guess without teaching,

decreases in prevalence S – supported by features in the

curriculum, operating at deep psychological level, so reinforced especially in secondary school

“Just a few little things left to learn”

Contrast between orientation to learning principles vs accumulating facts expert: a few math’l principles requiring mastery

of a web of complex relations between them some students and teachers: a large number of

facts to learn with weak links between them Maths education has a challenge to properly

deal with this accumulated facts approach for research (DCT2 was weak here) for improving teaching and learning of such

teachers and such students

How to use a detailed analysis of students’ thinking

Study grappled with what grain size of detail is useful – eventually worked with two (but both finer and coarser possible)

Coarse analysis is useful for human teaching Fine analysis is useful for machine teaching

several games using artificial intelligence to present the items from which a student can

see that there is something for him/her to learn learn something new.

Good results from just a little attention to this.

Flying Photographer

Hidden Numbers

Features of games and other instruction: students need to find out that they don’t know everyone needs to learn about the same principles different students need to learn in the context of different

items

Their teaching No special treatment

(but see our website & papers for many suggestions)

Usually start decimals around Grade 4, e.g. as alternative notation for tenths

Common modelsMAB, area (but better to use length)

Often restricted to one or two place decimals for a long time (Brousseau and others comment on this)

Rounding to 2dp standard in later years

0.13

Better to use length

0.2 0.28 0.3

Stacey, Helme, Archer, Condon – Ed. Studies Math.(2001)Accessibility leads to better retention and classroom discussion

Thank you

http://extranet.edfac.unimelb.edu.au/DSME/decimals

travelling the road to expertise: a longitudinal study of learning

Documents