modelling early mathematical competencies and misconceptions erasmus intensive seminar graz 2005...
Post on 20-Dec-2015
219 views
TRANSCRIPT
Modelling early mathematical competencies and misconceptions
Erasmus Intensive Seminar Graz 2005
Gisela Dösinger
Objective
Development of an instrument Adaptive assessment Broad range of early mathematical knowledge Including misunderstandings Against the background of internalisation Remedial instruction
Motivation
Analysis of existing instruments Neglecting prenumerical knowledge Only limited sub-domains assessed Not varying presentation format Only coarsely defining which competencies assessed Not covering misunderstandings Redundant assessment
Young children and children with disabilities
Methodology
For modelling correct knowledge Competence - Performance Theory Korossy (1993)
For modelling misconceptions Information System Based Approach Scott (1982) Applied by Lukas (1997)
Investigation 1
Brief introduction to competence - performance theory
Performance: observable solution behaviour Competence: underlying knowledge Latent level explains manifest level Levels related to each other
Competence - Performance Theory
Investigation 1
Performance structure (Q,P) Competence structure (E,K)
Interpretation function k: Q (K)kq: { 1, 2,…,r }
Representation function p: K (Q)
Competence - Performance Theory
Investigation 1
Step 1. Define solution ways Step 2. Represent solution steps by competencies
L = { f(q)|qQ} = { {3},{1,2},{2,4},{3,4} , ... , {2,3,4,6}}
q f(q)a {3},{1,2}b {2,4},{3,4}c {2},{1,3}d {2,5},{3,5}e {1,2,4,6},{2,3,4,6}
Competence - Performance Theory
Investigation 1
Step 3. Construct competence space 3.1 Apply surmise function
3.2 Close basis under union
e ke s(e)1 {1,2},{1,3},{1,2,4,6} {1,2},{1,3}2 {1,2},{2,4},{2},{2,5},{1,2,4,6},{2,3,4,6} {2}3 {3},{3,4},{1,3},{3,5},{2,3,4,6} {3}4 {2,4},{3,4},{1,2,4,6},{2,3,4,6} {2,4},{3,4}5 {2,5},{3,5} {2,5},{3,5}6 {1,2,4,6},{2,3,4,6} {1,2,4,6},{2,3,4,6}
Competence - Performance Theory
Investigation 1
Step 4. Apply interpretation function
q kq
a {3},{3,5},{3,4},{1,2},{2,3},{1,2,3},{3,4,5},{1,2,5} …b {2,4},{3,4},{3,4,5},{1,2,4},{2,3,4},{1,3,4},{1,2,3,4} …c {2},{2,5},{2,4},{1,2},{2,3},{1,3},{1,2,3} …d {2,5},{3,5},{3,4,5},{2,4,5},{1,2,5},{2,3,5} …e {1,2,4,6},{2,3,4,6},{1,2,3,4,6},{1,2,4,5,6} …
Competence - Performance Theory
Investigation 1
Step 5. Apply representation function
k p(k)
… …{1,3,4} {a,b,c}{3,4,5} {a,b,d}{1,2,3,4,5} {a,b,c,d}... …
Competence - Performance Theory
Investigation 1
Step 6. Construct performance space 6.1 Apply surmise function
6.2 Close basis under union
q pq s(q)
a {a},{a,d},{a,b},{a,c},{a,b,d},{a,c,d},{a,b,c},{a,b,c,d},{a,b,c,e},{a,b,c,d,e} {a}b {a,b},{b,c},{a,b,d},{a,b,c},{a,b,c,d},{a,b,c,e},{a,b,c,d,e} {a,b},{b,c}c {c},{a,c},{c,d},{b,c},{a,c,d},{a,b,c},{b,c,d},{a,b,c,d},{a,b,c,e},{a,b,c,d,e} {c}d {a,d},{c,d},{a,b,d},{a,c,d},{b,c,d},{a,b,c,d},{a,b,c,d,e} {a,d},{c,d}e {a,b,c,e},{a,b,c,d,e} {a,b,c,e}
Competence - Performance Theory
Investigation 1
State of research on early mathematical knowledge Proto-quantitative schemata Enumerative processes Calculation: addition and subtraction Internalisation
Twenty-six competencies and three internalisation levels Derive dependencies among competencies
Early mathematical knowledge
Investigation 1
Problem construction: making competencies accessible Take a competency Search available instruments for problems Describe solution way Represent solution steps by competencies Supplement competencies by prerequisites Copy problem to other internalisation levels
Forty-nine problems in eighteen problem classes
Problem construction
Investigation 1
A
B
F C
G
R={(A,B),(A,C),(A,F),(A,G),(B,C),(B,F),(B,G),(F,G),(C,G)}P={{A},{A,B},{A,B,F},{A,B,C},{A,B,C,F},{A,B,C,F,G}}
Deriving problem order
Investigation 1
Hypotheses For each pair element of the relation it is expected
that more difficult problem…
…is solved less frequent than or as frequent as less difficult problem
…is not solved if the less difficult problem is not solved
The performance states are expected to fit the empirical solution patterns
Hypotheses
Investigation 1
Method Ninety-four kindergarteners Mean 62.64, standard deviation 9.89 Problems partitioned into subsets 13 problems each Overlapping substructures Subjects tested individually
Method
Investigation 1
Solution frequency Solution frequencies in accordance with hypothesis 1
A .. 74%
B .. 79%
F .. 42%
C .. 89%
G .. 37%
Results
Investigation 1
Gamma-Index Derived from -Index (Goodman & Kruskal, 1954) Measure of association indicating whether two
classifications are ordered likely or unlikely
Results
Investigation 1
Gamma-Index
Gamma-Indices varying from 0.64 to 0.96, significantly differing from 0, thus supporting hypothesis 2
dc
dc
NN
NNG
Results
qj
1 0qi 1 N c
0 N d
dc
dc
NN
NN
2
2
Investigation 1
Symmetric distance
Distance distribution, average symmetric distance, and standard deviation are calculated
Distance distributions compared to 'random' ones ‘Random‘ distributions obtained by using power set as
data set
Results
MNNMNMNMd \\min,min
Investigation 1
Symmetric distance
Symmetric distances ranging from 0.13 to 0.94, with distance distributions significantly differing from 'random‘ ones ,supporting hypothesis 3
Results
Distance Random data Empirical dataAbs. frequency Rel. Frequency Obs. frequency Exp. frequency Chi²
0 11 0,3438 54 34,38 11,201 16 0,5000 29 50,00 8,822 5 0,1562 17 15,62 0,12
32 1,0000 100 100,00 20,14
Investigation 1
Distance Agreement Coefficient Proposed by Schrepp (1993) For comparing fit of different knowledge structures
The smaller, the better the fit
pot
dat
d
dDA
Results
Investigation 1
Distance Agreement Coefficient Adapted for testing relative fit of one knowledge
structure Compare DA to its maximal possible value which is
got when ddat is set to its maximal possible value
Distance Agreement Coefficient ranging from 0.05 to 0.32, much smaller than DAmax which was about 2, thus supporting hypothesis 3
Results
Investigation 1
Reproducibility Coefficient Proposed by Guttman (1944) Another measure explaining extent of concordance
between data and hypothesised structure Proportion of cells explained by model Example: a b c d
np
dREP
1a b c d
Subject 1 1 1 1 0Subject 2 0 1 0 1Subject 3 1 0 0 0Subject 4 1 1 0 0
Results
Investigation 1
Reproducibility Coefficient Reproducibility coefficient ranging from 0.93 to 0.99,
thus supporting hypothesis 3
Results
Investigation 1
Substructures proved valid Adaptive assessment Problems on different internalisation levels and on proto-
quantitative/quantitative level not varying in difficulty Use of abstract materials: difficulty to transfer knowledge
from concrete, everyday materials Reason for large number of competencies: broad range
intended to be covered and fine grained dissolution required
Discussion
Investigation 2
Brief introduction to information system based approach Manifest level: correct solutions and bugs Latent level: competencies and misconceptions
Two main concepts on the latent level Implication Incompatibility
Information system based approach
Investigation 2
Implication and incompatibility impose an algebraic structure
Information system A is a structure
‹D,Con, › where
D is set of data objects XCon is a set of finite consistent subsets of D is a binary relation ConD
Information system based approach
Investigation 2
On manifest level there is a polytomous response format: correct responses, bugs, and slips
For every qQ there is a set of possible responses
Rq = {q0,q1,q2,…,qn}
If R = Rq \ q0 a response pattern T is a subset of R
Information system based approach
Investigation 2
Relating latent and manifest level
Gq: A Rq
G: A (R)
G(x) = Gq(x)|q0
Information system based approach
Investigation 2
Identify set of knowledge entities D and their structure
Invariance principle a Additive principle b Spatial distortion c
b a c incompatible with a and b
Information system based approach
Investigation 2
Derive information system A Build power set Cancel inconsistent subsets Cancel subsets incompatible with
Con = {{},{a},{c},{a,b}}
a b c
0 0 0
1 0 0
0 1 0
0 0 1
1 1 0
1 0 1
0 1 1
1 1 1
Information system based approach
Investigation 2
Construct problems and determine responses Problem A
Spatial, number conserving change: row-to-circle
A1…same number, A2…more, A3…less Problem B
Spatial, number conserving change: spread-row
B1…same number, B2…more Problem C
Splitting set into subsets: partition-set
C1…same number, C2…more
Information system based approach
Investigation 2
Relate responses to elements xof information system
Determine response patterns
R q \q 0 x
A1 {a}
B1 {a}
C1 {a,b}
A2 {c}
A3 {c}
B2 {c}C2 {c}
Information system based approach
x T{} {}{a} {A1,B1}
{c} {A2,B2,C2},{A3,B2,C2}
{a,b} {A1,B1,C1}
Investigation 2
Measure for validating structure: discrepancy between empirical and hypothetical response patterns Number of problems in which patterns disagree
- 2
Comparison to 'random' case: randomly generated response patterns
U-Test for testing statistical significance
21,1 ,cba
Measure for validation
112 ,, cba
Investigation 2
Method Sixty-four kindergarteners Mean 61.84, standard deviation 9.40 Problems partitioned into subsets 12 problems each Subjects tested individually
Method
Investigation 2
Results Proportions of correct and buggy solutions Discrepancy
Discouraging output Re-modelling
Results
Investigation 2
No valid model found, but… Set of misconceptions identified Problems able to provoke their application designed Empirical evidence proven
No age effect Application of misconceptions seems to depend from
kind of problem Bugs arising from perceptual distraction play important
role
Discussion
Investigation 2
Misconceptions are not stable better use a probabilistic approach
Information system based approach: implications need to be neglected, because only excluding responses can be contained in response pattern
Discussion