Download - BSherin LAK presentation
![Page 1: BSherin LAK presentation](https://reader031.vdocuments.mx/reader031/viewer/2022021923/5a6defce7f8b9ae3258b4cc7/html5/thumbnails/1.jpg)
Using computational methods to discover
student science conceptions in interview
data
Bruce Sherin
School of Education and Social Policy
Northwestern University
LAK 2012
![Page 2: BSherin LAK presentation](https://reader031.vdocuments.mx/reader031/viewer/2022021923/5a6defce7f8b9ae3258b4cc7/html5/thumbnails/2.jpg)
Goals of this work
• Use computational analytic methods with traditional data
• Videos of interviews intended to study kids’ “prior conceptions” in
science.
• Automate the traditional analysis
• The traditional analysis:
1. Identify a set of “conceptions”
2. Code the data in terms of these conceptions.
• Go as far as possible with simple analytic techniques.
![Page 3: BSherin LAK presentation](https://reader031.vdocuments.mx/reader031/viewer/2022021923/5a6defce7f8b9ae3258b4cc7/html5/thumbnails/3.jpg)
Some specifics
• The data: 54 interviews with middle school students.
• The subject matter: The earth’s seasons
• The approach: Simple vector space models, clustering
There are reasons to think automating the analysis of this
data should be difficult
• The amount of data is small
• Student speech is halting and ambiguous
• Gestures and diagrams are important
![Page 4: BSherin LAK presentation](https://reader031.vdocuments.mx/reader031/viewer/2022021923/5a6defce7f8b9ae3258b4cc7/html5/thumbnails/4.jpg)
Prior science conceptions• Prior science conceptions: The prior understandings that students
bring to science learning
• Bibliography by Duit (2009) lists over 8000 papers
Health and disease
Genetics
Evolution
Geologic Time
Nature of matter
Ecosystems
Water cycle
Weather
• Two theoretical poles:
• Theory-Theory: Prior science knowledge consists of relatively well-
elaborated theories.
• Knowledge-in-Pieces. Prior science knowledge consists of a moderately
large number of not-well-organized conceptions.
• In an interview, students may construct explanations in-the-moment, drawing on
some of these conceptions.
![Page 5: BSherin LAK presentation](https://reader031.vdocuments.mx/reader031/viewer/2022021923/5a6defce7f8b9ae3258b4cc7/html5/thumbnails/5.jpg)
The seasons corpus
• 54 interviews with middle school students
• Our interview protocol, in brief:
1. “Why is it warmer in the summer and colder in the winter?”
2. Follow up questions for clarification.
3. Asked to make a drawing.
4. Follow up questions for clarification.
5. Challenges for certain answers.
![Page 6: BSherin LAK presentation](https://reader031.vdocuments.mx/reader031/viewer/2022021923/5a6defce7f8b9ae3258b4cc7/html5/thumbnails/6.jpg)
Prototypical explanations
![Page 7: BSherin LAK presentation](https://reader031.vdocuments.mx/reader031/viewer/2022021923/5a6defce7f8b9ae3258b4cc7/html5/thumbnails/7.jpg)
Example Interview: Edgar
Starts with side-based, emphasizing the sun’s rays:
E: Here’s the earth slanted. Here’s the axis. Here’s the
North Pole, South Pole, and here’s our country. And
the sun’s right, and the rays hitting like directly right
here. So everything’s getting hotter over the summer
and once this thing turns, the country will be here
and the sun can't reach as much. It's not as hot as
the winter.
Shifts to typical closer-farther
E: Actually, I don't think this moves it turns and it moves like that and it
turns and that thing like is um further away once it orbit around the s-
Earth- I mean the sun.
![Page 8: BSherin LAK presentation](https://reader031.vdocuments.mx/reader031/viewer/2022021923/5a6defce7f8b9ae3258b4cc7/html5/thumbnails/8.jpg)
Example Interview: Zelda
Tilt-based explanation, with the tilt causing light to be more or less direct
Z: Because, I think because the earth is on a tilt, and then, like that side of
the Earth is tilting toward the sun, or it’s facing the sun or something so
the sun shines more directly on that area, so its warmer.
![Page 9: BSherin LAK presentation](https://reader031.vdocuments.mx/reader031/viewer/2022021923/5a6defce7f8b9ae3258b4cc7/html5/thumbnails/9.jpg)
Example Interview: Caden
Tilt-based explanation, with the tilt causing closer-farther
I: So the first question is why is it warmer in the summer and colder in the
winter?
C: Because at certain points of the earth’s rotation, orbit around the sun,
the axis is pointing at an angle, so that sometimes, most times,
sometimes on the northern half of the hemisphere is closer to the sun
than the southern hemisphere, which, change changes the
temperatures. And then, as, as it’s pointing here, the northern
hemisphere it goes away, is further away from the sun and get’s colder.
I: Okay, so how does it, sometimes the northern hemisphere is, is toward
the sun and sometimes it’s away?
C: Yes because the at—I’m sorry, the earth is tilted on its axis. And it’s
always pointed towards one position.
![Page 10: BSherin LAK presentation](https://reader031.vdocuments.mx/reader031/viewer/2022021923/5a6defce7f8b9ae3258b4cc7/html5/thumbnails/10.jpg)
Analysis Procedure
1. Clean transcripts, removing everything except words
spoken by students
2. Break each transcript into 100-word segments, with a
moving window that steps forward 25 words
• Results in 794 segments
3. Map each segment to a vector
4. DeviationalizeTM the vectors
5. Cluster the vectors
6. Interpret the clusters
7. Apply clusters to analyze transcripts
![Page 11: BSherin LAK presentation](https://reader031.vdocuments.mx/reader031/viewer/2022021923/5a6defce7f8b9ae3258b4cc7/html5/thumbnails/11.jpg)
Mapping segments to vectors4
2
0
2
1
1
2
0
1
3
…
2.1
1.7
0
1.7
1
1
1.7
0
1
2.1
…
sun
earth
side
away
tilted
closer
axis
day
farther
time
…
• Compile the vocabulary
• Stop list consisting of 782 words
• Results in vocabulary with 647 words
• For each segment count number of
occurrences of each of these words.
• Weight as 1 + log(count)
• Normalize
• Result: 794 vectors, each with 647
dimensions.
![Page 12: BSherin LAK presentation](https://reader031.vdocuments.mx/reader031/viewer/2022021923/5a6defce7f8b9ae3258b4cc7/html5/thumbnails/12.jpg)
DeviationalizeTM
• Average all of the segment vectors and replace each by
their difference from this average.
![Page 13: BSherin LAK presentation](https://reader031.vdocuments.mx/reader031/viewer/2022021923/5a6defce7f8b9ae3258b4cc7/html5/thumbnails/13.jpg)
Cluster
• Use hierarchical agglomerative clustering
# of clusters Sizes of the clusters
10 19 72 9 68 140 62 44 122 136 122
9 19 72 68 62 44 122 136 122 149
8 19 72 68 44 122 136 122 211
7 72 68 44 122 122 211 155
6 68 44 122 122 211 227
5 68 122 122 211 271
4 122 122 271 279
3 271 279 244
What do the clusters mean?
![Page 14: BSherin LAK presentation](https://reader031.vdocuments.mx/reader031/viewer/2022021923/5a6defce7f8b9ae3258b4cc7/html5/thumbnails/14.jpg)
![Page 15: BSherin LAK presentation](https://reader031.vdocuments.mx/reader031/viewer/2022021923/5a6defce7f8b9ae3258b4cc7/html5/thumbnails/15.jpg)
Apply clusters to transcripts
For each transcript:
• Segment into 100-word chunks
• Find the vector for each segment
• For each segment, find the dot product between the
segment vector and each of the cluster centroids
• Plot the results
![Page 16: BSherin LAK presentation](https://reader031.vdocuments.mx/reader031/viewer/2022021923/5a6defce7f8b9ae3258b4cc7/html5/thumbnails/16.jpg)
EdgarStarts with side-based, emphasizing the sun’s rays:
E: … and the rays hitting like directly right here. … once this thing turns, the
country will be here and the sun can't reach as much
Shifts to typical closer-farther
E: that thing like is um further away once it orbit around the s- Earth- I
mean the sun.
![Page 17: BSherin LAK presentation](https://reader031.vdocuments.mx/reader031/viewer/2022021923/5a6defce7f8b9ae3258b4cc7/html5/thumbnails/17.jpg)
ZeldaTilt-based explanation, with the tilt causing light to be more or less direct
Z: … that side of the Earth is tilting toward the sun, or it’s facing the sun or
something so the sun shines more directly on that area, so its warmer.
![Page 18: BSherin LAK presentation](https://reader031.vdocuments.mx/reader031/viewer/2022021923/5a6defce7f8b9ae3258b4cc7/html5/thumbnails/18.jpg)
CadenTilt-based explanation, with the tilt causing closer-farther
C: … the axis is pointing at an angle, so that sometimes … the northern
half of the hemisphere is closer to the sun… .
![Page 19: BSherin LAK presentation](https://reader031.vdocuments.mx/reader031/viewer/2022021923/5a6defce7f8b9ae3258b4cc7/html5/thumbnails/19.jpg)
Summary
• Used traditional data set:
• Videos of interviews intended to study kids’ “prior conceptions” in
science.
• Set out to produce a “knowledge-in-pieces” analysis
• Notable difficulties:
• Small amount of data
• Halting and ambiguous speech.
• Gestures, diagrams are referenced
• Keep the methods as simple as possible
• Deviationalizing is an exception
• Results are “suggestive”
• (That we can capture features of student knowledge that are widely
recognized to be important)
![Page 20: BSherin LAK presentation](https://reader031.vdocuments.mx/reader031/viewer/2022021923/5a6defce7f8b9ae3258b4cc7/html5/thumbnails/20.jpg)
What does this buy me?
What role might these computational techniques play in the
toolkit of researchers who study prior conceptions science
students?
• Can we replace human coders?
• Actually, a human played an important role here.
• Can play a role as kind of independent support for the
work of human analysts!
![Page 21: BSherin LAK presentation](https://reader031.vdocuments.mx/reader031/viewer/2022021923/5a6defce7f8b9ae3258b4cc7/html5/thumbnails/21.jpg)
Open issues and next steps
1. Apply to subject matter other than the seasons
2. Systematic comparison to human analysis
3. Apply to answer some new research questions
4. Systematic investigation of alternative analysis methods
• In the paper: (1) Different segment size, (2) Without deviationalizing
5. Why does this work?