knowing what students know
DESCRIPTION
In an interactive digital-game, traces of a learner’s progress, problem-solving attempts, self-expressions and social communications can entail highly detailed and time-sensitive computer-based documentation of the context, actions, processes and products. This talk will present measurement and analysis considerations that are needed to address the challenges of finding patterns and making inferences based on these data. Methods based in data-mining, machine learning, model-building and complexity theory form a new theoretical foundation for dealing with the challenges of time sensitivity, spatial relationships, multiple layers of aggregations at different scales, and the dynamics of complex behavior spaces. Examples of these considerations in game-based learning analytics are presented and discussed, with implications for game-based e-learning design.TRANSCRIPT
![Page 1: Knowing what students know](https://reader034.vdocuments.mx/reader034/viewer/2022052523/5561da2bd8b42aa83f8b5c4a/html5/thumbnails/1.jpg)
Knowing what students know from game-based learning
David GibsonCurtin University
![Page 2: Knowing what students know](https://reader034.vdocuments.mx/reader034/viewer/2022052523/5561da2bd8b42aa83f8b5c4a/html5/thumbnails/2.jpg)
The Premise
In an interactive digital-game, traces of a learner’s progress, problem-solving attempts, self-expressions and social communications can entail highly detailed and time-sensitive computer-based documentation of the context, actions, processes and products.
![Page 3: Knowing what students know](https://reader034.vdocuments.mx/reader034/viewer/2022052523/5561da2bd8b42aa83f8b5c4a/html5/thumbnails/3.jpg)
Example
• Contexts: Farm, Playground, Science Lab• Actions: Talking, Testing, Walking to…• Processes & Products: Test Results, Explanations
Clarke-Midura & Gibson, 2013
![Page 4: Knowing what students know](https://reader034.vdocuments.mx/reader034/viewer/2022052523/5561da2bd8b42aa83f8b5c4a/html5/thumbnails/4.jpg)
Interaction Traces = Evidence
There is a need for new frameworks, concepts and methods for measuring what someone knows and can do based on game interactions and artifacts created during serious play
Why? (It’s a mouthful) Ubiquitous, unobtrusive, interactive big data created by people working in digital media performance spaces
![Page 5: Knowing what students know](https://reader034.vdocuments.mx/reader034/viewer/2022052523/5561da2bd8b42aa83f8b5c4a/html5/thumbnails/5.jpg)
Example
• Ecological rationality & Empirical probabilityClarke-Midura & Gibson, 2013
![Page 6: Knowing what students know](https://reader034.vdocuments.mx/reader034/viewer/2022052523/5561da2bd8b42aa83f8b5c4a/html5/thumbnails/6.jpg)
Sensors
• Wireless EEG– Facial muscles, emotional
clusters, raw EEG• Wireless Galvanic Skin
Conductance – Arousal level
• Eye Tracker– Gaze-point, duration, mouse-
clicks• Haptics– Button presses, head tilt
![Page 7: Knowing what students know](https://reader034.vdocuments.mx/reader034/viewer/2022052523/5561da2bd8b42aa83f8b5c4a/html5/thumbnails/7.jpg)
Anatomy of the System
Helen Chavez & Javier Gomez, ASU
![Page 8: Knowing what students know](https://reader034.vdocuments.mx/reader034/viewer/2022052523/5561da2bd8b42aa83f8b5c4a/html5/thumbnails/8.jpg)
Challenge: New Psychometrics
• What are some of the measurement and analysis considerations needed to address the challenges of finding patterns and making inferences based on data from digital learning experiences?
![Page 9: Knowing what students know](https://reader034.vdocuments.mx/reader034/viewer/2022052523/5561da2bd8b42aa83f8b5c4a/html5/thumbnails/9.jpg)
Biometric Sensor Nets
• What patterns do we find?
• How do they change over time?
• How do they relate to baseline and experimental activities?
![Page 10: Knowing what students know](https://reader034.vdocuments.mx/reader034/viewer/2022052523/5561da2bd8b42aa83f8b5c4a/html5/thumbnails/10.jpg)
Network Graphs
Digraphs illustrate structural relationships in the causative factors during a time slice or event frame.
![Page 11: Knowing what students know](https://reader034.vdocuments.mx/reader034/viewer/2022052523/5561da2bd8b42aa83f8b5c4a/html5/thumbnails/11.jpg)
Network Analysis
Adjacency tablesCentrality
Digraphs
AF3 F7 F3 FC5 T7 P7 O1 O2 P8 T8 FC6 F4 F8 AF4 GX GY
AF3
F7
F3
FC5
T7
P7
O1
O2
P8
T8
FC6
F4
F8
AF4
GX
GY
![Page 12: Knowing what students know](https://reader034.vdocuments.mx/reader034/viewer/2022052523/5561da2bd8b42aa83f8b5c4a/html5/thumbnails/12.jpg)
Symbolic Regression
Automated search for algorithms
Clarke-Midura & Gibson, 2013
![Page 13: Knowing what students know](https://reader034.vdocuments.mx/reader034/viewer/2022052523/5561da2bd8b42aa83f8b5c4a/html5/thumbnails/13.jpg)
New Space for Performance
• Unfold in time • Cover a multivariate space of possible actions• Assets contain both intangible (e.g. value,
meaning, sensory qualities, and emotions) and tangible components (e.g. media, materials, time and space)
NOTE: Asset utilization during performance provides evidence of what a user knows and can do
![Page 14: Knowing what students know](https://reader034.vdocuments.mx/reader034/viewer/2022052523/5561da2bd8b42aa83f8b5c4a/html5/thumbnails/14.jpg)
Example
Students who had this pattern of resources were most likely to show evidence of forming a hypothesis
Clarke-Midura & Gibson, 2013
![Page 15: Knowing what students know](https://reader034.vdocuments.mx/reader034/viewer/2022052523/5561da2bd8b42aa83f8b5c4a/html5/thumbnails/15.jpg)
Performance Space Features
• Unconstrained complex multidimensional stimuli and responses
• Dynamic adaptation of items to user, which entails interactivity and dependency
• Nonlinear behaviors with both temporal and spatial components
NOTE: Higher order and creative thinking is supported in such a space
![Page 16: Knowing what students know](https://reader034.vdocuments.mx/reader034/viewer/2022052523/5561da2bd8b42aa83f8b5c4a/html5/thumbnails/16.jpg)
Research Questions
• What patterns are found within & between sensors?
• How do these patterns relate to baseline and experimental activities?
![Page 17: Knowing what students know](https://reader034.vdocuments.mx/reader034/viewer/2022052523/5561da2bd8b42aa83f8b5c4a/html5/thumbnails/17.jpg)
Data Dashboard at ASUHelen Chavez and Javier Gomez
![Page 18: Knowing what students know](https://reader034.vdocuments.mx/reader034/viewer/2022052523/5561da2bd8b42aa83f8b5c4a/html5/thumbnails/18.jpg)
Thinking States
Rise inuncertainty and interest
Agreement & concentration drop
During thinking
![Page 19: Knowing what students know](https://reader034.vdocuments.mx/reader034/viewer/2022052523/5561da2bd8b42aa83f8b5c4a/html5/thumbnails/19.jpg)
The Game-Based Psychometric Landscape
• A “do over” for performance assessment• New ways of performing = new methods of
data capture, analysis and display• Complex tasks and artifacts containing– higher order thinking (e.g. decision sequences)– physical performances demonstrating skills– emotional responses
![Page 20: Knowing what students know](https://reader034.vdocuments.mx/reader034/viewer/2022052523/5561da2bd8b42aa83f8b5c4a/html5/thumbnails/20.jpg)
What Games & Sims Teach
• Understanding big ideas - systems knowledge• Dealing with time and scale• Practice in decision-making• Active problem-solving• Concepts, strategies, & tactics• Understanding processes beyond experience• Practice makes improvement
(Aldrich, 2005)
![Page 21: Knowing what students know](https://reader034.vdocuments.mx/reader034/viewer/2022052523/5561da2bd8b42aa83f8b5c4a/html5/thumbnails/21.jpg)
Conclusion
Methods based in data-mining, machine learning, model-building and complexity theory form a theoretical foundation for dealing with the challenges of time sensitivity, spatial relationships, multiple layers of aggregations at different scales, and the dynamics of complex behavior spaces.