#gamesur conference: from body signals to brainy player insights
TRANSCRIPT
![Page 1: #GamesUR Conference: From Body Signals to Brainy Player Insights](https://reader031.vdocuments.mx/reader031/viewer/2022030213/589d82db1a28ab69538b709f/html5/thumbnails/1.jpg)
From Body Signals to Brainy Player Insights
Lennart Nacke, Ph.D. Associate Professor
CC by Lennart Nacke/Flickrwww.hcigames.com, @acagamic, #gamesUR, [email protected]
![Page 2: #GamesUR Conference: From Body Signals to Brainy Player Insights](https://reader031.vdocuments.mx/reader031/viewer/2022030213/589d82db1a28ab69538b709f/html5/thumbnails/2.jpg)
About me• Associate Professor
• Director of the HCI Games Group at the University of Waterloo
![Page 3: #GamesUR Conference: From Body Signals to Brainy Player Insights](https://reader031.vdocuments.mx/reader031/viewer/2022030213/589d82db1a28ab69538b709f/html5/thumbnails/3.jpg)
CC by Liz Henry/Flickr
Body Signals
![Page 4: #GamesUR Conference: From Body Signals to Brainy Player Insights](https://reader031.vdocuments.mx/reader031/viewer/2022030213/589d82db1a28ab69538b709f/html5/thumbnails/4.jpg)
Electrodermal Activity• EDA
• Primary index of arousal
• Increased sweat gland activity
• Sympathetic nervous system
![Page 5: #GamesUR Conference: From Body Signals to Brainy Player Insights](https://reader031.vdocuments.mx/reader031/viewer/2022030213/589d82db1a28ab69538b709f/html5/thumbnails/5.jpg)
Facial Electromyography• EMG • Muscle contraction result in electrical activity • Primary index of hedonic valence
Brow muscle (negative valence)
Periocular muscle (high arousal, positive valence)
Cheek muscle (positive valence)
![Page 6: #GamesUR Conference: From Body Signals to Brainy Player Insights](https://reader031.vdocuments.mx/reader031/viewer/2022030213/589d82db1a28ab69538b709f/html5/thumbnails/6.jpg)
C3 CMS DRLCz C4
F3 F4
FC5 FC1 FC2 FC6
Fz
Pz
Oz
P4
AF4
Fp2
PO4
O2O1
CP2
F8
T8T7
P8
CP6
PO3
P3P7
CP1CP5
AF3
F7
Fp1
Chatrian, G. E., Lettich, E., & Nelson, P. L. (1988). Modified nomenclature for the "10%" electrode system. Journal of Clinical Neurophysiology: Official Publication of the American Electroencephalographic Society, 5(2), 183-186.
Frontal (F)
Parietal (P)
Occipital (O)
Temporal (T)Central (C)
Odd Numbers Even Numbers
Electroencephalography
![Page 7: #GamesUR Conference: From Body Signals to Brainy Player Insights](https://reader031.vdocuments.mx/reader031/viewer/2022030213/589d82db1a28ab69538b709f/html5/thumbnails/7.jpg)
Psychophysiological Inference
• Body and Mind Relations • One-to-one • One-to-many • Many-to-one • Many-to-many • Null relations
• Measure operation and activity of • Muscles • Nerve cells • Glands
![Page 8: #GamesUR Conference: From Body Signals to Brainy Player Insights](https://reader031.vdocuments.mx/reader031/viewer/2022030213/589d82db1a28ab69538b709f/html5/thumbnails/8.jpg)
Raw Physiological Signal (e.g., EMG)
0 1 2 3 4 5 6 7
x 104
6
6.5
7
7.5
8
8.5
9
9.5
10
10.5
11
![Page 9: #GamesUR Conference: From Body Signals to Brainy Player Insights](https://reader031.vdocuments.mx/reader031/viewer/2022030213/589d82db1a28ab69538b709f/html5/thumbnails/9.jpg)
Signal Processing
1. Rectify signal (RMS)
2. Filter (bandpass)
3. Smooth (average window)
4. If EEG → Fast-Fourier Transformation
![Page 10: #GamesUR Conference: From Body Signals to Brainy Player Insights](https://reader031.vdocuments.mx/reader031/viewer/2022030213/589d82db1a28ab69538b709f/html5/thumbnails/10.jpg)
Better EMG Signals
![Page 11: #GamesUR Conference: From Body Signals to Brainy Player Insights](https://reader031.vdocuments.mx/reader031/viewer/2022030213/589d82db1a28ab69538b709f/html5/thumbnails/11.jpg)
Normalized Skin Conductance
![Page 12: #GamesUR Conference: From Body Signals to Brainy Player Insights](https://reader031.vdocuments.mx/reader031/viewer/2022030213/589d82db1a28ab69538b709f/html5/thumbnails/12.jpg)
CC by Nefci/Flickr
![Page 13: #GamesUR Conference: From Body Signals to Brainy Player Insights](https://reader031.vdocuments.mx/reader031/viewer/2022030213/589d82db1a28ab69538b709f/html5/thumbnails/13.jpg)
Biometric Storyboards• A single User
Experience (UX) graph • Intended UX • Actual UX
• Comparison study (N=24) of different player testing approaches
Pejman Mirza-Babaei, Lennart E. Nacke, John Gregory, Nick Collins, and Geraldine Fitzpatrick. 2013. How does it play better?: exploring user testing and biometric storyboards in games user research. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '13). ACM, New York, NY, USA, 1499-1508. DOI=10.1145/2470654.2466200
![Page 14: #GamesUR Conference: From Body Signals to Brainy Player Insights](https://reader031.vdocuments.mx/reader031/viewer/2022030213/589d82db1a28ab69538b709f/html5/thumbnails/14.jpg)
Repidly• Online tool
• Rapid evaluation
• Annotate videos and physiological signals
Bachelor’s Thesis project from Mike Schaekermann
![Page 15: #GamesUR Conference: From Body Signals to Brainy Player Insights](https://reader031.vdocuments.mx/reader031/viewer/2022030213/589d82db1a28ab69538b709f/html5/thumbnails/15.jpg)
Repidly• Easy to associate
game events with physiological events
• Rapid annotation
Bachelor’s Thesis project from Mike Schaekermann
![Page 16: #GamesUR Conference: From Body Signals to Brainy Player Insights](https://reader031.vdocuments.mx/reader031/viewer/2022030213/589d82db1a28ab69538b709f/html5/thumbnails/16.jpg)
Repidly• Level of detail
changes
• Aggregation done smoothly
Bachelor’s Thesis project from Mike Schaekermann
![Page 17: #GamesUR Conference: From Body Signals to Brainy Player Insights](https://reader031.vdocuments.mx/reader031/viewer/2022030213/589d82db1a28ab69538b709f/html5/thumbnails/17.jpg)
Player Data Visualization
• Mixed visualization of qualitative and quantitative player data overlaid on top of a level
• The overall hue of the paths conveys areas with high (B and C) and low (A) player arousal
Pejman Mirza-Babaei, Günter Wallner, Graham McAllister, and Lennart E. Nacke. 2014. Unified visualization of quantitative and qualitative playtesting data. In CHI '14 Extended Abstracts on Human Factors in Computing Systems (CHI EA '14). ACM, New York, NY, USA, 1363-1368. DOI=10.1145/2559206.2581224
![Page 18: #GamesUR Conference: From Body Signals to Brainy Player Insights](https://reader031.vdocuments.mx/reader031/viewer/2022030213/589d82db1a28ab69538b709f/html5/thumbnails/18.jpg)
Takeaways
• Physiological data is complex
• Explore visual options for rapid insights
• Online tools allow collaboration of GURs
CC by David D/Flickr
![Page 19: #GamesUR Conference: From Body Signals to Brainy Player Insights](https://reader031.vdocuments.mx/reader031/viewer/2022030213/589d82db1a28ab69538b709f/html5/thumbnails/19.jpg)
Thank youContact me
@acagamic (Twitter)
www.hcigames.com
uwaterloo.ca/games-institute/
www.immerse-network.com