assessing engagement in an emotionally-adaptive applied game
TRANSCRIPT
Assessing Engagement in an Emotionally-Adaptive
Applied GameBoyan Bontchev and Dessislava Vassileva
Dep. of Software Technologies,
Fac. of Mathematics and Informatics, Sofia Univ., Bulgaria
TEEM-2016
Salamanca, November 2, 2016
Assessing Engagement in an Emotionally-Adaptive Applied Game
2 Player-centric adaptation
Player-centric adaptation in video games needs to answer some questions: Who – player character structural and behavioral
changes => requires player modelling How – how adaptation will be realized => requires
adaptive loop design What – which game features can be adapted =>
requires adaptive gameplay design Why – advantages of game adaptation => requires
analysis of outcomes in playability and learning results
02/11/2016
Assessing Engagement in an Emotionally-Adaptive Applied Game
3 The ADAPTIMES FP7 project ADAPTIve player-centric serious video gaMES ADAPTIMES aims at investigating how
cognitive abilities (the cognitive part), psycho-emotional status (the affective part) and playing style (the conative part – that of “desire, volition, and
striving”)
can be used in a holistic approach for efficient and effective player-centric adaptation
02/11/2016
Assessing Engagement in an Emotionally-Adaptive Applied Game
4 Adaptation Model
02/11/2016
Game dynamics
Gam
e m
echa
nics
Gam
e aesthetics
Player
Player style metrics
Perfo
rman
ce
indi
cato
rs
Emotional
indicatorsPlaying styles
Performance and efficiency
(abilities, skills and knowledge)
Emotions(flow, immersion &
intrinsic motivation)
Assessing Engagement in an Emotionally-Adaptive Applied Game
5 Measuring process
Measuring player’s performance – by tracking player’s results, incl. time, task difficulties and efficiency
Measuring playing styles – based on tracking specific gaming metrics describing player’s style of playing
Measuring player’s affect (emotions and arousal):By means of analysis of psychophysiology signals
(arousal)By means of visual expressions analysis (emotional
states)
02/11/2016
Assessing Engagement in an Emotionally-Adaptive Applied Game
6 Affect inferred by psychophysiology signals EDA (Electro-Dermal Activity) Custom EDA meter – custom hardware device and
software (H2020 RAGE asset developed by SU)
02/11/2016
Assessing Engagement in an Emotionally-Adaptive Applied Game
7 Emotions inferred by visual expressions analysis Custom emotion recognizer using Affectiva SDK Recognizes: sadness, disgust, anger, surprise, fear, joy;
engagement, attention (fixation), eye closures (saccades)
02/11/2016
Assessing Engagement in an Emotionally-Adaptive Applied Game
8 Workflow of emotion-based game dynamic adaptation control
02/11/2016
Implicit EmotionalIndicators
(via sensors & other devices)Facial expr.
GSRHeart Rhythm
ECGEEG
Pressure…
ChangeProcedures
TimeVelocity
AccelerationRouteSize
Defeat range Luminosity
Contrast…
Real-TimeExplicitPlayer
Feedbackthrough
AdaptationControl Panel
Analysis
Change
Visuali-zation
Triggering Events
on Behavior Templates
Assessing Engagement in an Emotionally-Adaptive Applied Game
9 Emotional Game Adaptation 1/2
02/11/2016
Adaptation of flying objects’ velocity, puzzle difficulty, and ambient light for lower phasic arousal
Assessing Engagement in an Emotionally-Adaptive Applied Game
10 Emotional Game Adaptation 2/2
02/11/2016
Adaptation of flying objects’ velocity, puzzle difficulty, and ambient light for higher phasic arousal
Assessing Engagement in an Emotionally-Adaptive Applied Game
11 Box plots of player’s average emotions inferred by facial expressions
02/11/2016
Assessing Engagement in an Emotionally-Adaptive Applied Game
12 Box plots of measured GEngQ terms
02/11/2016
GEngQ = Game Engagement Questionnaire (Brockmyer et al., 2009)
Assessing Engagement in an Emotionally-Adaptive Applied Game
13 Player´s joy, engagement, attention, eye closure, and SCR (skin conductance response)
02/11/2016
Successful shot Unsuccessful shot
Assessing Engagement in an Emotionally-Adaptive Applied Game
14 Correlations of psychological states (by facial expressions) and arousal with GEngQ terms
02/11/2016
Game Engagement Questionnaire (GEngQ)
State / arousal
Presen-ce
Absorp-tion Flow Immer-
sionEngage-
ment
Atten-tion 0.3878 0.0645 0.1779 0.0139 0.1978
Enga-gement 0.0087 -0.0510 0.1870 -0.0532 0.0221
Eye closure 0.3818 0.0885 0.3711 0.1737 0.1554
SCLlevel 0.1341 -0.1312 0.0459 -0.1067 -0.0318
SCRlevel 0.0119 -0.0558 -0.2177 0.1042 -0.0674
Assessing Engagement in an Emotionally-Adaptive Applied Game
15 Correlations of psychological states and arousal with emotions inferred by facial expressions
02/11/2016
Emotions
State / arousal
Sad-ness Disgust Anger Surp-
rise Fear Joy
Atten-tion -0.0226 0.3712 0.2329 0.1835 -0.1677 0.2040
Enga-gement 0.0782 0.7049 0.1653 0.4296 0.1013 0.8346
Eye closure -0.1257 0.2293 0.0918 0.4657 0.1713 0.3196
SCLlevel 0.0226 -0.1484 -0.0759 0.3873 0.3007 -0.0027
SCRlevel -0.0275 0.3616 -0.0307 0.1285 -0.0959 0.4008
Conclusions 1/2 Statistically significant correlations found:
attention and presenceeye closure and presenceeye closure and flowengagement and surpriseengagement and joySCL arousal level and surpriseSCR and joyeye closure & average efficiencySCL & relative time
Facial expressions (emotions) are complementary to EDA (arousal)
16
02/11/2016Assessing Engagement in an Emotionally-Adaptive Applied Game
Conclusions 2/2
Player-centric adaptive game play possesses essential advantages compared to the non-adaptive gameplay
The synergy of using player´s performance, emotions, and playing styles is very promising
All they are identified at run time and used for adapting game mechanics, dynamics and audio-visual content, in a dynamic and implicit way
More experiments are needed for revealing how such emotion-based adaptations affect game experience and playability, learning outcomes and consequences of flow
17
02/11/2016Assessing Engagement in an Emotionally-Adaptive Applied Game
Assessing Engagement in an Emotionally-Adaptive Applied Game
18 Future work Emotionally-adaptive games and other software app’s Dynamic adaptation control Software app’s using user affectation:
HRVTEMP….
Web-based platforms for generation of educational games targeted to non-ICT people
Semantic content management in applied games
02/11/2016
Thank you for your attention! Discussion
Videos at: https://www.youtube.com/watch?v=rSzM2RrceNw&index=1&list=PLs4ZK9XAjRxpROiCNQ1kdDR8SFpH-itmY
More info at: http://adaptimes.eu/
19
02/11/2016Assessing Engagement in an Emotionally-Adaptive Applied Game