+

Dinesh Babu Jayagopi, Assistant Professor, IIIT Bangalore Co authors: Bharath and Harsha, Fields of View, India TEEM 2014, Gamification Ecosystems, 2 Oct 2014

Analyzing gaming simulations using video based techniques

+Energy game: a sample session

2

Goal of the group: Energy policy for the country Observe 3 small groups

(representing departments – coal, atomic, renewable) Conflict/ Cooperation among groups Conflict/ Cooperation across groups

Goal serious Game designer: •  Use game elements to maximize learning •  Create awareness •  Abstract reality

+Role of game observation

3

+Quantitative data

4

+Qualitative data: game observation

5

n  Group 1 (Dept of Atomic Energy) had heavy discussions

n  Individual 2 in Group 3 was negotiating with Group 2 and was quite influential

n  Individual 3 in Group 2 was dominating and disturbing

Behavior Analysis of Group dynamics

Group 1

Group 2 Group 3

+

Behavior (Multimodal / audio-visual) analysis of group interactions

+Human Communication: Several channels of info.

7

+Behavior Coding: extracting cues

8

Source: Adam Kendon Conducting Interaction

+Technological trends: Human-centered Sensing

9

Microcone – Array microphone

Kinect – Image + Depth Camera

Sociometer – Wearable badge

Google glasses

+Technological trends: Human-centered Signal Processing

10

+Technological trends: Human-centered Signal Processing

© CHIL

11

+

Label

Observations

Hidden Variable

Observations

Technological trends: Machine Learning/Probabilistic Formulations

12

+Behavior Tracking and Perception [Human Human Interaction]

Cue Summarization

Behavior Perception

Multimodal Signal Processing

Speaking, Looking Gesturing style

Who speaks, Looks at Whom, who nods

Feedback/Analysis

Machine Learning

13

Sensing

P(Perception/Behavior)

Behavior Tracking

VIDEO

+Example: Gaze tracking

VFOA: Visual Focus of Attention

Observation: Texture (HOG) & color features

State space:

Location, scale,

eccentricity - continuous

Pose – discrete

14

+ 15 Example: Analysis HHI

speaking activity

inferred focus

person ID

head pose

tracking

side-view cameras

VIDEO

+Conclusion

n  Serious game [Face-to-face] design

n  Need for automatic observation (summarization and querying)

n  Automatic behavior analysis

n  Speech – who speaks when to whom, interruption, negotiation, word usage, dialog acts (question, answer)

n  Video – who looks at whom, who gestures what

n  Challenges: Canceling speech interference from other groups, occlusion.

16