Emotion Recognition using the GSR Signal on

Android DevicesShuangjiang Li

Outline

• Emotion Recognition• The GSR Signal• Preliminary Work• Proposed Work• Challenges• Discussion

Emotion Recognition

• Human-Computer Interaction• Speech recognition• Gesture/Action recognition• Facial expression recognition• Emotion recognition• …

• Affective Computing ( Picard @MIT Media Lab around late 90s)

Emotion Recognition

• Physiological Signals

Source: http://biomedikal.in/2011/05/important-physiological-signals-in-the-body/

The GSR Signal

• Galvanic Skin Response (GSR)• measuring the electrical conductance of the skin• due to the response of the skin and muscle tissue to external and internal

stimuli, the conductance can vary by several microsiemens (unit of ohm).• GSR is highly sensitive to emotions (fear, anger, startle response, etc.)

http://en.wikipedia.org/wiki/Skin_conductance

The GSR Signal

• GSR Sensor• SHIMMER (Sensing Health with Intelligence, Modularity, Mobility and Experimental

Reusability) Platform• The goal of SHIMMER is to provide an extremely compact extensible platform

for long-term wearable sensing in both connected and disconnected settings using proven system building blocks. • a highly extensible wireless sensor platform

• SHIMMER firmware is based on TinyOS• Data transmit via Bluetooth• Can sense EMG, ECG, GSR, etc.• Support Matlab, LabView, Android, C#/.Net etc.

http://shimmer.sourceforge.net/http://www.shimmer-research.com/Adrian Burns, SHIMMER: An Extensible Platform for Physiological Signal Capture, IEEE EMBS, 2010

Preliminary Work

• Emotion recognition based on the GSR signal• Four emotion categories: amusement, fear, relax, sadness• Using GSR + Accelerometer signal

• Preprocessing• Using supported accelerometer data• Denoising using median filter• Data rescaling and normalization

• Feature Extraction• 6 statistical features + 10 time domain features + 4 frequency domain features

+ feature selection (SFFS)

Preliminary Work

• Recognition rate• KNN• 10-fold cross-validation• Subject dependent / single subject

Proposed Work

• Emotion recognition on Android Devices• Android GUI for reading GSR sensor data • GSR data preprocessing• GSR data classification

• Sequential learning

Challenge

• Emotion signal tend to very noisy.

• Emotion signal generally lacks ground truth and emotion is very subjective.

• Recognition algorithms on Android devices should be light weight

• Dealing with sequential data

Discussion

Q&A