classifying different emotional states by means of eeg-based functional connectivity patterns
TRANSCRIPT
Classifying Different Emotional States by Means of EEG-Based Functional Connectivity Patterns
You-un Lee, Shulan HsiehPLOS ONE, 2014
Emotion Classification
What is emotion classification?
Motivation
Business aspect: Assist in the decision-making process
Research opportunity: Help us to understand other behaviours such as intentions
Emotion Classification - Textual information
I am very excited today :)
I am very excited today
I am very excited today... but also feeling very tired
Multilingual Emotion Classification
Informality is a major issue
Imbalanced data
Alternatives
Brainwaves(Universal)
(Sophisticated)
Audio(tones)
Brainwaves as data
Pros:
Universal - reliable data source (language not an issue)
Sophisticated - opportunity to mine gems of knowledge (lots of data)
Cons:
Universal - difficult to interpret
Sophisticated - data preprocessing (e.g., noise filtering)
First thing is first...
How to collect the neuroimaging data?
● Functional magnetic resonance imaging (fMRI)
● Positron emission tomography (PET)
● Electroencephalography (EEG)
● Magnetoencephalography (MEG)
EEG(Electroencephalography)
EEG - Fundamental Concepts
Monitors electrical activity in the brain through electrodes placed along the scalp
EEG - Fundamental Concepts
EEG Bands - are defined by the frequency of brainwaves.
5 different types of brainwaves:
❖ Gamma
❖ Beta
❖ Alpha
❖ Theta
❖ Delta
Source: http://psychedelic-information-theory.com/eeg-bands
EEG - Fundamental Concepts
Each band can be associated with different emotional and mental states
Examples:❖ Rapid eye movement (REM) sleep (slower frequencies involved)❖ ADHD (too much theta, not enough alpha and beta)
EEG - Fundamental Concepts
Artifacts - electrical activities usually not originating from the brain.
EEG - Fundamental Concepts
Brain Maps - illustrates the electrical powerat each frequency
Green region - normal electrical activity
Red region - abnormal electrical activity
ObjectiveCapture the relationship between brain activity and emotional states.
Important considerations
Single-electrode level vs. Functional connectivity
Emotion is a complex behaviour Electrical activity is usually dispersed
How to estimate EEG functional connectivity?
Using three popular connectivity indices:
Correlation - (independent of amplitude)
Coherence - (amplitude and phase important)
Phase synchronization - (phase important)
Combination is important since each connectivity index is sensitive to different characteristics of EEG signals (phase, polarity, and amplitude).
http://predictablynoisy.com/correlation-and-coherence-whats-the-difference/
Basic intuition
A particular connectivity index might be better at recognizing a particular emotion
No such thing as a perfect measure
Materialand
Method
Participants
40 healthy studentsNo psychiatric illness
24 hour away from caffeine or tobaccoNT $1000
For 6 hours
Clips use for Emotional stimuli
Standard Chinese Emotional Film Clips Database (not for free; need to pay)
Clips consideration
Overpowering of a particular emotion was counterbalanced using Latin Square Design
Sad Joy Anticipation
Anticipation Sad Joy
Joy Anticipation Sad
EEG Measurement
Electrooculography (EOG) was measured to capture ocular artifacts. The eye component was later removed.
EOG and EEG amplified (500 Hz per channel)
NeuroScan 4.3.1
Feature selection
Functional connectivity in four bands for all pairs of 19 electrodes: Theta band (4-7 Hz)Alpha band (8-12 Hz)Beta band (13-30 Hz)Gamma band (31-50 Hz)
Transformation of raw EEG signals: Fast Fourier Transformation (FFT)
The connectivity indices for all pairs of electrodes at each frequency band were selected
as features. Features where ANOVAs results was significant (p<= 0.05) were kept.
Capture relevant interactions within the brain
Pattern Classification
Quadratic Discriminant Analysis (QDA)
❖ Reason: performs extremely faster evaluations compared to other algorithms
Two 2-fold cross validation
❖ Reason: each data point used for training and testing on each fold
Accuracy as an evaluation metric
❖ Reason: Imbalanced dataset problem.
Experiments
Experimental procedure
1. 60-s go/nogo task to keep participants in a neutral state.
2. Two 90-s baseline resting EEGs (eyes open, then closed)
3. The film was then shown to participant
4. Spacebar when emotion changes or is triggered
5. 60-s resting period
6. SAM self-assessment
16s (8192 data-points) signal
Experimental setup
Scale in terms of valence:
Negative (surprising / amusing clips)
Positive - (disgust / fear clips)
Neutral - (no emotion clips)
Data cleaning: remove data of users that did not felt the correct emotion when viewing
the clips (29 out of 40 got it right!)
1 2 3 4 5 6 7 8 n
Valence scores
Negative Neutral Positive
Better dataset or more participants?
Evaluation metric
“Balanced accuracy” across 50 trials
½(TP/P + TN/N) Where P = TP+FN, N = TN+FP
Confusion Matrix
Actual Prediction
Malignant Benign
Benign Benign
Benign Benign
Benign Benign
Malignant Benign
Malignant Benign
Benign Benign
Benign Benign
Benign Benign
Problem with imbalanced data
Connectivity Indices - Correlation
Significant only in:
Theta
Neg-N → T,O
N-P → T,P,O
Alpha
Neg-N → F7-P7
Neg-P → P,O
N-P → RT
Result (Correlation)
Connectivity Indices - Coherence
Significant results in Theta, Alpha, Beta
Any other patterns?
Result (Coherence)
Connectivity Indices - Phase synchronization (PSI)
Significant results in all bands
Any other patterns?
Result (PSI)
Result - Multiple bands
All frequency bands combined as features
Conclusion
● Did other stimuli affect the results? SAM helps to remove this concern.
● Better results with feature selection
● All bands can help towards emotion analysis and not just one in particular.
● PSI performs better in most cases
● Gender was not an underlying factor in the study
Research Opportunities
Utilize other emotion-eliciting stimuli such as music and picture viewing.
Analysis of other emotions (e.g., anticipation)
Deep Learning algorithms for automatic feature learning
References
http://neurosky.com/2015/04/reading-your-brainwaves-understanding-the-basics-of-eeg
/
https://addyssey.wordpress.com/2013/09/27/qeeg-as-a-diagnostic-tool-in-the-assessme
nt-of-addadhd/
http://mentalhealthdaily.com/2014/04/15/5-types-of-brain-waves-frequencies-gamma-b
eta-alpha-theta-delta/