A stress recognition system using HRV parameters and machine learning techniques

Giorgos Giannakakis, Kostas Marias, Manolis Tsiknakis

Institute of Computer Science, Foundation for Research and Technology Hellas (FORTH), Greece

• Which are the Heart Rate Variability parameters mostly involved in stress?

• Which is their best combination for discriminating stress?

Problem position

Stress

“… similar to the spectrum of color, emotions seem to lack the discrete borders that would clearly differentiate one emotion from another..”

Russell & Fehr, 1994

Arousal

Valence

• Stress can be evaluated by various techniques [1]• Psychometric scales / Questionnaires

• Clinical interviews

• Laboratory examinations (cortisol levels, …)

• Biosignals (EEG, HRV, EDA, Respiration…)

• Speech

• Facial cues

Stress detection techniques

[1] G. Giannakakis et al., “Review on psychological stress detection using biosignals.,” IEEE Transactions on Affective Computing, 2019.

Physiological/physical measures

Involuntary

Voluntary• Facial expressions

• Gaze direction

• Head direction

Semi involuntary• Eye Blinks

• Head motility (e.g. during recall)

• Micro expressions

• Eyelid response

• Pupil variation

• Biosignals

• ECG/HRV, EEG, EDA, EMG

• Various studies exist in the literature employing HRV parameters in order to study stress states.

• However, most of the studies • use only a subset of features (e.g. only HR, SDNN)

• no information on the most relevant HRV parameters on stress response

• provide just statistical evidence on their reasoning.

• The investigation of the most robust combination of HRV parameters • leads to better results

• but also provides a more complete picture of heart activity in SNS/PNS activation during stress conditions.

Introduction

• A experimental protocol with 4 phases

• Social exposure

• Emotion recall

• Cognitive stress

• Stressful videos

• Each subject performed 11 tasks

• 3 neutral

• 7 stress

• 1 relaxed

• 72 neutral and 168 stressful sessions

Experimental protocol

Intended affective state (N: neutral, S: stress, R: relaxed).

Phase Experimental taskDuration

(min)Affective

State

1. Exposure1. Reference period 1 N

2. Interview 1 S

2. Emotion recall

1. Reference period 1 N

2. Anxious event recall (AER)

1 S

3. Stressful event recall (SER)

1 S

3. Cognitive stress

1. Stressful images (IAPS)

2 S

2. Stroop Color Word test (SCWT)

2 S

4. Stressful videos

1. Reference period 1 N

2. Relaxing video 2 R

3. Adventure video (AV) 2 S

4. Psychological pressure video (PPV)

2 S

wide range of external/ internal stressors

• The dataset was collected during a research project related to contactless cardiometabolic diseases risk assessment(SEMEOTICONS)

• Semeoticons Reference Dataset for Stress Assessment • 24 subjects, 11 tasks

• 7 women, 17 men

• age 47.3±9.3 years

Study Population

Stress recognition proposed architecture

• The signal was detrended by subtracting time series polynomial fit and bandpass filtered.

• Spikes and artifacts (due to the subject’s activity/ body movements, etc) were suppressed with proper filters.

• The R components of the QRS complex were detected and the RR Intervals (RRI) were calculated.

• The ectopic heartbeats were detected and excluded• HRV signal approach [1]. (change of 40% over the averaged

previous 5 heartbeats).

• The RRI time series were interpolated to a frequency of

ECG preprocessing

in10

trpf Hz

[1] D. Nabil and F. B. Reguig, “Ectopic beats detection and correction methods: A review,” Biomedical Signal Processing andControl, vol. 18, pp. 228-244, 2015.

HRV parameters extraction

Feature Description

Time DomainRR The mean of RR intervalsSDNN The standard deviation (std) of RR intervalsHRm mean heart rate

HRstd std of instantaneous heart rate values

RMSSD Square root of the mean squared differences of successive RR intervals

NN50 Number of successive RR interval pairs that differ more than 50 ms

pNN50 NN50 /total number of RR intervals

HRV triangular index

The integral of the RR interval histogram divided by the height of the histogram

ECG envelope ECG timeseries envelope

Frequency DomainTotal power Total spectral power

LF LF Power (0.04–0.15 Hz)

HF HF Power (0.15–0.4 Hz)

LF/HF Ratio between LF and HF band powers

LFnorm Normalized LF power

HFnorm Normalized HF power

LF peak Frequency of the LF band peak

HF peak Frequency of the HF band peak

Feature DescriptionTime DomainRR The mean of RR intervalsSDNN The standard deviation (std) of RR intervalsHRm mean heart rate

HRstd std of instantaneous heart rate values

RMSSD Square root of the mean squared differences of successive RR intervals

NN50 Number of successive RR interval pairs that differ more than 50 ms

pNN50 NN50 /total number of RR intervals

HRV triangular index

The integral of the RR interval histogram divided by the height of the histogram

ECG envelope ECG timeseries envelope

Frequency Domain

Total power Total spectral power

LF LF Power (0.04–0.15 Hz)

HF HF Power (0.15–0.4 Hz)

LF/HF Ratio between LF and HF band powers

LFnorm Normalized LF power

HFnorm Normalized HF power

LF peak Frequency of the LF band peak

HF peak Frequency of the HF band peak

• Time-Frequency Representation (TFR) of the RRI

• Time-varying autoregressive (TVAR) model estimated using Kalman filter [1]

HRV parameters extraction

[1] M. P. Tarvainen et al., “Time-varying analysis of heart rate variability signals with a Kalman smoother algorithm,” Physiological Measurement, vol. 27, p. 225, 2006.

• A relaxed period was established corresponding to each subject’s baseline

• This period was removed from all other experimental tasks -> a common reference across subjects providing data normalization.

Baseline removal

Brown: neutral Red: stressGreen : relaxed

• Stress detection in 2 different tasks • can be viewed as a ranking problem

• transformation from ranking into a 2-class classification problem [no stress, stress]

• pairwise transformation was used [1]

• where i,j : indices of stress/no stress respectively

Pairwise Transformation

: ( ) ( ), correspoing ,

{Y( ) ( )}

i j

i j

T X X t X ti j

Y sign t Y t

[1] R. Herbrich, T. Graepel, and K. Obermayer, "Support vector learning for ordinal regression," 1999.

• Non smoker normal : 70bpm

• Non smoker stress : 76bpm

• Smoker normal : 90bpm

• Smoker stress : 98bpm

Pairwise transformation

+6bpm

+8bpm

Results

• One way ANOVA • effects of stress on the HRV features in each experimental task

• Pairwise comparisons among tasks • in relation to a neutral recording at the beginning of each phase

FeatureInterview

Emotional Recall 1

EmotionalRecall 2

IAPS images Stroop Colour Word Task

Adventure video Pychological pressure video

direction αdirectio

nα

direction

αdirecti

onα direction α direction α

direction

α

SDNN ↓ 0.05 ↓ 0.05 ↓ 0.05

HRm ↑ 0.01 ↑ 0.05 ↑ 0.05 ↑ 0.01

STD HR ↑ 0.01

RMSSD ↓ 0.01 ↓ 0.01 ↓ 0.05

NN50 ↓ 0.05 ↓ 0.05 ↓ 0.05

pNN50 ↓ 0.05 ↓ 0.01 ↓ 0.05

Total power

LF/HF ↑ 0.01 ↑ 0.01 ↑ 0.01 ↑ 0.01

LF norm ↑ 0.01 ↑ 0.01 ↑ 0.01 ↑ 0.01

HF norm ↓ 0.01 ↑ 0.01 ↑ 0.01

• variable importance ranking

• minimum Redundancy Maximum Relevance (mRMR) algorithm [1]

• Optimization according to the Mutual Information Quotient (MIQ) criterion

• Feature selection• Based on this ranking, the top ranked

features are inserted iteratively in the feature subset evaluating their misclassification error with SVM.

• The best features subset minimizes the misclassification error

Features ranking/selection

[1] C. Ding and H. Peng, “Minimum redundancy feature selection from microarray gene expression data,” J Bioinform Comput Biol, vol. 3, pp. 185-205, 2005.

• Classification Procedure• 10-fold cross validation, 5 repetitions of the procedure

• Performance measure:

Performance evaluation

TP TNAccuracy

TP FP FN TN

ClassifierSample basis

Accuracy (%)

After pairwise transform

Accuracy (%)

KNN 66.7 73.8

GLM 56.2 76.2

NVB 61.5 64.9

LDA 65.6 69.9

SVM 73.6 84.4

RF 75.1 70.0

• The 11 top-ranked features HRV features in terms of their stress relevance on this study are HRm, LF, NN50, LFnorm, HRstd, pNN50, LF/HF, RMSSD, HFnorm, total power, HRVtri.

• The best performance achieved (10-fold CV) utilizing HRV parameters is 84.4% classification accuracy (pairwise, SVM).

• The pairwise transformation outperforms conventional approach

• Promising results, serving the growing scientific interest on the usage of only one ECG channel that can be conveniently measured wearable devices in daily monitoring.

Conclusions

Thank you for your attention