gair.media.gunma-u.ac.jp€¦ · web viewevery patient, who is willing to participate in this...

2018 年度博士学位論文

Brain Informatics Based Objective Diagnosis and Evaluation for Major Depressive Disorder

脳情報学に基づくうつ病の客観的な診断や評価に関する研究

指導教員　鍾　寧教授

前橋工科大学大学院

環境・生命工学専攻　博士後期課程

1556503

Wan Zhijiang

万　志江

　　　　　　　　　　　　　審査員

　　　　　　　　　主査　今村一之　　　　教授　　　　　　　　　　副査　坂田克己　　　　教授

　　　　王　鋒　　　　　教授

　　　　　　　　　　　　　宮崎均　　　　　教授

　　　　　　　　　　　　　白尾智明　　　　教授

To my new beginning

Acknowledgements

Firstly, I would like to express my sincere gratitude to my advisor Prof. Ning

Zhong for the continuous support of my Ph.D study and related research, for his

patience, motivation, and immense knowledge. His guidance helped me in all the time

of research and writing of this thesis. I could not have imagined having a better advisor

and mentor for my Ph.D study.

Besides my advisor, I would like to thank the rest of my thesis committee: Prof.

Kazuyuki Imamura, Prof. Katumi Sakata, Prof. Feng Wang, Prof. Hitoshi Miyazaki, and

Prof. Tomoaki Shirao, for their insightful comments and encouragement, but also for the

hard question which incented me to widen my research from various perspectives.

My sincere thanks also give to Dr. Muneaki Oshima, who helped me to polish the

Japanese abstract of this dissertation. I thank my fellow lab mates for the stimulating

discussions, for the sleepless nights we were working together before deadlines, and for

all the fun we have had in the last four years. Also I thank my friends in the following

institution: International WIC Institute, Beijing University of Technology. In particular,

I am grateful to Dr. Haiyan Zhou and Dr. Jiajin Huang for helping me to collect the data

and enlightening me the first glance of research, respectively.

Last but not the least, I would like to thank my family for supporting me spiritually

throughout the Ph.D. career.

Abstract

This study mainly focuses on the four research contents to study the objective

diagnosis and evaluation method for depression: (1) a systematic method integrating the

multi-modality data collection, analysis, and integration is summarized by Brain

Informatics (BI) methodology to provide the objective diagnosis and evaluation for

depression; (2) guiding by the BI methodology, this study adopts the

electroencephalograph (EEG) as the physiological measurement, and tries to explore

effective EEG biomarkers to discriminate Major Depressive Disorder (MDD); (3)

develop a ubiquitous MDD prescreening method based on the forehead EEG collected

by portable EEG device. Furthermore, utilize the forehead EEG data and the

quantitative mood state data obtained by self-rating tool to evaluate the mental disorder

of depressive inpatients; (4) follow the Wisdom as a Service (WaaS) architecture to

develop an objective evaluation system based on the multi-modality data for providing

the adjunctive depression treatment to the clinicians.

For the challenges existed in the traditional depression diagnosis and evaluation

methods, an objective method is valuable for diagnosing and evaluating the depression

without bias. The BI methodology aims at providing an adjunctive depression diagnosis

and evaluation from the objective perspectives based on various quantitative multi-

modality data. Taking the depressive inpatients as the experiment subject, the BI

methodology designs the multi-modality data collection method and clarifies which

kind of depressive symptom is quantitatively evaluated by which kind of quantitative

data. The Data-Brain model, which is a data fusion model based on ontology

technology,

iv

defines the concepts or properties to integrate the collected data and the related data

analysis results. The WaaS architecture is introduced to integrate the collected data, the

information of the data and the related data analysis result, and the knowledge in the

depression clinical field to provide the services in platform level to aid the clinicians.

Taking the EEG as the quantitative measurement, the specific case studies about

the objective depression discrimination are illustrated. The case study aims at utilizing

the deep learning method to differentiate normal controls from depressives, and even

from medicated and unmedicated depressives based on multi-channel

electroencephalogram (EEG) data, and further to analyze the features learned by the

method. Because of the Convolutional Neural Network (CNN) model possesses a good

performance in classification task, we propose a CNN architecture merging the two kind

of convolutional filters (TcfCNN for short) to simultaneously learn the synchronous

EEG pattern and single EEG pattern from the raw EEG data and discriminate the MDD.

After that, the FFT is adopted to analyze the frequency component of the feature maps,

and explore the effective EEG biomarkers for MDD discrimination. 16 normal controls

and 23 depressive patients are recruited to collect the EEG signal using a 32-channel

EEG system produced by Brain Product (BP) Company. The ten-fold validation method

and threshold based leave-one-participant-out cross validation (LOPOCV) method are

used to evaluate the classification performance of the CNN model. Based on the ten-

fold validation method, the result shows that the model achieves the maximum

accuracy, sensitivity, and specificity are 85.6%, 87.8%, and 81.4% for two-category

classification, and achieves the maximum accuracy, sensitivity, and specificity are

Maebashi Institute of Technology, Doctor Dissertation of Engineering Zhijiang Wan: Brain Informatics Based Objective Diagnosis and Evaluation for Major Depressive Disorder

v

68.6%, 68.7%, and 81.9%, for three-category classification. Compared with other CNN

architectures, our model obtains the best overall classification performance. Based on

the LOPOCV method when the threshold equals with 0.5, the model achieves the

maximum accuracy, sensitivity, and specificity of 85.7%, 82.6%, and 91.6%,

respectively. Furthermore, the FFT result shows that the alpha rhythm is the primary

frequency component in the feature learned by the last convolution layer, which

indicates that the alpha rhythm has a good MDD discrimination ability and gives a new

evidence to support the alpha rhythm might possess a crucial position in discriminating

depression.

Many studies developed the machine learning method for discriminating MDD and

normal control based on multi-channel EEG data, less concerned with using single

channel EEG collected from forehead scalp to discriminate MDD. In this study, two

kinds of EEG datasets collected by different EEG devices are used to validate the MDD

discrimination performance of the forehead single channel. The first EEG dataset is

collected by the Fp1 and Fp2 electrode of a 32-channel EEG system. The data collected

from the two single EEG channel is used to execute the MDD discrimination task,

respectively. A feature vector, including 53 time features, 67 frequency features, 120

wavelet features and 23 nonlinear features, is extracted from the data sample to train the

classifiers. The classifiers are built by the machine learning methods including K-

Nearest Neighbor (KNN), Random Forest (RF), Linear Discriminant Analysis (LDA),

Classification and Regression Tree (CART). The Genetic Algorithm (GA) is adopted to

select the effective feature subset and improve the accuracy of classifiers. The threshold


vi

based LOPOCV method is adopted to estimate the classification performance of the

classifiers. The result shows that the KNN-GA achieves the highest accuracy of 94.29%

based on nonlinear features extracted from the EEG data of Fp1 channel. This result

indicates that the single-channel EEG analysis can provide MDD discrimination at the

level of multi-channel EEG analysis. Furthermore, in order to develop a machine

learning method for prescreening the MDD in the ubiquitous environment, the second

EEG dataset is collected from the Fp1 location using a portable EEG device produced

by NeuroSky Company. 15 MDD patients and 15 normal controls are recruited to join

the data collection. The same data processing and classification methods as the first

dataset are utilized. The result shows that the CART-GA achieves the highest accuracy

of 86.67% based on time features, which is considered to be a further evidence for

demonstrating the single-channel EEG based machine learning method is promising to

support MDD prescreening application. Additionally, we also compare the superiority of

the classification performance by the accuracy results of different methods, feature

domains and EEG channels. The accuracy result indicates that the CART-GA, time

domain, and Fp1 channel of BP device has the best classification performance for

discriminating the EEG samples of normal controls and depressives.

After demonstrating the forehead EEG collected by portable EEG device provides

MDD discrimination at the level of multi-channel EEG, we try to explore whether the

single-channel EEG also provides an effective objective evaluation for assessing the

mental disorder of the MDD or not. In order to do that, a reliable ground truth for

validating the evaluation performance of the objective method based on single channel


vii

EEG data is necessary. The Hamilton Depression Scale (HAMD) and an electronic diary

log application named quantitative log for mental state (Q-Log for short) are used to

assess the severity of the depressive mood status. 7 unipolar depressives are recruited

from the hospital to join the experiment which lasts for two weeks. The clinician rates

the depression severity of inpatients using the HAMD scale once a week. Using the Q-

Log, the inpatients are asked to self-rate their mental state like writing a diary in the

morning and evening every single day. Hitherto, the experiment total collected 19 data

samples of the HAMD and 164 data samples of the Q-Log from 7 unipolar depressives.

The participant-independent analysis based on Principle Component Analysis (PCA) is

utilized to analyze the Q-Log data of each inpatient. A curve reflects the dynamic

change of the major depressive mood state is extracted by mapping the Q-Log data

matrix into the first principal component space, which contains the maximum energy or

information of the Q-Log data matrix. We named the curve as the First Principle

Component (FPC) and employed it as a quantitative tool to quantify the depressive

mood status for each inpatient. The result shows that the overall tendency of the FPC

curve is congruent with the HAMD scale, and the local variation reflects that the

feelings of the depressive in the morning are often worse than the evening. This

conclusion is congruent with the clinical treatment experience of the clinician, and

provides an evidence to support the effectiveness of the Q-Log for assessing the mental

disorder of the MDD. Furthermore, a quantitative model for objectively assessing the

mood status of the MDD is constructed by a regression method based on the RF

algorithm. The independent variables of the model are forehead EEG features and the


viii

dependent variables are the FPC values of the Q-Log data. The LOPOCV is adopted to

estimate the model performance, the result shows that the model outcomes have a

moderately strong uphill relationship (the average coefficient equals 0.6556 and the P-

value less than 0.01) with the FPC values, which illustrates that the forehead single-

channel EEG data provides an effective evaluation for objectively assessing the mental

disorder of the MDD.

In order to support the adjunctive depression treatment, the objective

discrimination and assessment methods should be integrated into a system for providing

the platform level services. Taking the quantitative evaluation for the mental disorder

based on forehead EEG and the Q-Log data as example, we follow the WaaS

architecture to develop a Data-Information-Knowledge-Wisdom (DIKW) system for

supporting the objective evaluation of depression. In the Data as a service (DaaS) layer,

the data and the quantitative analysis results are managed and saved in the SQL

database. In the Information as a Service (InaaS) layer, the Resource Description

Framework (RDF) model is developed to instantiate the concepts and the properties

designed in the Knowledge as a Service (KaaS) layer. That is, the RDF model builds a

bridge to map the instantiation data in the SQL database with the concepts in the

Ontology. In the KaaS layer, a portable EEG ontology, scale ontology, and related

objects and properties are constructed using Protege 4.3. The constructed quantitative

model is manually transformed into Jena format and derive feature notation from built

ontologies. The advantage of doing this is to transform the non-intuitive data into the

intuitive knowledge and facilitate the clinical users to understand the meanings of the


ix

data and the quantitative analysis results. In the WaaS layer, an exemplary application

named quantitative analysis for depression mood status and knowledge sharing is

actualized. The example demonstrates that the WaaS architecture driven technique route

for the objective evaluation of depressive symptom is feasible to support the mental

health services in the platform level.

Keywords: Brain Informatics, WaaS, Machine Learning, EEG, Ontology, Objective

Diagnosis, Quantitative Evaluation, Major Depression Disorder


Abstract Japanese

本研究は、うつ病の症状を客観的に量的診断・評価するために、次の４つの

点から行う。

（１）うつ病の客観的な評価と診断のために、脳情報学に基づき、生理・心

理・行動など多面的なデータの収集、管理、モデリング、分析といった基本的

なデータ処理のプロセスを体系的かつ一貫的に支援する方法を検討する。

（２）脳情報学に従い、脳波を利用しうつ病を量的に診断・評価する方法に

ついて提案する。単極性うつ病患者と健常成人を弁別するのに効果的な脳波の

指標を探る。

（３）ユビキタス環境において脳波計測のモバイルデバイスを使い、うつ

病患者のプレスクリーニングの方法を開発する。更に、前額部の脳波や気分障

害の自己評価データを元に、うつ病の気分症状を量化して評価する方法を提供

する。

（４）Wisdom as a Service (WaaS) アーキテクチャに従い、多様式の生理・心

理・行動データを元に、うつ病の治療補助のために、うつ病の気分・身体症状

を量化して評価するシステムを構築する。

これまでの典型的なうつ病の評価と診断は、バイアスの無い客観的が重要

とされている。脳情報学では、定量的なマルチ形式のデータに基づき、客観的

な視点からうつ病の診断と評価を提供することを目的としている。本研究で

は、うつ病の入院患者を対象とし、脳情報学に基づきデータを収集し、どのよ

うな

xi

うつ病症状をどのデータによって定量的に評価されるかを明らかにする。

データブレイン（Data-Brain）は、オントロジに基づくデータ融合モデルで

あり、収集されたデータと関連するデータ分析結果を統合する。 WaaS アーキ

テクチャは、収集されたデータ、データや分析結果の情報、うつ病の臨床分野

における知識を統合し、臨床医を支援するためにプラットフォームレベルで

サービスを提供する。

定量的な基準として脳波を採用することで、客観的なうつ病の評価と診断を

明確にできる。脳波の同期的パターンは、脳内ニューロンが同期放電から生じ

る脳波の特徴である。うつ病は脳内ニューロンの活動パターンの変動と関連

する精神疾患のため、脳波の同期的パターンから異常を推定できる。

本研究では、ドイツの Brain Product (BP) 社の脳波計測システムを利用し、

被験者の 6 チャネルの脳波データを記録した。この脳波データを分類性能の高

い畳み込みニューラルネットワークを利用し、脳波の同期性パターンを元に

うつ病患者と健常成人に分類した。被験者は、DSM-IV により単極性障害と診

断された患者群 23名と健常対照群 16名である。

これらのデータを元に、高速フーリエ変換を利用し、畳み込みニューラル

ネットワークの特徴マップの周波数成分を分析し、うつ病患者や健常成人を弁

別するために効果的な脳波指標を推定した。

この指標を、10-分割交差検証や閾値に基づいた被験者交差検証により、妥

当性を検証した。10-分割交差検証の結果においては、2 分類においては、最高

精度、感度、特異度がそれぞれ 85.6%、87.8%、81.4%であった。また、3 分類

においては、それぞれ 65.4%、65.4%、79.1%であった。被験者交差検証の結果


xii

においては、 0.5 の閾値では、 2 分類において、それぞれ

85.7% 、 82.6% 、 91.6% であった。また、 3 分類においては、それぞれ

65.7%、66.6%、83.3%であった。他の畳み込みニューラルネットワークの手法

と比べ、本研究ではより良い分類性能を得られた。

このことから、脳波の同期的パターンはうつ病患者群と健常対照者群の弁別

において役に立つことが示された。更に、高速フーリエ変換を利用した結果

では、アルファ波が畳み込み層の特徴マップの主成分であったことから、ア

ルファ波はうつ病患者群と健常対照者群の弁別において効果的な指標であるこ

とが示された。

前額部の脳波データがうつ病を弁別できることを証明するために、BP社の

脳波計測システムを利用して Fp1 と Fp2 チャネルの脳波データを収集した。単

一チャネルの脳波データを基に機械学習により病患者群と健常対照者群を弁別

した。

まず、各サンプルデータから 263 次元の特徴ベクトルを抽出した。特徴ベク

トルは 53 次元の時間領域の特徴、67 次元の周波数領域の特徴、120 次元の

ウェーブレット特徴、23 次元の非線形特徴である。特徴ベクトルを元に、K

近傍法(KNN)、ランダムフォレスト(RF)、線形判別分析(LDA)、分類や回帰木

(CART)など 4 つの機械学習による分類器を訓練した。また、遺伝的アルゴリ

ズムにより、特徴ベクトルから特徴ベクトルのサブセットを選択し、うつ病

患者群と健常対照者群に分類した。

被験者交差検証を利用し、閾値に基づいた分類方法の性能を評価する。うつ

病患者群と健常対照者群への分類結果では、Fp1 チャネルの脳波データの非線


xiii

形特徴による KNN-GA 方法で正解率 94.29％を得ることができた。このことか

ら、単一チャネルの前額部脳波データで、多チャネルの脳波データと同様に

うつ病患者を弁別できることが示された。

ユビキタス環境におけるうつ病の弁別のために、脳波計測のモバイルデバ

イスを利用し、前額部の脳波データを収集し、プレスクリーニングの方法を

開発した。ここでは、脳波計測に、ウェアラブル脳波デバイスである

NeuroSky社の「ThinkGear」を採用した。このシステムを利用し、Fp1 チャネ

ルで前額部脳波データを収集した。

被験者は、DSM-IV により単極性障害と診断された患者群 15名と健常対照群

15名である。

うつ病患者群と健常対照者群への分類結果より、時間領域の特徴に基づく

CART-GA 方法で正解率 88.57％を得ることができた。これにより、単一チャネ

ルの脳波データを基づく機械学習でうつ病患者のプレスクリーニングが可能

であることが示された。

また、上記 4 つの機械学習や 5 つの領域特徴、2 つの脳波デバイスチャネル

を利用した場合の、うつ病患者の弁別性能を比較した。本実験結果では、

CART-GA 方法・時間領域・BP 脳波デバイスの Fp1 チャンネルでは高正解率と

なった。

前額部の脳波データを利用した、単極性障害のうつ病患者の気分に客観的な

量的評価を提供する。うつ病患者の気分についての効果的な評価方法を参考に


xiv

し、客観的に量化評価方法の性能を検証する。そのために、ウェアラブル脳波

デバイスで前額部の脳波データを収集するだけではなく、うつ病用のハミル

トン評価尺度や「気分評価用量化日誌(Q-Log)」（日誌アプリ）を使い、うつ

病患者の気分状態を評価するためのデータを収集した。各被験者において二週

間程度の気分状態の自己評価の実験に参加してもらった。Q-Log アプリでは、

被験者は日誌を書くような方式で、毎日の朝と夕方に気分状態を自己評価する。

このログデータを基に、ハミルトン評価尺度による評価を週毎に行い、3回の

評価結果を得た。

被験者は、DSM-IV により単極性障害と診断された患者群 7名である。主成

分分析手法により、被験者毎の Q-Log データから第一主成分ベクトルを解析し、

被験者の主要的な気分状態を客観的に量的に評価した。この結果より、第一主

成分の全体的な傾向がハミルトン評価尺度の変化傾向と同じであることが示さ

れ、局所的変化が朝の気分状態より夕方の気分状態のほうがより良いことが示

された。この分析結果は、医師の臨床評価と同じであっただけではなく、Q-

Log データによるうつ病患者の気分状態に量化評価の方が優れた性能であった。

更に、前額部の脳波データを利用し、うつ病の気分状態を客観的に量的評価

するように、RF アルゴリズムを使って回帰モデルを構築した。被験者毎の

データでは、回帰モデルの説明変数は脳波データの特徴であり、予測変数は

Q-Log データの第一主成分である。被験者交差検証方法を利用して、気分状態

の量化評価におけるモデルの評価効果を検証した。検証結果から、モデルの出

力結果は検証用の Q-Log データの第一主成分と正の相関関係があり、回帰モデ

ルは気分状態の量的評価における効果的な結果を得られた。


xv

うつ病の治療補助のために、客観的な量的評価のデータ分析方法や結果を医

療システムに統合する必要がある。

本研究では前額部の脳波データと Q-Log データに基づいたうつ病状態の量的

評価を例にし、脳情報学の Data-Information-Knowledge-Wisdom (DIKW) system

による、うつ病補助治療用のシステムを開発した。また、医師が脳波データ

と Q-Log データの分析方法や結果を理解できるようにオントロジ技術を利用

し、データの意味を追加した。さらに、Jena を利用し、データやデータ分析結

果から Resource Description Framework (RDF) モデルを構築した。

うつ病の治療補助のシステムは「WaaS」のサービスポータルに組み込み、

気分状態を量的評価するウェブサービスとして提供する。これにより、脳情

報学の方法論に従ってうつ病の症状を客観的に量的評価することが可能である

ことを示した。


Contents

Acknowledgements.....................................................................................iAbstract...................................................................................................iiAbstract Japanese......................................................................................ixContents................................................................................................xvList of Figures.........................................................................................xxList of Tables........................................................................................xxiii

1. Prologue............................................................................................11.1 Introduction......................................................................................11.2 Organization of the Thesis....................................................................5

2. Related Work......................................................................................92.1 Traditional Depression Diagnosis and Evaluation.......................................92.2 Brain Informatics Methodology for Depression Treatment...........................102.3 Wisdom as a Service for Depression Treatment........................................122.4 Multi-modality Data Based Depression Diagnosis and Evaluation.................13

3. BI Based Depression Diagnosis and Evaluation..........................................153.1 Introduction....................................................................................153.2 Systematic Data Acquirement..............................................................18

xvii

3.3 Systematic Experiment Design.............................................................233.4 Systematic Data Analysis and Fusion.....................................................25

4. Multi-Channel EEG Based Objective Depression Discrimination....................304.1 Introduction....................................................................................304.2 Materials and Methods.......................................................................36

4.2.1 Data Acquisition.........................................................................364.2.2 Methods....................................................................................37

4.3 Results..........................................................................................514.3.1 Experimental Setup......................................................................514.3.2 Classification Results Comparison...................................................534.3.3 Analyze the Feature of the Convolutional Layer..................................564.3.4 Analyze the Feature of the Softmax Layer..........................................59

4.4 Discussion......................................................................................61

5. Single Channel EEG Based Objective Depression Prescreening......................675.1 Introduction....................................................................................675.2 Materials and Methods.......................................................................71

5.2.1 Technical Route..........................................................................715.2.2 Data Acquisition and Preprocessing.................................................725.2.3 Feature Extraction.......................................................................755.2.4 Feature Selection and Classification Methods.....................................805.2.5 Classifier Training and Sample Classification.....................................84


xviii

5.3 Results..........................................................................................855.3.1 Classification Result Based on Multi-channel EEG Device.....................855.3.2 Classification Result Based on Portable EEG Device............................895.3.3 Friedman and Multiple Comparison Test...........................................915.3.4 Comparison with Other Methods.....................................................93

5.4 Discussion......................................................................................97

6. Forehead EEG Based Objective Evaluation for Depressive Mood State..........1006.1 Introduction..................................................................................1006.2 Materials and Methods.....................................................................104

6.2.1 Data Acquisition.......................................................................1046.2.2 Data Processing........................................................................1076.2.3 Quantitative Model Construction and Performance Estimation..............111

6.3 Results........................................................................................1126.3.1 Depression Level of Inpatients Rated by HAMD Scale........................1126.3.2 Objective Evaluation for Mental Disorder........................................1136.3.3 Quantitative Model Evaluation......................................................114

6.4 Discussion....................................................................................116

7. Multi-modality Data and WaaS Architecture Based Objective Evaluation for

Depression...........................................................................................1197.1 Introduction..................................................................................1197.2 Technical Route.............................................................................121


xix

7.3 Multi-modality data collection...........................................................1237.4 Multi-modality data analysis.............................................................126

7.4.1 Depression Type Recognition.......................................................1267.4.2 Voice Analysis..........................................................................1287.4.3 Diary Activity Analysis...............................................................1297.4.4 Concentration Ability Analysis.....................................................131

7.5 Multi-modality Data Fusion by Data-Brain Model...................................1327.6 Case Study for the Service Construction Following WaaS Architecture.........133

7.6.1 DaaS Layer: SQL Database Construction.........................................1357.6.2 InaaS Layer: RDF Construction....................................................1367.6.3 KaaS Layer: Ontology Construction...............................................1377.6.4 WaaS Layer: Service Construction.................................................142

7.7 Discussion....................................................................................148

8. Epilogue........................................................................................1508.1 Contributions and Conclusions...........................................................1508.2 Future Work..................................................................................152

8.2.1 BI Based Depression Diagnosis and Evaluation.................................1528.2.2 Depression Prescreening Based on the EEG Data...............................1538.2.3 Quantitative Analysis for Depressive Mood Status.............................1558.2.4 Conjoint Analysis Based on the Multi-modality Data..........................155

Bibliography.........................................................................................157


xx

研究業績リスト- Publications.................................................................170


List of Figures

Figure 3.1 The schematic diagram for illustrating the BI based systematic investigation

on pathology of depression.........................................................................16Figure 3.2 Three aspects for illustrating data collection paradigm.........................25Figure 3.3 Diagram for illustrating the depression-related concepts depicted in the four

dimensions of Data-Brain model..................................................................29

Figure 4.1 The architecture of the TcfCNN model for three-category classification....44Figure 4.2 The technical route for obtaining the features of the convolutional layers or

the softmax layer of the TcfCNN model.........................................................51Figure 4.3 The comparison result of two-category and three-category classification

using 10-fold cross validation......................................................................54Figure 4.4 The FFT result of the features output by the 2nd, 4th, 8th convolutional layers

of SynCNN part in two-category classification................................................57Figure 4.5 The FFT result of the features output by the 2nd, 4th, 8th convolutional layers

of SinCNN part in two-category classification.................................................57Figure 4.6 The FFT result of the features output by the 2nd, 4th, 8th convolutional

layers of SynCNN part in three-category classification.......................................58Figure 4.7 The FFT result of the features output by the 2nd, 4th, 8th convolutional

layers of SinCNN part in three-category classification.......................................58Figure 4.8 The FFT result of the feature output by the softmax layer of TcfCNN model

xxii

intwo-category classification.......................................................................60Figure 4.9 The FFT result of the feature output by the softmax layer of TcfCNN model

in three-category classification....................................................................61

Figure 5.1 Flowchart illustrating the data processing procedure for classifying MDD

patients and NCs......................................................................................72Figure 5.2 The signal wave and corresponding FFT transform of original EEG signal,

denoised EEG signal and five subbands.........................................................75Figure 5.3 Classification performance comparison of different classifiers, feature

domains and EEG channels........................................................................93

Figure 6.1 The interface of the Q-Log application in an Android phone................106Figure 6.2 Information about overall tendency and local variation of the FPC........114Figure 6.3 Meshgrid plot of average correlation coefficient and average P-value.....115

Figure 7.1 Technical route for constructing the service system based on multi-modality

data following the WaaS architecture...........................................................123Figure 7.2 Multi-modality data collection tools deployed in the ward...................125Figure 7.3 Multi-modality data collecting strategy in different time points............126Figure 7.4 The mood category of the major depressive mood state for each patient..128Figure 7.5 A Matlab GUI tool for manually extracting the conversation voice........129Figure 7.6 A Matlab tool for showing the accelerator sensor data and automatically

recognizing the activity fragment...............................................................131Figure 7.7 An Android app for showing the forehead EEG curve and the Attention meter


xxiii

.........................................................................................................132Figure 7.8 Data Brain model construction by Protege tool.................................133Figure 7.9 Several datasheets of Q-Log data and the bonds that linked them..........136Figure 7.10 RDF initiations of ontology concepts...........................................137Figure 7.11 The structure of portable EEG ontology........................................140Figure 7.12 The structure of clinical evaluation tool ontology for depression.........141Figure 7.13 Partial resource descriptions of the Data-Brain model.......................145


List of Tables

Table 4.1 The parameter illustration of the TcfCNN model............................................45

Table 4.2 Two-category classification using the voting based LOPOCV with different

thresholds.........................................................................................................................55

Table 4.3 Three-category classification using the voting based LOPOCV with different

thresholds.........................................................................................................................55

Table 5.1 Classification results based on the feature set of time domain........................87

Table 5.2 Classification results based on the feature set of frequency domain...............87

Table 5.3 Classification results based on the feature set of wavelet domain..................88

Table 5.4 Classification results based on the feature set of nonlinear domain................88

Table 5.5 Classification results based on the fused feature set........................................89

Table 5.6 Classification results based on the feature set of 4 domains...........................90

Table 5.7 Classification results based on the fused feature set........................................91

Table 5.8 Comparison studies in literature with our approach........................................96

Table 6.1 Depression state changes and HAMD scores of 7 inpatients........................112

Table 6.2 Result about the effectiveness evaluation of quantitative model...................116

Table 7.1 Part of the object properties of portable EEG ontology................................142

Table 7.2 Part of the datatype properties of portable EEG ontology.............................142

Chapter 1

1. Prologue

1.1 Introduction

Major depressive disorder (MDD: unipolar depression), also known simply as

depression, is widely distributed in the world-wide populations and it is one of the

leading causes of disability in both adolescents and adults [1]. It is a mental illness that

involves several symptoms such as low mood, feeling worthless or guilty, loss of

interest in activities, weight loss or gain, insomnia, anxiety, being tired, trouble

concentrating, thoughts of suicide, as well as others. The Depression and Other

Common Mental Disorders published by World Health Organization (WHO) in 2017

shows that at a global level, over 300 million people are affected by depression,

equivalent to 4.4% of the world's population [2, 3]. In China, over 54.82 million people

are conservatively estimated to suffer from depression, accounts for 4.4% of the

Chinese population. At its most severe, depression can lead to suicide. The WHO

Global Health Estimates provide a comprehensive assessment of mortality due to

depression for all regions of the world [4, 5]. In the year 2015, it is estimated that

788,000 people died due to suicide, many more than this number attempted (but did not

die by) suicide. Meanwhile, as opposed to many other illnesses, the MDD or depression

often causes significant losses and burdens to the economic system, as well as the

social, educational, and justice systems. In this situation, accurate diagnosis and

assessment of depression are essential criteria for optimizing treatment selection and

improving outcomes, thus reducing the economic and psychological burdens.

2

However, depression is often mistaken for other mental health disorders [6, 7].

This misdiagnosis creates a cascade of negative outcomes. Patients will probably

receive inadequate or inappropriate treatment that will not alleviate the symptoms or

impairment of the disorder and may even further destabilize their mood. The related

statistics reported by Tanvir et al. shows that 69% of people with bipolar disorder are

initially misdiagnosed with another mental health disorder, most commonly unipolar

depression [6]. More than 30% of those remain misdiagnosed for 10 years or more and

the average patient remains misdiagnosed for 5.7-7.5 years. These individuals are then

at risk for experiencing numerous social and occupational impairments, alcohol or

substance abuse, and suicidal behavior. One of the reasons causes that is the

misevaluation for the depression symptoms depending on the clinical interview [8, 9].

The patient interview provides a subjective assessment for the depression symptoms is

affected by the clinical level of clinicians, and the evaluation result of the depression

symptoms made by different clinicians might differ greatly. Reliance upon clinical

assessments and patient interviews for diagnosing depression is frequently associated

with misevaluation or even misdiagnosis and result in suboptimal treatment outcomes.

Corresponding to the subjective diagnosis and assessment made by the patient

interview, an objective diagnosis and assessment for the depression might be a feasible

way to reduce the misdiagnosis rate and improve the therapy effectiveness of depression

[10, 11]. The advantages at least include four aspects: (1) The objective diagnosis and

evaluation method for depression is driven by the quantitative data. This facilitates

researchers to find out the biomarkers for quantitatively discriminating the difference of

the depression and other diseases or healthy condition. (2) Consistent results for


3

assessing depression symptoms could be given by the objective evaluation method. The

quantitative data monitored by the physiology and behavior measurements provide a

quantitative ground for objectively assessing the depression symptoms and avoid the

symptoms being subjectively concealed by depressives. (3) The objective methods are

able to evaluate depression in a fine granularity. In the clinical practices for depression

treatment, the usage of the related clinical assessment tools (scales or questionnaires)

should consider the test-retest reliability, which indicates the clinical evaluation is

carried out in a coarse-granularity. That is, the clinical evaluation tools are always used

to provide a cross-sectional assessment of psychological status and unable to assess the

daily change of psychological status. However, some studies found that a continuous

assessment on depression symptoms with fine granularity might give circumstantial

evidence to reflect the depression idiosyncrasy of patients. Because of the objective

methods need not consider the test re-test reliability, it is suitable to give circumstantial

evidence for evaluating depression symptoms in a fine granularity. (4) The objective

methods for depression evaluation are easily being implemented in the primary care

setting and clearly aid in affirming a diagnosis. If the depression symptoms could be

monitored in the primary care setting, there is no doubt that the monitoring result could

be used in aiding the clinical treatment.

In order to actualize the objective diagnosis and evaluation method, Prof. Ning

Zhong proposes a systematic method following the Brain Informatics (BI) methodology

to cope with the depression treatment difficulties, including the pathology of depression,

endogenous biomarkers, and the evolution mechanism for depression progression [12-

14]. The BI methodology for depression tries to collect the multi-modality data,


4

including physiology, psychology, and behavior data, from healthy, depression with

high risk, depressive, and depression rehabs as complete as possible, and further to

explore the evolution mechanism of depression and mining out several endogenous

biomarkers. Furthermore, taking the real application as the research target, the BI

methodology for depression focuses on the objective diagnosis and evaluation to

provide the adjunctive depression treatment for the clinical practices.

There has been a surge of research activity in recent years that has shed light on

both the neurobiological, physiological, and behavioral effects of depression, which

indicates the feasibility of using the multimodality data to objectively evaluate the

depressive symptoms. For instance, the psychological data obtained by the self-rating

tool provides an effective reference to observe the mental state change. Behavioral

characteristics such as reduced emotional responsivity, reduced physical

activity/psychomotor retardation, decreased socialization/social dysfunction are all well

documented as symptoms of depression. These behavioral symptoms can be viewed as

the behavioral manifestations of the autonomic blunting caused by major depression.

Psychomotor retardation (defined as the physical slowing down of thought, speech, and

movement) is another primary symptom of depression. Psychomotor retardation may be

the only group of symptoms of depression that can distinguish depression subtypes and

can have a high discriminative validity in determining depression as well as predicting

the response of a patient to certain types of treatment such as antidepressants. Changes

in diurnal sleep patterns, specifically disturbances of sleep, are typical for most

depressed patients and also belong to one of the core symptom groups of the disorder.

There is a strong bi-directional relationship between sleep, sleep alterations, and


5

depression.

In the beginning, we tried to use single modality data to deal with the objective

depression discrimination and depressive mood state assessment. Since the pervasive

and persistent nature of depressive symptoms has made scalp-recorded

electroencephalograph (EEG) as an appropriate approach for understanding the

underlying mechanisms of major depressive disorder, the EEG is chose as the

measurement tool to explore the EEG biomarker for supporting the objective depression

discrimination. Considering the traditional EEG measurement relates to intensive works

in the lab environment, we prefer to develop the method for objectively discriminate the

depression in the ubiquitous environment. Combining with the machine learning

methods, the portable EEG device equipped with forehead single EEG channel is used

to support the MDD prescreening application. After demonstrating the forehead EEG

contains effective information for discriminate depression, the effectiveness of the

forehead EEG for objectively evaluating the depressive mood status is also explored.

Based on the previous findings, a Data-Information-Knowledge-Wisdom (DIKW)

system for supporting the objective evaluation of depression driven by Wisdom as a

Service (WaaS) architecture is developed.

1.2 Organization of the Thesis

This thesis consists of 8 chapters.

Chapter 2 gives a brief introduction about the related works of this thesis. This

chapter firstly presents the traditional depression diagnosis and evaluation methods.


6

Opposite to the subjective characterize of the traditional methods, the BI methodology

proposed a systematic method to study the depression diagnosis and evaluation from the

objective perspective. The WaaS architecture is also introduced for implementing the

objective methods in the system level. As the main physiological measurement in this

study, the EEG data is used to measure the brain activity and support the objective

depression diagnosis and evaluation. Lastly, the relation between the EEG and

depression is also illustrated.

Chapter 3 introduces the BI methodology based depression diagnosis and

evaluation. This chapter aims at giving a complete introduction about the BI

methodology based depression and evaluation methods, including the systematic data

acquirement, systematic experiment design and implementation, and the systematic data

analysis and fusion. The work of this chapter is sorted into a journal paper which is

prepared to submit and accept review. The name of the journal paper is “A Brain

Informatics Methodology Based Objective Evaluation Method for Depression”.

Chapter 4 takes the EEG as the measurement tool to explore the EEG biomarker

for discriminating the MDD and normal controls. This Chapter aims at utilizing the

deep learning method to differentiate normal controls from depressives, and even from

medicated and unmedicated depressives based on multi-channel electroencephalogram

(EEG) data, and further to analyze the features learned by the method. We propose a

CNN architecture merging the two kind of convolutional filters (TcfCNN for short) to

simultaneously learn the synchronous EEG pattern and single EEG pattern. The ten-fold

validation method and threshold based leave-one-participant-out cross validation

(LOPOCV) method are used to evaluate the classification performance of the CNN


7

model. The gradient ascent method and the Fast Fourier Transformation (FFT) are

adopted to analyze the frequency component of the feature maps, and explore the

effective EEG biomarkers for MDD discrimination. We are sorting out the work of this

chapter into a journal paper, and the name of the journal paper is temporarily defined as

“EEG Based Feature Learning and Analysis of Convolutional Neural Network for

Depression Discrimination”.

Chapter 5 validates the MDD discrimination ability of the forehead single channel

EEG. After that, combining with the machine learning methods, a portable EEG device

is used to prescreen the depression for supporting the adjunctive depression

discrimination in the ubiquitous environment. The machine learning methods including

K-Nearest Neighbor (KNN), Random Forest (RF), Linear Discriminant Analysis

(LDA), Classification and Regression Tree (CART) are combined with Genetic

Algorithm (GA) to train the classifiers based on the features in different domains. The

threshold based LOPOCV method is adopted to estimate the performance of the

classification methods. The work of this chapter is sorted into a journal paper and

submitted to the International Journal of Information Technology and Decision Making.

The name of the journal paper is “Single Channel EEG Based Machine Learning

Method for Prescreening Major Depressive Disorder”.

Chapter 6 proposes a quantitative method for objectively evaluating the depressive

mood state based on the forehead EEG and the self-rating data of the mood state, which

is acquired by an electronic diary log application named quantitative log for mental state

(Q-Log for short). Specifically, as one of the depression symptoms, the depressive mood

status is quantitatively evaluated based on the forehead EEG data. The Hamilton


8

depression scale and the Q-Log data are used to assess the severity of the depressive

mood status. The RF regression method is adopted to construct the quantitative model

based on the forehead EEG data and the Q-Log data. The effectiveness of the

quantitative model for assessing the depressive mood status is estimated by comparing

the model outcomes with the real analysis results of the Q-Log data. The work of this

chapter is published at a conference proceeding in document format. The name of the

conference paper is “A Quantitative Analysis Method for Objectively Assessing the

Depression Mood Status Based on Portable EEG and Self-rating Scale”.

Chapter 7 describes the quantitative evaluation method for objectively assessing

the depression based on multi-modality data. Driven by the WaaS architecture, the detail

implementation methods about using what kind of tools to collect the multi-modality

data, utilizing what kind of data analysis method to analyze the multi-modality data,

how to integrate the multi-modality data and the related data analysis results, and etc.

Taking the quantitative evaluation for the mental disorder based on forehead EEG and

the Q-Log data as case study, a DIKW system for supporting the objective evaluation of

depression driven by the WaaS architecture is developed. The work of this chapter could

be divided into two parts, which are published at journal and conference proceeding in

document format, respectively. The name of the journal paper is “WaaS Architecture

Driven Depressive Mood Status Quantitative Analysis Based on Forehead EEG and

Self-Rating Scale”, and the name of the conference paper is “A Depressive Mood Status

Quantitative Reasoning Method Based on Portable EEG and Self-rating Scale”.

Chapter 8 summarizes this thesis. It includes contributions of this thesis and some

topics for future researches.


Chapter 2

2. Related Work

2.1 Traditional Depression Diagnosis and Evaluation

The traditional depression diagnosis and evaluation is executed by the clinicians.

Physical examination by a physician is the first step to decide an appropriate treatment

method for the depressives [15]. Certain medications as well as some medical

conditions such as a viral infection can cause the same symptoms as depression, and the

physician should rule out these possibilities through examination, interview, and lab

tests. Guided by the established classification criteria, such as the 10th International

Classification and Diseases (ICD-10) and 5th Diagnostic and Statistical Manual of

Mental Disorders (DSM-Ⅳ), the depression diagnosis and evaluation made by the

clinicians relies on the patient-reported information and the observed clinical symptoms.

A good diagnostic evaluation will include a complete history of symptoms, i.e., when

they started, how long they have lasted, how severe they are, whether the patient had

them before and, if so, whether the symptoms were treated and what treatment was

given. The doctor should ask about alcohol and drug use, and if the patient has thoughts

about death or suicide. Further, a history should include questions about whether other

family members have had a depressive illness and, if treated, what treatments they may

have received and which were effective. In addition, a diagnostic evaluation should

include a mental status examination to determine if speech or thought patterns or

memory have been affected, as sometimes happens in the case of a depressive or manic-

depressive illness.

Chapter 2

However, identifying people with established depression does not usually present

11

as a clinical challenge with standard clinical instruments. The potential for ambiguity

bias and low reliability of depression diagnosis based on clinical descriptions can be

compounded by the heterogeneous nature of the disorder. There are a number of DSM-5

defined depressive disorders (e.g. major depressive disorder [MDD], dysthymia,

depressive disorder not otherwise specified [NOS]) and, for unipolar MDD, there are

symptom based subtypes (e.g. melancholic, psychotic and atypical depression);

symptoms can also vary by gender, age and even race. Precise treatment that provides

objective diagnosis and evaluation criteria is crucial to reduce the misdiagnosis rate and

improve the therapy effectiveness of the traditional depression treatment methods.

2.2 Brain Informatics Methodology for Depression Treatment

We can understand the Brain Informatics methodology from the concept definition

and application, respectively. For the definition, the BI is an emerging interdisciplinary

and multidisciplinary research field that focuses on studying the mechanisms underlying

the human information processing system (HIPS) [12, 13]. For the application, the BI

methodology for depression treatment aimed at providing an adjunctive depression

diagnosis and evaluation from the objective perspectives based on various quantitative

data. It tries to provide objective depression diagnosis and evaluation from three

aspects: long term, multi-modality, and multi-scale.

(1) The long term aspect means that the BI advocates to consecutively collecting

the quantitative data. The consecutive collection not only indicates the quantitative data

is continuously collected in a certain time range or collected in discontinuous time


12

interval, but also implies the quantitative data acquired in different disease states (i.e.

health, health but high risk, depression, and rehabilitation) is significant.

(2) The multi-modality aspect means that the various quantitative data is collected

for objectively quantifying the severity of various depressive symptoms. The idea about

using different quantitative data to objectively quantify different depressive symptoms

is origin from the traditional depression evaluation method by clinical scales, which are

simultaneously assessing the multiple depressive symptoms and giving a comprehensive

assessment. Specifically, a systematic evaluation is needed for objectively evaluating

the depression symptoms. In the traditional depression treatment, the evaluation and

diagnosis tools, such as Structured Clinical Interview for DSM-5 (SCID-5), Hamilton

Depression Scale (HAMD), Beck Depression Inventory (BDI) et al, always consider

multiple depressive symptoms and provide a comprehensive evaluation. Referring to

this, the objective evaluation for depression should also consider the comprehensive

evaluation for multiple symptoms. The BI methodology adopts a specific single

modality data to quantitatively reflect the severity of one depressive symptom. A

comprehensive quantitative evaluation for depression could be obtained by gathering

the analysis result of every single modality data.

(3) The multi-scale aspect indicates that the BI tries to study the depression from

macro scale, meso scale, and micro scale. The three scales are corresponding with the

various quantitative data, which means the different data reflects different depressive

symptoms in different scales. For example, the quantitative data reflecting the

behavioral activity belongs to the data collected from macro scale; the EEG data

reflecting the electrophysiological activity of brain belongs to the data collected from


13

meso scale; the gene data reflecting the biological characteristics belongs to the data

collected from micro scale.

To conclude, the BI methodology tries to collect the depression related data as

complete as possible.

2.3 Wisdom as a Service for Depression Treatment

The final goal of this study is to support the adjunctive depression treatment.

Taking the real application as the research target, the BI provides a systematic data

analysis and fusion method following the WaaS architecture, which is a

multidisciplinary, interdisciplinary research field for open intelligence service

architectures [16, 17]. The WaaS focuses on the DIKW organization and transformation

to organize, manage, integrate, and utilize the collected multi-modality data.

Corresponding with the DIKW, four layers are designed to finish the organization and

transformation. The Data as a Service (DaaS) layer collects and analyzes the multi-

modality data for the depressive symptoms. The Information as a Service (InaaS) layer

summarizes the quantitative analysis results and saves them in the SQL database. The

Knowledge as a Service (KaaS) layer integrates the multi-modality data by ontology

technology, and supports the conjoint analysis of the quantitative analysis results and

clinical treatment knowledge. The Web Ontology Language (OWL) and the Resource

Description Framework (RDF) are used to annotate the data meanings and instantiate

the concepts or properties for the data in the database. The WaaS layer provides the

service application for depression treatment based on the previous works.


14

2.4 Multi-modality Data Based Depression Diagnosis and

Evaluation

In order to give an effective evaluation for those depressive symptoms, some

symptoms had better to be evaluated by the device which is common used in clinical

research, such as functional Magnetic Resonance Imaging (fMRI), Positron Emission

Tomography (PET), and Magnetic Resonance Spectroscopy (MRS), and there are also

some symptoms could be evaluated by the wearable devices. The promising domain of

research has been executed in studying the types of physiology that can be sensed

through the use of non-invasive and wearable means. In particular, there have been

various studies on correlating depression to measures such as portable EEG, heart rate,

and voice.

Compared to some other proposed brain imaging biomarkers derived from fMRI,

PET, and MRS, quantitative measurement of brain electrical signals taken from the

scalp-recorded EEG is a neuroimaging technique with clear practical advantages as it

does not involve invasive procedures, is widely available, easy to administer, well

tolerated, and has a relatively low cost [18, 19]. EEG is sensitive to a continuum of

states ranging from stress states, alertness to resting state, and sleep, and various regions

of the brain do not emit the same oscillatory activity simultaneously. During the normal

state of wakefulness with eyes open fast frequency (beta) oscillations are dominant in

central-frontal scalp areas. During relaxation recorded in an eyes-closed resting

condition, alpha activity in the EEG is dominant in posterior scalp regions and is


15

markedly diminished when individuals open their eyes, perhaps reflecting widespread

communication of cortical and thalamocortical interactions to aid information

processing of visual input.

In addition to its growing potential as a biomarker in the therapeutic drug

development process [20, 21] and in predicting antidepressant treatment response [22-

25], power spectral measures of resting state EEG oscillatory activity in different

frequency bands (delta [<4 Hz], theta [4–8 Hz], alpha [8–12 Hz], beta [12–30 Hz]) have

been shown to distinguish between depressed patients and healthy controls [26-28].

Many studies adopt the EEG biomarkers characterizing brain abnormalities to compare

the difference of the depressed group and normal control group by the group-level

comparisons, and the results demonstrate that the EEG biomarkers are informative in

elucidating the neuropath physiology of depression. Other studies utilize the individual

dependent analysis method to systematically examine whether the EEG measurements

can be useful in diagnosing or not or whether a given subject is depressed or not, and

many promising results that indicating the EEG signal contains useful information for

the depression discrimination are obtained.


Chapter 3

3. BI Based Depression Diagnosis and Evaluation

3.1 Introduction

Figure 3.1 shows the schematic diagram for illustrating the BI based systematic

investigation on pathology of depression. The BI methodology integrates multi-

modality tools, the Data Brain model and multi-aspect data analysis approaches are

utilized as core components to support the systematic investigation on pathology of

depression. Based on those components, BI aims at exploring the evolution mechanism

of depression to dig out several endogenous biomarkers for representing the disease

progression. In order to achieve that goal, BI emphasizes on recruiting the subjects in

different disease stages for collecting their multi-modality data, including physiology,

psychology, and behavior data, clinical diagnosis information, and treatment record. By

comparing the differences of the multi-modality data in different stages, the new

discoveries in endogenous biomarkers, depression pathology, and disease progression

could be obtained. There are four stages might be encountered by a subject: normal,

normal but with high depression risk, depressive patient, and rehabilitation. The four

stages could form a closing loop to represent the evolution process of depression, each

stage could further be subdivided into three progression levels: mild, moderate, and

severe. Moreover, four service methods, include evaluation, diagnosis, treatment and

prognosis, are defined to develop service applications served in the four stages,

respectively. For example, the evaluation or diagnosis method could be performed on

Chapter 3

normal people for assessing if he or she has potential for being depressed; the treatment

18

method could be executed on depressive inpatients to make their symptoms ameliorated.

Figure 3.1 The schematic diagram for illustrating the BI based systematic investigation on pathology of depression

For the challenges existed in the traditional depression diagnosis and evaluation

methods, an objective method is valuable for diagnosing and evaluating the depression

without bias. The BI gives a top-down principle for guiding the practices in four core

issues to study depression: systematic investigation on pathology of depression,

systematic experiment design by multi-modality tools, systematic data integration and

management by Data Brain [14], and systematic data analysis by multi-aspect

approaches. The posterior three core issues could be redeemed as the fundamental

infrastructure to support the systematic investigation on pathology of depression. In this

section, we emphasized the construction method for the posterior three core issues and


19

give a detail illustration. As mentioned in the chapter 2, many studies show that the

different depression symptoms related with different physiological, psychological, and

behavioral activities [15-20], which indicates that the depressive symptoms could be

quantitatively analyzed and represented by the multi-modality data. On one hand, multi-

modality data collection method (tools and experiment paradigm) should be developed

to support the systematic data acquirement which aims at collecting the depressive

symptoms related data as complete as possible. On the other hand, a data integration

tool is required for integrating the multi-modality data and the related data analysis

procedures. Considering the symptoms are affected by the treatment efficacy, it is

significant to record the provenance information of the multi-modality data and the

related analysis results for better observing the dynamic changes of depression

symptoms, and supporting the further treatment decisions. Not only the multi-modality

data should be integrated by data fusion model which is compatible with multiple data

formats, but also the related data collection and analysis procedures should be saved.

The BI methodology based objective evaluation method for depression is

introduced to address the questions raised above, the main contributions of this chapter

are summarized as follows:

(1) To give the objective evaluation of multiple depression symptoms, the BI

provides a systematic methodology for guiding the practices in multi-modality data

collection. In the following sections, we clarify what kind of subject is eligible to join

our experiment, and what kind of subject is ineligible. We also designate a specific

single modality data to correspond with one single depressive symptom. That is, a

specific single modality data is adopted to quantify the severity of one single depressive


20

symptom.

(2) As mentioned above, the BI advocates to consecutively collecting the

quantitative data. Regarding this, an experiment paradigm should be designed to satisfy

the requirement of consecutive data acquirement by the multi-modality data collection

tools.

(3) To support the systematic data analysis and fusion, the BI proposes an ontology

based data fusion model -- Data Brain for integrating the multi-mode physiological,

psychological and behavioral data, and recording the related data analysis process.

3.2 Systematic Data Acquirement

The systematic data acquirement refers to what kind of data should be collected

from what kind of subject by what kind of experiment paradigm. The multi-modality

data collected from the depressive inpatients is crucial to achieve the effective objective

diagnosis and evaluation for depression. In addition, the depressive inpatients are easy

to be recruited from the mental hospital, and the data collection experiment is

convenient to be executed in the ward. Such two advantages could facilitate the

systematic data acquirement. In our study, all depressive patients are recruited from

Beijing Anding hospital, China. Every patient, who is willing to participate in this

project, must meet the following inclusion and exclusion criteria:

Inclusion criteria: (1) Inpatient with age 18-60; (2) Right-handed and willingness

to give written informed consent; (3) Meet DSM-IV criteria for Major Depressive

Episode without psychotic symptoms; (4) HAMD score ≥ 20 upon initial evaluation; (5)


21

Meet Mini-international Neuropsychiatric interview (MINI) criteria for unidirectional

depression; (6) Education level above junior high school; (7) Not accept any

psychotropic drug therapy 2 weeks before experiment.

Exclusion criteria: (1) Severe physical illness and unable to complete the

questionnaire; (2) Severe suicide risk; (3) Brain damage with organic disease such as

epilepsy and have other brain disease with random discharge phenomenon; (4) Prior

ECT treatment within the last three months; (5) In accordance with other psychiatric

diagnosis criteria; (6) Alcohol or drug abuse within the last 1 year or psychoactive

addiction in the past and at present.

The goal of multi-modality data collection is to collect the data which is capable of

quantitatively representing the depression symptom. In our design, the portable devices

are chose as the primary tools for collecting the multi-modality data since they are easy

to be operated and able to detect the physiological or behavioral data in a long term.

Besides, the devices with big volume such as fMRI, EEG, and PET are also employed

as data collection tools for measuring physiological signal. We also take the genetic

factor into account to find some evidences of the genetic contribution to depression

susceptibility. Ten categories of data collection methods are used to collect the multi-

modality data, every depressive symptom is objectively evaluated by a corresponding

single modality data. The depressive symptoms and the corresponding data collection

methods are illustrated as follows:

(1) Mental disorder related depressive symptoms including depressed mood,

anxiety psychic, thoughts and feelings of incapacity, fatigue or weakness related to

activities [15]. According to the definition given by the Wikipedia, the MDD is a mental


22

disorder characterized by at least two weeks of low mood that is present across most

situations. Thus, the principal clinical manifestations of the symptom focus on the

disorder of the subjective emotions or feelings. The depressed mood indicates the

negative emotions such as feeling of sadness, hopeless, helpless, and worthless. The

anxiety psychic refers to the depressives feeling subjective tension and irritability, or

worrying about minor matters. It is significant to recognize what kind of mental

disorder does the patient belongs to, and further to quantify the severity level of the

mental disorder. In order to do that, a self-rating tool for assessing the mood status

named Q-Log is developed for self-rating the mental state by inpatients themselves. The

subjects are required to self-rate their mental state just like writing a diary by using the

Q-Log.

(2) Insomnia related depressive symptoms [29]. The clinical interviews assess the

insomnia by observing the sleeping state in three scenarios: early in the night, middle of

the night and early hours of the morning. Several sleeping events contain useful

information for accurately depicting the insomnia symptoms, such as the subject nightly

difficulty falling asleep, waking during the night, and is unable to fall asleep again if

he/she gets out of bed. In our study, the sleep mattress equipped with micro-vibration

sensor is employed as an insomnia monitor to detect the raw vibration signal in real

time, the raw signal is further used to analyze the sleeping events and give a quantitative

representation of insomnia.

(3) Slowness of thought and speech is one of typical depressive symptoms [30].

The clinicians always observe the voice speed during clinical interview. In order to

quantitatively rate the slowness of thought and speech, we develop an android


23

application to record the interview conversation of clinician by using a lavalier

microphone. The mic is clipped on the clinician’s collar so as to make the patient feel

relax to talk his real thought. The speech data of patient is separated from the

conversation and further used to analyze the voice speed and other phonetic features.

(4) Impaired ability to concentrate is one of manifestation patterns of retardation

symptom [31]. A worse impaired ability to concentrate could cause interview difficult

and even complete stupor to the patient. There are many evidences to manifest that the

human attention could be recognized by the EEG signals. For example, NeuroSky

company (San Jose, USA), a leading biosensor company, has developed an EEG chip

with dry electrode located on forehead region [32]. The biosensor integrates an attention

meter algorithm indicates the intensity of mental “focus” or “attention.” The output

value of the algorithm ranges from 0 to 100. The attention level increases when a user

focuses on a single thought or an external object, and decreases when distracted. Users

can observe their ability to concentrate using the algorithm. In order to quantify the

impaired ability to concentrate of patients, a portable EEG device equipped with the

NeuroSky biosensor is used to collect the raw EEG signal and the attention index.

(5) Decrease in actual time spent in activities or decrease in productivity indicates

that the patient spend few hours a day in activities, jobs, or hobbies. With the

development of wearable device for detecting human body movement, this symptom is

easy to be quantified using accelerator sensor [33]. In our study, we develop a diary

behavioral data collection module using accelerator sensor and micro SD card. The

module is required to be carried in a certain position of human body for a long term

during daytime.


24

(6) The Cardiovascular disease (CVD) is one of methods to represent the anxiety

somatic symptom [34]. Persons with depression are more likely to eventually develop

CVD and also have a higher mortality rate than the general population. With the

development of system-on-chip (SoC) technology, the cardiac arrhythmia could be

measured by the portable or mobile ECG recordings. For instance, the NeuroSky has

developed a small sized and low energy consumption biosignal SoC device, named

CardioChip. It is a single chip solution designed to accurately measure, process and

detect bioelectrical signal, such as ECG, by electrodes attached to the human skin

surface. In order to quantify the cardiovascular symptom of patient, a portable ECG

device equipped with CardioChip is adopted to collect the raw ECG signals of patients

and analyze the cardiovascular abnormality.

(7) Cognitive dysfunction refers to deficits in attention, verbal and nonverbal

learning, short-term and working memory, visual and auditory processing, problem

solving, processing speed, and motor functioning. Cognitive complaints are core

symptoms of acute MDD, and diminished ability to think or concentrate or

indecisiveness are criterion items for the diagnosis of MDD [35]. Considering the

cognitive dysfunction is high related with the brain functions, the brain signals

measurement tools such as fMRI and EEG are also included in our experiment.

Combined with cognitive tasks, the cognitive dysfunction could be quantitatively

evaluated.

(8) Genetic factors play important roles in the development of MDD, and may

reveal important information about disease mechanisms [36]. Although the genetic

factor could not be directly explained as one of depressive symptoms, as a systematic


25

data collection method, we also considered the genetic data collection and tried to

investigate the MDD from the micro-aspect.

(9) The reliability of the quantitative evaluation method for depressive symptom

needs to be validated by clinical evaluation criterions. A good reference for validating

the objective method had better to be approved by authorities in clinical field. Regard

this, the ICD-10 and the DSM-Ⅳ, which are common used in the clinical depression

treatment around the world, could provide a reference result for validating the reliability

of objective methods. The Structured Clinical Interview for DSM-5 (SCID-5), which is

a semi-structured interview guide for making the major DSM-5 diagnoses, is common

used in the clinical practice. It is administered by a clinician or trained mental health

professional who is familiar with the DSM -5 classification and diagnostic criteria. In

our study, the SCID-5 is employed as reference to validate the reliability of the

objective evaluation methods for depressive symptoms.

3.3 Systematic Experiment Design

As mentioned above, one of advantages of the objective evaluation for depression

is that it can evaluate the depressive symptoms in fine granularity without subjective

bias. That is, the multi-modality data collection for quantitatively rating the depressive

symptoms could be executed multiple times in a short term. For this regard, the BI

methodology defines the systematic data collection as the method that collects the

multi-modality data from three aspects: multi-scales, multiple cross sections and long

term. The multi-scales data collection indicates the data is collected from three scales:


26

macro scale, meso scale, and micro scale. The multiple cross-sectional data collection

means the data is acquired multiple times in a short term and consecutively collected in

a long term. In this way, not only the multiple symptoms of a subject could be recorded

and quantitatively evaluated simultaneously just like the traditional evaluation way by

clinical interview, but also the symptoms could be observed and compared in the same

time scale.

Figure 3.2 illustrates the multi-modality data collection paradigm form three

aspects. The aspect 1 shows the multi-modality data collection tools. The aspect 2 gives

a further illustration for the experiment paradigm by dividing it into 3 dimensions. The

Z axis represents the scale dimension which indicates that multi-modality data collected

in macro scale, meso scale, and micro scale. For example, the genetic data belongs to

the micro scale; the fMRI and EEG data belong to the meso scale; the sleeping mattress

and diary behavior data belong to the macro scale. The X axis means time scale and

gives the information about the time interval of data collection. In this experiment,

different kinds of data adopt different time intervals to collect the data. For example, we

require the subject to self-evaluate the mood state by the Q-log two times a day (once in

the morning and once in the night); the diary behavior data is consecutively collected

above 8 hours during daytime; the speech data of the clinical interview is recorded once

a week. The Y axis indicates modality scale and shows the multi-modality data collected

by the corresponding data collection tool.


27

Figure 3.2 Three aspects for illustrating data collection paradigm

3.4 Systematic Data Analysis and Fusion

After collecting the multi-modality data, BI provides a systematic data analysis and

fusion method following the WaaS architecture, which is a multidisciplinary,

interdisciplinary research field for open intelligence service architectures [17, 37]. The

WaaS focuses on the DIKW organization and transformation. The technical route of the

WaaS for adjunctive depression treatment is illustrated as follows:

The Data as a Service (DaaS) layer aims at collecting and analyzing the multi-

modality data for the objective evaluation of depressive symptoms. Specifically, the

multi-modality data of subject is acquired by various data collection tools. Data analysis

procedures such as data cleaning, time alignment and feature extraction are executed by

the machine learning and mathematical statistics methods. The objective evaluation


28

works for depressive symptoms are finished and the quantitative rating results are

obtained.

The Information as a Service (InaaS) layer aims at concluding the quantitative

analysis results and saving them in a database. In order to support the conjoint analysis

of the quantitative analysis results and clinical treatment knowledge, the database

structure is designed according to the ontologies constructed in the KaaS layer. Every

element in the datasheet is an instance of the concept or the object in the ontology. The

MySQL database is used to save the multi-modality data and the quantitative analysis

results. Meanwhile, BI provenances, including data provenance and analysis

provenance, are adopted as metadata to depict the data in the database. The data

provenance is a metadata set that describes the BI data origin by multi-aspect

experimental information, including subjects information, how experimental data of

subjects were collected, what instrument was used, etc. The analysis provenance is a

metadata set that describes what processing in a brain data set has been carried out,

including what analytic tasks were performed, what experimental data were used, what

data features were extracted, and so on. In this way, we tried to record the related

information as complete as possible so as to ensure the reproducibility of the

quantitative analysis results, and further to provide instantiations for mapping the data

in the MySQL database with the concepts and objects defined in the Knowledge as a

Service layer.

The Knowledge as a Service (KaaS) layer aims at integrating the multi-modality

data by ontology technology, and supporting the conjoint analysis of the quantitative

analysis results and clinical treatment knowledge. The Data-Brain model comprises four


29

dimensions (function dimension, data dimension, analysis dimension, and experiment

dimension) is used to support the systematic data fusion. The Web Ontology Language

(OWL) is adopted to construct the four dimensions and define the relationships of data,

quantitative results, and depressive symptoms. And then, the Resource Description

Framework (RDF) is utilized to instantiate the data in the InaaS layer as the instances of

the concepts and properties of the Data-Brain model. Figure 3.3 gives a diagram for

illustrating the concepts depicted in the four dimensions of Data-Brain model. Each

subfigure represents a dimension of Data-Brain. In the subfigure, the center of the red

triangle shows the name of the dimension, and the vertex gives three chief purposes of

the dimension. The function dimension aims at describing the concepts about the

symptoms of unipolar, bipolar depression, or other psychiatric disorders; the experiment

dimension aims at describing the details about the data collection experiment, which

could be performed in resting state, ubiquitous state, and using a cognitive task; the data

dimension aims at describing the concepts about the collected data for quantitative

evaluation of depressive symptoms, including the data collected from clinical record,

clinical therapy ontology, and multi-modality data; the analysis dimension aims at

describing the concepts about the methods for processing the collected data, such as the

statistical analysis method and data mining method for rating the symptoms, the

semantic analysis method for processing the ontology data. The concepts described in

each dimension are further to be divided into several categories, which are represented

by the blue words. The specific examples represented by the red words are listed at the

outmost layer. Specifically, the function dimension categorized the depression

symptoms into six categories, including body movement, insomnia, cognitive ability,


30

speech, physiological abnormality, and mental disorder. The outmost layer gives some

examples for every symptom, such as the mental disorder including fatigue, depressed

mood, and feeling of incapacity. Every specific symptom is objective evaluated by a

single modality data. The experiment dimension categorized the data collection

experiment into six categories, such as the behavior detection experiment, scale

assessment experiment, sleeping signal detection experiment, and etc. The outmost

layer shows the experiment tool for executing the experiments, such as the portable

EEG, accelerator sensor, portable EEG, Q-Log, and etc. The data dimension categorized

the multi-modality data into three categories, including the physiology data, psychology

data, and behavior data. The outmost layer shows the specific data collected in the

experiment, such as the portable EEG, portable EEG, speech, Q-Log, and etc. The

analysis dimension categorized the data analysis methods into six categories, such as the

data cleaning, feature extraction and analysis, classification, and etc. The outmost layer

shows the specific analysis methods used in our study, such as diary activity evaluation,

insomnia analysis, speech analysis, mental state recognition, and etc.

The WaaS layer aims at providing a web service to help users for querying the data

information and the relating data analysis results, and giving intuitive illustrations for

explaining the relationship of them based on the concepts or properties in the ontology.

In the beginning, two kinds of services are provided in the WaaS layer: multi-modality

data query and quantitative evaluation for depressive symptom. The service of the

multi-modality data query is constructed based on the RDF data, the SPARQL Protocol

and RDF Query Language are used to query the RDF data and obtain the information of

the multi-modality data. The service of the quantitative evaluation for depressive


31

symptom is constructed based on the analysis results of multi-modality data and the

corresponding clinical explanations. Since we try to quantitatively evaluate every single

depressive symptom in fine granularity by one modality data, the service could be

provided by the means of routine monitor and feedback to report the dynamic change of

the symptom.

Figure 3.3 Diagram for illustrating the depression-related concepts depicted in the four dimensions of Data-Brain model


Chapter 4

4. Multi-Channel EEG Based Objective

Depression Discrimination

4.1 Introduction

Based on the various physiology measurement tools, such as functional magnetic

resonance imaging (fMRI), electroencephalogram (EEG), Positron Emission

Tomography (PET), many studies tried to measure the psychological data and develop

adjunctive diagnostic approach in clinical practice. As one of the measurement tools,

quantitative measurement of brain electrical signals taken from the EEG is a

neuroimaging technique with clear practical advantages as it does not involve invasive

procedures, is widely available, easy to administer, well tolerated, and has a relatively

low cost. The pervasive and persistent nature of depressive symptoms has made scalp-

recorded EEG as an appropriate approach for understanding the underlying mechanisms

of major depressive disorder.

Recent advances in EEG acquisition and processing for discriminating depression

have been paralleled by the increased availability of machine learning methods. To date,

many machine learning studies on resting state EEG in depression have used varying

classification algorithms and have been found to classify depressives and healthy

controls with an overall accuracy ranging between 60–90 % [38-41]. Despite their

promise as a supplementary, computer-aided diagnostic approach for depression, these

Chapter 4

analytic methods are semi-automatic since they are designed to deal with multivariate

features extracted from the EEG signals. Few studies adopted the raw EEG data as the

34

model input directly for classifying the depressives and healthy controls. Furthermore,

feature extraction step is always followed by the feature selection, which is mainly

performed by the grouped-analysis and trial-and-error method. The grouped-analysis

method based on the analysis of variation (ANOVA) statistical test is suitable for

comparing every specific feature of depressives and healthy groups, and discovers

which feature showing the most meaningful differences between the two groups. The

trial-and-error through numerical experimentation aims at obtaining the feature subset,

which has the highest accuracy in discrimination, from a high dimensional feature

vector. Such feature extraction and selection steps required manual operation are time-

consuming and labor-intensive. The classification method, which can directly learn

from the raw EEG data and automatically extract the EEG features, is more suitable to

construct an automated EEG analytical approach for depression discrimination. Deep

learning is such kind of machine learning method which is commonly used in many

EEG data based classification scenarios. As one of the deep learning methods, the

convolutional neural network (CNN) is able to directly learn the EEG features or

patterns from the raw data, and does not require a hand-crafted set of features to be fed

into a classifier for classification [42].

In this chapter, we focus on utilizing Convolution neural network (CNN) model to

discriminate the EEG samples of depressives and normal controls, and try to

discriminate the EEG samples of medicated and unmedicated depressives. There are

many studies used CNN to execute the classification tasks based on the EEG

measurement. For example, Schirrmeister et al. studied convolutional neural networks

(deep ConvNets) with a range of different architectures for decoding imagined or


35

executed movements from raw EEG [43]. Putten et al. used deep convolutional neural

networks to explore if brain rhythms contain sex specific information for discriminating

sex [44]. Li et al. utilized CNN model for extracting task-related features, as well as

mining inter-channel and inter-frequency correlation, to carry out trial-level emotion

recognition task [45]. A deep net learned discriminative features between imagining

music or listening to music or rhythm perception. Differentiation between early stage

Creutzfeldt-Jakob disease and other forms of rapid progressive dementias with deep

learning achieved a sensitivity of 92% at specificity of 89%. Classification of sleep

stages using deep nets performed on par with human sleep experts. Detection of

interictal epileptiform discharges is another promising application for deep nets, and

may soon become a standard clinical tool to assist in the diagnostic process in epilepsy.

Acharya et al. presented the first application of the deep neural network concept and

CNN for depression diagnosis. The model was tested using EEGs obtained from 15

normal and 15 depressed patients. The algorithm attained accuracies of 93.5% and

96.0% using EEG signals from the left and right hemisphere, respectively.

Although those studies demonstrate that the deep learning method may become a

successful diagnostic tool for brain-related illness, most of them merely adopted one

kind of convolutional filter to learn the EEG pattern and less concerned multiple

convolutional filters. For a deep learning method, the convolutional operation decides

which kind of EEG pattern could be learned by the CNN model. Considering the scalp

EEG activity always reflects the summation of the synchronous activity over a network

included thousands or millions of neurons that have similar spatial orientation, the EEG

synchronous pattern generated from multiple brain regions might contain effective


36

depression-specific information. That is, the EEG synchronous pattern might be an

effective feature for discriminating depression. Many studies also show that the spatial

distribution of the EEG feature, which is reflected by the channel location, contains

depression-specific information. The representative case is the study about the

functional connection network, which extracts the feature from every single channel and

analyzes the correlation. Learning the two kinds of EEG patterns simultaneously and

using them to discriminate depression, the CNN model should obtain a better

classification performance. In addition, explaining the physical meaning of the features

which contain specific depressive and even unmedicated depressive information is

significant to discriminate the depression in the early stage. Those studies have yet to

explain the physical meaning of the features learned by the deep learning network.

In addition, we are interested in the feature learned by the CNN model to

differentiate normal controls from depressives, and even from medicated and

unmedicated depressives. Because of scaling in resting-state EEG has been suggested as

a biomarker for pathophysiology in neurodevelopmental disorders, many studies tried to

mine the EEG features that have significant difference of depressives and normal

controls, and investigate if the features could be a trait markers for depression [46-49].

For instance, the previous studies have frequently reported the frontal alpha asymmetry

(FAA) potential discriminator between depressed and healthy individuals [50]. And the

elevated EEG alpha activity during rest in depressed patients has been one of the main

and most consistent findings from studies of recent decades [49]. Besides, since the

brain is a complex nonlinear system, the search for nonlinear biomarkers of emotional

diseases has seen fast growth in recent years [46, 51-53]. However, there are many


37

incongruous and even contradicting results questioned the validity of the EEG

biomarkers for depression, and there have yet to demonstrate a reliable EEG biomarker

effectively differentiate the normal controls from depressives. Considering one of

advantages of the CNN model is it can obtain the feature with the most discriminative

power for classifying objects through multiple filters, the feature learned by the CNN

model and the related feature analysis are possible to reveal the effective EEG features

for discriminate depression in early stage, and facilitate the work of finding the EEG

biomarker for depression discrimination. In this regard, we would like to explore

whether the features learned by the CNN model contain the depression specific

information and the specific information for diagnosing the depression in early stage or

not.

The procedure we used to obtain the features learned by the CNN is similar to a

technique called “deep-dreaming” which has been described elsewhere in more detail.

The trained CNN network is consisted of several layers and there are lots of weights

between the layers. The essence of the method is that the network is activated “top-

down”, meaning that from a desired output (e.g. normal control=0) to a desired input

layer, which is initialized to an artificially generated matrix in the beginning and

iteratively updated by the weights between the layers until obtaining a desired input.

That is, the method does not try to adjust the weights of the CNN model, but merely

update the input feature matrix iteratively. The desired input representing an archetypal

feature or pattern that most likely would produce the desired output.

The final goal of this study is to develop a CNN model based depression screening

application, and further to discovery the effective EEG biomarkers for discriminating


38

MDD. For achieving this, the classification performance of the model should be

estimated for ensuring the model could provide a reliable classification result. The n-

fold cross validation method is common used in estimating the skill of machine learning

models, which aims at assessing how the results of a statistical analysis will generalize

to an independent data set. However, for a disease screening application in the real

world, we are more concerned with how many participants are classified correctly. That

is, the generalization of the model to an independent participant is more significant than

the independent data set. In this regard, the threshold based leave-one-participant-out

cross validation (LOPOCV) method is used to estimate the classification performance

of the CNN model. Here, we hypothesized that the spontaneous EEG of depressives is

not always accompanied with depressive characteristics, and the several EEG fragments

of healthy people might contain depressive characteristics. Thus, the majority voting

strategy could be used to determine the participant is classified correctly if the number

of his data samples classified correctly exceeds a certain threshold. With a higher

threshold, the higher classification performance the model achieves the better

generalization ability the model has.

The main contributions of this chapter are as follows:

(1) We propose a CNN architecture merging the two kind of convolutional filters

(TcfCNN for short) to learn EEG patterns, and tackle two-category (depressives and

normal controls) and three-category (medicated, unmedicated depressives and normal

controls) classification tasks.

(2) The FFT is used to analyze the spectral distribution on the feature learned by

the CNN model. A procedure similar to the “deep-dreaming” algorithm is used to


39

generate the feature produced by the convolutional layer or the softmax layer. The

physical meanings of the features learned by different convolution operations are

illustrated.

(3) A threshold based LOPOCV method and the n-fold cross validation method are

used to estimate the classification performance of the CNN model, respectively.

Combined with majority voting strategy, the threshold based LOPOCV method

estimates the model performance multiple times by different thresholds, and the

corresponding estimation results are also given. 10-fold cross validation method is used

to validate the classification performance of the TcfCNN model. We also constructed

four types of CNN models as the baseline. Specifically, two kind of convolutional filters

are used to construct a CNN model, respectively. Another two kinds of CNN models

used in other studies are resurrected. The comparison results are given and analyzed.

4.2 Materials and Methods

4.2.1 Data Acquisition

All depressive patients were recruited from Beijing Anding hospital, China. Every

patient, who is willing to participate in this project, must meet the following inclusion

and exclusion criteria mentioned in the Section 3.2. The normal control group of the

experiment is required to have no psychiatric disorders in the past and also selected

according to the HAMD scores. Finally, 16 normal controls (8 females and 8 males),

ranged in age from 21 to 55 years (26.4±9.8 years; mean±standard deviation (Std.)), and


40

23 depressive patients (12 females and 11 males), ranged in age from 20 to 56 years

(29.3±9.6 years; mean±Std.), are recruited. The 23 depressive patients include 12

unmedicated patients (6 females and 6 males), ranged in age from 25 to 54 years

(28.6±7.3 years; mean±Std.), and 11 medicated patients (6 females and 5 males), ranged

in age from 20 to 56 years (29.8±10.6 years; mean±Std.).

All subjects are asked to record their EEG data in the resting state. Specifically, the

subjects are sitting on a sofa and keeping with eye closed for 8 mins while not to think

anything purposefully in a dim illuminated and acoustical room. They are also asked to

maintain a minimum arousal level without falling into sleep. In order to save the

operation time of the data collection (e.g. daub conductive gel onto the electrode

channel), we select several representative electrode channel located on different brain

regions to collect the multi-channel EEG data. 8 surface electrodes (Fp1, Fp2, F3, F4,

P3, P4, A1, A2) placed on the scalp according to 10-20 electrode international system to

record the multi-channel EEG data. Among the surface electrodes, the A1 and A2 are

reference electrodes which data are not analyzed in this study. EEG recordings are

performed using a standardized methodology and platform (Brain Products Ltd.,

Germany), which data sampling rate is 500 Hz and equipped with BrainAmp 16-bit A/D

convertor (ADC) amplifier. The software used for analysis is Python 3.0 configured

with Tensorflow.


41

4.2.2 Methods

4.2.2.1 Data Preprocessing

The EEG data recorded in one trial is cut into three snippets, and the median

snippet with the time length of 5 minutes is kept for analyzing. The time snippets of the

beginning 30 seconds and the last approximate 2 minutes 30 seconds of each EEG

sample are removed. The Z-score normalization is adopted to overcome the amplitude

scaling problem and remove the offset effect. After that, the EEG record of each channel

is partitioned into several fragments without overlap. Every fragment contains 3072

sampling points (6.144 seconds). According to the channel order of Fp1, Fp2, F3, F4,

P3, P4, the fragments of 6 channels are realigned into a data matrix. The 1th row of the

data matrix is corresponding with the EEG data of Fp1 channel, the 2nd row of the data

matrix is corresponding with the EEG data of Fp2 channel, the 3rd row of the data

matrix is corresponding with the EEG data of F3 channel, by analog. Every data matrix

is fed into the CNN model as a new independent data sample. For every subject, the

data record of 5 minutes is fragmented into 50 data samples. The dataset used in this

work includes a total of 1872 data samples (800 data samples of normal control, 550

data samples of medicated depressive and 600 data samples of unmedicated depressive).

4.2.2.2 Input Data Representation and EEG Feature Learning

We assume that there are M EEG records, and every record is a data matrix with

size of C×D, where C means the number of channel and D represents the length of the


42

record. According to the EEG data collection channels, we realigned the channel

sequence in the matrix as Fp1, Fp2, F3, F4, P3, P4. Then, the record can be further

represented by a sequence of fragments{X1,X2 ,…,XT}. The size of each fragment Xt

is C×d, where d equals the quotient of D/(number of the fragment). Inspired by

successful architectures in computer vision, we represent the input data of the first layer

as a 2D-matrix which comprises 6 channels and each channel contains 3072 data points.

Each fragment Xt also has a corresponding category label yt, for two-category

classification, the yt∈{0,1}, for three-category classification, the yt∈{0,1,2}. With the

aforementioned notations, the inputs of the proposed CNN model are the set of time-

ordered sequences {X1,X2 ,…,XT } with a set of corresponding labels {y1,y2,…,yT}.

To tackle the task of learning the EEG feature, two types of feature learning

methods or convolution operations are designed. One is that the convolution operation

is split into a first convolution across single channel and a second convolution across

time. And the other is that the convolution operation is split into a first convolution

across multiple channels and a second convolution across time. The feature learned by

the first type could be regarded as the EEG characteristic of single channel. This feature

learning method is very similar with the traditional EEG feature learning, which learns

the feature channel by channel and form a vector to represent the raw EEG data. The

feature learned by the second type could be regarded as the EEG characteristic of

multiple channels. For convenience, we named the feature learned by the first type as

the single EEG pattern, and named the feature learned by the second type as the

synchronous EEG pattern.


43

4.2.2.3 CNN Model Construction

Figure 4.1 shows the model architecture of TcfCNN designed in this work. We

designed two parallel lines to run two independent tensorflow graphs. That is, the

TcfCNN model is consisted of two independent CNN sub-models. Every independent

CNN model comprises 8 convolutional layers and 8 max pooling layers. 3 fully

connected layers and 1 softmax layer are shared by them to form the TcfCNN. The first

fully connected layer is also known as concatenate layer. The specific operations are

illustrated as follow:

(1) Convolutional layer. In order to facilitate description, we use the uppercase and

lowercase letters to distinguish the layer operations in the two sub-models, respectively.

Let layer l be a convolutional layer. Then, the input of layer l comprises m1(l−1) feature

maps from the previous layer, each of size m2(l−1)×m3

(l−1). In the case where l=1, the input

is an EEG fragment Xm. This way, a convolutional neural network directly accepts the

fragment of multi-channel EEG data as input. The output of layer l consists of m1l

feature maps of size m2l ×m3

l . The ith feature map in layer l of the two sub-models,

denoted Y il and y i

l, is computed as

Y il=Bi

(l)+∑j=1

m1(l−1)

K i , j(l)∗¿Y i

( l−1)¿ y il=b i

(l)+∑j=1

m1(l−1)

k i , j(l )∗y i

(l−1) (4-1)

where Bi(l) and b i

( l) are bias matrixs, K i , j(l ) and k i , j

(l) are the convolutional filters connecting

the jth feature map in layer (l-1) with the ith feature map in layer l. In the Figure 4.1, the

convolutional filter is marked by red rectangle, the size of the filter for learning EEG

synchronous pattern is 6 × 8, the size of the filter for learning EEG single pattern is 1 ×


44

8. The corresponding convolution operation result is marked by black rectangle which

size is 1 × 1.

(2) Activation function. The leaky rectifier linear unit (LeakyRelu) is applied as the

activation function after the convolution operation. We utilize the the f (•) as the

activation function used in layer l and operates point wise, the formula is listed as

follow:

f (Y i(l))={ Y i

(l )if Y i(l)>0

0.01∗Y i(l)otherwise

f ( y i(l))={ y i

(l) if y i(l)>0

0.01∗y i(l)otherwise

. (4-2)

(3) Pooling layer. Let l be a pooling layer. Its output comprises m1(l)=m1

(l−1) feature

maps of reduced size. In general, pooling operates by placing windows at non-

overlapping positions in each feature map and keeping one value per window such that

the feature maps are subsampled. Max pooling is taken as the down-sampling operation.

In the Figure 4.1, the max pooling filter is marked by green rectangle, the filter size is 1

× 2. The corresponding down-sampling operation result is marked by black rectangle

which size is 1 × 1.

(4) Concatenate Layer. The last three layers of the model are fully connected

layers, which neurons are fully-connected with previous layer. The concatenate layer is

also the first fully connected layer, we concatenate the output tensorflows of the

previous layers of two sub-models into vector and feed it into the first fully connected

layer. We use the sign ∪ to represent the tensorflow concatenation operation in the

Figure 4.1. The equation for computing output of the ith unit in the concatenate layer l is


45

listed as follow:

Zi(l)=∑

j=1

m1(l−1)

∑r=1

m2(l−1)

∑s=1

m3( l−1)

∑t=1

m4

wi , j ,r ,s , t(l ) ( y j

(l−1 ))r ,s , t (4-3)

where w i , j ,r , s(l) denotes the weight connecting the unit at position (r,s) in the j th feature

map of layer (l-1) of tth sub-model and the ith unit in layer l. The m4 represents the

number of CNN sub-models.

(5) Fully connected layer. Let layer l not be the first fully connected layer, then the

input of the layer l is the m1(l−1) feature maps of size m2

(l−1)×m3(l−1), and the output of the ith

unit in layer l is computed as:

Zi(l)=∑

k=1

m1(l−1)

w i ,k(l) yk

(l−1) (4-4)

where w i ,k(l) and yk

(l−1) denote the corresponding weights of the ith unit in the layer l and

the outputs of the layer (l-1), respectively. The figure shows that the last layer contains 3

neurons which indicate three classification labels. One hot coding strategy is exerted to

represent the three labels: normal, medicated depressive, and unmedicated depressives.

We are also able to set the number of neuron in this layer to 2 for representing the

normal and depressive data samples and executing the two-category classification.

Table 4.1 gives the parameter illustration of the TcfCNN model. The input size of

ith layer indicates how many neurons (channels× data points× feature maps) are included

in the ith layer. The output size of ith layer indicates how many neurons are included in

the (i+1)th layer. It is noteworthy that the input and output size of i th layer of each

independent CNN models are congruent. For each feature map, we define the direction

along the data points as x-axis, and the direction along the channels as y-axis. The filter

size means the size of the convolutional filter or max pooling filter. The ‘syn’ represents


46

the convolutional filter for learning the synchronous EEG pattern, the convolutional

filter (N channels × m points) shifts along the y-axis one stride per time. The shift time

of the filter along the y-axis is equal with the channel number. In this way, the first row

of the output of the convolutional layer could be deemed as the convolution operation

result of the EEG data of N channels. The second row of the output of the convolutional

layer could be deemed as the convolution operation result of the EEG data of N-1

channels. And the last row of the output of the convolutional layer could be deemed as

the convolution operation result of the EEG data of the channel. It is noteworthy that we

adopt zero-padding to fit the convolutional filter. The ‘sin’ represents the convolutional

filter for learning the single EEG pattern, the convolutional filter (1 channel × m points)

also shifts along the y-axis one stride per time, but each row of the output of the

convolutional layer could be deemed as the convolution operation result of the EEG

data of the corresponding single channel. The stride gives the information about the

filter shifts along the x-axis and y-axis. The convolutional filters shift along the x-axis

and y-axis 1 unit per time, respectively. The max pooling filters shift along the x-axis

and y-axis 2 units per time and 1 unit per time, respectively.


47

Figure 4.4 The architecture of the TcfCNN model for three-category classification


48

Table 4.1 The parameter illustration of the TcfCNN model

Layers

Type Input size Output size

Filter size Stride‘syn’ ‘sin’ y x

0 Input 6*3072 6*3072 - - - -1 Convolution 6*3072 6*3072*6 6*8 1*8 1 12 Max-pooling 6*3072*6 6*1536*6 1*2 1*2 1 23 Convolution 6*1536*6 6*1536*6 6*8 1*8 1 14 Max-pooling 6*1536*6 6*768*6 1*2 1*2 1 25 Convolution 6*768*6 6*768*6 6*8 1*8 1 16 Max-pooling 6*768*6 6*384*6 1*2 1*2 1 27 Convolution 6*384*6 6*384*6 6*8 1*8 1 18 Max-pooling 6*384*6 6*192*6 1*2 1*2 1 29 Convolution 6*192*6 6*192*12 6*8 1*8 1 110 Max-pooling 6*192*12 6*96*12 1*2 1*2 1 211 Convolution 6*96*12 6*96*12 6*8 1*8 1 112 Max-pooling 6*96*12 6*48*12 1*2 1*2 1 213 Convolution 6*48*12 6*48*12 6*8 1*8 1 114 Max-pooling 6*48*12 6*24*12 1*2 1*2 1 215 Convolution 6*24*12 6*24*12 6*8 1*8 1 116 Max-pooling 6*24*12 6*12*12 1*2 1*2 1 217 Concatenate 6*12*12*3 32 - - - -18 Fully connected 32 16 - - - -19 Fully connected 16 3 - - - -

4.2.2.4 CNN Model Training and Testing

To train the CNN model, all variables of CNN model are initialized with random

Gaussian distributions and trained for 1000 epochs, the batch size of every epoch is 300

data fragments. In each epoch, a random resampling strategy for selecting training data


49

is adopted to avoid the model performance decrease caused by the sample

disequilibrium. For the two-category classification, 150 depressive data fragments and

150 normal control data fragments are used to train the model in every iteration. For the

three-category classification, 100 medicated depressive data fragments, 100

unmedicated depressive data fragments, and 100 normal control data fragments are used

to train the model in every iteration. The other parameters are momentum: 0.9; weight

decay: 0.0005; learning rate: 0.001; and dropout rate: 0.9. The gladiation descent

method is chose to update the weights, and the categorical cross-entropy is used as the

loss function.

The n-fold cross validation is a common used method for training the classifier and

testing the classification model. However, in the scenario of partitioning the signals of

subjects into several data samples, the n-fold cross validation method might influence

the model performance and provide an inaccurate estimation. Specifically, partial testing

data and training data selected by the n-fold cross validation might generate from the

same subject. Due to the brain is a steady biological system that possesses stable

electrical discharge activity, the EEG data generated from the same person could contain

the resembling characteristics. Meanwhile, because of developing a depression

prescreening application is the final goal, the participant-independent analysis method,

such as the leave-one-participant-out cross validation (LOPOCV), is intuitive to know

how many subjects are correctly identified. Besides, we hypothesize that the

spontaneous EEG of depressives is not always accompanied with depressive

characteristics, and the several EEG fragments of healthy people might contain

depressive characteristics. The subject could be deemed as being classified correctly if


50

the number of his data samples classified correctly exceeds a certain threshold. Actually

such method belongs to one kind of majority voting strategy. That is, the threshold

should be set to a high threshold (at least greater than 0.5), which indicates that the most

EEG samples should be classified as the MDD or normal control. Regard this, a

threshold based LOPOCV method is utilized to train the classifier and discriminate the

MDDs and normal controls.

The 10-fold cross validation and the threshold based LOPOCV method are used to

estimate the classification performance of the model, respectively. For the 10-fold cross

validation, 90% of the data samples are used to train the CNN model, and the remaining

10% of the data samples are adopted to test the model. The classification results of 10

times are averaged for illustrating the model performance. For the threshold based

LOPOCV method, we choose the data samples of every subject as the testing data in

turn, and the data samples of remaining 38 subjects are used as the training data. A

majority voting strategy is employed to determine the subject is correctly identified if

the number of the data samples correctly classified exceeds a certain threshold. The

final classification accuracy is the ratio between the number of subjects identified

correctly and the total number of the subjects. Specifically, for every subject, the EEG

data record with 5 minutes length is fragmented into m data samples, the size of every

sample is 6×3072. N indicates the total number of the subject. For every validation, m

EEG samples of the ith subject are used as the testing data, and the m×(N-1) EEG data

samples of the remaining (N-1) subjects are used as the training data. Train the CNN

model based on the m×(N-1) EEG data samples, and feed the testing data samples into

the model and count the number n of the samples which are correctly classified. A


51

threshold T is given to determine whether the ith subject is correctly classified or not.

That is, the ith subject is correctly classified if the ratio of n and m is greater than the

threshold T. Repeat the procedures mentioned above until the data samples of every

subject are chosen as the testing data. Calculate the evaluation indexes for estimating

the classification performance of CNN model.

4.2.2.5 Classification Performance Evaluation

The statistical measures include sensitivity (SEN), specificity (SPC) and

recognition accuracy (ACC) are used to evaluate the classification performance of CNN

model. For the two-category classification, the two categories are labeled as normal

control and depressive, and the depressive is defined as the positive label. The

calculation formulas are given bellow:

SEN=TP /(TP+FN )SPC=TN /(TN +FP)

ACC=(TP+TN)/ (TP+FP+FN +TN ) (4-5)

where P (Positive) means the sample number of depressives; N (Negative) means the

sample number of normal controls; TP (True Positive) indicates the number of samples

which are labeled as depressives and correctly identified as depressives; FN (False

Negative) indicates the number of samples which are labeled as depressive and

incorrectly identified as normal control; TN (True Negative) indicates the number of

samples who are labeled as normal control and corrected identified as normal control;

FP (False Positive) indicates the number of samples who are labeled as normal control

and incorrectly identified as depressive. Sensitivity refers to the classification ability of


52

classifier to correctly detect depressed patients. Specificity refers to the classification

ability of classifier to correctly detect normal controls. Recognition accuracy refers to

the classification ability of classifier to correctly detect normal controls and depressed

patients.

For the three-class classification, the three categories are labelled as normal

control, medicated depressive and unmedicated depressive. The one against all approach

is utilized to calculate the three evaluation indexes. Take the medicated depressive

which is defined as the positive label for example, the other two labels are defined as

negative label. “TP of medicated depressive” means all medicated depressive data

samples that are identified as medicated depressive, “TN of medicated depressive”

imdicates all non-medicated depressive data samples that are not identified as medicated

depressive, “FP of medicated depressive” represents all non-medicated depressive data

samples that are identified as medicated depressive, “FN of medicated depressive”

depicts all medicated depressive data samples that are not identified as medicated

depressive. Similar with the method mentioned above, it is easy to figure out the

calculation method for the three indexes when define the normal control or the

unmedicated depressive as the positive label. The final result of the evaluation indexes

is the average of the three situations.


53

4.2.2.6 Analyze the Features Learned by Convolutional

Layers and Softmax Layer

Taking the three-category classification for example, Figure 4.2 shows the

technical route for obtaining the features of the convolutional layer or the softmax layer

of the TcfCNN model. The feature analysis procedure is divided into two steps. The first

step is the CNN model training. We input the training data samples into the model, and

use gradient descent to optimize network weights. After finishing the model training, we

save the trained model, that is, the weights of different layers are kept unchanged. The

second step is feature generation. With the aforementioned notations, the feature is

initialized to an artificially generated matrix in the beginning and iteratively updated by

the weights between the layers until obtaining a desired input. The desired input

representing an archetypal feature or pattern that most likely would produce the desired

output. Specifically, for the softmax layer, the desired ouputs are medicated,

unmedicated depressive, or normal control. We can start from any of the three neurons

to iteratively update the feature matrix until obtaining the same classification label of

the corresponding neuron. For the convolutional layer (e.g. 8 th convolutional layer), we

first sort the tensorflow value of neurons and then choose the maximum one to

iteratively update the feature matrix. During this process, we generated the feature

matrix by retrograde ascending of the gradients in the trained network model.

Since the convolutional layer or the softmax layer producing the feature matrix that

reveals specific frequency features in the time domain when computing spectral analysis


54

on each row, the FFT method is adopted to analyze the frequency characteristics of each

row of the feature matrix. For the feature matrix generated by the convolutional layer of

the sub-model learning the EEG single pattern, it shares similar frequency properties

with original EEG data due to the convolutional filter merely operates the data of single

channel per time. For the feature matrix generated by the convolutional layer of the sub-

model learning the EEG synchronous pattern, except the last row, the FFT result on

each row of the matrix should reflect the synchronous properties of multiple EEG

channels due to the convolutional filter merely operates the data of multiple channels

per time. For the softmax layer of the TcfCNN model, since the model output is a

comprehensive result affected by the two sub-models, the feature matrix might

simultaneously contain the EEG characteristics of single channel and multiple channels.

Figure 4.5 The technical route for obtaining the features of the convolutional layers or the softmax layer of the TcfCNN model


55

4.3 Results

4.3.1 Experimental Setup

On one hand, we separately construct and train the sub-models of the TcfCNN to

execute the two-category and three-category classifications, respectively. On the other

hand, the state-of-the-art deep learning approaches adopted in other studies, which have

similar background of EEG based classification task, are also resurrected. Totally, four

kinds of CNN models are constructed as baseline approaches for result comparison.

(1) Baseline approaches. The details about the four baseline network architectures

are described as follows:

a. SynCNN. The CNN model merely learning the EEG synchronous pattern is

named as SynCNN. Based on the construction and parameter setting of the TcfCNN, we

only keep the sub-model part which learns the synchronous EEG pattern and remove the

concatenate operation in the first fully connected layer. The last two fully connected

layer, softmax layer, and model input are as the same as the TcfCNN.

b.SinCNN. The CNN model merely learning the EEG single pattern is named as

SinCNN. Similar with building the SynCNN model, we only keep the sub-model part

which learns the single EEG pattern and also remove the concatenate operation in the

first fully connected layer. The last two fully connected layer, softmax layer, and model

input are also as the same as the TcfCNN.

c. SchCNN. The CNN model constructed by Schirrmeister et al. is named as

SchCNN. They designed a CNN model with four convolution-max-pooling blocks, a


56

special first block designed to handle EEG input and followed by three standard

convolution-max-pooling blocks and a dense softmax classification layer. The primary

difference between the approach and our study is the output of the first convolutional

layer is a 2D-array with the number of data points as the width and the number of

feature maps as the length. That is, the convolutional filter mixed the EEG data of

multiple channels into a row, which is unable to reflect the spatial distribution

characteristics of EEG channels.

d. AchCNN. The CNN model constructed by Acharya et al. is named as AchCNN.

They presented the first application using the CNN model for EEG-based screening of

depression. The primary difference between the approach and our study is the data

representation of the input layer. The input data of their model is represented by an array

with the size of 1 channel * M data points. The model consists of 13 layers include 5

convolutional layers, 5 max pooling layers, and 3 fully connected layers.

(2) Our approaches. The TcfCNN model shown in the Figure 4.1 is used to

compare with the four baseline CNN networks.

4.3.2 Classification Results Comparison

4.3.2.1 Classification Results Using 10-fold Cross Validation

Figure 4.3 compares the overall performance of the five network architectures on

two-category and three-category classification. From the figure, we can observe that the

TcfCNN and the SynCNN achieve the best performance on two-category and three-


57

category classification task, respectively. For two-category classifications, the overall

classification performance of the TcfCNN is better than other four networks. The

maximum accuracy, sensitivity, and specificity are 85.6%, 87.8%, and 81.4%,

respectively. For the three-category classification, the TcfCNN achieves the best

classification performance. The maximum accuracy, sensitivity, and specificity are

68.6%, 68.7%, and 81.9%, respectively. This result shows that the TcfCNN, which

merges the synchronous pattern of multiple EEG channels and the EEG pattern of single

channel, is more suitable to discriminate depressives than other kinds of CNN

architectures.

Figure 4.6 The comparison result of two-category and three-category classification using 10-fold cross validation


58

4.3.2.2 Classification Results of Threshold Based LOPOCV

This section only gives the classification results of TcfCNN model based on the

threshold based LOPOCV. Tables 4.2 and 4.3 show the two-category and three-category

classification using the voting based LOPOCV with different thresholds, respectively.

From the Table 4.2, we can observe that with the increase of the threshold value, the

three classification indexes decline, which is congruent with the normal sense. The

TcfCNN achieves the maximum accuracy of 85.7%, the maximum sensitivity of 82.6%,

and the maximum specificity of 91.6% when the threshold equals 0.5. That is, for

82.6% depressives, the ratio of their own data samples classified correctly is above

50%; for 91.6% normal controls, the ratio of their own data samples classified correctly

is above 50%. This is a promising result to demonstrate that the TcfCNN provides a

trustworthy result for discriminating normal controls. From the Table 4.3, although the

TcfCNN does not achieve such good results as the two-category classification, it still

achieves the maximum accuracy of 65.7%, the maximum sensitivity of 66.6%, and the

maximum specificity of 83.3% when the threshold equals 0.5. This result also supports

that the TcfCNN provides a trustworthy result for discriminating normal controls in

three-category classification.

Table 4.2 Two-category classification using the voting based LOPOCV with different

thresholds

Thred ACC SEN SPE


59

avg±std max avg±std max avg±std max0.5 0.7±0.07 0.85 0.69±0.08 0.82 0.70±0.11 0.910.6 0.59±0.07 0.74 0.61±0.08 0.73 0.55±0.10 0.750.7 0.39±0.05 0.54 0.45±0.07 0.60 0.29±0.10 0.410.8 0.28±0.05 0.42 0.34±0.06 0.43 0.17±0.09 0.410.9 0.17±0.04 0.31 0.23±0.05 0.30 0.07±0.07 0.38

Table 4.3 Three-category classification using the voting based LOPOCV with different thresholds

Thred ACC SEN SPEavg±std max avg±std max avg±std max

0.5 0.54±0.05 0.65 0.55±0.04 0.66 0.77±0.02 0.830.6 0.47±0.04 0.60 0.49±0.04 0.61 0.54±0.02 0.600.7 0.40±0.03 0.51 0.41±0.03 0.52 0.40±0.01 0.460.8 0.36±0.02 0.45 0.38±0.02 0.47 0.39±0.01 0.430.9 0.33±0.02 0.40 0.35±0.01 0.41 0.37±0.01 0.40

4.3.3 Analyze the Feature of the Convolutional Layer

Because of the TcfCNN is consisted of two sub-models, we defined the sub-model

learning the synchronous EEG pattern as the SynCNN part, and the sub-model learning

the single EEG pattern as the SinCNN part. The FFT method is adopted to reveal

specific frequency characteristic of each row of the feature matrix generated by the

convolutional layer of the SynCNN and SinCNN part, respectively. The artificially

computed matrixes reflect input patterns with the largest activation of the net, the

frequency property of each row of the matrix inherits from or even sharing similar

frequency properties of original EEG data. Figures 4.4 and 4.5 show the FFT analysis

result of the feature learned by the 2nd, 4th, 8th convolutional layers of the SynCNN part


60

and SinCNN part in two-category classification, respectively. We find that the peak

position of the FFT amplitude of each row is moving back with the layer deeper into the

network. After that, we analyze the features learned by the convolutional layers of the

two parts in three-category classification. The same result is got from Figures 4.6 and

4.7 which show the FFT analysis result of the features learned by the 2nd, 4th, 8th

convolutional layers of the SynCNN part and SinCNN part in three-category

classification, respectively. Comparing with the 2nd and 4th convolutional layers, the 8th

convolutional layer plays a more important role for discriminating depression. For all

rows of the feature learned by the 8th convolutional layer, the signal component with the

frequency around 10 Hz, which belongs to the frequency range of alpha rhythm, is the

primary frequency component. This indicates that the alpha rhythm of EEG signal plays

a primary role in discriminating depressive and normal control.

Figure 4.7 The FFT result of the features output by the 2nd, 4th, 8th convolutional layers


61

of SynCNN part in two-category classification

Figure 4.8 The FFT result of the features output by the 2nd, 4th, 8th convolutional layers of SinCNN part in two-category classification

Figure 4.9 The FFT result of the features output by the 2nd, 4th, 8th convolutional


62

layers of SynCNN part in three-category classification

Figure 4.10 The FFT result of the features output by the 2nd, 4th, 8th convolutional layers of SinCNN part in three-category classification

4.3.4 Analyze the Feature of the Softmax Layer

The softmax layer in the TcfCNN contains two neurons (depressive and normal

control) or three neurons (medicated, unmedicated, and normal control). As mentioned

above, the artificially generated feature is iteratively updated by the weight between

layers and most likely would produce the desired output. Advancing deeper into the

network, the feature is more likely to represent archetypal features of the desired output.

In this regard, we can hypothesize that the significant difference could be found

between the features learned by different neurons of softmax layer. The feature matrix

learned by the softmax layer is a composite result which contains the information of the


63

synchronous and single EEG pattern. In order to facilitate explaining the physical

meaning of the feature learned by the TcfCNN, the each row of the feature matrix is

deemed as the EEG feature extracted from the corresponding channel. That is, the

artificially generated matrix updated by the softmax layer reveals specific frequency

features in the time domain when computing spectral analysis on each row, representing

EEG-channel data.

Figures 4.8 and 4.9 display the FFT result of the feature learned by the softmax

layer of the TcfCNN in two-category classification and three-category classification,

respectively. From the Figure 4.8, we find that for the channel located in the anterior

region of brain (Fp1, Fp2, F3, F4), the FFT amplitude of the signal component with the

frequency around 10 Hz and 50 Hz of normal control is higher than depressives. For the

channel located in the posterior region of brain (P3, P4), the FFT amplitude of the signal

component with the frequency around 10 Hz and 50 Hz of depressives is higher than

normal controls. From the Figure 4.9, we can get the same result that the FFT amplitude

of the signal component with the frequency around 10 Hz and 50 Hz of normal control

is higher than depressives in the anterior region of brain (Fp1, Fp2, F3, F4), and

conversely in the posterior region of brain (P3, P4). This result indicates that the

softmax layer of the TcfCNN focus on spatial patterns of EEG frequencies in different

brain regions, which shows depression-specific spatial differences in brain rhythms. The

spatial distribution of the two kinds of signal frequency (around 10 Hz and 50 Hz)

might be effective for depression discrimination.


64

Figure 4.11 The FFT result of the feature output by the softmax layer of TcfCNN model in two-category classification

Figure 4.12 The FFT result of the feature output by the softmax layer of TcfCNN model in three-category classification


65

4.4 Discussion

This study aims at using the CNN model to executing the two-category (depressive

and normal control) and three-category (medicated, unmedicated depressive, and

normal control) classification tasks. The works could be divided into several parts: input

representation of the CNN model, feature extraction for learning EEG pattern, model

training and testing by 10-fold cross validation, analyze the feature learned by

convolution layer and softmax layer using the FFT, and participant-independent

individual classification results obtained by the LOPOCV method. Inspired by the

image data representation, we represent the EEG as a time series of topographically

organized images. The convolution operations are designed to extract the single EEG

pattern and synchronous EEG pattern from the original multi-channel EEG data, the two

kinds of EEG patterns are demonstrated to be effective for depression discrimination.

Adopting a technology similar with “deep-dreaming” algorithm, we artificially generate

matrixes to reveal the EEG feature learned by the convolution layer and softmax layer.

The conclusion is that the spatial distribution of the two kinds of EEG frequency

component (around 10 Hz and 50 Hz) might contain depression-specific spatial

differences in brain rhythms. A voting-based leave-one-participant-out procedure is

utilized to test the participant-independent individual classification performance of our

CNN model and shows that the model provides a trustworthy result for discriminating

depressives and normal controls.

Deep learning for analysis of EEG patterns has been applied to other studies with


66

resting state EEG. For instance, a deep net learned discriminative features between

imagining music [54] or listening to music or rhythm perception. Differentiation

between early stage Creutzfeldt-Jakob disease and other forms of rapid progressive

dementias with deep learning achieved a sensitivity of 92% at specificity of 89% [55].

Classification of sleep stages using deep nets performed on par with human sleep

experts [56]. Detection of interictal epileptiform discharges is another promising

application for deep nets, and may soon become a standard clinical tool to assist in the

diagnostic process in epilepsy [57, 58]. Acharya et al. presented the first application of

the deep neural network concept and CNN for depression diagnosis [59]. The model

was tested using EEGs obtained from 15 normal and 15 depressed patients. The

algorithm attained accuracies of 93.5% and 96.0% using EEG signals from the left and

right hemisphere, respectively. Those studies demonstrate that the deep learning method

may become a successful diagnostic tool for brain-related illness, but they did not try to

explore the difference between the features for discriminating different categories based

on deep learning method.

This study is the first application to utilize the CNN model to differentiate normal

controls from depressives, and even from medicated and unmedicated depressives,

further to analyze and compare the difference between the features learned by the

convolution layer or softmax layer using the technology similar with “deep-dreaming”

algorithm. The reasons of using the CNN model to discriminate depression including

two aspects. On one hand, the CNN model does not require the semi-manually-selected

extraction and selection of features for classification. Rather, the model can self-learn

the feature from the raw EEG data and pick up distinct features during the training of


67

the algorithm. On the other hand, our final goal is to find out the effective biomarker for

the MDD discrimination, the classifiers with good classification performance facilitate

obtaining the features which possess the high discrimination ability.

In the beginning, we should consider the input representation of the multi-channel

EEG data for the CNN model based classification method. One possibility is to

represent the EEG as a time series of topographically organized images. In this view, we

represent the input data of first layer as a 2D-array with the 6 channels as the height (y-

axis) and the 3072 data points as the width (x-axis). EEG signals are assumed to

approximate a linear superposition of spatially global voltage patterns caused by

multiple dipolar current sources in the brain. Unmixing of these global patterns using a

number of spatial filters is therefore typically applied to the whole set of relevant

electrodes as a basic step in many successful examples of EEG decoding. The

synchronous EEG pattern is learned by this kind of method and focuses on the spatial

characteristics of brain activity. In addition, the EEG pattern reflected on a single scalp

position can be learned by a number of filters which merely operate the data of single

EEG channel. In this method, two kinds of EEG patterns (single EEG pattern and

synchronous EEG pattern) are learned.

Besides the CNN models for learning single EEG pattern and synchronous EEG

pattern are constructed, we also resurrect the CNN model architecture implemented in

other studies with the background of EEG based depression discrimination. Five

network architectures are used for comparison. The result shows that our CNN models

achieve the best performance on two-category and three-category classification. This

result not only indicates the single EEG pattern and synchronous EEG pattern learned


68

by the CNN model are effective for discriminating depressives, but also gives a new

evidence to support the EEG synchronous pattern of whole brain is significant for

exploring the trait marker for depression.

Adopting a technology similar with “deep-dreaming” algorithm, we artificially

generate matrixes to reveal the EEG feature learned by the convolution layer and

softmax layer. The FFT is used to analyze the specific frequency features in the time

domain when computing spectral analysis on each row of the matrix. The FFT analysis

result of the feature output by the 2nd, 4th, 8th convolution layer in the sub-models of

TcfCNN demonstrate that the signal component with low frequency (around 10 Hz),

which is overlapped by the theta or alpha component of the signal, plays a primary role

in discriminating depression. In addition, the FFT result of the feature output by the

softmax layer in the TcfCNN shows that in the anterior region of brain (Fp1, Fp2, F3,

F4), the FFT amplitude of the signal component with the frequency around 10 Hz and

50 Hz of normal control is higher than depressives, and in the posterior region of brain

(P3, P4), the FFT amplitude of the signal component with the frequency around 10 Hz

and 50 Hz of depressives is higher than normal controls. This indicates that the softmax

layer restricted its focus on spatial patterns of EEG rhythms in different brain regions,

which shows the spatial distribution of the two kinds of EEG frequency component

(around 10 Hz and 50 Hz) might contain depression-specific spatial differences in brain

rhythms.

Our results demonstrate that the EEG frequency component (around 10 Hz), which

belongs to the alpha rhythm, is effective for depression discrimination. Actually there

are many studies investigated the relation between alpha rhythm and depression [60-62].


69

For example, the frontal alpha asymmetry has been investigated multiple times to

validate the effectiveness in depression diagnosis. The increased alpha power is to date

considered a hallmark of depression, and there is strong evidence that people with

depression have impaired alpha oscillations. The discovery from the CNN model that

information in the alpha-range differs between the depressives and normal controls

gives a new evidence to support the alpha rhythm might possess a crucial position in

discriminating depression. In addition, there are also many studies investigated the

relation between gamma rhythm (≥ 32 Hz) and depression. Paul et al. summarize some

key findings on gamma rhythms (especially their amplitude) as a biomarker for major

depression, they suggest that gamma rhythms may provide objective information on

major depressive disease status. Li et al. studied abnormally increased connectivity of

brain functional networks in patients with depression, and found that the global EEG

coherence of patients with depression was significantly higher than that of healthy

controls in the gamma bands.

In order to ensure a good classification performance could be obtained by our CNN

model after implementing it in the clinical application, the leave one participant out

cross validation is used to train the CNN model and validates the classification

performance. We also give the classification results based on the LOPOCV method

using different threshold values. This work is worthy to be done because of we are more

concerned with how many participants are classified correctly for a disease screening

application in the real world. We also would like to obtain a good classification

performance based on the LOPOCV method using a high threshold. Actually the

threshold based LOPOCV method is equivalent to a voting-based leave-one-participant-


70

out procedure, they are all adopted the majority voting strategy to test the participant-

independent individual classification performance. It is easy to understand that with a

higher threshold, the higher classification performance the model achieves the better

generalization ability the model has. For the two-category classification, the TcfCNN

achieves the maximum accuracy of 85.7%, the maximum sensitivity of 82.6%, and the

maximum specificity of 91.6% when the threshold equals 0.5. For the three-category

classification, the TcfCNN achieves the maximum accuracy of 65.7%, the maximum

sensitivity of 66.6%, and the maximum specificity of 83.3% when the threshold equals

0.5. This is a promising result to demonstrate that the TcfCNN provides a trustworthy

result for discriminating normal controls.


5. Single Channel EEG Based Objective

Depression Prescreening

5.1 Introduction

To provide adjunctive depression discrimination, a prescreening tool based on

physiological data for automated classifying MDD patients and normal controls is

significant. The pervasive and persistent nature of depressive symptoms has made

functional magnetic resonance imaging (fMRI) and scalp-recorded

electroencephalogram (EEG) as appropriate approaches for understanding the

underlying mechanisms of major depressive disorder [63-65]. Quantitative

measurement of brain electrical signals taken from the scalp-recorded EEG is a



low cost [66]. Besides, the machine learning methods facilitate researchers to construct

the EEG based system for automatically providing the MDD discrimination. Regard

this, many studies proposed various EEG data based methods for depression

discrimination in recent years [39-41]. Generally, the researchers always extracted the

EEG features from raw signal to represent biological markers using linear domain

analysis, frequency domain analysis, wavelet analysis and nonlinear method. For

instance, power spectral measures of resting state EEG oscillatory activity in different

frequency bands have been shown to distinguish between depressives and normal

controls [39]. Bairy et al. extract features (skewness, energy, kurtosis, standard

deviation, mean and entropy) at

73

various detailed coefficients levels of discrete wavelet transform (DWT) for

automatically classifying normal and depressive EEG signals, the classification accurate

result of 88.92% is obtained by Support Vector Machine (SVM) classifier [46].

Hosseinifard et al. adopts nonlinear features extracted from five EEG frequency bands

for discriminating MDD patients and normal controls, a classification accuracy of 90%

is achieved by Linear Regression (LR) classifier [47]. Mohammadi et al. uses a data

mining methodology for classifying EEGs of 53 MDD patients and 43 normal controls,

and shows average classification accuracy of 80% [66]. The studies mentioned above

suggest that the automated EEG analytical approach by machine learning algorithm is

feasible to discriminate the MDD patients and normal controls.

However, the studies mentioned above relate to the intensive works launched under

the lab or hospital environment, there are still many difficulties for applying the EEG

based MDD discrimination in the dairy life. For example, the EEG acquisition systems

involving wet electrodes are time-consuming and uncomfortable for subjects. Reversely,

dehydration of the gel affects the quality of the acquired data and reliability of long-

term monitoring. Regard this, many researchers choose to use dry electrodes for

acquiring the EEG signal. Studies show that the advances in EEG recording and

analysis ensure a promising future in support of personal healthcare solutions [67-69].

The most dramatic example in developing EEG based real-life applications by dry

electrodes is made by Hu et al. [69], who develop a pervasive prefrontal-lobe three

electrode EEG system to collect the data from Fp1, Fp2, and Fpz locations. A

psychophysiological database is constructed, which containing 213 (92 depressed

patients and 121 normal controls) subjects. The result demonstrates that the highest


74

classification accuracy of 79.27% is got by K-Nearest Neighbor (KNN) algorithm.

Moreover, for actualizing the EEG based MDD prescreening in the diary life, the

ideal measurement is to use the EEG electrode as less as possible, and the electrode is

not covered by hair. Regard this, the forehead scalp is a superb location for collecting

the EEG data. Studies show that the forehead region contains rich information that

originates from the prefrontal cortex and the frontoparietal cortical network, which are

associated with cognitive abilities and dysfunctions [70, 71]. Given the fact of that,

many brain-computer interfaces (BCI) products prefer to choose forehead region for

achieving EEG signals. One such product is the ThinkGear system produced by the

NeuroSky company. The ThinkGear is a dry electrode device which consists of

proprietary firmware within a single channel which claims ease of use and portability in

a research-grade apparatus [72]. An initial validity study reported the EEG data derived

from a ThinkGear system is comparable to EEG recorded from a conventional multi-

electrode and lab-based system. The EEG data is sensitive to standard variations in

resting and active mental processing states [73]. Rogers et al. evaluated the reliability of

this portable system using test-retest and reliable change analyses, and their findings

encourage application of the portable EEG system for the study of brain function [67].

Therefore, the portable EEG device with single channel is trustworthy to measure

frontal brain activity and be further used to study the MDD discrimination.

The aim of this study is to develop a ubiquitous prescreening method for

discriminating depressives and normal controls based on the EEG data collected from

single forehead channel. The EEG dataset collected from the Fp1 and Fp2 location (two

surface sites placed on forehead scalp) by a 32-channel EEG system is firstly used to


75

demonstrate the MDD discrimination performance of the forehead single channel. The

MDD discrimination performance of Fp1 and Fp2 channel are compared for illustrating

which one is optimal for classifying the MDDs and normal controls. For each single

channel, the discrete wavelet transform is used to denoise the raw EEG and decompose

it into five subband components (delta, theta, alpha, beta and gamma). A feature vector

with 263-dimensions is extracted from the denoised EEG and the 5 rhythms by using

linear, wavelet and nonlinear analysis methods. The feature vector is employed for

training classifiers including K-Nearest Neighbor (KNN), Random Forest (RF), Linear

Discriminant Analysis (LDA), and Classification and Regression Tree (CART). The

Genetic Algorithm (GA) is used to select effective feature subset. Furthermore, a

portable EEG device produced by NeuroSky is utilized to collect the second EEG

dataset from the Fp1 location. The same data processing procedure is used to process

the data.

In order to estimate the classification performance of the classifiers, three

evaluation indexes including accuracy, sensitivity and specificity are calculated. The

MDD discrimination performance based on different EEG channels, feature sets and

classification methods are compared by the Friedman and multiple comparison test,

respectively. The methods published in other literatures are also compared with the

method used in this study.


76


5.2.1 Technical Route

Figure 5.1 shows the flowchart illustrating the data processing procedure for

classifying MDD patients and NCs. The procedure is divided into two phrases: offline

and online. Offline phrase aims at training classification model based on the training

data. Online phrase aims at classifying and labeling the testing data sample into MDD

or NC.

The offline phrase includes three data processing steps: data preprocessing, feature

extraction, feature selection and classifier training. For the data preprocessing step, the

raw EEG signal of every subject is partitioned into several fragments, and the fragments

are employed as the training data samples to train the classifiers. An adaptive filter is

adopted to remove 50 Hz line noise from the training data samples. And a wavelet filter

is utilized to remove the high frequency (>128 Hz) and low frequency (<1 Hz) from the

training data samples. After that, an eight-level discrete wavelet transform is used to

decompose the EEG sample into five subband components. For the feature extraction

step, time domain, frequency domain, wavelet and nonlinear analysis based feature

extraction methods are utilized to extract features from the EEG sample and the five

corresponding subband components. For the feature selection and classifier training

step, the GA algorithm encapsulated with the basic classifiers (KNN, RF, LDA, CART)

are employed to select optimal feature set and form the final classification model.

Congruent with the offline phrase, the same data processing steps including data


77

preprocessing, feature extraction and selection are executed in the online phrase to

process the testing samples. According to the optimal feature set selected in the offline

phrase, the feature subset is selected from 263-dimension vector and input into the

corresponding classifier for identifying the EEG testing sample.

Figure 5.13 Flowchart illustrating the data processing procedure for classifying MDD patients and NCs

5.2.2 Data Acquisition and Preprocessing

5.2.2.1 Data Acquisition

Two experiments are conducted not only for validating the single-channel EEG

data provides the MDD discrimination at the level of multi-channel EEG, but also for

giving a preliminary investigation to develop a ubiquitous MDD prescreening

application based on the portable EEG device. For experiment 1, all subjects are asked

to record their EEG data in the resting state. Specifically, the subjects are sitting on a

sofa and keeping with eye closed for 8 mins while not to think anything purposefully in


78

a dim illuminated and acoustical room. They are also asked to maintain a minimum

arousal level without falling into sleep. 16 normal controls (8 females and 8 males),

ranged in age from 21 to 55 years (26.4±9.8 years; mean±standard deviation (Std.)), and

23 depressive patients (12 females and 11 males), ranged in age from 20 to 56 years

(29.3±9.6 years; mean±Std.), are recruited. EEG data is recorded from 32 surface

electrodes placed on the head scalp according to standard international 10/20 system.

The EEG collection device is produced by Brain Products (BP) company of Germany

[74], which data sampling rate is 500 Hz and equipped with BrainAmp 16-bit A/D

convertor (ADC) amplifier. After finishing the multi-channel EEG data collection, the

data recorded from two forehead channels (Fp1 and Fp2) is extracted for data analysis.

For experiment 2, the experiment paradigm is as the same as the experiment 1. 15

normal controls (4 females and 11 males), ranged in age from 24 to 55 years (33.6±11.5

years; mean±Std.), and 15 depressive patients (6 females and 9 males), ranged in age

from 26 to 52 years (34±9.2 years; mean±Std.), are recruited. The EEG data is recorded

from Fp1 location by using NeuroSky MindWave device [32], which sampling

frequency is 512 Hz and equipped with 12-bit ADC precision.

5.2.2.2 Data Preprocessing

Considering 8 minutes is too long for patients to keep the resting state, raw EEG

signal recorded in one trial is cut into three snippets, and the median snippet which time

length of 5 minutes is kept for further data analysis. The time snippets of the beginning

30 seconds and the last approximate 2 minutes 30 seconds are removed. The Z-score


79

normalization is adopted to overcome the amplitude scaling problem and remove the

offset effect. After that, the remaining EEG signal of every subject is partitioned into

146 data samples with 50% overlap. Every data sample contains 2048 sampling points

(4.096 seconds). Finally, the dataset used in this work includes 5694 data samples (2336

normal controls and 3358 depressives). A custom Matlab R2014a (Waltham, MA, USA)

software is used for offline data analysis.

To remove 50-Hz noise originating from the ubiquitous environment, every sample

is passed through an adaptive filter with a 50-Hz reference signal. Furthermore, a data

denoising method based on the DWT is used to filter the segment. Due to the frequency

of electromyograph (EMG) is concentrated primarily in the high frequency band (> 100

Hz), the frequency of electrocardiogram (ECG) and electrooculogram (EOG) are

concentrated primarily in the low frequency band (< 1 Hz), a soft thresholding

algorithm with `db5` wavelet base is utilized to remove the high frequency component

(> 100Hz) and low frequency component (< 1 Hz) from data samples. The formula of

soft-thresholding wavelet transform is defined as follow.

f (i )={c ( i )−τ∗e1−( c (i )

τ ), c (i )>τ

0 , else

c ( i )+τ∗e1−( c (i)

τ ), c ( i )<τ

(5-1)

where τ is threshold and c(i) is the wavelet coefficients extracted from the DWT of the

the EEG sample.

Besides the data denoising, the DWT method is also used in data decomposition.

An eight-level DWT with `db5` wavelet base is used to decompose the denoised EEG


80

sample into five subband components. Specifically, after the first level decomposition,

the denoised EEG sample is decomposed into its higher resolution component d1 (250-

500 Hz) and lower resolution a1 (0-250 Hz). The a1 component is further decomposed

into its higher resolution component d2 (125-250 Hz) and lower resolution component

a2 (0-125 Hz). After eight-level decomposition, the denoised EEG sample is

decomposed into five frequency subbands that approximate to delta, theta, alpha, beta,

and gamma subbands: delta (2-4 Hz), theta (4-8 Hz), alpha (8-15 Hz), beta (16-31 Hz)

and gamma (31-62 Hz). The signal wave and the corresponding FFT of original EEG

sample, denoised EEG sample and five subbands are depicted in Figure 5.2.

Figure 5.14 The signal wave and the corresponding FFT transform of original EEG signal, denoised EEG signal and five subbands


81

5.2.3 Feature Extraction

5.2.3.1 Time-Domain Feature Extraction

Mean and standard deviation (SD) have been used in the previous studies to

characterize windows of the EEG signal data. The other statistic features including

maximum, minimum, median and 25th and 75th percentile are also common used.

Kurtosis and skewness are calculated to characterize the location and variability of EEG

segments. These time domain parameters are usually used to analyze the EEG signals

for extracting the time domain feature of the signal.

5.2.3.2 Frequency-Domain Feature Extraction

For each denoised EEG sample, the Welch method is applied to estimate the power

spectrum. According to the frequency vector outputted by Pwelch method in the Matlab,

the power spectral density of five subbands are accumulated according to the

corresponding frequency range, the summation of each subband is calculated to

represent their own relative power. In addition, the power spectral density between 0-

100 Hz is accumulated to represent the entire power of each denoised EEG segment.

The power ratio of beta band and theta band is extracted as a feature since it is claimed

to be an effective feature in the previous study. It is noteworthy that because of the five

subbands extracted by the wavelet decomposition method have their own principal

frequency respectively, the Welch method is not further performed on power spectral

estimation of the five subbands.


82

5.2.3.3 Wavelet Feature Extraction

The wavelet analysis method is only applied to extract wavelet features from

denoised EEG samples based on an eight-level DWT method with `db5` wavelet base.

Each EEG sample is decomposed into eight levels, the sum and the absolute sum of the

squared detail coefficients at level 3, 4, 5, 6, 7 which represent the signal power of the

five subbands are calculated. In congruent with the time domain feature extraction, the

time domain features (mean, SD, maximum, minimum, median and 25th and 75th

percentile, kurtosis and skewness) of the detail coefficients of five subbands are also

extracted. Additionally, the ratio between the square summation of the detail coefficients

of subbands and the square summation of the detail coefficients of the EEG sample is

calculated to represent the power energy distribution of the five subbands.

5.2.3.4 Nonlinear Feature Extraction

Because of the EEG signals are random and nonstationary, many studies focus on

using nonlinear dynamics features to characterize the nonlinearity particularities. The

nonlinear features extracted in this study including power spectral entropy, C0

complexity, approximate entropy and wavelet entropy.

(1) Power spectral entropy. The power spectral entropy is used as a physical

indicator to estimate the quality and intensity of brain activity, which indicates that the

larger indicator the more active the brain. The FFT is performed on time-series signal to


83

obtain the power spectrum, and the information entropy of power spectrum is called as

power spectral entropy. Suppose the EEG signal to be analyzed is {x(n),n=0,1,2…,N-1}

with a length of N samples, the calculation method is summarized as follows:

Step 1: Calculate the FFT of the signal by formula (5-2):

X (k )= 1N ∑

n=0

N −1

x (n ) e− j ( 2 kπn/N ) , k=0,1 ,…, N−1 (5-2)

Step 2: Calculate the power spectral density (PSD) of X(k) by formula (5-3):

P(k)= 1N |X (k)2| (5-3)

Step 3: Normalize P(k) and obtain power spectral density distribution function by

formula (4):

p (k )= P(k)

∑k=0

N−1

P(k ). (5-4)

Step 4: Calculate the power spectral entropy by formula (5-5):

H=−∑k=0

N −1

p (k ) ln ( p(k)). (5-5)

(2) C0 complexity. C0 complexity measures the percentage of the stochastic

components of a time-series signal. It assumes that a signal can be divided into regular

part and stochastic part. The calculation method of C0 complexity is depicted as follow:

Step 1: Calculate the FFT of the signal by formula (5-2).

Step 2: Calculate the power spectral density (PSD) of X(k) by formula (5-6):

M= 1N ∑

k=0

N−1

|X (k )2|. (5-6)

Step 3: If the square of X(k) less than M then the original X(k) is replaced by 0 to

get a new spectrum series Y(k):


84

Y (k )={X ( k )|X (k )|2>M0|X (k )|2<M

. (5-7)

Step 4: Calculate the inverse FFT (IFFT) of Y(k) by formula (5-8):

y (n )=∑n=0

N−1

Y (k )e j (2kπn/N ), n=0,1,…,N-1 (5-8)

Step 5: Subtract the power of stochastic part, and estimate the C0 complexity by

formula (5-9):

C 0=∑n=0

N −1

|x (n )− y (n)|2

∑n=0

N−1

|x (n)|2. (5-9)

(3) Approximate entropy. Approximate entropy (ApEn) is a recently developed

statistic quantifying regularity and complexity, which reflects the likelihood that similar

patterns of observations will not be followed by additional similar observations. A time

series containing many repetitive patterns has a relatively small ApEn, a less predictable

process has a higher ApEn. It appears to have potential application to a wide variety of

relatively short (greater than 100 sampling points) and noisy time-series data. The

calculation formulas of ApEn are depicted as follows:

Step 1: Give an integer m and a positive real number r. The value of m represents

the length of compared run of signal, and r specifies a filtering level.

Step 2: Form a sequence of vectors x(1), x(2), …, x(N-m+1), and define an m-

dimensional space by formula (5-10):

x (i )=[u ( i ) , u (i+1 ) , …,u(i+m−1)]. (5-10)

Step 3: a correlation integral is calculated by formula (5-11) using Heaviside


85

function which is defined asθ ( x )=0 for x<0 and θ ( x )=1 for x>0:

C im (r )=

∑i ≠ j

θ(r−|x (i )−x ( j)|)

N−m+1. (5-11)

Step 4: Define

βm (r )=∑i=1

N −m+1

log (C im (r ))

N−m+1. (5-12)

Step 5: Define ApEn as:

ApEn=βm (r )−βm+1 (r ). (5-13)

(4) Wavelet entropy. Wavelet entropy is an indicator which represents the energy

distribution of a signal, and the signal energy can be represented by wavelet

coefficients. The corresponding wavelet coefficient of five subbands (delta, theta, alpha,

beta and gamma) is extracted by the DWT and employed as basic parameters to obtain

wavelet entropy. The wavelet entropy is calculated as follow:

Step 1: Extract the wavelet coefficients D(k) of signal by DWT, and accumulate

the quadratic summation of D(k) by formula (5-14):

E=∑k=0

N −1

|D(k )|2 (5-14)

Step 2: Extract the corresponding wavelet coefficients Dj(k)of five subbands,

where j means the jth level decomposed by DWT. Accumulate the quadratic summation

of Dj(k) by formula (5-15), where L is the point number of Dj(k):

E j=∑k=0

L

|D j(k)|2 (5-15)

Step 3: Give the relative wavelet energy by formula (5-16):


86

p j=E j

E. (5-16)

Step 4: Calculate the wavelet entropy by formula (5-5).

5.2.4 Feature Selection and Classification Methods

5.2.4.1 Genetic Algorithm for Feature Selection

A genetic algorithm (GA) is applied for selecting optimum feature subsets that

maximizes classification accuracies of classifiers. Before starting the GA, the

parameters including the generation number, the population size and the number of

genes contained in each individual are initialized. Each gene indicates the single feature,

and the individual is consisted of a bunch of binary number (0 or 1). The value of 1

implied that the corresponding feature is selected by the algorithm and would be

included in the feature subset. Otherwise the feature is excluded from the feature subset.

In the first generation, the binary number formed genes are generated randomly. The

crossover rate and the mutation rate of ensuing generation are set to be a fixed decimal

fraction, respectively. The fitness function is applied to evaluate the superiority of the

individuals. The training datasets are used to train the classifiers, and the testing datasets

are employed to obtain the classification accuracy. The value of fitness function is set to

be equal with the classification accuracy, which gives an intuitive view to know the

highest classification accuracy belongs to the optimum feature subset.


87

5.2.4.2 K-Nearest Neighbor

K-Nearest Neighbor (KNN) is a non-parametric algorithm. It aims at using a

database in which the data points are separated into several classes to predict the

classification of a new sample point. The non-parametric character of the KNN means

that it does not make any assumptions on the underlying data distribution. The structure

of the classification model is determined from the data because of the most of the data

does not obey the typical theoretical assumptions made (as in linear regression models,

for example) in the real world. Therefore, the KNN is one of the first choices for a

classification study when there is little or no prior knowledge about the distribution

data. The KNN is also a lazy algorithm means that it does not use the training data

points to do any generalization. There is no explicit training phase or it keeps all (or

most) the training data to classify the object during the testing phase. The core idea of

the algorithm is based on feature similarity which is voted by comparing the space

distance (Euclidean Distance, for instance) between training data and testing data. An

object is classified by a majority vote of its neighbors, with the object being assigned to

the class most common among its k nearest neighbors.

5.2.4.3 Random Forest

Random forest algorithm is an ensemble learning method for classification or other

tasks such as regression. The algorithm constructs a multitude of decision trees at

training time and outputting the class that is the mode of the individual trees. The


88

training algorithm for random forests applies the general technique of bootstrap

aggregating, or bagging, to tree learners. Take the classification task as example, given a

training set X = x(1), ... , x(n)with labels Y = y(1), ..., y(n), bagging repeatedly (M

times) selects a random sample with replacement of the training set and fits trees to

these samples:

For m = 1, ..., M:

Sample, with replacement, n training examples from X, Y; call these X(m), Y(m).

Train a classification or regression tree f(m) on X(m), Y(m).

After training, predictions for unseen samples x' are made by taking the majority

vote in the case of classification trees. This bootstrapping procedure leads to better

model performance because it decreases the variance of the model, without increasing

the bias. This means that while the predictions of a single tree are highly sensitive to

noise in its training set, the average of many trees is not, as long as the trees are not

correlated. Simply training many trees on a single training set would give strongly

correlated trees (or even the same tree many times, if the training algorithm is

deterministic); bootstrap sampling is a way of de-correlating the trees by showing them

different training sets. Random decision forests correct for the overfitting of decision

trees to their training set.

5.2.4.4 Linear Discriminant Analysis

The basic concept of Linear discriminate analysis (LDA) is that searching for a

linear combination of variables that best separates two classes. The LDA function for


89

two classess (c1,c2) is defined as:

g ( x )=w t ( x )+w0, (5-17)

where x is the input feature vector, w is the linear model coefficients and w(0) is

threshold value. The problem is to estimate the linear model coefficients that maximize

the ratio of between-class variance to within-class variance, and guarantee maximal

separability of two classes. After optimizing the parameters, we classify the input

instance as class c1 if g(x)> 0, otherwise x is classified as class c2.

5.2.4.5 Classification and Regression Tree

Classification and Regression Trees (CART) is an algorithm that can be used for

classification modeling problems. This algorithm is referred to as `decision trees` which

indicates the representation for the CART model is a binary tree. Each root node

represents a single input variable (x) and a split point on that variable. The leaf nodes of

the tree contain an output variable (y) which is used to make a classification. Before

making a classification, a CART model should be created which involves selecting

input variables and split points on those variables until a suitable tree is constructed.

The selection of which input variable to use and the specific split or cut-point is chosen

using a greedy algorithm, also known as recursive binary splitting, to minimize a cost

function. The Gini index function is used which provides an indication of how ``pure``

the leaf nodes are.

G=∑i=1

k

p i∗(1− pi), (5-18)

where G is the Gini index over all classes, pi are the proportion of training instances


90

with class k in the rectangle of interest. Tree construction ends using a predefined

stopping criterion, such as a minimum number of training instances assigned to each

leaf node of the tree.

5.2.5 Classifier Training and Sample Classification

The threshold based LOPOCV method is utilized to train the classifiers and

classify the EEG data samples. For every validation, the classifier is trained for 100

generations, and every generation includes 100 individuals. Considering the dataset size

of depressive is greater than normal control, a resampling strategy without replacement

is adopted to randomly select the depressive and normal control data records from the

entire dataset (exclude the data samples which are already selected as the testing data)

as training data. The strategy aims at keeping the size of the two kinds of data samples

congruous, and avoiding the model performance decrease caused by the sample

disequilibrium.

The detail steps and the corresponding illustration of the LOPOCV are described

as follows:

Step 1: m EEG samples of the ith subject are used as the testing data, and the

m×(N-1) EEG samples of the remaining (N-1) subjects are used as the training data. N

indicates the number of subject. For every validation, m EEG samples of the ith subject

are used as the testing data, and the m×(N-1) EEG samples of the remaining (N-1)

subjects are used as the training data.

Step 2: A 263-dimension feature vector is extracted from every EEG sample using


91

the feature extraction methods.

Step 3: The GA algorithm encapsulated with basic classifiers is utilized to select

effective feature subset and build the final classification model.

Step 4: Feed the m testing data into the final classification model and count the

number n of the samples which are correctly classified and calculate the ratio by n/m.

Step 5: Repeat the steps mentioned above until the EEG sample of every subject is

chosen as the test data. The ith subject is correctly classified if the ratio between n and

m is greater than 0.5.

Step 6: Count the number of the subjects which are correctly classified, and

calculate the classification accuracy by (the number of correctly classified subjects)/N.

5.3 Results

5.3.1 Classification Result Based on Multi-channel EEG

Device

Tables 5.1-5.5 show the sensitivity, specificity and accuracy of 8 classifiers based

on 5 EEG feature sets of the Fp1 and Fp2 channel. No matter based on which kind of

feature set, the classification performance of the basic classifier combining with GA is

better than the performance of the corresponding basic classifier. Among them, the

KNN classifier combining with the GA algorithm (KNN-GA) obtains the best

classification accuracy of 94.29% based on the selected nonlinear feature set of the Fp1

channel. For the time feature set or the frequency feature set of Fp1 channel, the KNN-


92

GA also achieves the classification accuracy above 85%. Meanwhile, tables 5.1-5.5 also

give the information about the classification effectiveness comparison of Fp1 and Fp2

channel, the result demonstrates that no matter which kind of feature set we using, the

classification result of the Fp1 channel is better than Fp2, which indicates that the EEG

data collected from the Fp1 channel might contain more information for discriminating

the MDD than the EEG data collected from the Fp2 channel. This conclusion might

become an evidence to illustrate why some commercial wearable EEG product for

Brain-Computer Interface choose Fp1 location as the data collecting site. A multi-

category feature approach is also utilized for attempting to obtain better classification

results, especially for helping the basic classifiers which are not encapsulated into the

GA algorithm to improve the classification performance. However, not only the

performances of the basic classifiers are not improved, but also the performances of the

classifiers combining with the GA are declined. Whilst the result shows that the

depression classification results based on fused feature set is inferior to the single

feature domain based results, we are not intending to highlight such fact. Such results

could be caused by the parameter selection of GA, and the GA is not adroit at

processing the dataset with high dimension. This result might provide a suggestion for

the studies combining the GA algorithm to design the classification method that a fused

feature is not the first choice to construct a GA classification model.

Table 5.4 Classification results based on the feature set of time domain

Channel Fp1 Fp2


93

IndexAlgorithm

Sen Spe Acc Sen Spe Acc

KNN 0.50 0.43 0.46 0.42 0.30 0.34KNN-GA 0.67 1.0 0.89 0.83 0.87 0.86

RF 0.42 0.78 0.66 0.08 0.78 0.54RF -GA 0.58 1.0 0.86 0.41 0.91 0.74

LDA 0.42 0.52 0.49 0.25 0.48 0.40LDA-GA 0.75 0.78 0.74 0.67 0.74 0.71

CART 0.42 0.65 0.57 0.25 0.70 0.54CART-GA 0.58 1.0 0.86 0.75 0.87 0.77

Table 5.5 Classification results based on the feature set of frequency domain

Channel Fp1 Fp2Index

AlgorithmSen Spe Acc Sen Spe Acc

KNN 0.42 0.48 0.46 0.50 0.48 0.49KNN-GA 0.67 0.96 0.86 0.67 0.83 0.74

RF 0.33 0.87 0.69 0.0 0.74 0.49RF -GA 0.67 0.91 0.83 0.58 1.0 0.86

LDA 0.42 0.48 0.46 0.25 0.52 0.43LDA-GA 0.75 0.65 0.69 0.58 0.83 0.74

CART 0.58 0.74 0.69 0.08 0.39 0.29CART-GA 0.75 0.91 0.86 0.58 0.87 0.77

Table 5.6 Classification results based on the feature set of wavelet domain



KNN 0.67 0.26 0.40 0.25 0.43 0.37


94

KNN-GA 0.75 0.69 0.68 0.58 0.91 0.77RF 0.16 0.52 0.40 0.16 0.47 0.37

RF -GA 0.75 0.91 0.82 0.25 0.91 0.68LDA 0.41 0.69 0.60 0.33 0.56 0.48

LDA-GA 0.91 0.86 0.88 0.83 0.86 0.85CART 0.50 0.26 0.34 0.66 0.47 0.54

CART-GA 0.75 0.82 0.74 0.50 0.91 0.68

Table 5.7 Classification results based on the feature set of nonlinear domain



KNN 0.83 0.21 0.42 0.75 0.43 0.54KNN-GA 0.91 0.95 0.94 0.83 0.69 0.74

RF 0.50 0.52 0.51 0.50 0.34 0.40RF -GA 0.58 1.0 0.85 0.41 1.0 0.80

LDA 0.66 0.60 0.62 0.25 0.39 0.34LDA-GA 0.91 0.78 0.82 0.91 0.65 0.71

CART 0.66 0.26 0.40 0.66 0.30 0.42CART-GA 0.75 0.86 0.82 0.75 0.82 0.77

Table 5.8 Classification results based on the fused feature set



KNN 0.50 0.52 0.51 0.41 0.39 0.40KNN-GA 0.83 0.47 0.60 0.58 0.65 0.57


95

RF 0.50 0.60 0.57 0.50 0.65 0.60RF -GA 0.50 0.86 0.74 0.58 0.78 0.71

LDA 0.58 0.69 0.65 0.66 0.65 0.65LDA-GA 0.75 0.78 0.77 0.75 0.69 0.71

CART 0.58 0.52 0.54 0.58 0.56 0.57CART-GA 0.58 0.69 0.65 0.50 0.65 0.60

5.3.2 Classification Result Based on Portable EEG Device

In order to do a preliminary investigation on developing a ubiquitous application

for prescreening the MDD, the EEG data collected by portable device is further

analyzed. Tables 5.6 and 5.7 show the sensitivity, specificity and accuracy of 8

classifiers based on 4 EEG feature sets and the fused feature set, respectively. From the

entire classification result, we can know that the classification performance based on the

EEG data collected by the portable EEG device is inferior to the classification

performance based on the EEG data collected by the multi-channel EEG device. Despite

this, the CART-GA method still achieves the best classification accuracy of 86.67%

based on the selected time domain feature set, which is better than the corresponding

classification accuracy of 85.71% obtained in experiment 1. In addition, the sensitivity

of 93.33% achieved by the CART-GA shows that the method gives an effective

discrimination for normal control based on the selected feature set of time domain. For

the KNN-GA method, which achieves the best classification accuracy based on selected

nonlinear feature set in the experiment 1, it achieves the classification accuracy of 70%

and the specificity of 93.33% based on the selected nonlinear feature set. The high

specificity demonstrates that KNN algorithm based on the selected nonlinear features


96

supports the MDD discrimination with high accuracy.

Table 5.9 Classification results based on the feature set of 4 domains

Feature Time domain Frequency domainIndex


KNN 0.46 0.40 0.43 0.26 0.46 0.36KNN-GA 0.66 0.86 0.76 0.66 0.66 0.66

RF 0.33 0.40 0.36 0.20 0.40 0.30RF -GA 0.46 0.73 0.60 0.60 0.60 0.60

LDA 0.26 0.46 0.36 0.46 0.60 0.53LDA-GA 0.73 0.80 0.76 0.66 0.86 0.76

CART 0.40 0.53 0.46 0.46 0.33 0.40CART-GA 0.93 0.80 0.86 0.73 0.80 0.76

Feature Wavelet domain Nonlinear domainIndex


KNN 0.13 0.46 0.30 0.13 0.80 0.46KNN-GA 0.40 0.73 0.56 0.46 0.93 0.70

RF 0.13 0.40 0.26 0.40 0.73 0.56RF -GA 0.26 0.60 0.43 0.46 0.93 0.70

LDA 0.46 0.46 0.46 0.20 0.60 0.40LDA-GA 0.73 0.80 0.76 0.46 0.80 0.63

CART 0.33 0.20 0.26 0.40 0.66 0.53CART-GA 0.66 0.66 0.66 0.60 0.80 0.70

Table 5.10 Classification results based on the fused feature set

IndexAlgorithm

SEN SPE ACC

KNN 0.20 0.53 0.36KNN-GA 0.33 0.80 0.56


97

RF 0.33 0.40 0.36RF -GA 0.33 0.80 0.56

LDA 0.33 0.80 0.56LDA-GA 0.53 0.73 0.63

CART 0.33 0.66 0.50CART-GA 0.53 0.80 0.66

5.3.3 Friedman and Multiple Comparison Test

Based on the results across multiple classification attempts, the Friedman and

multiple comparison test rank the classification performance based on different

classification methods, feature domain, and EEG data collecting channel. That is, the

Friedman and multiple comparison test is used to compare the classification

performance of the classification methods (feature domain / channel) and know which

method (feature domain / channel) is the best one for discriminating the MDD and NC

based on the single-channel EEG data. Specifically, a superior classification method

could obtain high classification performance over various feature domains and the data

collecting channel; A superior feature domain for classifying MDDs and NCs could get

high classification performance over various classification methods and data collecting

channel; A superior EEG data channel could get high classification performance over

various classification methods and feature domains.

To compare the classification performance of the classification methods, the

classification accuracies obtained in experiment 1 and 2 are organized into a 15×8

matrix. The column represents methods and the row represents the classification

accuracies of the classifiers based on the different combinations of EEG channels and


98

feature domains. As the same, a 24×5 matrix (row represents the classification

accuracies based on the different combinations of classification methods and EEG

channel) and 40×3 (row represents the classification accuracies based on the different

combinations of classification methods and feature domains) matrix are organized

respectively for comparing the classification performance of 5 feature domains and 3

EEG channels (Fp1 and Fp2 channel of BP device and another Fp1 channel of

NeuroSky device).

Figures 5.3 shows the result of the Friedman test for comparing the classification

performance of different classifiers, feature domains and EEG channels, respectively.

From the figure, we can know two estimates being compared are significantly different

if their intervals are disjoint, and are not significantly different if their intervals overlap.

The estimate represented by the blue line with a small circle in the middle is significant

different with the estimate represented by the red line, and is not significant different

with the estimate represented by the grey line. The superiority of the classification

performance of different classifiers, feature domains and EEG channels would be

known by comparing the position of the small circle located at x-axis. From the figure,

we can know that the CART-GA, time domain, and Fp1 channel of BP device has the

best classification performance for discriminating the EEG samples of normal controls

and depressives, respectively.


99

0 2 4 6 8

CART- GA

CART

LDA-GA

LDA

SVM-GA

SVM

KNN-GA

KNN

meanranks of different classifiers1 2 3 4 5

fused

nonlinear

wavelet

frequency

time

meanranks of different feature domains1 1.5 2 2.5 3

NS- Fp1

BP- Fp2

BP- Fp1

meanranks of different channels

Figure 5.15 Classification performance comparison of different classifiers, feature domains and EEG channels

5.3.4 Comparison with Other Methods

Although the multi-channel based EEG data collection method is the predominant

way for developing methods of MDDs and NCs classification, several studies chose to

collect the EEG data from frontal lobe and obtained good results. In order to compare

with the results in those studies, we recruited the representative studies in the literature

which use the EEG channel located on the frontal brain to collect data for classifying

MDDs and NCs. Table 5.8 shows the simple introduction of the comparison studies in

the recruited literatures. In the table, the author column gives the name of the first

author listed in the literature. The Database/features column demonstrates the sample

size of the MDD and NC, and introduces the categories of the feature domain used for

classification. The classifier column lists the classification method which achieves the


100

best classification accuracy in the study. The Acc(%) column gives the corresponding

best classification accuracy.

Cai et al. constructed a psychophysiological database, containing 213 (92

depressed patients and 121 normal controls) subjects [69]. The EEG signals of all

participants under resting state and sound stimulation are collected using a pervasive

prefrontal-lobe three electrode EEG system at Fp1, Fp2, and Fpz electrode sites. After

the data denoising step, a feature vector with 176 linear and nonlinear features is

extracted. Four classification methods (Support Vector Machine, K-Nearest Neighbor,

Classification Trees, and Artificial Neural Network) distinguished the depressed

participants from normal controls. The results showed that K-Nearest Neighbor (KNN)

obtains the highest accuracy of 79.27%. Their study proves the feasibility of a pervasive

three-electrode EEG acquisition system for depression diagnosis. Li et al. conducted an

experiment based on facial expression viewing task (Emo_block and Neu_block), and

EEG data of 20 university students were collected using a 128 channel HydroCel

Geodesic Sensor Net (HCGSN) [75]. Their results indicate that optimal performance is

achieved using a combination of feature selection method GreedyStepwise (GSW)

based on Correlation Features Selection (CFS) and classifier KNN for beta frequency

band. They find that left parietotemporal lobe in beta EEG frequency band has greater

effect on mild depression detection. And fewer EEG channels (FP1, FP2, F3, O2 and

T3) combined with linear features may be good candidates for usage in portable systems

for mild depression detection. Lin et al. presents the usefulness of the forehead EEG

with advanced sensing technology and signal processing algorithms to support people

with healthcare needs, such as treating depression [68]. They recruited 20 participants


101

(four men and 16 women, with a mean age of 44.3±13.0 years) among the outpatients of

the Psychiatric Department of Taipei Veterans General Hospital. The theta EEG powers

of all channels were then used as the featured input to train a two-class classier. The

RBF-based SVM classier reached the highest accuracy of 73.5±11.2% in classifying

EEG patterns of responders and non-responders, which indicates that the frontal theta

EEG power at baseline may be useful for predicting pretreatment responses. Bachmann

et al. performed the EEG recordings on a group of 13 medication-free depressive

outpatients and 13 gender and age matched controls [76]. The recorded 30-channel EEG

signal was analysed using linear methods spectral asymmetry index, alpha power

variability and relative gamma power and nonlinear methods Higuchi`s fractal

dimension, detrended fluctuation analysis and Lempel-Ziv complexity. Maximal

classification accuracy of 92% was indicated using mixed combination of three linear

and three nonlinear measures. Ahmadlou et al. presents an investigation of the frontal

brain of MDD patients using the wavelet-chaosmethodology and Katz`s and Higuchi`s

fractal dimensions (KFD and HFD) as measures of nonlinearity and complexity [77]. 12

non-drugged MDD (20 to 28 years old), and 12 non-MDD adults (20 to 28 years old)

are recruited from Atieh Comprehensive Center for Psych and Nerve Disorders, Tehran,

Iran. Based on the HFDs of left, right, and overall frontal brain beta sub-band, a high

accuracy of 91.3% is achieved for classification of MDD and non-MDD EEGs by using

enhanced probabilistic neural network (EPNN) classifier.

From the studies mentioned above, we can know that the single-channel EEG

analysis can provide discrimination of depression at the level of multichannel EEG

analysis. The best classification accuracy of 86.67% achieved in this study is a relative


102

good result, which shows a promising future for developing the ubiquitous MDD

prescreening application.

Table 5.11 Comparison studies in literature with our approach

Authors Year Database/features Classifier Acc(%)Cai et al. [69] 2018 - 92 MDDs and 121

NCs- linear and nonlinear features

KNN 76.83

Li et al. [78] 2016 - 10 mild depressives and 10 NCs- linear and nonlinear features

SVM+GSW 96

Lin et al. [68] 2017 - 20 participants for predicting treatment efficacy- frequency features of theta wave

RBF-based SVM

73.5±11.2

Bachmann et al. [76]

2018 - 13 MDDs and 13 NCs- linear and nonlinear features

LDA 92

Ahmadlou et al. [77]

2012 - 12 MDDs and 12 NCs- wavelet-chaos methodology and fractal dimensions

EPNN 91.3

This study (multi-channel device)

2018 - 23 MDDs and 16 NCs- linear, frequency, wavelet and nonlinear

KNN-GA 94.29

This study (wearable device)

2018 - 15 MDDs and 15 NCs- linear, frequency, wavelet and nonlinear

CART-GA 86.67


103

5.4 Discussion

The aim of this study is to explore whether the single forehead EEG channel could

provide an effective discrimination for MDD and NC at the level of the multiple EEG

channels or not, and further to support the feasibility of portable EEG device based

ubiquitous MDD prescreening application. To gain this, we conduct the first experiment

by using multi-channel EEG device and deriving the EEG data from the Fp1 and Fp2

channels. The DWT methods are used to denoise the raw EEG signals and decompose

them into 5 subbands (delta, theta, alpha, beta and gamma). A 263-dimensions feature

vector is extracted by time domain, frequency domain, wavelet and nonlinear analysis

methods. The GA algorithm is utilized to select effective feature subsets and train the

basic classifiers including KNN, RF, LDA and CART. The best classification accuracy

of 94.29% is achieved by the KNN-GA method based on the selected nonlinear features

extracted from the EEG data of Fp1 channel. The result shows that the single forehead

EEG channel could provide an effective discrimination for MDD and NC at the level of

the multiple EEG channels. Additionally, tables 5.1-5.5 also demonstrate that in the

most cases, the classification results based on the EEG data of the Fp1 channel is better

than the classification results based on the EEG data of the Fp2 channel. This might

indicate the EEG data collected from Fp1 channel contains more discriminative

information for depressive state than the EEG data collected from Fp2 channel. The

conclusion is also supported by other studies. For instance, Bachmann et al. calculate

the classification accuracy between depressive and control subjects using logistic

regression analysis with leave-one-out cross-validation [76]. The calculations were


104

performed separately for each EEG channel. The result shows that the maximal

accuracy in FP1 channel based on linear measures is 69%, which is better than the

maximal accuracy of 65% in FP2 channel. Based on nonlinear measures, the maximal

accuracy in FP1 channel is 72% which is better than the maximal accuracy of 62% in

FP2 channel.

Tables 5.6 and 5.7 show that the CART-GA methods obtained the best

classification accuracy of 86.67% in the MDDs and NCs discrimination based on the

portable EEG device. Although the classification accuracy is inferior to the

classification accuracy based on the multiple channel EEG device, it shows a relative

good result when compared with other literatures. And we believe that this result

indicates a promising future for developing a ubiquitous application for prescreening

depression. Furthermore, the Friedman and multiple comparison test is used to compare

the classification performance by the accuracy results of the classification methods,

feature sets and EEG data collection channels. Figures 5.3 gives the result and shows

that the CART-GA, time domain, and Fp1 channel of BP device has the best

classification performance for discriminating the EEG samples of normal controls and

depressives, respectively.

The GA algorithm is common used in selecting effective feature subset for

discriminating depressive and control subjects [79-81]. Mohammadi et al. applied LDA

to map the features into a new feature space and applied GA to identify the most

significant features [66]. Their finding suggests that the proposed automated EEG

analytical approach could be a useful adjunctive diagnostic approach in clinical practice.

Hosseinifard et al. employed LDA as the basic classifier to discriminate depressive and


105

control subjects, the GA is used to select the most important features, the classification

accuracy of 86% combined with nonlinear features is obtained [47]. Many studies used

the measure of nonlinearity to investigate the brain complexity of MDD patients, and

reported that nonlinear analysis of EEG can be a useful method for prescreening

depressed patients and normal subjects [51-53, 82]. Although the sensitivity and

accuracy results in this study show that the classification performance of nonlinear

features is inferior to the time domain features, there is no significant difference

between the results. In addition, it is well known that the classification performance of

the classifier and feature domain is high related with the data quality. While the EEG

data collected by the portable EEG device is reliable to detect brain activity, the data

quality is affected by the high resistance of the dry electrode. In other words, the quality

of the data collected by the portable EEG device (dry electrode) is inferior to the quality

of the data collected by the multi-channel EEG device (wet electrode). Thus, this might

be an explanation for illustrating why the FP1 channel of BP device is the EEG channel

with optimal classification performance.


6. Forehead EEG Based Objective Evaluation for

Depressive Mood State

6.1 Introduction

Traditional diagnostic and evaluation approaches for MDD are based on patient

interviews, which attempt to give a comprehensive evaluation for several depressive

symptoms and form a treatment decision. However, such kind of depression evaluation

method provides a subjective assessment for the depressive symptoms and frequently

shared with other maladies [83]. That is, reliance upon clinical assessments and patient

interviews for diagnosing MDD is frequently associated with misdiagnosis and

suboptimal treatment outcomes. In addition, the most clinical tools, such as Hamilton

Depression Scale (HAMD) and Beck Depression Inventory (BDI), always have good

test re-test reliability, which could reliably replicate the result more than once in the

same situation and population [84]. Such tools are always used to provide a cross-

sectional assessment of psychological status but unable to assess the daily change of

psychological status. A continuous assessment on mood status with fine granularity

might give circumstantial evidence to reflect the depression idiosyncrasy of patients.

Moreover, fear of discrimination is one of difficulties for depression treatment [85].

Depressed people feel the world is deeply suspicious of them, is unlikely to befriend

them. Researchers at King’s College London’s Institute of Psychiatry used detailed

questionnaires to ask 1,082 people being treated for depression in 35 different countries

about their experiences of discrimination. The study shows that around 71% active wish

107

to conceal their depression from other people, which has led to concerns that people

with depression might not seek help, doing so would more than likely make their

condition become chronic.

Objectively quantify the severity of depression symptoms might be a viable way to

decline the misdiagnosis rate and improve the effectiveness of treatment outcomes.

Compared with clinical scales, objective methods based on physiological data not only

have the advantage of consistency result for quantitatively assessing depression, but also

provide a continuous assessment in a daily granularity. More importantly, using

wearable devices to detect the physiological data, those methods are easy to be

implemented in the pervasive environment and avoid the privacy disclosure. Therefore,

there is an increasing interest in studying objective methods for quantitatively

evaluating depression based on physiological and behavioral data [11, 86-90]. In this

study, we mainly focus on finding out a feasible and pervasive way to objectively

quantify depression symptoms based on physiological data.

To gain this, two questions should be dealt with:

(1) Which physiological data is eligible to provide a quantitative evaluation for

which symptom?

(2) How to validate the effectiveness of the quantitative evaluation method?

For the first question, depression is defined as a psychiatric illness that at least one

of depressive symptoms must be depressed mood or loss of interest in activities, and the

symptom should be continuously experienced for at least two weeks. Additionally,

clinical rating scales for MDD are all contain items for rating depressed mood, such as

feeling guilty, anxiety and other emotional status. Therefore, the depressed mood is a


108

critical symptom for diagnosing and evaluating depression. The studies showed that

faulty mood regulation by brain is one of causes of depression [48]. Based on the

various physiology measurement tools, such as functional magnetic resonance imaging

(fMRI), electroencephalogram (EEG), gene etc, many studies tried to measure the

psychological data and measure the brain disorder. As one of the measurement tools,

quantitative measurement of brain electrical signals taken from the EEG is a



low cost. The pervasive and persistent nature of depressive symptoms has made scalp-

recorded EEG as an appropriate approach for understanding the causes of major

depressive disorder. Techniques based on EEG are used to investigate the predictive

utility with regard to mood improvement. Many studies show that anterior regions of

the brain may modulate the differential effects of emotional arousal on the information-

processing capacities of the cerebral hemispheres. Deldin et al. designed a cognitive

experiment with four EEG recording blocks to find out the difference of EEG activities

between controls and depressives, the study discovered that depressed responders

further exhibits a cortical asymmetry of greater right relative to left activity in frontal

areas [49, 91]. Kan et al. aimed at determining the differences of alpha waves between

normal and depressive groups [49]. The conclusions show that the frontal lobe has much

lower alpha waves in the depression group compared to the normal group. Many other

studies are also show a high correlation between depressed mood and EEG wave in

frontal areas of brain [50, 75]. Therefore, it is a feasible way to objectively quantify

depressive mood status based on forehead EEG data.


109

For the second question, current clinical methods for depression diagnosis and

evaluation are still based on subjective clinical interview or clinician-rated scales. Few

studies claimed to develop a quantitative method for evaluating depression to supersede

traditional approaches, the majority prefer to choose clinical assessment tool as a

ground truth to test the effectiveness of the quantitative evaluation method. Cummins et

al. suggested that researchers should select a commonly used scale with clinical validity,

i.e. HAMD, to quantify depression severity and even suicide risk [92]. They also claim

that self-rated scale is a good choice, which is potentially more sensitive to detect

changes than clinician-rated scale, to rate the severity in milder forms of depression.

Sung et al. used non-invasive physiological and behavioral measures and visual analog

self-rating scale to collect a novel dataset of physiology, behavior, and emotional data

over long-term periods [93]. They show that it is possible to create an individualized

objective metric for depression based on simple physiological measures, and have

evidence that there are universal physiological features that can potentially be used to

create a universal objective metric for depression. A desire quantitative evaluation

method for evaluating depression severity is that the effectiveness is congruent with the

traditional evaluation approaches used in the clinical treatment. That is, compared with

the depression severity evaluation result rated by clinician-rated or self-rated tools, the

effectiveness of the quantitative evaluation method could be validated.

In order to solve the problems mentioned above, the main contributions of this

chapter are illustrated as follow:

(1) The EEG signal collected from the brain forehead is used to give a quantitative

evaluation for objectively assessing the depressive mood status. The methods including


110

Discrete Wavelet Transform (DWT), linear and nonlinear feature extraction are adopted

to extract the EEG rhythm and the related features.

(2) To provide a ground truth to test the effectiveness of the objective evaluation

method, an electronic diary log application named quantitative log for mental state (Q-

Log for short) is developed for self-rating the mental state by inpatients themselves. The

Principle Component Analysis (PCA) is used to process the Q-Log data. For each

patient, a curve called First Principle Component (FPC) is extracted to reflect the

overall tendency and daily change of the major depressive mood status.

(3) A regression analysis method based on Random Forest (RF) is selected for

modeling the relationship between a scalar dependent variable (FPC values of the Q-

Log) and explanatory variables (EEG features). The LOPOCV method is used to

calculate the correlation coefficient and P-value of prediction values and real FPC

values, and further to illustrate the model performance.


6.2.1 Data Acquisition

6.2.1.1 Portable EEG Data Acquisition

The forehead EEG data collected from Fp1 and Fp2 lobe simultaneously using B3

Band. The B3 Band is a portable EEG device equipping with the NeuroSky EEG

biosensor (512 Hz sampling frequency and 12-bit ADC precision). All subjects are

asked to record their EEG data in the resting state. The subjects are sitting on a sofa and


111

keeping with eyes closed for 5 mins while not to think anything purposefully in a dim

illuminated and acoustical room. They are also asked to maintain a minimum arousal

level without falling into sleep. The subjects are required to collect their forehead EEG

data twice daily (one time in the morning and evening respectively) and continuously

last for around 2 weeks.

6.2.1.2 Data Collection Using the Q-Log

The data acquisition time of the Q-Log is congruent with the portable EEG data for

each subject. The Q-Log is a visual analog scale which is deployed in the Android

platform and operated via a slider bar. The subjects are asked to daily self-rate their

current mood state just like recording or expressing the emotions by writing a diary. 40

adjectives of emotions vocabulary are cited from the Short Form of the Profile of Mood

Status (POMS-SF) in Chinese [94, 95] and employed to instruct the subjects for rating

their current mood states. The adjectives could be divided into 7 mood categories,

including tension-anxiety, depression-dejection, anger-hostility, fatigue-inertia,

confusion-bewilderment, vigor-activity, and self-mood relevant. Every adjective is rated

on 100 points scale, that is, the emotion instructed by the adjective could be rated by a

certain value range from 1 to 100. 5 levels (not at all, a little, moderately, quite a bit and

extremely) are used to give a detailed sub-division for the 100 points, the interval

between each level has 25 points. Figure 6.1 gives the interface of the Q-Log

application in an Android phone.


112

Figure 6.16 The interface of the Q-Log application in an Android phone

The data collection is lasted for 2 weeks, and the inpatients are required to self-rate

the mood state in the current moment twice daily (one time in the morning and evening

respectively). The data size of different inpatients is not always congruous since the

incorrect operation of experimenter, or the subject asked for one-day off during the

experiment. Hitherto, the experiment total collected 164 data samples of the Q-Log

from 7 unipolar depressives in the hospital.

6.2.1.3 HAMD Scales

The Hamilton Depression Rating Scale (HAMD) is used to validate the

effectiveness of the Q-Log for assessing the depressive mood status. Since the HAMD


113

developed in 1960 by Dr. Max. Hamilton of the University of Leeds, England, the scale

has been widely used in clinical practice and become a standard in pharmaceutical trials.

The scale has been proven useful for many years as a way of determining the depression

level of a patient before, during, and after treatment. It should be administrated by a

clinician experienced in working with psychiatric patients. Considering the clinical

authority of the HAMD, we use it to assess the depression level of inpatient in the

beginning, middle, and the ending of the experiment. For each inpatient, the HAMD

scale is rated three times by clinicians during the experiment.

6.2.2 Data Processing

6.2.2.1 Portable EEG Data Analysis

For each subject, every EEG sample is divided into several segments that

overlapped by 50%. A data de-noising method based on Discrete Wavelet Transform

(DWT) is used to process the data. The DWT is used in two aspects: EEG data de-

nosing and decomposition. For data de-nosing, a soft thresholding algorithm with `db5`

wavelet base is utilized to de-noise the raw EEG segment. For data decomposition, an

eight-layer DWT with `db5` wavelet base is used to decompose the de-noised EEG

segment into five subband components. In this paper, 6 time series, which include the

de-noised data and five subbands (delta, theta, alpha, beta and gamma), are extracted

from the raw EEG segment. The frequency range of each subband is depicted as follow:

delta (2-4 Hz), theta (4-8 Hz), alpha (8-16 Hz), beta (16-32 Hz) and gamma (32-64 Hz).


114

The linear, wavelet and nonlinear features are extracted from the 6 time series

respectively. The linear feature includes two categories: time and frequency domain

feature. For the time domain, the features, such as mean, standard variation, percentile,

kurtosis and skewness etc.,) are extracted. For the frequency domain, the welch method

with modified periodogram is applied to calculate power spectrum. In addition, the

relative power, the absolute power and the relative power ratio of beta and theta

subband are also calculated. The wavelet coefficients from third to seventh layer are

extracted from the de-noised EEG segment and used as wavelet features. Based on the

ratio of the energy of each layer, which can be represented by quadratic summations of

the wavelet coefficients, a feature named wavelet entropy is calculated. For the

nonlinear feature, the C0 complexity and the approximate entropy are extracted.

Eventually, a feature vector with 176 dimensions (time domain: 54, frequency domain:

54, wavelet: 56 and nonlinear: 12) is extracted from the raw EEG segment.

6.2.2.2 Q-Log Data Analysis

Because of the Q-Log contains 7 mood categories and adopts multiple adjectives

with similar meaning to limn the factor repeat, there are a number of approximate

emotion ratings which are highly-correlated with other items. It would be better if we

could attenuate the correlation, and recognize the major mood category from the Q-Log

dataset of 2 weeks for each patient. Thus, the PCA is utilized to process the Q-Log data.

The specific calculation steps are illustrated as follows:

(1) Based on the Q-Log dataset of 2 weeks for each patient, a t-by-40 matrix can be


115

assembled. The matrix X can be defined as follow:

X=|x11 x2

1 …x1

2 x22 …

x401

x402

…x1

t…x2

t……

…x40

t | (6-1)

where t means the Q-Log sample size, x it indicates the i-th item rating in the t-th self-

rating data.

(2) Transpose the matrix X to X ' and the average xm for each row of X ' can be

defined as follow:

xm=1t ∑i=1

t

x im (6-2)

where m means the m-th item of 40 items and the value range is [1,40]. After

subtracting off the column means, the matrix X1 can be defined as：

X1=| x11−x1 x1

2−x1 …x2

1−x2 x22−x2 …

x1t −x1

x2t −x2

…x40

1 −x40…

x402 −x40

……

…x40

t −x40| (6-3)

(3) Calculate the covariance matrix C for X1. The matrix C can be depicted as:

C= 139

X1 X1T (6-4)

where X1T is the transport matrix of X1.

(4) Calculate the eigenvalue and the corresponding eigenvector of the matrix C.

The eigenvalues of matrix C can be formed into a vector λ and described as:

λ=[λ1 , λ2 , …, λn] (6-5)

where n represents the number of eigenvalue. The corresponding eigenvector U j of j-th


116

eigenvalue can be depicted as:

U j=[u1j , u2

j ,…, u40j ] , 1≤ j≤ n (6-6)

According to the position of the maximum eigenvalue in the vector λ, the

corresponding eigenvector Umax, which is also known as the first principal coefficients,

can be obtained. The first principal component space of the matrix X ' can be calculated

by the formula:

FPC=X1× (Umax ) ' (6-7)

Umax=[u1max ,u2

max ,…,u40max] (6-8)

The indication of FPC can be illustrated by ordering the summation of the

corresponding ummax of 7 mood categories. The summation formulas of the corresponding

ummax of 7 mood categories are listed as follow:

Sang=u1max+u7

max+u15max+u19

max+u37max+u39

max

Sdep=u4max+u9

max+u18max+u24

max+u29max+u32

max

Sner=u5max+u6

max+u10max+u12

max+u14max+u40

max

Sflu=u8max+u23

max+u26max+u31

max+u36max

S fat=u11max+u13

max +u17max+u20

max+u22max

Sene=u1max+u7

max+u15max+u19

max+u37max+u39

max

Sself=u2max+u3

max +u33max+u35

max+u38max

(6-9)

where Sang, Sdep, Sner, Sflu, Sfat , Seneand Sself indicate anger-hostility, depression-dejection,

confusion-bewilderment, tension-anxiety, fatigue-inertia, vigor-activity and self-

emotional relevant, respectively. Because of FPC is regarded as the vector that contains

maximum information of original matrix X which is simultaneously affected by 7 mood

categories, it is able to represent the composite depressive mood status. Besides, every

single value of the FPC vector indicates the self-rating data for assessing the composite


117

depressive mood status in the corresponding trail, it also provides an intuitive change of

the composite depressive mood status in data collection period.

6.2.3 Quantitative Model Construction and Performance

Estimation

In this study, the RF regression analysis method is selected to execute the

quantitative evaluation model construction task. It can generate binary trees which are

made of nodes, every node represents a predictor, and the path from root node to leaf

node is intuitive for illustrating the relationship between predictors and quantitative

outcomes. In order to ensure the model is able to assess the major depressive mood

status accurately, the effectiveness of the model should be evaluated. The model training

and testing steps are listed as follows:

(1) The LOPOCV is used to split the EEG features and the FPC curve into two

parts respectively. That is, we adopt LOPOCV to split the datasets of 7 subjects into

training and testing datasets.

(2) Model Training. In every validation, a random forest regression model is

trained and built based on the EEG features (independent variable) and the FPC curve

(dependent variable) of 6 subjects. The EEG features of the rest subject are inputted into

model and generate the quantitative outcomes.

(3) Correlation analysis is utilized to estimate the performance of the model. The


118

correlation of actual data (the testing data of FPC curve) and the quantitative outcomes

is calculated via correlation analysis. The P-value is obtained by t-test to interpret the

correlation value has no significance in a certain probability. The correlation coefficient

r is closer to 1 and the p value is less than 0.05, the quantitative outcomes are more

capable to accurately rate the major depressive mood status in the experimental period.

6.3 Results

6.3.1 Depression Level of Inpatients Rated by HAMD Scale

Table 6.1 shows the depression state changes and HAMD scores of 7 inpatients in

experimental period. The ‘type’ column gives the information about the depression type

of inpatients at the beginning of the experiment. The ‘amelioration’ column shows

whether the depression symptom of inpatient is ameliorated or not after receiving the

therapy of 2 weeks. ‘score1’, ‘score2’, ‘score3’ give the HAMD scores of three times

rated by clinician in 2 weeks, respectively. According to the assessment of clinicians,

there are 2 patients whose depression type is changed to manic depression after

receiving the therapy of 1 week. It is unsuitable to rate their depression severity via

HAMD. In this case, the mark ‘*’ is used to represent the third HAMD rating is not

executed by clinician. Excluding that two inpatients, the 4th patient still belongs to the

unipolar patient but his depression symptom is not ameliorated and began to be worse

after the first therapy week. For the resting patients, their depression symptom is

ameliorated gradually.

Table 6.12 Depression state changes and HAMD scores of 7 inpatients


119

Patient type amelioration score1 score2 score31 unipolar no 20 22 *2 unipolar yes 22 16 73 unipolar yes 19 8 74 unipolar no 20 15 205 unipolar yes 29 16 86 unipolar no 23 8 *7 unipolar yes 20 18 11

6.3.2 Objective Evaluation for Mental Disorder

Figure 6.2 shows the information about overall tendency and local variation of the

FPC. In each subfigure, the horizontal coordinate indicates the time order of collecting

the Q-Log data, the vertical coordinate shows the FPC values which implies the value of

the major depressive mood state. We can know that (1) for the patient 2, 3, 5, and 7, the

overall tendency of the FPC declined and keep steady, which indicates the depressive

symptom is ameliorated after 2 weeks therapy. In contrast, for the patient 1, 4 and 6, the

overall tendency of those curves declined in the first week but rebounded in the second

week, which means the depressive severity is not remitted after receiving the drug

therapy of 2 weeks. In order to give a further illustration, the FPC values are also

represented in red line and the HAMD scores of three times are represented in green

line. As shown, the overall tendency of HAMD scores is congruent with FPC curve; (2)

bar graph shows the local variation of the FPC. The scores in the FPC have two types:

the value of the major emotional factor rated in the morning, which is marked by the

white bar, and value of the major emotional factor rated in the evening, which is marked


120

by the blue bar. As shown, for the patients whose state is ameliorated, the value rated in

the morning is higher than the value rated in the evening. This rhythm is more obviously

shown in the first week. This conclusion is congruous with the clinical experience,

which depicts the feelings of the depressives are often worse in the morning than in the

afternoon or evening.

Figure 6.17 Information about overall tendency and local variation of the FPC

6.3.3 Quantitative Model Evaluation

6.3.3.1 Parameter Selection for RF

For actualizing a pervasive method for objectively rating depressive mood status as

effective as the Q-Log, the RF algorithm is used to construct a regression model to


121

objectively rate the depressive mood status based on the forehead EEG features and the

FPC values. The performance of the RF is highly affected by the inputted parameters,

especially the number of features (mtry) and the number of trees (ntree). The grid

optimization method is used to select the optimal two parameters. The ntree parameter

is set from 1 to 500, and step size is 10. The mtry parameter is set from 1 to 100, and

step size is 5. For every LOOPCV, we can obtain a pair of correlation coefficients (CC)

and P-value. In order to find out the parameters with optimal CC and P-value, we

average the CCs and P-values of 7 times and select the two parameters with maximum

CC and minimum P-value as the optimal parameter pair. Figure 6.3 shows the meshgrid

plot of average CC and P-value when various feature number and tree number are

selected. According to Figure 6.3, the best CC and P-value are obtained when ntree

equals to 1 and mtry equals to 66. This indicates that one tree with high depth is

sufficient to obtain the effective outcomes for quantitatively rating the depressive mood

status. In the following work, we choose (ntree=1, mtry=66) as the optimal parameter

pair to train the regression model and obtain quantitative outcomes.


122

Figure 6.18 Meshgrid plot of average correlation coefficient and average P-value

6.3.3.2 Model Evaluation Using T-test

Table 6.2 shows the results about the effectiveness evaluation of quantitative

model. TrS means the sample size of training data, TeS indicates the sample size of

testing data. Because of the size of collected samples is different for each patient, the

number of TrS and TeS are also different in every LOPOCV. The correlation coefficient

and P-value are the results of correlation analysis obtained by T-test. The average CC of

7 inpatients is 0.6556, and the standard deviation is 0.0476. The P-values are all below

0.01, which implies that there is less than 1% probability happened in chance that the

quantitative outcomes for objectively rating the major depressive mood status have a

moderately strong uphill relationship with FPC values. This result may provide a

promising evidence to demonstrate that the built quantitative model is feasible to

objectively rate the depressive mood status as effective as the Q-Log.


123

Table 6.13 Result about the effectiveness evaluation of quantitative model

patient TrS TeS CC P-value1 202 16 0.72 6.54e-042 194 24 0.60 0.00163 200 18 0.71 8.61e-044 190 28 0.62 3.67e-045 192 26 0.61 1.96e-046 194 24 0.59 0.00217 192 26 0.67 1.36e-04

average 0.65 0.047

6.4 Discussion

In this chapter, a quantitative analysis method based on forehead EEG signal and

the Q-Log data is proposed to assess the depressive mood status in fine-granularity. The

PCA method is used to process the Q-Log data of each patient and obtain the FPC

values, which depict the overall tendency and local variability of depressive mood. It is

worthy to highlight that we summarized the FPC results of each patient, and discuss

with clinicians who did not demur our analysis results and support the intensive study in

hospital. Actually, the Q-Log belongs to Ecological Momentary Assessments (EMA),

which is an electronic diary method. Participants are repeatedly assessed for a certain

period of time (usually days to weeks), by administering a single or a set of

questionnaires on a relatively high frequency (e.g., daily or multiple times per day) [96-

99]. Similarly, the dramatic feature of the Q-Log is that it can be self-rated in a daily

granular, which facilitate clinicians to observe the mood status change of inpatients


124

more detail. Markowetz et al. mentioned that the temporal granularity at which

traditional methods collect data commonly is too coarse to reveal fine-granular patterns

[84]. The Q-Log provides a possible way to execute the identical psychometric test

multiple times over the course of a single day and reveal a daily pattern for the

depressive mood status.

Other researchers have previously explored the combination of self-rated scale data

with sensor data [100, 101]. The result demonstrates that the combination could provide

new insights into the pervasive and objective depression evaluation processes in daily

life. Based on the FPC curve and the features extracted from forehead EEG data, we

adopted RF regression algorithm to construct a quantitative model for objectively rating

the depressive mood status (independent variable: the EEG features, and dependent

variable: the FPC curve). The FPC values are used as a reference standard to evaluate

the quantitative model. The LOPOCV method is used to split the dataset into training

and testing datasets. For every LOPOCV, the model evaluation is finished by the

correlation analysis which is utilized to quantify the correlation of the quantitative

outcomes and the FPC. The result demonstrates that the quantitative outcomes for

objectively rating the major depressive mood status have a moderate uphill relationship

with FPC values, which provides a promising evidence to illustrate the built quantitative

model is feasible to objectively rate the major depressive mood status.


Chapter 7

7. Multi-modality Data and WaaS Architecture

Based Objective Evaluation for Depression

7.1 Introduction

This chapter aims at illustrating the detail implementation method of the objective

evaluation for depression based on multi-modality data and the WaaS architecture.

Following the WaaS architecture, the implementation of the objective evaluation for

depression is divided into four parts: multi-modality data collection, data analysis, data

fusion, and the specific depression applications. The four parts are respectively

illustrated as follows:

(1) The primary job of the multi-modality data collection includes the development

of the multi-modality data collection tools and the design of the data collection

paradigm. According to the description in the chapter 3, the systematic evaluation for

depression is mainly based on evaluating the severity of depressive symptoms by multi-

modality data. For every single depressive symptom, the chapter lists a specific

quantitative data to execute the objective evaluation. The data collection tool for

collecting what kind of specific quantitative data and quantifying what kind of

depressive symptom is designated in the following section. Besides, an experiment

paradigm is proposed to support the consecutive data collection in a long term.

(2) For every single modality data, a corresponding data analysis method is

adopted to analyze the data. The data analysis results are closely related to the

126

corresponding depressive symptoms. The physical meaning of the data analysis result

should be clarified so as to make the comprehensive assessment.

(3) The data fusion aims at integrating the collected multi-modality data, the

related data analysis results, and the clinical treatment information. The ontology based

Data Brain model is adopted to integrate those data and tries to form the data in

knowledge level. The data in knowledge level indicates that the data meaning could be

easily understood by human being and utilized for guiding the real clinical practices.

(4) Based on the works mentioned above, there are several service applications for

depression could be constructed. For instance, the objective evaluation service for the

single depressive symptom, the multi-modality data query, the data analysis result query

and knowledge sharing, and etc. The method for providing the real service applications

is illustrated in the following sections.

The main contributions of this chapter are illustrated as follows:

(1) A technical route follows the WaaS (Wisdom as a Service) architecture [37,

105] for objectively rating the depressive symptoms based on multi-modality data is

proposed. The technical route includes four steps, which are respectively corresponding

with the four layers in the WaaS architecture. A detail illustration is given for illustrating

the primary job in each step or layer.

(2) The depressive symptoms related data is collected by the multi-modality data

collection tools and the experiment paradigm. According to the description in the

section 3.2, this chapter describes the specific information of the multi-modality data

collection tools. Combining with the tools, a complete experiment paradigm for

consecutively collecting the data is also elaborated.


127

(3) In order to explain how to objectively quantify the severity of depressive

symptoms based on the multi-modality data, some preliminary works about the data

analysis methods and the related analysis results are described. Furthermore, the Data

Brain model based data integration method is illustrated.

(4) After finishing the primary works instructed in the service layer of the WaaS

architecture, the next step is to correlate the working accomplishment of each layer and

form a service system. In order to do that, a case study based on the forehead EEG and

Q-Log data is elaborated to explain the system construction method.

7.2 Technical Route

Figure 7.1 shows the technical route for constructing the service system based on

multi-modality data following the WaaS architecture. Following by the WaaS

architecture, the technical route can be illustrated as follows: in the DaaS layer, the

multi-modality data of users, including the behavior data, portable EEG data, Q-Log

data, multi-channel EEG data, and etc., are collected. After that, machine learning and

mathematical statistics methods are used to analyze the collected data and obtain the

data quantitative analysis result for the depressive symptoms. The SQL databased is

used to save the collected data and the related data analysis results. In the InaaS layer,

the RDF technology is utilized to describe the detail information of the data and the

related data analysis results, and further to generate the data provenance and analysis

provenance for corresponding the concepts and properties built in the KaaS layer. The

constructions of data provenance and analysis provenance are mainly finished manually.


128

The quantitative analysis methods or models could be manually structuralized into

several rules by using the Jena inference subsystem to support the reasoning and

deduction for depression symptoms. In the KaaS layer, the ontology technology is used

to construct the four dimensions of the Data-Brain model for describing the concepts or

properties about the relation between the quantitative data, data analysis results, and

depressive symptoms. The clinical knowledge extracted from the depression treatment

criterions such as the ICD-10 and CCDM3 is also integrated in the KaaS layer. The

clinical knowledges not only provide the ground truth to validate the effectiveness of

the objective diagnosis and evaluation for depression, but also they explain the clinical

connotation of the data analysis result and support the conjoint analysis by the

quantitative analysis results for the depressive symptoms. Several real applications are

provided in the WaaS layer, such as the multi-modality data query, quantitative

evaluation result for depressive symptoms, results query and knowledge sharing, and

etc.

Considering the WaaS architecture driven depression evaluation based on multi-

modality data relates a lot of works, we merely adopt the quantitative analysis result

based on forehead EEG and self-rating physiology data (Q-Log) to illustrate how to

develop the application for quantitatively analyzing depressive mood status and

knowledge sharing.


129

Figure 7.19 Technical route for constructing the service system based on multi-modality data following the WaaS architecture

7.3 Multi-modality data collection

Figure 7.2 gives the deployment of the multi-modality data collection tools in the

ward, which space size is 6.7 meters length, 4.8 meters weight, and 2.7 meters height.

The collected multi-modality data is transmitted and saved into a data server for further

data analysis. As mentioned in the chapter 2, the BI methodology adopts a specific

single modality data to quantitatively reflect the severity of one depressive symptom.

The specific single modality data is collected by one specific data collection tool.

Specifically, the psychology self-rating tool named Q-Log is used to recognize the


130

primary depressive mood category and evaluate the mental state change. The sleep

mattress equipped with micro-vibration sensor is employed as an insomnia monitor to

detect the raw vibration signal in real time, the raw signal is further used to analyze the

sleeping events and give a quantitative representation of insomnia. In order to

quantitatively rate the slowness of thought and speech, we develop an android

application to record the interview conversation by using a lavalier microphone. The

mic is clipped on the clinician’s collar so as to make the patient feel relax to talk his real

thought. A portable EEG device equipped with the NeuroSky biosensor is used to

collect the raw EEG signal and the attention index, and further to quantify the impaired

ability to concentrate of patients. We also develop a diary behavioral data collection

module using accelerator sensor and micro SD card. The module is required to be

carried in a certain position of human body for a long term during daytime. A portable

ECG device equipped with CardioChip is adopted to collect the raw ECG signals of

patients and analyze the cardiovascular abnormality, and further to quantify the

cardiovascular symptom of patient. Considering the cognitive dysfunction is high

related with the brain functions, the brain signals measurement tools such as fMRI and

EEG are also included in our experiment. Combined with cognitive tasks, the cognitive

dysfunction could be quantitatively evaluated. The genetic factor of depression is also

considered and utilized to investigate the MDD from the micro-aspect. In our study, the

SCID-5 is employed as reference to validate the reliability of the objective evaluation

methods for depressive symptoms.


131

Figure 7.20 Multi-modality data collection tools deployed in the ward

Furthermore, an experiment paradigm is designed to support the multi-modality

data collection. Figure 7.3 shows data collecting strategy in different time points. The

paradigm comprises two data collection ways: long term data collection which aims at

consecutively collecting the data above 8 hours in a single day, and short term data

collection supports a cross-sectional data acquirement for acquiring the data in 5

minutes or 10 minutes per time. Specifically, the sleep mattress is adopted to monitor

the sleeping data of inpatients in the night, the time length is always around 8 hours.

The diary behavioral data collection module is utilized to monitor the diary activity of

inpatients in the day time. The patient is asked to collect the ten kinds of data when

he/she begins to be hospitalized in the ward. While in the hospital, the multi-modality

data including interview voice, Q-Log, Portable EEG (PEEG), and Portable ECG

(PECG) is collected by multiple times. When the clinician thinks the patient’s condition


132

become steady and is eligible for being discharged from hospital, the patient is asked to

join the final multi-modality data collection.

Figure 7.21 Multi-modality data collecting strategy in different time points

7.4 Multi-modality data analysis

7.4.1 Depression Type Recognition

As mentioned, the PCA is utilized to process the Q-Log data. According to the data

collection order, the Q-Log data of every inpatient could be organized and formed a data

matrix. After that, the PCA is utilized to calculate the eigenvalue of the data matrix and

the corresponding principle components. Ranked by the eigenvalues, the first principal

component corresponds to the maximum eigenvalue is selected to quantitatively

represent the major depressive mood state. A curve reflects the dynamic change of the

major depressive mood state could be extracted by mapping the data matrix into the first

principal component space, which contains the maximum energy or information of the

data matrix. We named the curve as the first principle component (FPC) and employed


133

it as a quantitative tool to objectively rate the depressive mood status for each inpatient.

By adding up the corresponding FPC values of each mood categories respectively,

the mood category of the major depressive mood state is the one which has the

maximum summation. Figure 7.4 shows the mood category of the major depressive

mood state for each patient. In each subfigure, the horizontal coordinate is the 7 mood

categories, the alphabet A-G indicates the anger-hostility, self-mood relevant,

depression-dejection, tension-anxiety, confusion-bewilderment, fatigue-inertia and

vigor-activity, respectively. The vertical coordinate means the weight of each mood

category which is calculated by adding up the corresponding FPC values. For the major

depressive mood state of unipolar inpatients, the weight of one or several negative

emotions (such as depression-dejection, tension-anxiety, confusion-bewilderment) is

higher than the weight of the positive emotion (such as self-mood relevant and vigor-

activity). Thus, we can know that one of negative emotions is the major depressive

mood category of unipolar depressives and multiple negative emotions might be

concurrent simultaneously. This conclusion is strongly supported by the analysis results

of patients 3-5 and 7.

Combining with the results shown in the Figure 6.2, we can know that for the

patients with steady fluctuation of major depressive mood state, the weight of negative

emotions is higher than positive factors, but for the patients with unsteady fluctuation of

major depressive mood state, the weight of negative emotions is approximate with

positive emotions. In fact, the depression type of patients 1, 2, and 6 transformed to

bipolar depression after receiving the drug therapy of 2 weeks. From this, we can know

that the weight of negative emotion closes to the weight of positive emotion might be an


134

indication for showing the depression type changes from unipolar to bipolar depression.

Figure 7.22 The mood category of the major depressive mood state for each patient

7.4.2 Voice Analysis

Compared with the normal people, the voice speed and amount of depressives in

diary life is different. When someone is depressed, their range of pitch and volume

drop, so they tend to speak lower, flatter and softer. Speech also sounds labored, with

more pauses, starts and stops. Another key indicator is the tension or relaxation of the

vocal cords, which can make speech sound strained or breathy. Too much tension or

relaxation has been linked to depression and suicide risk. Depressed patients’ tongues

and breath may also become uncoordinated, resulting in a slight slurring of speech.

Regard this, we asked the clinicians to arrange a clinical interview and tried to record

the voice data of inpatients by the lavalier microphone. The voice of inpatient during the

interview conversation is extracted and analyzed by the voice processing technology.


135

Figure 7.5 shows a Matlab GUI tool for manually extracting the conversation voice

of inpatient. This tool is able to play, pause, and resume the interview conversation. The

first axes shows the voice data shown by the blue curve, a progress bar is located below

the first axes shows the current playback position of the whole interview conversation.

A red line is located above the blue curve for showing the pause position. We can click

the mouse to choose the start and the end of the conversation fragment of inpatient. The

start and the end position of the fragment are represented by the green line, respectively.

The selected voice fragment is displayed in the second axes for better observation.

Figure 7.23 A Matlab GUI tool for manually extracting the conversation voice


136

7.4.3 Diary Activity Analysis

Figure 7.6 shows a Matlab tool for showing the accelerator sensor data and

automatically recognizing the activity fragment. The first axes show the accelerator

sensor data comprises the data in three-axis. We represent the data in three-axis by three

different colors (X: red, Y: green, Z: blue). As shown in the figure, the curve with

irregular shape and spike wave means the inpatient is in an active state. The curve with

regular shape and smooth wave means the inpatient is in a static state. There is a button

named ‘Auto-Recognition’ which could automatically recognize the activity fragment

by a physical activity detection algorithm. The core of the physical activity detection

algorithm is to compare the entropy’s value of the data fragment, and extract the activity

fragment with high entropy’s value. The second axes show the entropy’s value of

different data fragments. In the first axes, we use the red line to mark the start position

of the physical activity fragment, and use the green line to mark the end position.

Besides, this tool is also able to play forward or backward the long term accelerator

sensor data, which is good for users to observe the data change.


137

Figure 7.24 A Matlab tool for showing the accelerator sensor data and automatically

recognizing the activity fragment

7.4.4 Concentration Ability Analysis

With brain injury in the frontal lobes, the person cannot concentrate on a single

subject or a single voice and is unable to ignore the other incentives, which means the

frontal lobe of the brain is related with the concentration ability of human being. As

mentioned above, the forehead EEG detected by the portable EEG device (NeuroSky

mindwave) is utilized to quantify the concentration ability of inpatient. The NeuroSky

algorithms provide the foundation of a universe of applications that can be built to

optimize brain health, education, alertness and overall function. There are algorithms for

rating the intensity of a user's level of mental “attention”. Specifically, the application

programming interface (API) of one NeuroSky algorithm provides Attention meter of


138

the user, which indicates the intensity of a user's level of mental “focus” or “attention”,

such as that which occurs during intense concentration and directed (but stable) mental

activity. Its value ranges from 0 to 100. Distractions, wandering thoughts, lack of focus,

or anxiety may lower the Attention meter levels. We can call the API in the Android

program to get the value of the Attention meter. Figure 7.7 shows the Android app for

showing the forehead EEG curve and the Attention meter represented by “Atte.”.

Figure 7.25 An Android app for showing the forehead EEG curve and the Attention

meter

7.5 Multi-modality Data Fusion by Data-Brain Model

Figure 7.8 shows the Data Brain model constructed by Protege tool. As mentioned

in section 3.4, ontologies are the core of the Data Brain model. The Data-Brain model

comprises four dimensions (function dimension, data dimension, analysis dimension,

and experiment dimension) is used to support the systematic data fusion. The Web


139

Ontology Language (OWL) is used to construct the four dimensions and define the

relationships of data, quantitative results, and depressive symptoms.

Figure 7.26 Data Brain model construction by Protege tool

7.6 Case Study for the Service Construction Following WaaS

Architecture

The combination of physiologic and behavioral metrics discussed above would be

complimentary in the study of response to treatment in clinical depression. However,

there exist some challenges in transferring the quantitative evaluation method for

depression from clinical application to real world: (1) physiological measurement tools

in hospital for depression treatment are cumbersome and difficult to be operated, which

indicates that the tools are not suitable for making pervasive application; (2)


140

physiological signal based quantitative analysis application for depression is an

interdisciplinary application, which involves clinical knowledge, data science and

computer-aided engineering. It is difficult for the clinicians or the people, who are not

adept at using Machine Learning technology, to understand the meaning of quantitative

results; (3) quantitative analysis method is just a reference tool and insufficient to make

a comprehensive treatment decision. That is, clinicians should consider conjointly

analyzing the quantitative results associated with the contextual and clinical therapy

information for making treatment decision. The contextual and clinical therapy

information is included in the electronic medical record. Thus, in order to support the

adjunctive treatment decision in the clinical trial, a data fusion model is needed to

integrate the quantitative results of the depressive symptoms with the electronic medical

record.

In order to solve the challenges mentioned above, a case study about the objective

evaluation service for depression based on EEG and Q-Log data is illustrated.

Following the WaaS architecture, which advantage is that it combines the data analysis

results with experience knowledge to support the decision in the real world, we would

like to integrate the quantitative evaluation results for mood status based on EEG data

with clinical knowledge to generate clinical feedback automatically for clinicians. In

addition, in order to facilitate clinicians or other users to retrieve and query contextual

information of inpatients, ontology technology is used to integrate data and annotate the

connotation of related concepts and properties. The quantitative model based on the

machine learning methods is manually transformed into several rules in Jena format,

and the Jena inference API is utilized to create an inference model for obtaining the


141

quantitative results. Take the forehead EEG and Q-Log data as example, an exemplary

application named quantitative analysis for depression mood status and knowledge

sharing is actualized.

7.6.1 DaaS Layer: SQL Database Construction

The objective evaluation for mental disorder based on the Q-Log data aims at

providing the quantitative evaluation service for the depressive inpatient. According to

the WaaS architecture, the Q-Log data collection and analysis are finished in the DaaS

layer. The next step is to summarize the provenance information of the Q-Log data and

the related data analysis results according to the ontology concept. The concepts and the

related objects are instantiated in the InaaS layer, the instances are saved in the MySQL

database. Figure 7.9 shows several datasheets of Q-Log data and the bonds that linked

them. The datasheet named ‘datatransfer_exp_subjects’ records the information such as

name, gender, age, and etc.; the datasheet named ‘Q-Log data’ contains the indexes

mapping with other datasheets, including ‘datatransfer_exp_subjects’, ‘data sample’,

‘analysis method’, and ‘analysis results’; the datasheet named ‘data sample’ records the

information such as the rating values for 40 emotion adjectives, data collecting time,

data collecting order, and etc.; the datasheet named ‘analysis method’ records the

information such as method name, code, parameters, running environment (tools), and

etc.; the datasheet named ‘analysis results’ records the indexes mapping with ‘principle

mood’ and ‘dynamic change’; the ‘principle mood’ records the analysis result about the

major depressive mood status of inpatient during experiment, and the ‘dynamic change’

records the analysis results including the first principle component values, the overall


142

tendency if the major depressive mood status, and the flag indicating whether the

inpatient transformed to bipolar or not.

Figure 7.27 Several datasheets of Q-Log data and the bonds that linked them

7.6.2 InaaS Layer: RDF Construction

The InaaS layer summarizes the quantitative analysis results and saves them in the

SQL database. The OWL and the RDF technology are used to annotate the data

meanings and instantiate the concepts or properties into the data in the SQL database. It

is noteworthy that the resource information described in the InaaS layer are the

instantiations of the concepts or properties in the KaaS layer, which integrates the multi-

modality data by ontology technology, and supporting the conjoint analysis of the

quantitative analysis results and clinical treatment knowledge.


143

Figure 7.28 RDF initiations of ontology concepts


144

7.6.3 KaaS Layer: Ontology Construction

The clinical connotation of the analysis results of multi-modality data should be

clarified. Because of the objective evaluation for depression relates to the knowledge

about data mining and mathematical statistics, the clinicians might not know the data

analysis process and the analysis result very well so that the clinical meaning of the

objective evaluation result is difficult for them to understand. There is no doubt that an

intuitive illustration of the meanings would help the clinicians to make an accurate

clinical decision. In addition, with the development of the computation technologies, a

mental health service system automatically provides the objective evaluation for

depression is more desirable in the real application. That is, an automated method for

generating or recommending the clinical treatment decision could be the basis for the

objective evaluation system of depression. In order to do that, the quantitative data

analysis result should be fused with clinical treatment experience to generate reliable

and effective decisions.

The BI follows the WaaS architecture to support the information query of the

multi-modality data and the corresponding data analysis results, and give intuitive

illustrations for the query results based on the concepts and properties in the Data-Brain

ontology. Knowledge sharing is defined as a function which is not only able to provide

contextual information for the users, but also can offer data annotations for facilitating

the users, who are not adept at Machine Learning technology, to understand the related

meaning. Thus, the fundamental work for actualizing the knowledge sharing function is

to annotate the contextual information and physiological feature of patients by


145

constructing ontologies. In this case, a portable EEG ontology and a ontology of clinical

tool for rating the depressive symptoms are constructed by using Protege 4.3. The

structure of the portable EEG and the clinical tool ontology are showed in Figures 7.11

and 7.12 respectively. The ontology structure is depicted from 3 levels: category level,

class level and individual level. The category level defines the most abstract entity

concepts, which are also known as the parent classes. The parent classes are subdivided

into multiple subclasses in the class level. The individual level describes the instances of

the subclasses.

For portable EEG ontology, the related EEG concepts and trial details are

summarized in the category level. The subclasses of the related EEG concepts depict the

information of EEG recording parameter (equipment, scalp region, sample rate, and

etc.) and EEG features (linear, wavelet and nonlinear). The subclasses of trial details

describe the subject information, experiment variable and data processing. The subject

information is recorded to define the subject number, name, sexual, the owner-member

relationship between the collected dataset and the subject, etc. The information of

experiment variable illustrates the detailed experiment method, such as the data

collecting time, the experiment status (resting status or task status), the materials used in

the experiment, etc. The data processing information describes the related information

of extracted features, such as nonlinear feature extraction (C0 complexity and

approximate entropy), and data de-noising method.


146

Figure 7.29 The structure of portable EEG ontology

For clinical tool ontology, three parent classes are introduced in the category level:

self-rating tools, subject basic information and clinician-rating scale. In the class level,

the structure of subject basic information is congruent with the one in the portable EEG

ontology. The classes of self-rating and clinician-rating scale depict all the scales used

in this experiment. For example, the MINI is a short structured clinical interview which

enables researchers to make the diagnosis for psychiatric disorders. The HAMD is a

common clinical other-rating scale which can evaluate the depression symptom. The

Young Mania Rating Scale (YMRS) is also a common used clinical other-rating scale


147

which can evaluate the mania symptom. Since we use the Q-Log to subjectively assess

the major depression mood status, the ontology structure of the Q-Log is elaborated. In

the class level, the Q-Log class contains three subclasses: item, log and analysis method.

The item class defines the specific items in the Q-Log, such as item number, item name,

item score, item options, etc. The log class describes the detailed Q-Log information,

such as scale score, scale instruction, property, the major emotional factor, etc. The

analysis method class depicts the mathematical statistics method adopted for processing

the Q-Log data. The individual level instantiates the subclasses defined in the class

level.

Figure 7.30 The structure of clinical evaluation tool ontology for depression


148

Besides the ontology structure, the object property is designed to describe the

relationship between individuals, and the datatype property is designed to link the

individual with a kind of numeric data. Some definitions of the object and datatype

property in the portable EEG ontology are illustrated in Tables 7.1 and 7.2 respectively.

Table 7.14 Part of the object properties of portable EEG ontology

Object property Domain RangehasEEGFeature Subject EEG_FeaturecontainSubFeature Nonlinear Raw_C0onElectrode EEG_Feature Electrode

Table 7.15 Part of the datatype properties of portable EEG ontology

Datatype property Domain datatypehasValue EEG_Feature doubledeviceName EEG_equipment stringhasName Subject string

7.6.4 WaaS Layer: Service Construction

As mentioned above, the elements in the datasheet are the instances of the concepts

depicted in the ontology. In order to do that, we adopt Web Ontology Language (OWL)

to design ontology, and define the object property and data property to define the

semantic relationships of different concepts. The Protege tool is employed to execute

the ontology design and generate the Resource Description Framework (RDF) data.


149

After that, we also develop a plugin to embed the data in MySQL database into the RDF

data. Figure 7.13 gives an example to demonstrate the semantic relationships between

entities and properties. The meaning of the example is illustrated as follow:

A unipolar inpatient named Li XX and with age 35 joined the experiment using Q-

Log to quantitatively evaluate the symptom of mental disorder. 24 Q-Log data samples

are collected, the collecting time, order and item scores of every Q-Log sample are

recorded. The method named Principle Component Analysis (PCA) is used to analyze

the data. The corresponding method code and parameters are recorded to support the

automatic generation of the analysis results. The results for quantitatively evaluating

the major depressive mood status (principle mood) and the dynamic change of the mood

status during experiment period are generated. The weights of 7 mood categories are

figured out and the meaning of the major depressive mood status (represented by the

first principle component of Q-Log data matrix) is given. Besides, the flag indicating

whether the unipolar inpatient transforms to bipolar or not, and the overall tendency of

the major depressive mood status over experiment period are described and recorded.

The figure utilizes three layers (data layer, semantic layer, and owl layer) to depict

the ontology structure of the example. The three layers are illustrated as follows:

(1) The data layer adopts the instances and the specific property values to describe

the example. The instance of class and the relating properties is represented by ellipse

and circles, respectively. The string in the middle of the arrow line explains the semantic

relationship of instances, or instance and the property values.

(2) Compared with the data layer, the semantic layer provides an abstract

description for the example. The instances and the relationships in the data layer are


150

summarized into terminologies in the OWL including class, object property, datatype

property, and so forth. The green arrow dotted line between data layer and semantic

layer indicates that the subject represented by ‘www.subject.com/patient/1’ is an

instance of the subject class depicted in the semantic layer. The black arrow line means

‘rdfs:domain’, which is an instance of property that is used to state that any resource

that has a given property is an instance of one or more classes. The black arrow dotted

line means ‘rdfs:range’, which states that any resource that is the value of an

‘rdfs:domain’ property is an instance of rdfs:Class. The red arrow dotted line gives an

illustration for the datatype of several property values. For instance, the datatype of

property value of ‘has_moodWeights’ is list, the datatype of property value of

‘has_Birth’ is date, the datatype of property value of ‘FP_meaningOf’ is string, and the

datatype of property value of ‘has_Age’ is integer.

(3) The OWL layer shows four OWL terminologies to express the connotations of

the shapes with different colors in the semantic layer. For example, the red rectangle

represents a specific class which is described by ‘owl:Class’ in the OWL, the blue

rectangle represents a specific datatype property which is described by ‘owl:Datatype

Property’ in the OWL, the azure rectangle represents a specific object property which is

described by ‘owl:Object Property’ in the OWL, and the white rectangle represents a

specific literal value which is described by ‘rdfs:Literal’ in the OWL.

Based on the RDF data, the SPARQL is use to support a web service for

quantitatively evaluating the mental disorder of inpatient. On one hand, the specific data

information and the relating data analysis results, such as the personal information of

subject, the number of the Q-Log data sample, the major depressive mood status and the


151

corresponding dynamic change over experiment period, could be queried. On the other

hand, based on the concepts and properties in the Data-Brain ontology, the specific

meaning of the data analysis results of the Q-Log data could be intuitively illustrated.

Figure 7.31 Partial resource descriptions of the Data-Brain model


152

According to the quantitative model built in the chapter 6, the trees contained in

the model are manually transformed into rules in Jena format, and Jena inference APIs

are used to derive feature notations from built ontologies. Because of the tree in

quantitative model has a high depth, a rule with long length is transformed from the

tree. As the length limit, the rule is simplified to give an exemplary illustration. The

simplified rule is given and explained as follows.

[rule1: (?subject rdf:type base:Subject) (?v1 rdf:type base: original_min_CD6) (?

v1 base: hasValue ?value1) greaterThan(?value1, -0.2539) (?v1 base:onElectrode ?

point1) (?point 1 rdfs:label "FP1")(?v2 rdf:type base: alpha_psd) (?v2 base: hasValue ?

value2) greaterThan (?value2, -0.1422)(?v2 base: onElectrode ? point 1) (?point 1

rdfs:label "FP1")(?v3 rdf:type base: denoised_sum_CD4) (?v3 base: hasValue ?value3)

greaterThan (?value3, 0.3396) (?v3 base:onElectrode ? point 1) (?point 1 rdfs:label "

FP1")(?RealValue rdf:type base:MoodStatus) (?RealValue base: has MoodStatus

Value ? value4)-> AssignOne(?value4, ? MoodStatus)]

The words with prefix ? (such as ?v1, ?v1, ?value1 and etc.) are the variables

defined in Jena, base : indicates the class name or property name defined in the

ontology, such as the Subject is the subject class, has Value is one of defined properties.

The classes of the EEG feature are also followed by base :. For example, the feature

named original_min_CD6 is the minimum of the 6-th layer wavelet coefficient, which

indicates the minimum wavelet decomposition coefficient of the theta subband. The

denoised_sum_CD4 means the sum of the 4-th layer wavelet cofficient, which means

the energy of the beta subband. The Jena functions are also depicted in the rule, such as

greaterThan and AssignOne. The general meaning of the rule is that supposing the


153

feature original_min_CD6, alpha_psd and denoised_sum_CD4 of the EEG signal

collected from FP1 is greater than -0.2539, -0.1422 and 0.3396 respectively, then a

quantitative value of the major depressive mood status could be outputted by the model

and assigned to the variable MoodStatus.

The core idea of using Jena inference subsystem is to facilitate the clinician to

derive contextual information and connotation of physiological features. The subsystem

is designed to allow a range of inference engines to be plugged into Jena. Such engines

are used to derive additional RDF assertions which are entailed from some base RDF

together with any optional ontology information and the axioms and rules. In this study,

the provenance information about the extracted features and the relationship between

features and quantitative outcomes are described and provided. For actualizing that,

three steps are designed:

(1) Summarize the rules from the random forest regression model and transform

them into Jena format. Because of the random forest model is consisted of several

decision trees, several rules which number is equivalent with the number of trees can be

generated. That is, every decision tree could be transformed into a rule in certain format

used in Jena.

(2) According to the formatted rules, the Jena inference API is used to derive the

notation from the built ontologies. Based on the preceding work about annotating the

concepts and properties, the Jena inference API can be used to read these annotations

from ontologies. An inference model is created to query the information on the specified

meaning of features and the relationships between feature and quantitative outcomes.

(3) Organize the annotations and provide them as a feedback to clinician.


154

Combined with the contextual information of subject, we can create a Java API to

automatically organize the annotations and provide them to clinicians or other users.

7.7 Discussion

Taking the quantitative results for depressive mood status based on the Q-Log and

forehead EEG data as example, this chapter shows a technical route about how to apply

the quantitative results for depressive symptom based on the multi-modality data in the

adjunctive clinical trial. Following by the WaaS architecture, a case study about the

quantitative analysis for depression mood status and knowledge sharing is actualized to

provide contextual information for the users who are not adept at Machine Learning,

and offer data annotations to facilitate users to understand the related meaning. In the

DaaS layer, the Q-Log, forehead EEG, and the related data analysis results are managed

and saved using SQL database; In the InaaS layer, the Resource Description Framework

(RDF) model is developed to instantiate the concepts and the properties designed in the

KaaS layer by the data in the database. In the KaaS layer, a portable EEG ontology,

scale ontology, and related object and property are constructed using Protege 4.3. The

constructed quantitative model is manually transformed into Jena format and derive

feature notation from built ontologies. The advantage of doing this is to transform the

non-intuitive data into the intuitive knowledge and facilitate the clinical users to

understand the meanings of the data and the quantitative analysis results. Prior studies

always adopt an ontology model to build the terminology of depression and utilize the

Bayesian networks or other Machine Learning method to infer the probability of

depression [106-109]. In the WaaS layer, an exemplary application named quantitative


155

analysis for depression mood status and knowledge sharing is actualized. The example

demonstrates that the WaaS architecture gives a feasible way to build a platform to

integrate data from commercially available sensors, and provide the objective

evaluation for the depressive symptoms.


Chapter 8

8. Epilogue

8.1 Contributions and Conclusions

The main contributions of the thesis are illustrated as follows:

(1) We summarize the objective depression diagnosis and evaluation method based

on the BI methodology, which comprises the systematic method for instructing the data

collection, experiment design and implementation, and data fusion. The systematic data

collection method is illustrated to clarify which kind of quantitative data is used to

evaluate which kind of depressive symptom. Take the depressive inpatients as the data

collection subject, the systematic experiment design and implementation provides an

experiment paradigm to collect the multi-modality data in fine granularity. The

systematic data fusion adopts the Data-Brain model to systematically integrate data,

information, and knowledge for supporting the clinical depression services. Utilize the

EEG as the quantitative measurement and combine with the machine learning methods,

some specific case studies about the BI based objective depression diagnosis and

evaluation methods are given.

(2) The EEG biomarker for discriminating the depression is explored. We propose

a CNN architecture merging the two kind of convolutional filters to simultaneously

learn the synchronous EEG pattern and single EEG pattern. The result shows that the

model achieves the maximum accuracy, sensitivity, and specificity are 85.6%, 87.8%,

and 81.4% for two-category classification, and achieves the maximum accuracy,

sensitivity, and specificity are 68.6%, 68.7%, and 81.9%, for three-category

157

classification. Compared with other CNN architectures, our model obtains the best

overall classification performance. The feature analysis result shows that the spatial

distribution of the EEG frequency components (around 10 Hz and 50 Hz) plays a

primary role in discriminating depressive.

(3) The effectiveness of the forehead EEG to objectively discriminate depression

and assess depressive mood state is validated. The EEG signal of the forehead signal

channel can provide discrimination of the MDD at the level of multi-channel EEG

analysis. This provides an evidence to support the feasibility of the MDD prescreening

method based on forehead EEG signal in the ubiquitous environment. We continue to

use the portable EEG device to collect the forehead EEG from the depressives and

normal controls. The result shows that while the portable EEG device based

classification performance is inferior to the multi-channel EEG device based

classification performance, it still possesses good discrimination ability for the MDD

and normal controls. In addition, the objective depression evaluation method based on

forehead EEG data and self-rating tool for mood status is developed. This method

combining physiological sensor data with psychological data could provide new

insights into the pervasive and objective depression evaluation processes in the daily

life.

(4) Following by the WaaS architecture, the implementation of the service system

for depression treatment based on multi-modality data is illustrated. Taking the EEG and

Q-Log data as example, an exemplary application of knowledge sharing, which is

developed by using ontology technology and Jena inference subsystem, is given to


158

illustrate the preliminary work for annotating data and facilitating clinical users to

understand the meaning of quantitative analysis results. This example demonstrates the

feasibility of the WaaS architecture in combining the machine learning and ontology

technologies to apply the multi-modality data in the objective diagnosis and evaluation

services for depression.

8.2 Future Work

8.2.1 BI Based Depression Diagnosis and Evaluation

Since the BI methodology advocates considering the depression-related symptoms

comprehensively, the quantitative evaluations for other symptoms are the primary works

in the further study. Many studies show that the several physiology and behavior

symptoms of depression could be quantitatively evaluated. For example, the losing

ability of the brain to transmit signals and information that caused depressed mood

which makes a person influences their normal daily activities. Troubled sleep is one of

common depressive symptoms and could be quantitatively evaluated by the sleep

mattress which provides the functions of detecting sleep quality. The retardation and

agitation symptoms could be quantitatively rated by the objective rating instrument.

Besides the EEG signal, the other modality data should be also analyzed for supporting

the comprehensive depression evaluation.

In addition, the objective method for depression diagnosis and evaluation method

should not be merely validated by the clinician-rated scale or the self-rated tools, the


159

clinical experience and the criteria of depression should also be considered. Although

we have already given attentions about using DSM-5 and ICD-10 criteria to validate the

effectiveness of the objective methods, the specific method has not yet been designed.

Further, the quantitative results for the depression symptom could give an accurate

evaluation for the depression symptoms. The clinicians could comprehensively consider

the quantitative results and the clinical information of the inpatients to make a joint

analysis for the depression severity. The electronic medical record is suitable to provide

the clinical information, the joint analysis using the quantitative results for the

depression symptoms and the electronic medical record is worthy to be done in the

intensive work.

8.2.2 Depression Prescreening Based on the EEG Data

Several research points about the depression prescreening method based on the

forehead EEG data are worthy to be further studied:

(1) The robustness of the classification model for prescreening MDDs and NCs is

affected by the limited number of subjects. Because of affecting by the experiment cost

(high payment is needed for requiting the subjects and experimenters), clinician-

involved workload, and the EEG data sample source of MDDs, it is difficult to recruit

the subjects of depression and NC in a huge size. The size of depression and NC

subjects recruited in this study is imbalance, a further corporation with hospital to

recruit enough NC subjects is necessary.


160

(2) Although the spatial distribution of the EEG frequency components (around 10

Hz and 50 Hz) plays a primary role in discriminating depressive, we cannot claim that

the frequency components are definitely an effective biomarker for the MDD

discrimination. A further investigation should be executed to support the conclusion.

Besides, while the classification performance of EEG feature domains is discussed, the

effective biomarkers for depression are more cared by researchers. To achieve that,

grouped-analysis method based on the analysis of variation (ANOVA) statistical test is

suitable for comparing every specific feature of MDD and healthy groups, and discovers

which feature showing the most meaningful differences between the two groups.

(3) The ability of the CNN model to detect the features in spatiotemporal data

could assist in knowledge discovery. The result shows that the spatial distribution of the

two kinds of EEG frequency component (around 10 Hz and 50 Hz) might contain

depression-specific spatial differences in brain rhythms. We believe that the methods

and findings described in this study pave the way for a widespread application of deep

CNN for depression discrimination both in clinical applications and neuroscientific

research.

(4) The proposed machine learning methods can be implemented in a clinical

setting as a tool for objective depression discrimination using EEG signal. Furthermore,

the proposed discrimination system can be installed as a web-based application to be

used by non-specialist clinicians remotely. Once the EEG signals are obtained from the

patients, they will be sent to severs (located in the hospitals) in the cloud where the

proposed CNN model can be used to make the discrimination. The result can be


161

immediately sent to the clinic.

8.2.3 Quantitative Analysis for Depressive Mood Status

The quantitative analysis for depressive mood status based on the forehead EEG

and Q-Log data should be further studied in the following aspects:

(1) The EMA approach has several advantages over traditional cross-sectional

approaches in which an assessment is conducted from a large population sample at one

or a few points in time, the Q-Log data of 7 inpatients is absolutely insufficient to make

it being applied to the real clinical use. More subjects should be recruited so as to test

the clinical value of the Q-Log in the intensive study.

(2) In every LOPOCV, the constructed pervasive model is different for every

inpatient. A uniform model trained based on fixed dataset is more appreciate for

objectively rating the major depressive mood status. Corresponding to the Q-Log data,

the forehead EEG samples should also be further collected to search effective quantified

feature or feature set and build the uniform quantitative model.

(3) Because of our platform is being developed, the graphic application interface is

unable to be provided. The exemplary application for knowledge sharing given in the

Chapter 7 is not intuitive to illustrate the usage of data annotation. Platform

development for showing the advantage and effectiveness of ontology and quantitative

analysis method should continue to be executed in the following works.


162

8.2.4 Conjoint Analysis Based on the Multi-modality Data

In this dissertation, we mainly illustrated the work of the objective depression

discrimination and depressive mood state assessment based on the EEG data. The

mental disorder, which is one of the typical depressive symptoms, is objective evaluated

using forehead EEG and Q-Log data. The method and the related analysis results about

the other depressive symptoms quantitatively evaluated by other kind of quantitative

data should also be illustrated. Actually, we have already finished part of the work, such

as the evaluation of diary activity time based on behavior data, the analysis of the voice

speed during the clinical interview, and the concentration ability analysis based on the

portable EEG device. The work mentioned above would be sorted out in the future

work.

In addition, the depression is a mental illness that comprises several depressive

symptoms, different combinations of the depressive symptoms might indicate different

depression types. Because of one specific depressive symptom is objectively evaluated

by one specific quantitative data, the next step is to develop the conjoint analysis

method based on the quantitative results by the multi-modality data. The preliminary

idea to do that is to combine the clinical knowledges of depression since the existed

knowledges have clarified the relation between different depressive symptoms. Provide

the depression diagnosis and evaluation from the objective perspective could challenge

the traditional methods of depression treatment and explore new discoveries.


Bibliography

[1] R. H. Belmaker, and G. Agam. Major Depressive Disorder. New England

Journal of Medicine, 358(1), pp. 55-68, 2008.

[2] W. H. Organization. Depression (Fact Sheet N. 369). 2016.

[3] C. D. Mathers, and D. Loncar. Projections of global mortality and burden of

disease from 2002 to 2030. Plos Medicine, 3(11), pp. e442, 2006.

[4] J. Olesen, A. Gustavsson, M. Svensson et al. The economic cost of brain

disorders in Europe. European Journal of Neurology, 19(1), pp. 155, 2012.

[5] H. A. Whiteford, L. Degenhardt, J. Rehm et al. Global burden of disease

attributable to mental and substance use disorders: findings from the Global Burden of

Disease Study 2010. The Lancet, 382(9904), pp. 1575-1586, 2013.

[6] T. Singh, and M. Rajput. Misdiagnosis of Bipolar Disorder. Psychiatry

(Edgmont), 3(10), pp. 57-63, 2006.

[7] P. A. Vohringer, and R. H. Perlis. Discriminating Between Bipolar Disorder and

Major Depressive Disorder. Psychiatric Clinics of North America, 39(1), pp.1-10, 2016.

[8] T. E. Davison, D. M. Mellor, and D. Mellor. An examination of the "gold

standard" diagnosis of major depression in aged-care settings. American Journal of

Geriatric Psychiatry, 17(5), pp.359-367, 2009.

[9] J. Guidi, P. F.Bech, E. B. Paykel et al. The Clinical Interview for Depression: a

comprehensive review of studies and clinimetric properties. Psychotherapy &

Psychosomatics, 80(1), pp.10-27, 2010.

[10] J.X. Pan, J.J. Xia, F.L. Deng et al. Diagnosis of major depressive disorder

164

based on changes in multiple plasma neurotransmitters: a targeted metabolomics study.

Translational Psychiatry, 8(1), pp.130, 2018.

[11] C. Solomon, M. F. Valstar, R. K. Morriss et al. Objective Methods for Reliable

Detection of Concealed Depression. Frontiers in Ict, 2(5), pp. 5, 2015.

[12] N. Zhong, J. M. Bradshaw, J. Liu et al. Brain Informatics. IEEE Intelligent

Systems, 26(5), pp. 16-21, 2011.

[13] N. Zhong, S. S. Yau, J. Ma et al. Brain Informatics-Based Big Data and the

Wisdom Web of Things. IEEE Intelligent Systems, 30(5), pp. 2-7, 2015.

[14] N. Zhong, and J. Chen. Constructing a New-Style Conceptual Model of Brain

Data for Systematic Brain Informatics. IEEE Transactions on Knowledge and Data

Engineering, 24(12), pp. 2127-2142, 2012.

[15] G. M. Goodwin. Depression and associated physical diseases and symptoms.

Dialogues in Clinical Neuroscience, 8(2), pp. 259-265, 2006.

[16] Y. Li, Z. Wan, J. Huang, et al. A Smart Hospital Information System for Mental

Disorders. IEEE / Wic / ACM International Conference on Web Intelligence and

Intelligent Agent Technology. pp. 321-324, 2015.

[17] J. H. Chen, Ning Zhong. Research challenges and perspectives on Wisdom

Web of Things (W2T). The journal of Supercomputing, 64(3), pp. 862-882, 2013.

[18] S. Salle, J. Choueiry, D. Shah et al. Effects of Ketamine on Resting-State EEG

Activity and Their Relationship to Perceptual/Dissociative Symptoms in Healthy

Humans. Front Pharmacol, 7, pp. 348, 2016.

[19] C. M. Michel, and M. M. Murray. Towards the utilization of EEG as a brain

imaging tool. NeuroImage, 61(2), pp. 371-385, 2012.

[20] S. C. Leiser, J. Dunlop, M. R. Bowlby et al. Aligning strategies for using EEG

as a surrogate biomarker: a review of preclinical and clinical research. Biochemical

165

Pharmacology, 81(12), pp. 1408-1421, 2011.

[21] V. J. Knott. Quantitative EEG methods and measures in human

psychopharmacological research. Hum Psychopharmacol, 15(7), pp.479-498, 2015.

[22] A. H. Kemp, A. J. Rush, L. M. Williams et al. Improving the prediction of

treatment response in depression: integration of clinical, cognitive, psychophysiological,

neuroimaging, and genetic measures. Cns Spectr, 13(12), pp.1066-1086, 2008.

[23] A. F. Leuchter, I. A. Cook, S. P. Hamilton et al. Biomarkers to predict

antidepressant response. Current psychiatry reports, 12(6), pp. 553-562, 2010.

[24] A. F. Leuchter, A. M. Hunter, A. S. Korb et al. A new paradigm for the

prediction of antidepressant treatment response. Dialogues in Clinical Neuroscience,

11(4), pp.435-446, 2009.

[25] G. MacQueen. Neuroimaging and electrophysiology in predicting treatment

responsiveness in depression: bridging the lab-to-clinic divide?. Can J Psychiatry, 58(9),

pp.497-498, 2013.

[26] H. Alhaj, and R. H. Williams. The use of the EEG in measuring therapeutic

drug action: focus on depression and antidepressants. Journal of Psychopharmacology,

25(9), pp.1175-1191, 2011.

[27] A. Baskaran, and R. S. McIntyre. The neurobiology of the EEG biomarker as a

predictor of treatment response in depression. Neuropharmacology, 63(4), pp.507-513,

2012.

[28] A. F. Leuchter, A. Cook et al. Use of clinical neurophysiology for the selection

of medication in the treatment of major depressive disorder: the state of the evidence.

Clinical Eeg & Neuroscience, 40(2), pp.78-83, 2009.

[29] R. Gupta, and V. Lahan. Insomnia Associated with Depressive Disorder:

Primary, Secondary, or Mixed?. Indian Journal of Psychological Medicine, 33(2), pp.

166

123-128, 2011.

[30] J. S. Buyukdura, S. M. McClintock, and P. E. Croarkin. Psychomotor

retardation in depression: Biological underpinnings, measurement, and treatment.

Progress in neuro-psychopharmacology & biological psychiatry, 35(2), pp. 395-409,

2011.

[31] N.H. Liu, C.Y. Chiang, and H.C. Chu. Recognizing the Degree of Human

Attention Using EEG Signals from Mobile Sensors. Sensors, 13(8), pp. 10273-10286,

2013.

[32] NeuroSky. Available: https://store.neurosky.com.

[33] F. Wahle, T. Kowatsch, E. Fleisch et al. Mobile Sensing and Support for People

With Depression: A Pilot Trial in the Wild. JMIR mHealth and uHealth, 4(3), pp. e111,

2016.

[34] C. I. Lo, S. S. Chang, J. P. Tsai et al. Evaluation of the Accuracy of ECG

Captured by CardioChip through Comparison of Lead I Recording to a Standard 12-

Lead ECG Recording Device. Acta Cardiologica Sinica, 34(2), pp.144-151, 2018.

[35] R. W. Lam, S. H. Kennedy, R. S. McIntyre et al. Cognitive Dysfunction in

Major Depressive Disorder: Effects on Psychosocial Functioning and Implications for

Treatment. Canadian Journal of Psychiatry. Revue Canadienne de Psychiatrie, 59(12),

pp. 649-654, 2014.

[36] F. W. Lohoff. Overview of the Genetics of Major Depressive Disorder. Current

psychiatry reports, 12(6), pp. 539-546, 2010.

[37] J. Chen, J. Ma, N. Zhong et al. WaaS: Wisdom as a Service. IEEE Intelligent

Systems, 29(6), pp. 40-47, 2014.

[38] Brainclinics. Depression history of EEG and QEEG findings. Avaliable:

https://www.brainclinics.com/depresssion-history-of-eeg-researchx.

167

[39] D.J. Kim, A. R. Bolbecker, J. Howell et al. Disturbed resting state EEG

synchronization in bipolar disorder: A graph-theoretic analysis. NeuroImage: Clinical,

2, pp. 414-423, 2013.

[40] H. Helgadóttir, G. Baldursson et al. Electroencephalography as a clinical tool

for diagnosing and monitoring attention deficit hyperactivity disorder: a cross-sectional

study. BMJ Open, 5(1), pp. e005500, 2015.

[41] S. B. Freitas, A. A. Marques, M. C. Bevilaqua et al. Electroencephalographic

findings in patients with major depressive disorder during cognitive or emotional tasks:

a systematic review. Revista Brasileira De Psiquiatria, 38(ahead), pp.338-346, 2016.

[42] I. Arel, D. C. Rose, and T. P. Karnowski. Deep Machine Learning - A New

Frontier in Artificial Intelligence Research. Computational Intelligence Magazine IEEE,

5(4), pp. 13-18, 2010.

[43] R. T. Schirrmeister, J. T. Springenberg, L. D. Fiederer et al. Deep learning with

convolutional neural networks for EEG decoding and visualization. Human Brain

Mapping, 38(11), pp. 5391-5420, 2017.

[44] M. P. van, S. Olbrich, and M. Arns. Predicting sex from brain rhythms with

deep learning. Sci Rep, 8(1), 2018.

[45] X. Li, D. Zhang, P. Hou, B. Hu. Deep fusion of multi-channel

neurophysiological signal for emotion recognition and monitoring. International Journal

of Data Mining and Bioinformatics, 18(1), pp. 1-27, 2017.

[46] M. Bairy, S. Bhat, L. Wei Jie Eugene et al. Automated Classification of

Depression Electroencephalographic Signals Using Discrete Cosine Transform and

Nonlinear Dynamics. Journal of Medical Imaging & Health Informatics, 5(3), pp.635-

640, 2015.

168

[47] B. Hosseinifard, M. Moradi, and R. Rostami. Classifying depression patients

and normal subjects using machine learning techniques and nonlinear features from

EEG signal. Comput Methods Programs Biomed. 109(3), pp. 339-345, 2012.

[48] D. M. Tucker, C. E. Stenslie, R. S. Roth et al. Right frontal lobe activation and

right hemisphere performance. Decrement during a depressed mood. Arch Gen

Psychiatry, 38(2), pp. 169-174, 1981.

[49] D. P. Kan, and P. F. Lee. Decrease alpha waves in depression: An

electroencephalogram(EEG) study. pp. 156-161, 2015.

[50] J. L. Stewart, J. A. Coan, D. N. Towers et al. Resting and task-elicited

prefrontal EEG alpha asymmetry in depression: support for the capability model.

Psychophysiology, 51(5), pp. 446-55, 2014.

[51] U. R. Acharya, V. K Sudarshan, H. Adeli et al. A Novel Depression Diagnosis

Index Using Nonlinear Features in EEG Signals. Eur Neurol. 74(1-2) , pp. 79-83, 2015.

[52] S. Akdemir Akar, S. Kara, S. Agambayev et al. Nonlinear analysis of EEGs of

patients with major depression during different emotional states. Computers in Biology

& Medicine, 67(C), pp.49-60, 2015.

[53] T. Takahashi. Complexity of spontaneous brain activity in mental disorders.

Progress in Neuro-Psychopharmacology and Biological Psychiatry, 45(3), pp. 258-266,

2013.

[54] S. Stober, A. Sternin, A. M. Owen et al. Deep Feature Learning for EEG

Recordings. Computer Science, 165, pp. 23–31, 2016.

[55] F. C. Morabito, M. Campolo, N. Mammone et al. Deep Learning

Representation from Electroencephalography of Early-Stage Creutzfeldt-Jakob Disease

and Features for Differentiation from Rapidly Progressive Dementia. International

169

Journal of Neural Systems, 27(2), pp. 1650039, 2017.

[56] S. Chambon, M. N. Galtier, P. J. Arnal et al. A Deep Learning Architecture for

Temporal Sleep Stage Classification Using Multivariate and Multimodal Time Series.

IEEE Transactions on Neural Systems & Rehabilitation Engineering, 26(4), pp. 758-

769, 2018.

[57] D. F. Wulsin, R. Gupta et al. Modeling electroencephalography waveforms

with semi-supervised deep belief nets: fast classification and anomaly measurement.

Journal of Neural Engineering. 8(3), pp.036015, 2011.

[58] A. R. Johansen, J. Jin, T. Maszczyk et al. Epileptiform spike detection via

convolutional neural networks. IEEE International Conference on Acoustics. pp. 754-

758, 2016.

[59] U. R. Acharya, S. L. Oh, Y. Hagiwara et al. Deep convolutional neural network

for the automated detection and diagnosis of seizure using EEG signals. Computers in

Biology & Medicine. 100, 2017.

[60] E. J. Nestler, M. Barrot, R. J. DiLeone et al. Neurobiology of Depression.

Neuron, 34(1), pp. 13-25, 2002.

[61] S. C. Roh, E. J. Park, M. Shim et al. EEG beta and low gamma power

correlates with inattention in patients with major depressive disorder. Journal of

Affective Disorders. 204, pp.124-130, 2016.

[62] Y. Li, C. Kang, Z. Wei et al. Beta oscillations in major depression – signalling a

new cortical circuit for central executive function. Scientific Reports. 7(1), pp. 18021,

2017.

[63] R. Kerestes, C. G. Davey, K. Stephanou et al. Functional brain imaging studies

of youth depression: A systematic review. NeuroImage, 4, pp. 209-231, 2014.

[64] O. Stelt, and A. Belger. Application of Electroencephalography to the Study of

170

Cognitive and Brain Functions in Schizophrenia. Schizophrenia Bulletin, 33(4), pp.

955-970, 2007.

[65] S. Olbrich, and M. Arns. EEG biomarkers in major depressive disorder:

Discriminative power and prediction of treatment response. International Review of

Psychiatry, 25(5), pp. 604-618, 2013.

[66] M. Mohammadi, F. Al-Azab, B. Raahemi et al. Data mining EEG signals in

depression for their diagnostic value. BMC Med Inform Decis Mak, 15, pp. 108, 2015.

[67] J. M. Rogers, S. J. Johnstone, A. Aminov et al. Test-retest reliability of a single-

channel, wireless EEG system. International Journal of Psychophysiology. 106, pp. 87-

96, 2016.

[68] C.T. Lin, C.H. Chuang, Z. Cao et al. Forehead EEG in support of future

feasible personal healthcare solutions: Sleep management, headache prevention, and

depression treatment. IEEE Access, 5, pp. 10612-10621, 2017.

[69] H. Cai, J. Han, Y. Chen et al. A Pervasive Approach to EEG-Based Depression

Detection. Complexity. 3, pp.1-13, 2018.

[70] E. Walter, and P. Dassonville. Activation in a Frontoparietal Cortical Network

Underlies Individual Differences in the Performance of an Embedded Figures Task.

PLOS ONE, 6(7), pp. e20742, 2011.

[71] S. Tekin, and J. L. Cummings. Frontal–subcortical neuronal circuits and

clinical neuropsychiatry: An update. Journal of Psychosomatic Research. 53(2), pp.

647-654, 2002.

[72] J. I. Ekandem, I. Davis et al. Evaluating the ergonomics of BCI devices for

research and experimentation. Ergonomics, 55(5), pp.592-598, 2012.

[73] S. J. Johnstone, J. M. Blackman et al. EEG from a single-channel dry-sensor

recording device. Clinical Eeg & Neuroscience, 43(2), pp.112, 2012.

171

[74] B. Products. Avaliable: https://www.brainproducts.com/.

[75] X. Li, B. Hu, S. Sun et al. EEG-based mild depressive detection using feature

selection methods and classifiers. Comput Methods Programs Biomed. 136, pp. 151-61,

2016.

[76] M. Bachmann, et al. Methods for classifying depression in single channel EEG

using linear and nonlinear signal analysis. Comput Methods Programs Biomed, 155, pp.

11-17, 2018.

[77] M. Ahmadlou, H. Adeli, and A. Adeli. Fractality analysis of frontal brain in

major depressive disorder. International Journal of Psychophysiology, 85(2), pp. 206-

211, 2012.

[78] Y. Guo, H. Zhang, C. Pang. EEG-based mild depression detection using multi-

objective particle swarm optimization. IEEE on Control and Decision Conference. Pp.

4980-4984, 2017.

[79] T. Tekin Erguzel, C. Tas, and M. Cebi. A wrapper-based approach for feature

selection and classification of major depressive disorder-bipolar disorders. Computers in

Biology & Medicine, 64, pp.127-137, 2015.

[80] W. Mumtaz, L. Xia, M. A. Mohd Yasin et al. A wavelet-based technique to

predict treatment outcome for Major Depressive Disorder. PLOS ONE. 12(2), pp.

e0171409, 2017.

[81] S. Devaraj, and P. Sathurappan. An Efficient Feature Subset Selection

Algorithm for Classification of Multidimensional Dataset. The Scientific World Journal.

18(3), pp. 1-9, 2015.

[82] M. A. Mendez, R. Zuluaga et al. Complexity analysis of spontaneous brain

activity: effects of depression and antidepressant treatment. Journal of

172

Psychopharmacology, 26(5), pp.636-643, 2012.

[83] K. M. Smith, P. F. Renshaw, and J. Bilello. The diagnosis of depression: current

and emerging methods. Comprehensive psychiatry, 54(1), pp. 1-6, 2013.

[84] A. Markowetz, K. Błaszkiewicz, C. Montag et al. Psycho-Informatics: Big

Data shaping modern psychometrics. Medical Hypotheses, 82(4), pp. 405-411, 2014.

[85] A. Lasalvia, S. Zoppei, T. Van Bortel et al. Global pattern of experienced and

anticipated discrimination reported by people with major depressive disorder: a cross-

sectional survey. The Lancet, 381(9860), pp. 55-62, 2013.

[86] T. Guo, Y. T. Xiang, L. Xiao et al. Measurement-Based Care Versus Standard

Care for Major Depression: A Randomized Controlled Trial With Blind Raters.

American Journal of Psychiatry, 172(10), pp. 1004, 2015.

[87] S. E. Crowell, B. R. Baucom, M. Yaptangco et al. Emotion dysregulation and

dyadic conflict in depressed and typical adolescents: evaluating concordance across

psychophysiological and observational measures. Biological Psychology, 98(9), pp. 50-

58, 2014.

[88] D. Howe, M. Costanzo, P. Fey et al. Big data: The future of biocuration.

Nature, 455(7209), pp. 47-50, 2008.

[89] N. J. Schork. Personalized medicine: Time for one-person trials. Nature,

520(7549), pp. 609-11, 2015.

[90] J. A. Bilello. Seeking an objective diagnosis of depression. Biomark Med.

10(8), pp. 861-75, 2016.

[91] P. J. Deldin, and P. Chiu. Cognitive restructuring and EEG in major depression.

Biol Psychol. 70(3), pp. 141-51, 2005.

[92] N. Cummins, S. Scherer, J. Krajewski et al. A review of depression and suicide

risk assessment using speech analysis. Speech Communication. 71, pp. 10-49, 2015.

173

[93] M. Sung, C. Marci, and S. Pentland. Objective Physiological and Behavioral

Measures for Identifying and Tracking Depression State in Clinically Depressed

Patients. 2010.

[94] D. M. McNair, L. F. Droppleman, M. Lorr et al. Profile of Mood States.

Educational and Industrial Testing Service. 1971.

[95] S. L. Curran, M. A. Andrykowski, and J. Studts, Short form of the Profile of

Mood States (POMS-SF). Psychometric information. 1995.

[96] F. J. Blaauw, H. M. Schenk, B. F. Jeronimus et al. Let’s get Physiqual – An

intuitive and generic method to combine sensor technology with ecological momentary

assessments. Journal of Biomedical Informatics. 63, pp. 141-149, 2016.

[97] M. aan het Rot, K. Hogenelst, and R. A. Schoevers. Mood disorders in

everyday life: a systematic review of experience sampling and ecological momentary

assessment studies. Clin Psychol Rev. 32(6), pp. 510-23, 2012.

[98] P. C. M. Molenaar. A Manifesto on Psychology as Idiographic Science:

Bringing the Person Back Into Scientific Psychology, This Time Forever. Measurement:

Interdisciplinary Research and Perspectives, 2(4), pp. 201-218, 2004.

[99] E. Hamaker. Why researchers should think "within-person": A paradigmatic

rationale. Handbook of Research Methods for Studying Daily Life. 3, pp.43-61,2012.

[100] X. Li, B. Hu, J. Shen et al. Mild Depression Detection of College Students: an

EEG-Based Solution with Free Viewing Tasks. J Med Syst. 39(12), pp. 187, 2015.

[101] S. Potvin, G. Charbonneau, R.P. Juster et al. Self-evaluation and objective

assessment of cognition in major depression and attention deficit disorder: Implications

for clinical practice. Comprehensive Psychiatry, 70, pp. 53-64, 2016.

[102] D. V. Iosifescu, S. Greenwald, P. Devlin et al. Frontal EEG predictors of

treatment outcome in major depressive disorder. Eur Neuropsychopharmacol. 19(11),

174

pp. 772-7, 2009.

[103] M. Bares, M. Brunovsky, T. Novak et al. QEEG Theta Cordance in the

Prediction of Treatment Outcome to Prefrontal Repetitive Transcranial Magnetic

Stimulation or Venlafaxine ER in Patients With Major Depressive Disorder. Clin EEG

Neurosci. 46(2), pp. 73-80, 2015.

[104] V. E. M. Griffeth, J. E. Perthen, and R. B. Buxton. Prospects for Quantitative

fMRI: Investigating the Effects of Caffeine on Baseline Oxygen Metabolism and the

Response to a Visual Stimulus in Humans. NeuroImage. 57(3), pp. 809-816, 2011.

[105] J. Chen, J. Han, Y. Deng et al. Wisdom as a Service for Mental Health Care.

IEEE Transactions on Cloud Computing. PP(99), pp. 1-1, 2015.

[106] S. Berrouiguet, M. M Perez-Rodriguez, M. Larsen et al. From eHealth to

iHealth: Transition to Participatory and Personalized Medicine in Mental Health.

Journal of Medical Internet Research. 20(1), pp.e2, 2018.

[107] Y. S. Chang, W. C. Hung, and T. Y. Juang. Depression Diagnosis Based on

Ontologies and Bayesian Networks. pp. 3452-3457, 2013.

[108] T. R. Gruber. Toward principles for the design of ontologies used for

knowledge sharing?. International Journal of Human-Computer Studies, 43(5), pp. 907-

928, 1995.

[109] H. Mi, and P. D. Thomas. Ontologies and Standards in Bioscience Research:

For Machine or for Human. Frontiers in Physiology. 2, pp. 5, 2011.

研究業績リスト- Publications

研究分野(Research Interests)

人工知能、機械学習、信号処理、画像処理、データマイニング

(Artificial Intelligence, Machine Learning, Signal Processing, Image Processing, Data

Mining)

博士論文題目(The Topic of the Ph.D. Thesis)

Brain Informatics Based Objective Diagnosis and Evaluation for

Major Depressive Disorder

(脳情報学に基づくうつ病の客観的な診断や評価に関する研究)

1. International Journal Papers

Zhijiang Wan, Yishan He, Ming Hao, Jian Yang, Ning Zhong. M-AMST: an Automatic

3D Neuron Tracing Method Based on Mean Shift and Adapted Minimum Spanning

Tree. BMC Bioinformatics. 2017 Mar 29;18(1):197. doi: 10.1186/s12859-017-1597-9.

Zhijiang Wan, Ning Zhong, et al. WaaS Architecture Driven Depressive Mood Status

Quantitative Analysis Based on Forehead EEG and Self-Rating Scale. Brain

Informatics, Springer, 2018 Dec 08; 5:15. doi: 10.1186/s40708-018-0093-y.

176

Zhijiang Wan, Hao Zhang, Haiyan Zhou, Jie Yang, Ning Zhong. Single Channel EEG

Based Machine Learning Method for Prescreening Major Depressive Disorder.

International Journal of Information Technology and Decision Making, World

Scientific, ISSN: 0219-6220, 2018. (条件付き採用)

Zhijiang Wan, Jianhui Chen, Haiyan Zhou, Ning Zhong. Objective evaluation for

depression using multi-modality data: a review, 2018. (準備中)

Zhijiang Wan, Jiajin Huang, Haiyan Zhou, Jie Yang, Ning Zhong. HybridEEGNet: A

Convolutional Neural Network for EEG Feature Learning and Depression

Discrimination, 2018. (準備中)

Zhijiang Wan, Haiyan Zhou, Jie Yang, Ning Zhong. Reading Speech Based Non-verbal

Paralinguistic Biomarkers Mining for Quantitatively Evaluating Depressive Emotions,

2018. (準備中)

2. International Conference Papers

Zhijiang Wan, Qiang He, Haiyan Zhou, Jie Yang, Jianzhuo Yan, Ning Zhong. A

Quantitative Analysis Method for Objectively Assessing the Depression Mood Status

Based on Portable EEG and Self-rating Scale. In: Zeng Y. et al. (eds) Brain Informatics.

BI 2017. Lecture Notes in Computer Science, vol 10654. Springer, Cham. doi:

Maebashi Institute of Technology, Doctor Dissertation of Engineering Zhijiang Wan: BI Based Objective Diagnosis And Evaluation for Major Depressive Disorder

177

10.1007/978-3-319-70772-3_21.

Zhijiang Wan, Ning Zhong, Jianhui Chen, Haiyan Zhou, Jie Yang, Jianzhuo Yan. A

Depressive Mood Status Quantitative Reasoning Method Based on Portable EEG and

Self-rating Scale. In Proceedings of the International Conference on Web Intelligence

(WI '17). ACM, New York, NY, USA, 389-395. doi:10.1145/3106426.3106471.

Zhijiang Wan, Yishan He, Ming Hao, Jian Yang, Ning Zhong. An Automatic Neuron

Tracing Method Based on Mean Shift and Minimum Spanning Tree. In: Ascoli G.,

Hawrylycz M., Ali H., Khazanchi D., Shi Y. (eds) Brain Informatics and Health. BIH

2016. Lecture Notes in Computer Science, vol 9919. Springer, Cham. doi: 10.1007/978-

3-319-47103-7_4.

3. Other Publications in China

(a) 公開、出願中の特許（中国国内）

Zhijiang Wan, Ning Zhong. Hierarchical Mechanism Based Human Body Behavior

Recognition Method, 2016. (In Chinese)

Zhijiang Wan, Ning Zhong. Electronic POMS Scale Based Self-rating System for

Depressive Mood Status, 2016. (In Chinese)

Zhijiang Wan, Ning Zhong. Brain Informatics Methodology Based Multi-modality data


178

collection and analysis system, 2016. (In Chinese)

Zhijiang Wan, Ning Zhong. Human Body Movement Data Based Quantitative

Evaluation Method for Mood Status, 2016. (In Chinese)

(b) ソフトウェアの著作権（中国国内）

Zhijiang Wan, Ning Zhong. A Behavior Displaying, Recognition, and Annotation

Software for The Long Term Behavior Data, 2017. (In Chinese)

Zhijiang Wan, Ning Zhong. A Self-rating Scale for mood status based on Android

System, 2017. (In Chinese)

Zhijiang Wan, Ning Zhong. An Android multiple Bluetooth communication software

based on BLE4.0 Protocol, 2017. (In Chinese)

Zhijiang Wan, Ning Zhong. A Matlab Based Artifact Manually Removing Software for

the EEG Data of the Portable Device, 2017. (In Chinese)


gair.media.gunma-u.ac.jp€¦ · web viewevery patient, who is willing to participate in this...

Documents