mot robust cross-subject klassificering av...
Post on 31-Jul-2020
1 Views
Preview:
TRANSCRIPT
INOM EXAMENSARBETE TEKNIK,GRUNDNIVÅ, 15 HP
, STOCKHOLM SVERIGE 2019
Mot robust cross-subject klassificering av electroencephalogram (EEG) baserad brain-computer interfacing (BCI):En genomförbarhetsstudie
SHUAI WU
KTHSKOLAN FÖR TEKNIKVETENSKAP
INOM EXAMENSARBETE TEKNIK,GRUNDNIVÅ, 15 HP
, STOCKHOLM SVERIGE 2019
Towards robust cross-subject classification of electroencephalogram (EEG) patterns for brain-computer interfacing (BCI):A feasibility study
SHUAI WU
KTHSKOLAN FÖR TEKNIKVETENSKAP
Abstract
A brain-computer interface (BCI) is a system that enables the subject to send
commands with merely brain activity. Such interface is important for people
affected by multiple motor disabilities, where BCI made it possible for machine
to better understand the patient and thus fulfill their demands.
The BCI variante that base on motor imagery require classification on subject’s
brain activity on imagining movement of body parts, which could be done by
using different classifier. There exists multiple difficulty when developing such an
system, one of them is generalization of trained models, this accuracy of trained
model could not be guaranteed when using on a different subject or in a different
session. Even within the same session, the classification result is not optimal
due to brain activity’s non-stationary nature. This paper tackle the problem of
intersubject classification with adaptive importance weighted linear discriminant
analysis(AIWLDA), which shows promising result on both intersession and intra-
session classification of offline EEG based BCI. This research has shown that there
exist subject pairs with inter-subject generalizable potential, more pairs could be
revealed by using AIWLDA, but this method fail to robustly classify across every
subject-pairs.
Keywords
covariate shift, brain-computer interface, motor imagery, EEG,
inter-subject
i
Sammanfattning
Brain-computer interface(BCI) är ett system där man kan skicka kommandon till
dator med bara hjärnaktivitet. En sådan system är viktigt för människor lider av
flera motorisk funktionshinder, då maskinen skulle kunna förbättra patienters liv
genom att uppfylla deras behov.
Denna rapport fokusera på en variant av BCI, kallas motor imagery based
BCI, vilken basera på att klassificera försökspersons hjärnaktivitet då han/hon
tänka sig att röra sin kroppsdelar. Det finns flera svårighet för att bygga en
fungerande system, en av de är generalisering av tränad model. En tränad model
garanti inte exakthet på annat försöksperson eller annat session. Även i samma
session, kan model ger sämre resultat på grund av hjärnaktiviteten nonstationary
natur. Denna rapport försöka hantera inter-subject klassificering problem
med adaptive importance weighted linear discriminant analysis(AIWLDA), som
gav bra resultat i både intra-session och inter-session klassificering av offline
EEG baserad BCI. Det kommer visa i resultat att det finns försökspersons par
där inter-subject generalisering är möjligt och AIWLDA kan avslöja mer av
sådana par, men misslyckas att bevisa om det denna egenskap finns mellan alla
försöksperson.
Nyckelord
covariate shift, brain-computer interface, motor imagery, EEG,
inter-subject
ii
Contents
1 Introduction 11.1 Background introduction . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 Aim . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.3 Problem formulation . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.4 Delimitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.5 Outline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
2 Background 52.1 Electroencephalography(EEG) . . . . . . . . . . . . . . . . . . . . . 5
2.2 Brain-computer interface(BCI) . . . . . . . . . . . . . . . . . . . . . 5
2.3 Previous works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
2.4 LDA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
2.5 Covariate-shift adaptation of LDA . . . . . . . . . . . . . . . . . . . 7
3 Method 93.1 Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
3.2 Accuracy measurement . . . . . . . . . . . . . . . . . . . . . . . . . 11
3.3 Sub-problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
4 Result 14
5 Discussion 175.1 Weakness . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
5.2 Strength . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
5.3 Future work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
5.4 Outlook . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
6 Conclusion 20
iii
1 Introduction
This section provide an overview within the topic of brain computer interface and
formulate the research question.
Abbreviations
· BCI: Brain-computer interface
· EEG: Electroencephalography
· LDA: Linear discriminant analysis
· AIWLDA: Adaptive importance weighted LDA
· BP: Band Power
·MI: Motor imagery
1.1 Background introduction
Brain-computer interface, allows the user to communicate with machines,
providing a new way of communication and control (Wolpaw & Mcfarland
1994). This new channel of control could serve multiple purposes e.g. post-
stroke rehabilitation (Prasad & Herman & Coyle & Mcdonough & Crosbie
2009), creates new means of communication for people who suffer from motor
disabilities(Hoffmann & Vesin & Ebrahimi & Diserens 2008) or even controlling
video games (Van de Laar & Gürkök & Plass-Oude Bos & Poel & Nijholt
2013).
In order to control a BCI, the user must produce a brain activity pattern
recognizable by the system. Most of the existing BCI relies on either
regression(Mcfarland &Wolpaw 2005) or classification(Pfurtscheller &Neuper &
Flotzinger & Pregenzer 1997). Themost commonway is to utilize a classifier(Lotte
& Congedo & Lécuyer & Lamarche & Arnaldi 2007). This identification process is
done mainly by means of training the subject to create a specific brain activity
pattern, at the same time, an adaptation of classifier is introduced, where it
1
calibrates its model to match the subject’s brain activity pattern(Lotte et al.
2007).
1.2 Aim
Although many of the classification algorithms works fine as it is within the same
BCI session for the same subject, the statistical distribution of the data collected
from BCI varies across both session and subjects, which limits the transferability
of the training data and the trained model across subjects and session (Jayaram
et al. 2015). This inconsistency makes each previous model unusable once a new
session is started or a new subject is introduced, which results in slow calibration
time prior to each session. There have been many studies with the means of
decreasing number of calibration trials needed, which, aside from proposing
better classifiers, could be summarized by two types of approaches: The first
approach is to better utilize calibration data, by either extracting better features
(Wang & Gao, S & Gao, X 2005; Boostani & Moradi 2005) or better utilization
of these features (Li & Guan & Zhang & Ang & 2014; Sugiyama et al. 1996). The
second approach is to make use of existing data, extracting generalized features
from earlier data obtained in other sessions or even other subjects (Bolagh &
Shamsollahi & Jutten & Congedo 2016; Shenoy & Miller & Ojemann & Rao
2008).
Although generalized features exist in other subjects, it is not guaranteed that
these features from every subject will give positive contribution (Shenoy &
Miller & Ojemann & Rao 2008). The aim of this study is therefore to find an
algorithm that robustly classifies across every subject, which will result in fast or
no calibration time for each new installation, making BCI open to everyone in
need.
1.3 Problem formulation
The inconsistency of BCI classificationmainly due to the test and training samples
does not follow the same probability distribution caused by the non-stationary
2
nature of brain activities. (Klonowski 2009). Covariate shift refers to the change
in the distribution of the input variable in the training and testing sample.
Classifier adaptive importance weighted linear discriminant analysis, also known
as AIWLDA proposed by Sugiyama et al(2007), could robustly classify samples
from a different probability distribution than the test samples.
Non-stationary nature of brain activity is the cause of covariate shift when doing
intra-subject classification, non-stationary meaning probability distribution a
sample change with time (Klonowski 2009). This nature is not the primary
concern of cross-subject classification, where the samples were collected from two
or more different sample set, i.e. subjects. Although covariate shift could still be a
major problem on the topic since the subjects share similar physiologic structure
i.e. are all humans. The research question can, therefore, formulate as such: Is it
possible to classifyMI tasks robustly across every subject pairs by using covariate-
shift adaptation of classifiers? This will help us find out whether the difficulty of
intersubject classification lies on covariate shift, thus showingmore insight on the
topic.
1.4 Delimitations
Due to the limitation of time, the scope of this study is limited to examine a
single linear classifier, i.e. LDA. Although many classifiers does have better
accuracy compared with LDA (Lotte et al. 2007) due to brain activity’s non-linear
nature (Klonowski 2009), LDA is easy to implement and does show a reasonable
high accuracy on classifying task of discriminating between left- and right- hand
motion imagination (Boostani & Moradi 2005), therefore is chosen to be studied
in this project.
Subject-specific frequencies bands are not investigated in this study, which is
rather important for motor imagery (MI) BCI. Investigate it will increase the
performance of the classification task (Suk & Lee 2011)
Data from 10 sessions of MI experiment was investigated, with 2 sessions for each
subject, with a total number of 5 subjects. Each session contains around 140 trials
and each trial consist of 4 seconds of EEG-signal.
3
1.5 Outline
In Chapter2 (Background), the reader can expect to learn about the concepts
introduced in Chapter1 (Introduction) like BCI and EEG, in a more in-depth
manner, followed by an overview of methods used in this project and data
processing. In Chapter3 (Method), the layout of this project will be presented
to reader, giving insight on how the experiment and results are formulated.
The research question is answered along with multiple worth noting results in
Chapter4 (Result). In Chapter5 (Discussion), the implication of the result is
discussed, followed by the analysis of weakness and strength of this method,
hoping to show a direction to future studies.
4
2 Background
In this section, the reader will get to understand multiple terms on the topic. The
method used in the following chapter is also be presented.
2.1 Electroencephalography(EEG)
Electroencephalography (EEG), is a monitoring method to record the electrical
activity of the brain, it usually gathers data from electrodes placed on the subject’s
scalps (Wolpaw & Mcfarland 1994). These EEG signals collected on the human
scalp are a reflection of corresponding activities in upper layers of the brain cortex
below the scalp surface (Vidal 1973). Much research has indicated that human has
the ability to manipulate a variety of EEG phenomena, which implies multiple
possibilities for EEG based BCI (Travis & Kondo & Knott 1975; Mcfarland & A.
Miner & Vaughan &Wolpaw 2000).
2.2 Brain-computer interface(BCI)
As stated in the introduction section, BCI enables the user to interact with
machines using brain activities, this could be done by letting the system identify
patterns of brain activity relevant to commands, which is a task usually given to
the classifiers.
The performance of the system depends on both features extracted from the
EEG signal and the classifier implemented(Lotte et al 2007). Where features are
data extracted from original data by reducing irrelevant parts, it’s intended to be
informative and in some cases lead to better human interpretations(Wikipedia
2019b).
Different features e.g. Power spectral density (PSD) (Kim & Sun & Liu & Wang &
Paek 2018) or BandPower (BP) (Pfurtscheller & Neuper,& Flotzinger & Pregenzer
1997) are extracted from the original data in a phase called preprocessing. Usage
of the different feature depends on the method chosen, therefore one can not
simply say that one feature yield better performance than others.
5
This project is built on so-called motor imagery (MI) BCI, where the subject is
requested to imaginemoving specific body partswhen instructed, while electrodes
placed along the subject’s scalp record the EEG signal. each of such instruction is
called a trial and each electrode is called a channel, with specific names depending
on where it’s located (figure 2.1).
Figure 2.1: Electrode placement international 10-20 system
2.3 Previous works
There have been numerous attempts on the cross-subject classification of EEG
based BCI, these attempts mainly try to adapt the existing model to decrease
calibration time. For instance, a study done by Lu and Zhang (2009) has shown
that for P300 speller, a BCI based on decisionmaking, utilizing a so-called subject
independent model learned by offline samples could drastically decrease the
number of calibration trials needed. Similar studies concerning MI, also shown a
positive result(Reuderink & Farquhar & Poel & Nijholt 2011; Jayaram & Alamgir
& Altun & Schölkopf & Grosse-Wentrup 2015).
There have been fewer studies on cross-subject classification of MI tasks without
using any labeled data from the test sample. Presumably due to that not
all training samples from different subjects may improve the performance of
cross-subject classification (Bolagh & Shamsollahi & Jutten & Congedo 2016).
Cross-subject classification of MI task require either subject selection so that
6
only subjects with positive contribution are used as the training set(Bolagh &
Shamsollahi & Jutten & Congedo 2016), whereas others report large variance
on accuracy between different subjects, when using inter-subject generalized
features(Shenoy & Miller & Ojemann & Rao 2008)
2.4 LDA
Earlier studies have shown that Linear discriminant analysis(LDA), a linear
classifier, which discriminates between two classes, is reasonably accurate on
classifying MI tasks (Herman, 2015). The classifier tries to find a line, where
the labeled samples are separated by the origin when projected on the line. The
unlabeled samples could then be projected on the line, and each observation are
labeled depending on the position of the projection compare with originn.
When classifying a set of offline samples using 4-fold cross-validation, LDA shows
similar average accuracy for intra-session(62.7%) and inter-session classification
(60.2%), but is lacking in an inter-subject(<50%) classification overall, except for
some specific subject pairs. Although the accuracy is not as high as the study done
by Herman et al. (2015) due to no subject-specific parameters was investigated
in this study, but the result can still serve the purpose of comparison with the
covariate-shift adaptation of LDA.
2.5 Covariate-shift adaptation of LDA
Assume the ratio of the test and training probability density function is finite and
known:P1(x)
P2(x)
Where the P denotes the probability density function of the respective sample
set and x denotes the input. This expression is known as importance, first
introduced by Fishman(1996) for importance sampling. A method introduced
by Sugiyama(2007) has shown that the importance could be used to address
covariate-shift problems in machine learning problems.
7
covariate-shift refer to the effect, where the change in probability distribution
presents in the training and test data. AIWLDA is a modified LDA classifier that
taken in to account of this by utilizing the importance, making this classifier more
accurate when encounter covariate-shift.
AIWLDA has a model of:
f(x, θ) = θ0 +∑i
θixi
Where θ is learned as following:
θ = argminθ[n∑
i=1
1
n((
Ptest(xi)
Ptrain(xi)
)λ(f(xi, θ)− yi)2]
The classifying result, or the labels are then obtained by:
u = sgn(f(x, θ))
Here eachxi and yi pairs denotes one labeled observation, wherexi is the input and
yi indicates its label. The importance is between the testing and training input’s
probability density function, which the input in importance expression is just the
training input of each observation.
Note that λ is the parameter that controls the tradeoff between accuracy and
precision(Sugiyama, Krauledat & Muller 2007), known as the bias-variance
tradeoff(Lotte et al 2007). Model selection is needed to choose a suitable λ. Worth
noting is that when λ = 0, AIWLDA is no other than the normal LDA.
Note that this adaptation of LDA does not address any potential model error,
therefore other classifiers such as Gaussian support vector machines(GSVM) with
better performance onMI tasks (Lotte et al. 2007)might still give higher accuracy
when the difference between distribution of test and training input is little.
8
3 Method
This section will providemore insight on both data andmethod used in this study,
an evaluation on measures of results are also be presented.
3.1 Data
In this project, the subject’s EEG pattern is recorded by 2 electrodes, placed on the
C3, C4 channels according to the international 10-20 system (Figure 2.1), which
corresponding to the hand area in M1 (Wang & Gao & Hong & Gao 2010). For
every four seconds, the subjects are instructed to imagine moving either right or
left hand, each of such an instruction is called a trial and each session consists
around 140 trials.
3.1.1 Preprocessing
This study make use of the feature called band power(BP). In order to obtain
this feature, raw EEG signal recorded from each session needed to be processed,
extracting band power from it with respect to frequencies. This could be achieved
by using Fourier transformation on different time intervals that are reasonably
small, in this study, the time interval is chosen to be ¼ of a second. This creates
one 3d-array of trial × Time × Frequency for each channel respectively, where
frequency spans from 0 to 41 Hz. Each element in the observation indicates the
BP of the particular frequency in the time interval of that trial. This process is
called preprocessing.
After preprocessing, the features usually contain noise and have large
dimensionality (Lotte et al 2oo7). These features are, therefore, participates in
another feature extraction, where data irrelevant to the event is filtered out. The
outcome of this final feature extraction is then fed into the classifier for eventual
training and testing purposes.
9
3.1.2 Feature extraction
Earlier studies have shown the correlation betweenmotor imagery of left and right
hand and activities in Mu- and Beta-rhythm. By performing MI tasks, the subject
can learn to control band powers (BP) in respective rhythms band(Mcfarland &
A. Miner & Vaughan & Wolpaw 2000). While the opposite can also occur, i.e.
classifier could use BP from respective bands to obtain enough information to
determine whether the subject imagine moving right or left hand(Pfurtscheller
& Neuper, & Flotzinger & Pregenzer 1997). In order to extract features inMu- and
Beta- rhythm. BP needs to be averaged over the respective rhythm band, reducing
each observation to 4 elements(2 rhythm for each electrode).
Furthermore, Since EEG signals might not generalize well across the whole
trial duration, a selection is then performed for every trail in each session, on
time windows of 1 second with 0.25 seconds overlap, obtaining 7 different time
windows for each session. Thesewindows, which, contain 4 different observations
for each trail, are viewed as subsets of observations with the same probability
distribution.
A method called cross-validation is introduced here. The basic idea is to divide a
set of samples into training sets and validation sets, the risk is then estimated by
the performance of the validation. A commonly used cross-validation type called
k-fold cross validation is applied here, which divides the sample set into k equal-
sized subsets. Using one subset at a time as a validation set and every other k-1
subsets as training sets, this process is repeated k times, until all the subset had
been validated once. The accuracy is estimated by the mean accuracy of all the
validation.
Using 4-fold cross validation with LDA as classifier within each time window and
comparing with other time windows, each session are represented by the window
that performs best in cross-validation. Each trail now consists of 4 observations,
and since the time window is small, it is assumed that these observations are
independent and are from the samepopulation. Thus obtaining the final extracted
features, with each session containing 4 times as many observations as before, i.e.
around 550. These extra variables could be used to perform ensemble methods
10
like voting to increase accuracy of classification, but in this case where test sample
size is small, the extra observations are used as it is, so the accuracy could be better
discriminated with the binomial distribution of p = 0.5, which have a decreasing
variance when the sample size increases.
3.1.3 Parameter selection
Recall that the term consisting importance in AIWLDA requires the probability
density function of test and training input. It is assumed that the Mu- and Beta-
frequency follows amultidimensional normal distribution(Sugiyama & Krauledat
& Muller 2007), with probability density function:
P (x) =exp(−1
2(x− µ)TΣ−1(x− µ))√det(Σ)(2π)k
. Here Σ denotes covariance matrix, calculated by:
Σij = E(xi−µj)(µi−xj)
µ denotes the mean vector and k the sample size.
Both the covariance matrix and the mean vector could be calculated using
input from testing and training samples respectively, thus obtaining the
importance.
Parameter λ that controls the bias-variance tradeoff in AIWLDA is a parameter
dependent on both training and testing samples. Different λ indicates a different
model, therefore, a new optimized λmust be decided for each new learning-testing
subject pair. This is done bymodel selection using 4-fold cross-validation on each
subject pair with λ chosen from {0.1, 0.2…1.0}.
3.2 Accuracy measurement
The result of this study are presented in classification accuracy, which is
measured as the ratio between correctly labeled sample and total number of test
11
sample.
Two sort of classification are performed in this project, namely intra-subject and
cross-subject classification.
Intra-subject classification is performed on all 10 sessions, where the accuracy is
obtained by 4-fold cross validation.
The session with best intra-subject result from each subject are used in cross-
subject classification, by using one of the five sessions as training set, another
as testing set. This gives in total 20 different training-testing pairs, where the
whole training session are used for training and the accuracy is obtained by
classifying testing session with trained model. Note that cross-validation could
not be performed here since testing and training samples are from different
populations.
Accuracy within 95% confidence interval of binomial distribution of p=0.5 is
deemed to be not significant enough. With a sample size of 560, this corresponds
to 46% - 54%, result outside this interval are referred as valid results, where pair
with lower than 46% accuracy signify not cross-subject classifiable and pairs with
higher than 54% are cross-subject classifiable.
3.3 Sub-problems
The research question: ’Is it possible to classify MI tasks robustly across every
subject pairs by using covariate-shift adaptation of classifiers?’ could be divided
into three minor parts. In this section, we shell first formulate these sub-
problems, then showing how they are solved.
3.3.1 Formulation
The first part is to determine whether implementing the covariate-shift
adaptation, i.e. AIWLDA, out perform the original classifier LDA on cross-subject
classification. If this gives a negative result, then there is no special reason to
implement this adaptation over LDA on cross-subject tasks.
12
Secondly, we need to show that all of the pairs are cross-subject classifiable. This
part is responsible for generalizability, i.e. whether this method could extend to
every subject pairs.
lastly, determine whether the accuracy of cross-subject classification have a
comparable magnitude to intra-subject classification when using AIWLDA. This
part answers for the robustness of the covariate-shift adaptation. Since the
accuracy of cross-subject classification is limited to be lower than or equal to intra-
subject classification. If covariate shift is the underlying problem on cross-subject
classification, solving it indicate cross-subject classification have similar accuracy
as intra-subject classification.
If all three parts could be validated and show a positive result, we can conclude
that covariate shift adaptation of classifiers can classify MI tasks robustly across
every subject pairs.
3.3.2 Measure evaluation
The first part is solved first by determining if result of cross-subject classification
obtained by two classifiers are from the same probability, using Mann–Whitney
U test (Nachar 2008), then show that AIWLDA have a higher average accuracy
than LDA on cross-subject classification.
The second part is then evaluated by examine the result obtained by AIWLDA in
cross-subject classification, so that none of the pairs shows < 46% accuracy. Note
that if some of the result lies within 46% - 54%, we can neither proof or disprove
the research question. In this case, a more accurate classifier and better feature
extraction section is required, in order to obtain a valid result.
Measures taken in the third part is the same as the first part, only the comparing is
done between intra-subject and cross-subject classification using AIWLDA.
13
4 Result
Results are presented in this chapter, with the helps from these results, the
research question could then be answered.
4.0.1 Justification
AIWLDA(Table 4.2) shows 9 out of 20 learning-testing pairs with higher than 54%
accuracy, which in Section 5.2 described as cross-subject classifiable, compare
to LDA (Table 4.1) with only 4 out of 20 pairs. This proves that covariate
shift adaptation does outperform the original classifier, which justifies using the
covariate shift adaptation on cross-subject MI classifying.
Mann–Whitney U test gives z = -2.62, which signify that result from respective
samples are from different distributions. Figure 4.1 further show that AIWLDA
outperform LDA in cross-subject MI tasks, by comparing the mean accuracy of
55.3% with LDSs 45.1%.
Worth noting is, none of the pairs from Table 4.2 are lower than 46%, thus neither
prove or disprove on generalizability aspect.
Table 4.1: Cross-subject classification with LDA.Letter indicates subject, (o) indicates cross-subject classifiable
14
Table 4.2: Cross-subject classification with AIWLDA.letter indicates subject, (o) indicates cross-subject classifiable
4.0.2 Robustness
performing U test again on result from intra-subject classification and inter-
subject classification with AIWLDA shows they belongs to different distribution,
with z = 2.95, showing result from intra- and cross-subject classification are not
from the same distribution with confidence level < 99%. After that we compare
mean accuracy over them (Figure 4.2), where intra-subject classification with
AIWLDA have mean accuracy of 62.4%. This far exceed the 55.3% obtained in
cross-subject classification.
This result proves, there exist factors other than covariate shift, which contribute
to the inaccuracy of cross-subject classification. This method is proven to be not
robust enough to classify an cross-subject MI task.
In conclusion, although AIWLDA does show better performance on cross-subject
tasks compared with LDA, due to multiple subject-pairs with high accuracy. It is
not robust enough to be used as classifier for cross-subject classification.
15
LDA AIWLDA0
20
40
60
80
100
45.1
55.3
#Accuracy(%)
Figure 4.1: Cross-subject classification mean accuracy with standard deviation
Intra-sub Cross-sub0
20
40
60
80
100
62.455.3
#Accuracy(%)
Figure 4.2: AIWLDA classifications mean accuracy with standard deviation
16
5 Discussion
In this section, future implications of result are discussed, along side with the
weakness and the strength of the proposed method. At last, we shell discuss the
future of BCI.
5.1 Weakness
As described in Chapter 4 (Result), using adaptive importance weighted LDA will
increase overall accuracy of cross-subject classification, but the mean accuracy is
far from optimal and the variance is large. Beside this, an optimized parameter
λ is not easy to find, due to the limitation of computation power. Even when an
optimized λ is found for a specific training-testing pair, the model confine this
parameter to only work on the specific model. Moreover, due to non-stationary
nature of brain-activity, model trained on two specific cross subject sessions is not
guaranteed to give good result on other sessions from the same testing subject, or
even different samples from the same test session. This variant of covariate shift
adaptation is therefore used, more as a verification for the possibility of cross-
subject classification, when addressing covariate shift.
5.2 Strength
Although we conclude that the adaptive importance weight method is not
desirable in reality situation, it still shown that addressing covariate shift in
cross-subject classification indicates better accuracy. Hence other covariate
shift addressing classifiers could still be utilized to shorten the calibration time
using data from other subjects, assuming the set of all human subjects could
be divided into subsets of cross-subject classifiable subject-pairs. The variant of
BCI that utilize generalized features obtained from other subject to minimize or
even exterminated calibration phase is called Subject independent BCI. Recent
studies have shown many progress (Lotte & Guan & Ang 2009; Cantillo-Negrete
& Martinez & Carino-Escobar & Carrillo-Mora & Elías-Viñas 2014), combining
17
method presented in this study with these findings might further increase
performance of subject independent BCIs based on Motor imagery.
5.3 Future work
As to future studies, one could investigate the subject pairs that are not cross-
subject classifiable (<44% accuracy) with LDA, but being the opposite when using
AIWLDA(>55% accuracy). An idea is to compare the cross-subject classification
accuracy of multiple different classifier with their covariate shift adaptation, this
might show more interesting results and a glimpse of the reason behind said
problem.
Due to the limitation of the project time, no subject specific parameters e.g. Mu-
and Beta- bands were investigated, which contributes to the low classification
accuracy. Having better feature extracted from preprocessed data might raise
the accuracy. Since cross-subject classification should work both way and there
are many subject-pairs with only one learning-testing pair that is cross-subject
classifiable, it is safe to assume that a higher accuracy will most definitely reveals
more classifiable pairs. Generalizability could also be determined this way, either
all subject pairs being cross-subject classifiable or showing a higher variance that
indicates the opposite.
5.4 Outlook
The generalizability of BCI is an important topic, this allow features obtained
from one subject to be used on others. Many people who urgently require
assistant of BCI are patients suffering motor impairment, these patients might
not have sufficient mental and physical strength to undergo the long lasting
calibration session. Therefore by constructing a subject independent BCI using
data collected from healthy subject, requirements earlier placed on the user could
be minimized.
This project have further confirmed the fact that it is not possible to classify across
every subject pair, an groupwise generalizable BCI could still be viable. If future
18
studies could help to identify easily as to which individual belongs to which cross-
subject classifiable group, then we could achieve a subject independent BCI that is
robust on every new user withminimal calibration. There are already studies with
similar problem formulated, e.g. study done by Cantillo-Negrete et al. have taken
gender in to account and shown increasing performance on subject independent
BCI.
There aremanymore properties on brain activities that haven’t been investigated,
which might not be as obvious. Although these properties are still unknown to
date, but due to increasing interest in the subject independent BCI recently, it will
surely not remain this way in the future.
19
6 Conclusion
Using covariate shift adaptation of LDA, also known as AIWLDA does show
better performance on cross-subject classification ,with mean accuracy of 55.4%
compared with 45.1% obtained by LDA (Figure 4.1). Overall only some specific
pairs contribute to higher cross-subject accuracy when using AIWLDA, not all
subject pairs show explicitly being cross-subject classifiable, i.e. accuracy > 54%.
The generalizability is therefore still to be determined.
Comparing 55.3% of cross-subject accuracy and 62.4% of intra-subject accuracy
(Figure 4.2) and applying U-test between these results shows, that the cross-
subject accuracy with AIWLDA is far from optimal, thus conclude that themethod
is not robust when classifying across all subjects.
In conclusion, the covariate shift adaptation of classifier is not robust across every
subject, but does out perform the original classifier in cross-subject classification
accuracy. The generalizability across every subject pair is still to be determine,
due to the low accuracy achieved in this study.
The hypothesis: covariate shift adaptation of classifier could use to robustly
classify across every subject, has not been positively validated, due to the lack of
robustness using LDA classifier.
20
Reference
Bolagh, Samaneh Nasiri Ghosheh & Shamsollahi, Mohammad Bagher & Jutten, Christian & Congedo, Marco (2016). Unsupervised Cross-Subject BCI Learning and Classification using Riemannian Geometry. ESANN . Boostani, Reza & Moradi, Mohammad. (2005). A new approach in the BCI research based on fractal dimension as feature and Adaboost as classifier. Journal of neural engineering. 1. 212-7. 10.1088/1741-2560/1/4/004. Cantillo-Negrete, Jessica & Martinez, Josefina & Carino-Escobar, Ruben & Carrillo-Mora, Paul & Elías-Viñas, David. (2014). An approach to improve the performance of subject-independent BCIs-based on motor imagery allocating subjects by gender. Biomedical engineering online. 13. 158. 10.1186/1475-925X-13-158. Costantini, Giovanni & Todisco, Massimiliano & Casali, Daniele & Carota, M & Saggio, Giovanni & Bianchi, Luigi & Abbafati, M & Quitadamo, Lucia. (2009). SVM Classification of EEG Signals for Brain Computer Interface. Frontiers in Artificial Intelligence and Applications. 204. 229-233. 10.3233/978-1-60750-072-8-229. Guger, Christoph & Edlinger, Günter & Harkam, W & Niedermayer, I & Pfurtscheller, Gert. (2003). How many people are able to operate an EEG-based brain-computer interface (BCI)?. IEEE transactions on neural systems and rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society. 11. 145-7. 10.1109/TNSRE.2003.814481. Herman, Pawel & Prasad, Girijesh & Martin McGinnity, Thomas. (2016). Designing an Interval Type-2 Fuzzy Logic System for Handling Uncertainty Effects in Brain-Computer Interface Classification of Motor Imagery Induced EEG Patterns. IEEE Transactions on Fuzzy Systems. PP. 1-1. 10.1109/TFUZZ.2016.2637934. Hoffmann, Ulrich & Vesin, Jean-Marc & Ebrahimi, Touradj & Diserens, Karin. (2008). An efficient P300-based brain computer interface for disabled subjects. Journal of neuroscience methods. 167. 115-25. 10.1016/j.jneumeth.2007.03.005. Jayaram, Vinay & Alamgir, Morteza & Altun, Yasemin & Schölkopf, Bernhard & Grosse-Wentrup, Moritz. (2015). Transfer Learning in Brain-Computer Interfaces. J. Vidal, Jacques(1973). Toward direct brain-computer communication.
Kaper, Matthias & Meinicke, Peter & Grossekathoefer, Ulf & Lingner, Thomas & Ritter, Helge. (2004). BCI competition 2003 - Data set IIb: Support vector machines for the P300 speller paradigm. IEEE_J_BME. 51. 1073-1076. 10.1109/TBME.2004.826698.
Kim, Chungsong & Sun, Jinwei & Liu, Dan & Wang, Qisong & Paek, Sunggyun. (2018) An effective feature extraction method by power spectral density of EEG signal for 2-class motor imagery-based BCI. Medical & Biological Engineering & Computing. 56. 1-14. 10.1007/s11517-017-1761-4. Klonowski, Wlodzimierz (2009). Everything you wanted to ask about EEG but were afraid to get the right answer. Nonlinear biomedical physics, 3(1), 2. doi:10.1186/1753-4631-3-2 Li, Xinyang & Guan, Cuntai & Zhang, Haihong & Ang, Kai & Ong, Sim. (2014). Adaptation of motor imagery EEG classification model based on tensor decomposition. Journal of Neural Engineering. 11. 056020. 10.1088/1741-2560/11/5/056020. Lotte, Fabien & Congedo, Marco & Lécuyer, Anatole & Fabrice, Lamarche & Arnaldi, Bruno. (2007). A review of classification algorithms for EEG-based brain-computer interfaces. Journal of Neural Engineering. 4. 10.1088/1741-2560/4/2/R01. Lotte, Fabien & Guan, Cuntai & Ang, Kai. (2009). Comparison of Designs Towards a Subject-Independent Brain-Computer Interface based on Motor Imagery. IEEE Engineering in Medicine and Biology Society. Conference. 2009. 4543-6. 10.1109/IEMBS.2009.5334126. Lu, Shijian & Guan, Cuntai & Zhang, Haihong. (2009). Unsupervised Brain Computer Interface Based on Intersubject Information and Online Adaptation. Neural Systems and Rehabilitation Engineering, IEEE Transactions on. 17. 135 - 145. 10.1109/TNSRE.2009.2015197. Mcfarland, Dennis & A. Miner, Laurie & Vaughan, Theresa & Wolpaw, Jonathan. (2000). Mu and Beta Rhythm Topographies During Motor Imagery and Actual Movements. Brain topography. 12. 177-86. 10.1023/A:1023437823106. Mcfarland, Dennis & Wolpaw, Jonathan. (2005). Sensorimotor Rhythm-Based Brain–Computer Interface (BCI): Feature Selection by Regression Improves Performance. IEEE transactions on neural systems and rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society. 13. 372-9. 10.1109/TNSRE.2005.848627. Nachar, Nadim. (2008). The Mann-Whitney U: A Test for Assessing Whether Two Independent Samples Come from the Same Distribution. Tutorials in Quantitative Methods for Psychology. 4. 10.20982/tqmp.04.1.p013. Pfurtscheller, Gert & Neuper, Christa & Flotzinger, D & Pregenzer, M. (1997). EEG-based discrimination between imagination of right and left hand movement. Electroencephalography and clinical neurophysiology. 103. 642-51. 10.1016/S0013-4694(97)00080-1.
Prasad, Girijesh & Herman, Pawel & Coyle, Damien & Mcdonough, Suzanne & Crosbie, Jacqueline. (2009). Using motor imagery based brain-computer interface for post-stroke rehabilitation. Proceedings of the 4th IEEE/EMBS International Conference on Neural Engineering. 258 - 262. 10.1109/NER.2009.5109282. Raza, Haider & Prasad, Girijesh & Li, Yuhua. (2015). EWMA model based shift-detection methods for detecting covariate shifts in non-stationary environments. Pattern Recognition. 48. 659–669. 10.1016/j.patcog.2014.07.028. Reuderink, Boris & Farquhar, J & Poel, Mannes & Nijholt, Anton. (2011). A subject-independent brain-computer interface based on smoothed, second-order baselining. Annual International Conference of the IEEE Engineering in Medicine and Biology Society. Conference. 2011. 4600-4. 10.1109/IEMBS.2011.6091139. Suk, Heung-Il & Lee, Seong-Whan. (2011). Subject and Class Specific Frequency Bands Selection for Multiclass Motor Imagery Classification. International Journal of Imaging Systems and Technology. 21. 123 - 130. 10.1002/ima.20283. Shenoy, Pradeep & Miller, Kai & Ojemann, Jeffrey & Rao, Rajesh. (2008).. Biomedical Engineering, IEEE Transactions on. 55. 273 - 280. 10.1109/TBME.2007.903528. Sugiyama, Masashi & Krauledat, Matthias & Müller, Klaus-Robert. (2007). Covariate Shift Adaptation by Importance Weighted Cross Validation. Journal of Machine Learning Research. 8. 985-1005. S. Fishman, George. (1996). Monte Carlo: Concepts, Algorithms, and Applications. Travis, T.A., Kondo, C.Y. and Knott, J.R. Alpha enhancement research: a review. Biol. Psychiat., 1975, 10: 69-89. Wang, Yijun & Gao, Shangkai & Gao, Xiaorong. (2005). Common Spatial Pattern Method for Channel Selelction in Motor Imagery Based Brain-computer Interface. Annual International Conference of the IEEE Engineering in Medicine and Biology Society. Conference. 5. 5392-5. 10.1109/IEMBS.2005.1615701. Wang, Yijun & Gao, Xiaorong & Hong, bo & Gao, Shangkai. (2010). Practical Designs of Brain–Computer Interfaces Based on the Modulation of EEG Rhythms. 10.1007/978-3-642-02091-9_8. Wolpaw, Jonathan & Mcfarland, Dennis. (1994). Multichannel EEG-based brain-computer communication. Electroencephalogr Clin Neurophysiol. Electroencephalography and clinical neurophysiology. 90. 444-9. 10.1016/0013-4694(94)90135-X.
Van de Laar, Bram & Gürkök, Hayrettin & Plass-Oude Bos, Danny & Poel, Mannes & Nijholt, Anton. (2013). Experiencing BCI Control in a Popular Computer Game. Computational Intelligence and AI in Games, IEEE Transactions on. 5. 176-184. 10.1109/TCIAIG.2013.2253778.
www.kth.se
top related