INOM EXAMENSARBETE TEKNIK,GRUNDNIVÅ, 15 HP

, STOCKHOLM SVERIGE 2019

Mot robust cross-subject klassificering av electroencephalogram (EEG) baserad brain-computer interfacing (BCI):En genomförbarhetsstudie

SHUAI WU

KTHSKOLAN FÖR TEKNIKVETENSKAP

INOM EXAMENSARBETE TEKNIK,GRUNDNIVÅ, 15 HP

, STOCKHOLM SVERIGE 2019

Towards robust cross-subject classification of electroencephalogram (EEG) patterns for brain-computer interfacing (BCI):A feasibility study

SHUAI WU

KTHSKOLAN FÖR TEKNIKVETENSKAP

Abstract

A brain-computer interface (BCI) is a system that enables the subject to send

commands with merely brain activity. Such interface is important for people

affected by multiple motor disabilities, where BCI made it possible for machine

to better understand the patient and thus fulfill their demands.

The BCI variante that base on motor imagery require classification on subject’s

brain activity on imagining movement of body parts, which could be done by

using different classifier. There exists multiple difficulty when developing such an

system, one of them is generalization of trained models, this accuracy of trained

model could not be guaranteed when using on a different subject or in a different

session. Even within the same session, the classification result is not optimal

due to brain activity’s non-stationary nature. This paper tackle the problem of

intersubject classification with adaptive importance weighted linear discriminant

analysis(AIWLDA), which shows promising result on both intersession and intra-

session classification of offline EEG based BCI. This research has shown that there

exist subject pairs with inter-subject generalizable potential, more pairs could be

revealed by using AIWLDA, but this method fail to robustly classify across every

subject-pairs.

Keywords

covariate shift, brain-computer interface, motor imagery, EEG,

inter-subject

i

Sammanfattning

Brain-computer interface(BCI) är ett system där man kan skicka kommandon till

dator med bara hjärnaktivitet. En sådan system är viktigt för människor lider av

flera motorisk funktionshinder, då maskinen skulle kunna förbättra patienters liv

genom att uppfylla deras behov.

Denna rapport fokusera på en variant av BCI, kallas motor imagery based

BCI, vilken basera på att klassificera försökspersons hjärnaktivitet då han/hon

tänka sig att röra sin kroppsdelar. Det finns flera svårighet för att bygga en

fungerande system, en av de är generalisering av tränad model. En tränad model

garanti inte exakthet på annat försöksperson eller annat session. Även i samma

session, kan model ger sämre resultat på grund av hjärnaktiviteten nonstationary

natur. Denna rapport försöka hantera inter-subject klassificering problem

med adaptive importance weighted linear discriminant analysis(AIWLDA), som

gav bra resultat i både intra-session och inter-session klassificering av offline

EEG baserad BCI. Det kommer visa i resultat att det finns försökspersons par

där inter-subject generalisering är möjligt och AIWLDA kan avslöja mer av

sådana par, men misslyckas att bevisa om det denna egenskap finns mellan alla

försöksperson.

Nyckelord

covariate shift, brain-computer interface, motor imagery, EEG,

inter-subject

ii

Contents

1 Introduction 11.1 Background introduction . . . . . . . . . . . . . . . . . . . . . . . . 1

1.2 Aim . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

1.3 Problem formulation . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

1.4 Delimitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

1.5 Outline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

2 Background 52.1 Electroencephalography(EEG) . . . . . . . . . . . . . . . . . . . . . 5

2.2 Brain-computer interface(BCI) . . . . . . . . . . . . . . . . . . . . . 5

2.3 Previous works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

2.4 LDA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

2.5 Covariate-shift adaptation of LDA . . . . . . . . . . . . . . . . . . . 7

3 Method 93.1 Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

3.2 Accuracy measurement . . . . . . . . . . . . . . . . . . . . . . . . . 11

3.3 Sub-problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

4 Result 14

5 Discussion 175.1 Weakness . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

5.2 Strength . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

5.3 Future work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

5.4 Outlook . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

6 Conclusion 20

iii

1 Introduction

This section provide an overview within the topic of brain computer interface and

formulate the research question.

Abbreviations

· BCI: Brain-computer interface

· EEG: Electroencephalography

· LDA: Linear discriminant analysis

· AIWLDA: Adaptive importance weighted LDA

· BP: Band Power

·MI: Motor imagery

1.1 Background introduction

Brain-computer interface, allows the user to communicate with machines,

providing a new way of communication and control (Wolpaw & Mcfarland

1994). This new channel of control could serve multiple purposes e.g. post-

stroke rehabilitation (Prasad & Herman & Coyle & Mcdonough & Crosbie

2009), creates new means of communication for people who suffer from motor

disabilities(Hoffmann & Vesin & Ebrahimi & Diserens 2008) or even controlling

video games (Van de Laar & Gürkök & Plass-Oude Bos & Poel & Nijholt

2013).

In order to control a BCI, the user must produce a brain activity pattern

recognizable by the system. Most of the existing BCI relies on either

regression(Mcfarland &Wolpaw 2005) or classification(Pfurtscheller &Neuper &

Flotzinger & Pregenzer 1997). Themost commonway is to utilize a classifier(Lotte

& Congedo & Lécuyer & Lamarche & Arnaldi 2007). This identification process is

done mainly by means of training the subject to create a specific brain activity

pattern, at the same time, an adaptation of classifier is introduced, where it

1

calibrates its model to match the subject’s brain activity pattern(Lotte et al.

2007).

1.2 Aim

Although many of the classification algorithms works fine as it is within the same

BCI session for the same subject, the statistical distribution of the data collected

from BCI varies across both session and subjects, which limits the transferability

of the training data and the trained model across subjects and session (Jayaram

et al. 2015). This inconsistency makes each previous model unusable once a new

session is started or a new subject is introduced, which results in slow calibration

time prior to each session. There have been many studies with the means of

decreasing number of calibration trials needed, which, aside from proposing

better classifiers, could be summarized by two types of approaches: The first

approach is to better utilize calibration data, by either extracting better features

(Wang & Gao, S & Gao, X 2005; Boostani & Moradi 2005) or better utilization

of these features (Li & Guan & Zhang & Ang & 2014; Sugiyama et al. 1996). The

second approach is to make use of existing data, extracting generalized features

from earlier data obtained in other sessions or even other subjects (Bolagh &

Shamsollahi & Jutten & Congedo 2016; Shenoy & Miller & Ojemann & Rao

2008).

Although generalized features exist in other subjects, it is not guaranteed that

these features from every subject will give positive contribution (Shenoy &

Miller & Ojemann & Rao 2008). The aim of this study is therefore to find an

algorithm that robustly classifies across every subject, which will result in fast or

no calibration time for each new installation, making BCI open to everyone in

need.

1.3 Problem formulation

The inconsistency of BCI classificationmainly due to the test and training samples

does not follow the same probability distribution caused by the non-stationary

2

nature of brain activities. (Klonowski 2009). Covariate shift refers to the change

in the distribution of the input variable in the training and testing sample.

Classifier adaptive importance weighted linear discriminant analysis, also known

as AIWLDA proposed by Sugiyama et al(2007), could robustly classify samples

from a different probability distribution than the test samples.

Non-stationary nature of brain activity is the cause of covariate shift when doing

intra-subject classification, non-stationary meaning probability distribution a

sample change with time (Klonowski 2009). This nature is not the primary

concern of cross-subject classification, where the samples were collected from two

or more different sample set, i.e. subjects. Although covariate shift could still be a

major problem on the topic since the subjects share similar physiologic structure

i.e. are all humans. The research question can, therefore, formulate as such: Is it

possible to classifyMI tasks robustly across every subject pairs by using covariate-

shift adaptation of classifiers? This will help us find out whether the difficulty of

intersubject classification lies on covariate shift, thus showingmore insight on the

topic.

1.4 Delimitations

Due to the limitation of time, the scope of this study is limited to examine a

single linear classifier, i.e. LDA. Although many classifiers does have better

accuracy compared with LDA (Lotte et al. 2007) due to brain activity’s non-linear

nature (Klonowski 2009), LDA is easy to implement and does show a reasonable

high accuracy on classifying task of discriminating between left- and right- hand

motion imagination (Boostani & Moradi 2005), therefore is chosen to be studied

in this project.

Subject-specific frequencies bands are not investigated in this study, which is

rather important for motor imagery (MI) BCI. Investigate it will increase the

performance of the classification task (Suk & Lee 2011)

Data from 10 sessions of MI experiment was investigated, with 2 sessions for each

subject, with a total number of 5 subjects. Each session contains around 140 trials

and each trial consist of 4 seconds of EEG-signal.

3

1.5 Outline

In Chapter2 (Background), the reader can expect to learn about the concepts

introduced in Chapter1 (Introduction) like BCI and EEG, in a more in-depth

manner, followed by an overview of methods used in this project and data

processing. In Chapter3 (Method), the layout of this project will be presented

to reader, giving insight on how the experiment and results are formulated.

The research question is answered along with multiple worth noting results in

Chapter4 (Result). In Chapter5 (Discussion), the implication of the result is

discussed, followed by the analysis of weakness and strength of this method,

hoping to show a direction to future studies.

4

2 Background

In this section, the reader will get to understand multiple terms on the topic. The

method used in the following chapter is also be presented.

2.1 Electroencephalography(EEG)

Electroencephalography (EEG), is a monitoring method to record the electrical

activity of the brain, it usually gathers data from electrodes placed on the subject’s

scalps (Wolpaw & Mcfarland 1994). These EEG signals collected on the human

scalp are a reflection of corresponding activities in upper layers of the brain cortex

below the scalp surface (Vidal 1973). Much research has indicated that human has

the ability to manipulate a variety of EEG phenomena, which implies multiple

possibilities for EEG based BCI (Travis & Kondo & Knott 1975; Mcfarland & A.

Miner & Vaughan &Wolpaw 2000).

2.2 Brain-computer interface(BCI)

As stated in the introduction section, BCI enables the user to interact with

machines using brain activities, this could be done by letting the system identify

patterns of brain activity relevant to commands, which is a task usually given to

the classifiers.

The performance of the system depends on both features extracted from the

EEG signal and the classifier implemented(Lotte et al 2007). Where features are

data extracted from original data by reducing irrelevant parts, it’s intended to be

informative and in some cases lead to better human interpretations(Wikipedia

2019b).

Different features e.g. Power spectral density (PSD) (Kim & Sun & Liu & Wang &

Paek 2018) or BandPower (BP) (Pfurtscheller & Neuper,& Flotzinger & Pregenzer

1997) are extracted from the original data in a phase called preprocessing. Usage

of the different feature depends on the method chosen, therefore one can not

simply say that one feature yield better performance than others.

5

This project is built on so-called motor imagery (MI) BCI, where the subject is

requested to imaginemoving specific body partswhen instructed, while electrodes

placed along the subject’s scalp record the EEG signal. each of such instruction is

called a trial and each electrode is called a channel, with specific names depending

on where it’s located (figure 2.1).

Figure 2.1: Electrode placement international 10-20 system

2.3 Previous works

There have been numerous attempts on the cross-subject classification of EEG

based BCI, these attempts mainly try to adapt the existing model to decrease

calibration time. For instance, a study done by Lu and Zhang (2009) has shown

that for P300 speller, a BCI based on decisionmaking, utilizing a so-called subject

independent model learned by offline samples could drastically decrease the

number of calibration trials needed. Similar studies concerning MI, also shown a

positive result(Reuderink & Farquhar & Poel & Nijholt 2011; Jayaram & Alamgir

& Altun & Schölkopf & Grosse-Wentrup 2015).

There have been fewer studies on cross-subject classification of MI tasks without

using any labeled data from the test sample. Presumably due to that not

all training samples from different subjects may improve the performance of

cross-subject classification (Bolagh & Shamsollahi & Jutten & Congedo 2016).

Cross-subject classification of MI task require either subject selection so that

6

only subjects with positive contribution are used as the training set(Bolagh &

Shamsollahi & Jutten & Congedo 2016), whereas others report large variance

on accuracy between different subjects, when using inter-subject generalized

features(Shenoy & Miller & Ojemann & Rao 2008)

2.4 LDA

Earlier studies have shown that Linear discriminant analysis(LDA), a linear

classifier, which discriminates between two classes, is reasonably accurate on

classifying MI tasks (Herman, 2015). The classifier tries to find a line, where

the labeled samples are separated by the origin when projected on the line. The

unlabeled samples could then be projected on the line, and each observation are

labeled depending on the position of the projection compare with originn.

When classifying a set of offline samples using 4-fold cross-validation, LDA shows

similar average accuracy for intra-session(62.7%) and inter-session classification

(60.2%), but is lacking in an inter-subject(<50%) classification overall, except for

some specific subject pairs. Although the accuracy is not as high as the study done

by Herman et al. (2015) due to no subject-specific parameters was investigated

in this study, but the result can still serve the purpose of comparison with the

covariate-shift adaptation of LDA.

2.5 Covariate-shift adaptation of LDA

Assume the ratio of the test and training probability density function is finite and

known:P1(x)

P2(x)

Where the P denotes the probability density function of the respective sample

set and x denotes the input. This expression is known as importance, first

introduced by Fishman(1996) for importance sampling. A method introduced

by Sugiyama(2007) has shown that the importance could be used to address

covariate-shift problems in machine learning problems.

7

covariate-shift refer to the effect, where the change in probability distribution

presents in the training and test data. AIWLDA is a modified LDA classifier that

taken in to account of this by utilizing the importance, making this classifier more

accurate when encounter covariate-shift.

AIWLDA has a model of:

f(x, θ) = θ0 +∑i

θixi

Where θ is learned as following:

θ = argminθ[n∑

i=1

1

n((

Ptest(xi)

Ptrain(xi)

)λ(f(xi, θ)− yi)2]

The classifying result, or the labels are then obtained by:

u = sgn(f(x, θ))

Here eachxi and yi pairs denotes one labeled observation, wherexi is the input and

yi indicates its label. The importance is between the testing and training input’s

probability density function, which the input in importance expression is just the

training input of each observation.

Note that λ is the parameter that controls the tradeoff between accuracy and

precision(Sugiyama, Krauledat & Muller 2007), known as the bias-variance

tradeoff(Lotte et al 2007). Model selection is needed to choose a suitable λ. Worth

noting is that when λ = 0, AIWLDA is no other than the normal LDA.

Note that this adaptation of LDA does not address any potential model error,

therefore other classifiers such as Gaussian support vector machines(GSVM) with

better performance onMI tasks (Lotte et al. 2007)might still give higher accuracy

when the difference between distribution of test and training input is little.

8

3 Method

This section will providemore insight on both data andmethod used in this study,

an evaluation on measures of results are also be presented.

3.1 Data

In this project, the subject’s EEG pattern is recorded by 2 electrodes, placed on the

C3, C4 channels according to the international 10-20 system (Figure 2.1), which

corresponding to the hand area in M1 (Wang & Gao & Hong & Gao 2010). For

every four seconds, the subjects are instructed to imagine moving either right or

left hand, each of such an instruction is called a trial and each session consists

around 140 trials.

3.1.1 Preprocessing

This study make use of the feature called band power(BP). In order to obtain

this feature, raw EEG signal recorded from each session needed to be processed,

extracting band power from it with respect to frequencies. This could be achieved

by using Fourier transformation on different time intervals that are reasonably

small, in this study, the time interval is chosen to be ¼ of a second. This creates

one 3d-array of trial × Time × Frequency for each channel respectively, where

frequency spans from 0 to 41 Hz. Each element in the observation indicates the

BP of the particular frequency in the time interval of that trial. This process is

called preprocessing.

After preprocessing, the features usually contain noise and have large

dimensionality (Lotte et al 2oo7). These features are, therefore, participates in

another feature extraction, where data irrelevant to the event is filtered out. The

outcome of this final feature extraction is then fed into the classifier for eventual

training and testing purposes.

9

3.1.2 Feature extraction

Earlier studies have shown the correlation betweenmotor imagery of left and right

hand and activities in Mu- and Beta-rhythm. By performing MI tasks, the subject

can learn to control band powers (BP) in respective rhythms band(Mcfarland &

A. Miner & Vaughan & Wolpaw 2000). While the opposite can also occur, i.e.

classifier could use BP from respective bands to obtain enough information to

determine whether the subject imagine moving right or left hand(Pfurtscheller

& Neuper, & Flotzinger & Pregenzer 1997). In order to extract features inMu- and

Beta- rhythm. BP needs to be averaged over the respective rhythm band, reducing

each observation to 4 elements(2 rhythm for each electrode).

Furthermore, Since EEG signals might not generalize well across the whole

trial duration, a selection is then performed for every trail in each session, on

time windows of 1 second with 0.25 seconds overlap, obtaining 7 different time

windows for each session. Thesewindows, which, contain 4 different observations

for each trail, are viewed as subsets of observations with the same probability

distribution.

A method called cross-validation is introduced here. The basic idea is to divide a

set of samples into training sets and validation sets, the risk is then estimated by

the performance of the validation. A commonly used cross-validation type called

k-fold cross validation is applied here, which divides the sample set into k equal-

sized subsets. Using one subset at a time as a validation set and every other k-1

subsets as training sets, this process is repeated k times, until all the subset had

been validated once. The accuracy is estimated by the mean accuracy of all the

validation.

Using 4-fold cross validation with LDA as classifier within each time window and

comparing with other time windows, each session are represented by the window

that performs best in cross-validation. Each trail now consists of 4 observations,

and since the time window is small, it is assumed that these observations are

independent and are from the samepopulation. Thus obtaining the final extracted

features, with each session containing 4 times as many observations as before, i.e.

around 550. These extra variables could be used to perform ensemble methods

10

like voting to increase accuracy of classification, but in this case where test sample

size is small, the extra observations are used as it is, so the accuracy could be better

discriminated with the binomial distribution of p = 0.5, which have a decreasing

variance when the sample size increases.

3.1.3 Parameter selection

Recall that the term consisting importance in AIWLDA requires the probability

density function of test and training input. It is assumed that the Mu- and Beta-

frequency follows amultidimensional normal distribution(Sugiyama & Krauledat

& Muller 2007), with probability density function:

P (x) =exp(−1

2(x− µ)TΣ−1(x− µ))√det(Σ)(2π)k

. Here Σ denotes covariance matrix, calculated by:

Σij = E(xi−µj)(µi−xj)

µ denotes the mean vector and k the sample size.

Both the covariance matrix and the mean vector could be calculated using

input from testing and training samples respectively, thus obtaining the

importance.

Parameter λ that controls the bias-variance tradeoff in AIWLDA is a parameter

dependent on both training and testing samples. Different λ indicates a different

model, therefore, a new optimized λmust be decided for each new learning-testing

subject pair. This is done bymodel selection using 4-fold cross-validation on each

subject pair with λ chosen from {0.1, 0.2…1.0}.

3.2 Accuracy measurement

The result of this study are presented in classification accuracy, which is

measured as the ratio between correctly labeled sample and total number of test

11

sample.

Two sort of classification are performed in this project, namely intra-subject and

cross-subject classification.

Intra-subject classification is performed on all 10 sessions, where the accuracy is

obtained by 4-fold cross validation.

The session with best intra-subject result from each subject are used in cross-

subject classification, by using one of the five sessions as training set, another

as testing set. This gives in total 20 different training-testing pairs, where the

whole training session are used for training and the accuracy is obtained by

classifying testing session with trained model. Note that cross-validation could

not be performed here since testing and training samples are from different

populations.

Accuracy within 95% confidence interval of binomial distribution of p=0.5 is

deemed to be not significant enough. With a sample size of 560, this corresponds

to 46% - 54%, result outside this interval are referred as valid results, where pair

with lower than 46% accuracy signify not cross-subject classifiable and pairs with

higher than 54% are cross-subject classifiable.

3.3 Sub-problems

The research question: ’Is it possible to classify MI tasks robustly across every

subject pairs by using covariate-shift adaptation of classifiers?’ could be divided

into three minor parts. In this section, we shell first formulate these sub-

problems, then showing how they are solved.

3.3.1 Formulation

The first part is to determine whether implementing the covariate-shift

adaptation, i.e. AIWLDA, out perform the original classifier LDA on cross-subject

classification. If this gives a negative result, then there is no special reason to

implement this adaptation over LDA on cross-subject tasks.

12

Secondly, we need to show that all of the pairs are cross-subject classifiable. This

part is responsible for generalizability, i.e. whether this method could extend to

every subject pairs.

lastly, determine whether the accuracy of cross-subject classification have a

comparable magnitude to intra-subject classification when using AIWLDA. This

part answers for the robustness of the covariate-shift adaptation. Since the

accuracy of cross-subject classification is limited to be lower than or equal to intra-

subject classification. If covariate shift is the underlying problem on cross-subject

classification, solving it indicate cross-subject classification have similar accuracy

as intra-subject classification.

If all three parts could be validated and show a positive result, we can conclude

that covariate shift adaptation of classifiers can classify MI tasks robustly across

every subject pairs.

3.3.2 Measure evaluation

The first part is solved first by determining if result of cross-subject classification

obtained by two classifiers are from the same probability, using Mann–Whitney

U test (Nachar 2008), then show that AIWLDA have a higher average accuracy

than LDA on cross-subject classification.

The second part is then evaluated by examine the result obtained by AIWLDA in

cross-subject classification, so that none of the pairs shows < 46% accuracy. Note

that if some of the result lies within 46% - 54%, we can neither proof or disprove

the research question. In this case, a more accurate classifier and better feature

extraction section is required, in order to obtain a valid result.

Measures taken in the third part is the same as the first part, only the comparing is

done between intra-subject and cross-subject classification using AIWLDA.

13

4 Result

Results are presented in this chapter, with the helps from these results, the

research question could then be answered.

4.0.1 Justification

AIWLDA(Table 4.2) shows 9 out of 20 learning-testing pairs with higher than 54%

accuracy, which in Section 5.2 described as cross-subject classifiable, compare

to LDA (Table 4.1) with only 4 out of 20 pairs. This proves that covariate

shift adaptation does outperform the original classifier, which justifies using the

covariate shift adaptation on cross-subject MI classifying.

Mann–Whitney U test gives z = -2.62, which signify that result from respective

samples are from different distributions. Figure 4.1 further show that AIWLDA

outperform LDA in cross-subject MI tasks, by comparing the mean accuracy of

55.3% with LDSs 45.1%.

Worth noting is, none of the pairs from Table 4.2 are lower than 46%, thus neither

prove or disprove on generalizability aspect.

Table 4.1: Cross-subject classification with LDA.Letter indicates subject, (o) indicates cross-subject classifiable

14

Table 4.2: Cross-subject classification with AIWLDA.letter indicates subject, (o) indicates cross-subject classifiable

4.0.2 Robustness

performing U test again on result from intra-subject classification and inter-

subject classification with AIWLDA shows they belongs to different distribution,

with z = 2.95, showing result from intra- and cross-subject classification are not

from the same distribution with confidence level < 99%. After that we compare

mean accuracy over them (Figure 4.2), where intra-subject classification with

AIWLDA have mean accuracy of 62.4%. This far exceed the 55.3% obtained in

cross-subject classification.

This result proves, there exist factors other than covariate shift, which contribute

to the inaccuracy of cross-subject classification. This method is proven to be not

robust enough to classify an cross-subject MI task.

In conclusion, although AIWLDA does show better performance on cross-subject

tasks compared with LDA, due to multiple subject-pairs with high accuracy. It is

not robust enough to be used as classifier for cross-subject classification.

15

LDA AIWLDA0

20

40

60

80

100

45.1

55.3

#Accuracy(%)

Figure 4.1: Cross-subject classification mean accuracy with standard deviation

Intra-sub Cross-sub0

20

40

60

80

100

62.455.3

#Accuracy(%)

Figure 4.2: AIWLDA classifications mean accuracy with standard deviation

16

5 Discussion

In this section, future implications of result are discussed, along side with the

weakness and the strength of the proposed method. At last, we shell discuss the

future of BCI.

5.1 Weakness

As described in Chapter 4 (Result), using adaptive importance weighted LDA will

increase overall accuracy of cross-subject classification, but the mean accuracy is

far from optimal and the variance is large. Beside this, an optimized parameter

λ is not easy to find, due to the limitation of computation power. Even when an

optimized λ is found for a specific training-testing pair, the model confine this

parameter to only work on the specific model. Moreover, due to non-stationary

nature of brain-activity, model trained on two specific cross subject sessions is not

guaranteed to give good result on other sessions from the same testing subject, or

even different samples from the same test session. This variant of covariate shift

adaptation is therefore used, more as a verification for the possibility of cross-

subject classification, when addressing covariate shift.

5.2 Strength

Although we conclude that the adaptive importance weight method is not

desirable in reality situation, it still shown that addressing covariate shift in

cross-subject classification indicates better accuracy. Hence other covariate

shift addressing classifiers could still be utilized to shorten the calibration time

using data from other subjects, assuming the set of all human subjects could

be divided into subsets of cross-subject classifiable subject-pairs. The variant of

BCI that utilize generalized features obtained from other subject to minimize or

even exterminated calibration phase is called Subject independent BCI. Recent

studies have shown many progress (Lotte & Guan & Ang 2009; Cantillo-Negrete

& Martinez & Carino-Escobar & Carrillo-Mora & Elías-Viñas 2014), combining

17

method presented in this study with these findings might further increase

performance of subject independent BCIs based on Motor imagery.

5.3 Future work

As to future studies, one could investigate the subject pairs that are not cross-

subject classifiable (<44% accuracy) with LDA, but being the opposite when using

AIWLDA(>55% accuracy). An idea is to compare the cross-subject classification

accuracy of multiple different classifier with their covariate shift adaptation, this

might show more interesting results and a glimpse of the reason behind said

problem.

Due to the limitation of the project time, no subject specific parameters e.g. Mu-

and Beta- bands were investigated, which contributes to the low classification

accuracy. Having better feature extracted from preprocessed data might raise

the accuracy. Since cross-subject classification should work both way and there

are many subject-pairs with only one learning-testing pair that is cross-subject

classifiable, it is safe to assume that a higher accuracy will most definitely reveals

more classifiable pairs. Generalizability could also be determined this way, either

all subject pairs being cross-subject classifiable or showing a higher variance that

indicates the opposite.

5.4 Outlook

The generalizability of BCI is an important topic, this allow features obtained

from one subject to be used on others. Many people who urgently require

assistant of BCI are patients suffering motor impairment, these patients might

not have sufficient mental and physical strength to undergo the long lasting

calibration session. Therefore by constructing a subject independent BCI using

data collected from healthy subject, requirements earlier placed on the user could

be minimized.

This project have further confirmed the fact that it is not possible to classify across

every subject pair, an groupwise generalizable BCI could still be viable. If future

18

studies could help to identify easily as to which individual belongs to which cross-

subject classifiable group, then we could achieve a subject independent BCI that is

robust on every new user withminimal calibration. There are already studies with

similar problem formulated, e.g. study done by Cantillo-Negrete et al. have taken

gender in to account and shown increasing performance on subject independent

BCI.

There aremanymore properties on brain activities that haven’t been investigated,

which might not be as obvious. Although these properties are still unknown to

date, but due to increasing interest in the subject independent BCI recently, it will

surely not remain this way in the future.

19

6 Conclusion

Using covariate shift adaptation of LDA, also known as AIWLDA does show

better performance on cross-subject classification ,with mean accuracy of 55.4%

compared with 45.1% obtained by LDA (Figure 4.1). Overall only some specific

pairs contribute to higher cross-subject accuracy when using AIWLDA, not all

subject pairs show explicitly being cross-subject classifiable, i.e. accuracy > 54%.

The generalizability is therefore still to be determined.

Comparing 55.3% of cross-subject accuracy and 62.4% of intra-subject accuracy

(Figure 4.2) and applying U-test between these results shows, that the cross-

subject accuracy with AIWLDA is far from optimal, thus conclude that themethod

is not robust when classifying across all subjects.

In conclusion, the covariate shift adaptation of classifier is not robust across every

subject, but does out perform the original classifier in cross-subject classification

accuracy. The generalizability across every subject pair is still to be determine,

due to the low accuracy achieved in this study.

The hypothesis: covariate shift adaptation of classifier could use to robustly

classify across every subject, has not been positively validated, due to the lack of

robustness using LDA classifier.

20

Reference

Bolagh, Samaneh Nasiri Ghosheh & Shamsollahi, Mohammad Bagher & Jutten, Christian & Congedo, Marco (2016). Unsupervised Cross-Subject BCI Learning and Classification using Riemannian Geometry. ESANN . Boostani, Reza & Moradi, Mohammad. (2005). A new approach in the BCI research based on fractal dimension as feature and Adaboost as classifier. Journal of neural engineering. 1. 212-7. 10.1088/1741-2560/1/4/004. Cantillo-Negrete, Jessica & Martinez, Josefina & Carino-Escobar, Ruben & Carrillo-Mora, Paul & Elías-Viñas, David. (2014). An approach to improve the performance of subject-independent BCIs-based on motor imagery allocating subjects by gender. Biomedical engineering online. 13. 158. 10.1186/1475-925X-13-158. Costantini, Giovanni & Todisco, Massimiliano & Casali, Daniele & Carota, M & Saggio, Giovanni & Bianchi, Luigi & Abbafati, M & Quitadamo, Lucia. (2009). SVM Classification of EEG Signals for Brain Computer Interface. Frontiers in Artificial Intelligence and Applications. 204. 229-233. 10.3233/978-1-60750-072-8-229. Guger, Christoph & Edlinger, Günter & Harkam, W & Niedermayer, I & Pfurtscheller, Gert. (2003). How many people are able to operate an EEG-based brain-computer interface (BCI)?. IEEE transactions on neural systems and rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society. 11. 145-7. 10.1109/TNSRE.2003.814481. Herman, Pawel & Prasad, Girijesh & Martin McGinnity, Thomas. (2016). Designing an Interval Type-2 Fuzzy Logic System for Handling Uncertainty Effects in Brain-Computer Interface Classification of Motor Imagery Induced EEG Patterns. IEEE Transactions on Fuzzy Systems. PP. 1-1. 10.1109/TFUZZ.2016.2637934. Hoffmann, Ulrich & Vesin, Jean-Marc & Ebrahimi, Touradj & Diserens, Karin. (2008). An efficient P300-based brain computer interface for disabled subjects. Journal of neuroscience methods. 167. 115-25. 10.1016/j.jneumeth.2007.03.005. Jayaram, Vinay & Alamgir, Morteza & Altun, Yasemin & Schölkopf, Bernhard & Grosse-Wentrup, Moritz. (2015). Transfer Learning in Brain-Computer Interfaces. J. Vidal, Jacques(1973). Toward direct brain-computer communication.

Kaper, Matthias & Meinicke, Peter & Grossekathoefer, Ulf & Lingner, Thomas & Ritter, Helge. (2004). BCI competition 2003 - Data set IIb: Support vector machines for the P300 speller paradigm. IEEE_J_BME. 51. 1073-1076. 10.1109/TBME.2004.826698.

Kim, Chungsong & Sun, Jinwei & Liu, Dan & Wang, Qisong & Paek, Sunggyun. (2018) An effective feature extraction method by power spectral density of EEG signal for 2-class motor imagery-based BCI. Medical & Biological Engineering & Computing. 56. 1-14. 10.1007/s11517-017-1761-4. Klonowski, Wlodzimierz (2009). Everything you wanted to ask about EEG but were afraid to get the right answer. Nonlinear biomedical physics, 3(1), 2. doi:10.1186/1753-4631-3-2 Li, Xinyang & Guan, Cuntai & Zhang, Haihong & Ang, Kai & Ong, Sim. (2014). Adaptation of motor imagery EEG classification model based on tensor decomposition. Journal of Neural Engineering. 11. 056020. 10.1088/1741-2560/11/5/056020. Lotte, Fabien & Congedo, Marco & Lécuyer, Anatole & Fabrice, Lamarche & Arnaldi, Bruno. (2007). A review of classification algorithms for EEG-based brain-computer interfaces. Journal of Neural Engineering. 4. 10.1088/1741-2560/4/2/R01. Lotte, Fabien & Guan, Cuntai & Ang, Kai. (2009). Comparison of Designs Towards a Subject-Independent Brain-Computer Interface based on Motor Imagery. IEEE Engineering in Medicine and Biology Society. Conference. 2009. 4543-6. 10.1109/IEMBS.2009.5334126. Lu, Shijian & Guan, Cuntai & Zhang, Haihong. (2009). Unsupervised Brain Computer Interface Based on Intersubject Information and Online Adaptation. Neural Systems and Rehabilitation Engineering, IEEE Transactions on. 17. 135 - 145. 10.1109/TNSRE.2009.2015197. Mcfarland, Dennis & A. Miner, Laurie & Vaughan, Theresa & Wolpaw, Jonathan. (2000). Mu and Beta Rhythm Topographies During Motor Imagery and Actual Movements. Brain topography. 12. 177-86. 10.1023/A:1023437823106. Mcfarland, Dennis & Wolpaw, Jonathan. (2005). Sensorimotor Rhythm-Based Brain–Computer Interface (BCI): Feature Selection by Regression Improves Performance. IEEE transactions on neural systems and rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society. 13. 372-9. 10.1109/TNSRE.2005.848627. Nachar, Nadim. (2008). The Mann-Whitney U: A Test for Assessing Whether Two Independent Samples Come from the Same Distribution. Tutorials in Quantitative Methods for Psychology. 4. 10.20982/tqmp.04.1.p013. Pfurtscheller, Gert & Neuper, Christa & Flotzinger, D & Pregenzer, M. (1997). EEG-based discrimination between imagination of right and left hand movement. Electroencephalography and clinical neurophysiology. 103. 642-51. 10.1016/S0013-4694(97)00080-1.

Prasad, Girijesh & Herman, Pawel & Coyle, Damien & Mcdonough, Suzanne & Crosbie, Jacqueline. (2009). Using motor imagery based brain-computer interface for post-stroke rehabilitation. Proceedings of the 4th IEEE/EMBS International Conference on Neural Engineering. 258 - 262. 10.1109/NER.2009.5109282. Raza, Haider & Prasad, Girijesh & Li, Yuhua. (2015). EWMA model based shift-detection methods for detecting covariate shifts in non-stationary environments. Pattern Recognition. 48. 659–669. 10.1016/j.patcog.2014.07.028. Reuderink, Boris & Farquhar, J & Poel, Mannes & Nijholt, Anton. (2011). A subject-independent brain-computer interface based on smoothed, second-order baselining. Annual International Conference of the IEEE Engineering in Medicine and Biology Society. Conference. 2011. 4600-4. 10.1109/IEMBS.2011.6091139. Suk, Heung-Il & Lee, Seong-Whan. (2011). Subject and Class Specific Frequency Bands Selection for Multiclass Motor Imagery Classification. International Journal of Imaging Systems and Technology. 21. 123 - 130. 10.1002/ima.20283. Shenoy, Pradeep & Miller, Kai & Ojemann, Jeffrey & Rao, Rajesh. (2008).. Biomedical Engineering, IEEE Transactions on. 55. 273 - 280. 10.1109/TBME.2007.903528. Sugiyama, Masashi & Krauledat, Matthias & Müller, Klaus-Robert. (2007). Covariate Shift Adaptation by Importance Weighted Cross Validation. Journal of Machine Learning Research. 8. 985-1005. S. Fishman, George. (1996). Monte Carlo: Concepts, Algorithms, and Applications. Travis, T.A., Kondo, C.Y. and Knott, J.R. Alpha enhancement research: a review. Biol. Psychiat., 1975, 10: 69-89. Wang, Yijun & Gao, Shangkai & Gao, Xiaorong. (2005). Common Spatial Pattern Method for Channel Selelction in Motor Imagery Based Brain-computer Interface. Annual International Conference of the IEEE Engineering in Medicine and Biology Society. Conference. 5. 5392-5. 10.1109/IEMBS.2005.1615701. Wang, Yijun & Gao, Xiaorong & Hong, bo & Gao, Shangkai. (2010). Practical Designs of Brain–Computer Interfaces Based on the Modulation of EEG Rhythms. 10.1007/978-3-642-02091-9_8. Wolpaw, Jonathan & Mcfarland, Dennis. (1994). Multichannel EEG-based brain-computer communication. Electroencephalogr Clin Neurophysiol. Electroencephalography and clinical neurophysiology. 90. 444-9. 10.1016/0013-4694(94)90135-X.

Van de Laar, Bram & Gürkök, Hayrettin & Plass-Oude Bos, Danny & Poel, Mannes & Nijholt, Anton. (2013). Experiencing BCI Control in a Popular Computer Game. Computational Intelligence and AI in Games, IEEE Transactions on. 5. 176-184. 10.1109/TCIAIG.2013.2253778.

www.kth.se