the use of brain-computer interfaces in control...

JOURNAL OF MEDICAL INFORMATICS & TECHNOLOGIES Vol. 22/2013, ISSN 1642-6037

Brain Computer Interfaces,control processes, industrial PC

Szczepan PASZKIEL1

THE USE OF BRAIN-COMPUTER INTERFACES INCONTROL PROCESSES BASED ON THE

INDUSTRIAL PC IN TERMS OF THE METHODS OFEEG SIGNAL ANALYSES

The article presents applications of BCI - Brain Computer Interfaces technology in the control processesbased on the infrastructure of an IPC - an Industrial PC. Methods of the EEG signal analysis such as the PCA thePrincipal Component Analysis and the ICA the Independent Component Analysis are also discussed. Nowadaysindustrial computers are increasingly used in production, due to their specific technical parameters conducive toworking in difficult conditions. The use of control based on brain-computer interface speed definitely rate theperformance of the employees, reduce the response time to the case and allows you to remotely perform theactivity.

1. INTRODUCTION

Several years ago, the control technology based on voice signals or touch panels was rarely used.Nowadays the situation has changed dramatically. An increasing number of devices uses this type ofinterface such as smart phones, television, and industrial computers and those devices used in the variousbranches of the services, such as medical industry, catering etc. Currently, many research centers andscientists are working on the development of Brain Computer Interfaces technologies [12]. Scientificactivities at the Institute for Knowledge Discovery, University of Graz in Austria are particularlyimportant for the development of the above-mentioned techniques. In Poland, Dr. Piotr Durka of theUniversity of Warsaw is considered to be the precursor of brain computer interfaces. Considering theaspect of BCI cognitivist action, a large number of publications in this field has prof. Wlodzislaw Duchof Nicolaus Copernicus University in Torun. The fundamental issue underlying BCI is undoubtedlyNeurocybernetics and prof. Ryszard Tadeusiewicz of AGH in Krakow [13] has been dealing with thisarea. Attempts to connect the BCI interface with IPCs, according to a thorough study of literature inPoland has not made any scientific research center yet. Therefore, the main objective of this article is tocharacterize the BCI technology. It can be applied to control the computer industry with the particularemphasis on methods of analysis of EEG signals. A continuous development of innovative methods ofcommunication tends to create more and more interfaces that can be used on many levels, includingcontrol processes, automation, etc in the industry. One of them are brain-computer interfaces, which,despite its complexity at the stage of implementation can gain a wider range of applications. First of all,for the correct and reliable operation it is important to conduct filtering of the electroencephalographicsignal properly which is the primary source on the basis of which a control process is performed in

1Opole University of Technology, Faculty of Electrical Engineering, Automatic Control and Informatics, Gen. KazimierzaSosnkowskiego 31, 45-271 Opole, Poland.

MEDICAL DATA ANALYSIS AND MONITORING SYSTEMS

mostly devices based on the BCI technology. Unfortunately, the quality of our EEG signal is gettingworse in the era of ubiquitous range of mobile networks as well as wireless networks, electromagneticfield and other interferences emitted by many devices around us. Efficient and accurate solutions areneeded to build a new generation of BICI interfaces - Industrial Brain Computer Interfaces.

2. BRAIN COMPUTER INTERFACES

There are more and more possibilities in terms of Brain Computer Interfaces technology from yearto year. It can serve as a backup in the case of the authorization of the access to systems and computernetworks. It is also used in the non-contact DNS querying on the Internet. Emotivs and NeuroSkyproduce newer devices based on BCI technology applications typically entertaining. Many researchcenters in the world and well-known leading brands, including, for example, Toyota conducts researchon the implementation of BCI technology to support the work of wheelchairs, expanding them to theability to control signals flowing directly from human brain. In the field of economics, BCI technologyis also reflected in neuromarketing. Internet neuromarketing is also an area which is currently beingdeveloped and it also has its operation based on BCI. Conquests of science from brain-computerinterfaces have been also used in the treatment of children with ADHD. A large increase in interest inthe above-mentioned technology is noticeable not only in the field of science. As mentioned in the firstpart, the author of this publication presents the concepts of the implementation of BCI technology inthe control processes of an industrial computer, IPC. The BCI technology is based on a non-invasivemeasurement method, as it is simpler and less expensive to implement than an invasive method. Whatis the most important, it is not associated with any direct interference with the human structure. In thismethod it is possible to select the analysis of the signal flowing from the interior of the brain by MRIor an electroencephalographic study. fMRI is more expensive and more difficult to use in industrialenvironments. Therefore, the second method of EEG signal acquisition, an electroencephalographicstudy was used for the construction of a BICI system [6].

3. INDUSTRIAL PC IN THE CONTROL PROCESSES

Industrial computers are objects undergoing the control process in the authors BICI system. Theyare used mainly in production plants. Their characteristic feature is the fact that they are more resistantto the conditions in which they work. This fact directly impacts the increased level of their reliableoperation. Industrial PCs, IPCs, equipped with a network interface CANbus standard, allow engineers fora relatively quick and easy implementations of necessary hardware and software tools to improve theirreliability and functional flexibility and simplify subsequent activities related to the maintenance andupdating of the software version in industrial applications. Modern industrial applications are becomingincreasingly complex and require the use of appropriate tools equipped with features such as a quickcontrolling the rotational speed of drive systems, a synchronization of a complex axial drive system, amaintenance of specialized analog modules and advanced interfaces of HMI. Communication interfacessuch as network and peripherals interfaces, remote access capabilities, operator interfaces as a specialkeyboard or panels, including touch and graphic panels should be taken into account when industrialsystems with IPCs are designed. BCI technology allows for great opportunities in case of realizations ofremote access to IPCs. An industrial PC should be equipped with serial communication interfaces suchas RS 232, 422, 485, IrDA, Bluetooth, USB 3.0, WiFi or any other wireless communication standard oftelecommunication networks and the previously-stated connectors for additional keyboards, panels anddisplays in addition to the standard of a communication network. USB 3.0 and Bluetooth are importantfrom the point of view of the proposed system BICI.

56


4. THE CONCEPT OF BICI INCLUDING METHODS OF ANALYSES, PCA AND ICA

When a situation arises to change parameter values by using a touch panel which in most caseshave IPCs working in industrial automation applications, it is necessary to use arms to their setting.If a person at a given moment of time who is busy with his or her hands and has active headphoneson his head with built-in electrodes, could change these parameters / values without much trouble andwithout a loss of time and any interruption with the activity. This measure could also improve safety byreducing the response time to a given event. The BICI system based on the BCI technology, equippedwith appropriate mechanisms for the analysis of electroencephalographic signals which correct noiseand interferences in the form of technical and biological artifacts. For this purpose, the PCA and ICAmethod have been implemented. Figure 1 shows the general concept of the BICI operation. The conceptof the device is based on Emotiv Epoc NeuroHeadset, which is a multi-channel device that operatesin Bluetooth technology, therefore, in an easy and a fast way it is possible to establish communicationbetween the active elements in the form of electrodes placed on the head of a person who works in aproduction hall and the IPC.

Fig. 1. General concept of the BICI system.

For the purposes described in this article, Emotiv Systems Inc. presented in Figure 2 was used. Thegoal that led the manufacturer of the device was to obtain the best parameters with a minimum numberof electronic components. Working with electroencephalographic signals requires that you configure ahigh sensitivity and accuracy in mapping the analog to digital signal. A set of filters which eliminatesthe undesirable harmonic signal and several amplifier stages including measuring amplifiers was usedto obtain the output amplitude of a sufficient value.

Fig. 2. Emotiv EPOC Neuroheadset device.

Computers for research purposes are equipped with a specially written for the BICI system applicationEEGVis2 that after the right correlating to a specific responded person perform a particular activity onthe operator panel. A year earlier a similar application research was used to control the mobile robot[1]. An aforementioned engine of this application was written in Java, which interacts with softwareprovided by the manufacturer of NeuroHeadset Emotiv Epoc, Emotiv Inc. Tests in the laboratory for

57


biomedical measurements at Opole University of Technology were also performed on the basis of thepopulation neuronal cell fraction model. One method which can be successfully applied to the EEGsignal analysis is Principal Component Analysis [9]. It is a typical example of the statistical methods ofa factor analysis. As the main objective of the method of principal component analysis is the rotation ofthe coordinate system in such a way as to maximize the gain variation of the first coordinate, and thenthe next coordinates. With a sample set of input data in the form of the EEG signal, which is treated asN observations, which are each highlighting a number of variables M, one can assume that N determinesthe points in the M-dimension [5]. In this way, we obtain a new observation space, which constitutethe initial factors. The method for analyzing the EEG signal helps to reduce the amount of informationcontained in the signal by the rejection of certain components containing disturbing artifacts. When youuse this method it is possible to provide an input file, the EEG signal, in the form of a correlation matrixor a covariance matrix. From the point of view of the EEG signal analysis it is preferable to use thecovariance matrix, since the values of the variables in this case are comparable in size. According to astudy in the set of input variables having the largest variance have the greatest impact on the outcome[7]. The algorithm of the PCA method is powered by the matrix of the input data X consisting of anumber of observation of the EEG signal. On the basis of these data base vectors of the new spaceare stated. When performing an algorithm it is necessary to determine the average values for the rowsof the matrix, to calculate the matrix of the deviation, to calculate the eigenvalues of the covariancematrix, to choose the eigenvalues, to set eigenvectors, then make a projection for the above mentionedvectors. The input matrix of the covariance is formed with regard to formula (1), which defines thevector of mean values u[m].

u[m] =1

N

N∑n=1

X[m,n] (1)

Positions of this vector record average values of the corresponding rows in the matrix. From eachelement of the matrix the average of the line in which it is located is substracted (equation 2).

X‘[i, j] = X[i, j]− u[i] (2)

The next step is to calculate the matrix V of the eigenvectors which satisfies the dependence (3),assuming that D is a diagonal matrix of the eigenvalues dominant to the own values C.

V−1CV = D (3)

Then the dimension of the space shall be verified. The largest eigenvalues are selected, so that it willminimize the loss of information which occurs during the process of projecting the data on a smallernumber of dimensions. The result of the selection of a subset of eigenvalues λ in the matrix for thereceived data stored by the system of linear equations (4) Gaussian elimination algorithm should beused. aij is the covariance matrix.

a11 − λ a12 · · · a1na21 a22 − λ a2n

... . . . ...an1 · · · ann − λ

∗x1x2...xn

(4)

The following clarification of eigenvectors makes it possible to determine the point in the new space,which will correspond to a particular observation vector by multiplying the matrix (5), wherein V isthe matrix of eigenvectors, x is a vector projected, y is a vector of the new space, and N is a numberof eigenvectors. In this case, the vector y can be written using the formula (5).

y =

y0y1...

yn−1

= V T ∗ x =

vT0vT1...

vTn−1

∗ x (5)

58


Another method to analyze the EEG signal is ICA [2]. It is perfect for separating different types ofnoise sources that may have a negative effect on the EEG signal. Figure 3 shows the general idea ofthe EEG signal processing.

Fig. 3. EEG signal processing concept.

This method is well suited for solving the problem of BSS - Blind Signals Separation [4]. TheEEG signal which is stored in the form of a vector −→x , can be illustrated using the above-mentionedproblem. The signal comes from a number of active electrodes placed on the subjects head. It is alinear combination of several statistically independent signals −→s , originating from a population ofsimultaneously active neurons. To solve the above problems mixing matrix H were additionally used.Then the vector −→x is defined by the formula (6), which is the product of the mixing matrix andindependent signals.

−→x = H−→s (6)

We are looking for a solution for this problem in a separate matrix W for which a dependence appears(7).

−→y = W−→x (7)

This vector is strongly correlated with signals −→s .The need to maintain high statistical independenceof components −→y require the use of the correlation function. In this case, it is necessary to rotatethe diagonal covariance matrix PCA or to reset statistics to the second order. The process of operatingprocedures that implement the requirements of independence greatly simplifies that when a de-correlationis obtained. Specified artificial neural networks are often used for this purpose. In conclusion, byapplying the ICA method it is possible to make estimates of unknown source signals and the extractionof unwanted interfering signals in terms of their subsequent elimination [3]. According to the abovedescribed idea it can be assumed that the signals received by the active electrode from the scalp of thesubject form linear combinations that can be saved by using the following formula (8).

x(k) = Hs(k) + v(k) (8)

where the vector of the observed EEG signals is (9):

x(k) = [x1(k), x2(k),...., xm(k)]T (9)

the mixing matrix with mxn dimensions (10):

HεRmxn (10)

the vector of the source EEG signals (11):

s(k) = [s1(k), s2(k),...sn(k)]T (11)

59


the vector of the additive noise (12):

v(k) = [v1(k), v2(k),...vn(k)]T (12)

In Figure 4 examples of EEG signals are shown, obtained from electrodes placed on a persons headcontrolling a computer application, the IPC type.

Fig. 4. Examples of EEG signals from the electrodes.

According to the PCA algorithm stated in this article, the variance values have been calculated onthe following main components (see Figure 5) based on the data taken from the active electrodes andstored in the form of the EEG signal. This method can eliminate the distortion of associated biologicalsignals, in this case the signal of brain electrical activity. For sure, the main components with the greatestvariance value are the most important. In Figure 5, the values of the variance are shown on the verticalaxis and the next components of PCA are shown on the horizontal axis.

After applying a practical method of principal component analysis and independent componentanalysis methods for the EEG signal, it was possible to do a re-roaring of the EEG signal that inthe best way it is suitable to use it in the control process of the industrial computer, IPC. Figure 6shows an example of EEG waveforms including independent components, color-coded green, blue andred.

It is worth noting that the PCA method enables us to considerably reduce the amount of correlatedvariables in the model. This dependence implies that the higher the correlation between the successionof values of the EEG signal the greater the risk of reducing the factors that are necessary to describe theelectroencephalographic signal which is subjected to the observation. From a practical point of view, byusing a variable reduction by means of the PCA method certainly has a positive effect on the efficiencyof the signal modelling generated by neuronal populations. This is due to reduction of the time seriesthat are necessary for description of the mentioned signal. The process of forecasting time series isextremely complicated and time-consuming. It is also important that the generated key components ofthe EEG signal are independent of each other, which greatly simplifies ongoing research. The ICAmethod is relatively versatile and easy to use. Its disadvantage is occurring uncertainty associated withthe lack of one hundred percent verification of the results. It is associated with an attempt to solvethe problem of BSS, which is mentioned in the fourth paragraph of this article. In the method of theICA we have to deal with some ambiguity that arises from the fact that the source signals found are

60


Fig. 5. Variation of values for the next principal components of the EEG signal.

determined to the nearest multiplicative factor, and that for any solution, the solution is allowed witha variable sequence of the source signals which are the EEG signals. This is implied by the fact thatduring the operation of this method, there is a simultaneous estimation of the separation matrix and theEEG signals. It is worth noting that currently there are already many modifications of the ICA, whichinclude among other things more resistance to noise and precise temporal or spatial structure of theinvestigated signals. This fact is particularly important for the identification of the EEG signal basedon a temporal and spatial summation that is seen during the population approach [10],[11].

5. SUMMARY

In conclusion, the use of BCI technology for controlling IPCs in the industrial conditions is un-doubtedly associated with an appropriate programming for the EEG signal processing, as it is the casewhen the EEG signal is used in a biometric authentication of users in LAN networks [8]. Before thesignal will allow to conduct a specific action on the device which is an industrial computer, it mustbe fairly tested in terms artifacts which may appear there. Therefore, the article was devoted to theimportant role of the EEG signal analysis using the method of PCA and ICA. These methods are someof the best methods in terms of speed and relatively low difficulty of implementing these methods. Theyallow for the removal from the signal of the components which an industrial operating equipment orelectromagnetic fields can emit.

The use of BCI interfaces in the control processes of IPCs will accelerate the production process orthe response to different situations over which the control is required from an application running underthe operating system on the IPC.

61


Fig. 6. Examples of EEG signals after processing algorithms, PCA and ICA.

BIBLIOGRAPHY

[1] BLACHOWICZ A., PASZKIEL Sz., A mobile system for measurements of incomplete discharges controlled by electroencephalo-graphic waves, Journal of Automation, Mobile Robotics Intelligent Systems, JAMRIS, Warszawa, 2010, Vol. 4, No. 3, pp. 31-35.

[2] COMON P., Independent Component Analysis a new concept?, Signal Processing, 1994, Vol. 36, pp. 287-314.[3] DELORME A., MAKEIG S., Eeeglab: an open source toolbox for analysis of single-trial EEG dynamics including independent

component analysis; Journal of Neuroscience Methods, 2004, 134(1), pp. 9-21.[4] JUTTEN C., HERAULT J., Blind separation of sources, Part I: An adaptive algorithm based on neuromimetic architecture, Signal

Processing, 1991, Vol. 24, pp. 1-10.[5] KRZANOWSKI W. J., Principles of Multivariate Analysis: A User’s Per-spective. Oxford University Press, 2000.[6] MAJKOWSKI J., Elektroencefalografia kliniczna; Panstwowy zakład wydawnictw lekarskich, Warszawa, 1989.[7] MEGHDADI A.H., FAZEL-REZAI R., AGHAKHANI Y., Detecting determinism in EEG signals using principal component analysis

and surrogate data testing; Conf Proc IEEE Eng Med Biol Soc. 2006.[8] PASZKIEL Sz., ZMARZLY D., KAWALA A., SZMECHTA M., Zastosowanie pomiarow elektroencefalograficznych EEG w procesie

uwierzytelniania biometrycznego uzytkownikow; Miesiecznik naukowo-techniczny Pomiary, Automatyka, Kontrola, Warszawa, 1997,Vol. 53 BIS 9’2007, pp. 433-436.

[9] PASZKIEL Sz., Wykorzystanie metody PCA i ICA do analizy sygnalu EEG w kontekscie usuwania zaklocen, Pomiary AutomatykaKontrola - PAK 2013, Warszawa, 2013, Vol. 59, 3/2013, pp. 204-207.

[10] PASZKIEL Sz., Zastosowanie modeli populacyjnych w interfejsach mozg-komputer, Monografia "Mlodzi Innowacyjni 2012, podredakcja prof. Janusza Kacprzyka, Wyd.: Instytut Automatyki i Pomiarow PIAP we wsposlpracy z Narodowym Centrum Badan iRozwoju NCBiR, 2013, pp. 152-170.

[11] PASZKIEL Sz., Koncepcja systemu dwumodulowego, laczacego modelowanie populacyjne frakcji komorek neuronalnych zalgorytmami analizy artefaktow sygnalu EEG, Pomiary Automatyka Kontrola - PAK 4/2012, Warszawa, 2012, pp. 361-364.

[12] PASZKIEL Sz., BLACHOWICZ A., Zastosowanie BCI do sterowania robotem mobilnym, Pomiary Automatyka Robotyka PAR02/2012, Warszawa, 2012, pp. 270-274.

[13] TADEUSIEWICZ R., et. all, Neurocybernetyka teoretyczna; Wydawnictwo Uniwersytetu Warszawskiego, Warszawa, 2009.

62

the use of brain-computer interfaces in control...

Documents