Seizure Prediction Through Clustering andTemporal Analysis of Micro Electrocorticographic

DataYilin Song 1, Jonathan Viventi 2, and Yao Wang 1

1Department of Electrical and Computer Engineering, New York University, NY, USA2Department of Biomedical Engineering, Duke University, Durham, NC, USA

Abstract—We have developed flexible, active, multiplexedrecording devices for high resolution recording over large,clinically relevant areas in the brain. While this technology hasenabled a much higher-resolution view of the electrical activityof the brain, the analytical methods to process, categorize andrespond to the huge volumes of data produced by these deviceshave not yet been developed. In our previous work, we appliedadvanced video analysis techniques to segment electrographicspikes, extracted features from the identified segments, and thenused clustering methods (particularly Dirichlet Process Mixturemodels) to group similar spatiotemporal spike patterns. From thisanalysis, we were able to identify common spike motion patterns.In this paper, we explored the possibility of predicting seizuresin this dataset using the Hidden Markov Model (HMM) tocharacterize the temporal dynamics of spike cluster labels. HMMand other supervised learning methods are united under thesame framework to perform seizure prediction. These methodshave been applied to in-vivo feline seizure recordings and yieldedpromising results.

I. INTRODUCTION

Currently, many existing neurological data analyses relyon manual inspection. With new high-density, micro-electrocorticographic (µECoG) electrode arrays that providedramatically enhanced spatial resolution, the volume of dataproduced by arrays with hundreds or thousands of electrodesis too large for manual review. Further, manual inspectioncan miss subtle features that automated machine learningtechniques can detect. There is an urgent need for efficientand sensitive automated methods that can analyze the largevolumes of data produced by next generation neurologicdevices.

One major application of high resolution µECoG is torecord brain signals in patients with epilepsy. Preliminaryanalysis of our high resolution data have discovered repetitivespatiotemporal microscale spike patterns that appear to initiateand terminate seizures. These patterns are likely not observedby standard electrodes, as their spatial scale is smaller than thecurrent electrode spacing (approximately 1cm). Understandingthe relationships between these spike patterns is a key steptowards developing a deeper understanding of the microscalephenomena that generate seizures, and ultimately enable us todevelop better therapies to treat epilepsy.

In [11], we compared many video analysis and machinelearning technics to recognize the spatiotemporal patterns

Fig. 1: Photograph of a 360 channel, high density neural electrode array usedin a feline model of epilepsy. The electrode array was placed on the surfaceof visual cortex. The electrode size and spacing was 300 µm x 300 µm and500 µm, respectively.

of electrographic spikes. We have developed a three-layer,pipelined, framework for robust spike segmentation and pat-tern recognition. In the first layer, we extracted spike segmentsfrom raw signal recording via spatial temporal filtering andregion growing[1]. In the second layer, we extracted featuresfrom these segments. Finally, in the third layer we usedunsupervised learning to group recurrent patterns into clusters.The three layers were reflected in Fig. 2 as the first three layersin the diagram.

Most current seizure detection methods fall in two cate-gories: 1) examination of the EEG signals in the frequencydomain using wavelet transform [5], [6], [7] followed by aclassifier such as support vector machine (SVM) or NeuralNetwork. 2) Analysis of the linear or nonlinear temporalevolution of the EEG signals to find a governing rule [3], [4].To the best of the authors’ knowledge no research has beenpublished on an integrated and simultaneous application ofspike segmentation, clustering and temporal dynamic analysisfor seizure prediction.

Based on our preliminary analysis of our data of inducedseizures, it is hard to tell whether a given electrographic spikewere part of a series of interictal spikes or part of a seizurebased on itself. Therefore, classification of one spike should

Pre-­‐Processing Dimension  Reduc1on

Unsupervised  Learning

Supervised  Learning

Data  Preprocessing

Spike  Segmenta1on  

(Region  Growing  and    Labeling)

Feature  Extrac1on  

(Spike  Segment  Correla1on)

Clustering  (Dirichlet  Process  

Mixture  Clustering)

Temporal  dynamic  analysis  

(Mul1ple  Hidden  Markov  Model

Classifica1on  (Support  Vector  

Machine)

Spike  Segments

Reduced  Features

Spike  PaKern    Labels

Data  Recording

Seizure  Predic1on  Output  

Fig. 2: Block diagram showing 4-stage seizure prediction algorithm. In data preprocessing, we spatially interpolates missing channels and filters each channelwith a band pass filter

be based on how T successive segments changed surroundingthe current one. There have been several works using hiddenmarkov model (HMM) [12] for seizure classification andprediction. Most of these are based on a single channel signal[8] [9]. The work in [10] considers multiple channel signalsand is most closely related to our proposed research. Most ofthese prior work use a single HMM, with the hidden statesrepresenting the stage of seizure to be identified.

In our approach, we investigate seizure prediction throughclassifying a spike into one of the three classes (non-seizure,pre-seizure, and seizure). A spike that is classified into the pre-seizure stage indicates the on-set of a new seizure. We firstclassify this spike into one of the pre-determined spike clustersbased on features extracted from this spike signal. Seizurestage classification is based on how the the spike cluster labelchanges in time, over a sliding time window surroundingthe current spike time. This is accomplished through HMMmodeling of the dynamics of spike labels in different seizurestage. If there were inactive period between two spikes, thisperiod will be partitioned into fixed length segments (withlength equal to the average spike segment length), each givena special label that corresponds to its inactive status.

The rest of this paper is organized as follows: In SectionII, we review our prior work on spike segmentation and clus-tering. In Section III we present seizure prediction algorithmusing the clustering label. Section IV concludes the paper witha discussion of the results.

II. SPIKE SEGMENTATION AND CLUSTERING

A. Spatial-Temporal Segmentation

In our acquired dataset, the multi-channel µECoG signalhas a high spatial and temporal correlation, and therefore eachspike could be treated as a spatio-temporally connected set ofvoxels that have high intensity values. We applied 3D regiongrowing to get spike segments. Region growing first randomly

select pixels (called seeds) that has intensity values above apreset threshold Pt. Each seed is used as an initial detectedregion. The algorithm then examines whether any immediateneighbors of the boundary of each previously detected region,Sn, also have high intensity values, with a threshold that wasdetermined based on the mean, ηn, and standard deviation, σnof the pixels inside the region. Specifically, a neighboring pixelwith intensity P was included in Sn if P >= ηn−ασn. Thisprocess continues until no more pixels could be included in theregion Sn. We applied the region growing algorithm to the datacaptured from an acute in vivo feline model of seizures, usingthe parameters Pt = 0.8mV , α = 0.8, Tt = 40ms. Figure 3shows two detected spike segments from one dataset.

B. Feature Extraction

Clustering the spike patterns based on raw data presentsseveral challenges. First, spike segments will have varyingtime durations, leading to different dimensions of the featurevectors. Second, the dimensionality of such feature vectorswill be very large.

To circumvent the above challenges, we used the correlationcoefficients between each spike segment with all other spikesegments in a training dataset to form a feature vector. Assum-ing that the signals corresponding to two spike segments haveduration n and m with n ≥ m. We denoted their correspondingvideo signals by X ∈ RN1N2n, Y ∈ RN1N2m, where theintensity values at voxels not belonging to the spike segmentare set to zero. To compute the correlation between X andY , we found a length-m segment in X that has the highestcorrelation coefficient with Y , and use this maximum as thecorrelation between X and Y .

The raw signal correlation map has a dimension of N ∗N ,if N is the total number of segments in the training set. Eachcolumn in this correlation map served as the feature vectorfor one spike segment. The feature vector consists of the

(a) (b)

(c) (d)

Fig. 3: Examples of spike segmentation. Subfigure a,b are for a spike witha spiral motion: Subfigure a shows the segmented volume in blue overlaidwith the µECoG signals captured at different times, with vertical axis (timeaxis) corresponding to frame number, with the other two axis represent thespatial arrangement of µECoG electrodes; Subfigure b shows the trajectoryof the centroid of the segmented region in successive frames, with the red dotindicating the beginning position; Subfigure c,d are for a spike with a verticalmotion.

correlation of this spike segment with all other segments. Themotivation for using the correlation as the feature vector forclustering was that similar segments should have similar cor-relations with respect to other segments. Correlation featuresmay not depict the spatio-temporal characteristics of the spikesegment directly, but have proven to be more effective thanother features we explored [11], for unsupervised clusteringof spike patterns.

C. Unsupervised Clustering of Individual Spike Pattern

In order to identify all typical spike motion patterns, weperform unsupervised clustering of all spike segments in atraining set. Among the various clustering methods that weexplored [11], the DPM renders more consistent clusters undertwo of our evaluation metrics. DPM is a special mixturemodel, where the mixing coefficient is a random variable thatfollows a certain distribution. We used Mean-Field VariationalInference[2] to learn the DPM parameters that best described agiven set of samples. With this algorithm, only an upper boundneeds to be provided to set the maximum possible numberof mixtures, it automatically determines the mixture numberthat leads to the best fit (in terms of the likelihood) of thetraining data and the model found. The DPM-based clustering

is particularly effective when the training data tend to formclusters that vary greatly in size (i.e. some clusters containmany more samples than other clusters), which is the case withour data. We used a training set to find the cluster numberand the probability distribution of each cluster. For seizureprediction, for each new spike segment, we first derive itsfeature vector by finding its correlation with all spike segmentsin the training set, and then assign it to the cluster with themaximum likelihood.

III. SEIZURE PREDICTION

Once a spike is detected and clustered into one of the pre-determined clusters, we would like to classify it into one ofthree spike categories: interictal, pre-ictal, ictal (i.e seizure)based on the temporal variation pattern of the clustering label.For neural datasets, a significant portion of the data recordingis low amplitude signal, for which we assign a unique label0. For resting period d longer than 100 ms, the number ofzero labels is ceil(d/100), since the average duration for spikesegments is 100 ms. To train the classifier, we manually labelthe ictal stage of each spike segment in training sequences,and assign spikes that occur 8 seconds before ictal stage to bepre-ictal stage spikes.

A. Hidden Markov Model

Hidden Markov Model is quite powerful for temporal dataanalysis, especially when the observations at successive timescan be effectively described as discrete variables. The hiddenstate qt is assumed to consist of N possible values. Theprobability of qt = i is conditionally independent of all otherprevious variables given qt−1. The observation Ot at time t isassumed to vary among K possible values, and is conditionallyindependent of all other variables given qt. Parameters ofHMM are defined as following, where Aij are called transitionprobabilities, Bj(k) the emission probabilities of state i, andπi the stationary probability of state i.:

Ai,j = P (qt = Si|qt−1 = Sj)

Bj(k) = P (Ot = k|qt = Sj)

πi = P (q1 = Si)

B. Using a Single Hidden Markov Models

We have explored two different ways of using HMM forseizure stage classification. In this single HMM approach,hidden states in the HMM correspond to all possible seizurestages (i.e., interictal, pre-ictal, and ictal). Given a sequence ofT observations (observation at time t is the cluster label of thespike at time t), the most likely state transition path then tellsthe classification results for all T spikes. During the trainingstage, we collect observations and their corresponding statelabels to derive the transition and emission probabilities forHMM. Inferring the hidden state label in this case correspondsto finding the Viterbi path. We found that prediction accuracybegins to decrease as the observation sequence length reachedbeyond a certain point. Therefore instead of using all pastobservations up to time t, we use only a fixed number of past

observations. Through trial and error, we found that using 20observations yielded the best result. We have experimentedwith two ways for assigning the initial state of HMM, 1) usingthe stationary probability, 2) using the state assigned to the firstspike segment when applying HMM classifier to the previousoverlapping observation sequence.

C. Using Multiple Hidden Markov Models plus a second stageclassifier (HMM+SVM)

In this approach, we built a separate HMM for each indi-vidual seizure stage. λi = (Ai, Bi, πi) denotes the parametersfor the i-th stage. During the training stage, the goal is toadjust model parameters λi = (A,B, π) to maximize P (O|λi)using observation sequences in the training data for stage i.Model parameters are updated iteratively through Baum-Welchalgorithm [12]. Since we have no prior knowledge of howmany states to choose, the number of hidden states n is chosento maximize P (O|λi). We found that four hidden states workwell for all three different HMMs.

We have found that using the HMM trained separately foreach seizure stage cannot accurately classify a spike during thetransition period from non-seizure to seizure (or vice verse).We suspect that this is because, during the transition period,the observation sequence contains spikes from both the non-seizure period and the seizure period. If we record the actuallikelihood values for all three seizure stages in time, it is likelythat, during the transition period, the likelihood for the non-seizure stage decreases, and the likelihood for the seizure stageincreases. To exploit the likelihood variation pattern over time,we adopted a sliding window approach and find the likelihoodvector (consisting of the likelihood values of the three seizurestages) over each sliding window. As shown in Fig 4, toclassify the observation Ot as indicated by the red doubledashed vertical lines on the top, we collect a total of 2n + 1observations. For each sliding window k window containingn observations, we derive the log likelihood that it belongsto each of the three seizure stages using the 3 trained HMMmodels. Because there are a total of n + 1 sliding windowsover 2n+1 observations, we form a new 3n+3 feature vectorGt that contains the log likelihood values for the three seizurestages of each over the n + 1 sliding windows. We apply atrained classifier to this feature vector to determine the seizurestage of the spike Ot. For this second stage classifier, we usethe multi-class support vector machine with linear kernel andradial basis function (RBF) kernel.

D. Dataset and Prediction Result

We analyzed micro-electrocorticographic (µECoG) datafrom an acute in vivo feline model of epilepsy. Adult cats wereanesthetized with a continuous infusion (3 ∼ 10 mg/kg/hr)of intravenous thiopental. A craniotomy and durotomy wereperformed to expose a 2 x 3 cm region of cortex. The highresolution electrode array was then placed on the surface ofthe brain over primary visual cortex, localized by electrophys-iological recordings of visual evoked potentials. Picrotoxin,a GABA-A receptor antagonist that blocks inhibition, was

Dataset samplingfrequency

recordinglength

labeledseizures

spikessegments

cat 1 277.778Hz

53 min 41sec

7 1685

cat 2 925.925Hz

32 min 20sec

27 3706

cat 2 925.925Hz

26 min 10sec

27 2380

TABLE I: Statistics of the datasets collected from two different animals.Dataset 2 and Dataset 3 are from the same animal, with different implantposition.

topically applied adjacent to the anterior-medial corner of theelectrode array in an amount sufficient to induce abnormalelectrical spikes and seizures from the covered region[13].

The active electrode array placed on the cortex was usedto record data from 360 independent channels arranged in 20columns and 18 rows, spaced 500µm apart. Each electrodecontact was composed of a 300µm × 300µm square of plat-inum. Two high-performance, flexible silicon transistors foreach electrode buffered and multiplexed the recorded signals[13]. The total array size was 10mm× 9mm.

Because of the variation of implant position and timeeffect, the predictor we built is dataset specific. But to avoidoverfitting, we use leave one out cross validation. Namely weuse observation sequences extracted during one seizure period,pre-seizure period (8 sec) before this seizure, and the non-seizure period leading up to this seizure as our testing data,use the rest of the data for training. We repeat this approachfor each seizure.

We compared the performances of both the single HMMclassifier and the HMM+SVM classifier (Fig. 4) for deter-mining the seizure stage of a spike. The spike classificationperformance using the single HMM classifier is given inTable II , results obtained using the HMM+SVM approach aresummarized in Table III. Here the accuracy is defined as thepercentage of spikes that are correctly identified into the threeseizure stages as compared to the manually labeled groundtruth. We can see that the HMM+SVM approach achievessignificantly higher accuracy than the single HMM approach.

Tables II and III considered the spike classification accuracy.Even though the HMM+SVM classifier was able to achievevery high spike classification on average, the classificationaccuracy for the pre-ictal spike is much lower, because pre-ictal spikes occur much less frequently than inter-ictal andictal spikes. Furthermore, we cannot simply consider any spikeclassified as pre-ictal as the onset of a new seizure, as thatwill yield very noisy and inconsistent prediction. Instead, wepredict the on-set of a new seizure if the current spike and 4previous spikes are classified to either pre-seizure or seizurespike. If the seizure actually happened after the predictedonset within 8 sec, we consider the prediction accurate. TableIV summarizes the seizure prediction accuracy using theHMM+SVM approach. We further measure the delay betweenthe predicted on-set and the actual on-set, and report the mean

Recording  

−0.5 0 0.5 1 1.5 2 2.5 3 3.5 4 4.5−2

−1.5

−1

−0.5

0

0.5

1

1.5

2

2.5

3x 10−3

second

mv

Region  Growing  and  labeling

…  

…  

Segments  Sequence   Feature  Extrac:on  and  

DPM  clustering

Label  1  

Label  2  

…  

Label  20  

Label  21  

Label  23  

…  

Label  41  

Label  22  

Label  Sequence  

HMM  predic:on

…   Likelihood  Vector  

Predic:on  Decisions  

Classifica:on

Fig. 4: Illustration of the HMM+SVM seizure prediction scheme. Input of the diagram is the raw µECoG data. Black bracket indicates the process ofsegmenting the raw data into spike segments. Green bracket generalizes the procedure of extracting features for each spike and classifying it into one ofpredetermined spike clusters using pre-trained DPM model. Red bracket generalizes the process of multiple HMM modeling for each overlapping sequenceof 21 spike cluster labels. Three different colors in each box represent the likelihood of the sequence belonging to one out of three HMMs. Blue bracketrepresents SVM classification of the likelihood vector computed from a total of 21 overlapping sequences centered around the current spike. The output ofthe diagram is the classified seizure stage for each spike. Seizure onset is predicted when the current and previous 4 spikes are all classified as either pre-ictalor ictal spikes.

Delay Stationary initial Recursive initialno delay 0.765 0.77010 segments(≈ 1 sec de-lay)

0.779 0.784

TABLE II: Spike classification accuracy for dataset 2 using the single HMMclassifier. The observation sequence length is fixed at 21 spike segments, spikesegment to be predicted is either located in the middle of the sequence(delayapproximately 1 sec) or in the end (no delay). ”Stationary initial” meansthat we used the stationary state distributions to set the initial state of eachobservation sequence. ”Recursive initial” means that we use the state assignedto the first spike segment by the classifier for the previous observationsequence.

Observationlength

svm-rbf svm-linear

21 segments (≈ 1sec delay)

0.920 0.846

41 segments (≈ 2sec delay)

0.940 0.852

TABLE III: Spike classification accuracy for dataset 2 using the HMM+SVMclassifier. Spike segment to be predicted is located in the middle of theobservation

delay in Table IV. Figure 5 shows the distribution of theprediction delay.

IV. CONCLUSION

µECoG has tremendous potential for many research andclinical applications. In this work, we have combined advanced

Dataset prediction ac-curacy

number offalse positive

average timebefore seizure

cat 1 5/7: 0.7143 0 1.86 seccat 2 20/27: 0.7407 1 2.22 seccat 2 24/27: 0.8889 0 2.33 sec

TABLE IV: Seizure prediction accuracy for three datasets using theHMM+SVM approach. Seizure onset is predicted if 5 consecutive spikesare classified to either pre-seizure or seizure stage. The time between thisprediction and the actual onset time is defined as the delay. Seizure predictionis considered correct if -8 sec ≤ delay ≤ 0 sec. The last column reports theaverage of the negative delays among all predictions (including those withpositive delays).

video analysis and machine learning algorithms to analyzethese challenging datasets in novel ways. From our previouswork [11], we have developed efficient methods for identifyingand localizing the spatial and temporal extent of inter-ictal andictal spikes and detecting the spike wavefront through regiongrowing. For those identified spikes we derive correlation fea-tures relying on their relationship with one another. DPM hasrendered consistent clustering patterns using the correlationfeatures.

In this work, we focused on seizure prediction using eachspike segment’s cluster label. We found that a two stageHMM+SVM classifier had a much higher spike classificationaccuracy than a single HMM classifier. The two stage approachyielded accurate seizure detection and possibly prediction,by detecting 49 out of 61 seizures before the human-labeledseizure onset. Our framework has demonstrated the potential

−10 −5 0 5 10 150

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

se

izu

re d

ete

ctio

n r

ate

second

(a) dataset 1

−5 −4 −3 −2 −1 0 1 20

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

se

izu

re d

ete

ctio

n r

ate

second

(b) dataset 2

−5 −4 −3 −2 −1 0 1 20

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

se

izu

re d

ete

ctio

n r

ate

second

(c) dataset 3

Fig. 5: Histogram of seizure prediction delay. Horizontal axis indicate the detection delay. Negative delay means that the seizure is predicted before the actualseizure onset. Vertical axis is the cumulative distribution of the delay. For datasets 2 and 3, all seizures were predicted with up to 2 seconds delay, and morethan 70% of seizures were predicted before the actual onsets. However, for dataset 1, 2 out of 7 seizures were not predicted within 5 seconds of onset.

for seizure prediction, although for most of the inducedseizures of our dataset there did not appear to exist a signif-icant pre-seizure state. Our framework could yield improvedprediction accuracy for more accurate seizure models where asignificant pre-seizure stage exists.

V. ACKNOWLEDGEMENT

This work was funded by National Science Foundationaward CCF-1422914.

REFERENCES

[1] R. Adams and L. Bischof, Seeded region growing, Pattern Analysis andMachine Intelligence, IEEE Transactions on, vol. 16, no. 6, pp. 641647,1994.

[2] David Blei, Michael Jordan.”Variational Inference for Dirichlet ProcessMixtures” Bayesian Analysis (2006). 1, Number 1, pp. 121144.

[3] Arnold, Matthias, et al. ”Adaptive AR modeling of nonstationary timeseries by means of Kalman filtering.” Biomedical Engineering, IEEETransactions on 45.5 (1998): 553-562. APA

[4] MLA Chisci, Luigi, et al. ”Real-time epileptic seizure prediction usingAR models and support vector machines.” Biomedical Engineering, IEEETransactions on 57.5 (2010): 1124-1132.

[5] H. Adeli, Z. Zhou, and N. Dadmehr, Analysis of EEG records in anepileptic patient using wavelet transform, J. Neurosci. Meth., vol. 123,pp. 6987, 2003.

[6] P. J. Durka, From wavelets to adaptive approximations: Time-frequencyparameterization of EEG, Biomed. Eng. Online, vol. 2, p. 1(130), 2003.

[7] Ghosh-Dastidar, Samanwoy, Hojjat Adeli, and Nahid Dadmehr. ”Mixed-band wavelet-chaos-neural network methodology for epilepsy and epilep-tic seizure detection.” Biomedical Engineering, IEEE Transactions on 54.9(2007): 1545-1551.

[8] Chiu, Alan WL, et al. ”Wavelet-based Gaussian-mixture hidden Markovmodel for the detection of multistage seizure dynamics: A proof-of-concept study.” Biomed Eng Online 10 (2011): 29.

[9] Wong, Stephen, et al. ”A stochastic framework for evaluating seizureprediction algorithms using hidden Markov models.” Journal of neuro-physiology 97.3 (2007): 2525-2532.

[10] Santaniello, Sabato, et al.”Quickest detection of drug-resistant seizures:An optimal control approach.” Epilepsy and Behavior 22 (2011): S49-S60

[11] B. Akyildiz, Y. Song, J. Viventi, and Y. Wang, Improved clusteringof spike patterns through video segmentation and motion analysis ofmicro Electrocorticographic data, IEEE Signal Processing in Medicineand Biology Symposium, 2013.

[12] Rabiner, Lawrence. ”A tutorial on hidden Markov models and selectedapplications in speech recognition.” Proceedings of the IEEE 77.2 (1989):257-286.APA

[13] Viventi, Jonathan, et al. ”Flexible, foldable, actively multiplexed, high-density electrode array for mapping brain activity in vivo.” Natureneuroscience 14.12 (2011): 1599-1605.