Ordinal analysis of EEG time series

Karsten Keller1, Heinz Lauffer2, Mathieu Sinn1,1 Institute of Mathematics, University Lubeck

2 Department of Pediatric Medicine, University Greifswald

Abstract

Ordinal time series analysis is a new approach to the qualitative investigationof long and complex time series. The idea behind it is to transform a giventime series into a series of ordinal patterns each describing the order rela-tions between the present and a fixed number of equidistant past values at agiven time. Here we consider ordinal pattern distributions and some measuresderived from them in order to detect differences between EEG data.

Keywords: Time series, Symbolic dynamics, Ordinal Patterns, Sliding time win-dow analysis

1 Introduction

The idea of ordinal time series analysis is to transform given time series . . . , xt−4,xt−3, xt−2, xt−1, xt, xt+1, xt+2, . . . into series of ordinal patterns describing the orderstructure of time-dependent delay vectors (xt, xt−τ , . . . , xt−dτ ) of length d + 1 for agiven delay τ and order d. Having the transformed data on the hand the distributionsof ordinal patterns in different parts of the time series are of a particular interest.(Note that different delays reveal different details of the structure of a time series,in the applications given in this paper, however, the delay is always equal to one.)There are many advantages of this method, despite the fact that the mean amplitudeinformation in the neighborhood of some time point is left.

First of all, the method is conceptionally simple. The system behind the time series,which we assume to be the brain here, can be considered as a source producing pat-terns. Those patterns are recognized by an electroencephalogram (EEG) measuringthe brain activity from a system of electrodes placed on the scalp. Roughly speak-ing, the described transformation maps ‘brain patterns’ to ordinal patterns, whichvary in time and between different regions of the brain. Comparing the patterndistributions obtained one can go from the top to the bottom. For example, onefirst asks for temporal changes in the pooled pattern distribution not distinguishingthe different electrodes. If there are some, one specifies changes for the differentlobes or loci of the brain represented by ensembles of electrodes or single electrodes.This idea is demonstrated in Keller and Lauffer [6] and in Keller and Wittfeld [9].

1

The analysis based on the method is robust and flexible. In particular, ordinalpatterns are invariant with respect to monotone changes of the original data. Thissometimes implies that a non-stationary process transforms to a (nearly) stationary‘ordinal process’. The transformation of a time series into a series of ordinal patternscan be done in a computationally very fast way. Once given the ordinal patternsand their distribution, one can use a variety of characteristics from nominal statis-tics and information theory to quantify and visualize (long term) changes in timeseries and differences between time series and the systems behind. For example,the permutation entropy introduced by Bandt and Pompe [1], who have initiatedordinal time series analysis, is based on the Shannon entropy. Note that the useof permutation entropy in EEG analysis was first illustrated in Cao et al. [4] andKeller and Lauffer [6]. A generalized correspondence analysis for ordinal patterndistributions has been introduced in Keller and Wittfeld [9].

In the following section we outline the computational background behind ordinaltime series analysis. Section 3 discusses a sliding window analysis for illustratingdynamics of brain activity on the base of the ordinal method. In particular, weconsider the permutation entropy and introduce some new quantifiers for ordinaltime series analysis. Finally, we give an illustration of the described methods by amore clinical example in Section 4.

2 Ordinal calculus

As noted above, ordinal time series analysis is based on transforming a time seriesinto a series of ordinal patterns. Here we give the exact definition of an ordinalpattern and present different ways of describing it. For a presentation of the detailssee [8].

Ordinal patterns. We assume a real-valued time series (xt)t∈T , where T = Z forthe sake of simplicity. The adaption to the practical case with T = {r, r +1, . . . , s−1, s} for some r, s ∈ Z, however, will be obvious. The ordinal pattern of orderd ∈ N = {1, 2, 3, . . .} and delay τ ∈ N at time t is defined as the unique permutation

πτd(t) =

(0 1 2 . . . dr0 r1 r2 . . . rd

)= : (r0, r1, r2, . . . , rd)

of the set {0, 1, . . . , d} satisfying

xt−r0τ ≥ xt−r1τ ≥ . . . ≥ xt−rd−1τ ≥ xt−rdτ(1)

and

rl−1 > rl if xt−rl−1τ = xt−rlτ .(2)

What in fact πτd(t) provides is a ranking of the times t, t − τ, . . . , t − dτ such that

the corresponding values are in descending order. Note that we do not emphasize

2

equality of two values: If there are two equal values among those at times t, t −τ, . . . , t − dτ we consider the more recent one as if it was smaller. In particular,constant parts within the original time series yield the same ordinal patterns asdecreasing ones. Of course there are other ways to handle the case of equality. Wedo it in the way given mainly to describe ordinal structure by permutations, havingin mind time series in which equality is rare.

For example, let d = 5 and τ = 3 and consider the following vector obtained fromsome fictive time series at time t = 40 (see Figure 1):

(xt, xt−τ , xt−2τ , xt−3τ , xt−4τ , xt−5τ ) = (x40, x37, x34, x31, x28, x25)

= (0.2, 0.5, 0.7, 0.6, 0.8, 0.3).

Here xt−4τ > xt−2τ > xt−3τ > xt−τ > xt−5τ > xt, hence πτd(t) = π3

5(40) =(4, 2, 3, 1, 5, 0).

Figure 1: Ordinal pattern

Alternative representations. Let us consider representations of ordinal patternsbeing alternative to its permutation form.

Given a permutation π = (r0, r1, r2, . . . , rd) of {0, 1, . . . , d}, for l = 1, 2, . . . , d let

il = il(π) = #{r ∈ {0, 1, . . . , l − 1} | π−1(r) > π−1(l)}

be the number of all r = 0, 1, . . . , l − 1 with position greater than that of l. It is awell known fact that π is uniquely coded by the sequence (i1, i2, . . . , id) (see [10]).For a time series as given above one easily sees that

il(πτd(t)) = iτl (t) := #{r ∈ {0, 1, . . . , l − 1} | xt−lτ ≥ xt−rτ}

for l = 1, 2, ..., d, hence πτd(t) is coded by the inversion representation (iτ1(t), i

τ2(t),

. . . , iτd(t)). Here note that if πτd(t) is coded by (iτ1(t), i

τ2(t), . . . , i

τd(t)), then for l =

1, 2, . . . , d−1 the permutation πτl (t) is coded by (iτ1(t), i

τ2(t), . . . , i

τl (t)). The following

equality allows an efficient determination of successive τ -distant ordinal patterns:

iτl (t + τ) =

{iτl−1(t) + 1 if xt−(l−1)τ ≥ xt+τ

iτl−1(t) else.

3

The consideration of lexicographic order on the set of inversion representations oforder d yields a somehow natural linear arrangement of ordinal patterns. Moreprecisely, the assignment

(i1, i2, . . . , id) 7→d∑

l=1

il(d + 1)!

(l + 1)!.

defines a bijection from the set Id = {0, 1} × {0, 1, 2} × . . . × {0, 1, . . . , d} onto{0, 1, . . . , (d + 1)!− 1} turning the lexicographic order on Id into the usual order on{0, 1, . . . , (d + 1)!− 1}, hence

nτd(t) =

d∑l=1

iτl (t)(d + 1)!

(l + 1)!

provides a number representation of ordinal patterns: each pattern of order d iscoded by one of the numbers 0, 1, . . . , (d + 1)! − 1. This follows from the relation∑d

l=k l (d+1)!(l+1)!

= (d+1)!k!

− 1, which can be shown for k = d, d− 1, . . . , 2, 1 by induction.

Remark. The inversion representation of an ordinal pattern can be obtained fromthe number representation by an inductive procedure on the base of the formula

nτj (t) = (j + 1)

j−1∑l=1

iτl (t)j!

(l + 1)!+ iτj (t) = (j + 1) nτ

j−1(t) + iτj (t)

for j = 1, 2, . . . , d with nτ0(t) := 0. It provides that nτ

j−1(t) is the integer part ofnτ

j

j+1

and iτj (t) the remainder of nτj (t) for division by j + 1.

The number coding is advantageous for determining pattern distributions sincethe comparison of two ordinal patterns is reduced to the comparison of two in-tegers. In particular, we will use the linear arrangement of ordinal patterns forthe visualizations of time-dependent pattern distributions. Note that nτ

d(t) = 0,(iτ1(t), i

τ2(t), . . . , i

τd(t)) = (0, 0, . . . , 0) and πτ

d(t) = (0, 1, . . . , d) are equivalent, andnτ

d(t) = (d + 1)! − 1, (iτ1(t), iτ2(t), . . . , i

τd(t)) = (1, 2, . . . , d) and πτ

d(t) = (d, d −1, . . . , 1, 0) too. This means that with respect to the linear arrangement the permu-tations (0, 1, . . . , d) and (d, d− 1, . . . , 1, 0) have maximal distance, which is naturalsince these patterns represent completely opposite monotone behavior of the origi-nal time series. The arrangement of the patterns is discussed in detail from a moregeneral viewpoint in Keller and Sinn [8].

3 Pattern distributions and derived measures

Our analysis of a time series is based on the distribution of ordinal patterns init. Since we are interested in time-dependent results, we focus on a sliding timewindow analysis with fixed window length δ, i.e. for each t ∈ T we determine

4

ordinal patterns at times t, t− 1, ..., t− δ +1 for given d, τ ∈ N and the frequency ofeach of the (d + 1)! ordinal patterns of order d. So we obtain a series (Dτ

d(t))t∈T =(Dτ

d(t, δ))t∈T of pattern distributions, where distributions are considered as vectors(p0, p1, . . . , p(d+1)!−1) with pi denoting the relative frequency of the ordinal patternwith number representation i.

H13 (45s) up1

3(45s) down13(45s) changes1(45s) H1

3 (73s) up13(73s) down1

3(73s) changes1(73s)

a) b)

c) d)

e) f)

Figure 2: Illustration. a), b): Transformed EEG-data and time dependent patterndistributions. c), d): Pattern distributions at 45 and 73 seconds, respectively. e), f):Derived measures.

The upper plots in Figures 2 a), b) show the number representations of two trans-formed EEG-samples, each of a length of 200 seconds, corresponding to 200 · 256 =51200 data points. (The sampling rate of all data considered in this paper is 256Hz.)Here d = 7 and τ = 1, and the number representing an ordinal pattern was dividedby (d+1)!, in order to get numbers in [0, 1]. The extremely truncated representationin time direction allows to make out black and white horizontal ‘stripes’ suggestingstationarity of the underlaying ‘ordinal processes’. Figures 2 a), b) indicate that thestructure of the two time series considered seems to be quite different. Looking atthe corresponding pattern distributions allows to quantify this observation.

The time dependent pattern distributions for order d = 3, delay τ = 1 and timewindow length δ = 512 (i.e. 2 seconds) are illustrated by the plots in the secondrow of Figures 2 a), b). At any time t the distribution Dτ

d(t) = (p0, p1, . . . , p23) isvisualized by the space between the bottom line and the first curve representing p0,the space between the first and the second curve representing p1, ..., and the spacebetween the upper curve and the top line representing p23. Matching the observa-tions from above there are only small differences within the series of distributions

5

while the both distribution series can be clearly distiguished. In the following welook at some measures characterizing ordinal pattern distributions and so allowingto quantify differences between them.

Permutation entropy. One detail of a given distribution Dτd(t) = (p0, p1, . . . ,

p(d+1)!−1) is described by the permutation entropy

Hτd (t) = −

(d+1)!−1∑i=0

pi ln pi

introduced by Bandt and Pompe as a measure for quantifying the complexity of atime series and the system behind it. It has turned out to be relatively robust with re-spect to observational and dynamical noise and is strongly related to topological andKolmogorov-Sinai entropy in the case of time series coming from one-dimensionaldynamical systems (see [1, 2]). This relationship has been also discussed by Mis-iurewicz [11]. Roughly speaking, it suggests that the partition defined by ordinalpatterns is (approximately) generating, saying that the ‘ordinal symbolic dynamics’representation contains much information on the original dynamical system.

Have a closer look at the pattern distribution for the data behind the left part ofFigure 2 at time t = 45s(= 45 · 256). Here D1

3(t) is given by

p0 = 0.118, p1 = 0.048, p2 = 0.036, p3 = 0.026, p4 = 0.057, p5 = 0.027,

p6 = 0.020, p7 = 0.023, p8 = 0.013, p9 = 0.017, p10 = 0.028, p11 = 0.072,

p12 = 0.067, p13 = 0.037, p14 = 0.013, p15 = 0.023, p16 = 0.027, p17 = 0.019,

p18 = 0.027, p19 = 0.045, p20 = 0.042, p21 = 0.028, p22 = 0.049, p23 = 0.137.

This distribution and the distribution of ordinal patterns for the ‘right’ EEG part att = 73s are illustrated by Figure 2 c) and d), respectively. The resulting permutationentropies are H1

3 (45s) = 2.054 and H13 (73s) = 2.086. The normalized entropies

obtained by dividing the original ones by ln 24 = 3.178 are shown in Figure 2 e), f).Note that the permutation entropy Hτ

d ranges in [0, ln(d + 1)!].

Despite the obvious differences between the distributions illustrated in the his-tograms the resulting permutation entropies are nearly identic. Clearly, permutationentropy is invariant under permutation of the pattern frequencies. This is why wehave proposed to consider not only the permutation entropy for the investigationof time series but also further information contained in the distribution of ordinalpatterns (see Keller and Sinn [8]). Here we derive some measures revealing differentdetails of this distributions.

Length of monotone parts. We have found that for many EEG time series muchof the quantitative changes of permutation entropy is due to changes in the amountof monotonically increasing and monotonically decreasing behavior. More precisely,for low d and τ a low entropy is often correlated to a more frequent occurance of

6

ordinal patterns with number representation 0 or (d + 1)! − 1 (corresponding toincreasing and decreasing parts in the original time series).

In order to quantify monotonic behavior, we introduce upτd and downτ

d as measuresfor the mean length of monotone parts. For this at each time s in the time window{t− δ +1, t− δ +2, . . . , t} we consider the present value xs and count the number ofvalues in the delay vector (xs, xs−τ , . . . , xs−dτ ) one can go back in time in a (strictly)monotone way. Averaging provides

upτd(t) = upτ

d(t, δ) :=1

δ

t∑s=t−δ+1

] {l ∈ {1, 2, . . . , d} |xs−lτ < . . . < xs−τ < xs}

and

downτd(t) = downτ

d(t, δ) :=1

δ

t∑s=t−δ+1

] {l ∈ {1, 2, . . . , d} |xs−lτ ≥ . . . ≥ xs−τ ≥ xs}.

Indeed upτd(t, δ) and downτ

d(t, δ) only depend on the distribution Dτd(t, δ) = (p0, p1,

. . . , p(d+1)!−1) since

xs−lτ < . . . < xs−τ < xs ⇐⇒ iτ1(s) = iτ2(s) = . . . = iτl (s) = 0

⇐⇒ nτd(s) <

(d + 1)!

(l + 1)!

with the latter equivalence following from ]{0, 1, . . . , l +1}×{0, 1, . . . , l +2}× . . .×{0, 1, . . . , d} = (d+1)!

(l+1)!, and

xs−lτ ≥ . . . ≥ xs−τ ≥ xs ⇐⇒ iτ1(s) = 1, iτ2(s) = 2, . . . , iτl (s) = l

⇐⇒ nτd(s) > (d + 1)!− 1− (d + 1)!

(l + 1)!

with the same argument as above, implying that

upτd(t) =

d∑l=1

∑j<

(d+1)!(l+1)!

pj(3)

and

downτd(t) =

d∑l=1

∑j>(d+1)!−1− (d+1)!

(l+1)!

pj.(4)

For the distribution in Figure 2 c) with the above given pattern frequencies (p0, p1,. . . , p23) we obtain

up13(45s, 512) =

11∑i=0

pi +3∑

i=0

pi + p0 = 1 ·11∑i=4

pi + 2 ·3∑

i=1

pi + 3 · p0

= 1 · 0.257 + 2 · 0.110 + 3 · 0.118 = 0.831

7

and

down13(45s, 512) =

23∑i=12

pi +23∑

i=20

pi + p23 = 1 ·19∑

i=12

pi + 2 ·22∑

i=20

pi + 3 · p23

= 1 · 0.258 + 2 · 0.119 + 3 · 0.137

= 0.907.

These numbers divided by d = 3 (which is the maximum value any up13 and down1

3

can take) are provided in Figure 2 e). The distribution from the right histogramyields up1

3(73s) = 0.572 and down13(73s) = 0.654. For the normalized values see

Figure 2 f).

In contrast to the permutation entropies the numbers upτd and downτ

d, respectively,obtained from the distributions in Figure 2 c) and d) are clearly different. Thenumbers say that the left EEG sample contains longer monotone parts than theright one while their complexity is about the same. The first fact is particularlyrelated to the less frequent occurance of patterns with number representations 0and 23 in the right distribution.

Number of changes. The second measure we want to introduce bases on countingthe number of changes between downward to upward behaviour one finds when goingback in time. Let

changesτ (t) = changesτ (t, δ)(5)

:=1

δ]{s ∈ {t− δ + 1, t− δ + 2, ..., t} |

xs−2τ ≥ xs−τ < xs or xs−2τ < xs−τ ≥ xs}.

In contrast to upτd and downτ

d this number quantifies the amount of ‘zigzag’ in theoriginal time series. Since

xs−2τ ≥ xs−τ < xs ⇐⇒ iτ1(s) = 0 and iτ2(s) > 0

andxs−2τ < xs−τ ≥ xs ⇐⇒ iτ1(s) = 1 and iτ2(s) < 2

one obtains changesτ (t, δ) from Dτd(t, δ) = (p1, p2, . . . , p(d+1)!−1) for d ≥ 2 by

changesτ (t) =

5(d+1)!6

−1∑j=

(d+1)!6

pj.

In particular, changesτ (t) = p1 + p2 + p3 + p4 = 1 − p0 − p5 for d = 2. Clearly,changesτ (t) is maximally equal to 1, and in the case τ = 1 the value 1 is assumedif at each s ∈ {t − δ, t − δ + 1, . . . , t − 1} the time series changes between up anddown. Note that the frequencies of patterns obtained by xs−2τ ≥ xs−τ < xs andxs−2τ < xs−τ ≥ xs, respectively, differ by at most 1, such that doubling one of thefrequencies approximates the whole frequency.

8

Remark. Clearly, upτd, downτ

d and changesτ can be also obtained from the simplepatterns of order 1 comparing only values for time difference τ , but not from theirdistribution itself. Here time information is necessary. Formulae (3), (4) and (5) areparticularly interesting for considering pooled ordinal pattern distributions.

The distributions represented by Figure 2 c), d) provide changes1(45s, 2s) = 0.514and changes1(73s, 2s) = 0.798, respectively, which are represented in Figures 2 e),f). Again in contrast to the permutation entropies these numbers substantially differ,saying that the right EEG signal contains (much) more ‘zigzag’ then the left one,matching the above statement about the amount of monotone parts in the signals.

Figure 3: Side differences. Upper two plots: Time dependent pattern distributionsbelonging to channels O2 (black) and O1 (gray). Third plot: Time dependent(normalized) permutation entropies. Fourth plot: Time dependence of the measuresup1

3 and down13. Fifth plot: Time dependence of changes1.

4 Application to EEG analysis

The above provided methods have turned out to be well suited for long term EEGanalysis. Let us look at an example. For the medical terms see [3]. The MRI inFigure 4 shows the brain of an infant of three months with an extended cerebral mal-formation in the right temporal, parietal and occipital lobes. Note the enlargementof the whole posterior right cerebral hemisphere with a blurred border of gray andwhite matter. (The ‘R’ on the left side indicates the right side of the brain; the MRIreverses left-right orientation.) The clinical picture consisted of a marked hypotonia

9

with motor retardation and partial seizures in the left arm and face. Seizures couldbe controlled by phenobarbital and phenytoin.

Figure 4: MRI and electrode placing

For order d = 3, delay τ = 1 and time window length δ = 512 the two above plotsin Figure 3 show the time-dependent ordinal pattern distributions obtained fromthe EEG data of channel O2 (first plot) and O1 (second plot). The correspondingelectrodes are placed at the right and left occipital cortex, respectively. For thewhole electrode placement see Figure 4. (Again ‘R’ indicates the right side. Thedirection from the front to the back is the same as for the MRI.) The original EEGdata show focal slowing in the region of the cerebral malformation (T4, T6, P4, O2).This in particular yields a higher frequency of the ordinal patterns 0 and 23 whichrepresent monotone increasing and decreasing parts in the original time series.

The third plot in Figure 3 shows the (normalized) permutation entropies H13 of

channels O2 (black) and O1 (gray). It is reduced over the right occipital region (O2)in comparison with the opposite side (O1). The fourth plot visualizes the numbersupτ

d and downτd for both channels. The curves belonging to O2 are again drawn in

black, those for O1 in gray. One finds downτd being higher for both channels all the

time expressing that in the mean there is a longer decreasing than increasing in theoriginal data. This asymmetry is very interesting, we however have no explanationfor it. Here note in particular that the numbers upτ

d and downτd obtained from

channel O2 are higher than those from channel O1 which matches with the observedfocal slowing in the right parieto-occipital cortex.

The fifth plot finally visualizes the time-dependency of changesτ for O2 (black) andO1 (gray). Here in contrast to upτ

d and downτd the numbers obtained for O1 are

higher than those for O2.

10

5 Conclusions

We have discussed ordinal time series analysis and its applicability to EEG data. Inparticular, we have introduced some simple measures based on the distribution ofordinal patterns of a time series, supplementary and alternatively to the permuta-tion entropy. The measures have been applied for describing qualitative differencesbetween EEG data. For visualizing the differences we have used curve represen-tations (compare Figure 3). These representations can be simply interpreted andindicate differences rather than evaluating in terms of statistical significance, thereis however a general need of statistical modelling in ordinal time series analysis.The results presented (and results in [4, 6, 7]) show that ordinal time series analysiscould be a promising approach for exploring EEG time series and studying braindynamics.

References

[1] C. Bandt, B. Pompe, Permutation entropy: A natural complexity measure fortime series, Phys. Rev. Lett. 88 (2002), 174102.

[2] C. Bandt, G. Keller, B. Pompe, Entropy of interval maps via permutations,Nonlinearity 15 (2002), 1595 – 1602.

[3] W. T. Blume, M. Kaibara, Atlas of Pediatric Electroencephalography, Lippincott-Raven, Philadelphia 1999.

[4] Y. Cao, W. Tung, J.B. Gao, V.A. Protopopescu, L.M. Hively, Dedecting dy-namical changes in time series using the permutation entropy, Phys. Rev. E 70(2004),

[5] P. Diaconis, Group representations in probability and statistics, IMS, Hayward,California, 1988.

[6] K. Keller, H. Lauffer, Symbolic analysis of high-dimensional time series,Int. J. Bifurcation Chaos 13 (2003), 2657 – 2668.

[7] K. Keller, M. Sinn, Ordinal analysis of time series, Physica A 356 (2005), 114 –120.

[8] K. Keller, M. Sinn, Ordinal symbolic dynamics, Technical Report A-05-14,Lubeck 2005.

[9] K. Keller and K. Wittfeld, Distances of time series components by means ofsymbolic dynamics, Int. J. Bifurcation Chaos 14 (2004), 693 – 704.

11

[10] D. Knuth, The art of computer programming. Volume 3: Sorting and searching,Addison Wesley, Reading, Massachusetts, 1973.

[11] M. Misiurewicz, Permutations and topological entropy for interval maps, Non-linearity 16 (2003), 971 – 976.

12