Slide 1

Lecture series: Data analysis Lectures: Each Tuesday at 16:00 (First lecture: May 21, last lecture: June 25) Thomas Kreuz, ISC, CNR thomas.kreuz@cnr.it http://www.fi.isc.cnr.it/users/thomas.kreuz/

Slide 2

Lecture 1: Example (Epilepsy & spike train synchrony), Data acquisition, Dynamical systems Lecture 2: Linear measures, Introduction to non-linear dynamics Lecture 3: Non-linear measures Lecture 4: Measures of continuous synchronization Lecture 5: Measures of discrete synchronization (spike trains) Lecture 6: Measure comparison & Application to epileptic seizure prediction Schedule

Slide 3

Example: Epileptic seizure prediction Data acquisition Introduction to dynamical systems First lecture

Slide 4

Non-linear model systems Linear measures Introduction to non-linear dynamics Non-linear measures - Introduction to phase space reconstruction - Lyapunov exponent Second lecture

Slide 5

Non-linear measures - Dimension [ Excursion: Fractals ] - Entropies - Relationships among non-linear measures Third lecture

Slide 6

Characterizition of a dynamic in phase space Predictability (Information / Entropy) Density Self-similarityLinearity / Non-linearity Determinism / Stochasticity (Dimension) Stability (sensitivity to initial conditions)

Slide 7

Dimension (classical) Number of degrees of freedom necessary to characterize a geometric object Euclidean geometry: Integer dimensions Object Dimension Point0 Line1 Square (Area)2 Cube (Volume)3 N-cuben Time series analysis: Number of equations necessary to model a physical system

Slide 8

Hausdorff-dimension

Slide 9

Box-counting

Slide 10

Richardson: Counter-intuitive notion that a coastline's measured length changes with the length of the measuring stick used. Fractal dimension of a coastline: How does the number of measuring sticks required to measure the coastline change with the scale of the stick?

Slide 11

Example: Koch-curve Some properties: - Infinite length - Continuous everywhere - Differentiable nowhere - Fractal dimension D=log4/log3 1.26

Slide 12

Strange attractors are fractals Logistic map Hnon map 2,01

Slide 13

Self-similarity of the logistic attractor

Slide 14

Generalized dimensions

Slide 15

Generalized entropies

Slide 16

Lyapunov-exponent

Slide 17

Summary

Slide 18

Motivation Measures of synchronization for continuous data Linear measures: Cross correlation, coherence Mutual information Phase synchronization (Hilbert transform) Non-linear interdependences Measure comparison on model systems Measures of directionality Granger causality Transfer entropy Todays lecture

Slide 19

Motivation

Slide 20

Motivation: Bivariate time series analysis Three different scenarios: Repeated measurement from one system (different times) Stationarity, Reliability Simultaneous measurement from one system (same time) Coupling, Correlation, Synchronization, Directionality Simultaneous measurement from two systems (same time) Coupling, Correlation, Synchronization, Directionality

Slide 21

Synchronization [Huygens: Horologium Oscillatorium. 1673]

Slide 22

Synchronization [Pecora & Carroll. Synchronization in chaotic systems. Phys Rev Lett 1990]

Slide 23

Synchronization [Pikovsky & Rosenblum: Synchronization. Scholarpedia (2007)] In-phase synchronization

Slide 24

Synchronization [Pikovsky & Rosenblum: Synchronization. Scholarpedia (2007)] Anti-phase synchronization

Slide 25

Synchronization [Pikovsky & Rosenblum: Synchronization. Scholarpedia (2007)] Synchronization with phase shift

Slide 26

Synchronization [Pikovsky & Rosenblum: Synchronization. Scholarpedia (2007)] No synchronization

Slide 27

Synchronization [Pikovsky & Rosenblum: Synchronization. Scholarpedia (2007)] In-phase synchronizationAnti-phase synchronization No synchronizationSynchronization with phase shift

Slide 28

Measures of synchronization SynchronizationDirectionality Cross correlation / Coherence Mutual Information Index of phase synchronization - based on Hilbert transform - based on Wavelet transform Non-linear interdependence Event synchronization Delay asymmetry Transfer entropy Granger causality

Slide 29

Linear correlation

Slide 30

Static linear correlation: Pearsons r -1 - completely anti-correlated r = 0 - uncorrelated (linearly!) 1 - completely correlated Two sets of data points:

Slide 31

Examples: Pearsons r Undefined [An example of the correlation of x and y for various distributions of (x,y) pairs; Denis Boigelot 2011]

Slide 32

Cross correlation Maximum cross correlation:

Slide 33

Coherence Linear correlation in the frequency domain Cross spectrum: Coherence = Normalized power in the cross spectrum Welchs method: average over estimated periodograms of subintervals of equal length Complex number Phase

Slide 34

Mutual information

Slide 35

Shannon entropy ~ Uncertainty Binary probabilities:In general:

Slide 36

Mutual Information Marginal Shannon entropy: Joint Shannon entropy: Mutual Information: Estimation based on k-nearest neighbor distances: [Kraskov, Stgbauer, Grassberger: Estimating Mutual Information. Phys Rev E 2004] Kullback-Leibler entropy compares to probability distributions Mutual Information = KL-Entropy with respect to independence

Slide 37

Mutual Information Properties: Non-negativity: Symmetry: Minimum: Independent time series Maximum: for identical systems Venn diagram (Set theory)

Slide 38

Cross correlation & Mutual Information 1.0 0.5 0.0 C max I 1.0 0.5 0.0 C max I 1.0 0.5 0.0 C max I

Slide 39

Phase synchronization

Slide 40

Definition of a phase - Rice phase - Hilbert phase - Wavelet phase Index of phase synchronization - Index based on circular variance - [Index based on Shannon entropy] - [Index based on conditional entropy] Phase synchronization [Tass et al. PRL 1998]

Slide 41

Linear interpolation between marker events - threshold crossings (mostly zero, sometimes after demeaning) - discrete events (begin of a new cycle) Problem: Can be very sensitive to noise Rice phase

Slide 42

Hilbert phase [Rosenblum et al., Phys. Rev. Lett. 1996] Analytic signal: Artificial imaginary part: Instantaneous Hilbert phase: - Cauchy principal value

Slide 43

Wavelet phase Basis functions with finite support Example: complex Morlet wavelet Wavelet = Hilbert + filter [ Quian Quiroga, Kraskov, Kreuz, Grassberger. Phys. Rev. E 2002 ] Wavelet phase:

Slide 44

Index of phase synchronization: Circular variance (CV)

Slide 45

Non-linear interdependence

Slide 46

Takens embedding theorem [F. Takens. Detecting strange attractors in turbulence. Springer, Berlin, 1980]

Slide 47

Non-linear interdependences Nonlinear interdependence SNonlinear interdependence H Synchronization Directionality [Arnhold, Lehnertz, Grassberger, Elger. Physica D 1999]

Slide 48

Non-linear interdependence

Slide 49

Slide 50

Slide 51

Slide 52

Slide 53

Slide 54

Slide 55

Slide 56

Slide 57

Slide 58

Event synchronization

Slide 59

Event synchronization Event times: Synchronicity: Event synchronization:Delay asymmetry: [Quian Quiroga, Kreuz, Grassberger. Phys Rev E 2002] Window: with Avoids double-counting

Slide 60

Event synchronization [Quian Quiroga, Kreuz, Grassberger. Phys Rev E 2002] Q q

Slide 61

Measure comparison on model systems

Slide 62

Measure comparison on model systems [Kreuz, Mormann, Andrzejak, Kraskov, Lehnertz, Grassberger. Phys D 2007]

Slide 63

Model systems & Coupling schemes

Slide 64

Hnon map Introduced by Michel Hnon as a simplified model of the Poincar section of the Lorenz model One of the most studied examples of dynamical systems that exhibit chaotic behavior [M. Hnon. A two-dimensional mapping with a strange attractor. Commun. Math. Phys., 50:69, 1976]

Slide 65

Hnon map

Slide 66

Coupled Hnon maps Driver: Responder: Identical systems: Coupling strength:

Slide 67

Coupled Hnon maps

Slide 68

Coupled Hnon systems

Slide 69

Rssler system designed in 1976, for purely theoretical reasons later found to be useful in modeling equilibrium in chemical reactions [ O. E. Rssler. An equation for continuous chaos. Phys. Lett. A, 57:397, 1976 ]

Slide 70

Rssler system

Slide 71

Coupled Rssler systems Driver: Responder: Parameter mismatch: Coupling strength:

Slide 72

Coupled Rssler systems

Slide 73

Slide 74

Lorenz system Developed in 1963 as a simplified mathematical model for atmospheric convection Arise in simplified models for lasers, dynamos, electric circuits, and chemical reactions [ E. N. Lorenz. Deterministic non-periodic flow. J. Atmos. Sci., 20:130, 1963 ]

Slide 75

Lorenz system

Slide 76

Coupled Lorenz systems Driver: Responder: Small parameter mismatch in second component Coupling strength:

Slide 77

Coupled Lorenz systems

Slide 78

Slide 79

Noise-free case

Slide 80

Criterion I: Degree of monotonicity = 1 - strictly monotonic increase M(s) = 0 - flat line (or equal decrease and increase) = -1 - strictly monotonic decrease

Slide 81

Degree of monotonicity: Examples Sequences: 100 values 5050 pairs Left: Monotonicity Right: # positive # negative

Slide 82

Comparison: No Noise

Slide 83

Summary: No-Noise-Comparison Results for Rssler are more consistent than for the other systems Mutual Information slightly better than cross correlation (Non-linearity matters) Wavelet phase synchronization not appropiate for broadband systems (inherent filtering looses information)

Slide 84

Robustness against noise

Slide 85

Criterion II: Robustness against noise

Slide 86

Example: White noise

Slide 87

Hnon system: White noise

Slide 88

Rssler system : White noise

Slide 89

Lorenz system: White noise

Slide 90

Hnon system

Slide 91

Comparison: White noise

Slide 92

Summary: White noise For systems opposite order as in the noise-free case (Lorenz more robust then Hnon and than Rssler) the more monotonous a system has been without noise, the less noise is necessary to destroy this monotonicity Highest robustness is obtained for cross correlation followed by mutual information.

Slide 93

Iso-spectral noise: Example

Slide 94

Iso-spectral noise: Fourier spectrum complex Autocorrelation Fourier spectrum Time domain Frequency domain x (t) Amplitude Physical phenomenon Time series

Slide 95

Generation of iso-spectral noise Phase-randomized surrogates: Take Fourier transform of original signal Randomize phases Take inverse Fourier transform Iso-spectral surrogate (By construction identical Power spectrum, just different phases) Add to original signal with given NSR

Slide 96

Lorenz system: Iso-spectral noise

Slide 97

Comparison: Iso-spectral noise

Slide 98

Summary: Iso-spectral noise Again results for Rssler are more consistent than for the other systems Sometimes M never crosses critical threshold (monotonicity of the noise-free case is not destroyed by iso-spectral noise). Sometimes synchronization increases for more noise: (spurious) synchronization between contaminating noise-signals, only for narrow-band systems

Slide 99

Correlation among measures

Slide 100

Correlation among measures

Slide 101

Slide 102

Slide 103

Slide 104

Summary: Correlation All correlation values rather high (Minimum: ~0.65) Highest correlations for cross correlation and Hilbert phase synchronization Event synchronization and Hilbert phase synchronization appear least correlated Overall correlation between two phase synchronization methods low (but only due to different frequency sensitivity in the Hnon system)

Slide 105

Overall summary: Comparison of measures Capability to distinguish different coupling strengths Obvious and objective criterion exists only in some special cases (e.g., wavelet phase is not very suitable for a system with a broadband spectrum). Robustness against noise varies (Important criterion for noisy data) Pragmatic solution: Choose measure which most reliably yields valuable information (e.g., information useful for diagnostic purposes) in test applications

Slide 106

Measures of directionality

Slide 107

Measures of directionality

Slide 108

Granger causality [Granger: Investigating causal relations by econometric models and cross-spectral methods. Econometrica 37, 424-438 (1969)]

Slide 109

Granger causality Univariate model: Bivariate model: Model parameters; Prediction errors; [Granger: Investigating causal relations by econometric models and cross-spectral methods. Econometrica 37, 424-438 (1969)] Fit via linear regression

Slide 110

Transfer Entropy: Conditional entropy Venn diagram (Set theory) Conditional entropy: Mutual Information:

Slide 111

Transfer entropy : Conditional entropy [T. Schreiber. Measuring information transfer. Phys. Rev. Lett., 85:461, 2000]

Slide 112

Transfer entropy [T. Schreiber. Measuring information transfer. Phys. Rev. Lett., 85:461, 2000]

Slide 113

Transfer entropy [T. Schreiber. Measuring information transfer. Phys. Rev. Lett., 85:461, 2000]

Slide 114

Motivation Measures of synchronization for continuous data Linear measures: Cross correlation, coherence Mutual information Phase synchronization (Hilbert transform) Non-linear interdependences Measure comparison on model systems Measures of directionality Granger causality Transfer entropy Todays lecture

Slide 115

Measures of synchronization for discrete data (e.g. spike trains) Victor-Purpura distance Van Rossum distance Schreiber correlation measure ISI-distance SPIKE-distance Measure comparison Next lecture