nonlinear time series analysis with the tisean package · 2008-05-26 · nonlinear time series...
TRANSCRIPT
Nonlinear time series analysis with the TISEAN package
Eckehard Olbrich
MPI MIS Leipzig
Potsdam SS 2008
Olbrich (Leipzig) 23.05.2008 1 / 16
Deterministic chaos
Deterministic chaos: Aperiodic irregular temporal behaviour inlow-dimensional deterministic systems
Nonlinear time series analysis: A battery of methods to characterize,predict and model time series as nonlinear deterministicsystems instead as linear stochastic systems as in the“classical” time series analysis
Successful applications: Electric circuits, Laser, Hydrodynamicexperiments (Taylor–Couette flow), ...
Unsuccessful attempts: Climate data, EEG data, finance data, ...
Olbrich (Leipzig) 23.05.2008 2 / 16
The henon map
The logistic map:xn+1 = 1 − ax2
n
Add a second degree of freedom in order to make it invertible:
xn+1 = 1 − ax2n + byn
yn+1 = xn
Example: 10 000 data points with henon - delay plot with delay
-1.5
-1
-0.5
0
0.5
1
1.5
0 20 40 60 80 100
x n
n
-1.5
-1
-0.5
0
0.5
1
1.5
-1.5 -1 -0.5 0 0.5 1 1.5
x n+
1
xnOlbrich (Leipzig) 23.05.2008 3 / 16
Embedding - False nearest neighbours with false nearest
Delay vector: xxxn = (xn, xn−1, xn−2, . . . , xn−m+1)
False nearest neighbour: next neighbour in m dimensions, distanceincreases by more than the factor f in m + 1 dimensions
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
1 2 3 4 5 6 7 8 9 10
frac
tion
of fa
lse
near
est n
eigh
bour
s
embedding dimension m
f=2f=3
Olbrich (Leipzig) 23.05.2008 4 / 16
Estimating entropies with boxcount
Block entropies Hm(ǫ)
0
2
4
6
8
10
12
0.001 0.01 0.1 1
H(m
,ε)
ε
log(N)1.6-1.25*log(ε)
Olbrich (Leipzig) 23.05.2008 5 / 16
Estimating entropies with boxcount
Conditional entropies hm(ǫ) = Hm(ǫ) − Hm+1(ǫ)
0
0.5
1
1.5
2
2.5
3
3.5
4
0.001 0.01 0.1 1
h(m
,ε)
ε
λ=0.4190.82-log(ε)
Olbrich (Leipzig) 23.05.2008 5 / 16
Correlation sum and dimension - d2
Time series {x(t0), x(t0 + ∆), . . . , x(t0 + N∆)}Delay embedding xxx = (xt , xt−d∆, . . . , xt−(m−1)d∆)
Correlation sum
C (m, ǫ) =2
(N − m) · (N − m − 1)
N−m∑
i=1
N∑
j=i+1
Θ(ǫ − ||~Xi − ~Xj ||)
∝ ǫD2
Correlation dimension D2
D2 = limǫ→0
D2(m, ǫ) = limǫ→0
d log C (m, ǫ)
d log ǫ
Olbrich (Leipzig) 23.05.2008 6 / 16
Correlation sum and dimension - d2
1e-05
1e-04
0.001
0.01
0.1
1
0.001 0.01 0.1 1
C(m
,ε)
ε
0
0.5
1
1.5
2
2.5
3
3.5
4
0.001 0.01 0.1 1
D(m
,ε)
ε
DKS=1.25
correlation sum correlation dimension
Olbrich (Leipzig) 23.05.2008 6 / 16
Lyapunov exponents
Estimating the largest Lyapunovexponent using lyap k
-8
-7
-6
-5
-4
-3
-2
-1
0
0 5 10 15 20
S(n
)
n
0.425*x-6.95
Stretching factor vs. time steps fordifferent neighborhood sizesλmax = 0.425
Lyapunov exponents from thesystem equations:
λ1 = 0.419, λ2 = −1.62,DKY = 1.26
Lyapunov exponents from fittingthe Jacobian with lyap spec:
λ1 = 0.41, λ2 = −1.55,DKY = 1.27
Olbrich (Leipzig) 23.05.2008 7 / 16
Flow data — the Lorenz system
Lorenz equations:
x = s(−x + y)
y = −xz + rx − y
z = xy − bz
Integrated with lorenz at stan-dard parameters
s = 10
r = 28
b = 8/3
-20
-15
-10
-5
0
5
10
15
20
0 1000 2000 3000 4000 5000
t
-20 -15 -10 -5 0 5 10 15 20-25-20
-15-10
-5 0
5 10
15 20
25 5
10 15 20 25 30 35 40 45
z(t)
x(t)
y(t)
z(t)
Olbrich (Leipzig) 23.05.2008 8 / 16
Embedding — finding the optimal delay time
Delay embedding of the x-component
-20-15
-10-5
0 5
10 15
20 -20-15
-10-5
0 5
10 15
20
-20-15-10-5 0 5
10 15 20
x(t+2∆)
x(t) x(t+∆)
x(t+2∆)
-20-15
-10-5
0 5
10 15
20 -20-15
-10-5
0 5
10 15
20
-20-15-10-5 0 5
10 15 20
x(t+20∆)
x(t) x(t+10∆)
x(t+20∆)
-20-15
-10-5
0 5
10 15
20 -20-15
-10-5
0 5
10 15
20
-20-15-10-5 0 5
10 15 20
x(t+36∆)
x(t) x(t+18∆)
x(t+36∆)
-20-15
-10-5
0 5
10 15
20 -20-15
-10-5
0 5
10 15
20
-20-15-10-5 0 5
10 15 20
x(t+100∆)
x(t)
x(t+50∆)
x(t+100∆)
Olbrich (Leipzig) 23.05.2008 9 / 16
Embedding — finding the optimal delay time
Rule of thumb: First zero of the autocorrelation function (corr) or firstminimum of the mutual information function (mutual)
0
0.5
1
1.5
2
0 100 200 300 400 500
τ
Correlation functionMutual information
0
0.5
1
1.5
2
0 10 20 30 40 50
τ
Correlation functionMutual information
First zero of the autocorrelation function at 314 time steps. First minimumof the mutual information function at 18 time steps.
Olbrich (Leipzig) 23.05.2008 9 / 16
Embedding — finding the optimal delay time
-20
-15
-10
-5
0
5
10
15
20
-20 -15 -10 -5 0 5 10 15 20
x(t+
18∆)
x(t)
-20
-15
-10
-5
0
5
10
15
20
-20 -15 -10 -5 0 5 10 15 20
x(t+
314∆
)x(t)
Mutual information Correlation functiond = 18 d = 314
Olbrich (Leipzig) 23.05.2008 9 / 16
What is an optimal state space?
One possibility - maximize the scaling range in dimension estimates.Let us consider the correlation dimension:
C (ǫ) =2
N(N − 1)
N∑
i=1
N∑
j=i+1
Θ(ǫ − ||xxx i − xxx j ||)
Scaling range ǫl ≤ ǫ ≤ ǫuFor m > D0
C (ǫ) ∝ ǫD2
Lower end of the scaling range in the noise free case:
C (ǫl) =2
N(N − 1)
Olbrich (Leipzig) 23.05.2008 10 / 16
Estimating the scaling range
What about the upper bound?Correlation sum approximates Renyi entropies of second order
− log Cm(ǫ) ≈ H(q=2)m (ǫ)
For q = 1 we know Hm(ǫ) ≥ mhKS . Assuming the same for q = 2 andcalling the corresponding entropy hC one gets
−logCm(ǫu) ≥ mhC
−logCm(ǫl) ≥ mhC − D2 logǫlǫu
Olbrich (Leipzig) 23.05.2008 11 / 16
Estimating the scaling range
What about the upper bound?
−logCm(ǫl) ≥ mhC − D2 logǫlǫu
and therefore
ǫuǫl
≤(
e−mhCN2
2
)1/D2
.
In order to get a scaling in one decade of ǫ:
N ≥√
2 · 10D2/2 · emhC /2
For D2 = 4, m = 5 and hC = ln 2 we would get N > 4525.
Olbrich (Leipzig) 23.05.2008 11 / 16
Maximizing the scaling range
Maximizing the scaling range ⇒ Minimizing H(ǫu).
Hm(ǫu) = mhm−1(ǫ) +m−1∑
n=1
nδhn(ǫ)
with δhn(ǫ) = hn−1(ǫ) − hn(ǫ) and h0(ǫ) = H1(ǫ). In a first approximation
Hm(ǫu) = mhKS + MI MI = limǫ→0
MI (ǫ)
with the mutual information MI = δh1. For different delays τ we have
hKS = τ hKS
thus minimizing Hm(ǫ) leads to
dMI (τ)
dτ
∣
∣
∣
∣
τ=τopt
= −mhKSd2MI (τ)
dτ2
∣
∣
∣
∣
τ=τopt
> 0
For hKS = 0 this gives the first minimum of the mutual information.Olbrich (Leipzig) 23.05.2008 12 / 16
Correlation dimension for different delays
0
0.5
1
1.5
2
2.5
0.1 1 10
D2(
ε)
ε
delay d=1
0
0.5
1
1.5
2
2.5
0.1 1 10
D2(
ε)
ε
Delay d=10
0
0.5
1
1.5
2
2.5
3
3.5
0.1 1 10
D2(
ε)
ε
Delay d=18
0
2
4
6
8
10
12
0.1 1 10
D2(
ε)
ε
Delay d=50
Olbrich (Leipzig) 23.05.2008 13 / 16
Temporal correlations — the Theiler correction
For finite data temporal correlations might lead to a based estimate of thecorrelation sum. Theiler correction: remove W temporal neighbours fromthe correlation sum
C (ǫ) =2
N − W (N − W − 1)
N∑
i=1
N∑
j=i+W+1
Θ(ǫ − ||xxx i − xxx j ||)
Space-time separation plot: plots cumulative distribution for distances ǫconditioned on the temporal differences.Space-time separation plot (stp) for the Lorenz system (m=3,d=18)
0
5
10
15
20
25
30
35
40
0 50 100 150 200 250 300 350 400 450 500
ε
∆t
Olbrich (Leipzig) 23.05.2008 14 / 16
EEG — spurious dimension due to temporal correlations
1
10
100
1000
0 10 20 30 40 50 60 70 80 90 100
ε
t [∆t]
0
5
10
15
20
10 100
D2(
m,ε
)
ε
tW=0tW=3
tW=601.64
Space-time separation plot Correlation dimension
Olbrich (Leipzig) 23.05.2008 15 / 16
Prediction and Modeling
xxxn+1 = FFF (xxxn+1)
which reduces toxn+1 = F (xn, . . . , xn−m+1)
in the case of delay embedding.
Local methods: Basic idea: Looking for similar events in the past andusing their future for prediction.Local constant (lzo-run): Using the average as prediction.Local linear (lfo-run): Fitting a linear model for similarevents, i.e. neighbors in phase space.
Global models: Parameterizing the function FFF and fitting the parameters.Polynomials (polynom)Radial basis functions (rbf) — related to neural networks(not included in TISEAN)
Olbrich (Leipzig) 23.05.2008 16 / 16
Prediction and Modeling
-30
-25
-20
-15
-10
-5
0
5
10
15
20
0 100 200 300 400 500 600 700 800 900 1000
x(t)
t
originallocal constant prediction
local linear predictionglobal model rbf
global model polynom
Relative forecast errors:
lfo 1.12e-04poly 8.28e-04lzo 1.25e-02rbf 4.03e-02
Polynomial fit used 4th order polynomials.
Olbrich (Leipzig) 23.05.2008 16 / 16