Lecture 6Discrete Time Series

Colin Rundel02/06/2017

1

Discrete Time Series

2

Stationary Processes

A stocastic process (i.e. a time series) is considered to be strictly stationaryif the properties of the process are not changed by a shift in origin.

In the time series context this means that the joint distribution of{yt1 , . . . , ytn} must be identical to the distribution of {yt1+k, . . . , ytn+k} forany value of n and k.

3

Stationary Processes

A stocastic process (i.e. a time series) is considered to be strictly stationaryif the properties of the process are not changed by a shift in origin.

In the time series context this means that the joint distribution of{yt1 , . . . , ytn} must be identical to the distribution of {yt1+k, . . . , ytn+k} forany value of n and k.

3

Stationary Processes

A stocastic process (i.e. a time series) is considered to be strictly stationaryif the properties of the process are not changed by a shift in origin.

In the time series context this means that the joint distribution of{yt1 , . . . , ytn} must be identical to the distribution of {yt1+k, . . . , ytn+k} forany value of n and k.

3

Weak Stationary

Strict stationary is too strong for most applications, so instead we often optfor weak stationary which requires the following,

1. The process has finite variance

E(y2t ) < ∞ for all t

2. The mean of the process in constant

E(yt) = µ for all t

3. The second moment only depends on the lag

Cov(yt, ys) = Cov(yt+k, ys+k) for all t, s, k

When we say stationary in class we almost always mean this version ofweakly stationary.

4

Weak Stationary

Strict stationary is too strong for most applications, so instead we often optfor weak stationary which requires the following,

1. The process has finite variance

E(y2t ) < ∞ for all t

2. The mean of the process in constant

E(yt) = µ for all t

3. The second moment only depends on the lag

Cov(yt, ys) = Cov(yt+k, ys+k) for all t, s, k

When we say stationary in class we almost always mean this version ofweakly stationary.

4

Autocorrelation

For a stationary time series, where E(yt) = µ and Var(yt) = σ2) for all t, wedefine the autocorrelation at lag k as

ρk = Cor(yt, yt+k)

=Cov(yt, yt+k)√Var(yt)Var(yt+k)

=E ((yt − µ)(yt+k − µ))

σ2

this is also sometimes written in terms of the autocovariance function (γk)as

γk = γ(t, t+ k) = Cov(yt, yt+k)

ρk =γ(t, t+ k)√

γ(t, t)γ(t+ k, t+ k)=

γ(k)γ(0)

5

Autocorrelation

For a stationary time series, where E(yt) = µ and Var(yt) = σ2) for all t, wedefine the autocorrelation at lag k as

ρk = Cor(yt, yt+k)

=Cov(yt, yt+k)√Var(yt)Var(yt+k)

=E ((yt − µ)(yt+k − µ))

σ2

this is also sometimes written in terms of the autocovariance function (γk)as

γk = γ(t, t+ k) = Cov(yt, yt+k)

ρk =γ(t, t+ k)√

γ(t, t)γ(t+ k, t+ k)=

γ(k)γ(0)

5

Covariance Structure

Based on our definition of a (weakly) stationary process, it implies acovariance of the following structure,

γ(0) γ(1) γ(2) γ(3) · · · γ(n)γ(1) γ(0) γ(1) γ(2) · · · γ(n − 1)γ(2) γ(1) γ(0) γ(1) · · · γ(n − 2)γ(3) γ(2) γ(1) γ(0) · · · γ(n − 3)...

......

.... . .

...γ(n) γ(n − 1) γ(n − 2) γ(n − 3) · · · γ(0)

6

Example - Random walk

Let yt = yt−1 + wt with y0 = 0 and wt ∼ N (0, 1). Is yt stationary?

−20

−10

0

10

0 250 500 750 1000

t

y

Random walk

7

ACF + PACF

0 10 20 30 40 50

0.0

0.2

0.4

0.6

0.8

1.0

Lag

AC

F

Series rw$y

0 10 20 30 40 50

0.0

0.4

0.8

Lag

Par

tial A

CF

Series rw$y

8

Example - Random walk with drift

Let yt = δ + yt−1 + wt with y0 = 0 and wt ∼ N (0, 1). Is yt stationary?

0

25

50

75

100

0 250 500 750 1000

t

y

Random walk with trend

9

ACF + PACF

0 10 20 30 40 50

0.0

0.2

0.4

0.6

0.8

1.0

Lag

AC

F

Series rwt$y

0 10 20 30 40 50

0.0

0.2

0.4

0.6

0.8

1.0

Lag

Par

tial A

CF

Series rwt$y

10

Example - Moving Average

Let wt ∼ N (0, 1) and yt = 13 (wt−1 + wt + wt+1), is yt stationary?

−1.0

−0.5

0.0

0.5

1.0

1.5

0 25 50 75 100

t

y

Moving Average

11

ACF + PACF

0 10 20 30 40 50

−0.

20.

20.

61.

0

Lag

AC

F

Series ma$y

0 10 20 30 40 50

−0.

20.

20.

40.

6Lag

Par

tial A

CF

Series ma$y

12

Autoregression

Let wt ∼ N (0, 1) and yt = yt−1 − 0.9yt−2 + wt with yt = 0 for t < 1, is ytstationary?

−5

0

5

0 100 200 300 400 500

t

y

Autoregressive

13

ACF + PACF

0 10 20 30 40 50

−0.

50.

00.

51.

0

Lag

AC

F

Series ar$y

0 10 20 30 40 50

−0.

8−

0.4

0.0

0.4

Lag

Par

tial A

CF

Series ar$y

14

Example - Australian Wine Sales

Australian total wine sales by wine makers in bottles

Time series

20000

30000

40000

1980 1985 1990 1995

date

sale

s

16

Basic Model Fit

20000

30000

40000

1980 1985 1990 1995

date

sale

s

17

Residuals

resid_q

resid_l

1980 1985 1990 1995

−10000

−5000

0

5000

10000

15000

−10000

−5000

0

5000

10000

15000

date

resi

dual type

resid_l

resid_q

18

Autocorrelation Plot

0 5 10 15 20 25 30 35

−0.

4−

0.2

0.0

0.2

0.4

0.6

0.8

1.0

Lag

AC

F

Series d$resid_q

19

Partial Autocorrelation Plot

0 5 10 15 20 25 30 35

−0.

20.

00.

20.

40.

6

Lag

Par

tial A

CF

Series d$resid_q

20

lag_09 lag_10 lag_11 lag_12

lag_05 lag_06 lag_07 lag_08

lag_01 lag_02 lag_03 lag_04

−10000−5000 0 500010000 −10000−5000 0 500010000 −10000−5000 0 500010000 −10000−5000 0 500010000

−10000

−5000

0

5000

10000

−10000

−5000

0

5000

10000

−10000

−5000

0

5000

10000

lag_value

resi

d_q

21

Auto regressive errors

#### Call:## lm(formula = resid_q ~ lag_12, data = d_ar)#### Residuals:## Min 1Q Median 3Q Max## -12286.5 -1380.5 73.4 1505.2 7188.1#### Coefficients:## Estimate Std. Error t value Pr(>|t|)## (Intercept) 83.65080 201.58416 0.415 0.679## lag_12 0.89024 0.04045 22.006

Residual residuals

−10000

−5000

0

5000

1980 1985 1990 1995

date

resi

d

23

Residual residuals - acf

0 5 10 15 20 25 30 35

−0.

20.

00.

20.

40.

60.

81.

0

Lag

AC

F

Series l_ar$residuals

24

lag_09 lag_10 lag_11 lag_12

lag_05 lag_06 lag_07 lag_08

lag_01 lag_02 lag_03 lag_04

−10000 −5000 0 5000 −10000 −5000 0 5000 −10000 −5000 0 5000 −10000 −5000 0 5000

−10000

−5000

0

5000

−10000

−5000

0

5000

−10000

−5000

0

5000

lag_value

resi

d

25

Writing down the model?

So, is our EDA suggesting that we then fit the following model?

sales(t) = β0 + β1 t+ β2 t2 + β3 sales(t − 12) + ϵt. . .

the implied model is,

sales(t) = β0 + β1 t+ β2 t2 + wt

where

wt = δ wt−12 + ϵt

26

Discrete Time Series