state space models - rob j hyndman · 1the state space perspective 2simple exponential smoothing...

Rob J Hyndman

State space models

1: Exponential smoothing

Outline

1 The state space perspective

2 Simple exponential smoothing

3 Trend methods

4 Seasonal methods

5 Taxonomy of exponential smoothingmethods

6 Innovations state space models

7 ETS in R

State space models 1: Exponential smoothing 2

Outline

3 Trend methods

4 Seasonal methods

7 ETS in R

State space perspective

Observed data: y1, . . . , yT.

Unobserved state: x1, . . . ,xT.

Forecast yT+h|T = E(yT+h|xT).The “forecast variance” is Var(yT+h|xT).A prediction interval or “interval

forecast” is a range of values of yT+hwith high probability.

Outline

3 Trend methods

4 Seasonal methods

7 ETS in R

Simple Exponential Smoothing

Component form

Forecast equation yt+h|t = `t

Smoothing equation `t = αyt + (1− α)`t−1

`1 = αy1 + (1− α)`0

Component form

`1 = αy1 + (1− α)`0

Component form

`1 = αy1 + (1− α)`0

`2 = αy2 + (1− α)`1 = αy2 + α(1− α)y1 + (1− α)2`0

Component form

`1 = αy1 + (1− α)`0

`2 = αy2 + (1− α)`1 = αy2 + α(1− α)y1 + (1− α)2`0

`3 = αy3 + (1− α)`2 =2∑j=0

α(1− α)jy3−j + (1− α)3`0

Component form

`1 = αy1 + (1− α)`0

`2 = αy2 + (1− α)`1 = αy2 + α(1− α)y1 + (1− α)2`0

`3 = αy3 + (1− α)`2 =2∑j=0

α(1− α)jy3−j + (1− α)3`0

`t =t−1∑j=0

α(1− α)jyt−j + (1− α)t`0

Forecast equation

yt+h|t =t∑

α(1− α)t−jyj + (1− α)t`0, (0 ≤ α ≤ 1)

Weights assigned to observations for:Observation α = 0.2 α = 0.4 α = 0.6 α = 0.8

yt 0.2 0.4 0.6 0.8yt−1 0.16 0.24 0.24 0.16yt−2 0.128 0.144 0.096 0.032yt−3 0.1024 0.0864 0.0384 0.0064yt−4 (0.2)(0.8)4 (0.4)(0.6)4 (0.6)(0.4)4 (0.8)(0.2)4

yt−5 (0.2)(0.8)5 (0.4)(0.6)5 (0.6)(0.4)5 (0.8)(0.2)5

Limiting cases: α→ 1, α→ 0.

Forecast equation

yt+h|t =t∑

α(1− α)t−jyj + (1− α)t`0, (0 ≤ α ≤ 1)

yt 0.2 0.4 0.6 0.8yt−1 0.16 0.24 0.24 0.16yt−2 0.128 0.144 0.096 0.032yt−3 0.1024 0.0864 0.0384 0.0064yt−4 (0.2)(0.8)4 (0.4)(0.6)4 (0.6)(0.4)4 (0.8)(0.2)4

yt−5 (0.2)(0.8)5 (0.4)(0.6)5 (0.6)(0.4)5 (0.8)(0.2)5

Forecast equation

yt+h|t =t∑

α(1− α)t−jyj + (1− α)t`0, (0 ≤ α ≤ 1)

yt 0.2 0.4 0.6 0.8yt−1 0.16 0.24 0.24 0.16yt−2 0.128 0.144 0.096 0.032yt−3 0.1024 0.0864 0.0384 0.0064yt−4 (0.2)(0.8)4 (0.4)(0.6)4 (0.6)(0.4)4 (0.8)(0.2)4

yt−5 (0.2)(0.8)5 (0.4)(0.6)5 (0.6)(0.4)5 (0.8)(0.2)5

Component form

State space form

Observation equation yt = `t−1 + etState equation `t = `t−1 + αet

et = yt − `t−1 = yt − yt|t−1 for t = 1, . . . ,T, the one-stepwithin-sample forecast error at time t.

`t is an unobserved “state”.

Need to estimate α and `0.

Component form

State space form

Component form

State space form

Component form

State space form

SES in R

library(fpp)

fit <- ses(oil, h=3)

plot(fit)

summary(fit)

Outline

3 Trend methods

4 Seasonal methods

7 ETS in R

Holt’s linear trend

Component form

Forecast yt+h|t = `t + hbtLevel `t = αyt + (1− α)(`t−1 + bt−1)

Trend bt = β∗(`t − `t−1) + (1− β∗)bt−1,

Two smoothing parameters α and β∗

(0 ≤ α, β∗ ≤ 1).

`t level: weighted average between yt one-stepahead forecast for time t, (`t−1 + bt−1 = yt|t−1)

bt trend (slope): weighted average of (`t − `t−1)and bt−1, current and previous estimate of thetrend.

Component form

Trend bt = β∗(`t − `t−1) + (1− β∗)bt−1,

(0 ≤ α, β∗ ≤ 1).

Component form

Trend bt = β∗(`t − `t−1) + (1− β∗)bt−1,

(0 ≤ α, β∗ ≤ 1).

Component form

Trend bt = β∗(`t − `t−1) + (1− β∗)bt−1,

(0 ≤ α, β∗ ≤ 1).

Holt’s linear trendComponent form

Forecast yt+h|t = `t + hbt

Level `t = αyt + (1− α)(`t−1 + bt−1)

Trend bt = β∗(`t − `t−1) + (1− β∗)bt−1,

State space form

Observation equation yt = `t−1 + bt−1 + etState equations `t = `t−1 + bt−1 + αet

bt = bt−1 + βet

β = αβ∗

et = yt − (`t−1 + bt−1) = yt − yt|t−1

Need to estimate α, β, `0,b0.State space models 1: Exponential smoothing 12

Level `t = αyt + (1− α)(`t−1 + bt−1)

Trend bt = β∗(`t − `t−1) + (1− β∗)bt−1,

State space form

bt = bt−1 + βet

β = αβ∗

et = yt − (`t−1 + bt−1) = yt − yt|t−1

Level `t = αyt + (1− α)(`t−1 + bt−1)

Trend bt = β∗(`t − `t−1) + (1− β∗)bt−1,

State space form

bt = bt−1 + βet

β = αβ∗

et = yt − (`t−1 + bt−1) = yt − yt|t−1

Level `t = αyt + (1− α)(`t−1 + bt−1)

Trend bt = β∗(`t − `t−1) + (1− β∗)bt−1,

State space form

bt = bt−1 + βet

β = αβ∗

et = yt − (`t−1 + bt−1) = yt − yt|t−1

Holt’s method in R

fit2 <- holt(ausair, h=5)

plot(fit2)

summary(fit2)

Exponential trend

Level and trend are multiplied rather than added:

Component form

yt+h|t = `tbht

`t = αyt + (1− α)(`t−1bt−1)

bt = β∗`t`t−1

+ (1− β∗)bt−1

State space form

Observation equation yt = (`t−1bt−1) + etState equations `t = `t−1bt−1 + αet

bt = bt−1 + βet/`t−1

Trend methods in R

fit3 <- holt(air, h=5, exponential=TRUE)

plot(fit3)

summary(fit3)

Additive damped trend

Component form

yt+h|t = `t + (φ+ φ2 + · · ·+ φh)bt

`t = αyt + (1− α)(`t−1 + φbt−1)

bt = β∗(`t − `t−1) + (1− β∗)φbt−1.

State space form

Observation equation yt = `t−1 + φbt−1 + etState equations `t = `t−1 + φbt−1 + αet

bt = φbt−1 + βet

Damping parameter 0 < φ < 1.If φ = 1, identical to Holt’s linear trend.As h→∞, yT+h|T → `T + φbT/(1− φ).Short-run forecasts trended, long-run forecasts constant.

Component form

yt+h|t = `t + (φ+ φ2 + · · ·+ φh)bt

`t = αyt + (1− α)(`t−1 + φbt−1)

bt = β∗(`t − `t−1) + (1− β∗)φbt−1.

State space form

Component form

yt+h|t = `t + (φ+ φ2 + · · ·+ φh)bt

`t = αyt + (1− α)(`t−1 + φbt−1)

bt = β∗(`t − `t−1) + (1− β∗)φbt−1.

State space form

Component form

yt+h|t = `t + (φ+ φ2 + · · ·+ φh)bt

`t = αyt + (1− α)(`t−1 + φbt−1)

bt = β∗(`t − `t−1) + (1− β∗)φbt−1.

State space form

Component form

yt+h|t = `t + (φ+ φ2 + · · ·+ φh)bt

`t = αyt + (1− α)(`t−1 + φbt−1)

bt = β∗(`t − `t−1) + (1− β∗)φbt−1.

State space form

Trend methods in R

fit4 <- holt(air, h=5, damped=TRUE)

plot(fit4)

summary(fit4)

Example: Sheep in Asia

●●

● ●

● ●●

●●

● ● ●● ●

●●

Forecasts from Holt's method with exponential trend

1970 1980 1990 2000 2010

●●

● ●

DataSESHolt'sExponentialAdditive DampedMultiplicative Damped

Outline

3 Trend methods

4 Seasonal methods

7 ETS in R

Holt-Winters additive methodHolt and Winters extended Holt’s method to captureseasonality.

Component form

yt+h|t = `t + hbt + st−m+h+m

`t = α(yt − st−m) + (1− α)(`t−1 + bt−1)

bt = β∗(`t − `t−1) + (1− β∗)bt−1

st = γ(yt − `t−1 − bt−1) + (1− γ)st−m,

h+m = b(h− 1) mod mc+ 1 - the largest integer not

greater than (h− 1) mod m. Ensures estimatesfrom the final year are used for forecasting.Parameters: 0 ≤ α ≤ 1, 0 ≤ β∗ ≤ 1, 0 ≤ γ ≤ 1− αand m = period of seasonality (e.g. m=4 forquarterly data).

Component form

`t = α(yt − st−m) + (1− α)(`t−1 + bt−1)

bt = β∗(`t − `t−1) + (1− β∗)bt−1

st = γ(yt − `t−1 − bt−1) + (1− γ)st−m,

Component form

`t = α(yt − st−m) + (1− α)(`t−1 + bt−1)

bt = β∗(`t − `t−1) + (1− β∗)bt−1

st = γ(yt − `t−1 − bt−1) + (1− γ)st−m,

Holt-Winters additive method

Component form

`t = α(yt − st−m) + (1− α)(`t−1 + bt−1)

bt = β∗(`t − `t−1) + (1− β∗)bt−1

st = γ(yt − `t−1 − bt−1) + (1− γ)st−m,

State space form

yt = `t−1 + bt−1 + st−m + et`t = `t−1 + bt−1 + αetbt = bt−1 + βetst = st−m + γet.

Holt-Winters multiplicative

Component form

yt+h|t = (`t + hbt)st−m+h+m.

`t = αyt

st−m+ (1− α)(`t−1 + bt−1)

bt = β∗(`t − `t−1) + (1− β∗)bt−1

st = γyt

(`t−1 + bt−1)+ (1− γ)st−m

State space form

yt = (`t−1 + bt−1)st−m + et`t = `t−1 + bt−1 + αet/st−mbt = bt−1 + βet/st−mst = st−m + γet/(`t−1 + bt−1).

Seasonal methods in R

aus1 <- hw(austourists)aus2 <- hw(austourists, seasonal="mult")

plot(aus1)plot(aus2)

summary(aus1)summary(aus2)

Holt-Winters damped method

Often the single most accurate forecasting method forseasonal data:

yt+h|t = [`t + (φ+ φ2 + · · ·+ φh)bt]st−m+h+m

`t = α(yt/st−m) + (1− α)(`t−1 + φbt−1)

bt = β∗(`t − `t−1) + (1− β∗)φbt−1

st = γyt

(`t−1 + φbt−1)+ (1− γ)st−m

State space form

yt = (`t−1 + φbt−1)st−m + et`t = `t−1 + φbt−1 + αet/st−mbt = φbt−1 + βet/st−mst = st−m + γet/(`t−1 + φbt−1).

Seasonal methods in R

aus3 <- hw(austourists, seasonal="mult",damped=TRUE)

summary(aus3)

plot(aus3)

Outline

3 Trend methods

4 Seasonal methods

7 ETS in R

Exponential smoothing methods

Seasonal ComponentTrend N A M

Component (None) (Additive) (Multiplicative)

N (None) N,N N,A N,M

A (Additive) A,N A,A A,M

Ad (Additive damped) Ad,N Ad,A Ad,M

M (Multiplicative) M,N M,A M,M

Md (Multiplicative damped) Md,N Md,A Md,M

N,N: Simple exponential smoothing

N,N: Simple exponential smoothingA,N: Holt’s linear method

N,N: Simple exponential smoothingA,N: Holt’s linear methodAd,N: Additive damped trend method

N,N: Simple exponential smoothingA,N: Holt’s linear methodAd,N: Additive damped trend methodM,N: Exponential trend method

N,N: Simple exponential smoothingA,N: Holt’s linear methodAd,N: Additive damped trend methodM,N: Exponential trend methodMd,N: Multiplicative damped trend method

N,N: Simple exponential smoothingA,N: Holt’s linear methodAd,N: Additive damped trend methodM,N: Exponential trend methodMd,N: Multiplicative damped trend methodA,A: Additive Holt-Winters’ method

N,N: Simple exponential smoothingA,N: Holt’s linear methodAd,N: Additive damped trend methodM,N: Exponential trend methodMd,N: Multiplicative damped trend methodA,A: Additive Holt-Winters’ methodA,M: Multiplicative Holt-Winters’ method

There are 15 separate exponential smoothing methods.

Component form 7/ exponential smoothing 145

Trend SeasonalN A M

yt+h|t = `t yt+h|t = `t + st−m+h+m

yt+h|t = `tst−m+h+m

N `t = αyt + (1−α)`t−1 `t = α(yt − st−m) + (1−α)`t−1 `t = α(yt/st−m) + (1−α)`t−1st = γ(yt − `t−1) + (1−γ)st−m st = γ(yt/`t−1) + (1−γ)st−m

yt+h|t = `t + hbt yt+h|t = `t + hbt + st−m+h+m

yt+h|t = (`t + hbt)st−m+h+m

A `t = αyt + (1−α)(`t−1 + bt−1) `t = α(yt − st−m) + (1−α)(`t−1 + bt−1) `t = α(yt/st−m) + (1−α)(`t−1 + bt−1)bt = β∗(`t − `t−1) + (1− β∗)bt−1 bt = β∗(`t − `t−1) + (1− β∗)bt−1 bt = β∗(`t − `t−1) + (1− β∗)bt−1

st = γ(yt − `t−1 − bt−1) + (1−γ)st−m st = γ(yt/(`t−1 − bt−1)) + (1−γ)st−myt+h|t = `t +φhbt yt+h|t = `t +φhbt + st−m+h+

myt+h|t = (`t +φhbt)st−m+h+

Ad `t = αyt + (1−α)(`t−1 +φbt−1) `t = α(yt − st−m) + (1−α)(`t−1 +φbt−1) `t = α(yt/st−m) + (1−α)(`t−1 +φbt−1)bt = β∗(`t − `t−1) + (1− β∗)φbt−1 bt = β∗(`t − `t−1) + (1− β∗)φbt−1 bt = β∗(`t − `t−1) + (1− β∗)φbt−1

st = γ(yt − `t−1 −φbt−1) + (1−γ)st−m st = γ(yt/(`t−1 −φbt−1)) + (1−γ)st−myt+h|t = `tb

ht yt+h|t = `tb

ht + st−m+h+

myt+h|t = `tb

ht st−m+h+

M `t = αyt + (1−α)`t−1bt−1 `t = α(yt − st−m) + (1−α)`t−1bt−1 `t = α(yt/st−m) + (1−α)`t−1bt−1bt = β∗(`t/`t−1) + (1− β∗)bt−1 bt = β∗(`t/`t−1) + (1− β∗)bt−1 bt = β∗(`t/`t−1) + (1− β∗)bt−1

st = γ(yt − `t−1bt−1) + (1−γ)st−m st = γ(yt/(`t−1bt−1)) + (1−γ)st−m

yt+h|t = `tbφht yt+h|t = `tb

φht + st−m+h+

myt+h|t = `tb

φht st−m+h+

Md `t = αyt + (1−α)`t−1bφt−1 `t = α(yt − st−m) + (1−α)`t−1b

φt−1 `t = α(yt/st−m) + (1−α)`t−1b

φt−1

bt = β∗(`t/`t−1) + (1− β∗)bφt−1 bt = β∗(`t/`t−1) + (1− β∗)bφt−1 bt = β∗(`t/`t−1) + (1− β∗)bφt−1st = γ(yt − `t−1b

φt−1) + (1−γ)st−m st = γ(yt/(`t−1b

φt−1)) + (1−γ)st−m

Table 7.8: Formulae for recursivecalculations and point forecasts. Ineach case, `t denotes the series levelat time t, bt denotes the slope at timet, st denotes the seasonal componentof the series at time t, and m denotesthe number of seasons in a year; α, β∗,γ and φ are smoothing parameters,φh = φ + φ2 + · · · + φh and h+

m =b(h− 1) mod mc+ 1.

Method Initial values

(N,N) l0 = y1

(A,N) (Ad,N) l0 = y1, b0 = y2 − y1

(M,N) (Md,N) l0 = y1, b0 = y2/y1

(A,A) (Ad,A) l0 = 1m (y1 + · · ·+ ym)

b0 = 1m

[ym+1−y1

m + · · ·+ ym+m−ymm

s0 = y1 − `0, s−1 = y2 − `0, . . . , s−m+1 = ym − `0

(A,M) (Ad,M) l0 = 1m (y1 + · · ·+ ym)

b0 = 1m

[ym+1−y1

m + · · ·+ ym+m−ymm

s0 = y1/`0, s−1 = y2/`0, . . . , s−m+1 = ym/`0

Table 7.9: Initialisation strategies forsome of the more commonly usedexponential smoothing methods.

Outline

3 Trend methods

4 Seasonal methods

7 ETS in R

Methods V Models

Algorithms that return point forecasts.

Innovations state space models

Generate same point forecasts but can alsogenerate forecast intervals.

A stochastic (or random) data generatingprocess that can generate an entire forecastdistribution.

Allow for “proper” model selection.

Methods V Models

ETS models

Each model has an observation equation andtransition equations, one for each state (level,trend, seasonal), i.e., state space models.

Two models for each method: one with additiveand one with multiplicative errors, i.e., in total30 models.ETS(Error,Trend,Seasonal):

Error= {A, M}Trend = {N, A, Ad, M, Md}Seasonal = {N, A, M}.

ETS models

General notation E T S : ExponenTial Smoothing

Examples:A,N,N: Simple exponential smoothing with additive errorsA,A,N: Holt’s linear method with additive errorsM,A,M: Multiplicative Holt-Winters’ method with multiplicative errors

General notation E T S : ExponenTial Smoothing

Examples:A,N,N: Simple exponential smoothing with additive errorsA,A,N: Holt’s linear method with additive errorsM,A,M: Multiplicative Holt-Winters’ method with multiplicative errors

General notation E T S : ExponenTial Smoothing↑

TrendExamples:

A,N,N: Simple exponential smoothing with additive errorsA,A,N: Holt’s linear method with additive errorsM,A,M: Multiplicative Holt-Winters’ method with multiplicative errors

General notation E T S : ExponenTial Smoothing↑ ↖

Trend SeasonalExamples:

General notation E T S : ExponenTial Smoothing↗ ↑ ↖

Error Trend SeasonalExamples:

å All ETS models can be written in innovationsstate space form.

å Additive and multiplicative versions give thesame point forecasts but different predictionintervals.

ETS(A,N,N)

Observation equation yt = `t−1 + εt,

State equation `t = `t−1 + αεt

et = yt − yt|t−1 = εt

Assume εt ∼ NID(0, σ2)

“innovations” or “single source of error”because same error process, εt.

ETS(A,N,N)

ETS(M,N,N)

SES with multiplicative errors.

Specify relative errors εt =yt−yt|t−1

yt|t−1∼ NID(0, σ2)

Substituting yt|t−1 = `t−1 gives:yt = `t−1 + `t−1εtet = yt − yt|t−1 = `t−1εt

Observation equation yt = `t−1(1 + εt)

State equation `t = `t−1(1 + αεt)

Models with additive and multiplicative errorswith the same parameters generate the samepoint forecasts but different predictionintervals.

ETS(M,N,N)

yt|t−1∼ NID(0, σ2)

ETS(M,N,N)

yt|t−1∼ NID(0, σ2)

ETS(M,N,N)

yt|t−1∼ NID(0, σ2)

ETS(M,N,N)

yt|t−1∼ NID(0, σ2)

ETS(M,N,N)

yt|t−1∼ NID(0, σ2)

ETS(M,N,N)

yt|t−1∼ NID(0, σ2)

Holt’s linear method

ETS(A,A,N)

yt = `t−1 + bt−1 + εt

`t = `t−1 + bt−1 + αεt

bt = bt−1 + βεt

ETS(M,A,N)

yt = (`t−1 + bt−1)(1 + εt)

`t = (`t−1 + bt−1)(1 + αεt)

bt = bt−1 + β(`t−1 + bt−1)εt

Holt’s linear method

ETS(A,A,N)

yt = `t−1 + bt−1 + εt

`t = `t−1 + bt−1 + αεt

bt = bt−1 + βεt

ETS(M,A,N)

yt = (`t−1 + bt−1)(1 + εt)

`t = (`t−1 + bt−1)(1 + αεt)

bt = bt−1 + β(`t−1 + bt−1)εt

ETS(A,A,A)

Holt-Winters additive method with additive errors.

Forecast equation yt+h|t = `t + hbt + st−m+h+m

Observation equation yt = `t−1 + bt−1 + st−m + εt

State equations `t = `t−1 + bt−1 + αεt

bt = bt−1 + βεt

st = st−m + γεt

Forecast errors: εt = yt − yt|t−1

h+m = b(h− 1) mod mc+ 1.

Additive error models7/ exponential smoothing 149

ADDITIVE ERROR MODELS

Trend SeasonalN A M

N yt = `t−1 + εt yt = `t−1 + st−m + εt yt = `t−1st−m + εt`t = `t−1 +αεt `t = `t−1 +αεt `t = `t−1 +αεt/st−m

st = st−m +γεt st = st−m +γεt/`t−1

yt = `t−1 + bt−1 + εt yt = `t−1 + bt−1 + st−m + εt yt = (`t−1 + bt−1)st−m + εtA `t = `t−1 + bt−1 +αεt `t = `t−1 + bt−1 +αεt `t = `t−1 + bt−1 +αεt/st−m

bt = bt−1 + βεt bt = bt−1 + βεt bt = bt−1 + βεt/st−mst = st−m +γεt st = st−m +γεt/(`t−1 + bt−1)

yt = `t−1 +φbt−1 + εt yt = `t−1 +φbt−1 + st−m + εt yt = (`t−1 +φbt−1)st−m + εtAd `t = `t−1 +φbt−1 +αεt `t = `t−1 +φbt−1 +αεt `t = `t−1 +φbt−1 +αεt/st−m

bt = φbt−1 + βεt bt = φbt−1 + βεt bt = φbt−1 + βεt/st−mst = st−m +γεt st = st−m +γεt/(`t−1 +φbt−1)

yt = `t−1bt−1 + εt yt = `t−1bt−1 + st−m + εt yt = `t−1bt−1st−m + εtM `t = `t−1bt−1 +αεt `t = `t−1bt−1 +αεt `t = `t−1bt−1 +αεt/st−m

bt = bt−1 + βεt/`t−1 bt = bt−1 + βεt/`t−1 bt = bt−1 + βεt/(st−m`t−1)st = st−m +γεt st = st−m +γεt/(`t−1bt−1)

yt = `t−1bφt−1 + εt yt = `t−1b

φt−1 + st−m + εt yt = `t−1b

φt−1st−m + εt

Md `t = `t−1bφt−1 +αεt `t = `t−1b

φt−1 +αεt `t = `t−1b

φt−1 +αεt/st−m

bt = bφt−1 + βεt/`t−1 bt = b

φt−1 + βεt/`t−1 bt = b

φt−1 + βεt/(st−m`t−1)

st = st−m +γεt st = st−m +γεt/(`t−1bφt−1)

MULTIPLICATIVE ERROR MODELS

Trend SeasonalN A M

N yt = `t−1(1 + εt) yt = (`t−1 + st−m)(1 + εt) yt = `t−1st−m(1 + εt)`t = `t−1(1 +αεt) `t = `t−1 +α(`t−1 + st−m)εt `t = `t−1(1 +αεt)

st = st−m +γ(`t−1 + st−m)εt st = st−m(1 +γεt)

yt = (`t−1 + bt−1)(1 + εt) yt = (`t−1 + bt−1 + st−m)(1 + εt) yt = (`t−1 + bt−1)st−m(1 + εt)A `t = (`t−1 + bt−1)(1 +αεt) `t = `t−1 + bt−1 +α(`t−1 + bt−1 + st−m)εt `t = (`t−1 + bt−1)(1 +αεt)

bt = bt−1 + β(`t−1 + bt−1)εt bt = bt−1 + β(`t−1 + bt−1 + st−m)εt bt = bt−1 + β(`t−1 + bt−1)εtst = st−m +γ(`t−1 + bt−1 + st−m)εt st = st−m(1 +γεt)

yt = (`t−1 +φbt−1)(1 + εt) yt = (`t−1 +φbt−1 + st−m)(1 + εt) yt = (`t−1 +φbt−1)st−m(1 + εt)Ad `t = (`t−1 +φbt−1)(1 +αεt) `t = `t−1 +φbt−1 +α(`t−1 +φbt−1 + st−m)εt `t = (`t−1 +φbt−1)(1 +αεt)

bt = φbt−1 + β(`t−1 +φbt−1)εt bt = φbt−1 + β(`t−1 +φbt−1 + st−m)εt bt = φbt−1 + β(`t−1 +φbt−1)εtst = st−m +γ(`t−1 +φbt−1 + st−m)εt st = st−m(1 +γεt)

yt = `t−1bt−1(1 + εt) yt = (`t−1bt−1 + st−m)(1 + εt) yt = `t−1bt−1st−m(1 + εt)M `t = `t−1bt−1(1 +αεt) `t = `t−1bt−1 +α(`t−1bt−1 + st−m)εt `t = `t−1bt−1(1 +αεt)

bt = bt−1(1 + βεt) bt = bt−1 + β(`t−1bt−1 + st−m)εt/`t−1 bt = bt−1(1 + βεt)st = st−m +γ(`t−1bt−1 + st−m)εt st = st−m(1 +γεt)

yt = `t−1bφt−1(1 + εt) yt = (`t−1b

φt−1 + st−m)(1 + εt) yt = `t−1b

φt−1st−m(1 + εt)

Md `t = `t−1bφt−1(1 +αεt) `t = `t−1b

φt−1 +α(`t−1b

φt−1 + st−m)εt `t = `t−1b

φt−1(1 +αεt)

bt = bφt−1(1 + βεt) bt = b

φt−1 + β(`t−1b

φt−1 + st−m)εt/`t−1 bt = b

φt−1(1 + βεt)

st = st−m +γ(`t−1bφt−1 + st−m)εt st = st−m(1 +γεt)

Table 7.10: State space equationsfor each of the models in the ETSframework.

Multiplicative error models

7/ exponential smoothing 149

ADDITIVE ERROR MODELS

Trend SeasonalN A M

N yt = `t−1 + εt yt = `t−1 + st−m + εt yt = `t−1st−m + εt`t = `t−1 +αεt `t = `t−1 +αεt `t = `t−1 +αεt/st−m

st = st−m +γεt st = st−m +γεt/`t−1

yt = `t−1 + bt−1 + εt yt = `t−1 + bt−1 + st−m + εt yt = (`t−1 + bt−1)st−m + εtA `t = `t−1 + bt−1 +αεt `t = `t−1 + bt−1 +αεt `t = `t−1 + bt−1 +αεt/st−m

bt = bt−1 + βεt bt = bt−1 + βεt bt = bt−1 + βεt/st−mst = st−m +γεt st = st−m +γεt/(`t−1 + bt−1)

yt = `t−1 +φbt−1 + εt yt = `t−1 +φbt−1 + st−m + εt yt = (`t−1 +φbt−1)st−m + εtAd `t = `t−1 +φbt−1 +αεt `t = `t−1 +φbt−1 +αεt `t = `t−1 +φbt−1 +αεt/st−m

bt = φbt−1 + βεt bt = φbt−1 + βεt bt = φbt−1 + βεt/st−mst = st−m +γεt st = st−m +γεt/(`t−1 +φbt−1)

yt = `t−1bt−1 + εt yt = `t−1bt−1 + st−m + εt yt = `t−1bt−1st−m + εtM `t = `t−1bt−1 +αεt `t = `t−1bt−1 +αεt `t = `t−1bt−1 +αεt/st−m

bt = bt−1 + βεt/`t−1 bt = bt−1 + βεt/`t−1 bt = bt−1 + βεt/(st−m`t−1)st = st−m +γεt st = st−m +γεt/(`t−1bt−1)

yt = `t−1bφt−1 + εt yt = `t−1b

φt−1 + st−m + εt yt = `t−1b

φt−1st−m + εt

Md `t = `t−1bφt−1 +αεt `t = `t−1b

φt−1 +αεt `t = `t−1b

φt−1 +αεt/st−m

bt = bφt−1 + βεt/`t−1 bt = b

φt−1 + βεt/`t−1 bt = b

φt−1 + βεt/(st−m`t−1)

st = st−m +γεt st = st−m +γεt/(`t−1bφt−1)

MULTIPLICATIVE ERROR MODELS

Trend SeasonalN A M

N yt = `t−1(1 + εt) yt = (`t−1 + st−m)(1 + εt) yt = `t−1st−m(1 + εt)`t = `t−1(1 +αεt) `t = `t−1 +α(`t−1 + st−m)εt `t = `t−1(1 +αεt)

st = st−m +γ(`t−1 + st−m)εt st = st−m(1 +γεt)

yt = (`t−1 + bt−1)(1 + εt) yt = (`t−1 + bt−1 + st−m)(1 + εt) yt = (`t−1 + bt−1)st−m(1 + εt)A `t = (`t−1 + bt−1)(1 +αεt) `t = `t−1 + bt−1 +α(`t−1 + bt−1 + st−m)εt `t = (`t−1 + bt−1)(1 +αεt)

bt = bt−1 + β(`t−1 + bt−1)εt bt = bt−1 + β(`t−1 + bt−1 + st−m)εt bt = bt−1 + β(`t−1 + bt−1)εtst = st−m +γ(`t−1 + bt−1 + st−m)εt st = st−m(1 +γεt)

yt = (`t−1 +φbt−1)(1 + εt) yt = (`t−1 +φbt−1 + st−m)(1 + εt) yt = (`t−1 +φbt−1)st−m(1 + εt)Ad `t = (`t−1 +φbt−1)(1 +αεt) `t = `t−1 +φbt−1 +α(`t−1 +φbt−1 + st−m)εt `t = (`t−1 +φbt−1)(1 +αεt)

bt = φbt−1 + β(`t−1 +φbt−1)εt bt = φbt−1 + β(`t−1 +φbt−1 + st−m)εt bt = φbt−1 + β(`t−1 +φbt−1)εtst = st−m +γ(`t−1 +φbt−1 + st−m)εt st = st−m(1 +γεt)

yt = `t−1bt−1(1 + εt) yt = (`t−1bt−1 + st−m)(1 + εt) yt = `t−1bt−1st−m(1 + εt)M `t = `t−1bt−1(1 +αεt) `t = `t−1bt−1 +α(`t−1bt−1 + st−m)εt `t = `t−1bt−1(1 +αεt)

bt = bt−1(1 + βεt) bt = bt−1 + β(`t−1bt−1 + st−m)εt/`t−1 bt = bt−1(1 + βεt)st = st−m +γ(`t−1bt−1 + st−m)εt st = st−m(1 +γεt)

yt = `t−1bφt−1(1 + εt) yt = (`t−1b

φt−1 + st−m)(1 + εt) yt = `t−1b

φt−1st−m(1 + εt)

Md `t = `t−1bφt−1(1 +αεt) `t = `t−1b

φt−1 +α(`t−1b

φt−1 + st−m)εt `t = `t−1b

φt−1(1 +αεt)

bt = bφt−1(1 + βεt) bt = b

φt−1 + β(`t−1b

φt−1 + st−m)εt/`t−1 bt = b

φt−1(1 + βεt)

st = st−m +γ(`t−1bφt−1 + st−m)εt st = st−m(1 +γεt)

Table 7.10: State space equationsfor each of the models in the ETSframework.

Let xt = (`t,bt, st, st−1, . . . , st−m+1) and εtiid∼ N(0, σ2).

yt = h(xt−1)︸︷︷︸+ k(xt−1)εt︸︷︷︸µt et

xt = f(xt−1) + g(xt−1)εt

Additive errors:k(x) = 1. yt = µt + εt.

Multiplicative errors:k(xt−1) = µt. yt = µt(1 + εt).εt = (yt − µt)/µt is relative error.

All the methods can be written in this statespace form.

The only difference between the additive errorand multiplicative error models is in theobservation equation.

Additive and multiplicative versions give thesame point forecasts.

Some unstable models

Some of the combinations of (Error, Trend,Seasonal) can lead to numerical difficulties; seeequations with division by a state.

These are: ETS(M,M,A), ETS(M,Md,A),ETS(A,N,M), ETS(A,A,M), ETS(A,Ad,M),ETS(A,M,N), ETS(A,M,A), ETS(A,M,M),ETS(A,Md,N), ETS(A,Md,A), and ETS(A,Md,M).

Models with multiplicative errors are useful forstrictly positive data – but are not numericallystable with data containing zeros or negativevalues. In that case only the six fully additivemodels will be applied.

Exponential smoothing models

Additive Error Seasonal ComponentTrend N A M

N (None) A,N,N A,N,A A,N,M

A (Additive) A,A,N A,A,A A,A,M

Ad (Additive damped) A,Ad,N A,Ad,A A,Ad,M

M (Multiplicative) A,M,N A,M,A A,M,M

Md (Multiplicative damped) A,Md,N A,Md,A A,Md,M

Multiplicative Error Seasonal ComponentTrend N A M

N (None) M,N,N M,N,A M,N,M

A (Additive) M,A,N M,A,A M,A,M

Ad (Additive damped) M,Ad,N M,Ad,A M,Ad,M

M (Multiplicative) M,M,N M,M,A M,M,M

Md (Multiplicative damped) M,Md,N M,Md,A M,Md,M

Estimation

L∗(θ,x0) = n log

( n∑t=1

ε2t /k

2(xt−1)

n∑t=1

log |k(xt−1)|

= −2 log(Likelihood) + constant

Estimate parameters θ = (α, β, γ, φ) and initialstates x0 = (`0,b0, s0, s−1, . . . , s−m+1) byminimizing L∗.

Estimation

L∗(θ,x0) = n log

( n∑t=1

ε2t /k

2(xt−1)

n∑t=1

log |k(xt−1)|

= −2 log(Likelihood) + constant

Estimate parameters θ = (α, β, γ, φ) and initialstates x0 = (`0,b0, s0, s−1, . . . , s−m+1) byminimizing L∗.

Parameter restrictionsUsual region

Traditional restrictions in the methods0 < α, β∗, γ∗, φ < 1 — equations interpreted asweighted averages.

In models we set β = αβ∗ and γ = (1− α)γ∗ therefore0 < α < 1, 0 < β < α and 0 < γ < 1− α.

0.8 < φ < 0.98 — to prevent numerical difficulties.

Admissible region

To prevent observations in the distant past having acontinuing effect on current forecasts.

Usually (but not always) less restrictive than theusual region.

For example for ETS(A,N,N):usual 0 < α < 1 — admissible is 0 < α < 2.

Admissible region

Model selection

Akaike’s Information Criterion

AIC = −2 log(L) + 2k

where L is the likelihood and k is the number ofparameters initial states estimated in the model.

Corrected AIC

AICc = AIC +2(k + 1)(k + 2)

T − k

which is the AIC corrected (for small sample bias).

Bayesian Information Criterion

BIC = AIC + k(log(T)− 2).

Model selection

Corrected AIC

AICc = AIC +2(k + 1)(k + 2)

T − k

Model selection

Corrected AIC

AICc = AIC +2(k + 1)(k + 2)

T − k

Automatic forecasting

From Hyndman et al. (IJF, 2002):

Apply each model that is appropriate to thedata. Optimize parameters and initial valuesusing MLE (or some other criterion).

Select best method using AIC:

AIC = −2 log(Likelihood) + 2p

where p = # parameters.

Produce forecasts using best method.

Obtain prediction intervals using underlyingstate space model.

Method performed very well in M3 competition.State space models 1: Exponential smoothing 46

Forecasting with ETS models

Point forecasts obtained by iterating equationsfor t = T + 1, . . . , T + h, setting εt = 0 for t > T.

Not the same as E(yt+h|xt) unless trend andseasonality are both additive.

Point forecasts for ETS(A,x,y) are identical toETS(M,x,y) if the parameters are the same.

Prediction intervals will differ between modelswith additive and multiplicative methods.

Exact PI available for many models.

Otherwise, simulate future sample paths,conditional on last estimate of states, andobtain PI from percentiles of simulated paths.

Point forecasts: iterate the equations fort = T + 1,T + 2, . . . ,T + h and set all εt = 0 for t > T.For example, for ETS(M,A,N):

yT+1 = (`T + bT)(1 + εT+1)

Therefore yT+1|T = `T + bTyT+2 = (`T+1 + bT+1)(1 + εT+1) =

[(`T + bT)(1 + αεT+1) + bT + β(`T + bT)εT+1] (1 + εT+1)

Therefore yT+2|T = `T + 2bT and so on.

Identical forecast with Holt’s linear method andETS(A,A,N). So the point forecasts obtained from themethod and from the two models that underly themethod are identical (assuming the same parametervalues are used).

yT+1 = (`T + bT)(1 + εT+1)

Prediction intervals: cannot be generated usingthe methods.

The prediction intervals will differ betweenmodels with additive and multiplicativemethods.

Exact formulae for some models.

More general to simulate future sample paths,conditional on the last estimate of the states,and to obtain prediction intervals from thepercentiles of these simulated future paths.

Options are available in R using the forecastfunction in the forecast package.

Outline

3 Trend methods

4 Seasonal methods

7 ETS in R

Exponential smoothingForecasts from ETS(M,Md,M)

1995 2000 2005 2010

Exponential smoothingForecasts from ETS(M,Md,M)

1995 2000 2005 2010

library(forecast)fit <- ets(h02)fcast <- forecast(fit)plot(fcast)

Exponential smoothing> fitETS(M,Md,M)

Smoothing parameters:alpha = 0.3318beta = 4e-04gamma = 1e-04phi = 0.9695

Initial states:l = 0.4003b = 1.0233s = 0.8575 0.8183 0.7559 0.7627 0.6873 1.2884

1.3456 1.1867 1.1653 1.1033 1.0398 0.9893

sigma: 0.0651

AIC AICc BIC-121.97999 -118.68967 -65.57195

The ets() function in R

ets(y, model="ZZZ", damped=NULL,alpha=NULL, beta=NULL,gamma=NULL, phi=NULL,additive.only=FALSE,lambda=NULLlower=c(rep(0.0001,3),0.80),upper=c(rep(0.9999,3),0.98),opt.crit=c("lik","amse","mse","sigma"),nmse=3,bounds=c("both","usual","admissible"),ic=c("aic","aicc","bic"), restrict=TRUE)

yThe time series to be forecast.

modeluse the ETS classification and notation: “N” for none,“A” for additive, “M” for multiplicative, or “Z” forautomatic selection. Default ZZZ all components areselected using the information criterion.

dampedIf damped=TRUE, then a damped trend will be used(either Ad or Md).damped=FALSE, then a non-damped trend will used.If damped=NULL (the default), then either a dampedor a non-damped trend will be selected according tothe information criterion chosen.

The ets() function in Ralpha, beta, gamma, phiThe values of the smoothing parameters can bespecified using these arguments. If they are set toNULL (the default value for each of them), theparameters are estimated.

additive.onlyOnly models with additive components will beconsidered if additive.only=TRUE. Otherwise allmodels will be considered.

lambdaBox-Cox transformation parameter. It will be ignoredif lambda=NULL (the default value). Otherwise, thetime series will be transformed before the model isestimated. When lambda is not NULL,additive.only is set to TRUE.

lower,upper bounds for the parameter estimates ofα, β, γ and φ.opt.crit=lik (default) optimisation criterion usedfor estimation.bounds Constraints on the parameters.

usual region – "bounds=usual";admissible region – "bounds=admissible";"bounds=both" (the default) requires theparameters to satisfy both sets of constraints.

ic=aic (the default) information criterion to be usedin selecting models.restrict=TRUE (the default) models that causenumerical difficulties are not considered in modelselection.

The forecast() function in Rforecast(object,h=ifelse(object$m>1, 2*object$m, 10),level=c(80,95), fan=FALSE,simulate=FALSE, bootstrap=FALSE,npaths=5000, PI=TRUE, lambda=object$lambda, ...)object: the object returned by the ets() function.

h: the number of periods to be forecast.

level: the confidence level for the predictionintervals.

fan: if fan=TRUE, suitable for fan plots.simulate

If simulate=TRUE, prediction intervals generatedvia simulation rather than analytic formulae.Even if simulate=FALSE simulation will be used ifthere are no algebraic formulae exist.

The forecast() function in R

bootstrap: If bootstrap=TRUE and simulate=TRUE,then the simulated prediction intervals usere-sampled errors rather than normally distributederrors.

npaths: The number of sample paths used incomputing simulated prediction intervals.

PI: If PI=TRUE, then prediction intervals areproduced; otherwise only point forecasts arecalculated. If PI=FALSE, then level, fan, simulate,bootstrap and npaths are all ignored.

lambda: The Box-Cox transformation parameter. Thisis ignored if lambda=NULL. Otherwise, forecasts areback-transformed via an inverse Box-Coxtransformation.

state space models - rob j hyndman · 1the state space perspective 2simple exponential smoothing...

Documents