Automatic Forecasting Algorithms 1
2017 Beijing Workshop onForecasting
AutomaticForecastingAlgorithmsRob J Hyndman
robjhyndman.com/beijing2017
Outline
1 Motivation
2 ETS
3 ARIMA models
4 STLM
5 TBATS
6 FASSTER
7 Comparisons
Automatic Forecasting Algorithms Motivation 2
Motivation
Automatic Forecasting Algorithms Motivation 3
Motivation
Automatic Forecasting Algorithms Motivation 3
Motivation
Automatic Forecasting Algorithms Motivation 3
Motivation
Automatic Forecasting Algorithms Motivation 3
Motivation
Automatic Forecasting Algorithms Motivation 3
Motivation
Common in business to have over 1000 products thatneed forecasting at least monthly.Forecasts are often required by people who areuntrained in time series analysis.
SpecificationsAutomatic forecasting algorithms must:
å determine an appropriate time series model;å estimate the parameters;å compute the forecasts with prediction intervals.
Automatic Forecasting Algorithms Motivation 4
Motivation
Common in business to have over 1000 products thatneed forecasting at least monthly.Forecasts are often required by people who areuntrained in time series analysis.
SpecificationsAutomatic forecasting algorithms must:
å determine an appropriate time series model;å estimate the parameters;å compute the forecasts with prediction intervals.
Automatic Forecasting Algorithms Motivation 4
Outline
1 Motivation
2 ETS
3 ARIMA models
4 STLM
5 TBATS
6 FASSTER
7 Comparisons
Automatic Forecasting Algorithms ETS 5
Exponential smoothing methodsSeasonal Component
Trend N A MComponent (None) (Additive) (Multiplicative)
N (None) (N,N) (N,A) (N,M)A (Additive) (A,N) (A,A) (A,M)Ad (Additive damped) (Ad,N) (Ad,A) (Ad,M)
(N,N): Simple exponential smoothing(A,N): Holt’s linear method(Ad,N): Additive damped trend method(A,A): Additive Holt-Winters’ method(A,M): Multiplicative Holt-Winters’ method(Ad,M): Damped multiplicative Holt-Winters’ method
There are also multiplicative trend methods (not recommended).Automatic Forecasting Algorithms ETS 6
Methods V ModelsExponential smoothing methods
Algorithms that return point forecasts.
Innovations state space modelsGenerate same point forecasts but can also generateforecast intervals.A stochastic (or random) data generating process thatcan generate an entire forecast distribution.Allow for “proper” model selection.ETS(Error,Trend,Seasonal):
Error = {A,M}Trend = {N,A,Ad}Seasonal = {N,A,M}.
Automatic Forecasting Algorithms ETS 7
Methods V ModelsExponential smoothing methods
Algorithms that return point forecasts.
Innovations state space modelsGenerate same point forecasts but can also generateforecast intervals.A stochastic (or random) data generating process thatcan generate an entire forecast distribution.Allow for “proper” model selection.ETS(Error,Trend,Seasonal):
Error = {A,M}Trend = {N,A,Ad}Seasonal = {N,A,M}.
Automatic Forecasting Algorithms ETS 7
Exponential smoothing methodsSeasonal Component
Trend N A MComponent (None) (Additive) (Multiplicative)
N (None) N,N N,A N,M
A (Additive) A,N A,A A,M
Ad (Additive damped) Ad,N Ad,A Ad,M
Automatic Forecasting Algorithms ETS 8
Exponential smoothing modelsSeasonal Component
Trend N A MComponent (None) (Additive) (Multiplicative)
N (None) N,N N,A N,M
A (Additive) A,N A,A A,M
Ad (Additive damped) Ad,N Ad,A Ad,M
General notation E T S : ExponenTial Smoothing↗ ↑ ↖
Error Trend SeasonalExamples:A,N,N: Simple exponential smoothing with additive errorsA,A,N: Holt’s linear method with additive errorsM,A,M: Multiplicative Holt-Winters’ method with multiplicative errors
Automatic Forecasting Algorithms ETS 9
Exponential smoothing modelsSeasonal Component
Trend N A MComponent (None) (Additive) (Multiplicative)
N (None) N,N N,A N,M
A (Additive) A,N A,A A,M
Ad (Additive damped) Ad,N Ad,A Ad,M
General notation E T S : ExponenTial Smoothing↗ ↑ ↖
Error Trend SeasonalExamples:A,N,N: Simple exponential smoothing with additive errorsA,A,N: Holt’s linear method with additive errorsM,A,M: Multiplicative Holt-Winters’ method with multiplicative errors
Automatic Forecasting Algorithms ETS 9
Exponential smoothing modelsSeasonal Component
Trend N A MComponent (None) (Additive) (Multiplicative)
N (None) N,N N,A N,M
A (Additive) A,N A,A A,M
Ad (Additive damped) Ad,N Ad,A Ad,M
There are 9 separate exp. smoothing methods.Each can have an additive or multiplicative error, giving 18separate models.Only 15 models are numerically stable.Additive and multiplicative error models give same pointforecasts but different prediction intervals.All models can be written in innovations state space form.
Automatic Forecasting Algorithms ETS 10
Exponential smoothing modelsSeasonal Component
Trend N A MComponent (None) (Additive) (Multiplicative)
N (None) N,N N,A N,M
A (Additive) A,N A,A A,M
Ad (Additive damped) Ad,N Ad,A Ad,M
There are 9 separate exp. smoothing methods.Each can have an additive or multiplicative error, giving 18separate models.Only 15 models are numerically stable.Additive and multiplicative error models give same pointforecasts but different prediction intervals.All models can be written in innovations state space form.
Automatic Forecasting Algorithms ETS 10
Exponential smoothing modelsAdditive Error Seasonal Component
Trend N A MComponent (None) (Additive) (Multiplicative)
N (None) A,N,N A,N,A A,N,M
A (Additive) A,A,N A,A,A A,A,M
Ad (Additive damped) A,Ad,N A,Ad,A A,Ad,M
Multiplicative Error Seasonal ComponentTrend N A M
Component (None) (Additive) (Multiplicative)
N (None) M,N,N M,N,A M,N,M
A (Additive) M,A,N M,A,A M,A,M
Ad (Additive damped) M,Ad,N M,Ad,A M,Ad,MAutomatic Forecasting Algorithms ETS 11
ETS state space modelsxt−1
εt
yt
{}
Automatic Forecasting Algorithms ETS 12
State space modelxt = (level, slope, seasonal)
ETS state space modelsxt−1
εt
yt
xt
{}
Automatic Forecasting Algorithms ETS 12
State space modelxt = (level, slope, seasonal)
ETS state space modelsxt−1
εt
yt
xt yt+1
εt+1
{}
Automatic Forecasting Algorithms ETS 12
State space modelxt = (level, slope, seasonal)
ETS state space modelsxt−1
εt
yt
xt yt+1
εt+1 xt+1
{}
Automatic Forecasting Algorithms ETS 12
State space modelxt = (level, slope, seasonal)
ETS state space modelsxt−1
εt
yt
xt yt+1
εt+1 xt+1 yt+2
εt+2
{}
Automatic Forecasting Algorithms ETS 12
State space modelxt = (level, slope, seasonal)
ETS state space modelsxt−1
εt
yt
xt yt+1
εt+1 xt+1 yt+2
εt+2 xt+2
{}
Automatic Forecasting Algorithms ETS 12
State space modelxt = (level, slope, seasonal)
ETS state space modelsxt−1
εt
yt
xt yt+1
εt+1 xt+1 yt+2
εt+2 xt+2 yt+3
εt+3
{}
Automatic Forecasting Algorithms ETS 12
State space modelxt = (level, slope, seasonal)
ETS state space modelsxt−1
εt
yt
xt yt+1
εt+1 xt+1 yt+2
εt+2 xt+2 yt+3
εt+3 xt+3
{}
Automatic Forecasting Algorithms ETS 12
State space modelxt = (level, slope, seasonal)
ETS state space modelsxt−1
εt
yt
xt yt+1
εt+1 xt+1 yt+2
εt+2 xt+2 yt+3
εt+3 xt+3 yt+4
εt+4
{}
Automatic Forecasting Algorithms ETS 12
State space modelxt = (level, slope, seasonal)
ETS state space modelsxt−1
εt
yt
xt yt+1
εt+1 xt+1 yt+2
εt+2 xt+2 yt+3
εt+3 xt+3 yt+4
εt+4
{}
Automatic Forecasting Algorithms ETS 12
State space modelxt = (level, slope, seasonal)
EstimationCompute likelihood L fromε1, ε2, . . . , εT.Optimize L wrt model parameters.
ETS state space modelsLet xt = (`t, bt, st, st−1, . . . , st−m+1) and εt
iid∼ N(0, σ2).
yt = h(xt−1)︸ ︷︷ ︸ + k(xt−1)εt︸ ︷︷ ︸ Observation equationµt et
xt = f(xt−1) + g(xt−1)εt State equation
Additive errors:k(xt−1) = 1. yt = µt + εt.
Multiplicative errors:k(xt−1) = µt. yt = µt(1 + εt).εt = (yt − µt)/µt is relative error.
Automatic Forecasting Algorithms ETS 13
Innovations state space models
Estimation
L∗(θ, x0) = n log( n∑
t=1ε2t /k
2(xt−1))
+ 2n∑t=1
log |k(xt−1)|
= −2 log(Likelihood) + constant
Estimate parameters θ = (α, β, γ, φ) and initial statesx0 = (`0, b0, s0, s−1, . . . , s−m+1) by minimizing L∗.
Automatic Forecasting Algorithms ETS 14
Automatic forecasting
From Hyndman et al. (IJF, 2002):
Apply each model that is appropriate to the data.Optimize parameters and initial values using MLE.Select best method using AICc.Produce forecasts using best method.Obtain forecast intervals using underlying state spacemodel.
Method performed very well in M3 competition.
Automatic Forecasting Algorithms ETS 15
Example: Asian livestock
livestock %>% ets() %>% forecast() %>% autoplot()
300
400
500
600
1960 1980 2000
Time
.
level
80
95
Forecasts from ETS(M,A,N)
Automatic Forecasting Algorithms ETS 16
Example: drug sales
h02 %>% ets() %>% forecast() %>% autoplot()
0.3
0.6
0.9
1.2
1.5
1995 2000 2005 2010
Time
.
level
80
95
Forecasts from ETS(M,Ad,M)
Automatic Forecasting Algorithms ETS 17
ETS
Automatic Forecasting Algorithms ETS 18
ETS
Automatic Forecasting Algorithms ETS 18
www.OTexts.org/fpp2
ETS
Automatic Forecasting Algorithms ETS 18
Outline
1 Motivation
2 ETS
3 ARIMA models
4 STLM
5 TBATS
6 FASSTER
7 Comparisons
Automatic Forecasting Algorithms ARIMA models 19
ARIMA models
yt−1
yt−2
yt−3yt
Inputs Output
Automatic Forecasting Algorithms ARIMA models 20
ARIMA models
yt−1
yt−2
yt−3
εt
yt
Inputs Output
Automatic Forecasting Algorithms ARIMA models 20
Autoregression (AR) model
ARIMA models
yt−1
yt−2
yt−3
εt
εt−1
εt−2
yt
Inputs Output
Automatic Forecasting Algorithms ARIMA models 20
Autoregression movingaverage (ARMA) model
ARIMA models
yt−1
yt−2
yt−3
εt
εt−1
εt−2
yt
Inputs Output
Automatic Forecasting Algorithms ARIMA models 20
Autoregression movingaverage (ARMA) model
EstimationCompute likelihood L fromε1, ε2, . . . , εT.Use optimization algorithm tomaximize L.
ARIMA models
yt−1
yt−2
yt−3
εt
εt−1
εt−2
yt
Inputs Output
Automatic Forecasting Algorithms ARIMA models 20
Autoregression movingaverage (ARMA) model
EstimationCompute likelihood L fromε1, ε2, . . . , εT.Use optimization algorithm tomaximize L.
ARIMA modelAutoregression movingaverage (ARMA) modelapplied to differences.
Auto ARIMAlivestock %>% auto.arima() %>% forecast() %>% autoplot()
300
400
500
1960 1980 2000
Time
.
level
80
95
Forecasts from ARIMA(0,1,0) with drift
Automatic Forecasting Algorithms ARIMA models 21
Auto ARIMAh02 %>% auto.arima() %>% forecast() %>% autoplot()
0.3
0.6
0.9
1.2
1995 2000 2005 2010
Time
.
level
80
95
Forecasts from ARIMA(3,1,3)(0,1,1)[12]
Automatic Forecasting Algorithms ARIMA models 22
How does auto.arima() work?A non-seasonal ARIMA process
φ(B)(1− B)dyt = c + θ(B)εtNeed to select appropriate orders p, q, d, and whether toinclude c.
Automatic Forecasting Algorithms ARIMA models 23
Algorithm choices driven by forecast accuracy.
How does auto.arima() work?A non-seasonal ARIMA process
φ(B)(1− B)dyt = c + θ(B)εtNeed to select appropriate orders p, q, d, and whether toinclude c.
Hyndman & Khandakar (JSS, 2008) algorithm:Select no. differences d via KPSS unit root test.Select p, q, c by minimising AICc.Use stepwise search to traverse model space, startingwith a simple model and considering nearby variants.
Automatic Forecasting Algorithms ARIMA models 23
Algorithm choices driven by forecast accuracy.
How does auto.arima() work?A non-seasonal ARIMA process
φ(B)(1− B)dyt = c + θ(B)εtNeed to select appropriate orders p, q, d, and whether toinclude c.
Hyndman & Khandakar (JSS, 2008) algorithm:Select no. differences d via KPSS unit root test.Select p, q, c by minimising AICc.Use stepwise search to traverse model space, startingwith a simple model and considering nearby variants.
Automatic Forecasting Algorithms ARIMA models 23
Algorithm choices driven by forecast accuracy.
How does auto.arima() work?A seasonal ARIMA process
Φ(Bm)φ(B)(1− B)d(1− Bm)Dyt = c + Θ(Bm)θ(B)εtNeed to select appropriate orders p, q, d, P,Q,D, andwhether to include c.
Hyndman & Khandakar (JSS, 2008) algorithm:
Select no. differences d via KPSS unit root test.Select D using OCSB unit root test.Select p, q, P,Q, c by minimising AICc.Use stepwise search to traverse model space, startingwith a simple model and considering nearby variants.
Automatic Forecasting Algorithms ARIMA models 24
Outline
1 Motivation
2 ETS
3 ARIMA models
4 STLM
5 TBATS
6 FASSTER
7 Comparisons
Automatic Forecasting Algorithms STLM 25
STL decompositionSTL: “Seasonal and Trend decomposition using Loess”,Very versatile and robust.Unlike X-12-ARIMA, STL will handle any type ofseasonality.Seasonal component allowed to change over time,and rate of change controlled by user.Smoothness of trend-cycle also controlled by user.Robust to outliersNot trading day or calendar adjustments.Only additive.Take logs to get multiplicative decomposition.Use Box-Cox transformations to get otherdecompositions.
Automatic Forecasting Algorithms STLM 26
STL decompositionfit <- stl(elecequip, s.window=5, robust=TRUE)autoplot(fit) +
ggtitle("STL decomposition of electrical equipment index")
data
tren
dse
ason
alre
mai
nder
2000 2005 2010
60
80
100
120
80
90
100
110
−20
−10
0
10
−20
−10
0
10
Time
STL decomposition of electrical equipment index
Automatic Forecasting Algorithms STLM 27
STL decompositionfit <- stl(elecequip, s.window="periodic", robust=TRUE)autoplot(fit) +
ggtitle("STL decomposition of electrical equipment index")
data
tren
dse
ason
alre
mai
nder
2000 2005 2010
60
80
100
120
80
90
100
110
−10
0
10
−10−5
05
10
Time
STL decomposition of electrical equipment index
Automatic Forecasting Algorithms STLM 28
Forecasting and decomposition
Forecast seasonal component using seasonal naivemethod.Forecast seasonally adjusted data using non-seasonaltime series method. E.g., ETS or ARIMA.Combine forecasts of seasonal component withforecasts of seasonally adjusted data to get forecastsof original data.
Automatic Forecasting Algorithms STLM 29
Electrical equipmentfit <- stl(elecequip, s.window=7)fit %>% seasadj() %>% ets() %>% forecast() %>% autoplot() + ylab("seasonally adjusted elecequip")
60
80
100
120
2000 2005 2010 2015
Time
seas
onal
ly a
djus
ted
elec
equi
p
level
80
95
Forecasts from ETS(A,Ad,N)
Automatic Forecasting Algorithms STLM 30
Electrical equipmentfit %>% forecast() %>%
autoplot()
50
75
100
125
2000 2005 2010 2015
Time
elec
equi
p level
80
95
Forecasts from STL + ETS(A,Ad,N)
Automatic Forecasting Algorithms STLM 31
Forecasting and decompositionelecequip %>% stlf() %>%
autoplot() + ylab('elecequip')
50
75
100
125
2000 2005 2010 2015
Time
elec
equi
p level
80
95
Forecasts from STL + ETS(A,Ad,N)
Automatic Forecasting Algorithms STLM 32
Decomposition and prediction intervals
It is common to take the prediction intervals from theseasonally adjusted forecasts and modify them withthe seasonal component.This ignores the uncertainty in the seasonalcomponent estimate.It also ignores the uncertainty in the future seasonalpattern.
Automatic Forecasting Algorithms STLM 33
Outline
1 Motivation
2 ETS
3 ARIMA models
4 STLM
5 TBATS
6 FASSTER
7 Comparisons
Automatic Forecasting Algorithms TBATS 34
Complex seasonality
7000
8000
9000
10000
1990 2000 2010 2020
Year
Tho
usan
ds o
f bar
rels
per
day
Gasoline data
Automatic Forecasting Algorithms TBATS 35
Complex seasonality
7000
8000
9000
10000
1990 2000 2010 2020
Year
Tho
usan
ds o
f bar
rels
per
day
Forecasts from TBATS(1, {0,0}, −, {<52.18,9>})
Automatic Forecasting Algorithms TBATS 36
Complex seasonality
7000
8000
9000
10000
1990 2000 2010 2020
Year
Tho
usan
ds o
f bar
rels
per
day
Forecasts from TBATS(1, {0,0}, −, {<52.18,9>})
blah
Automatic Forecasting Algorithms TBATS 37
fit <- tbats(gasoline)fcast <- forecast(fit)autoplot(fcast)
Complex seasonality
100
200
300
400
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
Week
Num
ber
of c
alls
in 5
min
ute
inte
rval
Number of calls to large American bank {7am..9pm}
Automatic Forecasting Algorithms TBATS 38
Complex seasonality
100
200
300
400
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
Week
Num
ber
of c
alls
in 5
min
ute
inte
rval
Forecasts from TBATS(1, {3,1}, 0.8, {<169,6>, <845,4>})
Automatic Forecasting Algorithms TBATS 39
Complex seasonality
100
200
300
400
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
Week
Num
ber
of c
alls
in 5
min
ute
inte
rval
Forecasts from TBATS(1, {3,1}, 0.8, {<169,6>, <845,4>})
blah
Automatic Forecasting Algorithms TBATS 40
fit <- tbats(calls)fcast <- forecast(fit)autoplot(fcast)
TBATS model
yt = observation at time t
y(ω)t =
(yωt − 1)/ω if ω 6= 0;
log yt if ω = 0.
y(ω)t = `t−1 + φbt−1 +
M∑i=1
s(i)t−mi
+ dt
`t = `t−1 + φbt−1 + αdtbt = (1− φ)b + φbt−1 + βdt
dt =p∑i=1φidt−i +
q∑j=1θjεt−j + εt
s(i)t =
ki∑j=1
s(i)j,t
Automatic Forecasting Algorithms TBATS 41
s(i)j,t = s(i)
j,t−1 cosλ(i)j + s∗(i)
j,t−1 sinλ(i)j + γ(i)
1 dts(i)j,t = −s(i)
j,t−1 sinλ(i)j + s∗(i)
j,t−1 cosλ(i)j + γ(i)
2 dt
TBATS model
yt = observation at time t
y(ω)t =
(yωt − 1)/ω if ω 6= 0;
log yt if ω = 0.
y(ω)t = `t−1 + φbt−1 +
M∑i=1
s(i)t−mi
+ dt
`t = `t−1 + φbt−1 + αdtbt = (1− φ)b + φbt−1 + βdt
dt =p∑i=1φidt−i +
q∑j=1θjεt−j + εt
s(i)t =
ki∑j=1
s(i)j,t
Automatic Forecasting Algorithms TBATS 41
s(i)j,t = s(i)
j,t−1 cosλ(i)j + s∗(i)
j,t−1 sinλ(i)j + γ(i)
1 dts(i)j,t = −s(i)
j,t−1 sinλ(i)j + s∗(i)
j,t−1 cosλ(i)j + γ(i)
2 dt
Box-Cox transformation
TBATS model
yt = observation at time t
y(ω)t =
(yωt − 1)/ω if ω 6= 0;
log yt if ω = 0.
y(ω)t = `t−1 + φbt−1 +
M∑i=1
s(i)t−mi
+ dt
`t = `t−1 + φbt−1 + αdtbt = (1− φ)b + φbt−1 + βdt
dt =p∑i=1φidt−i +
q∑j=1θjεt−j + εt
s(i)t =
ki∑j=1
s(i)j,t
Automatic Forecasting Algorithms TBATS 41
s(i)j,t = s(i)
j,t−1 cosλ(i)j + s∗(i)
j,t−1 sinλ(i)j + γ(i)
1 dts(i)j,t = −s(i)
j,t−1 sinλ(i)j + s∗(i)
j,t−1 cosλ(i)j + γ(i)
2 dt
Box-Cox transformation
M seasonal periods
TBATS model
yt = observation at time t
y(ω)t =
(yωt − 1)/ω if ω 6= 0;
log yt if ω = 0.
y(ω)t = `t−1 + φbt−1 +
M∑i=1
s(i)t−mi
+ dt
`t = `t−1 + φbt−1 + αdtbt = (1− φ)b + φbt−1 + βdt
dt =p∑i=1φidt−i +
q∑j=1θjεt−j + εt
s(i)t =
ki∑j=1
s(i)j,t
Automatic Forecasting Algorithms TBATS 41
s(i)j,t = s(i)
j,t−1 cosλ(i)j + s∗(i)
j,t−1 sinλ(i)j + γ(i)
1 dts(i)j,t = −s(i)
j,t−1 sinλ(i)j + s∗(i)
j,t−1 cosλ(i)j + γ(i)
2 dt
Box-Cox transformation
M seasonal periods
global and local trend
TBATS model
yt = observation at time t
y(ω)t =
(yωt − 1)/ω if ω 6= 0;
log yt if ω = 0.
y(ω)t = `t−1 + φbt−1 +
M∑i=1
s(i)t−mi
+ dt
`t = `t−1 + φbt−1 + αdtbt = (1− φ)b + φbt−1 + βdt
dt =p∑i=1φidt−i +
q∑j=1θjεt−j + εt
s(i)t =
ki∑j=1
s(i)j,t
Automatic Forecasting Algorithms TBATS 41
s(i)j,t = s(i)
j,t−1 cosλ(i)j + s∗(i)
j,t−1 sinλ(i)j + γ(i)
1 dts(i)j,t = −s(i)
j,t−1 sinλ(i)j + s∗(i)
j,t−1 cosλ(i)j + γ(i)
2 dt
Box-Cox transformation
M seasonal periods
global and local trend
ARMA error
TBATS model
yt = observation at time t
y(ω)t =
(yωt − 1)/ω if ω 6= 0;
log yt if ω = 0.
y(ω)t = `t−1 + φbt−1 +
M∑i=1
s(i)t−mi
+ dt
`t = `t−1 + φbt−1 + αdtbt = (1− φ)b + φbt−1 + βdt
dt =p∑i=1φidt−i +
q∑j=1θjεt−j + εt
s(i)t =
ki∑j=1
s(i)j,t
Automatic Forecasting Algorithms TBATS 41
s(i)j,t = s(i)
j,t−1 cosλ(i)j + s∗(i)
j,t−1 sinλ(i)j + γ(i)
1 dts(i)j,t = −s(i)
j,t−1 sinλ(i)j + s∗(i)
j,t−1 cosλ(i)j + γ(i)
2 dt
Box-Cox transformation
M seasonal periods
global and local trend
ARMA error
Fourier-like seasonal terms
TBATS model
yt = observation at time t
y(ω)t =
(yωt − 1)/ω if ω 6= 0;
log yt if ω = 0.
y(ω)t = `t−1 + φbt−1 +
M∑i=1
s(i)t−mi
+ dt
`t = `t−1 + φbt−1 + αdtbt = (1− φ)b + φbt−1 + βdt
dt =p∑i=1φidt−i +
q∑j=1θjεt−j + εt
s(i)t =
ki∑j=1
s(i)j,t
Automatic Forecasting Algorithms TBATS 41
s(i)j,t = s(i)
j,t−1 cosλ(i)j + s∗(i)
j,t−1 sinλ(i)j + γ(i)
1 dts(i)j,t = −s(i)
j,t−1 sinλ(i)j + s∗(i)
j,t−1 cosλ(i)j + γ(i)
2 dt
Box-Cox transformation
M seasonal periods
global and local trend
ARMA error
Fourier-like seasonal terms
TBATSTrigonometricBox-CoxARMATrendSeasonal
TBATS modelTBATSTrigonometric terms for seasonalityBox-Cox transformations for heterogeneityARMA errors for short-term dynamicsTrend (possibly damped)Seasonal (including multiple and non-integer periods)
Handles non-integer seasonality, multiple seasonalperiods.Entirely automatedPrediction intervals often too wideVery slow on long series
Automatic Forecasting Algorithms TBATS 42
Outline
1 Motivation
2 ETS
3 ARIMA models
4 STLM
5 TBATS
6 FASSTER
7 Comparisons
Automatic Forecasting Algorithms FASSTER 43
Pedestrian counts
Automatic Forecasting Algorithms FASSTER 44
Pedestrian counts
0
1000
2000
3000
4000
Jul 2016 Oct 2016 Jan 2017 Apr 2017
Date
Ped
estr
ians
cou
nted
Hourly pedestrian traffic at Southern Cross Station
0
1000
2000
3000
4000
Apr 01 Apr 15 May 01 May 15 Jun 01
Date
Ped
estr
ians
cou
nted
Hourly pedestrian traffic at Southern Cross Station
Automatic Forecasting Algorithms FASSTER 45
Pedestrian counts
Weekday Weekend
00 A
M
06 A
M
12 P
M
18 P
M
00 A
M
00 A
M
06 A
M
12 P
M
18 P
M
00 A
M
0
1000
2000
3000
4000
Time
Tota
l ped
estr
ians
cou
nted
Seasonality in pedestrian traffic at Southern Cross Station
Automatic Forecasting Algorithms FASSTER 46
Switching StructureFASSTER extends current state-space approaches byswitching between states.Dynamic linear model
yt = Ftθt + vt, vt ∼ N (0,V)θt = Gθt−1 + wt, wt ∼ N (0,W)
Switch between two groups:
yt = It∈G1Ftθ(1)t + It∈G2Ftθ
(2)t + vt vt ∼ N (0,V)
Groups G1 and G2 define the switching rule (say weekdaysand weekends).
Automatic Forecasting Algorithms FASSTER 47
Switching StructureFASSTER extends current state-space approaches byswitching between states.Dynamic linear model
yt = Ftθt + vt, vt ∼ N (0,V)θt = Gθt−1 + wt, wt ∼ N (0,W)
Switch between two groups:
yt = It∈G1Ftθ(1)t + It∈G2Ftθ
(2)t + vt vt ∼ N (0,V)
Groups G1 and G2 define the switching rule (say weekdaysand weekends).
Automatic Forecasting Algorithms FASSTER 47
Switching StructureFASSTER extends current state-space approaches byswitching between states.Dynamic linear model
yt = Ftθt + vt, vt ∼ N (0,V)θt = Gθt−1 + wt, wt ∼ N (0,W)
Switch between two groups:
yt = It∈G1Ftθ(1)t + It∈G2Ftθ
(2)t + vt vt ∼ N (0,V)
Groups G1 and G2 define the switching rule (say weekdaysand weekends).
Automatic Forecasting Algorithms FASSTER 47
Application to pedestrian trafficThis series contains a switching daily pattern overweekdays and weekends. Each group can be modelledusing level and hourly seasonal states.
0
1000
2000
3000
4000
Jan 30 Feb 06 Feb 13 Feb 20
Date
Ped
estr
ians
cou
nted
Pedestrian Traffic at Southern Cross Station
Automatic Forecasting Algorithms FASSTER 48
Application to pedestrian traffic
An appropriate model can be constructed using switchingstates:
yt = It∈WeekdayFtθ(Weekday)t + It∈WeekendFtθ(Weekend)
t + vt
where Ftθ(i)t = `(i)
t + f(i)t and vt ∼ N (0,V)
`t is a level componentft is a seasonal component based on Fourier terms
Automatic Forecasting Algorithms FASSTER 49
FASSTER allows flexible use of:
seasonal factorsfourier seasonal termspolynomial trendsBoxCox transformationsexogenous regressorsARMA processesstate switching
General measurement equation
yt = F(0)t θ
(0)t +
k∑j=1
It∈GjF(j)t θ
(j)t + vt, vt ∼ N (0,V)
where k is the number of switching combinations.Automatic Forecasting Algorithms FASSTER 50
FASSTER allows flexible use of:
seasonal factorsfourier seasonal termspolynomial trendsBoxCox transformationsexogenous regressorsARMA processesstate switching
General measurement equation
yt = F(0)t θ
(0)t +
k∑j=1
It∈GjF(j)t θ
(j)t + vt, vt ∼ N (0,V)
where k is the number of switching combinations.Automatic Forecasting Algorithms FASSTER 50
FASSTER and Dynamic Linear ModelsA FASSTER model can be represented as a time-varyingDLM.
yt = Ftθt + vt, vt ∼ N (0,V)θt = Gθt−1 + wt, wt ∼ N (0,W)
where θ0 ∼ N (m0, C0)
There are too many parameters to easily estimate.E.g., pedestrian model has 48 states:2 groups× (1 level + 23 fourier states).We use a “heuristic estimation” approach involvingonly two passes through the data.
Automatic Forecasting Algorithms FASSTER 51
FASSTER and Dynamic Linear ModelsA FASSTER model can be represented as a time-varyingDLM.
yt = Ftθt + vt, vt ∼ N (0,V)θt = Gθt−1 + wt, wt ∼ N (0,W)
where θ0 ∼ N (m0, C0)
There are too many parameters to easily estimate.E.g., pedestrian model has 48 states:2 groups× (1 level + 23 fourier states).We use a “heuristic estimation” approach involvingonly two passes through the data.
Automatic Forecasting Algorithms FASSTER 51
Usage (Pedestrian Counts)SthCross_fasster_fit <- SthCross_Ex %>%
fasster(Hourly_Counts ~ DayType %S% (poly(1) + trig(24)))
0
1000
2000
3000
4000
0 100 200 300 400 500
Date
Ped
estr
ians
cou
nted
Series
Response
Fitted
Pedestrian Traffic at Southern Cross Station
Automatic Forecasting Algorithms FASSTER 52
Forecasts (Pedestrian counts)
Automatic Forecasting Algorithms FASSTER 53
Outline
1 Motivation
2 ETS
3 ARIMA models
4 STLM
5 TBATS
6 FASSTER
7 Comparisons
Automatic Forecasting Algorithms Comparisons 54
Pharmaceutical sales
0.50
0.75
1.00
1.25
1995 2000 2005 2010
Time
h02
Forecast
ARIMA
ETS
FASSTER
STLF
TBATS
Automatic Forecasting Algorithms Comparisons 55
R packages
Automatic Forecasting Algorithms Comparisons 56
https://github.com/earowang/tsibble
http://pkg.earo.me/sugrrants
https://github.com/mitchelloharawild/fasster
http://pkg.robjhyndman.com/forecast
http://pkg.earo.me/hts
Textbook
Automatic Forecasting Algorithms Comparisons 57
OTexts.org/fpp2