automatic algorithms for time series forecasting

Rob J Hyndman

Automatic algorithmsfor time seriesforecasting

Outline

1 Motivation

2 Forecasting competitions

3 Exponential smoothing

4 ARIMA modelling

5 Automatic nonlinear forecasting?

6 Time series with complex seasonality

7 Hierarchical and grouped time series

8 Recent developments

Automatic algorithms for time series forecasting Motivation 2

Motivation

1 Common in business to have over 1000products that need forecasting at least monthly.

2 Forecasts are often required by people who areuntrained in time series analysis.

Specifications

Automatic forecasting algorithms must:

å determine an appropriate time series model;

å estimate the parameters;

å compute the forecasts with prediction intervals.

Motivation

1 Common in business to have over 1000products that need forecasting at least monthly.

2 Forecasts are often required by people who areuntrained in time series analysis.

Specifications

Automatic forecasting algorithms must:

å determine an appropriate time series model;

å estimate the parameters;

å compute the forecasts with prediction intervals.

Example: Asian sheep

Numbers of sheep in Asia

1960 1970 1980 1990 2000 2010

Example: Asian sheep

Automatic ETS forecasts

1960 1970 1980 1990 2000 2010

Example: Cortecosteroid sales

Monthly cortecosteroid drug sales in Australia

1995 2000 2005 2010

Example: Cortecosteroid sales

Forecasts from ARIMA(3,1,3)(0,1,1)[12]

1995 2000 2005 2010

Outline

1 Motivation

4 ARIMA modelling

Automatic algorithms for time series forecasting Forecasting competitions 7

Makridakis and Hibon (1979)

This was the first large-scale empirical evaluation oftime series forecasting methods.

Highly controversial at the time.

Difficulties:8 How to measure forecast accuracy?8 How to apply methods consistently and objectively?8 How to explain unexpected results?

Common thinking was that the moresophisticated mathematical models (ARIMAmodels at the time) were necessarily better.If results showed ARIMA models not best, itmust be because analyst was unskilled.

It is amazing to me, however, that after all thisexercise in identifying models, transforming and soon, that the autoregressive moving averages comeout so badly. I wonder whether it might be partlydue to the authors not using the backwardsforecasting approach to obtain the initial errors.

— W.G. Gilchrist

I find it hard to believe that Box-Jenkins, if properlyapplied, can actually be worse than so many of thesimple methods . . . these authors are more athome with simple procedures than withBox-Jenkins. — C. ChatfieldAutomatic algorithms for time series forecasting Forecasting competitions 10

It is amazing to me, however, that after all thisexercise in identifying models, transforming and soon, that the autoregressive moving averages comeout so badly. I wonder whether it might be partlydue to the authors not using the backwardsforecasting approach to obtain the initial errors.

— W.G. Gilchrist

I find it hard to believe that Box-Jenkins, if properlyapplied, can actually be worse than so many of thesimple methods . . . these authors are more athome with simple procedures than withBox-Jenkins. — C. ChatfieldAutomatic algorithms for time series forecasting Forecasting competitions 10

Consequences of M&H (1979)

As a result of this paper, researchers started to:

å consider how to automate forecasting methods;

å study what methods give the best forecasts;

å be aware of the dangers of over-fitting;

å treat forecasting as a different problem fromtime series analysis.

Makridakis & Hibon followed up with a newcompetition in 1982:

1001 seriesAnyone could submit forecasts (avoiding thecharge of incompetence)Multiple forecast measures used.

Consequences of M&H (1979)

As a result of this paper, researchers started to:

å consider how to automate forecasting methods;

å study what methods give the best forecasts;

å be aware of the dangers of over-fitting;

å treat forecasting as a different problem fromtime series analysis.

Makridakis & Hibon followed up with a newcompetition in 1982:

1001 seriesAnyone could submit forecasts (avoiding thecharge of incompetence)Multiple forecast measures used.

M-competition

Main findings (taken from Makridakis & Hibon, 2000)

1 Statistically sophisticated or complex methods donot necessarily provide more accurate forecaststhan simpler ones.

2 The relative ranking of the performance of thevarious methods varies according to the accuracymeasure being used.

3 The accuracy when various methods are beingcombined outperforms, on average, the individualmethods being combined and does very well incomparison to other methods.

4 The accuracy of the various methods depends uponthe length of the forecasting horizon involved.

M3 competition

“The M3-Competition is a final attempt by the authors tosettle the accuracy issue of various time series methods. . .The extension involves the inclusion of more methods/researchers (in particular in the areas of neural networksand expert systems) and more series.”

3003 seriesAll data from business, demography, finance andeconomics.Series length between 14 and 126.Either non-seasonal, monthly or quarterly.All time series positive.M&H claimed that the M3-competition supported thefindings of their earlier work.However, best performing methods far from “simple”.

Makridakis and Hibon (2000)Best methods:

A very confusing explanation.

Shown by Hyndman and Billah (2003) to be average oflinear regression and simple exponential smoothingwith drift, applied to seasonally adjusted data.

Later, the original authors claimed that theirexplanation was incorrect.

Forecast Pro

A commercial software package with an unknownalgorithm.

Known to fit either exponential smoothing or ARIMAmodels using BIC.

M3 results (recalculated)

Method MAPE sMAPE MASE

Theta 17.42 12.76 1.39

ForecastPro 18.00 13.06 1.47

ForecastX 17.35 13.09 1.42

Automatic ANN 17.18 13.98 1.53

B-J automatic 19.13 13.72 1.54

M3 results (recalculated)

Theta 17.42 12.76 1.39

ForecastPro 18.00 13.06 1.47

ForecastX 17.35 13.09 1.42

B-J automatic 19.13 13.72 1.54

ä Calculations do not match

published paper.

ä Some contestants apparently

submitted multiple entries but only

best ones published.

Outline

1 Motivation

4 ARIMA modelling

Automatic algorithms for time series forecasting Exponential smoothing 18

Exponential smoothing methods

Seasonal ComponentTrend N A M

Component (None) (Additive) (Multiplicative)

N (None) N,N N,A N,M

A (Additive) A,N A,A A,M

Ad (Additive damped) Ad,N Ad,A Ad,M

M (Multiplicative) M,N M,A M,M

Md (Multiplicative damped) Md,N Md,A Md,M

N,N: Simple exponential smoothing

N,N: Simple exponential smoothingA,N: Holt’s linear method

N,N: Simple exponential smoothingA,N: Holt’s linear methodAd,N: Additive damped trend method

N,N: Simple exponential smoothingA,N: Holt’s linear methodAd,N: Additive damped trend methodM,N: Exponential trend method

N,N: Simple exponential smoothingA,N: Holt’s linear methodAd,N: Additive damped trend methodM,N: Exponential trend methodMd,N: Multiplicative damped trend method

N,N: Simple exponential smoothingA,N: Holt’s linear methodAd,N: Additive damped trend methodM,N: Exponential trend methodMd,N: Multiplicative damped trend methodA,A: Additive Holt-Winters’ method

N,N: Simple exponential smoothingA,N: Holt’s linear methodAd,N: Additive damped trend methodM,N: Exponential trend methodMd,N: Multiplicative damped trend methodA,A: Additive Holt-Winters’ methodA,M: Multiplicative Holt-Winters’ method

There are 15 separate exp. smoothing methods.

There are 15 separate exp. smoothing methods.Each can have an additive or multiplicative error,giving 30 separate models.

There are 15 separate exp. smoothing methods.Each can have an additive or multiplicative error,giving 30 separate models.Only 19 models are numerically stable.

There are 15 separate exp. smoothing methods.Each can have an additive or multiplicative error,giving 30 separate models.Only 19 models are numerically stable.Multiplicative trend models give poor forecastsleaving 15 models.

General notation E T S : ExponenTial Smoothing

Examples:A,N,N: Simple exponential smoothing with additive errorsA,A,N: Holt’s linear method with additive errorsM,A,M: Multiplicative Holt-Winters’ method with multiplicative errors

General notation E T S : ExponenTial Smoothing

Examples:A,N,N: Simple exponential smoothing with additive errorsA,A,N: Holt’s linear method with additive errorsM,A,M: Multiplicative Holt-Winters’ method with multiplicative errors

General notation E T S : ExponenTial Smoothing↑

TrendExamples:

A,N,N: Simple exponential smoothing with additive errorsA,A,N: Holt’s linear method with additive errorsM,A,M: Multiplicative Holt-Winters’ method with multiplicative errors

General notation E T S : ExponenTial Smoothing↑ ↖

Trend SeasonalExamples:

General notation E T S : ExponenTial Smoothing↗ ↑ ↖

Error Trend SeasonalExamples:

Innovations state space models

å All ETS models can be written in innovationsstate space form (IJF, 2002).

å Additive and multiplicative versions give thesame point forecasts but different predictionintervals.

ETS state space model

xt−1

State space modelxt = (level, slope, seasonal)

xt−1

xt yt+1

xt−1

xt yt+1

εt+1 xt+1

xt−1

xt yt+1

εt+1 xt+1 yt+2

xt−1

xt yt+1

εt+1 xt+1 yt+2

εt+2 xt+2

xt−1

xt yt+1

εt+1 xt+1 yt+2

εt+2 xt+2 yt+3

xt−1

xt yt+1

εt+1 xt+1 yt+2

εt+2 xt+2 yt+3

εt+3 xt+3

xt−1

xt yt+1

εt+1 xt+1 yt+2

εt+2 xt+2 yt+3

εt+3 xt+3 yt+4

xt−1

xt yt+1

εt+1 xt+1 yt+2

εt+2 xt+2 yt+3

εt+3 xt+3 yt+4

EstimationCompute likelihood L fromε1, ε2, . . . , εT.Optimize L wrt modelparameters.

Let xt = (`t,bt, st, st−1, . . . , st−m+1) and εtiid∼ N(0, σ2).

yt = h(xt−1)︸︷︷︸+ k(xt−1)εt︸︷︷︸ Observation equation

µt et

xt = f(xt−1) + g(xt−1)εt State equation

Additive errors:k(xt−1) = 1. yt = µt + εt.

Multiplicative errors:k(xt−1) = µt. yt = µt(1 + εt).

εt = (yt − µt)/µt is relative error.

All models can be written in state space form.

Additive and multiplicative versions give samepoint forecasts but different predictionintervals.

Estimation

L∗(θ,x0) = n log

( n∑t=1

ε2t /k

2(xt−1)

n∑t=1

log |k(xt−1)|

= −2 log(Likelihood) + constant

Minimize wrt θ = (α, β, γ, φ) and initial statesx0 = (`0,b0, s0, s−1, . . . , s−m+1).

Estimation

L∗(θ,x0) = n log

( n∑t=1

ε2t /k

2(xt−1)

n∑t=1

log |k(xt−1)|

Estimation

L∗(θ,x0) = n log

( n∑t=1

ε2t /k

2(xt−1)

n∑t=1

log |k(xt−1)|

Estimation

L∗(θ,x0) = n log

( n∑t=1

ε2t /k

2(xt−1)

n∑t=1

log |k(xt−1)|

Estimation

L∗(θ,x0) = n log

( n∑t=1

ε2t /k

2(xt−1)

n∑t=1

log |k(xt−1)|

Estimation

L∗(θ,x0) = n log

( n∑t=1

ε2t /k

2(xt−1)

n∑t=1

log |k(xt−1)|

Q: How to choosebetween the 15 usefulETS models?

Cross-validationTraditional evaluation

● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● timeTraining data Test data

Standard cross-validation

● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ●● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ●● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ●●● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●●● ● ●

● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ●● ●

Time series cross-validation

● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ●● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ●● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ●●● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●●● ● ●

● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ●● ●

● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●

● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ●● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ●● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ●●● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●●● ● ●

● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ●● ●

● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●

● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ●● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ●● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ●●● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●●● ● ●

● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ●● ●

● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●

Also known as “Evaluation ona rolling forecast origin”

Akaike’s Information Criterion

AIC = −2 log(L) + 2k

where L is the likelihood and k is the number ofestimated parameters in the model.

This is a penalized likelihood approach.If L is Gaussian, then AIC ≈ c + T log MSE + 2kwhere c is a constant, MSE is from one-stepforecasts on training set, and T is the length ofthe series.

Minimizing the Gaussian AIC is asymptoticallyequivalent (as T →∞) to minimizing MSE fromone-step forecasts on test set via time seriescross-validation.Automatic algorithms for time series forecasting Exponential smoothing 25

Corrected AICFor small T, AIC tends to over-fit. Bias-correctedversion:

AICC = AIC + 2(k+1)(k+2)T−k

Bayesian Information Criterion

BIC = AIC + k[log(T)− 2]

BIC penalizes terms more heavily than AICMinimizing BIC is consistent if there is a truemodel.

What to use?

Choice: AIC, AICc, BIC, CV-MSE

CV-MSE too time consuming for most automaticforecasting purposes. Also requires large T.

As T →∞, BIC selects true model if there isone. But that is never true!

AICc focuses on forecasting performance, canbe used on small samples and is very fast tocompute.

Empirical studies in forecasting show AIC isbetter than BIC for forecast accuracy.

What to use?

ets algorithm in R

Based on Hyndman, Koehler,Snyder & Grose (IJF 2002):

Apply each of 15 models that areappropriate to the data. Optimizeparameters and initial valuesusing MLE.

Select best method using AICc.

Produce forecasts using bestmethod.

Obtain prediction intervals usingunderlying state space model.

ets algorithm in R

Exponential smoothing

Forecasts from ETS(M,A,N)

1960 1970 1980 1990 2000 2010

fit <- ets(livestock)fcast <- forecast(fit)plot(fcast)

Forecasts from ETS(M,A,N)

1960 1970 1980 1990 2000 2010

Forecasts from ETS(M,N,M)

1995 2000 2005 2010

fit <- ets(h02)fcast <- forecast(fit)plot(fcast)

Forecasts from ETS(M,N,M)

1995 2000 2005 2010

> fitETS(M,N,M)

Smoothing parameters:alpha = 0.4597gamma = 1e-04

Initial states:l = 0.4501s = 0.8628 0.8193 0.7648 0.7675 0.6946 1.2921

1.3327 1.1833 1.1617 1.0899 1.0377 0.9937

sigma: 0.0675

AIC AICc BIC-115.69960 -113.47738 -69.24592

M3 comparisons

Theta 17.42 12.76 1.39

ForecastPro 18.00 13.06 1.47

ForecastX 17.35 13.09 1.42

B-J automatic 19.13 13.72 1.54

ETS 17.38 13.13 1.43

www.OTexts.org/fpp

Outline

1 Motivation

4 ARIMA modelling

Automatic algorithms for time series forecasting ARIMA modelling 36

ARIMA models

yt−1

yt−2

yt−3

Inputs Output

ARIMA models

yt−1

yt−2

yt−3

Inputs Output

Autoregression (AR)model

ARIMA models

yt−1

yt−2

yt−3

εt−1

εt−2

Inputs Output

Autoregression movingaverage (ARMA) model

ARIMA models

yt−1

yt−2

yt−3

εt−1

εt−2

Inputs Output

EstimationCompute likelihood L fromε1, ε2, . . . , εT.Use optimizationalgorithm to maximize L.

ARIMA models

yt−1

yt−2

yt−3

εt−1

εt−2

Inputs Output

EstimationCompute likelihood L fromε1, ε2, . . . , εT.Use optimizationalgorithm to maximize L.

ARIMA modelAutoregression movingaverage (ARMA) modelapplied to differences.

ARIMA modelling

Auto ARIMA

Forecasts from ARIMA(0,1,0) with drift

1960 1970 1980 1990 2000 2010

Auto ARIMA

fit <- auto.arima(livestock)fcast <- forecast(fit)plot(fcast)

Forecasts from ARIMA(0,1,0) with drift

1960 1970 1980 1990 2000 2010

Auto ARIMA

1995 2000 2005 2010

Auto ARIMA

fit <- auto.arima(h02)fcast <- forecast(fit)plot(fcast)

1995 2000 2005 2010

Auto ARIMA

> fitSeries: h02ARIMA(3,1,3)(0,1,1)[12]

Coefficients:ar1 ar2 ar3 ma1 ma2 ma3 sma1

-0.3648 -0.0636 0.3568 -0.4850 0.0479 -0.353 -0.5931s.e. 0.2198 0.3293 0.1268 0.2227 0.2755 0.212 0.0651

sigma^2 estimated as 0.002706: log likelihood=290.25AIC=-564.5 AICc=-563.71 BIC=-538.48

How does auto.arima() work?

A non-seasonal ARIMA process

φ(B)(1− B)dyt = c + θ(B)εt

Need to select appropriate orders p,q,d, andwhether to include c.

Algorithm choices driven by forecast accuracy.

Hyndman & Khandakar (JSS, 2008) algorithm:Select no. differences d via KPSS unit root test.Select p,q, c by minimising AICc.Use stepwise search to traverse model space,starting with a simple model and consideringnearby variants.

A seasonal ARIMA process

Φ(Bm)φ(B)(1− B)d(1− Bm)Dyt = c + Θ(Bm)θ(B)εt

Need to select appropriate orders p,q,d, P,Q,D, andwhether to include c.

Hyndman & Khandakar (JSS, 2008) algorithm:Select no. differences d via KPSS unit root test.Select D using OCSB unit root test.Select p,q, P,Q, c by minimising AICc.Use stepwise search to traverse model space,starting with a simple model and consideringnearby variants.

M3 comparisons

Theta 17.42 12.76 1.39

ForecastPro 18.00 13.06 1.47

B-J automatic 19.13 13.72 1.54

ETS 17.38 13.13 1.43

AutoARIMA 19.12 13.85 1.47

Outline

1 Motivation

4 ARIMA modelling

Automatic algorithms for time series forecasting Automatic nonlinear forecasting? 47

Automatic nonlinear forecasting

Automatic ANN in M3 competition did poorly.

Linear methods did best in the NN3competition!

Very few machine learning methods getpublished in the IJF because authors cannotdemonstrate their methods give betterforecasts than linear benchmark methods,even on supposedly nonlinear data.

Some good recent work by Kourentzes andCrone on automated ANN for time series.

Watch this space!

Outline

1 Motivation

4 ARIMA modelling

Automatic algorithms for time series forecasting Time series with complex seasonality 49

Examples

US finished motor gasoline products

1992 1994 1996 1998 2000 2002 2004

Examples

Number of calls to large American bank (7am−9pm)

5 minute intervals

3 March 17 March 31 March 14 April 28 April 12 May

Examples

Turkish electricity demand

2000 2002 2004 2006 2008

TBATS model

TBATSTrigonometric terms for seasonality

Box-Cox transformations for heterogeneity

ARMA errors for short-term dynamics

Trend (possibly damped)

Seasonal (including multiple and non-integer periods)

Automatic algorithm described in AM De Livera,RJ Hyndman, and RD Snyder (2011). “Forecastingtime series with complex seasonal patterns usingexponential smoothing”. Journal of the AmericanStatistical Association 106(496), 1513–1527.

TBATS model

yt = observation at time t

y(ω)t =

t − 1)/ω if ω 6= 0;

log yt if ω = 0.

y(ω)t = `t−1 + φbt−1 +

M∑i=1

s(i)t−mi+ dt

`t = `t−1 + φbt−1 + αdt

bt = (1− φ)b + φbt−1 + βdt

p∑i=1

φidt−i +

q∑j=1

θjεt−j + εt

s(i)t =

ki∑j=1

s(i)j,t

s(i)j,t = s(i)j,t−1 cosλ(i)j + s∗(i)j,t−1 sinλ(i)j + γ(i)1 dt

s(i)j,t = −s(i)j,t−1 sinλ(i)j + s∗(i)j,t−1 cosλ(i)j + γ(i)2 dt

TBATS model

y(ω)t =

t − 1)/ω if ω 6= 0;

log yt if ω = 0.

y(ω)t = `t−1 + φbt−1 +

M∑i=1

s(i)t−mi+ dt

`t = `t−1 + φbt−1 + αdt

bt = (1− φ)b + φbt−1 + βdt

p∑i=1

φidt−i +

q∑j=1

θjεt−j + εt

s(i)t =

ki∑j=1

s(i)j,t

Box-Cox transformation

TBATS model

y(ω)t =

t − 1)/ω if ω 6= 0;

log yt if ω = 0.

y(ω)t = `t−1 + φbt−1 +

M∑i=1

s(i)t−mi+ dt

`t = `t−1 + φbt−1 + αdt

bt = (1− φ)b + φbt−1 + βdt

p∑i=1

φidt−i +

q∑j=1

θjεt−j + εt

s(i)t =

ki∑j=1

s(i)j,t

M seasonal periods

TBATS model

y(ω)t =

t − 1)/ω if ω 6= 0;

log yt if ω = 0.

y(ω)t = `t−1 + φbt−1 +

M∑i=1

s(i)t−mi+ dt

`t = `t−1 + φbt−1 + αdt

bt = (1− φ)b + φbt−1 + βdt

p∑i=1

φidt−i +

q∑j=1

θjεt−j + εt

s(i)t =

ki∑j=1

s(i)j,t

M seasonal periods

global and local trend

TBATS model

y(ω)t =

t − 1)/ω if ω 6= 0;

log yt if ω = 0.

y(ω)t = `t−1 + φbt−1 +

M∑i=1

s(i)t−mi+ dt

`t = `t−1 + φbt−1 + αdt

bt = (1− φ)b + φbt−1 + βdt

p∑i=1

φidt−i +

q∑j=1

θjεt−j + εt

s(i)t =

ki∑j=1

s(i)j,t

M seasonal periods

ARMA error

TBATS model

y(ω)t =

t − 1)/ω if ω 6= 0;

log yt if ω = 0.

y(ω)t = `t−1 + φbt−1 +

M∑i=1

s(i)t−mi+ dt

`t = `t−1 + φbt−1 + αdt

bt = (1− φ)b + φbt−1 + βdt

p∑i=1

φidt−i +

q∑j=1

θjεt−j + εt

s(i)t =

ki∑j=1

s(i)j,t

M seasonal periods

ARMA error

Fourier-like seasonalterms

TBATS model

y(ω)t =

t − 1)/ω if ω 6= 0;

log yt if ω = 0.

y(ω)t = `t−1 + φbt−1 +

M∑i=1

s(i)t−mi+ dt

`t = `t−1 + φbt−1 + αdt

bt = (1− φ)b + φbt−1 + βdt

p∑i=1

φidt−i +

q∑j=1

θjεt−j + εt

s(i)t =

ki∑j=1

s(i)j,t

M seasonal periods

ARMA error

Fourier-like seasonalterms

TBATSTrigonometric

Box-Cox

Seasonal

Examples

fit <- tbats(gasoline)fcast <- forecast(fit)plot(fcast)

Forecasts from TBATS(0.999, {2,2}, 1, {<52.1785714285714,8>})

1995 2000 2005

Examples

fit <- tbats(callcentre)fcast <- forecast(fit)plot(fcast)

Forecasts from TBATS(1, {3,1}, 0.987, {<169,5>, <845,3>})

5 minute intervals

3 March 17 March 31 March 14 April 28 April 12 May 26 May 9 June

Examples

fit <- tbats(turk)fcast <- forecast(fit)plot(fcast)

Forecasts from TBATS(0, {5,3}, 0.997, {<7,3>, <354.37,12>, <365.25,4>})

2000 2002 2004 2006 2008 2010

Outline

1 Motivation

4 ARIMA modelling

Automatic algorithms for time series forecasting Hierarchical and grouped time series 56

Hierarchical time seriesA hierarchical time series is a collection ofseveral time series that are linked together in ahierarchical structure.

AA AB AC

BA BB BC

CA CB CC

ExamplesNet labour turnoverTourism by state and region

AA AB AC

BA BB BC

CA CB CC

AA AB AC

BA BB BC

CA CB CC

Hierarchical time series

Yt : observed aggregate of allseries at time t.

YX,t : observation on series X attime t.

bt : vector of all series atbottom level in time t.

yt = [Yt, YA,t, YB,t, YC,t]′ =

1 1 11 0 00 1 00 0 1

︸︷︷︸

1 1 11 0 00 1 00 0 1

︸︷︷︸

1 1 11 0 00 1 00 0 1

︸︷︷︸

btyt = Sbt

Hierarchical time seriesTotal

AX AY AZ

BX BY BZ

CX CY CZ

1 1 1 1 1 1 1 1 11 1 1 0 0 0 0 0 00 0 0 1 1 1 0 0 00 0 0 0 0 0 1 1 11 0 0 0 0 0 0 0 00 1 0 0 0 0 0 0 00 0 1 0 0 0 0 0 00 0 0 1 0 0 0 0 00 0 0 0 1 0 0 0 00 0 0 0 0 1 0 0 00 0 0 0 0 0 1 0 00 0 0 0 0 0 0 1 00 0 0 0 0 0 0 0 1

︸︷︷︸

AX AY AZ

BX BY BZ

CX CY CZ

1 1 1 1 1 1 1 1 11 1 1 0 0 0 0 0 00 0 0 1 1 1 0 0 00 0 0 0 0 0 1 1 11 0 0 0 0 0 0 0 00 1 0 0 0 0 0 0 00 0 1 0 0 0 0 0 00 0 0 1 0 0 0 0 00 0 0 0 1 0 0 0 00 0 0 0 0 1 0 0 00 0 0 0 0 0 1 0 00 0 0 0 0 0 0 1 00 0 0 0 0 0 0 0 1

︸︷︷︸

AX AY AZ

BX BY BZ

CX CY CZ

1 1 1 1 1 1 1 1 11 1 1 0 0 0 0 0 00 0 0 1 1 1 0 0 00 0 0 0 0 0 1 1 11 0 0 0 0 0 0 0 00 1 0 0 0 0 0 0 00 0 1 0 0 0 0 0 00 0 0 1 0 0 0 0 00 0 0 0 1 0 0 0 00 0 0 0 0 1 0 0 00 0 0 0 0 0 1 0 00 0 0 0 0 0 0 1 00 0 0 0 0 0 0 0 1

︸︷︷︸

yt = Sbt

Forecasting notation

Let yn(h) be vector of initial h-step forecasts, madeat time n, stacked in same order as yt. (They maynot add up.)

Reconciled forecasts are of the form:yn(h) = SPyn(h)

for some matrix P.

P extracts and combines base forecasts yn(h)to get bottom-level forecasts.

S adds them up

for some matrix P.

S adds them up

for some matrix P.

S adds them up

for some matrix P.

S adds them up

for some matrix P.

S adds them up

General properties

yn(h) = SPyn(h)

Forecast biasAssuming the base forecasts yn(h) are unbiased,then the revised forecasts are unbiased iff SPS = S.

Forecast varianceFor any given P satisfying SPS = S, the covariancematrix of the h-step ahead reconciled forecasterrors is given by

Var[yn+h − yn(h)] = SPWhP′S′

where Wh is the covariance matrix of the h-stepahead base forecast errors.Automatic algorithms for time series forecasting Hierarchical and grouped time series 61

General properties

yn(h) = SPyn(h)

General properties

yn(h) = SPyn(h)

BLUF via trace minimizationTheoremFor any P satisfying SPS = S, then

= trace[SPWhP′S′]

has solution P = (S′W†hS)−1S′W†

W†h is generalized inverse of Wh.

yn(h) = S(S′W†hS)−1S′W†

hyn(h)

Revised forecasts Base forecasts

Equivalent to GLS estimate of regressionyn(h) = Sβn(h) + εh where ε ∼ N(0,Wh).

Problem: Wh hard to estimate.Automatic algorithms for time series forecasting Hierarchical and grouped time series 62

hyn(h)

Optimal combination forecasts

Solution 1: OLS

yn(h) = S(S′S)−1S′yn(h)

Solution 2: WLSApproximate W1 by its diagonal.Assume Wh = khW1.Easy to estimate, and places weight where wehave best one-step forecasts.

yn(h) = S(S′ΛS)−1S′Λyn(h)

hyn(h)

Solution 1: OLS

hyn(h)

Solution 1: OLS

hyn(h)

Solution 1: OLS

hyn(h)

Solution 1: OLS

hyn(h)

Solution 1: OLS

hyn(h)

Solution 1: OLS

hyn(h)

Challenges

Computational difficulties in big hierarchies dueto size of the S matrix and singular behavior of(S′ΛS).

Loss of information in ignoring covariancematrix in computing point forecasts.

Still need to estimate covariance matrix toproduce prediction intervals.

Challenges

Australian tourism

Hierarchy:

States (7)

Zones (27)

Regions (82)

Australian tourism

Hierarchy:

States (7)

Zones (27)

Regions (82)

Base forecastsETS (exponential smoothing)models

Base forecasts

Domestic tourism forecasts: Total

1998 2000 2002 2004 2006 2008

Base forecasts

Domestic tourism forecasts: NSW

1998 2000 2002 2004 2006 2008

Base forecasts

Domestic tourism forecasts: VIC

1998 2000 2002 2004 2006 2008

Base forecasts

Domestic tourism forecasts: Nth.Coast.NSW

1998 2000 2002 2004 2006 2008

Base forecasts

Domestic tourism forecasts: Metro.QLD

1998 2000 2002 2004 2006 2008

Base forecasts

Domestic tourism forecasts: Sth.WA

1998 2000 2002 2004 2006 2008

Base forecasts

Domestic tourism forecasts: X201.Melbourne

1998 2000 2002 2004 2006 2008

Base forecasts

Domestic tourism forecasts: X402.Murraylands

1998 2000 2002 2004 2006 2008

Base forecasts

Domestic tourism forecasts: X809.Daly

1998 2000 2002 2004 2006 2008

Reconciled forecasts

2000 2005 2010

2000 2005 20101000

2000 2005 2010

er2000 2005 201018

2000 2005 20104000

2000 2005 2010

LD2000 2005 201060

ital c

2000 2005 2010

Forecast evaluation

Select models using all observations;

Re-estimate models using first 12 observationsand generate 1- to 8-step-ahead forecasts;

Increase sample size one observation at a time,re-estimate models, generate forecasts untilthe end of the sample;

In total 24 1-step-ahead, 23 2-steps-ahead, upto 17 8-steps-ahead for forecast evaluation.

Forecast evaluation

Hierarchy: states, zones, regions

MAPE h = 1 h = 2 h = 4 h = 6 h = 8 AverageTop Level: Australia

Bottom-up 3.79 3.58 4.01 4.55 4.24 4.06OLS 3.83 3.66 3.88 4.19 4.25 3.94WLS 3.68 3.56 3.97 4.57 4.25 4.04Level: States

Bottom-up 10.70 10.52 10.85 11.46 11.27 11.03OLS 11.07 10.58 11.13 11.62 12.21 11.35WLS 10.44 10.17 10.47 10.97 10.98 10.67Level: Zones

Bottom-up 14.99 14.97 14.98 15.69 15.65 15.32OLS 15.16 15.06 15.27 15.74 16.15 15.48WLS 14.63 14.62 14.68 15.17 15.25 14.94Bottom Level: Regions

Bottom-up 33.12 32.54 32.26 33.74 33.96 33.18OLS 35.89 33.86 34.26 36.06 37.49 35.43WLS 31.68 31.22 31.08 32.41 32.77 31.89

hts package for R

hts: Hierarchical and grouped time seriesMethods for analysing and forecasting hierarchical and groupedtime series

Version: 4.5Depends: forecast (≥ 5.0), SparseMImports: parallel, utilsPublished: 2014-12-09Author: Rob J Hyndman, Earo Wang and Alan LeeMaintainer: Rob J Hyndman <Rob.Hyndman at monash.edu>BugReports: https://github.com/robjhyndman/hts/issuesLicense: GPL (≥ 2)

Outline

1 Motivation

4 ARIMA modelling

Automatic algorithms for time series forecasting Recent developments 71

Further competitions

1 2011 tourism forecasting competition.

2 Kaggle and other forecasting platforms.

3 GEFCom 2012: Point forecasting of

electricity load and wind power.

4 GEFCom 2014: Probabilistic forecasting

of electricity load, electricity price,

wind energy and solar energy.

Forecasts about forecasting

1 Automatic algorithms will become moregeneral — handling a wide variety of timeseries.

2 Model selection methods will take accountof multi-step forecast accuracy as well asone-step forecast accuracy.

3 Automatic forecasting algorithms formultivariate time series will be developed.

4 Automatic forecasting algorithms thatinclude covariate information will bedeveloped.

For further information

robjhyndman.com

Slides and references for this talk.

Links to all papers and books.

Links to R packages.

A blog about forecasting research.

automatic algorithms for time series forecasting

grouped time series

time series analysis

forecasting competitions

appropriate time series

asia year millionsofsheep

arima modelling

exponential smoothing

complex seasonality

Data & Analytics

automatic forecasts algorithms

automatic state-based test generation using genetic...

genetic algorithms for eq-algebras automatic -...

solar energetic particle forecasting algorithms and...

econometric forecasting -...

theory and algorithms for forecasting non-stationary time...

study of algorithms to combine multiple automatic speech ......

visualising forecasting algorithm performance using time...

automatic time series forecasting: the forecast...

algorithms for automatic alignment of arrays

rob j hyndman automatic algorithms for time … algorithms...

financial forecasting using genetic algorithms · pdf...

automatic diï¬€erentiation of numerical integration...

automatic time series forecasting

advances in automatic time series forecasting

flood prediction forecasting using machine learning...

learning algorithms for automatic data structure design

automatic forecasting software - fuqua school of...

automatic time series forecasting: the forecast package for...

key frame selection algorithms for automatic generation of...