ch 9. forecasting - missouri state...
TRANSCRIPT
Ch 9. FORECASTING
Time Series Analysis
Time Series Analysis Ch 9. FORECASTING
In this chapter, we assume the model is known exactly, andconsider the calculation of forecasts and their properties for bothdeterministic trend models and ARIMA models.
9.1 Minimum Mean Square Error Forecasting
Suppose the series is available up to time t, namelyY1,Y2, · · · ,Yt . We forecast the value of Yt+`. We call t theforecast origin, ` the lead time, and denote the forecast as Yt(`).We shall develop methods based on minimizing the mean squareforecasting error. It turns out that
Yt(`) = E (Yt+` | Y1,Y2, · · · ,Yt).
Time Series Analysis Ch 9. FORECASTING
9.2 Deterministic Trends
We assume that the model is Yt = µt + Xt , where µt is adeterministic function, and Xt is a white noise with zero mean andvariance γ0. Then
Yt(`) = E (µt+` + Xt+` | Y1, · · · ,Yt)
= E (µt+` | Y1, · · · ,Yt) + E (Xt+` | Y1, · · · ,Yt) = µt+`.
The forecast error is given by
et(`) = Yt+` − Yt(`) = Xt+`.
Since E (et(`)) = 0, the forecast is unbiased. The forecast errorvariance is
Var (et(`)) = Var (Xt+`) = γ0.
Time Series Analysis Ch 9. FORECASTING
Ex. In Exhibits 3.5 and 3.6, we estimate the temperature series inDubuque, Iowa as a cosine trend
µt = 46.2660 + (−26.7079) cos(2πt) + (−2.1697) sin(2πt)
The start time is January 1964, and the end time is December1975. Here µt is a periodic function of period 1 so we will get thesame forecast temperature on the same month each year. Forexample, the forecast temperature at June 1976 is
µ1976+5/12 = µ5/12 = 68.31◦F
Time Series Analysis Ch 9. FORECASTING
9.3 ARIMA Forecasting
For ARIMA models, the forecasts can be expressed in severaldifferent ways.
Time Series Analysis Ch 9. FORECASTING
9.3.1 AR(1)
The model is Yt − µ = φ(Yt−1 − µ) + et . Replace t by t + `:
Yt+` − µ = φ(Yt+`−1 − µ) + et+`
Take conditional expectation E (· | Y1, · · · ,Yt):
Yt(`)− µ = φ(Yt(`− 1)− µ) + E (et+` | Y1, · · · ,Yt)
Yt(`)− µ = φ(Yt(`− 1)− µ) or Yt(`) = µ+ φ(Yt(`− 1)− µ).
The last equation is the difference equation form of the forecast. Then
Yt(`)− µ = φ(Yt(`− 1)− µ) = φ2(Yt(`− 2)− µ) = · · · = φ`(Yt − µ)
So the lead ` forecast may also be expressed as
Yt(`) = µ+ φ`(Yt − µ).
Since |φ| < 1, we have Yt(`) ≈ µ for large `.
In numerical calculation, apply the difference equation form recursively
will accumulate the round off error. We should use many decimal places
in such calculations.Time Series Analysis Ch 9. FORECASTING
The estimated model is:Yt − 74.3293 = 0.5705(Yt−1− 74.3293) + et , and the last observedvalue of the color property is Yt = 67. We get the lead ` forecast
Yt(`) = 74.3293 + (0.5705)`(67− 74.3293).
We can implement a function to calculate Yt(`) for some `:
Yt(1) = 70.14793, Yt(1) = 71.94383,
Yt(1) = 73.88636, Yt(1) = 74.30253.
Time Series Analysis Ch 9. FORECASTING
Now we calculate the forecast error et(`) for AR(1) model. Recall
Yt+` − µ = et+` + φ(Yt+`−1 − µ)
= et+` + φet+`−1 + φ2(Yt+`−2 − µ) = · · ·= et+` + φet+`−1 + · · ·+ φ`−1et+1 + φ`(Yt − µ),
Yt(`)− µ = φ`(Yt − µ).
Therefore, for AR(1) model,
et(`) = Yt+` − Yt(`) = et+` + φet+`−1 + · · ·+ φ`−1et+1.
In particular, et(1) = et+1.
Time Series Analysis Ch 9. FORECASTING
9.3.2 MA(1)
The invertible model is: Yt = µ+ et − θet−1. Replace t by t + 1and take E (· | Y1, · · · ,Yt):
Yt(1) = µ+ E (et+1 | Y1, · · · ,Yt)− θE (et | Y1, · · · ,Yt) = µ− θet .
For ` > 1,
Yt(`) = µ+ E (et+` | Y1, · · · ,Yt)− θE (et+`−1 | Y1, · · · ,Yt) = µ.
Time Series Analysis Ch 9. FORECASTING
9.3.3 The Random Walk with Drift
The nonstationary model is: Yt = Yt−1 + θ0 + et . Then
E (Yt+` | Y1, · · · ,Yt) = E (Yt+`−1 | Y1, · · · ,Yt)+θ0+E (et+` | Y1, · · · ,Yt)
We getYt(`) = Yt(`− 1) + θ0 = · · · = Yt + θ0`.
Now we consider the forecast error.
Yt+` = Yt+`−1+θ0+et+` = · · · = Yt+θ0`+et+`+et+`−1+· · ·+et+1.
Hence
et(`) = Yt+` − Yt(`) = et+` + et+`−1 + · · ·+ et+1.
So the forecast is unbiased as E(et(`)) = 0, and
Var (et(`)) = σ2e`.
In general, an ARIMA process is nonstationary iff Var (et(`))grows without limit.
Time Series Analysis Ch 9. FORECASTING
9.3.4 ARMA(p,q)
For a stationary invertible ARMA(p,q) model, the differenceequation form is
Yt(`) =
p∑i=1
φi Yt(`− i) + θ0 −q∑
j=1
θjE (et+`−j | Y1, · · · ,Yt)
in which
E (et+j | Y1, · · · ,Yt) =
{0 for j > 0
et+j for j ≤ 0
The Yt(`) is the true forecast when ` > 0, but Yt(`) = Yt+` for` ≤ 0.
Time Series Analysis Ch 9. FORECASTING
Ex. Consider the ARMA(1,2) model:Ys = φYs−1 + θ0 + es − θ1es−1 − θ2es−2. Then
Yt(1) = φYt + θ0 − θ1et − θ2et−1,Yt(2) = φYt(1) + θ0 − θ2et ,Yt(`) = φYt(`− 1) + θ0, ` ≥ 3.
The forecast may be expressed explicitly in terms of µ =θ0
1− φ:
Yt(`)− µ = φ`(Yt − µ)− (φ`−1θ1 + φ`−2θ2)et − φ`−1θ2et−1.
Time Series Analysis Ch 9. FORECASTING
For ARMA(p,q) models, the noise terms et−(q−1), · · · , et−1, etappear directly in the computation of the forecasts for leads` = 1, 2, · · · , q. However, for ` > q, the autoregressive portion ofthe difference equation takes over, and we have
Yt(`) = φ1Yt(`−1)+φ2Yt(`−2)+· · ·+φpYt(`−p)+θ0 for ` > q. (1)
So the nature of the forecasts for long lead times will bedetermined by the autoregressive parameters φ1, · · · , φp.
Recall that θ0 = µ(1− φ1 − φ2 − · · · − φp). (1) may be rewrittenas
Yt(`)−µ = φ1[Yt(`−1)−µ]+φ2[Yt(`−2)−µ]+· · ·+φp[Yt(`−p)−µ] for ` > q.(2)
Time Series Analysis Ch 9. FORECASTING
Now we discuss the forecast error {et(`)} for general (stationary ornonstationary) ARIMA models. Appendix G shows that everyARIMA model has truncated linear process representation:
Yt+` = Ct(`) + It(`) for ` ≥ 1 (3)
where Ct(`) is a function of Yt ,Yt−1, · · · , and
It(`) = et+` + Ψ1et+`−1 + Ψ2et+`−2 + · · ·+ Ψ`−1et+1 for ` ≥ 1.(4)
Take E (· | Y1, · · · ,Yt) on (3):
Yt(`) = E (Ct(`) | Y1, · · · ,Yt) + E (It(`) | Y1, · · · ,Yt) = Ct(`)
Therefore,
et(`) = Yt(`)− Yt+` = It(`) = et+` + Ψ1et+`−1 + · · ·+ Ψ`−1et+1.
Time Series Analysis Ch 9. FORECASTING
It implies that the forecasts are unbiased:
E(et(`)) = 0,
and
Var (et(`)) = σ2e
`−1∑j=0
Ψ2j
for ` ≥ 1.
For stationary ARMA models and large `, the variances increasewith upper bounded:
Var (et(`)) ≈ σ2e
∞∑j=0
Ψ2j
= γ0.
For nonstationary ARIMA models, the forecasts are unbiased;however, the Ψj weights does not decay to zero as j increases, sothat the error variances increase without bounded.
Time Series Analysis Ch 9. FORECASTING
9.3.5 Nonstationary Models
Recall that {Yt} ∼ ARIMA(p, d , q) means that
∇dY t = φ1∇dY t−1+φ2∇dY t−2+· · ·+φp∇dY t−p+et−θ1et−1−θ2et−2−· · ·−θqet−q
where
∇dYt = ∇d−1Yt −∇d−1Yt−1 =d∑
i=0
(d
i
)(−1)iYt−i
= Yt −(d
1
)Yt−1 +
(d
2
)Yt−2 − · · ·+
(d
d − 1
)(−1)d−1Yt−d+1 + (−1)dYt−d .
So an ARIMA(p,d,q) model can be naturally expressed as anonstationary ARMA(p+d,q) model. We can get the forecastsYt(t) and the forecast errors et(`) similarly to those for stationaryARMA(p,q) models.
Time Series Analysis Ch 9. FORECASTING
Ex. The ARIMA(1,1,1) model isYt − Yt−1 = φ(Yt−1 − Yt−2) + θ0 + et − θet−1. It has anARMA(2,1) expression:
Yt = (1 + φ)Yt−1 − φYt−2 + θ0 + et − θet−1.
In the above equation, replace t by t + 1, t + 2, or t + `, and takeE (· | Y1, · · · ,Yt). We get the forecasts:
Yt(1) = (1 + φ)Yt − φYt−1 + θ0 − θet ,Yt(2) = (1 + φ)Yt(1)− φYt + θ0,
Yt(`) = (1 + φ)Yt(`− 1)− φYt(`− 2) + θ0.
Time Series Analysis Ch 9. FORECASTING
For all nonstationary ARIMA models, the forecasts are unbiased,
but the variance of forecast error , Var (et(`)) = σ2e
`−1∑j=0
Ψ2j
,
grows without bound. Some examples:
1 for the random walk with drift, Ψj = 1;
2 for IMA(1,1) model, Ψj = 1− θ for j ≥ 1;
3 for ARI(1,1) model, Ψj = (1− φj+1)/(1− φ) for j ≥ 1.
So with nonstationary series, the distant future is quite uncertain.Section 9.9 gives the Summary of Forecasting with Certain ARIMAModels.
Time Series Analysis Ch 9. FORECASTING
9.4 Prediction Limits
9.4.1 Deterministic Trends
The model is: Yt = µt + Xt . Recall that
Yt(`) = µt+`, et(`) = Xt+`, Var (et(`)) = γ0.
Suppose the stochastic component Xt is normally distributed. Then so isthe forecast error et(`). For a given confidence level 1− α, we could usea standard normal percentile z1−α/2 = F−1(1− α/2) (the inversefunction of the cdf of the standard normal distribution), to claim
P
[−z1−α/2 <
Yt+` − Yt(`)√Var (et(`))
]= 1− α (5)
Thus we may be (1− α)100% confident that the future observation Yt+`
will be contained within the prediction limits
Yt(`)± z1−α/2√
Var (et(`)).
Time Series Analysis Ch 9. FORECASTING
Ex. The monthly average temperature series in Dubuque, Iowa ismodeled by a trend
µt = 46.2660 + (−26.7079) cos(2πt) + (−2.1697) sin(2πt)
with√
Var (et(`)) =√γ0 = 3.7◦F . The predicted temperature of
June 1976 is 68.3◦F . Thus with 95% of confidence interval, theaverage June 1976 temperature is
68.3± 1.96(3.7) = 68.3± 7.252, or [61.05◦F , 75.55◦F ].
In practice, the correct forecast error variance will be slightly largerthan Var (et(`)) as we use the estimated parameters.
Time Series Analysis Ch 9. FORECASTING
9.4.2 ARIMA Models
If the white noise terms {et} are normally distributed, then so isthe forecast error et(`). We know that
Var (et(`)) = σ2e
`−1∑j=0
Ψ2j
.
Both σ2e and Ψ-weights must be estimated from the observed timeseries. For large sample sizes, the estimations will have little effecton the actual prediction limits.
Time Series Analysis Ch 9. FORECASTING
Ex. (TS-ch9.R) The AR(1) model estimation of industry color
property in Exhibit 9.1 estimates that
φ = 0.5705, µ = 74.3293, σ2e = 24.8,
and we have
Yt(`) = 74.3293 + (0.5705)`(67− 74.3293),
Var (et(`)) = σ2e
[1− φ2`
1− φ2
].
Thus the 95% confidence interval for `-step-ahead prediction is
74.3293 + (0.5705)`(67− 74.3293)± 1.96 ·
√24.8
(1− 0.57052`
1− 0.57052
).
We can calculated some intervals for ` = 1, 2, 5, 10, etc.
Time Series Analysis Ch 9. FORECASTING
9.5 Forecasting Illustrations
9.5.1 Deterministic Trends
We use the example of temperature series in Dubuque, Iowa toshow how to plot the series with forecast and confidence band.
Time Series Analysis Ch 9. FORECASTING
Ex. 9.2 (cont)
The model fits quite well with a relatively small error variance, theforecast limits are quite close to the fitted trend forecast.
Time Series Analysis Ch 9. FORECASTING
9.5.2 ARIMA Models
Notice how the forecasts approach the mean exponentially as thelead time increases. Also note how the prediction limits increase inwidth.
Time Series Analysis Ch 9. FORECASTING
Time Series Analysis Ch 9. FORECASTING
9.6 Updating ARIMA Forecasts
The origin forecast with origin time t and lead `+ 1 is Yt(`+ 1).Once the observation Yt+1 at time t + 1 is known, we may updatethe forecast as Yt+1(`). The truncated linear process shows that
Yt+`+1 = Ct(`+ 1) + et+`+1 + Ψ1et+` + · · ·+ Ψ`et+1.
Note that Ct(`+ 1) and et+1 are functions of Yt+1,Yt , · · · . TakeE (· | Y1, · · · ,Yt+1) on both sides.
Yt+1(`) = Ct(`+ 1) + Ψ`et+1 = Yt(`+ 1) + Ψ`[Yt+1 − Yt(1)]
where Yt+1 − Yt(1) is the actual forecast error at time t + 1 onceYt+1 has been observed.
Time Series Analysis Ch 9. FORECASTING
Ex. The AR(1) model for the industrial color property in Exhibit9.1 gives
Y35(1) = 70.14793, Y35(2) = 71.94383.
If we now observe the next value Y36 = 65, then we update theforecast for t = 37 as
Y36(1) = 71.94383 + 0.5705(65− 70.14793) = 69.00694.
Time Series Analysis Ch 9. FORECASTING
9.7 Forecast Weights and Exponentially WeightedMoving Averages
We hope to explicitly determine the forecasts from the observedseries Yt ,Yt−1, · · · ,Y1. In general, an invertible ARIMA(p,d,q)process has an inverted form:
Yt = π1Yt−1 + π2Yt−2 + π3Yt−3 + · · ·+ et .
ThenYt+` = π1Yt+`−1 + π2Yt+`−2 + · · ·+ et+`.
Apply E (· | Y1, · · · ,Yt) on both sides for ` = 1, 2, · · · We get
Yt(1) = π1Yt + π2Yt−1 + π3Yt−2 + · · · ,Yt(2) = π1Yt(1) + π2Yt + π3Yt−1 + · · · ,
...
Time Series Analysis Ch 9. FORECASTING
Now we determine the π-weights. An ARIMA(p,d,q) model maybe written as a nonstationary ARMA(p+d,q) model:
Yt = ϕ1Yt−1+ϕ2Yt−2+· · ·+ϕp+dYt−p−d+et−θ1et−1−· · ·−θqet−q.
Time Series Analysis Ch 9. FORECASTING
Ex. Consider the nonstationary IMA(1,1) model:Yt = Yt−1 + et − θet−1. Use (9.7.2) or substituteet = Yt − π1Yt−1 − π2Yt−2 − · · · for et and et−1. We get
πj = θπj−1 = θj−1π1 = (1− θ)θj−1, j ≥ 1.
Therefore,
Yt(1) = (1− θ)Yt + (1− θ)θYt−1 + (1− θ)θ2Yt−2 + · · ·
Here the π-weights decrease exponentially. Thus Yt(1) is called anexponentially weighted moving average (EWMA). We alsohave
Yt(1) = (1− θ)Yt + θYt−1(1) = Yt−1(1) + (1− θ)[Yt − Yt−1(1)].
They show how to update forecasts from origin t − 1 to origin t.
Time Series Analysis Ch 9. FORECASTING
9.8 Forecasting Transformed Series
9.8.1 Differencing
For nonstationary ARIMA models, we use differences to achievestationarity. Two methods of forecasting may be used:
1 forecasting the original nonstationary series,
2 forecasting the stationary differenced series, then sum theterms to obtain the forecast of original series.
Both methods will lead to the same forecast for any type of lineartransformation with constant coefficients, including differences.Because taking conditional expectation is a linear function.
Time Series Analysis Ch 9. FORECASTING
9.8.2 Log Transformations
Let Yt denote the original series value and let Zt = log(Yt). Wealways have
E (Yt+` | Y1, · · · ,Yt) ≥ exp[E (Zt+` | Z1, · · · ,Zt)]
with equality holding only in trivial cases. Thus exp[Zt(`)] is notthe minimum mean square error forecast of Yt+`.Fact: If X has a normal distribution with mean µ and variance σ2,then
E(exp(X)) = exp
[µ+
σ2
2
].
Time Series Analysis Ch 9. FORECASTING
In our case, µ = E (Zt+` | Z1, · · · ,Zt) and
σ2 = Var (Zt+` | Zt,Zt−1, · · · ,Z1)
= Var(
Zt(`) + et(`) | Zt,Zt−1, · · · ,Z1
)= Var (Ct(`) + et(`) | Zt,Zt−1, · · · ,Z1)
= Var (et(`) | Zt,Zt−1, · · · ,Z1)
= Var (et(`)) .
Thus the minimum mean square error forecast in the original seriesis given by
Yt(`) = exp
{Zt(`) +
1
2Var (et(`))
}.
Time Series Analysis Ch 9. FORECASTING
However, if Zt has a normal distribution, then Yt = exp(Zt) has alognormal distribution, for which a criterion based on the meanabsolute error would be better. The optimal forecast for thiscriterion is the median of Yt+` conditional on Yt ,Yt−1, · · · ,Y1.The log function preserves the mean. So in this case
Yt(`) = mean [Yt+` | Yt , · · · ,Y1]
= exp [mean(Zt+` | Zt · · · ,Z1)]
= exp[Zt(`)
]= expµ.
Time Series Analysis Ch 9. FORECASTING
9.9 Summary of Forecasting with Certain ARIMAModels
This section includes a summary of Yt(`), et(`), Var (et(`)), andΨj , for AR(1), MA(1), IMA(1,1), IMA(2,2) models.
Time Series Analysis Ch 9. FORECASTING
Time Series Analysis Ch 9. FORECASTING
Time Series Analysis Ch 9. FORECASTING
Time Series Analysis Ch 9. FORECASTING
Time Series Analysis Ch 9. FORECASTING
Time Series Analysis Ch 9. FORECASTING
Time Series Analysis Ch 9. FORECASTING