Download - Slides sales-forecasting-session2-web
Arthur CHARPENTIER - Sales forecasting.
Sales forecasting # 2
Arthur Charpentier
1
Arthur CHARPENTIER - Sales forecasting.
Agenda
Qualitative and quantitative methods, a very general introduction
• Series decomposition
• Short versus long term forecasting
• Regression techniques
Regression and econometric methods
• Box & Jenkins ARIMA time series method
• Forecasting with ARIMA series
Practical issues : forecasting with MSExcel
2
Arthur CHARPENTIER - Sales forecasting.
Time series decomposition
0 20 40 60 80
1500
020
000
2500
030
000
A13 Highway
3
Arthur CHARPENTIER - Sales forecasting.
Time series decomposition
0 20 40 60 80
1500
020
000
2500
030
000
A13 Highway
4
Arthur CHARPENTIER - Sales forecasting.
Time series decomposition
0 20 40 60 80
−500
00
5000
A13 Highway, removing trending
5
Arthur CHARPENTIER - Sales forecasting.
Time series decomposition
●
●
●
●
●
●●
●
●●
●●
●
●
●●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
● ●
● ●
● ●
●
● ●
●
● ●
●
●
●
●
●
●
●
● ●
●● ●
●
●
●
●
●●
● ●
●
●
●
●
●
●
●
2 4 6 8 10 12
−500
00
5000
A13 Highway, removing trending
Months
6
Arthur CHARPENTIER - Sales forecasting.
Time series decomposition
●
●
●
●
●
●●
●
●●
●●
●
●
●●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
● ●
● ●
● ●
●
● ●
●
● ●
●
●
●
●
●
●
●
● ●
●● ●
●
●
●
●
●●
● ●
●
●
●
●
●
●
●
2 4 6 8 10 12
−500
00
5000
A13 Highway, removing trending
Months
7
Arthur CHARPENTIER - Sales forecasting.
Time series decomposition
0 20 40 60 80
1500
020
000
2500
030
000
A13 Highway: trend and cycle
8
Arthur CHARPENTIER - Sales forecasting.
Time series decomposition
●
●
●
●
●
●
●●
●
●
●●
●
●
●●
●
●
●
●
●
●●
●●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●●
●
●
0 20 40 60 80
−300
0−1
000
010
0020
00
A13 Highway: random part
9
Arthur CHARPENTIER - Sales forecasting.
Time series decomposition
●
●
●
●
●
●
●●
●
●
●●
●
●
●●
●
●
●
●
●
●●
●●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●●
●
●
0 20 40 60 80
−300
0−1
000
010
0020
00
A13 Highway: random part
10
Arthur CHARPENTIER - Sales forecasting.
Time series decomposition
●
●
●
●
●
●
●●
●
●
●●
●
●
●●
●
●
●
●
●
●●
●●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●●
●
●
0 20 40 60 80
−300
0−1
000
010
0020
00
A13 Highway: random part
11
Arthur CHARPENTIER - Sales forecasting.
Time series decomposition
0 20 40 60 80 100
1500
020
000
2500
030
000
A13 Highway, prediction
12
Arthur CHARPENTIER - Sales forecasting.
Time series decomposition
0 20 40 60 80 100
1500
020
000
2500
030
000
A13 Highway, prediction
13
Arthur CHARPENTIER - Sales forecasting.
Time series decomposition
●
●
●
●
●
●
●●
●
●
●●
●
●
●●
●
●
●
●
●
●●
●●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
0 20 40 60 80
−200
0−1
000
010
0020
00
A13 Highway: random part (v2)
14
Arthur CHARPENTIER - Sales forecasting.
Time series decomposition
●
●
●
●
●
●
●●
●
●
●●
●
●
●●
●
●
●
●
●
●●
●●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
0 20 40 60 80
−200
0−1
000
010
0020
00
A13 Highway: random part (v2)
15
Arthur CHARPENTIER - Sales forecasting.
Time series decomposition, modeling the random part
Histogram of residuals (v2)
Den
sity
−3000 −2000 −1000 0 1000 2000
0e+0
02e
−04
4e−0
4
16
Arthur CHARPENTIER - Sales forecasting.
Time series decomposition, modeling the random part
●
●
●
●
●
●
●●
●
●
●●
●
●
●●
●
●
●
●
●
● ●
●●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
−2 −1 0 1 2
−200
0−1
000
010
0020
00
Normal QQ plot of residuals (v2)
Theoretical Quantiles
Sam
ple
Qua
ntile
s
17
Arthur CHARPENTIER - Sales forecasting.
Time series decomposition, forecasting
0 20 40 60 80 100
1500
020
000
2500
030
000
A13 Highway, forecast scenario
●
●
●
●
●
●
●
●●
●●
●●
●
●
●
●●
●
●
●
●●
●
●
18
Arthur CHARPENTIER - Sales forecasting.
Time series decomposition, forecasting
0 20 40 60 80 100
1500
020
000
2500
030
000
A13 Highway, forecast scenario
●
●●
●
●
●
●
●
●
●●
●●
●
●
●●
●
●●●
●
●
●
●
19
Arthur CHARPENTIER - Sales forecasting.
Time series decomposition, forecasting
0 20 40 60 80 100
1500
020
000
2500
030
000
A13 Highway, forecast scenario
●
●
●
●●
●
●
●●
●●
●●
●●
●
●
●
●●
●
●
●
●●
20
Arthur CHARPENTIER - Sales forecasting.
Time series decomposition, forecasting
0 20 40 60 80 100
1500
020
000
2500
030
000
A13 Highway, forecast scenario
●
●●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
21
Arthur CHARPENTIER - Sales forecasting.
Time series decomposition, modeling the seasonal
componant
●
●
●
●
●
●●
●
●●
●●
●
●
●●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
● ●
● ●
● ●
●
● ●
●
● ●
●
●
●
●
●
●
●
● ●
●● ●
●
●
●
●
●●
● ●
●
●
●
●
●
●
●
2 4 6 8 10 12
−500
00
5000
A13 Highway, removing trending
Months
22
Arthur CHARPENTIER - Sales forecasting.
Time series decomposition, modeling the seasonal
componant
0 20 40 60 80
1500
020
000
2500
030
000
A13 Highway: trend and cycle
23
Arthur CHARPENTIER - Sales forecasting.
Time series decomposition, modeling the seasonal
componant
●●
●
●●●
●
●
●
●
●●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
0 20 40 60 80
−400
0−2
000
020
0040
00
A13 Highway: random part
24
Arthur CHARPENTIER - Sales forecasting.
Modeling the random component
The unpredictible random component is the key element when forecasting. Most
of the uncertainty comes from this random component εt.
The lower the variance, the smaller the uncertainty on forecasts.
The general theoritical framework related to randomness of time series is related
to weakly stationary.
25
Arthur CHARPENTIER - Sales forecasting.
De�ning stationarity
Time series (Xt) is weakly stationary if
� for all t, E(X2t
)< +∞,
� for all t, E (Xt) = µ, constant independent of t,
� for all t and for all h, cov (Xt, Xt+h) = E ([Xt − µ] [Xt+h − µ]) = γ (h),independent of t.
Function γ (·) is called autocovariance function.
Given a stationary series (Xt) , de�ne the autocovariance function, as
h 7→ γX (h) = cov (Xt, Xt−h) = E (XtXt−h)− E (Xt) .E (Xt−h) .
and de�ne the autocorrelation function, as
h 7→ ρX (h) = corr (Xt, Xt−h) =cov (Xt, Xt−h)√V (Xt)
√V (Xt−h)
=γX (h)γX (0)
.
26
Arthur CHARPENTIER - Sales forecasting.
De�ning stationarity
A process (Xt) is said to be strongly stationary if for all t1, ..., tn and h we have
the following law equality
L (Xt1 , ..., Xtn) = L (Xt1+h, ..., Xtn+h) .
A time series (εt) is a white noise if all autocovariances are null, i.e. γ (h) = 0 for
all h 6= 0. Thus, a process (εt) is a white noise if it is stationary, centred and
noncorrelated, i.e.
E (εt) = 0, V (εt) = σ2 and ρε (h) = 0 for any h 6= 0.
27
Arthur CHARPENTIER - Sales forecasting.
Statistical issues
Consider a set of observations {X1, ..., XT }.
The empirical mean is de�ned as
XT =1T
T∑t=1
Xt.
The empirical autocovariance function is de�ned as
γ̂T (h) =1
T − h
T−h∑t=1
(Xt −XT
) (Xt−h −XT
),
while the empirical autocorrelation function is de�ned as
ρ̂T (h) =γ̂T (h)γ̂T (0)
.
Remark those estimators can be biased, but asymptotically unbiased. More
precisely γ̂T (h)→ γ (h) and ρ̂T (h)→ ρ (h) as T →∞.
28
Arthur CHARPENTIER - Sales forecasting.
Backward and forward operators
De�ne the lag operator L (or B for backward) the linear operator de�ned as
L : Xt 7−→ L (Xt) = LXt = Xt−1,
and the forward operator F ,
F : Xt 7−→ F (Xt) = FXt = Xt+1,
Note that L ◦ F = F ◦ L = I (identity operator) and further F = L−1 and
L = F−1.
� it is possible to compose those operators : L2 = L ◦ L, and more generally
Lp = L ◦ L ◦ ... ◦ L︸ ︷︷ ︸ where p ∈ N
with convention L0 = I. Note that Lp (Xt) = Xt−p.
� Let A denote a polynom,A (z) = a0 + a1z + a2z2 + ...+ apz
p. Then A (L) is the
29
Arthur CHARPENTIER - Sales forecasting.
operator
A (L) = a0I + a1L+ a2L2 + ...+ apL
p =p∑k=0
akLk.
Let (Xt) denote a time series. Series (Yt) de�ned by Yt = A (L)Xt satis�es
Yt = A (L)Xt =p∑k=0
akXt−k.
or, more generally, assuming that we can formally the limit,
A (z) =∞∑k=0
akzk et A (L) =
∞∑k=0
akLk.
30
Arthur CHARPENTIER - Sales forecasting.
Backward and forward operators
Note that for all moving average A and B, thenA (L) +B (L) = (A+B) (L)
α ∈ R, αA (L) = (αA) (L)
A (L) ◦B (L) = (AB) (L) = B (L) ◦A (L) .
Moving average C = AB = BA satis�es( ∞∑k=0
akLk
)◦
( ∞∑k=0
bkLk
)=
( ∞∑i=0
ciLi
)où ci =
i∑k=0
akbi−k.
31
Arthur CHARPENTIER - Sales forecasting.
Geometry and probability
Recall that it is possible to de�ne an inner product in L2 (space of squared
integrable variables, i.e. �nite variance),
< X,Y >= E ([X − E(X)] · [Y − E(Y )]) = cov([X − E(X)], [Y − E(Y )])
Then the associated norm is ||X||2 = E([X − E(X)]2
)= V (X).
Two random variables are then orthogonal if < X,Y >= 0, i.e.cov([X − E(X)], [Y − E(Y )]) = 0.
Hence conditional expectation is simply a projection in the L2, E(X|Y ) is the theprojection is the space generated by Y of random variable X, i.e.
E(X|Y ) = φ(Y ), such that
� X − φ(Y ) ⊥ X, i.e. < X − φ(Y ), X >= 0,� φ(Y ) = Z∗ = argmin{Z = h(Y ), ||X − Z||2}� E(φ(Y )) <∞.
32
Arthur CHARPENTIER - Sales forecasting.
Linear projection
The conditional expectation E(X|Y ) is a projection if the set of all functions
{h(Y )}.
In linear regression, the projection if made in the subset of linear functions h(·).
We call this linear function conditional linear expectation, or linear projection,
denoted EL(X|Y ).
In purely endogeneous models, the best forecast for XT+1 given past
informations {XT , XT−1, XT−2, · · · , XT−h, ...} is
X̂T+1 = E(XT+1|{XT , XT−1, XT−2, · · · , XT−h, · · · }) = φ(XT , XT−1, XT−2, · · · , XT−h, · · · ).
Since estimating a nonlinear function is di�cult (especially in high dimension),
we focus on linear functions, i.e. autoregressive models,
X̂T+1 = EL(XT+1|{XT , XT−1, XT−2, · · · , XT−h, · · · }) = α0XT+α1XT−1+α2XT−2+· · ·+αhXT−h+· · · .
33
Arthur CHARPENTIER - Sales forecasting.
De�ning partial autocorrelations
Given a stationary series (Xt), de�ne the partial autocorrelation function
h 7→ ψX (h) as
ψX (h) = corr(X̂t, X̂t−h
),
where X̂t−h = Xt−h − EL (Xt−h|Xt−1, ..., Xt−h+1)
X̂t = Xt − EL (Xt|Xt−1, ..., Xt−h+1) .
34
Arthur CHARPENTIER - Sales forecasting.
Time series decomposition, modeling the random part
●
●
●
●
●
●
●●
●
●
●●
●
●
●●
●
●
●
●
●
●●
●●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
0 20 40 60 80
−200
0−1
000
010
0020
00
A13 Highway: random part (v2)
35
Arthur CHARPENTIER - Sales forecasting.
Time series decomposition, modeling the random part
0 5 10 15
−0.2
0.0
0.2
0.4
0.6
0.8
1.0
Lag
AC
F
Autocorrelations of residuals (v2)
36
Arthur CHARPENTIER - Sales forecasting.
Time series decomposition, modeling the random part
5 10 15
−0.2
−0.1
0.0
0.1
0.2
Lag
Par
tial A
CF
Partial autocorrelations of residuals (v2)
37
Arthur CHARPENTIER - Sales forecasting.
Time series decomposition, modeling the detrended series
0 20 40 60 80
−500
00
5000
A13 Highway, removing trending
38
Arthur CHARPENTIER - Sales forecasting.
Time series decomposition, modeling the detrended series
0 5 10 15 20 25 30 35
−0.5
0.0
0.5
1.0
Lag
AC
F
Autocorrelations of detrended series
39
Arthur CHARPENTIER - Sales forecasting.
Time series decomposition, modeling the detrended series
0 5 10 15 20 25 30 35
−0.4
−0.2
0.0
0.2
0.4
0.6
Lag
Par
tial A
CF
Partial autocorrelations of detrended series
40
Arthur CHARPENTIER - Sales forecasting.
Time series decomposition, modeling Yt = Xt −Xt−12
0 10 20 30 40 50 60 70
−300
0−1
000
010
0020
00
A13 Highway: lagged detrended series
41
Arthur CHARPENTIER - Sales forecasting.
Time series decomposition, modeling Yt = Xt −Xt−12
0 10 20 30 40 50 60 70
−300
0−1
000
010
0020
00
A13 Highway: lagged detrended series
42
Arthur CHARPENTIER - Sales forecasting.
Time series decomposition, modeling Yt = Xt −Xt−12
0 5 10 15 20 25 30 35
−0.2
0.0
0.2
0.4
0.6
0.8
1.0
Lag
AC
F
Autocorrelations of lagged detrended series
43
Arthur CHARPENTIER - Sales forecasting.
Time series decomposition, modeling Yt = Xt −Xt−12
0 5 10 15 20 25 30 35
−0.2
−0.1
0.0
0.1
0.2
Lag
Par
tial A
CF
Partial autocorrelations of lagged detrended series
44
Arthur CHARPENTIER - Sales forecasting.
Time series decomposition, forecasting
A13 Highway: forecasting detrended series (ARMA)
1990 1992 1994 1996 1998 2000
−500
00
5000
●
●●
●●
●
●
●
●
●
●
●●
●●
●●
●
●
●
●
●
●
●
45
Arthur CHARPENTIER - Sales forecasting.
Time series decomposition, forecasting
A13 Highway: forecasting detrended series (ARMA)
1990 1992 1994 1996 1998 2000
−500
00
5000
●
●●
●●
●
●
●
●
●
●
●●
●●
●●
●
●
●
●
●
●
●●
●●
●●
●
●
●
●
●
●
●
46
Arthur CHARPENTIER - Sales forecasting.
47
Arthur CHARPENTIER - Sales forecasting.
Estimating autocorrelations with MSExcel
48
Arthur CHARPENTIER - Sales forecasting.
A white noise
A white noise is de�ned as a centred process (E(εt) = 0), stationary(V (εt) = σ2), such that cov (εt, εt−h) = 0 for all h 6= 0.
The so-called Box-Pierce test can be used to test H0 : ρ (1) = ρ (2) = ... = ρ (h) = 0
Ha : there exists i such that ρ (i) 6= 0.
The idea is to use
Qh = Th∑k=1
ρ̂2k,
where h is the lag number and T the total number of observations.
Under H0, Qh has a χ2 distribution, with h degrees of freedom.
49
Arthur CHARPENTIER - Sales forecasting.
A white noise
Another statistics with better properties is a modi�ed version of Q,
Q′h = T (T + 2)h∑k=1
ρ̂2k
T − k,
Most of the softwares return Qh for h = 1, 2, · · · , and the associated p-value. If p
exceeds 5% (the standard signi�cance level) we feel con�dent in accepting H0,
while if p is less than 5% , we should reject H0.
50
Arthur CHARPENTIER - Sales forecasting.
A white noise
0 100 200 300 400 500
−2
−1
01
23
Simulated white noise
0 10 20 30 40
0.0
0.2
0.4
0.6
0.8
1.0
Lag
AC
F
White noise autocorrelations
0 10 20 30 40
−0.
6−
0.4
−0.
20.
00.
20.
40.
6
Lag
Par
tial A
CF
White noise partial autocorrelations
51
Arthur CHARPENTIER - Sales forecasting.
Time series decomposition, testing for white noise
Box−Pierce statistic, testing for white noise on lagged detrended series
5 10 15 20
05
1015
20
0.0
0.2
0.4
0.6
0.8
1.0
Q B
ox−P
ierc
e st
atis
tics
p−va
lue
52
Arthur CHARPENTIER - Sales forecasting.
Time series decomposition, testing for white noise
Box−Pierce statistic, testing for white noise on residuals (v2)
5 10 15 20
010
2030
4050
60
0.0
0.2
0.4
0.6
0.8
1.0
Q B
ox−P
ierc
e st
atis
tics
p−va
lue
53
Arthur CHARPENTIER - Sales forecasting.
Autoregressive process AR(p)
We call autoregressive process of order p, denoted AR (p), a stationnary process
(Xt) satisfying equation
Xt −p∑i=1
φiXt−i = εt for all t ∈ Z, (1)
where the φi's are real-valued coe�cients and where (εt) is a white noise process
with variance σ2. (1) is equivalent to
Φ (L)Xt = εt where Φ (L) = I− φ1L− · · · − φpLp
54
Arthur CHARPENTIER - Sales forecasting.
Autoregressive process AR(1), order 1
The general expression for AR (1) process is
Xt − φXt−1 = εt for all t ∈ Z,
where (εt) is a white noise with variance σ2.
If φ = ±1, process (Xt) is not stationary. E.g. if φ = 1, Xt = Xt−1 + εt (called
random walk) can be written
Xt −Xt−h = εt + εt−1 + ...+ εt−h+1,
and thus E (Xt −Xt−h)2 = hσ2.
But it is possible to prove that for any stationary process
E (Xt −Xt−h)2 ≤ 4V (Xt). Since it is impossible to have for any h,
hσ2 ≤ 4V (Xt), it means that the process cannot be stationary.
55
Arthur CHARPENTIER - Sales forecasting.
Autoregressive process AR(1), order 1
If |φ| < 1 it is possible to invert the polynomial lag operator
Xt = (1− φL)−1εt =
∞∑i=0
φiεt−i (as a function of the past) (εt) ). (2)
For a stationary process,the aucorelation function is given by ρ (h) = φh.
Further, ψ(1) = φ and ψ(h) = 0 for h ≥ 2.
56
Arthur CHARPENTIER - Sales forecasting.
A AR(1) process, Xt = 0.7Xt−1 + εt
Simulated AR(1)
0 100 200 300 400 500
−4
−2
02
0 10 20 30 40
0.0
0.2
0.4
0.6
0.8
1.0
Lag
AC
F
AR(1) autocorrelations
0 10 20 30 40
−0.
6−
0.4
−0.
20.
00.
20.
40.
6
Lag
Par
tial A
CF
AR(1) partial autocorrelations
57
Arthur CHARPENTIER - Sales forecasting.
A AR(1) process, Xt = 0.4Xt−1 + εt
Simulated AR(1)
0 100 200 300 400 500
−3
−2
−1
01
23
0 10 20 30 40
0.0
0.2
0.4
0.6
0.8
1.0
Lag
AC
F
AR(1) autocorrelations
0 10 20 30 40
−0.
6−
0.4
−0.
20.
00.
20.
40.
6
Lag
Par
tial A
CF
AR(1) partial autocorrelations
58
Arthur CHARPENTIER - Sales forecasting.
A AR(1) process, Xt = −0.5Xt−1 + εt
Simulated AR(1)
0 100 200 300 400 500
−2
02
4
0 10 20 30 40
−0.
50.
00.
51.
0
Lag
AC
F
AR(1) autocorrelations
0 10 20 30 40
−0.
6−
0.4
−0.
20.
00.
20.
40.
6
Lag
Par
tial A
CF
AR(1) partial autocorrelations
59
Arthur CHARPENTIER - Sales forecasting.
A AR(1) process, Xt = 0.99Xt−1 + εt
Simulated AR(1)
0 100 200 300 400 500
−10
−5
05
10
0 10 20 30 40
0.0
0.2
0.4
0.6
0.8
1.0
Lag
AC
F
AR(1) autocorrelations
0 10 20 30 40
−0.
6−
0.4
−0.
20.
00.
20.
40.
6
Lag
Par
tial A
CF
AR(1) partial autocorrelations
60
Arthur CHARPENTIER - Sales forecasting.
Autoregressive process AR(2), order 2
Those processes are also called Yule process, and they satisfy(1− φ1L− φ2L
2)Xt = εt,
where the roots of Φ (z) = 1− φ1z − φ2z2 are assumed to lie outside the unit
circle, i.e. 1− φ1 + φ2 > 0
1 + φ1 − φ2 > 0
φ21 + 4φ2 > 0,
61
Arthur CHARPENTIER - Sales forecasting.
Autoregressive process AR(2), order 2
Autocorrelation function satis�es equation
ρ (h) = φ1ρ (h− 1) + φ2ρ (h− 2) for any h ≥ 2,
and the partial autocorrelation function satis�es
ψ (h) =
ρ (1) for h = 1[ρ (2)− ρ (1)2
]/[1− ρ (1)2
]for h = 2
0 for h ≥ 3.
62
Arthur CHARPENTIER - Sales forecasting.
A AR(2) process, Xt = 0.6Xt−1 − 0.35Xt−2 + εt
Simulated AR(2)
0 100 200 300 400 500
−4
−2
02
0 10 20 30 40
−0.
20.
00.
20.
40.
60.
81.
0
Lag
AC
F
AR(2) autocorrelations
0 10 20 30 40
−0.
6−
0.4
−0.
20.
00.
20.
40.
6
Lag
Par
tial A
CF
AR(2) partial autocorrelations
63
Arthur CHARPENTIER - Sales forecasting.
A AR(2) process, Xt = −0.4Xt−1 − 0.5Xt−2 + εt
Simulated AR(2)
0 100 200 300 400 500
−4
−2
02
0 10 20 30 40
−0.
4−
0.2
0.0
0.2
0.4
0.6
0.8
1.0
Lag
AC
F
AR(2) autocorrelations
0 10 20 30 40
−0.
6−
0.4
−0.
20.
00.
20.
40.
6
Lag
Par
tial A
CF
AR(2) partial autocorrelations
64
Arthur CHARPENTIER - Sales forecasting.
Moving average process MA(q)
We call moving average process of order q, denoted MA (q), a stationnary
process (Xt) satisfying equation
Xt = εt +q∑i=1
θiεt−i for all t ∈ Z, (3)
where the θi's are real-valued coe�cients, and process (εt) is a white noise
process with variance σ2. (3) processes can be written equivalently
Xt = Θ (L) εt whereΘ (L) = I + θ1L+ ...+ θqLq.
The autocovariance function satis�es
γ (h) = E (XtXt−h)
= E ([εt + θ1εt−1 + ...+ θqεt−q] [εt−h + θ1εt−h−1 + ...+ θqεt−h−q])
=
[θh + θh+1θ1 + ...+ θqθq−h]σ2 if 1 ≤ h ≤ q0 if h > q,
65
Arthur CHARPENTIER - Sales forecasting.
Moving average process MA(q)
If h = 0, then γ (0) =[1 + θ21 + θ22 + ...+ θ2q
]σ2. This equation can be written
γ (k) = σ2
q∑j=0
θjθj+k with convention θ0 = 1.
Autocovariance function satis�es
ρ (h) =θh + θh+1θ1 + ...+ θqθq−h
1 + θ21 + θ22 + ...+ θ2qif 1 ≤ h ≤ q,
and ρ (h) = 0 if h > q.
66
Arthur CHARPENTIER - Sales forecasting.
Moving average process MA(1), order 1
The general expression of MA (1) is
Xt = εt + θεt−1, for all t ∈ Z,
where (εt) is a white noise with variance σ2. Autocorrelations are given by
ρ (1) =θ
1 + θ2, and ρ (h) = 0, for h ≥ 2.
Note that −1/2 ≤ ρ (1) ≤ 1/2 : MA (1) processes only have small
autocorrelations.
Partial autocorrelation of order h is given by
ψ (h) =(−1)h θh
(θ2 − 1
)1− θ2(h+1)
.
67
Arthur CHARPENTIER - Sales forecasting.
A MA(1) process, Xt = εt + 0.7εt−1
Simulated MA(1)
0 100 200 300 400 500
−3
−2
−1
01
23
0 10 20 30 40
0.0
0.2
0.4
0.6
0.8
1.0
Lag
AC
F
MA(1) autocorrelations
0 10 20 30 40
−0.
6−
0.4
−0.
20.
00.
20.
40.
6
Lag
Par
tial A
CF
MA(1) partial autocorrelations
68
Arthur CHARPENTIER - Sales forecasting.
A MA(1) process, Xt = εt − 0.6εt−1
Simulated MA(1)
0 100 200 300 400 500
−3
−2
−1
01
23
0 10 20 30 40
−0.
50.
00.
51.
0
Lag
AC
F
MA(1) autocorrelations
0 10 20 30 40
−0.
6−
0.4
−0.
20.
00.
20.
40.
6
Lag
Par
tial A
CF
MA(1) partial autocorrelations
69
Arthur CHARPENTIER - Sales forecasting.
Autoregressive moving average process ARMA(p, q)
We call autoregressive moving average process of orders p and q, denoted
ARMA (p, q), a stationnary process (Xt) satisfying equation
Xt =p∑j=1
φjXt−j + εt +q∑i=1
θiεt−i for all t ∈ Z, (4)
where the φj 's and θi's are real-valued coe�cients, and process (εt) is a white
noise process with variance σ2. (4) processes can be written equivalently
Φ (L)Xt = Θ (L) εt,
where Φ (L) = I− φ1L− ...− φqLq and Θ (L) = I + θ1L+ ...+ θqLq .
70
Arthur CHARPENTIER - Sales forecasting.
Autoregressive moving average process ARMA(p, q)
Note that under some technical assumptions, one can write
Xt = Φ−1 (L) ◦Θ (L) εt,
i.e. the ARMA(p, q) process is also an MA(∞) process, and
Φ (L) ◦Θ−1 (L)Xt = εt,
i.e. the ARMA(p, q) process is also an AR(∞) process.
Wald's theorem claims that any stationary process (satisfying further technical
conditions) can be written as a MA process.
More generally, in practice, a stationary series can be modeled either by an
AR(p) process, a MA(q), or an ARMA(p′, q′) whith p < p′ and q < q′.
71
Arthur CHARPENTIER - Sales forecasting.
A ARMA(1, 1) process, Xt = 0.7Xt−1εt − 0.6εt−1
Simulated ARMA(1,1)
0 100 200 300 400 500
−2
−1
01
23
0 10 20 30 40
0.0
0.2
0.4
0.6
0.8
1.0
Lag
AC
F
ARMA(1,1) autocorrelations
0 10 20 30 40
−0.
6−
0.4
−0.
20.
00.
20.
40.
6
Lag
Par
tial A
CF
ARMA(1,1) partial autocorrelations
72
Arthur CHARPENTIER - Sales forecasting.
A ARMA(2, 1) process, Xt = 0.7Xt−1 − 0.2Xt−2εt − 0.6εt−1
Simulated ARMA(2,1)
0 100 200 300 400 500
−2
02
4
0 10 20 30 40
0.0
0.2
0.4
0.6
0.8
1.0
Lag
AC
F
ARMA(2,1) autocorrelations
0 10 20 30 40
−0.
6−
0.4
−0.
20.
00.
20.
40.
6
Lag
Par
tial A
CF
ARMA(2,1) partial autocorrelations
73
Arthur CHARPENTIER - Sales forecasting.
Fitting ARMA processes with MSExcel
74
Arthur CHARPENTIER - Sales forecasting.
Forecasting with AR(1) processes
Consider an AR (1) process, Xt = µ+ φXt−1 + εt then
• TX∗T+1 = µ+ φXT ,
• TX∗T+2 = µ+ φ.TX
∗T+1 = µ+ φ [µ+ φXT ] = µ [1 + φ] + φ2XT ,
• TX∗T+3 = µ+ φ.TX
∗T+2 = µ+ φ [µ+ φ [µ+ φXT ]] = µ
[1 + φ+ φ2
]+ φ3XT ,
and recursively TX∗T+h can be written
TX∗T+h = µ+ φ.TX
∗T+h−1 = µ
[1 + φ+ φ2 + ...+ φh−1
]+ φhXT .
or equivalently
TX∗T+h =
µ
φ+ φh
[XT −
µ
φ
]= µ
1− φh
1− φ︸ ︷︷ ︸1+φ+φ2+...+φh−1
+ φhXT .
75
Arthur CHARPENTIER - Sales forecasting.
Forecasting with AR(1) processes
The forecasting error made at time T for horizon h is
T∆h = TX∗T+h −XT+h =T X
∗T+h − [φXT+h−1 + µ+ εT+h]
= ...
= TX∗T+h −
[φh1XT +
(φh−1 + ...+ φ+ 1
)µ
+εT+h + φεT+h−1 + ...+ φh−1εT+1,
(6)
thus, T∆h = εT+h + φεT+h−1 + ...+ φh−1εT+1, with variance having variance
V̂ =[1 + φ2 + φ4 + ...+ φ2h−2
]σ2, where V (εt) = σ2.
thus, variance of the forecast error increasing with horizon.
76