regression with autocorrelated...
TRANSCRIPT
OLS Regression Auto-correlated Models Regression with Autocorrelated Errors
Regression With Autocorrelated ErrorsEDU 7309 Project
Xiaowen Hu & Wenkai Bao
Southern Methodist University
Apr. 7th, 2010
Xiaowen Hu & Wenkai Bao Regression With Autocorrelated Errors
OLS Regression Auto-correlated Models Regression with Autocorrelated Errors
Linear Regression Model
Settings and Assumptions
Linear Regression Model
yi = β0 + β1xi1 + . . . + βkxik + εi ,
where yi , xi1, . . . , xik are observations of k + 1 variables, andεi
iid∼ N(0, σ2).
E(εi) = 0 for i = 1, . . . , nVar(εi) = σ2 for i = 1, . . . , ncov(εi , εj) = E(εiεj) = 0 for i 6= j
Xiaowen Hu & Wenkai Bao Regression With Autocorrelated Errors
OLS Regression Auto-correlated Models Regression with Autocorrelated Errors
Linear Regression Model
Least Square Regression
β0, . . . , βk unknownResidual ei = yi − β̂0 − β̂1xi1 − . . .− β̂kxik
β̂0, . . . , β̂k minimizes∑n
i=1 e2i
Xiaowen Hu & Wenkai Bao Regression With Autocorrelated Errors
OLS Regression Auto-correlated Models Regression with Autocorrelated Errors
Autocorrelated Errors
Relaxing The Assumptions
What if cov(εi , εj) 6= 0?More specific, autocorrelation among errorsThis may occur when
Missing true explanatory variablesMisspecification of models (linear vs. quadratic)Pure correlated errors (true autocorrelation)
β̂0, . . . , β̂k are still unbiasedthat is, expectations of β̂’s are β’s.
Xiaowen Hu & Wenkai Bao Regression With Autocorrelated Errors
OLS Regression Auto-correlated Models Regression with Autocorrelated Errors
Autocorrelated Errors
Relaxing The Assumptions
However...Variance of errors may be underestimatedVariance of β̂’s may be underestimatedConfidence intervals may not be applicableSpurious regressione.g. two uncorrelated variables may appear related.
Xiaowen Hu & Wenkai Bao Regression With Autocorrelated Errors
OLS Regression Auto-correlated Models Regression with Autocorrelated Errors
Autocorrelated Errors
Remedial Options
Add predictorsHigher order predictorsTransform variablesCochrane-Orcutt (1949)
Xiaowen Hu & Wenkai Bao Regression With Autocorrelated Errors
OLS Regression Auto-correlated Models Regression with Autocorrelated Errors
ARMA Models
Basic Definitions
Time series processA collection of random variables Xt ’s. i.e. {Xt}, where t istime index.Autocovariance
γ(t1, t2) = cov(Xt1 , Xt2)
Autocorrelation
ρ(t1, t2) =γ(t1, t2)
σ(Xt1)σ(Xt2)
White noiseA type of time series that
Xt ’s are identically distributedγ(t1, t2) = 0γ(t , t) = σ2
Xiaowen Hu & Wenkai Bao Regression With Autocorrelated Errors
OLS Regression Auto-correlated Models Regression with Autocorrelated Errors
ARMA Models
Autoregressive Models
AR(p) ModelAn autoregressive model of order p is
Xt − µ− φ1(Xt−1 − µ)− . . .− φp(Xt−p − µ) = wt ,
where µ is mean, and wt is white noise.
Xiaowen Hu & Wenkai Bao Regression With Autocorrelated Errors
OLS Regression Auto-correlated Models Regression with Autocorrelated Errors
ARMA Models
Operator Form of AR Models
Introduce backward shift operator0.8BXt = 0.8Xt−1, 1.2B2Xt = 1.2Xt−2Similar properties as algebraic counterparts(0.3B + 1.6B)Xt = 1.9BXt = 1.9Xt−1
Rewrite AR(p) model as
(1− φ1B − . . .− φpBp)(Xt − µ) = wt .
Denote it simply as φ(B)(Xt − µ) = wt
Xiaowen Hu & Wenkai Bao Regression With Autocorrelated Errors
OLS Regression Auto-correlated Models Regression with Autocorrelated Errors
ARMA Models
AR(1) Examples
Xt − 0.9Xt−1 = wt , Xt + 0.9Xt−1 = wt
realization of AR(1) with positive phi
Time
0 20 40 60 80 100
−6
−4
−2
02
realization of AR(1) with negative phi
Time
0 20 40 60 80 100
−4
−2
02
4
Xiaowen Hu & Wenkai Bao Regression With Autocorrelated Errors
OLS Regression Auto-correlated Models Regression with Autocorrelated Errors
ARMA Models
Moving Average Models
MA(q) ModelA moving average model of order q is
Xt − µ = wt − θ1wt−1 − . . .− θqwt−q,
where µ is mean, and wt is white noise.
Xiaowen Hu & Wenkai Bao Regression With Autocorrelated Errors
OLS Regression Auto-correlated Models Regression with Autocorrelated Errors
ARMA Models
Operator Form of MA Models
Rewrite MA(q) model as
Xt − µ = (1− θ1B − . . .− θqBq)wt .
Denote it simply as Xt − µ = θ(B)wt
Xiaowen Hu & Wenkai Bao Regression With Autocorrelated Errors
OLS Regression Auto-correlated Models Regression with Autocorrelated Errors
ARMA Models
MA(1) Examples
Xt = wt − 0.9wt−1, Xt = wt + 0.9wt−1
realization of MA(1) with positive theta
Time
0 20 40 60 80 100
−3
−1
12
3
realization of MA(1) with negative theta
Time
0 20 40 60 80 100
−4
−2
02
Xiaowen Hu & Wenkai Bao Regression With Autocorrelated Errors
OLS Regression Auto-correlated Models Regression with Autocorrelated Errors
ARMA Models
Stationarity
Stationary Processes
A process {Xt}, t = 0,±1,±2, . . . is stationary ifµt is constantσ2
t is constantcov(Xt1 , Xt2) only depends on |t1 − t2|
All MA(q) models are stationaryNot all AR(p) models are stationaryStationary AR(p) models can be written as MA(∞) form
Xiaowen Hu & Wenkai Bao Regression With Autocorrelated Errors
OLS Regression Auto-correlated Models Regression with Autocorrelated Errors
ARMA Models
Invertibility
Invertible MA ProcessesAn MA process is invertible if it can be written as AR(∞) form
(1− φ1B − φ2B2 − . . .)(Xt − µ) = wt
All AR(p) models are invertibleNot all MA(q) models are invertible
Xiaowen Hu & Wenkai Bao Regression With Autocorrelated Errors
OLS Regression Auto-correlated Models Regression with Autocorrelated Errors
ARMA Models
ARMA(p, q) Models
ARMA(p, q) ModelA time series is ARMA model of order p, q if it is stationary,invertible, and
Xt−µ−φ1(Xt−1−µ)−. . .−φp(Xt−p−µ) = wt−θ1wt−1−. . .−θqwt−q
A combination of AR model and MA modelOperator formφ(B)(Xt − µ) = θ(B)wt
Xiaowen Hu & Wenkai Bao Regression With Autocorrelated Errors
OLS Regression Auto-correlated Models Regression with Autocorrelated Errors
ARMA Models
ARMA(1,1) Examples
Xt − 0.9Xt−1 = wt − 0.7wt−1, Xt + 0.9Xt−1 = wt − 0.7wt−1
realization of ARMA(1,1) with +phi & +theta
Time
0 20 40 60 80 100
−3
−1
01
2
realization of ARMA(1,1) with −phi & +theta
Time
0 20 40 60 80 100
−1
00
51
0
Xiaowen Hu & Wenkai Bao Regression With Autocorrelated Errors
OLS Regression Auto-correlated Models Regression with Autocorrelated Errors
ARMA Models
Models Considered
More general: ARUMA, ARIMAe.g. random walkInfinite varianceOnly considering stationary and invertible modelsρ(t − k , t) = ρ(k) for any t
Xiaowen Hu & Wenkai Bao Regression With Autocorrelated Errors
OLS Regression Auto-correlated Models Regression with Autocorrelated Errors
ARMA Models
ACF Plots: MA(1)
Xt = wt − 0.9wt−1, Xt = wt + 0.9wt−1
realization of MA(1) with positive theta
Time
0 20 40 60 80 100
−3
−1
12
3
0 20 40 60 80 100
−1
.00
.00
.51
.0
true autocorrelation of MA(1) with positive theta
lag
au
toco
rre
latio
n
0 20 40 60 80 100
−1
.00
.00
.51
.0
sample autocorrelation of MA(1) with positive theta
lag
au
toco
rre
latio
n
realization of MA(1) with negative theta
Time
0 20 40 60 80 100
−4
−2
01
23
0 20 40 60 80 100
−1
.00
.00
.51
.0
true autocorrelation of MA(1) with negative theta
laga
uto
co
rre
latio
n
0 20 40 60 80 100
−1
.00
.00
.51
.0
sample autocorrelation of MA(1) with negative theta
lag
au
toco
rre
latio
n
Xiaowen Hu & Wenkai Bao Regression With Autocorrelated Errors
OLS Regression Auto-correlated Models Regression with Autocorrelated Errors
ARMA Models
ACF Plots: AR(1)
Xt − 0.9Xt−1 = wt , Xt + 0.9Xt−1 = wt
realization of AR(1) with positive phi
Time
0 20 40 60 80 100
−6
−4
−2
02
0 20 40 60 80 100
−1
.00
.00
.51
.0
true autocorrelation of AR(1) with positive phi
lag
au
toco
rre
latio
n
0 20 40 60 80 100
−1
.00
.00
.51
.0
sample autocorrelation of AR(1) with positive phi
lag
au
toco
rre
latio
n
realization of AR(1) with negative phi
Time
0 20 40 60 80 100
−4
−2
02
4
0 20 40 60 80 100
−1
.00
.00
.51
.0
true autocorrelation of AR(1) with negative phi
laga
uto
co
rre
latio
n
0 20 40 60 80 100
−1
.00
.00
.51
.0
sample autocorrelation of AR(1) with negative phi
lag
au
toco
rre
latio
n
Xiaowen Hu & Wenkai Bao Regression With Autocorrelated Errors
OLS Regression Auto-correlated Models Regression with Autocorrelated Errors
ARMA Models
ACF Plots: ARMA(1,1)
Xt − 0.9Xt−1 = wt − 0.7wt−1, Xt + 0.9Xt−1 = wt − 0.7wt−1
realization of ARMA(1,1) with +phi & +theta
Time
0 20 40 60 80 100
−3
−1
01
2
0 20 40 60 80 100
−1
.00
.00
.51
.0
true autocorrelation of ARMA(1,1) with +phi & +theta
lag
au
toco
rre
latio
n
0 20 40 60 80 100
−1
.00
.00
.51
.0
sample autocorrelation of ARMA(1,1) with +phi & +theta
lag
au
toco
rre
latio
n
realization of ARMA(1,1) with −phi & +theta
Time
0 20 40 60 80 100
−1
00
51
0
0 20 40 60 80 100
−1
.00
.00
.51
.0
true autocorrelation of ARMA(1,1) with −phi & +theta
laga
uto
co
rre
latio
n
0 20 40 60 80 100
−1
.00
.00
.51
.0
sample autocorrelation of ARMA(1,1) with −phi & +theta
lag
au
toco
rre
latio
n
Xiaowen Hu & Wenkai Bao Regression With Autocorrelated Errors
OLS Regression Auto-correlated Models Regression with Autocorrelated Errors
ARMA Models
PACF Plots
ACF plots identify the order of MA modelTo identify orders of AR(MA) modelsUse partial ACF plots
PACF ρ∗(k)
is the correlation coefficient between Xt and Xt−k with the lineareffect of {Xt−1, . . . , Xt−(k−1)}, on each, removed.
Xiaowen Hu & Wenkai Bao Regression With Autocorrelated Errors
OLS Regression Auto-correlated Models Regression with Autocorrelated Errors
ARMA Models
PACF Plots
5 10 15 20
−1
.00
.00
.51
.0
Lag
Pa
rtia
l AC
F
partial sample autocorrelation of AR(1) with positive phi
5 10 15 20
−1
.00
.00
.51
.0
Lag
Pa
rtia
l AC
F
partial sample autocorrelation of AR(1) with negative phi
5 10 15 20
−1
.00
.00
.51
.0
Lag
Pa
rtia
l AC
F
partial sample autocorrelation of MA(1) with positive phi
5 10 15 20
−1
.00
.00
.51
.0
LagP
art
ial A
CF
partial sample autocorrelation of MA(1) with negative phi
5 10 15 20
−1
.00
.00
.51
.0
Lag
Pa
rtia
l AC
F
partial sample autocorrelation of ARMA(1,1) with +phi & +theta
5 10 15 20
−1
.00
.00
.51
.0
Lag
Pa
rtia
l AC
F
partial sample autocorrelation of ARMA(1,1) with −phi & +theta
Xiaowen Hu & Wenkai Bao Regression With Autocorrelated Errors
OLS Regression Auto-correlated Models Regression with Autocorrelated Errors
Detecting Autocorrelated Errors
Graphical Check
Residual plotACF/PACF plots
AR(p) MA(q) ARMA(p, q)ACF Tails off Cuts off Tails off
after lag qPACF Cuts off Tails off Tails off
after lag p
Xiaowen Hu & Wenkai Bao Regression With Autocorrelated Errors
OLS Regression Auto-correlated Models Regression with Autocorrelated Errors
Detecting Autocorrelated Errors
Objective Tests
Durbin-Watson d test
Durbin-Watson statistics
Dk =
∑nt=k+1(et − et−k )2∑n
t=1 e2t
Dk ≈ 2(1− ρk )
Limited to AR(1) models
The runs testRun: an uninterrupted sequence of + or - signs of theresiduals
Xiaowen Hu & Wenkai Bao Regression With Autocorrelated Errors
OLS Regression Auto-correlated Models Regression with Autocorrelated Errors
Theories
OLS Regression
Modelyt = β′xt + εt , t = 1, 2, . . . , n
Matrix formy = Xβ + ε,
where X = [x1, . . . , xn]′, ε= (ε1, . . . , εn)′ with
variance-covariance matrix Γ= {γ(s, t)}
Xiaowen Hu & Wenkai Bao Regression With Autocorrelated Errors
OLS Regression Auto-correlated Models Regression with Autocorrelated Errors
Theories
ARMA Transformation
Suppose {εt} follows ARMA model
φ(B)εt = θ(B)wt ,
where {wt} is white noiseMatrix form
φ(B)
θ(B)ε = w ,
where ε= (ε1, . . . , εn)′, w= (w1, . . . , wn)
′
TransformationMultiply φ(B)
θ(B) on both sides of y = Xβ + ε,
φ(B)
θ(B)y =
φ(B)
θ(B)Xβ + w
Independent error term assumption is satisfied
Xiaowen Hu & Wenkai Bao Regression With Autocorrelated Errors
OLS Regression Auto-correlated Models Regression with Autocorrelated Errors
Theories
Cochrane and Orcutt Algorithm
Obtain residuals via OLS routine
yt = β̂′xt + et
Fit an ARMA model to et
φ̂(B)et = θ̂(B)wt
Apply ARMA transformation to linear model
φ̂(B)
θ̂(B)yt = β̂′ φ̂(B)
θ̂(B)xt + wt ,
denoted as ut =β̂′vt+wt
Run OLS regression again on transformed model
Xiaowen Hu & Wenkai Bao Regression With Autocorrelated Errors
OLS Regression Auto-correlated Models Regression with Autocorrelated Errors
Illustrative Example
Pollution, Temperature, Mortality Example
SettingsWeekly data 1970 through 1979 in Los AngelesCardiovascular Mortality (Mt ), Particulates (Pt ),Temperature (Tt )
Xiaowen Hu & Wenkai Bao Regression With Autocorrelated Errors
OLS Regression Auto-correlated Models Regression with Autocorrelated Errors
Illustrative Example
Pollution, Temperature, Mortality Example
DataCardiovascular Mortality
Time
mor
t
0 100 200 300 400 500
7090
110
130
Temperature
Time
tem
p
0 100 200 300 400 500
−20
010
20
Particulates
Time
part
0 100 200 300 400 500
2040
6080
100
Xiaowen Hu & Wenkai Bao Regression With Autocorrelated Errors
OLS Regression Auto-correlated Models Regression with Autocorrelated Errors
Illustrative Example
Pollution, Temperature, Mortality Example
Correlations
mort
50 60 70 80 90 100
7080
9010
011
012
013
0
5060
7080
9010
0
temp
70 80 90 100 110 120 130 20 40 60 80 100
2040
6080
100
part
Xiaowen Hu & Wenkai Bao Regression With Autocorrelated Errors
OLS Regression Auto-correlated Models Regression with Autocorrelated Errors
Illustrative Example
Obtain OLS Residuals
Models consideredMt = β0 + β1t + εt
Mt = β0 + β1t + β2(Tt − T.) + εt
Mt = β0 + β1t + β2(Tt − T.) + β3(Tt − T.)2 + εt
Mt = β0 + β1t + β2(Tt − T.) + β3(Tt − T.)2 + β4Pt + εt
Xiaowen Hu & Wenkai Bao Regression With Autocorrelated Errors
OLS Regression Auto-correlated Models Regression with Autocorrelated Errors
Illustrative Example
Obtain OLS Residuals
> fit <‐ lm(mort~ trend + temp + temp2 + part, na.action=NULL) > summary(fit) Call: lm(formula = mort ~ trend + temp + temp2 + part, na.action = NULL) Residuals: Min 1Q Median 3Q Max ‐19.0760 ‐4.2153 ‐0.4878 3.7435 29.2448 Coefficients: Estimate Std. Error t value Pr(>|t|) (Intercept) 81.592238 1.102148 74.03 < 2e‐16 *** trend ‐0.026844 0.001942 ‐13.82 < 2e‐16 *** temp ‐0.472469 0.031622 ‐14.94 < 2e‐16 *** temp2 0.022588 0.002827 7.99 9.26e‐15 *** part 0.255350 0.018857 13.54 < 2e‐16 *** ‐‐‐ Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
Residual standard error: 6.385 on 503 degrees of freedom Multiple R‐squared: 0.5954, Adjusted R‐squared: 0.5922 F‐statistic: 185 on 4 and 503 DF, p‐value: < 2.2e‐16
Xiaowen Hu & Wenkai Bao Regression With Autocorrelated Errors
OLS Regression Auto-correlated Models Regression with Autocorrelated Errors
Illustrative Example
Obtain OLS Residuals
80 90 100 110 120
0.0
0.5
1.0
1.5
2.0
Fitted values
Sta
nd
ard
ize
d r
esid
ua
ls
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●●
●● ●
●
●
●
●●
● ●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
● ●
●
●●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●●
●
●
●
●
●
●●
●●
●
●
●
●
●●
●
●
●●
●
●
●●●
●
●
●●
●
●
●
●
●
●●
●●●
●
●
●
●
●●
●
●●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●● ●
●
●
●
●
● ●
● ●
●
●
●●
●
●
●
●
●
●
●
●● ●
●●
●●
●
●
●
●
●
●●
●
●
● ●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
● ●
●
●
●
●
●
●●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●●
●
●●
●●
●
●●
●
●
●
● ●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●●
●
● ●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
● ●●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●●
●●
●
●
●
●●
●
●
●
●
●●
●●
●●
● ●
●
●●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●●
●
●
●●
●
●
●
●
●●●●
●
●
●
●
●●
●
Scale−Location152
154257
●
●
●
●
●●
●
●●
●
●● ●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●●
●
●●
●
●
●
●
●
●
●●●
●
●
●
●
●
●●●
●
●
●
●●
●●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●●●
●
●
●
●●●
●●●
●
●
●
●
●
●
●
●
● ●●
●
●
●●
●●
●
●
●
●●●
●
●
●
● ●●● ●
●
●
●
●
●● ●●
●
●
●●
●
●
● ●
●
●●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●●
●●
●●
●
●
●
●
●
●●
●
●●
●●
●
●
●
●●●
●
●
●●
●●
●
●
●
●● ●
●
●
●
●
●
●
●
●
●●
● ●
●●●
●
●
●
●●●
●
●
●●
●
●
●
●
●
●
●●
●●
●
●
●
●●
●
●
●●●●
●
●
●
●●
●●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
● ●●
●
●
●
●
●
●
●●
● ●
●●●
●●
●●●
●
●
●
● ●
●●
●
●
●
●●
●
●
●●
●
●
●●●
●●
●
●
●
● ●
●
●
●
●●
●
●
● ●
●●
●●
●
●
●
●
●●
●
●
●
●
●
●●
●●● ●
●●
●
●
●
●
●
●●
●
●
●●
●
●●● ●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●
●●●
●
●●
●
●
●
●
●
●
●● ●
●●
●
●
●●●
●
●●
●
●
●
●
●
●
●
●●
●
●●
●
●●
● ●
●●
●
●
●●●●●
●
●
●●
●
●●
●
●●
●
●
●
●
●
●●●●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●●
●●
●
●
●
●
●
●
●
● ●
●
●●
●
● ●
●
●
●
●● ●
●●●
●
●
●
●
●
●
−3 −2 −1 0 1 2 3
−2
02
4
Theoretical Quantiles
Sta
nd
ard
ize
d r
esid
ua
ls
Normal Q−Q
152
154257
0 5 10 15 20 25
0.0
0.2
0.4
0.6
0.8
1.0
Lag
AC
F
ACF of residuals
0 5 10 15 20 25
−0
.10
.00
.10
.20
.3
Lag
Pa
rtia
l A
CF
PACF of residuals
Xiaowen Hu & Wenkai Bao Regression With Autocorrelated Errors
OLS Regression Auto-correlated Models Regression with Autocorrelated Errors
Illustrative Example
Fit an AR(2) model to et
> (fit2<‐ar.ols(fit$resid, aic=F,order=2)) Call: ar.ols(x = fit$resid, aic = F, order.max = 2) Coefficients: 1 2 0.2205 0.3625 Intercept: ‐0.002895 (0.2472) Order selected 2 sigma^2 estimated as 30.92
Xiaowen Hu & Wenkai Bao Regression With Autocorrelated Errors
OLS Regression Auto-correlated Models Regression with Autocorrelated Errors
Illustrative Example
Apply ARMA transformation to linear model
> Mort<‐filter(mort, c(1,‐.2205,‐.3625),sides=1)[3:508] > Trend<‐filter(trend, c(1,‐.2205,‐.3625),sides=1)[3:508] > Temp<‐filter(temp, c(1,‐.2205,‐.3625),sides=1)[3:508] > Temp2<‐filter(temp2, c(1,‐.2205,‐.3625),sides=1)[3:508] > Part<‐filter(part, c(1,‐.2205,‐.3625),sides=1)[3:508]
Xiaowen Hu & Wenkai Bao Regression With Autocorrelated Errors
OLS Regression Auto-correlated Models Regression with Autocorrelated Errors
Illustrative Example
Fit OLS Regression on Transformed Model
> fit3 = lm(Mort~ Trend + Temp + Temp2 + Part) > summary(fit3) Call: lm(formula = Mort ~ Trend + Temp + Temp2 + Part) Residuals: Min 1Q Median 3Q Max ‐17.4256 ‐3.4915 ‐0.3200 3.0912 17.9067 Coefficients: Estimate Std. Error t value Pr(>|t|) (Intercept) 34.835498 0.672217 51.822 < 2e‐16 *** Trend ‐0.027775 0.003861 ‐7.193 2.32e‐12 *** Temp ‐0.196162 0.038710 ‐5.067 5.68e‐07 *** Temp2 0.016758 0.002210 7.582 1.66e‐13 *** Part 0.229008 0.022589 10.138 < 2e‐16 *** ‐‐‐ Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 Residual standard error: 5.281 on 501 degrees of freedom Multiple R‐squared: 0.3068, Adjusted R‐squared: 0.3012 F‐statistic: 55.43 on 4 and 501 DF, p‐value: < 2.2e‐16
Xiaowen Hu & Wenkai Bao Regression With Autocorrelated Errors
OLS Regression Auto-correlated Models Regression with Autocorrelated Errors
Illustrative Example
Fit OLS Regression on Transformed Model
30 35 40 45 50 55
0.0
0.5
1.0
1.5
Fitted values
Sta
nd
ard
ize
d r
esid
ua
ls
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
● ●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
● ●
●
●
●●
●●
● ●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
● ●
●●● ●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
● ●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●●
●
●
●
●
●●
●●●
●
●●
●
●
●
●●
●
●
●
●
●
●
● ●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●● ●
●
●
●
●●
●
●
●
●
●
●
●
● ●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●●
●
●
●
●
●
●
Scale−Location14989148
●
●●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●●
●●
●●
●
● ●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
● ●●●
●
●
●
●●
●
●
●
●
●
●●
●●
●●
●
●
●
●
●●
●
●
●
●
●●
● ●●
●
●
●
●
●●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●●
●●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●●
●●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
● ●●●
●
●
●●●
●
●●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●●●
●●
●●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●●
●
●●
●●
●
●
●●
●●
●
●
●
● ●
●
●●●
●
●
●●●
●
●
●
●
●
●
●●
●
●●
●
●
● ●●
●
●●
●
●
●●
●
●
●●●
●●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
● ●
●
●●
●●
●●
●
●
●
●●
●●
●
●
●●
●
●
●
●
● ●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●●
●
●
●
●●●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●●●
●
●
●
●
●
●
−3 −2 −1 0 1 2 3
−2
02
4
Theoretical Quantiles
Sta
nd
ard
ize
d r
esid
ua
ls
Normal Q−Q
149
89
148
0 5 10 15 20 25
0.0
0.2
0.4
0.6
0.8
1.0
Lag
AC
F
ACF of residuals after transformation
0 5 10 15 20 25
−0
.10
.00
.10
.20
.3
Lag
Pa
rtia
l A
CF
PACF of residuals after transformation
Xiaowen Hu & Wenkai Bao Regression With Autocorrelated Errors
OLS Regression Auto-correlated Models Regression with Autocorrelated Errors
Illustrative Example
Obtain OLS Residuals
##compare parameter estimates > rbind(summary(fit)$coefficients[,1], summary(fit3)$coefficients[,2]) (Intercept) trend temp temp2 part [1,] 81.59224 ‐0.02684 ‐0.47247 0.02259 0.25535 [2,] 0.67222 0.00386 0.03871 0.00221 0.02259 ##compare parameter standard errors > rbind(summary(fit)$coefficients[,2], summary(fit3)$coefficients[,2]) (Intercept) trend temp temp2 part [1,] 1.10215 0.00194 0.03162 0.00283 0.01886 [2,] 0.67222 0.00386 0.03871 0.00221 0.02259
Xiaowen Hu & Wenkai Bao Regression With Autocorrelated Errors