Homework 3

1 Data analysis problems

For the homework, be sure to give full explanations where required and toturn in any relevant plots.

1. Explore the behavior of the ACF and PACF. Simulate a single pathof 10,000 observations for each of the following models.

(a) xt = 0.6xt−1 − 0.8xt−2 + wt

(b) xt = wt + 0.8wt−1 + 1.1wt−2

(c) xt = 0.8xt−1 + wt + 0.8wt−1

Plot the ACF and PACF for each path. Discuss what you see in theplots and if it is what you would expect.

[Sol] The plots and discussion would be as follows:

(a) Since this is an AR(2), we expect an ACF that falls offexponentially and a PACF which falls off quickly afterthe second lag. This is seen in the plots.

(b) Since this is an MA(2), we expect an ACF that falls offafter a lag of two and a PACF which falls off exponen-tially. This is what is observed.

(c) This is an ARMA(1,1), we expect the PACF and theACF to fall off exponentially.

1

Figure 1: 1-(a) ACF and PACF of AR(2)

0 10 20 30 40

-0.5

0.0

0.5

1.0

Series: ar

LAG

ACF

0 10 20 30 40

-0.5

0.0

0.5

1.0

LAG

PACF

Figure 2: 1-(b) ACF and PACF of MA(2)

0 10 20 30 40

-0.4

0.0

0.4

0.8

Series: ma

LAG

ACF

0 10 20 30 40

-0.4

0.0

0.4

0.8

LAG

PACF

2

Figure 3: 1-(b) ACF and PACF of ARMA(1,1)

0 10 20 30 40

-0.5

0.0

0.5

1.0

Series: arma

LAG

ACF

0 10 20 30 40

-0.5

0.0

0.5

1.0

LAG

PACF

3

2. Load the Berkeley and Santa Barbara temperature data using thefollowing commands

berk=scan("berkeley.dat", what=list(double(0),double(0),double(0)))

time=berk[[1]]

berkeley=berk[[2]]

stbarb=berk[[3]]

See if berkeley and stbarb are stationary. If not, fix the nonsta-tionarity with the first order differencing. Do you have an opinion onpossible models based only on these plots?

[Sol] The plots are seen below. For berkeley and stbarb, inboth plots (left), there is a trend in the series and the mean isnot constant. It means that the data is not very stationary inappearance. So we need to take the first order differencing.From the plots below (right), it appears that both differencedseries have constant means.

Figure 4: 2 - time series plots for berkeley and stbarb (left), and theirdifferenced series (right)

Time

berkeley

0 20 40 60 80 100

12.5

13.5

14.5

Time

dberk

0 20 40 60 80 100

-1.0

0.0

0.5

1.0

1.5

Time

stbarb

0 20 40 60 80 100

1415

1617

18

Time

dstbarb

0 20 40 60 80 100

-1.0

0.0

1.0

4

Figure 5: 2 - ACF and PACF of berkeley

5 10 15 20

-0.2

0.2

0.6

1.0

Series: berkeley

LAG

ACF

5 10 15 20

-0.2

0.2

0.6

1.0

LAG

PACF

Figure 6: 2 - ACF and PACF of stbarb

5 10 15 20

-0.2

0.2

0.6

1.0

Series: stbarb

LAG

ACF

5 10 15 20

-0.2

0.2

0.6

1.0

LAG

PACF

5

For the differenced data dberk=diff(berkeley) and dstbarb=diff(stbarb),see Figures below. Since ACF cuts off at lag 1 and PACFtails off, so a possible model for dberkeley is MA(1); simi-larly, a possible model for dstbarb is MA(1) (Other modelscan also be correct and your proposed model need to be welldefensed.)

Figure 7: 2 - ACF and PACF of differenced berkeley

5 10 15 20

-0.5

0.0

0.5

1.0

Series: dberk

LAG

ACF

5 10 15 20

-0.5

0.0

0.5

1.0

LAG

PACF

Figure 8: 2 - ACF and PACF of differenced berkeley

5 10 15 20

-0.5

0.0

0.5

1.0

Series: dstbarb

LAG

ACF

5 10 15 20

-0.5

0.0

0.5

1.0

LAG

PACF

6

3. Load the global average temperature using the command

temperature=scan("globtemp.dat")

See if temperature is stationary. If not, fix the nonstationarity withthe first order differencing. Do you have an opinion on possible modelsbased on these plots?

[Sol] Similar to the previous exercise, from the plots it ap-pears to be non-stationary for the original series. Differenc-ing in needed here. For the differenced data,ACF cuts off atlag 2 and PACF tails off. So a possible candidate model isMA(2).(You need to defense your model in order to get a full credit.)

Figure 9: 3 - time series plots for temperature (left), and its differencedseries (right)

Time

temperature

0 20 60 100 140

-0.4

-0.2

0.0

0.2

0.4

Time

dtemp

0 20 60 100 140

-0.3

-0.1

0.1

0.3

7

Figure 10: 3 - ACF and PACF of temperature

5 10 15 20

-0.2

0.2

0.6

1.0

Series: temperature

LAG

ACF

5 10 15 20

-0.2

0.2

0.6

1.0

LAG

PACF

Figure 11: 3 - ACF and PACF of differenced temperature

5 10 15 20

-0.2

0.2

0.6

1.0

Series: dtemp

LAG

ACF

5 10 15 20

-0.2

0.2

0.6

1.0

LAG

PACF

8

4. Verify Example 3.5 (pages 91-92) from the textbook. Simulate the twoMA models of length 10,000 using your newly created function masim

and the two sets of parameters in this example. Now, fit those twoMA models using the command arima(x,c(0,0,1)) where x is thename of your time series. (We’ll talk more about this command later)Does this confirm what was shown in theory?

[Sol] For this problem, they should obtain something like

> x1=masim(c(5),1,10000)

>

> x2=masim(c(1/5),25,10000)

>

> arima(x1, c(0,0,1))

Call:

arima(x = x1, order = c(0, 0, 1))

Coefficients:

ma1 intercept

0.2095 0.0378

s.e. 0.0101 0.0600

sigma^2 estimated as 24.60: log likelihood = -30202.29, aic = 60410.59

>

> arima(x2, c(0,0,1))

Call:

arima(x = x2, order = c(0, 0, 1))

Coefficients:

ma1 intercept

0.2122 0.0117

s.e. 0.0099 0.0608

sigma^2 estimated as 25.2: log likelihood = -30323.32, aic = 60652.65

>

The important thing to notice was that while the parametersused to simulate are quite different R fits the same model. Rfinds the invertible parameters.

9

2 Theoretical problems

1. Check if there are any parameter redundancies. It there are, thenremove it. Determine whether the removed models are causal and/orinvertible.(Refer to the example 3.7 in the textbook)

(a) xt = 0.6xt−1 − 0.08xt−2 + wt − 0.7wt−1 + 0.1wt−2

[Sol] The ar polynomial is φ(z) = (1 − 0.2z)(1 − 0.4z) andthe ma polynomial is θ(z) = (1− 0.2z)(1− 0.5z). Since thear and ma polynomials for this model share a commonroot 5, there is a parameter redundancy. After removingit, the root of ar polynomial is greater than 1 and theroot of the ma polynomial is also greater than 1. So theremoved model is both causal and invertible.

(b) xt = −0.2xt−1 + 0.15xt−2 + wt + 2.5wt−1 + wt−2

[Sol] The ar polynomial is φ(z) = (1 − 0.3z)(1 + 0.5z) andthe ma polynomial is θ(z) = (1 + 2z)(1 + 0.5z). Since the arand ma polynomials for this model share a common root-2, there is a parameter redundancy. After removing it,the root of ar polynomial is greater than 1 and the rootof the ma polynomial is smaller than 1. So the removedmodel is causal but not invertible.

(c) xt = xt−1 + wt − 0.5wt−1 − 0.5wt−2

[Sol] The AR polynomial is φ(z) = 1−z, which has root 1.the MA polynomial is θ(z) = 1−z/2−z2/2, which has roots-2 and 1. since these polynomials share a common root,they have the common factor 1− z. Factoring these out,the reduced representation has AR polynomial φ(z) =1(which has no roots) and MA polynomial θ(z) = 1 + z/2(which has root -2). thus, this is an ARMA(0,1) process(or MA(1) process) which is causal and invertible.

(d) xt = 3xt−1 + wt + 2wt−1 − 8wt−2

[Sol] The AR polynomial is φ(z) = 1− 3z, which has root1/3. the MA polynomial is θ(z) = 1 + 2z − 8z2, which hasroots -1/4 and 1/2. Thus, this is an ARMA(1,2) processwhich is neither causal nor invertible.

2. Verify that for an MA(1) process, xt = wt+θ1wt−1 with wt ∼ wn(0, σ2w),

max−∞<θ1<∞

ρ(1) = 0.5, min−∞<θ1<∞

ρ(1) = 0.5

10

(Hint : calculate the autocorrelation at lag 1, ρ(1), and take the firstderivative of ρ(1) with respect to θ1 to see the maximum and mini-mum).

[Sol] Recall that ρ(1) = θ11+θ21

so that ∂ρ(1)∂θ1

=−θ21+1

(1+θ21)2 which is seen

to be positive on the range −1 < θ1 < 1 and zero at both -1 and+1. For | θ1 |> 1, the derivative is negative. Taken together,these facts imply the desired results. Or you can show thatthe second derivative evaluated at θ1 = −1 is positive and thesecond derivative evaluated at θ1 = 1 is negative.

3. For the ARMA(1,2) process, xt = 0.8xt−1+wt+0.7wt−1+0.6wt−2 withwt ∼ wn(0, σ2w), show that ρ(k) = 0.8ρ(k − 1) for k > 2 under the as-sumption that the mean of the series is zero (Hint : it is enough to showγ(k) = 0.8γ(k − 1) for k > 2 and use E(xt−1xt−k) = E(xtxt−(k−1)))

[Sol]

cov(xt, xt−k) = E[(0.8xt−1 + wt + 0.7wt−1 + 0.6wt−2)xt−k]

= 0.8E(xt−1xt−k) + E(wtxt−k) + 0.7E(wt−1xt−k) + 0.6E(wt−2xt−k)

= 0.8E(xt−1xt−k) because k > 2

= 0.8E(xtxt−(k−1)) = 0.8cov(xt, xt−(k−1)).

Thus, γ(k) = 0.8γ(k − 1) and ρ(k) = 0.8ρ(k − 1).

4. Consider the AR(1) process xt = φ1xt−1 + wt with wt ∼ wn(0, σ2w).Show that if | φ1 |= 1, the process cannot be stationary.

[Sol] Assume that {xt} is stationary. Then var(xt) = φ21var(xt−1)+

var(wt) and σ2y = φ21σ2y + σ2w or σ2y = σ2

w

1−φ21. If | φ1 |= 1, this is

impossible and we have a proof by contradiction.

5. Consider a process that satisfies the zero-mean, “stationary” AR(1)equation xt = φ1xt−1 + wt with wt ∼ wn(0, σ2w) and | φ1 |< 1. Let cbe any nonzero constant, and define yt = xt + cφt1.

1) Show that E(yt) = cφt1.

[Sol] E(yt) = E(xt + cφt1) = E(xt) + cφt1 = cφt1.

2) Is {yt} stationary?

11

[Sol] Since E(yt) = cφt1 is not constant in time, the solution isnot stationary.

3) Show that {yt} satisfies the “stationary” AR(1) equation yt =φ1yt−1 + wt.

[Sol] yt = xt + cφt1 = φ1xt−1 + wt + cφt1 = φ1(xt−1 + cφt−11 ) + wt =φ1yt−1 + wt.

12