Model Parameter Stability

2

Problem caused by breaks• Recall that the below equation can be thought of as a condition

expectation equation or an estimation of average relationships between variables.

• If a break occurs in this model during the sample period, then the OLS regression estimated over the full sample will estimate a relationship that holds on average, in the sense that the estimate combines the two different periods.

• Depending on the location and size of the break, the “average” regression function can be quite different from the true regression function.

• This leads to poor forecasts.

3

Parameter Stability testing• So far we have estimated regression of the type:

• Where the implicit assumption is the parameters (βs) are constant over the entire sample.

• In financial time series large fluctuations are common possibly invalidating this assumption.

• We can test this assumption using parameter stability tests.

• If the break point is known the Chow test can be used.

• The idea is to split the data into sub-periods and estimate up to three models, for each of the sub-parts and the whole sample and then compare the RSS from the models.

• The test can be simplified by using dummy variables to categorize the sub-periods around the known break point.

How do we decide the sub-parts to use?

• As a rule of thumb, we could use all or some of the following:

1. Plot the dependent variable over time and split the data accordingly to any obvious structural changes in the series.

2. Split the data according to any known important historical events (e.g. stock market crash, new government elected)

– Intuition here is that a major change in the underlying environment in which y is measured will cause a structural change in the model’s parameters.

4

5

Steps in performing the Chow test1. Split the data into two sub-periods.

2. Use a dummy variable to categorise these two periods.

3. Create interaction terms using this dummy variable for each explanatory variable in the model

4. Run a regression ignoring the categorisation. This is a pooled regression using the whole sample without the dummy or interaction terms (This would be the restricted model).

5. Run a second regression using the whole sample but now including a dummy variable and all the interaction terms (a fully interacted model) to distinguished between sub periods. – (This is the unrestricted model; you are allowing for the possibility of two sub periods in the model).

6. We can thus form an F-test which is the difference between the RSS’s.

6

Steps in performing the Chow test

• T=number of observations k=number of parameters who stability are being tested.

• RSSunrestricted = RSS from the regression in step 5.

• This is equivalent to the sum of the RSS from a regression on a sub-period before the break point plus the RSS from a regression on a sub-period after the breakpoint.

• This is a test of how much the RSS for the whole sample is bigger than the sum of the RSS of the two sub-samples.

• If coefficients don't change much then RSS will not rise much upon imposing restriction; that is we have stable parameters in our model!

STATA: Characteristic line of a Mutual Fund

• Consider a CAPM model of Century Share Funds excess returns:

• Using as sample from 1996m6 to 2012m2

• How stable will the OLS parameter estimates be in the above model?

• As a first step plot the dependent variable over time.

7

8

STATA: Chow test of known breakChoosing the most extreme value date (2001m3) we perform a chow test using the following.From the resultant F test statistic we reject the null hypothesis of parameter stability at the 1% level

9

What if structural break point isn't know?

• These previous tests will work well only if the structural break point is specified.

• In reality we may only know it lies within a given range of the sample period.

• Use a modified version of Chow Test know as Quandt Likelihood Ratio Test.

• QLR test automatically computes the usual Chow F-test repeatedly with differing break dates.

• The Break date with the largest F-statistic is chosen.

• QLR follows a non-standard distribution and is well behaved only when range of possible break points are far away from end of the sample.

10

11

As the Max QLR statistic is greater than 5% critical value we reject the null hypothesis of parameter stability.

Recursive estimation and Stability• If the researcher believes the series has a structural

break but cannot identify where then can use recursive estimation; recursive least squares.

• It involves starting with a sub-sample(normally the minimum required for estimation), estimating a regression, then sequentially adding one observation at a time, then rerunning the regression until the end of the sample is reached.

• Intuition is that parameter estimates at the start of this procedure will appear unstable; but as sample size increases the estimates should settle down.

• The results can then be plotted to give an excellent visually impression of how stable the parameters are.

12

Recursive estimation and Stability

• Two statistical tests based on recursive least squares are the CUSUM and CUSUM2.

• They are general tests of the stability of a time series process.

• The are derived from the residuals of the recursive estimation; recursive residuals.

• Under the null of perfect parameter stability

– The CUSUM statistic should be zero

– The CUSUM squared should range from zero at start of period and end at one.

• In practice both are normally plotted with a 95% confidence bands and the Null is rejected of the plot strays outside this band.

13

STATA: CSF characteristic line

• Run a recursive estimation method starting with 2 observations:• rolling, recursive window(2)clear: reg erCENSX er_GSPC, r

• This is a rolling window regression command in Stata that start with a sample of three and repeats the estimation adding one observation on each time.

• We can visually analyse the parameter estimates using:• tsset end

• tsline _b_cons if end>tm(1997m1), ytitle(Intercept) xtitle(" ") name(alpha, replace) nodraw

• tsline _b_er_GSPC if end>tm(1997m1), ytitle(CAPM Beta estimate) xtitle(" ") name(beta, replace) nodraw

• graph combine alpha beta, title(Century Share Fund CAPM stability) r(2)

14

• The recursive intercept fluctuates but decreases gradual with a sharp drop downward tick around 2000m1.

• Similarly the recursive CAPM beta fluctuates but sets to a fairly constant level beyond 200m1.

15

STATA: Apple’s characteristic line• To run the CUSUM squared test in Stata we can use a user written command

• cusum6 erCENSX er_GSPC

• The above command will graph the CUSUM squared statistic along with a 95% confidence band.

• The Null hypothesis of parameter stability is rejected if the statistic strays outside this band.

16

As the CUSUM squared statistics strays out the 95% confidence band we reject the null of model parameter stability

Solution to Failing stability tests

• Model is re-specified by including additional variables.

• Separate estimations are conducted for each of the sub-samples.

• If the Null cannot be rejected it is valid to pool all of the sample together in a single regression.

• In our Century Share fund example there is some evidence to suggest its characteristic line would be better modelled with consideration given to break points.

• As a first step the model should be re-estimated using a dummy variable specification and the previous recursive least squares tests performed again.

17