forecasting with vector autoregressive (var) models subject to business cycle restrictions

ELSEVIER International Journal of Forecasting 11 (1995) 569-583

Forecasting with vector autoregressive (VAR) models subject to business cycle restrictions

Scott Simkins Department of Economics, North Carolina A& T State University, Greensboro, NC 27411, USA

Abstract

In the last decade VAR models have become a widely-used tool for forecasting macroeconomic time series. To improve the out-of-sample forecasting accuracy of these models, Bayesian random-walk prior restrictions are often imposed on VAR model parameters. This paper focuses on whether placing an alternative type of restriction on the parameters of unrestricted VAR models improves the out-of-sample forecasting performance of these models. The type of restriction analyzed here is based on the business cycle characteristics of U.S. macroeconomic data, and in particular, requires that the dynamic behavior of the restricted VAR model mimic the business cycle characteristics of historical data. The question posed in this paper is: would a VAR model, estimated subject to the restriction that the cyclical characteristics of simulated data from the model "match up" with the business cycle characteristics of U.S. data, generate more accurate out-of-sample forecasts than unrestricted or Bayesian VAR models?

Keywords: VAR models; Business cycle behavior; Restricted forecasts; Prior restrictions

1. Introduction

It is well known that Bayesian vector autoregressive (VAR) models centered around a random-walk specification forecast aggregate macroeconomic time series more accurately than unrestricted VAR models. Unrestricted VAR models are prone to overfitting the data as more variables are added to the models so a reduction in dimensionality is often necessary to improve their forecasting accuracy. A Bayesian VAR modeler accomplishes this reduction in dimensionality by imposing Bayesian prior restrictions, typically centered around a random-walk prior specification, on the coefficients of the model. Li t terman (1979) and Doan, Lit terman and Sims

0169-2070/95/$09.50 (~) 1995 Elsevier Science B.V. All rights SSDI 0169-2070(95)00616-8

(1984) showed that such Bayesian VAR models produce more accurate forecasts than either unrestricted VAR models or univariate time series models, while Lit terman (1986) has shown that forecasts generated from Bayesian VAR models can successfully compete with forecasts from large structural models.

This paper focuses on the question of whether a more general type of prior restriction based on the business cycle characteristics of U.S. macroeconomic data can also improve the out-of-sample forecasting performance of unrestricted VAR models. Specifically, this restriction requires that the dynamic behavior of the restricted VAR model mimic the business cycle behavior of historical data. Such prior restrictions are not

reserved

570 S. Simkins / International Journal of Forecasting 11 (1995) 569-583

imposed on the model's coefficients directly, but rather on the sample paths generated from those coefficients, and provide an alternative to the standard random-walk prior often used in Bayesian VAR forecasting models.

The business cycle restrictions on the dynamic behavior of VAR models employed here are based on the classical methods of business cycle analysis developed by Arthur Burns and Wesley Mitchell at the National Bureau of Economic Research (NBER) during the 1930s and 1940s. Their work provided the first comprehensive description of business cycle activity in the economy and led to the first formal definition of the business cycle, a definition which underlies the modern concept of business cycle behavior. Overall, Burns and Mitchell's detailed description of the business cycle properties of economic aggregates forms the foundation of what economists generally accept as the "business cycle facts" to be explained by theoretical models and serves as a reference point against which to measure and define the business cycle properties of economic data.

The motivation for using Burns and Mitchell's methods of business cycle analysis to restrict the business cycle behavior of VAR models comes from the successful use of their methods to evaluate the dynamic behavior of theoretical business cycle models. Adelman and Adelman (1959) first used Burns and Mitchell's methods of business cycle analysis to compare the cyclical properties of simulated data from the Klein- Goldberger model with the cyclical properties of corresponding U.S. time series. Their work provided important empirical support for early Keynesian business cycle models. More recently, Simkins (1988, 1994) and King and Plosser (1994) carried out similar tests using Burns and Mitchell's methods for real business cycle models. In general, these tests underscore the need for economic models to generate time series behavior which replicates the business cycle characteristics of actual economies.

The present paper turns the Adelmans' test on its head. Instead of using Burns and Mitchell's methods to determine how well the cyclical behavior of model and actual economies match

up, their methods are used to force the cyclical behavior of a simple VAR model to mimic that of U.S. aggregate time series by restricting the dynamic behavior of the VAR model. Given the role that Burns and Mitchell's methods have played in developing a widely-accepted set of stylized business cycle "facts" that business cycle models must replicate, such a restriction seems reasonable. The question raised here is whether estimating a VAR model subject to the restriction that simulated data from the model mimic the business cycle properties of U.S. historical data leads to reduced out-of-sample forecasting errors. In the process of answering this question, additional insights are gained about how well unrestricted VAR models capture the salient features of U.S. business cycle behavior.

2. Measuring business cycle behavior

The measures of business cycle behavior used in this paper to restrict the dynamic behavior of unrestricted VAR models are drawn directly from the work of Burns and Mitchell (1946). An important part of their work focused on describing the cyclical characteristics of a wide variety of economic time series and then determining the similarities and differences in their patterns of cyclical behavior. To compare the cyclical characteristics of the hundreds of time series they analyzed, Burns and Mitchell constructed reference cycle patterns (or nine-point graphs), a simple summary measure of a series' business cycle properties, for each of the time series they analyzed. These reference cycle patterns com- pactly summarize both the average timing and volatility of the series over the course of a business cycle, and because they are unit-flee measures, they can be used to compare the cyclical characteristics of a wide range of economic activities. Such measures are used here both to describe the business cycle characteristics of historical data and also to construct the posterior distribution of forecasts from the restricted VAR model.

The first step in constructing reference cycle patterns for economic time series is determining

S. Simkins / International Journal of Forecasting 11 (1995) 569-583 571

the dates of business cycle peaks and troughs. For historical data, NBER business cycle turning point dates are used to date business cycle peaks and troughs. In simulated data from the VAR model, cyclical turning point dates are selected by using an automated turning point selection procedure to find cyclical peaks and troughs in the VAR model's output series, real GNP. This procedure first smoothes the output series with a three-term moving average to locate general areas of cyclical turns and then applies to the raw data a second-filter (whose weights are chosen to filter out all but the business cycle frequencies) in order to determine specific turning points within the general region of cyclical turns. This turning point selection process is similar to a procedure developed by Bry and Boschan (1971) to mimic the turning point selection process of Burns and Mitchell, but differs in the number of smoothed series that are used to select the areas of cyclical turns and the particular filter that is used to locate the specific dates of cyclical turning points. Further details about the turning point selection procedure and its ability to replicate the turning point selection process of Burns and Mitchell are given in Simkins (1988; 1994, Ap- pendix).

Once reference cycle dates are selected, a series' reference cycle pattern of cyclical move- ment for a particular business cycle is constructed by first expressing each observation in the cycle as a percentage of the cycle mean. Next, these cycle relatives are divided into nine stages. For cycles measured trough to trough, stages I, V, and IX represent the initial trough, the peak, and the terminal trough, respectively. Stages I through V correspond to a business cycle expansion while stages V through IX refer to a business cycle contraction. Using quarterly data, the cycle relative values at the initial trough, peak, and terminal trough dates become the cycle relatives for stages I, V, and IX respectively. The cycle relatives for the remaining stages are determined by dividing the business cycle expansion phase (stages l-V) and contraction phase (stages V-IX) into successive thirds (ex- cluding the peak and trough dates) and averaging the cycle relatives contained in each stage. When

plotted against the nine stages of the business cycle, these cycle relatives yield a graphical summary of the cyclical behavior of the series for a particular business cycle.

To obtain a representation of the average business cycle behavior for the series, the cycle relatives at each business cycle stage are aver- aged across all the business cycles in the sample period. These average reference cycle patterns form the classic NBER nine-point graphs which plot a series' average reference cycle values across the nine stages of the business cycle. It is these nine-point graphs, or reference cycle patterns, constructed from historical data, that form the basic summary measure of business cycle behavior used to restrict the dynamic behavior of the series in the VAR model. The time series behavior of the VAR model is judged to be consistent with the business cycle behavior of the historical data only if the reference cycle patterns of simulated data from the model mimic the reference cycle patterns of the corresponding U.S. time series.

3. Construction of the posterior distribution of restricted forecasts

To determine the effect of the business cycle prior on the forecasting performance of a repre- sentative VAR model, I use a variant of Litter- man's (1986) national forecasting model. Litter- man's model of the national economy has been cited as a benchmark model for comparisons of forecasting accuracy among VAR and other time- series models (see, for example, McNees, 1986 and Shoesmith, 1992) and provides a suitable framework for analyzing the business cycle properties of VAR models. In particular, Litterman's model is large enough to accurately reflect U.S. business cycle behavior, yet it is tractable for VAR work. The model analyzed here is a five- variable, six-lag, quarterly VAR model which includes the following variables (in levels): real GNP, the GNP price deflator, real business fixed investment, the money supply (M1), and the unemployment rate. The analysis here focuses on the model's performance in forecasting three

572 S. Sirnkins / International Journal of Forecasting I1 (1995) 569-583

of these aggregates, real GNP, the GNP deflator, and the unemployment rate. ~ These series are the ones most frequently cited in discussions of the national economy and provide a general view of the overall forecasting performance of the model.

3.1. U.S. reference cycle behavior

The business cycle measures that are used to restrict the dynamic behavior of the VAR model are computed by applying Burns and Mitchell's methods of business cycle analysis to the historical time series of the variables included in the model. Nine-point reference cycle patterns are constructed for each of the five variables using NBER business cycle peak and trough dates over the period 1949:1-1990:4. The NBER business cycle chronology for this period is shown in

Table 1 Business cycle turning points for historical U.S. data, 1949:1- 1990:4

Peak (P) or trough (T) NBER peak and trough dates a

T 1949:4 P 1953:2 T 1954:2 P 1957:3 T 1958:2 P 1960:2 T 1961:1 P 1969:4 T 1970:4 P 1973:4 T 1975:1 P 1980:1 T 1980:3 P 1981:3 T 1982:4

Note: a Source: Moore and Zarnowitz (1984). Business cycles are measured trough to trough here, so the 1990:2 business cycle peak is not included in the table.

1 Litterman's model also includes the three-month Treas- ury bill rate. However, the inclusion of this variable did not increase the forecasting accuracy of the model's variables for the sample period under consideration (1948:1-1990:4). This was true whether or not a Bayesian prior was imposed on the model. All data except the money supply are obtained from the Citibase database. The M1 series was supplied by Richard Todd. The data are contained in the Appendix.

Table 1 and includes seven complete cycles measured trough to trough. The resulting average reference cycle graphs for the five series are shown in Fig. 1. These reference cycle patterns summarize the cyclical movements of the series, relative to the cyclical mean, across a typical business cycle.

Note that all of the series except the unemployment rate display procyclical behavior, with cycle relatives rising through stages I -V (business cycle expansions) and falling (or rising less rapidly) through stages V-IX (business cycle contractions). The unemployment rate displays countercyclical behavior, falling during expansions and rising during recessions. The reference cycle patterns of the GNP deflator and the money supply rise throughout the nine stages of the business cycle, indicating that these series are dominated by a rising trend even within a business cycle. Note, however, that the series grow less rapidly, relative to the cycle mean, during contractions than expansions.

In addition to the average reference cycle patterns, the patterns in Fig. 1 also include standard-error bounds which measure the variability in cycle relatives over time. The standard- error bounds are calculated by first computing the standard deviation of the reference cycle relatives at each stage across the seven business cycles included in the sample period. The illustrated bounds represent---1.5 standard devia- tions of cycle-stage relatives around the mean U.S. reference cycle patterns at each stage of the business cycle. It is these bounds that are used to determine whether or not the business cycle properties of simulated data from the VAR model "match" the business cycle properties of the historical data.

3.2. Business cycle restrictions

Once reference cycle patterns of the historical data are constructed, Bayesian Monte Carlo methods are used to construct posterior distributions of restricted model forecasts for real GNP, the GNP deflator, and the unemployment rate. The first step in this process is to simulate the model and determine whether the simulated data

S. Sirnkins / International Journal of Forecasting 11 (1995) 569-583 573

are consistent with the underlying historical business cycle characteristics. Simulated sample paths of each of the model's variables are generated by drawing realizations from the underlying parameter space and adding stochastic shocks to the data-generating process. Following proce-

dures developed by Kloek and Van Dijk (1978), VAR model coefficients are first drawn from a normal-inverse Wishart posterior distribution whose mean is/3, the vector of unrestricted VAR parameter estimates. Serially independent stochastic shocks are then drawn from a multi-

a. Real GNP b. GNP Deflator

115"

110

105

>e 100 = _o g~ 9s

>- 90 U

85

80

7 5 '

Business Cycle Stage

I - - + / - 1.5 Std. Dev. ~ Average ]

c. Unemployment Rate

30

25

20

15

10

05

O0

95

90

85

l J

f /

/ f

J 80

'i Ill IV V V' VII Vill IX Business Cycle Stage

1 - - + / - 1 . 5 Std. Dev.-m- Average I

d. Investment

180 "

160

~ t 2 0

80 ~

Business Cycle Stage

I - - + / - 1.5 Std. Dev. + Average I

115

110

105

100 >,

90

85

8O

75

70 II lU IV V VI VII VIII

Business Cycle Slogs

1 - - + / - 1 . 5 Sld. Dev. + Average I

Fig. 1. Reference cycle pa t te rns -average of seven cycles with s tandard-error bounds: 1949:1-1990:4.


e. Money Supply

125'

120

115

.>_w 110

"6 lOS

IOO

o 95

9o

85

80

J

J

J

Business Cycle Stoge

- - + / - 1.5 Std. Dev..4_ Average

Fig. 1. Continued.

variate normal distribution with zero means and covariances based on residuals from the estimated unrestricted VAR model to generate artificial sample paths of each of the model's variables. 2 The length of the artificial sample paths is set to match the length of the estimation period for the model.

For each realized sample path of the model's variables two criteria must be met before reference cycle patterns are computed for the simulated data. First, each simulation must contain at least five complete business cycles, a condition that ensures that the average cycle length in the simulated data will be approximately the same as in historical business cycles. The nine-point reference cycle patterns illustrated in Fig. 1 are based on the seven postwar business cycles listed in Table 1, so to generate consistent reference cycle patterns in the simulated data, the simulated data should contain a similar number of

2 The specific procedure is outlined in example 10.1 in Doan (1992) and is commonly used to construct forecasts and posterior distributions in applied work with VARs. Drawing from both the distribution of VAR coefficients and stochastic shocks allows for sampling variability in the parameters as well as independent random shocks to the model's variables.

cycles. Business cycle turning points are selected from the simulated real GNP series using the turning point procedure summarized in Section 2 and simulations which contain fewer than five business cycles are excluded from the analysis. In addition, the time series must display positive values throughout the simulation period. Some simulations display negative values for some of the model's variables, a result which clearly has no economic meaning. Those simulations in which negative values of any of the model's variables appear are also excluded from the analysis.

For those simulations that meet these two criteria, nine-point reference cycle patterns are computed for each of the simulated series in the model. The nine-point reference cycle patterns from the simulated data are then compared to the nine-point reference cycle patterns from the corresponding U.S. historical time series. If the nine-point reference cycle patterns for each variable in the model fall completely within the standard error bounds of the U.S. data reference cycle patterns illustrated in Fig. 1, the simulated data are judged to be consistent with historical business cycle experience. Note that each of the simulated variable's reference cycle patterns

s. Simkins / International Journal of Forecasting 11 (1995) 569-583 575

must "match" that of the corresponding U.S. time series. The five reference cycle patterns constructed from the historical data represent the business cycle characteristics of the economy, so to be consistent with observed business cycle behavior, the simulated data should jointly replicate this underlying cyclical behavior. The standard-error bounds surrounding the mean reference cycle patterns in Fig. 1 can be "tightened" or "loosened" to make the joint set of business cycle restrictions more or less binding, determining how many of the realizations will lead to "matching" reference cycle patterns. T h e - 1 . 5 standard deviation bounds used here appear to be rather tight, resulting in about 10% of the simulations meeting the business cycle criteria imposed on the simulated behavior of the VAR model.

3.3. Restricted V A R mode l forecasts

are generated and used to compute overall out- of-sample forecast statistics for the restricted model. In concept, this is similar to applying the Kalman coefficient updating algorithm to the unrestricted VAR model to get updated coefficient estimates and forecasts. In this case, however, the entire Monte Carlo simulation must be repeated for each estimation period to obtain posterior distributions of restricted forecasts for the variables. The initial estimation period is 1949:3-1987:2, with forecasts generated for the period 1987:3-1989:2. The model is then re- estimated with data through 1987:3 and forecasts are generated for the period 1987:4-1989:3. This estimation/forecasting process is repeated through 1988:4, with the final forecasts ending in 1990:4, the end of the sample period of data. The procedure for generating the restricted VAR model forecasts is summarized in Table 2.

In simulations where the reference cycle pattern restriction is met, the draw of model parameters used to generate the simulated data is also used to generate dynamic forecasts-from one to eight steps ahead-of real GNP, the GNP deflator, and the unemployment rate. The forecasts for that simulation are then stored and the entire simulation process repeated until 500 k-step- ahead (k = 1 . . . . . 8) forecasts are obtained from VAR models that meet the business cycle restrictions. For each forecast step k the 500 out-of- sample forecasts from the restricted model form the posterior distribution of the restricted forecasts for a given estimation period. The means of these distributions are used as the out-of-sam~ple point forecasts for a given estimation period.

Additional one-to-eight step-ahead forecasts are obtained by adding one additional observation to the estimation period and repeating the entire simulation exercise outlined above. By successively re-estimating and updating the forecast distributions, seven k-step-ahead forecasts

3 Given the symmetric nature of the resulting posterior distributions (based on the underlying normal distribution of the stochastic shocks) the median of the posterior distribution could also be used as the point estimate of the forecast.

4. Forecasting performance of the restricted VAR model

Restricted forecasts for real GNP, the GNP deflator, and the unemployment rate are illustrated in Figs. 2-4 for three of the seven estimation periods examined (1949:3-1987:2, 1949:3- 1988:1, and 1949:3-1988:4), along with forecasts from the corresponding unrestricted VAR model, a Bayesian VAR model, and the actual values of the series. The three periods are chosen randomly to illustrate the relative forecasting performance of the three VAR models. The Bayesian VAR model is centered on a random-walk prior with small standard errors, limiting the interaction of the variables in the model. In the terminology frequently used by Bayesian VAR modelers, the prior distribution is "tight" around the random-walk specification. "Looser" priors which allow more interaction among the model's variables generally produced less accurate forecasts.

Note that no single estimation method leads to superior forecasts for each of the periods or for each of the three variables shown in Figs. 2-4. The relative forecasting accuracy of the three VAR models varies significantly across the illus-


Table 2 Procedure for generating business cycle restricted forecasts

1. Compute reference cycle patterns for historical U.S. data. 2. Simulate VAR model and impose business cycle restrictions

A. Simulate the VAR model for the given estimation period • Draw model coefficients from the underlying parameter distribution. • Add independent stochastic shocks based on historical behavior each period to generate simulated sample paths

for the model's variables, based on the drawn coefficient vector. B. Impose business cycle restrictions

(1) General restrictions • Determine that there are five or more business cycles in the simulated sample paths. • Determine that there are no negative observations in the simulated sample paths.

(2) Business cycle restrictions • Determine that reference cycle patterns from each of the simulated series are contained within the ---1.5

standard-error bounds of the corresponding U.S. reference cycle patterns. 3. Generate restricted model forecasts for a given estimation period

A. For simulations where all of the restrictions in (2.B) are met, use the drawn coefficient vector from (2.A) and add serially independent stochastic shocks to generate 1-to-8 step-ahead forecasts of real GNP, the GNP deflator, and the unemployment rate.

B. Construct a posterior distribution of restricted model forecasts by repeating steps 2 to (3.A) until 500 sets of 1-to-8 step-ahead restricted forecasts are generated. The mean values of the resulting posterior distributions of restricted forecasts become the 1-to-8 step-ahead point forecasts for the estimation period.

4. Forecast updating One additional observation is added to the estimation period and steps 2-3 are repeated. This process is repeated seven times to generate seven one-step-ahead forecasts, seven two-step-ahead forecasts, etc., from which forecasting accuracy statistics are computed.

t r a t e d t ime pe r iods , a resul t tha t is espec ia l ly e v i d e n t a m o n g the rea l G N P forecas ts . T h e B a y e s i a n V A R m o d e l forecas ts dur ing the 1987- 89 fo recas t p e r i o d (Fig. 2a) a re m o r e accura te t han those f rom the o t h e r two mode l s , whi le all t h r e e m o d e l s p r o d u c e s imi lar forecas t s for the 1988-90 fo recas t p e r i o d (Fig. 2b). D u r i n g the 1989-90 fo recas t p e r i o d (Fig. 2c), the unre- s t r i c t ed and re s t r i c t ed V A R m o d e l forecas t s a re s u p e r i o r at l onge r forecas t hor izons , whi le the B a y e s i a n V A R m o d e l p r o d u c e s the mos t accura te fo recas t s o v e r sho r t e r hor izons . 4

In t e rms of ind iv idua l ser ies , the res t r i c t ed m o d e l is m o r e accura te than the unres t r i c t ed m o d e l in fo recas t ing the rea l G N P ser ies in the p e r i o d s i l lus t ra ted , a l t hough ne i t he r m o d e l forecasts pa r t i cu l a r ly well in the 1987:3-1989:2 forecast pe r iod . T h e B a y e s i a n V A R m o d e l c lear ly p r o d u c e s s u p e r i o r forecas ts for this pe r iod . The

4 One reason why the relative forecasting performance of the models may be sensitive to the forecast period analyzed is that the U.S. economy experienced a cyclical turning point during the 1989-91 period, and model forecasts are likely to be more sensitive to updated information at those times.

Bayes i an V A R m o d e l also g e n e r a t e s the mos t accura te forecas t s o f the u n e m p l o y m e n t ra te . B o t h the un re s t r i c t ed and re s t r i c t ed m o d e l s forecas t u n e m p l o y m e n t ra tes tha t a re s ignif icant- ly h igher than ac tua l va lues ove r the p e r i o d s i l lus t ra ted . F o r the G N P def la to r , the r e s t r i c t ed m o d e l p r o d u c e s m o r e accu ra t e forecas t s t han the un res t r i c t ed m o d e l , bu t aga in the B a y e s i a n V A R m o d e l gene ra l ly o u t p e r f o r m s the o t h e r two mode l s .

T h e forecas t s in Figs. 2 - 4 r e p r e s e n t on ly t h ree of the seven fo recas t pe r i ods for which the full r ange of forecas t s can be c o m p u t e d . W h i l e the Bayes i an V A R m o d e l a p p e a r s to d o m i n a t e in the pe r iods i l lus t ra ted , an overa l l m e a s u r e o f forecast ing accuracy is r e q u i r e d to d e t e r m i n e w h e t h e r the bus iness cycle res t r i c t ions i m p o s e d on the un re s t r i c t ed V A R m o d e l l e ad to fo recas t s that a re generally m o r e accu ra t e than those f rom unres t r i c t ed or B a y e s i a n V A R mode l s . F o r each of the m o d e l s (and for each va r iab le ) the overa l l fo recas t ing accuracy at each s tep k is m e a s u r e d by The i l ' s U stat is t ic , which is ca l cu la t ed as the ra t io of the roo t m e a n squa re d e r r o r of the

S. Simkins I International Journal of Forecasting 11 (1995) 569-583 577

a. Estimation Period: 1949:3-1987:2 Forecast Period: 1987:3-1989:2 b. Estimation Period: 1949:3-1988:1 Forecast Period: 1988:2-1990:.1

4200

4150 ~4too ~ - f g 4050

J 4000

J ~ 3 9 5 0

5900 e

3 8 5 0

3800 ~ ' ~ _ ~ - w - - - - ~

3750 . . . . . . . 87:3 88:1 88:3 89:t

87:4 88:2 88:4 89:2 Dote

~ stricfed VAR

4450 -

4400

4350

4300

= 4250

- 4 2 0 0

4150

4 t 0 0

4050

4000

/

88:2 88:4 89:2 89:4 88:3 89:t 89:3 90:1

Date

A•_al Uric_restricted VAR

c. Estimation Period: 1949:3-1988:4 Forecast Period: 1989:1-1990:.4

4350

,~oo

=2 425o

~42oo

~ 4 1 5 0

4100

4050 8~:~ 89':3 96:1 9~:3 89:2 B9:4 90:2 90:4

Date

Fig. 2. Forecasts and actual values-real GNP.

Actual

Umesfr ic led VAR

Restricted VAR

BVAR Mode~

fo recas t k s teps a h e a d to the na ive forecas t of no change in the va r iab le . 5 In the case w h e r e the r o o t m e a n squa re e r r o r is the same as the na ive

5 5. Theil's U statistic is given by

~, y { , + k _ y , ÷ k 2 ,~2

I where Y, represents the value of variable Y at time t, Y,.,.k represents the k-step-ahead forecast of Y at time t, and T k represents the set of t's in the forecast period for which the k-step-ahead forecast errors are known.

forecas t e r r o r the s ta t is t ic is equa l to one , so The i l U stat is t ics less than one ind ica te tha t the m o d e l ' s forecas t s a re m o r e accura te than the naive forecas ts of no change ove r the fo recas t pe r iod . L o w e r va lues for the The i l U s ta t is t ic indica te m o r e accura te forecas ts .

The k - s t e p - a h e a d The i l U stat is t ics for each o f the m o d e l s ' fo recas t s of real GNP, the G N P def la tor , and the u n e m p l o y m e n t ra te a re g iven in Tab le 3. T h e r e the re la t ive fo recas t ing p e r f o r m - ance of the t h ree fo recas t ing m o d e l s b e c o m e s c leare r . In gene ra l , the r e s t r i c t ed m o d e l p r o d u c e s m o r e accura te forecas ts t han the unre -

578 S. Simkins I International Journal of Forecasting 11 (1995) 569-583

a. Estimation Period: 1949:3-1987:2 Forecast Period: 1987:3-1989:2 b. Estimation Period: 1949:3-1988:1 Forecast Pea~xl: 1988:2-1990:1

12

11

10 ~9 i 8 ~ 7

~ 5

strieted VAR

[ B~IrRicteL: AR

9

8.5

8 /'J "% / / /

6.5 / /

5.5 £ 5

4.5 ..¢

estricfed VAR

=

87:3 88:1 88:3 89:1 88:2 88:4 89:2 89:4 87:4 88:2 88:4 89:2 88:3 89:1 89:3 90:1

Date Date

c. Esl~mtion Period: 1949:3-1988:4 Forecast Period: 1989:1-199~.4

8.5

8

7.5 >

"O 2 7

6.8 I o o

5.5

Actual -->(-

~ stricted VAR

~ r lcted VAR

BVAR Model

s 89':1 8~:3 96:1 9d:3 89:2 89:4 90:2 90:4

Date

Fig. 3. Forecasts and actual va lues -unemployment rate.

stricted model, especially for real GNP and the GNP deflator. This result is especially true for forecasts at longer time horizons. At these longer time horizons, the business cycle restrictions reduce the Theil U statistics by as much as 30%. For the unemployment rate the improvement in the restricted forecasts is less noticeable. Here the Theil Us verify the poor forecasting performance of the unrestricted and restricted models for this variable, a result that is illustrated in Fig. 3.

Overall, the improvement in forecasts pro-

duced by the business cycle restrictions are small in comparison to the improvements produced by imposing random-walk Bayesian priors on the coefficients of the model. In most cases the Theil U statistics are well below one, indicating that the Bayesian VAR forecasts outperformed the naive forecast of no change in the variable. The relative improvement provided by the Bayesian priors is especially evident in the case of unemployment rate forecasts. The Bayesian VAR forecasts come much closer to the actual values than forecasts from the other two models. The


a. Estimation Period: 1949:3-1987:2 Forecast Period: 1987:3-1989".2 b. Estimation Period: 1949:3-1988:1 Forecast Period: 1988:2-1990:1

126

128 J = 124 9 :~ 123

-~ 122

. 121

~ 120

~ 1 1 9

118 I

117 . . . . 87:3 88:1 88:3 89:1

87:4 88:2 88:4 89:2 Date

U~.~estricted VAf

1 3 0

122 . 1 2 0 ~ [BVAR Model I

u -116

114

112 8~:2 8g:4 89':2 8g:4 88:3 89:1 89:3 90:1

Dote

c. Es',imation Period: 1949:3-1988:4 Forecast Period: 1989:.1-1990:A

135

134

133 m

132

> 131 "6

130

. 1 2 9 m

128

127

126

125

124

/ / 89:1 89:3 90:1 90:3

89:2 89:4 90:2 90:4 Date

U~_estrieted VAR

Fig. 4. Forecasts and actual values-GNP deflator.

Theil U statistics for the Bayesian VAR model are five to seven times smaller than the Theil U statistics for this variable in the unrestricted and restricted models. For real GNP and the GNP deflator, the improvement is less dramatic but is consistent over all forecast horizons. For these two series the Theil U statistics are often less than half those from forecasts generated by the unrestricted and restricted models. It is clear from these results that while the business cycle restrictions do marginally improve the forecasting accuracy of otherwise unrestricted VAR

models, prior restrictions constructed on the basis of results from modern time series techniques (i.e. the random-walk specification of the Bayesian priors) lead to more accurate forecasts of macroeconomic aggregates.

5. Conclusions

Bayesian VAR models have become a standard tool for forecasting macroeconomic aggregates. Forecasts from these models are often as accur-


Table 3 Theil's U statistics for unrestricted, restricted, and Bayesian VAR models based on 7 horizon

out-of-sample forecasts for each forecast

Variable Steps ahead Unrestricted VAR (k) model

Business cycle Bayesian VAR restricted VAR model model

Real GNP 1 1.062 1.043 0.303 2 1.227 I. 186 0.298 3 1.198 1.132 0.311 4 1.158 1.060 0.366 5 1.124 0.993 0.445 6 1.116 0.977 0.549 7 1.135 1.007 0.648 8 1.157 1.039 0.794

GNP deflator 1 0.551 0.510 0.290 2 0.651 0.582 0.289 3 0.721 0.605 0.284 4 0.786 0.621 0.274 5 0.838 0.634 0.262 6 0.869 0.631 0.253 7 0.900 0.634 0.252 8 0.941 0.646 0.266

Unemployment rate 1 3.153 3.117 0.656 2 4.332 4.212 0.635 3 5.601 5.342 0.779 4 6.569 6.124 0.939 5 7.861 7.160 1.302 6 8.227 7.406 1.523 7 8.606 7.750 1.945 8 8.529 7.775 2.458

ate as those from much larger models and serve as benchmarks for alternative forecasting methods. It appears that Bayesian prior restrictions on the coefficients of these models, in particular the random-walk prior, lead to more accurate forecasts than the type of business cycle restrictions examined in this paper. While Burns and Mitchell's methods of business cycle analysis provide a useful framework for describing the cyclical characteristics of time series, it appears that their benefit in reducing forecast errors is less dramatic than standard Bayesian priors.

However, while the forecasting results them- selves may be disappointing, this paper provides some useful insights into the role of classical business cycle methods and their relationship to modern time-series models. First, the forecasting exercise carried out here illustrates how classical business cycle methods can be used to restrict

the dynamic behavior of modern time-series models. It also suggests how such restrictions could be used to calibrate theoretical business cycle models, in much the same way that second- moment behavior is currently used to select parameter values in these models. The Burns and Mitchell measures of business cycle behavior provide a compact set of business cycle characteristics that summarize the cyclical activity of economic time series and represent a reasonable criterion for theoretical business cycles to meet. It would be useful to investigate how restricting theoretical models to mimic U.S. business cycle behavior (using Burns and Mitchell's methods) would affect the underlying dynamic behavior of these models and determine what implications this has for the underlying economic theory. Such an exercise can be accomplished using Monte Carlo simulation methods like those used

s. Simkins / International Journal of Forecasting 11 (1995) 569-583 581

in this paper. Insights gained from this type of exercise may be useful for discriminating among competing economic theories.

Finally, the results provide additional insights into the ability of unrestricted VAR models to describe the business cycle behavior of U.S. time series in a manner consistent with Burns and Mitchell's methods of business cycle analysis. The fact that the business cycle restrictions lead to only minor improvements in the forecasting accuracy of unrestricted VAR models implies that the business cycle restrictions imposed on the unrestricted VAR model are already being summarized by the unrestricted model. That is, the business cycle restrictions do not significantly affect the dynamic behavior of unrestricted VAR models. To the extent that the results of Burns and Mitchell's business cycle analysis are the source of business cycle "facts" which must be explained by macroeconomic models and to the extent that unrestricted VAR models are used to descr ibe the business cycle characteristics of macroeconomic data, this is an encouraging finding. It implies that the time series behavior generated by unrestricted VAR models is consistent with the business cycle facts that Burns and Mitchell's methods generate. Unfortunately, these same business cycle facts do not seem to significantly improve forecasts of macroeconomic aggregates. VAR models with explicit priors on the coefficients of the model still provide the best macroeconomic forecasts.

Appendix: Data used to estimate the restricted VAR model

Date Real G N P G N P Unemployment Real Money (Yr:Otr) Deflator Rate Business Supply

Fixed (M1) Investment

48:1 1086.8 23.2 3.73 196.3 111.16 48:2 1106.1 23.5 3.67 197.6 110.24

48:3 1116.3 23.9 3.77 195.7 110.34 48:4 1125.5 23.8 3.83 194.3 109.91

49:1 1112.4 23.6 4.67 183.6 109.32 49:2 1105.9 23.4 5.87 175.6 109.49

49:3 1114.3 23.4 6.70 114.6 109.16 49:4 1103.3 23.4 6.97 179.9 109.16 50:1 1148.2 23.5 6.40 190.9 110.14 50:2 1181.0 23.9 5.57 208.8 111.75

50:3 1225.3 24.2 4.63 50:4 1260.2 24.5 4.23

51:1 1286.6 25.1 3.50

51:2 1320.4 25.1 3.10

51:3 1349.8 25.11 3.17

51:4 1356.0 25.2 3.37 52:1 1369.2 25.2 3.07 52:2 1365.9 25.3 2.97

52:3 1378.2 25.5 3.23 52:4 1406.8 25.9 2.83 53:1 1431.4 25.9 2.7[)

53:2 1444.9 25.9 2.57 53:3 1438.2 26.0 2.73

53:4 1426.6 25.8 3.711 54:1 1406.8 26.2 5.27 54:2 14111.2 26.3 5.81/ 54:3 1418.0 26.3 5.97

54:4 1438.8 26.5 5.33 55:1 1469.6 26,8 4.73 55:2 1485.7 27.1 4.411

55:3 1505.5 27,3 4.1(I 55:4 1518.7 27.4 4.23

56:1 1515.7 27.7 4.113

56:2 1522.6 27.9 4.20 56:3 1523.7 28.2 4.13

56:4 1540.6 28.5 4.13 57:1 1553.3 28.8 3.93

57:2 1552.4 28.9 4.111 57:3 1561.5 29.2 4.23

57:4 1537.3 29.3 4.93 58:1 1506.1 29.5 6.311

58:2 1514.2 29.6 1.37 58:3 15511.0 29.1 7.33

58:4 1586.1 29.9 6.37 59:1 1606.4 3(1.2 5.83 59:2 1637.0 30.4 5.111

59:3 1629.5 311.6 5.27 59:4 1643.4 30.6 5.60 60:1 1611.6 30.9 5.13 60:2 1666.8 311.9 5.23

60:3 1668.4 31.0 5.53

60:4 1654.1 31.11 6.27 61:1 1611.3 31.0 6.811

61:2 1692.1 31.2 7.011 61:3 1116.3 31.4 6.77

61:4 1754.9 31.4 6.211 62:1 1771.9 31.7 5.63

62:2 1796.4 31.8 5.53 62:3 1813.1 31.9 5.57

62:4 1810.1 32.2 1.53 63:1 1834.6 32.2 5.77 63:2 1860.0 32.3 5.73 63:3 1892.5 32.4 5.50

63:4 1906.1 32.6 5.57 64:1 1948.7 32.7 5.47 64:2 1965.4 32.8 5.20

64:3 1985.2 33.0 5.011 64:4 1993.7 33.1 4.97

65:1 2036.9 33.5 4.911 65:2 2066.4 33.6 4.67

65:3 2099.3 33.9 4.37 65:4 2147.6 34.1 4.10 66:1 2190.1 34.5 3.87

66:2 2195.8 34.8 3.83 66:3 2218.3 35.1 3.77 66:4 2229.2 35.5 3.70

61:1 2241.8 35.7 3.83 67:2 2255.2 35.7 3.83

67:3 2287.7 36.11 3.80 67:4 2300.6 36.4 3.911 68:1 2327.3 37. I 3.73 68:2 2366.9 37.5 3.57

223.6 219.7

211.3

203.9

201.7 21XI,4 203.11 2114.9

193.6

205,9 213.9 214.4

214.11

213.0 21(I.4

214.1 220.8 224.1

233.4 243.0

241.9 249.5 244.9

246.0

245.8 242.1

242.l 239.3

241.9

238.1 226.1

219.0 2 2 0 7

233.5 241.1 255.5

257.9 254.8 262.0

254,1 247.5

247,5 245.5

247.4 253.3

261.0

265. l 214.4

216.3 273.2 275.1/

288.2 295.3 303.3 307.8 3118.3

310.7 313.9

329.8

340.4 344.5

352.5 361.2

357. i 355.2

341.2 332.1 343.6

347.7 359.1 368.6 365,8

112.99

113.98

115.16

116.21

117.68

119.84 121.42 122.43 123.68

124.92 125.41

126.27 126.46

126.56 126.92 i27.22

128.43 129.74 131.25

132,03 132.59

132.82

133.28 133.64 133.67 134.30

134.56

134.62 134.66

133.93 133.77

135.31

136.65 138.33 139.31

140.53 141.53 140.30

139.90 139.60

140.90 1411.87

14L53 142.60

143.43 144.13

145.61

146.63 146.41 I41.33

148.80 1511.20 151.73 153.23

154.20 I55.30

157.87 159:911

161.10 162.07

163.90 166.90

169.17

111.63 171.10 171.57

173.27 175.70 179,57 182.47 184.93 188.111


68:3 68:4 69:1 69:2 69:3 69:4 70:1 70:2 70:3 70:4 71:1 71:2 71:3 71:4 72:1 72:2 72:3 72:4 73:1 73:2 73:3 73:4 74:1 74:2 74:3 74:4 75:1 75:2 75:3 75:4 76:1 76:2 76:3 76:4 77:1 77:2 77:3 77:4 78:1 78:2 78:3 78:4 19:1 79:2 79:3 79:4 80:1 80:2 80:3 80:4 81:1 81:2 81:3 81:4 82:1 82:2 82:3 82:4 83:1 83:2 83:3 83:4 84:1 84:2 84:3 84:4 85:1 85:2 85:3 85:4 86:1 86:2

2385.3 37.9 3.53 368.8 191.17 2383.0 38.5 3.40 379.7 195.90 2416.5 39.0 3.40 385.4 199.40 2419.8 39.5 3.43 386,2 201.03 2433.2 40.1 3.57 390.3 201.90 2423.5 40.6 3,57 318.6 203.$3 2408.6 41.3 4.17 374.1 205,80 2406.5 41.9 4.11 366.5 201.20 2435,8 42.2 5.11 374.5 210.00 2413,8 42.1 5.83 318.2 213.10 2478,6 43.4 5.93 383.8 211.40 2478,4 44.2 5.9(} 397.8 221.93 2491,1 44.7 6.03 405.4 225,71 2491,0 45.3 5.93 411.7 221.87 2545.6 45.8 5.17 431.5 232.40 2595,1 46,1 5.70 436.7 236.01 2622.1 46,7 5.57 441.4 241.00 2671.3 47.3 5.37 465.0 246.93 2734.0 48,0 4.93 481.7 251.90 2741.0 49.0 4.93 482.1 254.83 2738,3 50.0 4.80 481.3 251.77 2762.8 51.2 4.77 417.9 261.03 2747,4 51.9 5.13 465.8 265.43 2755.2 53.0 5.20 459.0 267.83 2719,3 54.8 5.63 446,1 270.27 2695.4 56,3 6.60 420.6 273.57 2642,7 57.7 8.27 392.4 275.17 2669.6 58,6 8.87 388.4 279.27 2714.9 59.9 8.47 397.8 284.60 2752.7 61.0 8.30 405.7 286.87 2804.4 61.7 7.73 420.3 290.73 2816.9 62.5 7.57 425.9 295.40 2828.6 63.4 7.73 429.1 298.57 2856.8 64.5 7.77 450.3 304.27 2896.0 65.6 7,50 467.8 311.50 2942.7 66.9 7.13 493.1 317.03 3001.8 67.7 6.90 502.2 322,40 2994.1 68.9 6,67 505.5 329.30 3020.5 69.9 6.33 512.4 335.83 3115.9 71,6 6,00 543.5 343.30 3142,6 72.9 6.03 550.2 350.17 3181.6 74.4 5.90 554.6 356.33 3181.7 76.1 5.87 558.3 360.63 3178.7 77.8 5.70 557,3 369.67 3207.4 79.4 5.87 564.9 378.83 3201.3 81.0 5.97 560.5 382.30 3233,4 82.7 6.30 552.6 388.40 3157.0 84.6 7.33 496,9 385.10 3159.1 86.5 7.67 497.2 399.50 3199.2 89.0 7.40 518,1 410.73 3261.1 91.3 7,43 524.9 415.10 3250.2 92.8 7.40 529,4 425.23 3264.6 94.9 7.40 525.0 427.23 3219.0 96.7 8.23 501.4 432.83 3170.4 98.2 8.83 488.2 442.53 3179.9 94.4 9.43 473.0 446.93 3154.5 100,8 9.90 458.1 452.23 3159.3 101.7 10.67 468.1 470.90 3186.6 102,5 10.31 469.4 483.97 3258,3 103.3 10.13 496.2 499.10 3306.4 104,2 9.37 525.8 510.53 3365,1 105.4 8.53 550.3 519.73 3451,1 106.5 1.87 511.8 528.00 3498.0 107.3 7.43 595.1 537.11 3520,6 108,2 7.43 603.3 542.10 3535,2 109,0 7.30 614.0 547,97 3577,5 109,7 7.23 618.6 562.01 3599.2 110.6 7.30 630.6 516.47 3635,8 111.3 7.20 622.1 596.83 3662.4 112.2 7.03 640.4 613.60 3721.1 112.4 7.03 634.2 626.73 3704,6 113,2 1.17 635,2 651,90

86:3 3112.4 114.6 6.97 631.0 679.53 86:4 3733.6 115.1 6.83 636,0 108.17 87:1 3781.2 116.1 6,60 627.4 731.77 87:2 3820.3 117,0 6.27 642,0 745.27 87:3 3858,9 118.0 6.00 657.1 746.23 87:4 3920,7 118.5 5.83 658.1 753.60 88:1 3970,2 119.3 5.70 667.4 759,50 88:2 4005.8 120.6 5.47 688,3 774.20 88:3 4032.1 122.0 5.50 690.4 783.77 88:4 4059.3 123.4 5.30 682.2 785.63 89:1 4095,7 124.4 5.23 590.9 785.47 89:2 4112,2 125.8 5.27 693.6 777.47 89:3 4129.7 126.8 5.27 697.7 780.33 89:4 4133,2 128.0 5,30 690.2 190,13 90:1 4150.6 129.5 5.30 702.9 800.40 90:2 4155.1 131,0 5.33 691.2 808.90 90:3 4170,0 132,2 5.60 692.3 816.33 90:4 4153.4 133,1 5,90 682.7 823.30

References

Adelman, Irma and Frank L. Adelman, 1959, The dynamic properties of the Klein-Goldberger model, Econometrica 27, 596-625.

Bry, Gerhard and Charlotte Boschen, 1971, Cyclical analysis of time series: Selected procedures and computer programs (NBER, New York).

Burns, Arthur F. and Wesley C. Mitchell, 1946, Measuring business cycles (NBER, New York).

Doan, Thomas A., 1992, User's manual: RATS (Regression Analysis of Time Series), version 4.10 (Estima, Evanston, IL).

Doan, Thomas A., Robert Litterman and Christopher Sims, 1984, Forecasting and conditional projection using realistic prior distributions, Econometric Reviews 3, 1-100.

King, Robert G. and Charles I. Plosser, 1994, Real business cycles and the test of the Adelmans, Journal of Monetary Economics 33, 405-438.

Kloek, T. and H.K. Van Dijk, 1978, Bayesian estimates of equation system parameters: An application of integration by Monte Carlo, Econometrica 46, 1-20.

Litterman, Robert, 1979, Techniques of forecasting using vector autoregressions, Working Paper No. 115, Min- neapolis Federal Reserve Bank.

Litterman, Robert, 1986, Forecasting with Bayesian vector autoregressions-five years of experience, Journal of Busi- ness and Economic Statistics 4, 25-37.

McNees, Stephen K., 1986, Forecasting accuracy of alternative techniques: A comparison of U.S. macroeconomic forecasts, Journal of Business and Economic Statistics 4, 5-15.

Moore, Geoffrey H. and Victor Zarnowitz, 1984, The development and role of the National Bureau's business cycle chronologies, Working Paper No. 1394, National Bureau of Economic Research.

Shoesmith, Gary L., 1992, Multiple cointegration, error


correction and Litterman's model, Working paper, Bab- cock Graduate School of Management, Wake Forest Uni- versity.

Simkins, Scott P., 1988, Interpreting economic time series using the methods of Burns and Mitchell, Unpublished Ph.D. dissertation, University of Iowa.

Simkins, Scott P., 19c~4, Do real business cycle models really exhibit business cycle behavior?, Journal of Monetary Economics 33, 381-404.

forecasting with vector autoregressive (var) models subject to business cycle restrictions

Documents