increasing the transparency of macroeconometric forecasts: a report from the trenches

International Journal of Forecasting 18 (2002) 85–105www.elsevier.com/ locate / ijforecast

Increasing the transparency of macroeconometric forecasts:a report from the trenches

a,b ,*Ullrich Heilemanna

¨ ¨Rheinisch-Westf alisches Institut f ur Wirtschaftsforschung, Hohenzollernstraße 1-3, D-45128 Essen, Germanyb ¨Gerhard-Mercator-Universitat Duisburg, Lotharstraße 65, D-47048 Duisburg, Germany

Abstract

The acceptance of macroeconometric model results suffers from the procedures of model builders /users that are oftenopaque. This paper describes in detail the production of a macroeconomic forecast with a macroeconometric model for theGerman economy. The model used is the RWI-business cycle model, a medium sized macroeconometric model (41stochastic equations, 86 identities). It has been used since the late 1970s in the RWI and in the biannual ‘Joint Diagnosis’ ofGerman economic research institutes. The paper starts with a presentation of the analytical foundations of econometric modelforecasting in general and of various ways and forms of incorporating outside-model information in particular. Thepresentation of the main features of the model is followed by the detailed description of the various stages of forecastproduction. The paper ends with suggesting a more intensive use of the analytical possibilities of the models and theirforecast. 2002 International Institute of Forecasters. Published by Elsevier Science B.V.

Keywords: Macro models; Forecast evaluation; Forecast comparison; add-factoring

1. Introduction marks. Closer looks at the ‘critiques’, have,however, revealed their limited relevance, and

Econometric models are a widely used and the ‘new/old macroeconomic consensus’ (Blin-powerful tool in macroeconomic analysis and der, 1992) of the early 1990s seem to haveforecasting. Admittedly, their acceptance by the restored much of the lost credibility.academic community had some hard times Unfortunately this does not hold for theduring the 1970s and 1980s. The general decline charge of missing transparency. More specific,in the reputation of macroeconomics, the Lucas it is criticised that (1) the numerous, oftenand the Sims critique, as well as failures of the complex model relationships and the role andmodelling community to make their models and importance of endogenous /exogenous as welltheir practice more transparent have left their as other outside information are difficult to

understand and (2) the models and especiallytheir results are not free from bias, i.e. the*Tel.: 149-201-814-9221; fax: 149-201-814-9200.

E-mail address: [email protected] (U. Heilemann). models tell what their users want them to tell.

0169-2070/02/$ – see front matter 2002 International Institute of Forecasters. Published by Elsevier Science B.V.PI I : S0169-2070( 01 )00113-3

86 U. Heilemann / International Journal of Forecasting 18 (2002) 85 –105

This ‘black box’ accusation is expressed in provides users of these forecasts an understand-particular by members of the academic com- ing that model forecasts are not the result ofmunity, who, for various reasons, favor small, ‘black boxes’ but can be tracked down toreductionist text book models. In contrast to hypotheses and assumptions. The model used is

¨this, policy makers and business economists the model of the Rheinisch-Westf alisches In-¨value the informational content of large models. stitut f ur Wirtschaftsforschung (RWI), a

They are much less sceptical of these or over- medium sized, short term macroeconometric forcome their doubts about the models by using the (West) German economy. It has been regu-others’ models in addition to their own. The larly applied for forecasting since 1978. Thecomplaint about missing transparency is not forecast that is examined is the one that wasnew. Tinbergen’s models in the 1930s were made in autumn of 1996 for 1996 and 1997.confronted with it (Morgan, 1990, pp. 118ff.). The paper describes the various steps of forecastThis complaint figured prominently in the mac- generation, shows their consequences for theroeconometric model critique of the early 1970s final forecast and analyzes in particular the(Brunner, 1973) and it is still often heard today. accuracy of this and some competing forecasts.The problem of ‘reading’ (Boutillier, 1983) or The next section displays some analyticalunderstanding macroeconometric models re- foundations of econometric forecasting andceived some attention by employing verbal error analysis. Section 3 briefly presents themodel descriptions, block diagrams, logical model used and the macroeconomic situation inanalysis (graphs, incidence matrices), model late 1996. The generation of the forecast iscondensations, aggregated supply /demand described in Section 4, including its ex postcurves, and implicit Phillips curves. This, how- assessment. The paper ends with a summary andever, is not the case with respect to the gene- conclusions. The subtitle of the paper announcesration of model forecasts. Of course, the afore- a report from the trenches, hence the paper ismentioned techniques facilitate also the under- rather dense and brief, and, equally important, itstanding of the forecasting process, but the abstains from generalisations. Although theliterature on this is still rare (Klein & Young, general framework is likely to hold for many1980, pp. 75ff.; Adams, 1986, pp. 106ff.; Dicks macroeconometric model forecasts, the weight& Burrell, 1994), and the problem is ignored in given to the various stages of the forecastmost econometric textbooks. The main explana- production will vary with the purpose and thetion for all this is that there is not much design of the model, whether it is operated ineffective user demand for model transparency. the private, public or academic sector etc.Consequently, the model industry sees noreason to improve the situation (Daub, 1987, pp.

2. Analytical foundations73ff.).This paper deals with the possibilities of

A simplified form of an econometric modelmaking forecast practice transparent by describ-can be represented as follows:ing in detail the generation of a mac-

roeconometric forecast. It should serve two y 5 f( y , x , b, e ) (1)t t2k t2i tmain purposes. First, it structures the process ofmacroeconometric model forecasting using state where: f 5 ( f , . . . , f )5 vector of N functional1 N

of the art techniques. This should be a guide for relationships; y 5vector of (lagged) endogen-t2k

producers as well as for users. Second, it ous variables, k 5 0, . . . ,m; x 5vector oft2i

U. Heilemann / International Journal of Forecasting 18 (2002) 85 –105 87

(lagged) exogenous variables, i 5 0, . . . ,n; b 5 usually are put in at the start, the second ones(b , . . . b )5matrix of estimated structural pa- typically are made in later stages when the1 N

rameters; e 5 (e , . . . e )5vector of error results contradict macroeconomic reasoning ort 1t Nt

terms (with the usual assumptions). outside information. All this makes the processForecasts with this system of equations are of forecasting an iterative procedure, similar—

made by inserting the predetermined (exogenous though to a lesser extent and, of course, muchand lagged endogenous) variables, assuming quicker—to the procedures of the so-callede 5 0 and solving the model for the periods informal GDP model forecasts (Zarnowitz,t

k . 0: 1992, pp. 385ff.). Usually, the modifications aremade in the form of additions to the constanty 5 f y ; x ; b (2)s dt1j t2k1j t2i1j term (‘adds’) so that the model reactions arepreserved.For a number of reasons this basic scheme of

econometric forecasting is (usually) modified.First, there may be actual data or superioroutside information available for some of the 3. The RWI short-term modelendogenous variables to replace model

1 The RWI business cycle model is a medium-estimates. Second, there may be suggestedsized, quarterly model, which has been used forpolicies for which the model lacks appropriateshort-term forecasting (six to eight quarters) andvariables. Third, an equation may be mis-simulation since the late 1970s. The versionspecified, may have systematic bias etc., so thatexamined here consists of 41 stochastic equa-the assumption for the forecast period appearstions and 86 identities, which together form anunreasonable and is replaced. All this leads to ainterdependent, weakly non-linear model (seebroadening of (2):Heilemann, 1998a). In macroeconomic perspec-ay 5 f y ; x ; x ; b (3)s dt1j t2k1j t2i1j t2i1j tive, it can be partitioned into five sectors:

a production (five stochastic equations; 17 defini-where: x : vector of adjustments to thet2i1j tions), demand (8; 24), prices (8; 12), incomeconstant term; or, more specifically:distribution (6; 13), and government (14; 20).

0 0 The list of major exogenous variables includesy 5 f y ; x ; x , x ; b (4)s dt1j t2k1j t2i1j t2i1j t2i1jpolicy-determined variables, such as the Social

0 S security contribution rate, Government con-where: x /x : vectors of adjustmentst2i1j t2i1j

struction outlays, and Interest rates. Majoraccounting for a priori information or ‘objectiveinternationally determined exogenous variablesconsiderations’ about economic policy and datainclude World trade (volume index) and Importetc., or ‘subjective considerations’ such as mis-prices. The theoretical foundations of the equa-specification (Intriligator, Bodkin & Hsiao,tions are, as for most applied econometric1996, p. 520).models, somewhat eclectic, including neo-clas-These additions are made in various phases ofsical and Keynesian, as well as monetaristthe forecasting process. While the first oneselements. The architecture of the model, how-ever, is in the Keynes /Klein tradition. Withrespect to the roles played by demand and

1 money (interest rates) and the stability of theFor example, in Germany the annual negotiated wage rateprivate sector, the model may be labelled ascan be reliably predicted from the wage settlements of the

first 4–5 months of the current year. post-Keynesian. On the other hand much atten-


tion is paid to elements of the ‘new/old macro- with most short-run models the impact andeconomic consensus’ such as supply side elas- interim multipliers are of greater importance.ticity, symmetrical reaction of the Phillips curveor to ‘rational’ price expectations, etc. (Blinder,1992). 4. The 1997 forecast

The model is re-estimated twice a year fromAs indicated above, the process of forecastingseasonal unadjusted data. The sample period is

with an econometric models takes severalthe same for all equations, and, to avoid cyclicalrounds until the final result forecast is reached.bias and the inclusion of a past seen as irrele-Here, the process is broken up into five steps:vant, covers only the last 40 quarters of the(1) examination of the previous forecast; (2)available data (‘moving window’). For thestructural analysis of the newly estimatedcurrent study the sample period is 1986-III tomodel; (3) survey of starting conditions and1996-II, with West German data up to 1990-II,setting of the exogenous variables and of otherand thereafter all German data with a number ofassumptions; (4) production of the forecastone time and permanent dummy variables in

2 proper; (5) ex post examination of the forecast.most equations. The parameters are estimatedThe data reported here are for reasons ofby ordinary least squares (OLS). Like mostsimplicity annual data only, though in practiceother models of this type and size, it hasthe quarterly data have to be examined too,comparatively few within /between-sector inter-adding another important dimension to theactions. The sectors are more or less recursive.process.The model’s forecasting accuracy has been

widely examined and is part of the publishedspring forecasts with the model (e.g. Rheinisch- 4.1. Examination of the previous forecast

¨ ¨Westfalisches Institut fur WirtschaftsforschungThe examination of the previous forecast(Barabas, 1994; Hrsg., 1998)). The accuracy of

(made in Spring 1996, covering 1996-I to 1997-the typical ex post and ex ante forecasts as wellIV) is basically the same as what will be doneas the predictions for particular situations, arelater under (Section 4.5). As Table 1 discloses,examined. Based on its ex post performance, thethe assumptions for the exogenous variables forforecast accuracy of the models predictions of1996 were pretty accurate. On the other hand,important macroeconomic variables was foundsome of the assumptions for 1997 were, com-to be acceptable in general (Heilemann, 1998b,pared to their 1986/95 average, relatively inac-pp. 86ff.).curate. Public construction outlays and Long-It should be emphasized that in terms of itsterm interest rates were overestimated, whilegeneral structure, the model is basically short-the international development (World trade,term oriented. The sample period covers onlyImport prices) was much underestimated. Thethe last two cycles, and the specifications areerror of the original forecast of the rate ofmade with the eight quarters forecast horizon ingrowth of GDP is much below average (Tablemind. Though the model’s long-term properties2). However, when we correct for faulty as-and its determinants are also of interest andsumptions (for details see Table 4, below) ithave been found in general to be ‘stable’. But asdoubles and increases above average. The in-fluence of the add factors is negligible, with the

2 exception of the Government deficit, as Table 2A complete listing of equations, the various estimationresults, etc. are available from the author. shows. All in all, the forecast accuracy of the


Table 1aForecast errors of the exogenous variables 1996/97

bVariables Time of 1996 1997 Averageforecast 1986–1995

Public construction outlays, 1995, spring 10.7 10.1 4.1rates of change, percent 1995, autumn 8.7 8.7 3.9

1996, spring 6.7 8.7 2.51996, autumn 1.1 6.8 1.4

Social security contribution 1995, spring 20.5 20.3 0.2rates, percent 1995, autumn 0.0 20.1 0.2

1996, spring 0.1 0.1 0.11996, autumn 0.1 0.1 0.1

World exports (volume index) 1995, spring 1.0 22.3 2.319805100, rates of change 1995, autumn 1.0 22.2 1.7

1996, spring 20.2 22.8 1.41996, autumn 0.3 21.7 0.6

Price index of imports (deflator), 1995, spring 2.2 20.9 4.1rates of change, percent 1995, autumn 0.9 21.6 3.5

1996, spring 0.0 21.3 1.21996, autumn 20.4 0.4 0.6

Short-term interest rate, 1995, spring 1.2 0.8 0.9percent 1995, autumn 0.8 0.0 0.5

1996, spring 20.2 20.1 0.31996, autumn 20.1 0.0 0.1

Long-term interest rate, 1995, spring 1.3 1.2 0.7percent 1995, autumn 0.7 0.7 0.6

1996, spring 0.3 0.3 0.21996, autumn 0.1 0.0 0.1

Source: Own computations and Heilemann (1998a).a Forecast minus actual data.b Average of absolute differences.

Spring forecast was unusually high, although with the inspection of the estimation results fornot for the right reasons. This is not a new single equations. The first inspection examinesexperience with macroeconometric models the statistical and parameter characteristics of(Evans et al., 1972; McNees, 1990) but with all 41 stochastic equations over the new 40RWI model a still rare one (cf. Heilemann, quarter sample period of fit. This is made in the1998a). light of a more than 20 years’ experience with

most equations. In the present case, checkingthe estimation results did not reveal any4.2. Structural analysis of the newly estimatednoteworthy changes — neither as to the changesmodelin the parameters /elasticities nor as to the

‘Structural analysis’ means here the examina- statistical quality of the equations. The onlytion of the single stochastic equations as well as remarkable change in parameters was an in-the behaviour of the complete model. It starts creased value of the lagged endogenous variable


Table 2aForecast errors of the endogenous variables: 1996 and 1997, rates of change

Variable Time of Errors with exogenous variables Averagebforecast 1986–1995

Assumed Actual Actual, withoutadds

1996 1997 1996 1997 1996 1997

Wage and salary earner, 1995, spring 1047 – 843 – 1378 – 572in 1000 1995, autumn 828 – 728 – 1243 – 532

1996, spring 372 1117 375 1226 393 1351 1941996, autumn 128 962 118 975 118 971 174

Private consumption, 1995, spring 1.8 – 0.9 – 0.3 – 0.9real 1995, autumn 1.9 – 1.7 – 1.2 – 0.8

1996, spring 1.8 1.4 1.6 1.7 0.5 1.8 0.81996, autumn 0.8 1.9 0.5 1.7 0.4 1.6 0.3

Fixed investment, 1995, spring 4.4 – 20.2 – 20.9 – 3.6real 1995, autumn 3.7 – 20.5 – 21.1 – 2.5

1996, spring 22.0 2.6 21.6 1.8 20.8 3.3 2.11996, autumn 0.0 4.4 21.0 5.4 20.6 5.3 1.2

Exports, real 1995, spring 0.5 – 0.1 – 0.3 – 1.81995, autumn 1.3 – 0.7 – 0.8 – 2.01996, spring 21.4 24.9 21.1 22.8 1.8 0.1 2.51996, autumn 20.8 23.4 20.7 20.6 20.9 20.7 1.1

GDP, real 1995, spring 1.6 – 0.2 – 20.2 – 1.11995, autumn 1.5 – 0.3 – 0.0 – 0.91996, spring 20.1 0.0 0.0 0.2 0.1 1.3 0.71996, autumn 0.3 0.6 0.0 1.2 0.0 1.4 0.5

Price index Private con- 1995, spring 20.2 – 20.1 – 20.4 – 0.5sumption (deflator) 1995, autumn 20.1 – 20.5 – 20.9 – 0.3

1996, spring 20.5 20.2 20.6 20.7 20.7 20.6 0.31996, autumn 20.2 20.3 20.2 20.5 20.3 20.6 0.1

Government deficit, 1995, spring 48.4 – 55.0 – 103.7 – 23billion DM 1995, autumn 62.0 – 38.9 – 62 – 39

1996, spring 24.0 226.1 210.8 219.9 7.3 21.0 331996, autumn 2.0 224.1 29.2 226.1 217.3 236.2 5

Source: Own computations and Heilemann, 1998a.a Forecast minus actual data.b Average of absolute differences.

in the Private consumption function, but this did results for root-mean-square-percentage-errornot affect either the elasticity or the mean lag. (RMSPE) are shown, but additional error mea-

The single equation analysis is followed by sures (mean absolute / square error, Theil’san examination of the (complete) model’s static inequality coefficient etc.) are also examined.and six quarters dynamic simulation accuracy They inform about the model’s dynamic stabili-(Table 3, columns (2)–(6)). Here only the ty, though in general the insight from these

U. Heilemann / International Journal of Forecasting 18 (2002) 85 –105 91Table 3Ex post accuracy of the RWI model forecast for 1996/97

In-sample Out-of-samplea a bRMSPE RMSPE Absolute errors

1987-III /1996-II 1996-III /1997-IV 1996 1997

Type of model solution:

36 quarters 6*6 quarters 6 quarters

Single Model Model Model simulation—dynamicequation simulation— simulation—

static dynamic(1) (2) (3) (4) (5) (6)

GDP, originWage and salary earners – 0.3 0.4 2.4 0.5 2.3Productivity per hour – 0.7 0.7 3.6 20.1 21.4GDP, real – 0.7 0.7 1.5 0.1 1.1

Demand, realPrivate consumption 0.4 0.9 1.0 1.7 0.5 1.4Government consumption 1.1 1.1 1.1 1.1 20.4 0.0Gross fixed capital formation – 1.7 1.7 5.3 20.6 5.1Machinery 4.1 3.2 3.6 6.4 20.6 7.5Construction – 1.4 1.4 4.9 20.6 3.6

cChange in inventories, bill. DM 3.7 4.2 4.4 4.1 7.5 213.5cNet exports, bill. DM – 4.5 4.7 4.5 24.0 28.0

Exports 1.7 1.7 1.8 2.5 21.0 20.6Imports 1.2 2.2 2.2 0.9 20.1 20.2GDP – 0.7 0.7 1.5 0.1 1.1

Price deflators, 19915100Private consumption 0.3 0.3 0.3 0.6 20.2 20.6GDP – 0.4 0.4 0.7 0.3 0.0

GDP, incomeIncomeGross wage and salary income – 0.8 0.9 3.0 0.7 2.5Gross profits /assets – 4.0 3.9 5.8 21.5 22.4National income – 1.0 1.0 1.7 0.1 1.0Net wage income—Net wage and salary income – 0.9 0.9 3.3 0.7 2.8—Net profits /assets income – 4.8 4.7 7.4 23.0 22.8

GovernmentGross income – 1.1 1.1 1.7 20.2 1.5Expenditures – 1.1 1.0 1.7 0.5 1.1

cDeficit, bill. DM – 5.2 4.9 5.9 217.3 214.2

Source: Own computations.a Root mean square percentage error (RMSPE).b Annual instead of quarterly values.c Root mean square error (RMSE).


simulations are limited (Pagan, 1989; Chong & which can be attributed to (1) the investmentHendry, 1986). As to be expected, compared boom during unification, (2) government effortswith the single equation results the fit quality of to meet the Maastricht-deficit criterion, (3) themost equations is much reduced, though in aftermath of a strong Deutschmark, and (4) thesome cases (‘aggregation’) gains are also found high real interest rates (see, for example,(e.g. for Fixed investment). The results of those ¨ ¨Rheinisch-Westfalisches Institut furvariables that are primarily explained by pre- Wirtschaftsforschung (Hrsg., 1997a,b)).determined variables (e.g. Exports, Distributed The values fixed for the exogenous variablesprofits, Government consumption), of course, do were the same as those used for the RWI model

3not differ from those for the single equation in the JD, what is not always the case. Aresults. The differences between the static difficult problem to solve here is the quarterly(Table 3, column (2)) and the six quarter distribution of the values. When, as is usuallydynamic simulation results (column (3)) are the case for government variables, no particularsmall and after four quarters both are nearly the information is provided, a steady developmentsame—the model is a system of difference is assumed.equations of 15th order but 90% of the lags are For a number of reasons the process of fixingwithin the four quarters range. Although the these assumptions is, again, recursive and inter-quality of the stochastic equations had some- dependent. It is recursive because sometimes thewhat suffered, the results gave no cause for assumptions have to be changed when there-specification. results appear as implausible or incoherent. The

A complete analysis of the model’s multi- interest rates that are assumed may have to bepliers and elasticities was not made in the adjusted if and when the monetary authoritiespresent case—this is part of the annual generalmodel inspections. The size and the time profile

3They are gained by informal methods as follows: Publicof Government construction outlays (real terms)construction outlays are estimated on the basis of public

multiplier was basically the same for the old construction orders, the fiscal situation of the largeand the newly estimated model (0.6 and 0.7 for municipalities (municipalities are the largest public inves-

tors) and fiscal plans of federal and state governments; the1996 and 0.1 and 0.2 for 1997). Their sensitivi-Social security contribution rate is a parameter summa-ty as to shift of the sample period is usually notrizing the contribution rates of employers /employees tohigh, which is not too surprising given theunemployment insurance, health insurance and old age

stability of the parameters. pension insurance, usually fixed by the government forseveral years ahead, though, in recent time, often changedon short notice because of budgetary considerations; World4.3. Survey of the starting conditions, theexports and the Price index of imports are estimated on thesetting of the exogenous variables and ofbase of own estimations of the international developmentother assumptionscomplemented /checked by forecasts of international or-ganisations like the IMF or OECD; the values of the Short

The general economic environment had been term interest rate and of the Long term interest rate resultset out in detail in the autumn from a number of variables such as monetary aggregates,

inflation rates, expected economic growth, expected gov-Gemeinschaftsdiagnose [Joint Diagnosis (JD)]ernment deficit and internal forecasts of these variables.(Arbeitsgemeinschaft, 1996). It may be summa-Experiments with an endogenization of the interest ratesrised as follows: in 1996, the German economye.g. on the basis of the so-called Feldstein /Eckstein

was still in the early phase of a weak upswing, equations suffered from the fact that US interest rates weremostly driven by exports. The expansion was of particular influence and proved to be more difficult toslower than usual in this phase of the cycle, forecast for us than German interest rates.


reveal a different view. Another example may transferred to the federal budget and a numberbe that assumptions about government expendi- of tax reductions (ca. 9 billion DM net) and cutstures contradict the government’s policy of in government expenditure (about 12 billiondeficit reduction. The process is interdependent DM) as consequences of the Tax reforms (Table

5because assumptions are interdependent and a 5). The inclusion of outside-model informationrevision, e.g. of the assumptions for World trade is also a recursive and interdependent process,also requires a rethinking of those for Import particularly when including them makes modelprices or Interest rates. The process can partly results appear as incoherent. Because of thebe formalized by using policy reaction functions model interactions, correcting outside informa-or their variants, though our experiences with tion is often not as easy as changing assump-this type of explanation has so far been dis- tions, especially when the changes are made inappointing. New information on policies or one or more equations with lagged endogenousabout the world economy, etc. during this stage variables (for a formal treatment of these prob-may, of course, also lead to revisions of the lems, see e.g. Klein & Young, 1980, pp. 107ff.).assumptions. For the 1997 forecast no such While the effects of these corrections onchanges had to be made. growth, inflation, or employment are rather

The deviations of the assumptions from the small, their consequences for the governmentprevious forecast (Spring 1996) and their conse- deficit are large—about 0.3% of the GDP/deficitquences are shown in columns A and B of ratio. An even stronger intervention is the 0.5

4Table 4. Major revisions involve Public invest- reduction in the rate of change of the Negotiatedment, a consequence of greater attention paid by wage rate to 2.2%, because of strong evidencefiscal policy to the deficit, and for the Interest that wage policy could be expected to be muchrate. Only the former were of particular impor- more employment-oriented than the model’stance for the economic development within the function suggested (3.5%). The effects on em-next 18 months. All in all, the ‘new’ values for ployment and growth in an annual perspectivethe exogenous variables did not have much are negligible, but the inflation rate would haveinfluence on the forecast of growth, employ- been 0.4 percentage points higher.ment, inflation, and government deficit. Here as in most previous cases, the total

Having fixed the values of the exogenous effect of the various policies is rather small,variables, the forecasting process could have, in rarely exceeding 0.2% of GDP. Given this andprinciple, started. However, a variety of addi- the lags of macroeconomic reactions, it could betional information had to be assessed and some argued that their inclusion is not necessary. Inof it included. This information included the particular as the inclusion is often arbitrary,Surplus of the Deutsche Bundesbank to be because it is difficult to decide which policies to

include: those which are discussed in the ruling4 parties? or those which have been passed by theThe Bundesbank surplus is estimated on the basis of its

cabinet? or those which have been enacted? Oneinterest revenues, currency gains corrected for cost, appro-must also decide on the amounts and timing ofpriated retained earnings and accrued liabilities as shown

in its balance sheets for the first two or three quarters. The the policies. The amounts often create difficultdata on Tax reforms and its consequences for revenue and problems, e.g. when the government expectsexpenditures have been estimated using government in-formation accompanying tax laws or proposals. The same

5Later changes of these new values of the exogenousholds for the ‘Budget consolidation’ measures. Since therevariables in the following columns are due to revisions ofwas no other information, the values were evenly distribut-the data base.ed over the forecast period (1997).

94U

.H

eilemann

/International

Journalof

Forecasting

18(2002)

85–105

Table 4The evolution of the 1996/1997 RWI model forecast 1996 and 1997; rates of change against previous year in percent

b c e(A) Old model / (B) Old model / (C) New model / (D) New model / Actual dataa dassumption new assumption old assumption new assumption

1996 1997 1996 1997 1996 1997 1996 1997 1996 1997fAssumptions

Public construction outlays 20.9 0.2 24.9 21.2 25.7 3.5 26.5 21.2 26.8 29.9Social security, contribution rates 20.3 20.7 20.3 20.9 20.3 20.7 20.3 20.9 20.3 21.0World exports (volume index) 5.8 7.5 6.3 7.6 6.0 7.5 6.3 7.6 5.9 9.8Price index of the imports 0.6 2.1 0.2 1.4 0.8 2.1 0.2 1.4 0.7 3.0Short term interest rate, percent 3.2 4.1 3.3 3.3 3.2 4.1 3.3 3.3 3.3 3.3Long term interest rate, percent 5.9 6.3 5.7 5.8 5.9 6.3 5.7 5.8 5.6 5.1

ForecastGDP originWage and salary earners 20.1 0.6 20.1 0.7 20.8 0.9 20.8 0.9 21.2 21.4Productivity per hour 1.9 1.7 1.9 1.7 3.1 2.1 3.1 2.0 – –GDP, real 1.3 2.2 1.3 2.2 1.7 3.0 1.7 2.8 1.4 2.2

Demand, realPrivate consumption 3.1 1.6 3.1 1.8 2.0 2.0 2.1 2.1 1.3 0.2Government consumption 1.3 20.1 1.3 0.0 2.2 20.2 2.2 20.2 2.4 20.4Gross fixed capital formation 22.8 2.8 23.1 3.1 20.9 4.9 20.8 4.6 20.8 0.2Machinery 1.8 5.5 2.1 5.8 2.0 9.0 2.3 8.8 2.4 3.9Construction 25.6 1.0 26.2 1.3 22.6 2.3 22.7 2.0 22.7 22.2Change in inventories, bill. DM 25.0 33.2 25.1 33.5 32.7 43.3 33.0 42.9 23.9 57.0Net exports, bill. DM 27.6 3.1 25.4 1.3 27.9 5.9 27.9 2.8 2.4 31.1Exports 3.5 5.8 4.0 5.2 3.9 7.8 4.1 7.3 4.9 10.7Imports 3.7 4.4 3.9 4.3 2.9 6.1 3.1 5.9 2.6 7.0GDP 1.3 2.2 1.3 2.2 1.7 3.0 1.7 2.8 1.4 2.2

U.

Heilem

ann/

InternationalJournal

ofF

orecasting18

(2002)85

–10595

Price deflator, 19915100Private consumption 1.4 1.7 1.3 1.5 1.8 1.8 1.7 1.6 1.9 1.9GDP 1.1 1.1 1.0 1.0 1.4 1.3 1.4 1.1 1.0 0.6

GDP incomeIncomeWage income 3.3 3.6 3.0 3.3 2.8 3.8 2.7 3.5 1.0 0.2Profits /assets income 22.3 2.7 21.4 3.1 3.5 6.0 4.3 5.6 3.6 8.9National income 1.7 3.3 1.7 3.3 3.0 4.4 3.1 4.1 1.8 2.7Net wage income 2.2 21.8Net profits /assets income 23.1 4.0 22.0 4.3 2.6 6.8 3.5 6.3 4.3 10.7

GovernmentDeficit, bill. DM 2138 2128 2138 2119 2133 2137 2132 2126 2126 2102Revenues 1.2 3.0 0.9 3.2 1.2 3.6 1.1 3.6 0.5 1.9Expenditures 29.8 2.2 210.0 1.9 210.0 3.5 210.1 3.0 210.8 0.4

Source: Own computations and official data.a Forecast No. 43.b Sample period 1986-I to 1995-IV.c Sample period 1986-III to 1996-II.d Sample period 1986-III to 1996-II.e As of spring 1998.f Growth rates for old assumptions if fixed in levels changed with new sample period.


Table 5Modifying the RWI model forecast for 1996/97 billion DM

aMeasures and variables 1997 1997

Quarter

1 2 3 4

Surplus of Deutsche BundesbankEarned income of government 8.5 – 8.5 – –

Tax reformsIncome taxes on wages and salaries 28.4 22.1 22.1 22.1 22.1Increase of child allowance /benefit 23.8 21.0 21.0 21.0 21.0Abolishment of property tax 25.3 21.3 21.3 21.3 21.3Others 0.7 0.2 0.2 0.2 0.2

Income taxes on enterprise and propertyAbolishment of property tax 23.5 20.9 20.9 20.9 20.9

Excise and other ‘indirect’ taxes 1.6 0.4 0.4 0.4 0.4Reform of motor vehicle tax 2.1 0.5 0.5 0.5 0.5Abolishment of trade tax on business capital 23.7 20.9 20.9 20.9 20.9Increase of real property transfer tax from 2 to 3% 3.2 0.8 0.8 0.8 0.8

Changes of CPI (%)Reform of motor vehicle tax 0.1 0.1 0.1 0.1 0.1

Received property transfersAmendment of inheritance (gift) tax 1.6 0.4 0.4 0.4 0.4

Personal property /entrepreneurial incomeAmendment of inheritance (gift) tax (by 25vH) 0.4 0.1 0.1 0.1 0.1

Budget consolidationGovernment consumption 28.3 22.1 22.1 22.1 22.1Transfers 23.5 20.9 20.9 20.9 20.9

Other modifications – – – – –

¨ ¨Source: Rheinisch-Westfalisches Institut fur Wirtschaftsforschung (Hrsg., 1996).a In bold type: general type of measures, in italics: model variables affected, in normal type: elements of the measures

included.

revenues from a tax hike without taking its fiscal-data into National Accounts (NA)-datamacroeconomic repercussions into account, should be added.which is usually the case. The difficulties of Nevertheless, for the present model, thesedating the expected extra revenues have already ‘objective considerations’ have proved to be

6been mentioned, the problems of translating necessary. First, professional readers of theforecast, in particular when they look beyond

6If the model’s tax equations exactly mirrored the tax code, GDP growth or inflation, want to know whichthis would not be a problem, but the tax equations are short

policies have been included, in which way andcuts in which these kind of changes have to be introducedwith which consequences. Second, includingin rather simple ways. Luckily, after a few periods, the

newly estimated equations correct automatically. these policy considerations definitely improves


the accuracy of the deficit forecast, though not of minor importance, such as the deflator fornecessarily that of the rest of the model because business construction, caused problems. Theyof the rather weak links of the government were overcome by add factors. The secondsector with the real and the price sphere. check only required a change in the Negotiated

wage rate (see above). The fact that usuallythere are only a few changes of this type is a

4.4. Production of the forecast consequence of the short and hence rather‘actual’ sample period used to estimate the

The forecast proper starts with a number of model. This is in contrast to the often heavy andtest runs to get an impression of the effects of numerous add factoring with less actual modelthe new set of assumptions (here: the exogenous structures (see e.g. Dicks & Burrell, 1994, pp.variables with old ‘adds’). First, the old model 124ff.; McNees, 1990). As to the third check,version is re-run with the new assumptions, the differences between our predictions andone-by-one, or, all at a time. Then, the newly those of the JD and the IMF (see Table 7,estimated version of the model is run with old Appendix) were rather small. They were welland then with new assumptions. Though the within the usual tolerances and could be ex-RWI model is formally a non-linear model, its plained by the influence of different assump-reactions are linear. Thus the model results of tions made. If larger differences show up, thesingle variations can be added so that the implications of the competing forecasts arecontribution of each change to the final result tested by exogenization or add factoring one orcan be identified. As Table 4 displays, the several of the variables that are out of line.consequences for the forecast differ: While the Where possible, the model is re-run with one ornew assumptions (Table 4, columns B) do not more of the equations of a competing model. Ifchange the picture much, the ‘new sample the results are plausible they are summarized inperiod’ (1987-III to 1996-I with old assump- a list (see Table 5 above). Here no such changestions, columns C) does so to a considerable had to be made. These adjustments are also thedegree. This holds not only for real GDP but outcome of an often lengthy iterative process.also for employment, prices, and deficit. The Though all these changes are rather time con-coherence of the forecast is checked in three suming and costly, compared with the practiceways: first, by examining the match within the examined by McNees (1990), they seem to beforecast; second, with outside information (e.g. modest and, in general, of negligible importanceorders, retail data, housing starts, survey data, for the final forecast.etc.) third, by comparison with competing The final forecast (new model version, new

7forecasts. assumptions, and policy adds (Table 4, columnThe first check showed that only aggregates D) was published November 13, 1996. It por-

trays the German economy in 1997 very similarto that shown under column C: a considerableincrease of real growth in 1997, mainly attribut-able to the increase of Fixed investment, includ-ing Changes in stocks and Net exports. Inflation

7Since 1998, the check of the various forecasts also is still moderate and employment improvesincludes a cyclical classification of the quarterly forecast

slowly. The comparison of the old forecastresults based on a four-phase-scheme, derived by multi-(column A) and the new one indicates ‘forecast¨variate discriminance analysis (Heilemann & Munch,

1998). stability’. This is not surprising, given the small


changes of the model structure and the Our ex post forecast evaluation compares the8assumptions. forecast with the first published actual data

A comparison of this prediction with those of released by the Statistische Bundesamt [Federalmajor macroeconomic forecasters (Table 7) Statistical Office] in March 1998. For reasons ofshows that the differences are small. This is true simplicity this paper only analyses the 1997despite the fact that three of them were pro- forecasts, thus ignoring the effects of differentduced later and so have a broader information bases of the forecasts.base. The JD, published on October 29, 1996 Table 4 (columns D, E) indicate that the GDP(Arbeitsgemeinschaft, 1996), had forecast, prob- growth rate was overestimated by 0.6 percent-ably based on rather similar assumptions as the age points or more than 25%. This was mainly amodel forecast, GDP growth rates of 1.5% in consequence of a massive overestimation of1996 and 2.5% in 1997, and inflation rates Private consumption (nearly 2 percentage(consumption deflator) of 2.0 and 1.5%; the points) and Fixed investment (4.4 percentage

¨Sachverstandigenrat [Council of Economic Ex- points), partially offset by an even more severeperts (CEE)] forecast, published on November underestimation of the Changes in stocks and of

¨18, 1996 (Sachverstandigenrat, 1996, Ziffer Net exports. The result for employment is even215ff.), as well the Jahreswirtschaftsbericht der more unpleasant. Given a productivity-growthBundesregierung [Annual Economic Report of caused ‘employment barrier’ of about 2.0%the Federal Government (AER)] issued January growth of real GDP, instead of a considerable29, 1997 (Bundesregierung, 1997, Ziffer 122ff.), increase, the economy experienced a furtherand also the IMF (1996) forecast, issued in severe drop in employment. There is also someOctober 1996, came up with rather similar evidence of a steep increase of productivity, notgrowth and inflation forecasts. Only the OECD uncommon in the early upswing phase of theforecast, published in December 1996 (OECD, cycle. The forecasts for the Deflator of private1996), presented a different picture. consumption overestimated the actual change

because of both lower wage settlements andhigher productivity. The deficit for 1997 wasoverestimated by more than 25% — certainly an4.5. Ex post examination of the forecastoutcome of a number of additional measures to

Productions and presentations of new macro- meet the Maastricht-deficit criterion by a moreeconomic forecasts once used to start with an restrained expenditure policy than had been

9examination of the previous forecast. Today this assumed.is often ignored, or, at least, not many resources What caused these errors? First of all itare committed to this (for what is still an should be noted that though the size of theoutstanding example of such an analysis, cf. errors is high, the errors still lie more or lessSapir, 1949). within the margins which could be reasonably

8Though, it has to be realized that ‘forecast stability’9doesn’t matter much in present macroeconomic forecasts— The rate of change of Government expenditures was

regardless of the fact that it is nowadays so easy to overestimated by 2.6 percentage points (1986/1995 aver-demonstrate. Hopefully, recent practice of the IMF finds age error: 2.1), Government revenues by 1.7 (1.5) per-some followers. centage points.


expected (see above). Some causes are obvious models such as those of the JD, the CEE and the(see ‘Assumptions’ in columns D and E, Table AER, it is nearly impossible for third parties to4): the lower Construction outlays of the gov- do this. For macroeconometric model forecasts,ernment sector, the increases of social security there are less difficulties, but one is still re-contributions and the much lower wage in- quired to have the model and the forecast atcreases. Policy impulses outside exogenous one’s disposal and third parties rarely havevariables (Table 3) amounted to about 10 bill these data. (The difficulties of error analysis inDM, which ‘reduced’ real GDP growth in 1997 non-model based forecasts are illustrated inby 0.2 percentage points. Fintzen and Stekler, 1999). Assuming that all

Repeating the forecast with actual data for the forecasts were based on rather similar assump-exogenous variables (including Negotiated wage tions, which is probably too strong an assump-rates) unfortunately produces even larger fore- tion for the (rather early) forecasts of JD, thecast errors (Table 6, column B). Numerically, IMF and the RWI model, all forecast errors stillthis can be attributed to a further deterioration lie well within the margins set by past ex-of the Fixed investment forecast, which was perience (Heilemann, 1998a,b). This result isonly partly balanced by improvements in the even more impressive considering that there isforecasts of the Change of stocks and Net little evidence that the various policies to meetexports. The errors of the employment and the Maastricht Treaty have been allowed for ininflation forecasts are hardly affected. This is the exogenous variables.surprising in the context of an implicit pro- The assumption-corrected performance of theduction function. model yields also some conclusions about the

These findings should be compared with forecast accuracy of the various variables. How-those for competing forecasts (Table 7). Un- ever, given the definitional and behaviouralfortunately, the JD does not analyze its forecast- links between macroeconomic variables one hasing accuracy and the forecast analysis of the to be careful with determining their forecast

10CEE (1997, Ziffer 248ff.) is very brief and rank or the size of their genuine errors. Thegeneral. The real GDP forecasts of 2.5% of the forecast of consumption depends on forecasts ofJD and the CEE showed an overestimation of a host of other variables, more or less the

110.3, while both the IMF and the OECD had complete model. With this caveat, it seemsmade lower or no errors at all. The overestima- obvious that unusually large errors in all fore-tion of the model was 0.6. As to inflation, withthe exception of the IMF and the CEE, allforecasters had underestimated its rate of 1.9%(Deflator for Private consumption)—a conse- 10Formally speaking, multivariate /multiperiod macroecon-quence of the unexpected 15% devaluation of omic forecasts are ‘konjunktive Prognosen’ [linked singlethe DM. forecasts]. This is also true for VAR models (via defini-

The influence of the assumptions on these tions). In a strict sense this forbids any accuracy statementon single variables or on some aggregates like GDP,results are difficult to determine. Non-modelinflation, etc.based forecasts usually are not very clear about11Although the resulting empirical problems could betheir assumptions, and in any case the forecastsreduced for some periods by subsequent exogenization,

are difficult to correct for flaws in the assump- this is beyond the scope of this paper (for some sug-tions. Although it is possible in principle to gestions as to the solution by ‘logical’ analysis, see Gilli,correct the forecasts derived from informal GDP 1984).


Table 6Forecasts with the RWI model 1996 and 1997: rates of change against previous year in percent

a d(A) Model forecast (B) Old model /actual (C) New model /new (D) Actual datab cassumption assumption

1996 1997 1996 1997 1996 1997 1996 1997eAssumptions

Public construction outlays 26.5 21.2 28.9 213.1 26.8 29.9 26.8 29.9Social security, contribution rates 20.3 20.9 20.3 21.0 20.3 21.0 20.3 21.0World exports (volume index) 6.3 7.6 6.4 9.8 5.9 9.8 5.9 9.8Price index of the imports 0.2 1.4 0.6 3.0 0.7 3.0 0.7 3.0Short term interest rate, percent 3.3 3.3 3.3 3.3 3.3 3.3 3.3 3.3Long term interest rate, percent 5.7 5.8 5.6 5.1 5.6 5.1 5.6 5.1

ForecastGDP originWage and salary earners 20.8 0.9 20.8 0.9 21.3 21.2 21.2 21.4Productivity per hour 3.1 2.0 3.1 2.3 2.6 3.6 – –GDP, real 1.7 2.8 1.6 3.3 1.1 2.3 1.4 2.2

Demand, realPrivate consumption 2.1 2.1 2.0 1.9 1.3 0.0 1.3 0.2Government consumption 2.2 20.2 2.2 20.2 2.0 0.5 2.4 20.4Gross fixed capital formation 20.8 4.6 21.2 5.1 21.5 0.6 20.8 0.2Machinery 2.3 8.8 2.0 11.3 2.3 3.6 2.4 3.9Construction 22.7 2.0 23.1 1.1 23.9 21.3 22.7 22.2Change in inventories, billion DM 33.0 42.9 32.5 44.8 28.4 34.4 23.9 57.0Net exports, billion DM 27.9 2.8 26.1 17.0 25.5 51.5 2.4 31.1Exports 4.1 7.3 4.2 9.9 4.8 11.7 4.9 10.7Imports 3.1 5.9 3.0 7.1 3.2 4.8 2.6 7.0GDP 1.7 2.8 1.6 3.3 1.1 2.3 1.4 2.2

Price deflator, 19915100Private consumption 1.7 1.6 1.8 1.7 1.9 2.0 1.9 1.9GDP 1.4 1.1 1.4 1.0 1.1 0.4 1.0 0.6

GDP incomeIncomeWage income 2.7 3.5 2.4 2.9 1.1 0.5 1.0 0.2Profits /assets income 4.3 5.6 4.5 8.4 5.9 7.9 3.6 8.9National income 3.1 4.1 3.0 4.5 2.4 2.6 1.8 2.7Net wage income 2.2 21.8Net profits /assets income 3.5 6.3 3.7 9.3 6.0 10.1 4.3 10.7

GovernmentDeficit, billion DM 2132 2126 2136 2122 2132 2101 2126 2102Revenues 1.1 3.6 0.8 3.4 0.5 2.0 0.5 1.9Expenditures 210.1 3.0 210.2 2.4 210.5 0.1 210.8 0.4

Source: Own computations and official data.a See Table 4, columns D.b Sample period 1986-III to 1996-II, but actual data for assumptions.c Forecast No. 47 (Spring 1998).d As of Spring 1998.e Growth for old assumptions if fixed in levels changed with new sample period.

U.

Heilem

ann/

InternationalJournal

ofF

orecasting18

(2002)85

–105101

Table 7Selected macroeconomic forecasts for 1996/97: 1996 and 1997; rates of change against previous year in percent

IMF JD RWI model CEE OECD AER Actual data

a1996 1997 1996 1997 1996 1997 1996 1997 1996 1997 1996 1997 1996 1997October 1996 October 1996 November 1996 November 1996 December 1996 January 1997 March 1998

GDP originWage and salary earners 21.3 0.2 21.0 0.0 20.8 0.9 21.00 0.00 20.9 0.2 21.1 20.5 21.2 21.4Productivity per hour – – 3.0 3.0 3.1 2.0 3.50 3.00 – – – – – –GDP, real 1.3 2.4 1.5 2.5 1.7 2.8 1.50 2.50 1.1 2.2 1.4 2.5 1.4 2.2

Demand, realPrivate consumption 2.0 2.2 1.5 1.5 2.1 2.1 1.75 1.75 1.7 2.0 1.4 1.5 1.3 0.2Government consumption 2.7 1.5 2.5 1.0 2.2 20.2 2.75 1.00 2.7 0.7 2.8 0.5 2.4 20.4Gross fixed capital formation 21.2 3.8 22.0 1.0 20.8 4.6 21.25 1.25 22.2 1.4 20.7 1.5 20.8 0.2Machinery – – 0.5 4.0 2.3 8.8 0.50 3.50 0.6 3.9 2.6 5.0 2.4 3.9Construction – – 23.5 21.0 22.7 2.0 22.25 20.25 23.9 20.2 22.7 21.0 22.7 22.2Change in inventories, billion DM – – 28.0 43.5 33.0 42.9 27.50 40.50 – – – – 23.9 57.0Net exports, billion DM – – 22.0 12.0 27.9 2.8 23.00 12.50 – – – – 2.4 31.1Exports 3.3 5.4 4.5 6.0 4.1 7.3 3.50 6.75 3.7 5.9 4.6 6.5 4.9 10.7Imports 2.0 5.0 2.5 4.0 3.1 5.9 2.00 4.75 2.4 4.0 2.0 4.5 2.6 7.0GDP 1.3 2.4 1.5 2.5 1.7 2.8 1.50 2.50 1.1 2.2 1.4 2.5 1.4 2.2

Price deflator, 19915100Private consumption 1.6 1.7 2.0 1.5 1.7 1.6 1.75 1.75 1.7 1.5 1.8 1.5 1.9 1.9GDP 1.4 1.6 1.5 1.0 1.4 1.1 1.50 1.50 1.3 1.2 1.0 1.0 1.0 0.6

GDP incomeIncomeWage income – – 1.5 2.0 2.7 3.5 1.14 2.00 1.5 2.6 1.2 1.5 1.0 0.2Profits /assets income – – 5.5 7.5 4.3 5.6 4.75 9.75 – – 3.0 7.0 3.6 8.9National income – – 3.0 3.5 3.1 4.1 2.25 4.25 – – 1.7 3.0 1.8 2.7Net wage income – – 3.0 0.5 2.25 0.00 – – 2.3 0.0 2.2 21.8Net profits /assets income – – 5.5 8.0 3.5 6.3 5.00 10.50 – – 4.3 10.7

GovernmentDeficit, bill. DM – – 2144 2127 2132 2126 2150 2123 2146 2125 2138 2130 2134 2102Revenues – – 1.5 3.5 1.1 3.6 1.25 3.50 – – 1.2 2.0 0.9 1.9Expenditures – – 2.5 2.0 210.1 3.0 2.50 1.75 – – 2.0 2.0 1.5 0.4

Source: Own computations and official data. For sources of the forecasts see text.a First, preliminary official results for 1996 as of January 1997.


casts for Private consumption and for Fixed Orcutt correction of parameters is often used toinvestment were ‘responsible’ for a large part of eliminate serial correlation in the residuals (see,the GDP errors. To illustrate this point: if for example, Intriligator et al. (1996), pp.Private consumption is exogenized, the error of 140f.)). If all equations with the exceptions ofthe 1997 GDP forecast would have been re- Private consumption, Negotiated wage rate, andduced from 3.3 to 2.5%. The accuracy gains of Depreciation (where some signs of parametersthis are, however, not evenly distributed within had changed) were corrected, there would onlythe model. Some variables such as Net exports have been a marginal improvement in forecastor Change in stocks now show even larger accuracy. The GDP-error decreased by 0.1errors—underscoring the compound character of percentage points.macroeconomic forecasts. Finally, some remarks on the time and cost

The forecast evaluation is also used to check framework of the forecast. The work started onagain the (implicit) hypothesis of stability of October, 7 and the final forecast (Rheinisch-

¨ ¨model parameters. For that purpose the model is Westfalisches Institut fur Wirtschaftsforschungre-estimated over the period 1988-I to 1997-IV. (Hrsg.), 1996) was issued 5 weeks later. ItThe accuracy of this forecast (Table 6, column required, as usual, about two person weeksC) has greatly improved. In particular Private (net). It should, however, be noted that theconsumption and Fixed investment are now production can draw heavily on in-house exper-predicted nearly perfectly. Leaving aside that tise as to (1) collection and evaluation of thethis result is beyond expectations based on the data and their possible flaws, (2) the fixing ofex post-accuracy of the complete sample, it the exogenous variables and the additionalshould be noted that the parameters have not policy assumptions, and, (3) the general econ-changed much. The simulation results of the old omic environment and the future picture of thesample period for 1996/97 for Private con- economy. This information developed in thesumption and Fixed investment and as conse- semi-annual reports of the institute. About 20%quence for GDP had deteriorated considerably. of the time goes into the new estimation of theAgain, even in a medium sized model it is model and its structural analysis (if nothing hasdifficult to decide whether (yet small) changes to be fixed), 30% is spent on fixing the valuesin the economic reactions or just good luck have of the exogenous variables and the adds (despitebrought forward these improvements. While all the heavy reliance on in-house expertise!), 30%these examinations require a tremendous goes into finishing the forecast proper, and 20%amount of work without leading to ultimate into its ex post analysis. The complete directconclusions, the results stress again the need cost add up to about 20,000 DM, of whichthat the sample period be as up-to-date as about one half is due to non-model work.possible.

Could we have done better? Of course, withhindsight there are many ways in which theforecasts could have been improved. Here we 5. Summary and conclusionsrestrict this to the exploration of informationcontained in the residuals. Stimulated by criticism that the forecasts of

The exploitation of past error structures for macroeconometric models are not transparent,forecasts has a long tradition in mac- the paper has discussed the procedures used toroeconometric forecasting. The Cochrane- generate of a forecast with a medium sized


model for Germany for the year 1997. The the high flexibility of the macroeconometricpaper developed a five stage framework which model approach and the numerous analyticalproducers of forecasts can follow and which tools and (2) the limited resources available toshould make the whole process more transpar- forecast producers and consumers to apply andent to consumers of macroeconometric model to process the data, etc. Transparency still hasforecasts. The procedure is rather complex, its price.comprising a number of stages and recursive aswell as interdependent steps. However, by vari-

Acknowledgementsation of the sample period and a number ofsimulations, the role of hypotheses and assump-

¨Paper presented in the ‘Ausschuss fur em-tions for the final forecast can be exposed andpirische Wirtschaftsforschung und angewandteits accuracy assessed. Though the process ofOkonometrie’ at the Annual Conference of theforecasting is made transparent, it is, of course,Deutsche Statistische Gesellschaft, Hannover,not free from subjective elements. The influenceOctober 8, 1999. Previous versions of the paperof these subjective elements becomes more

¨were presented in seminars at the Universitatvisible and is laid open to discussion.¨Leipzig and at the Universitat Bielefeld. TheTo meet the ‘black box’ criticism, there are

author is grateful for the helpful comments andtwo things forecasters should do. First, thesuggestions of one editor and two referees ofpresentation of model forecasts should containthis Journal. This work has been supported bymuch more information than is usually the case.the Collaborative Research Centre ‘ReductionIt should report on the values of exogenousof Complexity in Multivariate Data Structures’variables and on important add factors, but also(SFB 475) of the German Research Foundationon relevant changes in the model structure and(DFG).their consequences. In the present case, no

stochastic simulations have been used to ap-proximate the expected forecast accuracy, but

Referencesthis is an important tool that can be used, giventhe decline in computing cost. Second, similarly

Adams, F. G. (1986). The business forecasting revolution.an ex post analysis should be made. At least theNation–industry–firm, Oxford University Press, New

role of the various assumptions should be York.12demonstrated. Though not a new idea, part of Arbeitsgemeinschaft deutscher wirtschaftswissenschaf-¨the presentation of a new forecast should also tlicher Forschungsinstitute e.V., Munchen (1996). Die

Lage der Weltwirtschaft und der deutschen Wirtschaftinclude an analysis of the preceding one as wellim Herbst 1996. Beurteilung der Wirtschaftslage durchas a discussion of the forecast stability. Finally,die folgenden Mitglieder, DIW, Ifo, IfW, IWH und RWI,it is obvious that there is a trade-off between (1)Berlin.

¨Barabas, G., Heilemann, U., & Munch, H. J. (1994).Forecasting accuracy and the length of the sampleperiod, RWI, Essen, RWI Papiere 34.

12 ´ `Of course, it would be preferable to leave the sensible Blinder, A. S. (1992). Comment: Deja vu all over again.undertaking of forecast analysis to third parties. However, In: Belongia, M. T., & Garfinkel, M. R. (Eds.), Theinstitutions such as the ESRC Macroeconomic Modelling business cycle: theories and evidence, Proceedings ofBureau in the UK (Wallis, 1985) are rather unique and the the 6th annual economic policy conference of thepublication lag of third parties’ reports usually is still long, Federal Reserve Bank of St. Louis, Kluwer, Boston, pp.though it may be reduced by using the Internet. 189–196.


Boutillier, M. (1983). The concept of reading as analysis SFB 475, Technical report, 16 /98, University Dor-of macroeconometric model. In: Fourth IFAC/IFORS/ tmund.IAAS conference on the modelling and control of IMF (1996). World economic outlook, World economic andnational economies and the 1983 SEDC conference on financial surveys, IMF, Washington, DC, October.economic dynamics and control, IFAC/IFORS/IAAS, Intriligator, M. D., Bodkin, R. G., & Hsiao, C. (1996).Washington, DC, pp. 115–121. Econometric models, techniques, and applications, 2nd

Brunner, K. (1973). Book review Economic models of ed, Prentice Hall, Upper Saddle River, NJ.cyclical behavior. Journal of Economic Literature XI, Klein, L. R., & Young, R. M. (1980). An introduction to926–933. econometric forecasting, Lexington Books, Lexington,

Bundesregierung (1997). Jahreswirtschaftsbericht 1997 MA.der Bundesregierung. (BT-Drucksache,13 /3601) vom McNees, S. K. (1990). Man vs. model? The role of31.1.1997, Hegner, Bonn. judgement in forecasting. New England Economic Re-

Chong, Y. Y., & Hendry, D. (1986). Econometric evalua- view July /August, 41–52.tion of linear macro-economic models. Review of Econ- Morgan, M. S. (1990). The history of econometric ideas,omic Studies 53, 671–690. Cambridge University Press, Cambridge.

Daub, M. (1987). Canadian economic forecasting. In a OECD (1996). Economic outlook, vol. 60, OECD, Paris,world where all’s unsure, McGill & Queens University December.Press, Montreal. Pagan, A. (1989). On the role of simulation in the

Dicks, G., & Burrell, A. (1994). Forecasting in practice. statistical evaluation of econometric models. Journal ofIn: Hall, S. (Ed.), Applied economic forecasting tech- Econometrics 40, 125–139.niques, Harvester Wheatsheaf, New York, pp. 110–143. ¨ ¨Rheinisch-Westfalisches Institut fur Wirtschaftsforschung

Evans, M. K., Haitovsky, Y., Treyz, G. I., & Su, V. (1972). ¨(1996). Bearbeiter: Dr Gyorgy Barabas & UllrichAn analysis of the forecasting properties of US econo- ¨Taureg Vierteljahrliche Prognose mit dem RWI-Konjun-metric models. In: Hickman, B. G. (Ed.), Econometric kturmodell 1996–3 bis 1997–4, Nr. 44, vol. 13, RWI,models of cyclical behaviour, Studies in income and Essen, November.wealth, 36, vol. II, Columbia University Press, New ¨ ¨Rheinisch-Westfalisches Institut fur WirtschaftsforschungYork, pp. 949–1139. (1997a). Die wirtschaftliche Lage im Ausland. RWI-

Fintzen, D., & Stekler, H. O. (1999). Why did forecasters Konjunkturbericht 48, 101–121.fail to predict the 1990 recession? International Journal ¨ ¨Rheinisch-Westfalisches Institut fur Wirtschaftsforschungof Forecasting 15, 309–323. ¨(1997b). Bearbeiter: Dr Gyorgy Barabas & Ullrich

¨Taureg Vierteljahrliche Prognose mit dem RWI-Konjun-Gilli, M. (1984). CAUSOR: A program for the analysis ofkturmodell 1997–1 bis 1998–4, Nr. 45, vol. 16, RWI,recursive and interdependent causal structures, Cahiers

´ ´ ´ Essen, June.du departement d’econometrie, 84.03, Universite de` ´Geneve, Faculte des sciences economiques et sociales, ¨ ¨Rheinisch-Westfalisches Institut fur Wirtschaftsforschung`Geneve. ¨(1998). Bearbeiter: Dr Gyorgy Barabas & Ullrich

¨Taureg Vierteljahrliche Prognose mit dem RWI-Konjun-Heilemann, U. (1998a). Erfahrungen mit dem RWI-Kon-kturmodell 1998–1 bis 1999–4, Nr. 47, vol. 2, RWI,junkuturmodell 1974 bis 1994. In: Heilemann, U., &Essen, July.Wolters, J. (Eds.), Gesamtwirtschaftliche Modelle in der

¨Bundesrepublik Deutschland: Erfahrungen und Perspek- Sachverstandigenrat zur Begutachtung der gesam-¨tiven, Schriftenreihe des Rheinisch-Westfalischen In- twirtschaftlichen Entwicklung (1996). Jahresgutachten

¨stituts fur Wirtschaftsforschung, N.F. Heft 61, Duncker 1996/97, BT-Drucksache 13/6200, Hegner, Bonn.& Humblot, Berlin, pp. 61–93. Sapir, M. (1949). Review of economic forecasts for the

transition period, Studies in income and wealth, vol. 11,Heilemann, U. (1998b). Paradigm lost-Zu den ProjektionenNBER, New York.des Jahreswirtschaftsberichts der Bunderegierung. In:

Heilemann, U., Kath, D., & Kloten, H. (Eds.), En- Wallis, K. F. (Ed.), (1985). Models of the UK economy. Atgrenzung als Erkenntnis- und Gestaltungsaufgabe- second review by the ESRC Macroeconomic Modelling

¨Festschrift fur Reimut Jochimsen zum 65. Geburtstag, Bureau, Oxford University Press, Oxford.Dunker & Humblot, Berlin, pp. 79–100. Zarnowitz, V. (1992). Business cycles: theory, history,

¨Heilemann, U., & Munch, H. J. (1998). Cyclical classifica- indicators, and forecasting, NBER studies in businesstion of the RWI-short term forecast 1997 –3 to 1999 –4, cycles, vol. 27, University of Chicago Press, Chicago.


Biography: Ullrich HEILEMANN graduated in 1973 from the RWI. In 1995 he declined a chair offer from thethe University of Mannheim, Germany, and joined the University of Leipzig; in 1997 he accepted such an offer

¨ ¨Rheinisch-Westfalisches Institut fur Wirtschaftsforschung from the University of Duisburg. He is on part-time leave(RWI) in Essen in 1974. In 1979 he received his doctorate there to serve as the vice president of the RWI. The author

¨from the University of Munster (on forecasting qualities of was a Visiting Scholar /Professor at Harvard, MIT, theWest German macroeconometric models) and was inaugu- Brookings Institution, and various other US and Canadianrated there in 1988 as a lecturer (on West German wage research institutions.dynamics). In 1987 he became a member of the board of

increasing the transparency of macroeconometric forecasts: a report from the trenches

Documents