unobserved component model with observed cycle _ dudek s. et alli

8/13/2019 Unobserved Component Model With Observed Cycle _ Dudek S. Et Alli

1/17

30th CIRET Conference, New York, October 2010

Unobserved component model with observed cycle

Use of BTS data for short-term forecasting of manufacturing

production

Sawomir Dudek

Dawid Pachucki

The Research Institute for Economic Development (RIED)

The Warsaw School of Economics (WSE)

Abstract

Business tendency survey data (BTS) is often used as an indicator of the cyclicalfluctuations in the real economy. The outcome of many empirical studies is that the survey datais usually leading or coincident with the quantitative one. In our paper we are using this propertyof the BTS to make short-term forecasts of manufacturing production. For that purpose, theunobserved component model (UCM), also known as the structural time series model was used.Within this model the time series of manufacturing production is decomposed into unobservedcomponents: the trend, the cycle. It was assumed that the trend is approximated with anunivariate time series model. As to the "unobserved cyclical component" it was assumed that itis common for reference quantitative variable and qualitative variable. In that sense the cyclicalfluctuation can be approximated by the fluctuations of BTS indicators. Such specification can becalled Unobserved component model with observed cycle" (UCM-OC). Such specified systemwas estimated with the application of Kalman filter technique. Then the model was used formaking recursive one-period ahead forecasts to check its out-of-sample data fit. In addition the

forecasting properties were evaluated against alternative models, ie, "pure" UCM and ARIMAmodel. The analysis was performed for Poland and selected European Union countries.

Key Words: industrial production, business tendency survey, short-term forecasting, unobserved

component model

JEL Classification: E23, E27, E32, C22, C53


2/17


3/17


3

tendency survey data (BTS) are often used as an indicators of the cyclical fluctuations in the realeconomy the specification is assuming that "unobserved cyclical component" can be extracted basingon the behaviour of qualitative indicator. In that sense unobserved cyclical fluctuation are in factobserved in fluctuation of qualitative indicator. So we decided to call our model: "Unobserved

component model with observed cycle" (UCM-OC).

Next to the work of Planas, Roeger and Rossi (2009) we decided for following state spacerepresentation of our model:

cttctct

ttt

tt

BTStBTSt

ttt

accc

a

t

acBTS

cty

++=

++=

=

++=

+=

2211

1

1

**

*)1(*

*

(1)

The first two equations are so called signal or measurement equations which describes

relationship between observed: country X manufacturing production (ty ) and country X selected

BTS indicator (tBTS ), and unobserved trend ( tt ) and cycle ( tc ). The next tree equations of the

system (1) named in literature the state or transition equations, describes the behaviour of unobservedcomponents. In terms of the cycle which in the model is kind of common component for themanufacturing output and the BTS indicator, an AR(2) process is assumed. For the trend (third andfourth equations of the above system) the dumped trend process is being considered. We testeddifferent trend specification in the system (1) (Pedregal 2002), however the dumped trend proposed

above seems to fit the best all the analyzed time series. The smoothing behaviour of the trend for being constrained to take values between 0 and 1 (if there is no additional shock to the system) is aquite good approximation of the behaviour of economic time series (Grander, McKenzie 2009). The

BTSa , ta , cta are white noise processes. As a qualitative variable we used separately three

indicators: the industrial confidence indicator (ICI) with 1 month lead to the common cycle, the balanceon question regarding production expectations (IPE) with 1 month lead, and the balance on questionregarding production trend observed in recent months (IPT) as coincident. Hence we estimated threemodels respectively: UCM-ICI, UCM-IPE and UCM-IPT.

Bearing in mind the above mentioned advantages of the unobserved component models overARIMA models, as an alternative we also tested univariate version of the system (1), where the onlyobserved signal is for manufacturing production, i.e. specification without second equation. This model

will be indicated as UCM.

For out-of-sample analysis purposes from the whole time sample last P=39 observations wereexcluded to compare forecasting properties. The exclude sample covers period 2007:M1-2010:M3 tocheck to assess the models reaction to the last global financial crisis. Thus the starting estimationsample include T=180 observations, it covers period 1992:M1-2006:M12 (for Poland sample startsfrom 1992:M3). Using defined above models (ARIMA(1,1,0), UCM, UCM-ICI, UCN-IPE, UCM-IPT), 39point (one month ahead) forecasts were calculated recursively with re-estimation of that models. Ateach recursion the estimation sample was increased by one month forward and forecasted point(month) also. For all models there were calculated forecast errors for out-of sample and averagemeasures like root mean squared error (RMSE) and mean absolute error (MAE).


4/17


4

( )==

==

P

t

f

tt

P

t

t yyP

eP

RMSE1

2

1

2 11 (2)

==

==

P

t

f

tt

P

t

t yyP

eP

MAE11

11 (3)

In order to check whether the forecasts from UCM-OC models are superior to the forecasts fromreference benchamrk, there were calculated relative RMSEs and MAEs, i.e. ratios of the root meansquared errors and mean absolute errors of the UCM-OC models to the reference ARIMA model. Therelative RMSE is called also as a Theils ratio (called in some papers as a U statistic). If Theils Ustatistic or relative MAE is smaller than one, then the forecasts based on the BTS indicators aresuperior to the forecasts of the benchmark model.

To check weather forecast superiority is statistically significant we focus on the test of equalpredictive accuracy of Diebold and Mariano (1995), which is widely used for comparing forecasts oftwo competing models. We use Diebold-Mariano test with squared error loss function and withabsolute error loss function. The loss differentials for out-of-sample are calculated as:

( ) ( )22 ARIMAtOCUCM

t

sqr

t eed =

(4)

ARIMA

t

OCUCM

t

abs

t eed = (5)

whereARIMA

t

OCUCM

t ee ,

are forecast errors from competing models.

Two forecasts have equal accuracy if and only if the loss differential (4 or 5) has zero expectation

for all t. Thus the null hypothesis of equal predictive accuracy is ( ) 0:0

=tdEH versus the alternative

hypothesis ( ) =tdEH :1 different from zero. When module of Diebold-Mariano test statistics (usedfor 4 or 5) is higher than critical value with given significance level than null hypothesis of equalpredictive accuracy have to be rejected. When Diebold-Mariano test statistics is negative andempirical p-value is less then assumed significance level (e.g. 5% or 10%) than forecasts receivedform UCM-OC models are significantly superior to the forecasts from ARIMA model.

It should be underlined that all the forecast errors used to calculated above statistics for eachperiod in out-of-sample have the same weight, henceforth we call them unweighted. But in manypractical situations precise forecasts for some periods are more important than for others. Forexample, accurate forecasting of the beginning of a recession is of special importance. In case ofmanufacturing production, which is strongly affected by cyclical fluctuations it is especially important.Very often the start of a recession correspond with a large decrease of manufacturing production.Hence, when selecting among competing forecasting models, it makes sense to focus on these crucialobservations and to put more weight on the errors in this periods.

For this purpose, we use approach proposed by van Dijk et al. (2003). To compare forecastaccuracy they proposed modified Diebold-Mariano statistic by using a weighted average lossdifferential. As an examples of sensible weighting function they proposed to use empirical cumulativedensity function of forecasted variable. Basing on CDF we can construct left tail (LT) weightingfunction and right tail (RT) weighting function. First one is putting more weight on periods when high


5/17


5

rate of growth of reference variable is observed, second one opposite, when rate of change is largelynegative. Formally, the weight functions for the left tail and right tail are given by:

( )

( )t

RT

t

t

LT

t

ywRT

ywLT

=

=

:

1: (6)

where ( )ty denotes the empirical cumulative density function of forecasted variable. In ourpaper we use distribution of log-change of reference variable because forecasting models areconstructed on levels. Figure 1 depicts the empirical cumulative density functions of reference variablefor analyzed countries which are used to construct weights.

Using above defined weights (6), weighted forecast errors are calculated:

tt

w

t ewe = (7)

This weighted errors are used to calculate relative RMSEs, MAEs and loss differentials (4 and 5)for Diebold-Mariano test. In all experiments, the competing forecasts are evaluated using unweightedand weighted (left tail and right tail weights) versions of the Diebold-Mariano test statistic and weightedand unweighted relative RMSEs and MAEs.


6/17


6

Figure 1 Empirical CDFs for dlog of reference variable.

0.0

0.2

0.4

0.6

0.8

1.0

-.08 -.06 -.04 -.02 .00 .02 .04

Probability

DLOG_DE_IP_SA

0.0

0.2

0.4

0.6

0.8

1.0

-.04 -.03 -.02 -.01 .00 .01 .02

Probability

DLOG_FR_IP_SA

0.0

0.2

0.4

0.6

0.8

1.0

-.04 -.03 -.02 -.01 .00 .01 .02 .03

Prob

bility

DLOG_IT_IP_SA

0.0

0.2

0.4

0.6

0.8

1.0

-.05 -.04 -.03 -.02 -.01 .00 .01 .02 .03 .04 .05

Probability

DLOG_PL_IP_SA

0.0

0.2

0.4

0.6

0.8

1.0

-.025 -.015 -.005 .000 .005 .010 .015 .020

Probability

DLOG_UK_IP_SA

Source: Own calculation; DLOG_ first difference of logarithm of reference variable.


7/17


7

3. Predictive power of UCM-OC models with BTS indicators (out-of-

sample analysis).

The UCM models, both univariate an multivariate versions, where identified for all the countries.

The only exception was Poland, where the univariate UCM identified the cycle with really strange

behaviour. The model had some problems with differentiate the trend and a the component of

business cycle frequencies. The only possibility to deal with this issue was to put some additional

constrains in the UCM, which made the model for Poland different form the others. As we decided to

not differentiate the systems for particular countries, in the further analysis the univariate unobserved

component model for Poland is skipped. This is a good example that this kind of models are quite

sensitive to the assumed parameters and formulation of particular components. On the other hand, the

multivariate specification allowed for solving the problem.

As was mentioned in the methodological part, to assess whether the UCM-OC model outperform

the benchmark we are looking for lower than one values for relatives MAE and RMSE or negativevalues for Diebold-Mariano statistics (DM-t-sqr and DM-t-abs). All the results are presented in Tables

1a-1c.

Comparison of unweighted forecast errors do not provide clear answer (see also Figure 2) to the

key question posed in the paper. On average, the forecast errors seem to be lower in the UCM-OC

type of models, however the Diebold-Mariano tests do allow for the statement that the difference is

statistically significant. On the other hand, in case of UK, the ARIMA(1,1,0) model provides

significantly better forecasts except the UCM-OC model where the IPT index was used as indicator for

the cycle. In this case there was no significant difference in the quality of the forecast between UCM-

IPT and ARIMA model. Another finding is that in general the multivariate version of UCM provides

lower forecasts errors than the univariate one. However for particular countries the confidenceindicator which allowed for reduction of the error was different. In case of Germany it was IPT, for

France - IPE, Italy IPT, Poland ICI, and UK IPT.

After giving weight for the periods of high growths (Table 1b right tail weighting function) or

high drops (Table 1c left tail weighting function) the conclusions changed. In case of high positive

growth rate (expansion phase) the simple ARIMA model seem to outperform the UCM approach. It is

possible to identify at least one UCM model for Germany, Poland and UK where at 10% significance

level the ARIMA forecasts where better then the UCM. In case of France it is 12% significant level. In

case of Italy or other not identified above models, the differences was statistically not significant, what

means that forecasts accuracy is the same for both types of the models.

From the Figure 2 it is visible that on general UCM models with qualitative indicators in terms offorecast accuracy performed better during the period of intensification of global finance crisis (end of

2008). Better performance of the UCM models when left tail weighting function is used is also proved

by analysis of relative RMSE sans MAEs (Table 1c). For all countries (except UK) the relative average

error statistics are lower than one. In the case of Poland and Italy UCM-OC models were able to

provide significantly better (according to DM statistics) forecast than the benchmark. In other cases,

left tail weighting do not significantly improved the UCM performance compared to the conclusions for

unweigted errors, however relative RMSEs and MAEs are lower than one.


8/17

Table 1a Evaluation of the out-of-sample forecasting power of BTS indicators (unweight

Country Model MAE Rank RMSE Rank DM-t sqr DM-t prob

DE UCM 1,061 3 1,008 4 0,13 0

UCM-ICI 1,118 4 0,904 3 -0,53 0

UCM-IPE 0,993 2 0,848 2 -0,93 0UCM-IPT 0,975 1 0,835 1 -1,01 0

FR UCM 0,974 3 0,982 4 -0,56 0

UCM-ICI 0,881 2 0,804 2 -1,25 0

UCM-IPE 0,858 1 0,794 1 -1,18 0

UCM-IPT 0,983 4 0,855 3 -0,83 0

IT UCM 1,019 4 0,990 4 -0,79 0

UCM-ICI 0,946 2 0,917 2 -1,59 0

UCM-IPE 0,988 3 0,945 3 -1,03 0

UCM-IPT 0,887 1 0,840 1 -1,53 0

PL UCM - - - - - - UCM-ICI 0,990 1 0,953 1 -1,30 0

UCM-IPE 1,049 2 1,028 3 0,56 0

UCM-IPT 1,053 3 1,022 2 0,95 0

UK UCM 1,045 2 1,046 2 1,73 0

UCM-ICI 1,277 3 1,196 3 1,98 0

UCM-IPE 1,426 4 1,410 4 2,04 0

UCM-IPT 0,860 1 0,842 1 -0,76 0

a) RMSE, MAE relative root means square error and mean absolute error of UCM-OC models over benchmthan 1 indicate superiority of UCM-OC model, Rank ranking of models based on relative RMSE, MAE.

b) DM-t sqr, DM-t prob sqr Diebold-Mariano t-statistics and empirical p-value for test with squared error los

abs Diebold-Mariano t-statistics and empirical p-value for test with absolute error loss function, negativeaverage forecast errors from UCM-OC models are lees than from benchmark model, p-value empirical s10% than accuracy of one model is significantly better than rival one.


9/17

Figure 2 Forecast errors for particular models and countries.

-12

-8

-4

0

4

8

I II III IV I II III IV I II III IV I

2007 2008 2009 2010

ARIMA UCM UCM-ICI

UCM-IPE UCM-IPT

DE

-6

-5

-4

-3

-2

-1

0

1

2

3


2007 2008 2009 2010

ARIMA UCM UCM-ICI

UCM-IPE UCM-IPT

FR

-6

-4

-2

0

2

4

6


2007 2008 2009 2010

ARIMA UCM UCM-ICI

UCM-IPE UCM-IPT

IT

-10.0

-7.5

-5.0

-2.5

0.0

2.5

5.0

7.5

10.0


2007 2008 2009 2010

ARIMA UCM-ICI

UCM-IPE UCM-IPT

PL

-5

-4

-3

-2

-1

0

1

2

3


2007 2008 2009 2010

ARIMA UCM UCM-ICI

UCM-IPE UCM-IPT

UK


10/17

Table 1b Evaluation of the out-of-sample forecasting power of BTS indicators (weighted


DE UCM 1,206 3 1,266 4 1,34 0

UCM-ICI 1,399 4 1,207 3 1,49 0

UCM-IPE 1,128 2 0,961 1 -0,25 0UCM-IPT 1,117 1 0,979 2 -0,14 0

FR UCM 0,979 2 0,985 2 -0,35 0

UCM-ICI 0,997 3 0,997 3 -0,03 0

UCM-IPE 0,960 1 0,952 1 -0,40 0

UCM-IPT 1,254 4 1,291 4 1,61 0

IT UCM 1,054 4 1,025 1 1,22 0

UCM-ICI 1,035 1 1,068 1 1,12 0

UCM-IPE 1,076 3 1,086 3 1,27 0

UCM-IPT 1,065 2 1,113 4 1,23 0

PL UCM - - - - - - UCM-ICI 1,111 1 1,096 1 1,53 0

UCM-IPE 1,301 3 1,327 3 2,03 0

UCM-IPT 1,138 2 1,131 2 1,66 0

UK UCM 1,027 2 1,016 2 1,22 0

UCM-ICI 1,358 3 1,147 3 0,77 0

UCM-IPE 1,433 4 1,218 4 1,39 0

UCM-IPT 0,977 1 0,818 1 -0,61 0





11/17


11

Table 1c Evaluation of the out-of-sample forecasting power of BTS indicators (weighted


DE UCM 0,972 4 0,925 4 -1,23 0

UCM-ICI 0,947 3 0,741 1 -1,11 0

UCM-IPE 0,910 2 0,774 3 -1,08 0UCM-IPT 0,889 1 0,754 2 -1,14 0

FR UCM 0,972 4 0,979 4 -0,92 0

UCM-ICI 0,815 2 0,735 3 -1,40 0

UCM-IPE 0,800 1 0,727 2 -1,33 0

UCM-IPT 0,828 3 0,706 1 -1,31 0

IT UCM 0,996 4 0,973 4 -1,17 0

UCM-ICI 0,891 2 0,856 2 -1,85 0

UCM-IPE 0,934 3 0,888 3 -1,76 0

UCM-IPT 0,777 1 0,720 1 -1,90 0

PL UCM - - - - - - UCM-ICI 0,901 2 0,881 2 -1,84 0

UCM-IPE 0,866 1 0,854 1 -1,76 0

UCM-IPT 0,991 3 0,973 3 -1,12 0

UK UCM 1,055 2 1,051 2 1,89 0

UCM-ICI 1,229 3 1,161 3 1,33 0

UCM-IPE 1,422 4 1,423 4 1,48 0

UCM-IPT 0,791 1 0,799 1 -1,23 0





12/17


13/17


13

References

Diebold, F.X. and R.S. Mariano (1995), Comparing Predictive Accuracy, Journal of Business & Economic

Statistics 13, 253-263.Dijk, van D. and P. H. Franses (2003), Selecting a Nonlinear Time Series Model using Weighted Tests of

Equal Forecast Accuracy, Oxford Bulletin of Economics and Statistics, 65, 727744.

European Commission DG ECFIN (1997), The Joint Harmonised EU Programme of Business andConsumer Surveys, European Economy Report and Studies, No 6, Brussels.

European Commission DG-ECFIN (2007), The Joint Harmonised EU Programme of Business andConsumer Surveys - User guide.

Harvey, A.C. (1985), Trends and Cycles in Macroeconomic Time Series, Journal of Business andEconomic Statistics, Vol. 3(3), 216-227.

Harvey, A.C. (1989), Forecasting, Structural Time Series Models and the Kalman Filter, CambridgeUniversity Press, Cambridge, New York and Melbourne.

Kuttner K. (1994),Estimating potential output as a latent variable, Journal of Business and Economic

Statistics,12,3,361-368.Planas C., Roeger W., Rossi A. (2009), Improving real-time TFP cycle estimates by using capacity

utilization European Commission, Joint Research Centre.


14/17


14

Appendix 1: Definitions and data sources

All variables used in the paper are encoded in a uniform manner. The syntax of the variable

code is as follows:

[country code]_[variable code]_SA

where: _SA means seasonal adjustment.

Country codes according to EUROSTAT:

Germany DE

France FR

Italy IT

Poland PL

United Kingdom - UK

Index of manufacturing production (IP)Description Index of manufacturing production (NACE Rev.2), monthly frequency, 1992.01-

2010.03, single-base index 2005=100, seasonally adjusted.Source EUROSTAT: on-line database:

http://epp.eurostat.ec.europa.eu/portal/page/portal/statistics/search_databaseIndustry production index - monthly data - (2005=100) (NACE Rev.2) (sts_inpr_m)

OECD: on-line database only for Poland for years 1992-1994http://stats.oecd.orgDataset: Production and Sales (MEI)/Production in total manufacturing sa,2005=100

Business tendency survey industry (ICI, IPE, IPT)

Description Business tendency survey industry, monthly frequency, 1992.01-2010.03,seasonally adjusted.ICI industrial confidence indicator is the arithmetic average of the balances (inpercentage points) of the answers to the questions on production expectations,order books and stocks of finished products (the last with inverted sign). according

to EU definition, balances for questions from harmonized questionnaire (see ECDG-ECFIN 2007).IPT Production trend observed in recent months - balanceIPE Production expectations for the months ahead balance.

Source EUROSTAT: on-line database:http://epp.eurostat.ec.europa.eu/portal/page/portal/statistics/search_databaseBusiness surveys (Source: DG ECFIN)/ Business surveys - NACE Rev. 1.1/Industry- monthly data (bsin_m)The Research Institute for Economic Development (RIED), The Warsaw School ofEconomics (WSE): Business Activity in Manufacturing Industry periodic survey.


15/17


15

Appendix 2: Graphs

Figure 3a Unobserved common cycle component and BTS indicators.

-50

-40

-30

-20

-10

0

10

20

-20

-15

-10

-5

0

5

10

15

1992 1994 1996 1998 2000 2002 2004 2006 2008 2010

DE_ICI_SA DE_IPICI_CYCLE

DE

-50

-40

-30

-20

-10

0

10

20

30

-25

-20

-15

-10

-5

0

5

10

15

1992 1994 1996 1998 2000 2002 2004 2006 2008 2010

DE_IPE_SA DE_IPIPE_CYCLE

DE

-50

-40

-30

-20

-10

0

10

20

30

-20

-16

-12

-8

-4

0

4

8

12

1992 1994 1996 1998 2000 2002 2004 2006 2008 2010

DE_IPT_SA DE_IPIPT_CYCLE

DE

-50

-40

-30

-20

-10

0

10

20

-16

-12

-8

-4

0

4

8

12

1992 1994 1996 1998 2000 2002 2004 2006 2008 2010

FR_ICI_SA FR_IPICI_CYCLE

FR

-40

-30

-20

-10

0

10

20

30

-16

-12

-8

-4

0

4

8

12

1992 1994 1996 1998 2000 2002 2004 2006 2008 2010

FR_IPE_SA FR_IPIPE_CYCLE

FR

-50

-40

-30

-20

-10

0

10

20

30

40

-25

-20

-15

-10

-5

0

5

10

15

20

1992 1994 1996 1998 2000 2002 2004 2006 2008 2010

FR_IPT_SA FR_IPIPT_CYCLE

FR

Source: Own calculations.


16/17


16

Figure 3b Unobserved common cycle component and BTS indicators.

-40

-30

-20

-10

0

10

20

-12

-8

-4

0

4

8

12

1992 1994 1996 1998 2000 2002 2004 2006 2008 2010

IT_ICI_SA IT_IPICI_CYCLE

IT

-30

-20

-10

0

10

20

30

40

-16

-12

-8

-4

0

4

8

12

1992 1994 1996 1998 2000 2002 2004 2006 2008 2010

IT_IPE_SA IT_IPIPE_CYCLE

IT

-70

-60

-50

-40

-30

-20

-10

0

10

20

-25

-20

-15

-10

-5

0

5

10

15

20

1992 1994 1996 1998 2000 2002 2004 2006 2008 2010

IT_IPT_SA IT_IPIPT_CYCLE

IT

-40

-30

-20

-10

0

10

20

30

40

-10

-8

-6

-4

-2

0

2

4

6

1992 1994 1996 1998 2000 2002 2004 2006 2008 2010

PL_ICI_SA PL_IPICI_CYCLE

PL

-40

-30

-20

-10

0

10

20

30

40

-12.5

-10.0

-7.5

-5.0

-2.5

0.0

2.5

5.0

7.5

1992 1994 1996 1998 2000 2002 2004 2006 2008 2010

PL_IPE_SA PL_IPIPE_CYCLE

PL

-40

-30

-20

-10

0

10

20

30

-12

-8

-4

0

4

8

12

16

1992 1994 1996 1998 2000 2002 2004 2006 2008 2010

PL_IPT_SA PL_IPIPT_CYCLE

PL



17/17


17

Figure 3c Unobserved common cycle component and BTS indicators.

-60

-50

-40

-30

-20

-10

0

10

20

-20

-16

-12

-8

-4

0

4

8

12

1992 1994 1996 1998 2000 2002 2004 2006 2008 2010

UK_ICI_SA UK_IPICI_CYCLE

UK

-80

-60

-40

-20

0

20

40

60

-25

-20

-15

-10

-5

0

5

10

1992 1994 1996 1998 2000 2002 2004 2006 2008 2010

UK_IPE_SA UK_IPIPE_CYCLE

UK

-60

-40

-20

0

20

40

60

-20

-15

-10

-5

0

5

10

1992 1994 1996 1998 2000 2002 2004 2006 2008 2010

UK_IPT_SA UK_IPIPT_CYCLE

UK


unobserved component model with observed cycle _ dudek s. et alli

Documents