ELSE-V1ER International Journal of Forecasting 13 (1997) 463-475

The performance of alternative forecasting methods for SETAR models

Michael E Clements*, Jeremy Srnith Departing'st ~ Economics. University of Warwick. Coventry. CV4 7AL. UK

Abs trac t

Wc c~mpare a number of methods 0tat have been proposed in the literature l~r obtaining /t-step ahead minimunt mean square error forecasts lbr self-exciting thrcstmld aumrcgrcssivc (SETAR) models. These forecasts are compared to those fxom an AR model. The ct~mparixon of f~recasting methods is made using Monte Carh~ simulation. The Monte Carlo mctlmd of calculating SI-TAR forecasts ix generally at leant as g~od as that ~i' the other ttlell|~tls we consider. An exceplitm is when the tlisturbatnces in the SI!TAR model come from a highly asyntntctric tlistributi~m, when a Bo~tstrap method ix I~ be preferred.

An empirical applic:ttitm c:tlculalcs multi-period forecasts It'(mt a SI"I'AR lnodel of [,IS gn)sn national product usi0tg a nund~cr of the lbrccaslmg mcttu~ds. Wc lind that whether there arc imprtwemcnls in ftwccast perfmmance relative to a linear AR tn~,.lcl tlepculds ~m the histt~tical epoch we select, and whether forecasts arc evaluated ctmditional on the regime the lm~ccss ~,ts mal the time the l~rccas! was nt;tdc. (~) 1997 I!lscvicr Science B.V.

hcvword~. l'hrcxhold motlcl; I:orcca:,ting; Simulations

I. In t roduc t ion

The aim of this paper is to compare the forecast

performance of a number of methods that can be

used to obtain h-step ahead nfinimum iIican sqtuarc

error forecasts for self-exciting threshold autoregres- sive (SETAR) models. We assess the relative merits

of the ahemative methods by Monte Carlo (MC) simulation, and vary the desi~,n of the MC to explore

the impact of a range of factors (such ;Is non-zero

intercepts in the SETAR models, non-Gaussian

innovations, etc.). The ntotivation for the paper is that the multi-step

(t~rrc,p~mtlmg author. Tel.: 1012031 5229¢~n; fax: t{11203) . _303_: |--mail: MP.CIcmcnts@~warwick.ac.uk

forecast perlbrnmnce of SETAR models is often not

reported in many of the papers that comprise tile

large and growing body of litcrature on applying such models to economics data. This is pcrhaps

surprising since one of the main uses of time-series models is for prediction. Moreover, neither in-sample

lit, nor the rejection of the null of linearity in a formal test for non-lincarity, guarantee that SETAR

models (or non-linear models more generally) will forecast more accurately than linear AR models (see, tbr example, Diebold and Nason (1990), and Dc

Gooijcr and Kumar (1992), who argue that the evidence for a superior forecast performance of non- linear models is patchy). One factor that has proba- bly retarded the widespread reporting of forecasts from SETAR models is that it is not possible to

Ol(¢)-21)71)/t17/$17.(X) ,~" 1~)97 l-I,,cvicr Science B.V. All right~, reserved.

I ' l l S~}1(¢)-2070( 97 )00(}17-4

464 M.P. Clements. J. Smith I International Journal of Forecasting 13 (19o7) 4~3-.175

obtain closed-form analytic expressions for multi- step ahead forecasts and exact numerical solutions require sequences of numerical integrations.

Thus in this paper we evaluate a number of alternative methods that are available that do not involve numerical integration. Their accuracy, ease of use, and robustness to certain non-standard as- pects of the forecasting problem, or forms of model- misspecification, are addressed. The five methods we consider are a MC simulation method, a Bootstrap method (BS), the normal forecast errors method (NFEM) due to AI-Qassam and Lane (1989) and adapted to SETAR models by De Gooijer and De Bruin (1997), a multi-step or Dynamic Estimation method (DE). and a 'naive' method (Skeleton. SK). Linear AR models are also used as a benchmark for comparison.

De Gooijer and Dc Bruin (1997) draw together some of the results in the literature on forecasting SETAR processes, and motivate the NFEM from the Chapman-Kolmogorov relations. They also report a comparison of multi-step forecasts and forecast variances produced by the NFFM, the MC mcth()tl. and an exact method involving numerical integration. for a small number of SI:TAR models. We extend their study by considering three additional mcth~tls and a broader range of Si~TAR models, which include models with non-Gaussian disturbances. We do not consider numerical integration teclmiques.

The comparison between the forecasting methods could be made at a number of different knowledge levels concerning the SETAR model. For example, we could assume that: (i) all the parameters of the SETAR model are known; (ii) the autoregressive parameters are unknown, but the number of regimes, thresholds, delay parameters, and autoregressive lag orders are known; (iii) all the parameters are un- known except the lag orders; (iv) all the parameters including the lag orders are unknown. De Gooijer and De Bruin (1997) operate at level ( i ) - -wc make the comparison at level (iii), except we assume that the delay lag is also known. We operate at this level following a suggestion of one of the referees. It allows a comparison of feasible forecasting methods. but also has a number of other advantages. It enables the SETAR and AR models to be compared. A: level (i) the parameters of the AR model would need to be replaced by their pseudo-true values under the

SETAR. which may be difficult to calculate. More- over, it allows the use of the BS method, which is not applicable at level (i). Lai and Zhu (1991) consider multi-step prediction for general non-linear AR models when the model parameters are un- known, and apply a number of the forecasting methods we consider to the exponential autoregres- sive model.

The paper is organised as follows. In Section 2 we introduce the basic SETAR model and the alternative forecasting methods. Section 3 discusses the MC comparison of forecasting methods. Section 4 applies some of the forecasting methods to SETAR models of US gross national product (GNP). As well as illustrating the performance of the methods, this empirical example is of considerable interest in its own right, following the recent papers of Tiao anti Tsay (1994) and Potter (1995), which suggest that US GNP can bc usefully modelled its a SETAR process. I iowcvcr, t lanscn (1996) linds the evidence for a SETAR stnictt,rc h)r US GNP weak. Wc approach the question lrom a f()rccasting perspective. and our lindings stress the importance of evaluating forecasts conditional on the state of the economy.

2. The SETAR and AR models

The SETAR model was introduced by Tong (1978), Tong iuld Linl (1980), lind discussed more fully in Tong (1995). ht Section 2.1 we brietly outline the model, and in Section 2.2 we consider methods of obtaining forecasts. Section 2.3 considers the linear AR model.

9. I. A brief description of the SETAR moth,I

i t is assumed that a variable, y,, is a linear autoregression within a regime, but may move between regimes depending on the value taken by a lag of.v,, say y,_,l, where d is known as the length of the delay. Formally. y,_,l is continuous on the real line. :)~.. so that partitioning the real line delines the number of distinct regimes, say q. The process is in the r h regime when G,-~ -<.v, ~,1 < G,. and in that ease y, fol lows an AR process of order Ph" For the two-regime case (q = 2) the model is denoted by

3d.P. Clements. J. Smith 1 Internanomd Journal +'t Foreca.stiml / 3 ( IqO7~ 4&T-475 -1.65

S E T A R t 2 ; P t . p . ) . indicat ing that the process is an AR(p~) in regime 1 and an A R ( p : ) in regime 2:

Pl

3'1 = ¢q¢11 ~'- E Oll,lS't- ! + £l.t" Yt-d <- r t = l

/+2

Y, = a : n + E ct:.jy,_~ + #2.,. Y, -a > r ( I a)

where t = I ..... n and G.,~G(0.cr- ' , ) . h = 1.2. The dis- tr ibution of the dis turbances . G(- ). may be Gauss ian . but this is not necessar i ly the case. and in Sect ion 3 we consider a ntin~ber of possibi l i t ies . The model can be written more compac t ly using the indicator func- tion as:

P

Y, = ~ . + N" cr,.x',.-~ + ":1.,

4- l, ,~lr) ~u + . U , ~ v , + c:., - ,v,., ( I b )

w h e r e . ,lrl = > , ) I,', so m: . co , .par-

ing I':qs. ( l a ) a m l ( Ih) : +q. =~t,. i = 1 ..... l q . and

% , = ~r, ~- fl,. i = I ..... p: . W h e n / , , ~p. , . some parame- ters will bc zero.

The necessary ;.uRI sul ' l icicnt ctmtli t ions for statitm- arity of SI -TAR Illotlclx only exist in the l i terature for ;t tm!nhcr of special cases (scc. for example . I)c ( ;oo i jc r and 1)c P, r u i n ( 1 9 9 7 ) ) .

2.2. Meth ,Ms .fi,r .l},reca.~ting SI '? 'AR nu,,h' ls

In this section, we hriclly trotline live forecast ing methods that will he invest igated in Sect ion 3. All the methods arc d iscussed in Granger and Tc6.isvirta ~ 1993) in the context of forecast ing from a general non-l inear model . For exposi t ional s impl ic i ty , the discussion assumes a S E T A R ( 2 ; I . I ) model and that the delay parameter ix one ( d = I ).

2.2. I . 77w N H ' M

The first forecast ing method we cons ider ix the NFEM suggestctl by AI -Qassam and Lane (1989) for a l i rs t-ordcr exponent ia l au toregress ive model and extended by De Goo i j c r and Dc Bruin (1997) to SF.TAR (q : l ' , .P , . ) with general d. This method essent ial ly takes a weighted average of the forecasts

from regime 1..~'l . . . . ~. = a i m + t r l . l f ~ l ~ t +1 and re-

g ime 2. y : . , + ~ = % . o + a : . ~ y , + ~ _ , , to form the

forecast as:

^ M

.~" tl ~ ,{. =Pk-I ) :~ .... ~ + ( I - p t - I ) - ~ : . . + k

( -+,, ) . r - - Y , , * k - I

+ ( < , -

k = 2 .... (2)

where the weight p~_, is the probabi l i ty of the process being in the lower regime at t ime n + k - I

assuming normal i ty , that is p~_~ : ( ~ { ( r - yn÷k_l)/-M

~ . ~ - i } - Subst i tut ing for x",.,+k.)"z.,~ k and P~-i in Eq. (2} we obtain the fol lowing recursive relat ion for

approx imate k-s tep ( k > l ) forecasts:

- M r - - }'n ~ l - - l ~ M

.%1 = +I' -7 . . . . / ( a I ,, + c+l.t.~ ) • n + . k - I V ~ ¢r.~-k-I / .

( " ) + +/~ . . . . ~ ( a : . . + a . . t y , . ~ _ l )

r - v , , , ~ i ( ( r e . , - ( r t l )+?" ~ i

k = 2 .... (3)

where +b(' ) and d~(" ) arc file dis tr ibut ion ;rod prob- ahili ty densi ty function for a standard N(0.1). The i 'ccursion hcgins with:

M y , , . , = ~,, + +qy,, + / , , ( r ) ( f l , , + fl, y,,).

F.q. (3) makes use of t~,,+~ . , . the ( n + k - I ) " ' - s t e p ahead forecast s tandard error. This is ca lcula ted from the rcctirsive formula:

.%, 2 ,4":++~ }p~ " : = {(trl u + trl.i v , .~ - ~)" + tr t,~ - i -t Irtt , I , . ,

. % 1 ~ • + {(% , + %.,Y,, ~k-, 12 + +r~. ~ 5"7, ++ _,}

- %1

^ N | + 2+r_,.,(tv_... + a_,,~y,, .~ _ , )

M

+ +rl.,.v,, +~ .~ ))}&,, .~ .,[,~ ..~ + r r - - . . . . ~)'-

which is appl icable lbr a model with a regime- invariant var iance of the dis turbance term. rr~. De Gooi j e r and De Bruin (1997) show how the method can be ex tended to models with q > 2. p, =pc > I and d > I. The N F E M may not he appropr ia te when the

4 6 6 M.P Clements. J. Smith I Internanonal Journal of Forecastint, 13 (1997) 463-.J75

underlying disturbances are n o t normal- -see , for example. Lai and Zhu (1991) (p. 318).

Z Z Z The MC method The MC method is a simple simulation method for

o b t a i n i n g multi-step forecasts that can be applied a s

easily to complex models (high autoregressive l ag

orders, several regimes) as to the simple SETAR(2 : I , I ) model. For forecasting the n + ! c o n d i t i o n a l o n y, . the regime is known with certain- ty. so that the one-step MC forecast is simply:

~ M C y,, , , -- a 0 + a,y,, + l,,(r)(fl0 + f l lY,) .

which is identical to the one-step forecast produced by the NFEM. However. for observation n 4-2 the forecast regime is determined by y, ~.,. but we only

~ M C have y,,~j (which differs from Y,,,I by a n error term). Deline the It)recasts for y,,,_:, y,, ~ and y,, ~k for a realisation of the error process, d "j'

y . , ~ : = ~r,, + . , y . , . , + / , , , ,(,'){~,, + , - . , ) , , , ,} + _ :.,

(4)

M ( ' / ------, ~ M ( ' I y . . , ~r,, + mY. , ~: + / , , . _.(r){/3,, + fl,.~"""},,,., +.r",.,

(5)

and:

M('i ,; M ( . ' I Y . ~ k = t r o + ~q ) , , . k - I

Nit ' i It + I,,,~t _,(r){fl,, + #,.~,, ~, _, } + ~t., (6)

,~NI ' " r ..-~.1,', > r ) = { t ' ,,,1 , and where. /,,~k_l(r) = / t v,,.~ . o I h c r w l ~ ¢

k > l . Now, averaging these forecasts across the j = I ..... N iterations of the MC yields:

N ~ M ( " = L ~ ~ - ; M C j

tbr k > I. Notice that the pseudo-random numbers are super-scripted by the regime, so that the drawing in period n + k is made from a distribution with a variance approprk, te for the regime tile process is in (determined by the forecast value of the process for n + k - I ) . We assume that the ~'i'., are independent

h • Gaussian variates. ~',.i-N(0.trT,). The normality as- sumption is used here regardless of the actual distribution o f eh.,. In the MC study of Section 3 we assess the performance of the MC medlod for some

non-Gaussian distributions of eh. ,. This method is used extensively by Clements and Smith (1996) to obtain forecasts for a number of empirical SETAR models.

Z Z 3 . The SK method The third method can be viewed as a special case

of the MC method, where the (i'.~ are set to zero. This "naive" or SK method (see Tong, 1995~ amounts to approximating the expectation o f a

function of random variables by that function of the expectation.

Z2.4. Tilt' BS method The fourth rnethod is the BS method, which is

naturally closely related to the MC method. The forecasts are constructed in similar manner to the

MC forecasts except that the ~'J' . ,.p are drawn randonlly (with replacement) from the wittlin-sample, regime- specilic residuals, rather than the normal distribution that is used in the MC method.

Z2.5. 77u' I)1:" mcthtM The lifih method is motivated by the I)F, nlelhod

discussed by Granger and Tcriisvirla (1993). Con- sider the usual method of estimation of tile model given by F.qs. ( la) and (Ih). Conditional on the regime threshold r and the delay lag d being known. the sample can simply be split into two groups and an ordinary least squares (OLS) regression can he run on tile observations belonging to each regifne separately. Alternatively, indicator functions can be used in a single regression, constraining the residual error variance to be constant across regimes (see, for example, Potter. 1995, p. 113.). When r and d are unknown, the model is estimated by searching over these two parameters--that is. the model is estimated by Ol.S at each pair of values (r,d). The range of values of r is usually restricted to those between the 15"' and 85 m percentile of the empirical distribution of Y,-d' following Andrews (1993) and H:msen (1996). while d typically takes on the values 0.1.2... up to the nlaximum lag length allowed. For a known lag order, the selected model is that for which the pair (r.d) minimize the overall residual sum of squares (RSS. equal to the sum of the RSS in each regime). This is the model estimation method we adopt in Section 3 for all tile forecasting methods

M.P Clements. J. Smith I International Journal of Forecastme 13 11997) ~ll93-475 ,1.67

other than DE. except to simplify matters we set d equal to unity. It is a 'one-step" estimation method. since OLS minimizes the sum of squares of one-step e lTOrs .

In the spirit of DE applied to linear models (see. for example. Clements and Hendry (1996). for references), for forecasting k periods ahead, we forecast .v, , k directly from information known at period n (and earlier) by using the in-sample relation- ship between y, and y,_, . For example, for a two- step ahead forecast horizon, the model is estimated by minimizing the in-sample sum of squares of two-step errors. From Eq. ( Ib) the two-step version of the model can be written as:

Yt~-2 ~---

i t . + crl((~ . + a j y , + It(r)([3~ + flj 3',) + ~'t ~ J )

+ I,~ ,(r){[3,, + [3,(,,,, + c~, Yt

4- I , ( r ) ( f l . + f l lY , ) + ~:, ~t )} + ";', ~-" (8)

If wc ignore terms multiplied by I t , l ( r ) in Eq. (8) and simply relate ) ' t , : lo a two-regime model based on Yt wc ohlain:

y, ,2 = ~'r,,, + Crj,,y, + I ,(~:)( f i , , . : + f i~ . :y , ) + ;:t ,.'

(9)

the parameters of which can be estimated by mini- raising the sum of squares of ~3 r ~, over &,.,. fl,.2, i = 0.1, and ~,, for the sample period t = I ..... n. Tltis requires a "grid-search' over ~, as for "one-step' estimation. The gcneralisation to k-step force:isis entails minimising the sum of squares of ~?,,, over ~,._,, fi,._,, i=0 ,1 , and F~. where:

.vt , , = a . , , + ~ , , , y , + / , ( ~ , ) / f i , , . , + f i . . , : ' t l + ~, , ,

The forecast function for y,, .~ is linear in estimutcd parameters and y,, (and /,,(r~)y,,):

( I 0 )

the

Y , , . k = % . k + & L . ~ Y , + / , , k ..k + I.D',, (11)

where ~,.k, fl,.k and fk are the least squares estimates. In a linear setting. DE would not be expected to

yield gains in terms of forecast accuracy over the traditional minimization of one-step in-sample errors (OLSt when the model is corrcctly-specilicd. Here

the model is correctly-specified for the data generat- ing process, but gains might be expected to accrue from the simpler means of obtaining forecasts from models estimated by DE. and also when the distur- bances are non-Gaussian.

An obvious drawback with this method is the computational cost of estimating the model for each forecast horizon.

2.3. The l inear A R m o d e l

Finally, we consider forecasting from linear AR models, which we denote by AR(p) , where p is the lag length. Fitting a linear AR(p) process to Yt yields a linear predictor, where the forecasts solve the recursivc formula:

y , , , ~ = S3. - r . : . . , % y , , ~ _ / J - I

with = , o for k cocl'licicnt estimates from intercept) to the series Yr

(12)

--0, and the ~b; are the litting an AR(p) (with an t = I . . . . . n .

3. MC design

The forecast method comparisons reported in this paper are for tile SETAR(2 ; I , I ) model with d = I:

Yt -----" t r l .o + a ' t . l Y t - i ÷ 'e't.P ) ' , . t - -

y , = a ~ , . + a 2 , ~ y , _ i + e , , t , Yt - i > r ( 1 3 )

wtlerc ~;,.,-Gt0.~r~). We consider Eq. (13) for vari- ous combinations of the parameters drawn from the parameter spaces: ( a L t . a , . i ) ( ~ ( - 0 . 8 , - 0 . 6 , - 0 . 4 , - 0.2,0.2.0.6,0.8) and (al.o,a,_,.o) = ( - 0 . 5 , -0.25.0.0,0.25.0.5), with G. the distribution of the error term, being either normal iN). uniform (U) or chi-squared (2). (X2(2)). For the results reported in this section, d is assumed to be known to be unity.

The comparison of the various forecasting meth- ods for the SETAR model is made by MC as follows. For each iteration of the MC we simulate a series of 210 observations from Eq. (13). The tirst 200 observations are used to estimate the SETAR and AR models (n =200). The lag orders Pl and P2 are assumed to be known to be I, and the parameters {al . , ,cr l . l .a , . .o .a , . I .~r~.r } are al l estimated including

4 6 8 M.P. Clements . J. Smi th I In terna t iona l J o u r n a l o f Forecus t in~ 13 ¢ 1997) - 1 0 3 - 4 7 5

the analogues for DE. We impose the stationarity conditions for a SETAR(2 : I . I ) model, that is: aq.t < I. a~:.~ < 1, and oct.,a,., < 1 (and similarly for the DE estimates). Since forecast comparison is by mean squared forecast error {MSFE). failure to do so might result in the estimated SETAR models generating explosive forecasts on a small number of MC replications which would inflate the calculated MSFEs and make interpretation of the results more difficult.

One- to ten-step ahead forecasts of the ten remain- ing observations are then generated from the esti- mated SETAR model for each of the methods described in Section 2. and from the estimated AR models. For the MC method this entails a set of MC simulations consisting of N = 5 0 0 replications for

each iteration. The forecast errors are then calculated and stored for each forecasting (nethod and model. The above is then repeated for each of I()00 itera- tions to generate a MC sample of forecast errors, and the MC estimates of the MSFEs are the sums of the sqt,ares of the forecast crrors over the I00(} itcra- tions. Notice that the model is re-esti,nattcd on each iteration. Moreover. for the !)1! method the motlcl is estimated for each Iorccast horizon, on each replica- lion.

Only first and sccond-ortlcr AR motlcls arc consid- crctl, since prelimi,mry invcstig:flions suggested thatl longer lag ortlcrs tit) not yield :m improved forecast perlbrm;,nce.

The results are organiscd It) highlight the in)pact of certain aspects of the design of the experiments tm the outcomes of the forecast comparisons. We con-

sider the relative performance of the methods for the SETAR model, and also draw out the gains (if any) to using the SETAR over an AR for forcc.tsting. We do not consider the impact of changing the variance of thc SETAR (nodel disturbances--De Gooijcr and De Bruin (1997) document the deterioration iu the NFEM as this variance increases.

3. I. Results

3. I.I. Gattssh:n disturbances and zero intert'ept.v The first set of experiments focus on the SE"I'AR

model slope parameters aq.~.a~z.~. We assume that the disturbances are Gaussian. t:,,.,~N(O.rr~,), with ,r~ = ~r z = 0 . 5 , a n d set the intercepts to zcro. t h. , = %., = O.

A selection of our results for the MSFEs of the AR(I) . NFEM. BS. SK and DE. all divided by MC. are reported in Table I for one-. two- and five-step ahead forecasts. The results for the AR(2) were never better than for the AR( I ) and hence are not reported. It is apparent that there is little to choose between the MC. NFEM and BS methods for this experimental design. However. the DE method is worse than the BS. MC and NFEM for this set of experiments, and using the "naive method" SK yields much worse tbrecasts in some instances. There appears to be no tendency for the SK MSFEs to converge on those of the other methods as the horizon lengthens, chiefly due to the bias.

T a b l e I

M S F E s o f S E T A R fiwecast m e t h o d s ;,rid AR( I ) m o d a l re la t ive to

M C m e t h o d (~r, ,, = a ' : . = 0. tr, = ~r: : (/.5. G a u s s i a n d i s t u r b a n c e s )

D e s i g n F o r e c a s t AR( 11 NI :EM BS S K DI"

h,t~rizol|

~r,~ = - 0 . ~ 1 ().q9 I.(X) I . ( ) ) I.(X) I.(X)

~r:.j - --().{~ 2 I.(X) I.(X) 1,01 1 .03 1 .03

5 I.(XI I.(X) 1.01 1315 13)4 ~r~ - O.X I 1()~1 I.(X) I.(X) I.(X) I.(X)

er:~ : (1.(, 2 1 0 2 1.01 I.OI 1.10 I.I)A

5 I.(X) I.(X) I.(X) I. It) 1.05

~r~.~ . . . . ().(, I 0 . g S I . ( ) ) I,(X) 1.04) I.(X)

~r,, : -'(L4 2 I.(X) I.(X) 1.01 1.04 1.02 5 I.(X) ] .(XI I.(X) I.(bl 1.05

~q , = - 0 . 6 l 1 .09 I.(X) I.(X) 1 ,00 I.(X)

~ : t - ( ) .g 2 1.(14 1.04) I.(X) 1.08 1.03

5 1.()1 I.(X) I.(X) I. If, I.(15

~ , j : - 0 . 4 I 0 .98 l .O0 I.(X) 1.04) I.(X)

~r,.., : - 0 ,6 2 I.(X) I.(10 I.{X) 1.04 I.(bl

5 1.00 I.{X) 1.(11 1.03 1.03

~ q , = - 0 . 4 1 1.03 I.(X) I.OI I.(XI I.(X)

t~, ~ : 0 . 6 2 1.01 1.1)1 I.(½1 1.116 I .(1"

5 I.(X) I.(10 I.()O 1.13 1.04

~r,.~ = - 0 . 2 I 0 . 97 I.(.~) I.(X} I.(XI I.(XI

~r..., = 0 . 2 2 I.(~1 I.(XI 1.01 1.04 1.03

5 1.04J I.(XP I.OI I.(12 1.04

~r~ : n . 2 1 0 .9~ I.(X) I .(1(1 I.(X) 1.00

~r:, = - 0 . 2 2 I.(X) I.O() I.OI 1.05 1.(}3 5 1.04) 1.00 I .(X) I.(M 1.05

~r~ = 0 . 6 I 1.18 I.(X) I.()O 1.04) I.(X)

% , = -O.S 2 1.02 1.0(1 1.01 1.10 1.05

5 I,(X) I.(X) I.(R) 1.21 I.(12

~r,~ = 0 . 6 I I . lO l.{X) l .O0 I . ( ) ) 1.00

~r: ~ = - { 1 . 4 2 I.{)2 I.(X) 1.00 I.(Iq 1.05

5 1.04) I.(XI I .(X) I. 13 1.05

(i 'rl : ().~} I 0 .98 I.(X) I.(X) 1.00 I.(X) ~q.~ =0.4 2 l.(X) 1.00 1.()) 1.03 I.(15

5 13)0 1.00 1.00 1.07 I.(M

M.P Clements. J. Smith I International Journal of Forecasting 13 (1997) 463-475 469

The gains relative to an AR model can be large when og., and ~..., are of opposite sign. and close to the unit circle. Not surprisingly, the AR model is penalised less heavily when these parameters are of the same sign and close together, since the process generating the data is then closer to a single-regime (i.e. linear) model. The closer these parameters are to zero. the closer the process to white noise, which can be optimally forecast by an AR model with AR coefficients of zero.

More interestingly, the penalty to using an AR model is greater when og.~>0 and t r : . j<0 than when these parameters have the reverse signs (com- pare. for example, the entry in Table I for og.~ =0 .6 and trz. ~ = - 0 . 8 with that for og: = - 0 . 6 and or..., = 0.8). ~,, = -0.4

When crj . )<0 and c£,.)>0 (e.g.. (:r).) = - 0 . 6 and a,, =-0 .o

c~:., =0.8). once the process enters the lower regime ~,, = -0.4

due to a sufficiently large negative shock e ' , ~ - , : ,=0.6 (L,.tY,+). the process will llip back to the upper regime the next period unless ~; ,)<--~rj .~y, . This , , , = - 0 . 2 'anti-persistent' behaviour can bc exploited by the ,,:, =o.2 SETAR model to yield improved forecasts. It is

~r,, =0.2 similar when %.) > 0 ~,nd ~r,.~ < 0 . in which case the ,,.,, = -0.2 prt)ccss will tend not to remain in the upper regime, which also suggests improved forecasts from the ,, , ---1):) S E T A R model. ,,:, ::-o,~

Table I indicates that any gains to the SETAR tq., =0.6

relative to the AR model will be negligible after t t : , = - I ) . 4 f ive-steps ahead, and wc can do no better than

forecasting the unconditional (zero) mean of the ,, ,=o.6 p r o c e s s , tL'.' = 0.4

3. 1.2. UniJ'orm dist,,rbances We investigated the irnpact of uniform distur-

bances in Eq. (13), ~',/trh ~{U[0.11 - 0 . 5 } x/~_, for a range of values of trt. t and tr.,.t (with zero intercepts, ~r,.. = or2:)=0). The NFEM is only strictly applicable when the disturbances of the SETAR process are normally distributed, and De Gooijcr and De Bruin (1997) conjecture that it may be possible to adapt the method for other distributions. However, it is of interest to ascertain the robustness of the NFEM to various departures from normality, since in practical applications this assumption will typically only hold appmxinmtely.

Table 2 reports a selection of results for the same values of ~,.) and ot:.t as in Table I for the Gaussian

Table 2

MSFEs of SETAR forecast methods and ARt 1 ) model relative to

MC method ( a a = a_, )=0 . % =o'., =0.5, uniform disturbances)

Design Forecast ARt I ) NFEM BS SK DE

horizon

og., = - 0 . 8

or:., = - 0 . 6

og) = - 0 . 8 a , , =0 .6

og., = - 0 . 6

~t:., = - 0 . 4

0% = - 0 . 6

tr:., =0 .8

0.98 1.00 1.00 1.00 1.00

0.99 0.99 0.99 1.01 1.04

1.00 1.130 1.00 1.01 1.02 1.08 1.00 1.01 1.00 1.00 1.00 1.00 1.00 1.09 1.04 1.00 1.00 1.130 1.20 1.03

0.96 1.00 1.00 1.00 1.00

0.99 0.99 1.00 1.02 1.05

1.00 0.99 0.99 1.02 1.03

1.07 1.00 1.00 1.00 1.00 1.03 1.00 1.130 1.08 1.01

1.00 1.01 1.01 1.23 1.05 0.98 1.00 1.00 1.00 1.00

1.00 1.00 1.00 1 .I)2 1.05 I .iX) 1.00 1.00 1.02 1.02 1.03 I.(X) I.(X) I.(X) I.(X)

1 .00 I . iX) 1 .00 1 .07 1.01 I .(X) l . (X) l . (X) I . 14 1.03 0.99 I .(X) I .(X) I .(X) 1.1X1

1),99 I.(X) I.(X) 1.02 1.04

I ,(X) I .(X) lAX) I .(R') 1.03

0.97 I.(X) 1.00 lAX) 1.00 I,(X) lAX) lAX) I.IX) 1.0.t

I .(x) I .(x) I .(Xl 1 .04 1 .02 1 .06 I .(X) 1 .(X) 1 .(XI I .(X)

0.99 I.(X) 0.99 I. 14 1.0-4

I.(X) I.IX) I.(X) 1.24 1.04

I.(X) I.(X) I.(X) I.IX) I.(X)

0.99 I.IX) 0.99 I .I)9 1.03

I.(X) I.(X) I,(~) I. 15 1.02

0.¢)7 I .(X) I .(X) I .(X) 1.00

0.99 1.00 1.00 1.03 l .(X)

I .(X) 1 .00 1.01 1 .09 1.03

case. It is evident from Table 2 that there are no gains to the BS method relative to the MC method. Although not shown, the NFEM performs badly at the longer horizons (eight- to ten-steps ahead) for experiments where one or both of a~.~ and oL,.t are close to zero. The problem appears to be that the recursive formula for the forecast error variance starts to generate explosive values.

Relative to the ARt I ) model, the large gains to the SETAR in the Gaussian case, when og.~ and 0% are large and og : > 0 and a:.~ <0 , are reduced, but the (smaller) gains to the SETAR when the slope param- eters take on the reverse signs are affected less by the move from Gaussian to uniform disturbances.

.170 M.P Clements. J. Smith I International Journal of Forecastim¢ 13 (1997j .163-473

3. 1.3. Chi-squared disturbances

We also study the impact of asymmetric distur- bances in the SETAR model, with the errors in Eq.

( 131 drawn from a x~-distribution, et,/trj, ~ {x-'t2) -

2}/ , .4. Table 3 reports a selection of results tbr the same values of a i j and a:., as in Tables I and 2 tbr ease of comparison (and again zero intercepts). It is readily apparent that in some cases there are gains to BS relative to MC. but these are rarely much larger than 2%, and occur for values of a~., and %., closer to zero. in fact the gain to using the BS method is maximized at aq., = a , . , = 0 (not shown), where the process is a series of lid g~ variates. The NFEM sometimes produces very large numbers at the longer

Table 3 MSFEs of SETAR forecast methods and ARt I ) model relative to MC method (a'~ ,, = a: ,, =0. rr, = ~r: =0.5. X:(2} disturbances)

Design Forecast A R t I ) N F E M BS SK DE horizon

~r, , = - ( ) . 8

t r : , = - 0 . 6

~r,, :: - 0 . 8

tr n , ~- -- 0.0 ~r: , -: - 0 . 4

( ~ , , :: - ( ) . 6

tq., =O.g

t r t , = - 0.4

% = - tl.e,

tr, t = - 0 . 4

tr: ~ = 0 . 6

Cr, , = - 1 1 . 2

er a , =0.2

~r,, = 0 . 2

~r:., = - 0 . 2

a,., =0.6 ( r : , = - 0.8

~q ~ = 0.6 ~t,~ = - 0 , 4

5 oq, =0.6 I ~r.,. z = 0 . 4 2

5

n,t)7 i.o11 I .(111 I . ( io I . ix)

n .99 i .oo I .o11 I.(12 I .o I

11319 I.(X) I .n I 1.04 1.112

1.117 I .no 1.1R1 1.111 I .ix)

I.(X) 1.11(1 11.9g I. I I5 1.112

0.99 - ().gtl I. I I I . IXI

().95 I . IXI l.(X) I.(XI l.{X)

().98 I.(X) I.(Xl I.(11 1.01

I.(XI lAX) I .(X) 1,03 1,112

1,113 I , ( l ) l , l ) l lAX) 1,110

1.02 I,IIO ().98 1.02 I.(H) 0.98 - 0.98 1.12 I.(12 (),~R) I.(R) 1.01 lAX) I.(R}

0.09 I .(R) I .<11 1.113 I .(11

I . IXI I.{X) 1.01 I.O2 1.()2

I.I13 I.(11) 1.00 I.IX) 1,00 0.99 I.IX) O.tJ8 I.()3 I .(X) (1.9 t) - 0 , 9 8 1 .0 t) 1 .01

0.98 l.(X) I .(X) 1.0() 1.00 0.95 0.99 ().g8 0 .96 0.99 I .(R) - 1.(10 1 .03 1 . 0 3

1 . 0 0 I .(X) I .(R) lAX) I.IR)

0.98 1.011 0 .g9 1.04 1.01

n.98 - 0 .99 I,(X) 1.112

1.24 1.00 1.00 1,00 I.(X)

1.02 I .IX) I .IX) I. I 0 1.00 I .(X) l .O0 1.00 1.21 1.02

l.Og l.(X) I .I'X) I.(X) 1.011 1.01 I.I10 1.00 IAY) I .()5

0.gq I.(X) lAX) I. I 1 1.03 ().97 I.(X) 1,00 lAX) I.(X)

I .(X) I .iX) 1.01 1.03 I ,n2 I.(X) 1.0o I .(x) 1.()5 I .()3

horizons (not shown) due to the forecast variance exploding, occasioned by ~,.~ < - i , i = i.2.

The AR model dominates the SETAR when the slope parameters are of the same sign (e.g. aq.i = -

0.6.az. I = - 0 . 4 ) or are of opposite sign but close to zero (e.g. at , ~ = - 0 . 2 , a : . , =0.2) .

3.1.4. Non-zero intercepts

The effect of allowing non-zero intercepts in the SETAR model Eq. ( 131 was investigated for the case of Gaussian disturbances. For most of the combina- tions of intercepts (¢h.o.a:. ,) and slope parameters (cfl.~.cqj) we considered, the MC. BS and NFEM all perfommd almost equally as well. For the larger values of the intercepts (--0.5) . the NFEM was sometimes a little worse than MC and BS at longer horizons, but any such differences were usually only of the ordcr of I or 2%. The entry in "Fable 4 is

onfitted for the experiment with ~h.i)= a , . , =0 .5 and cL,.a =0 .8 bccausc there were insuflicicnt observa- tions in the h)wcr regime to reliably estimate thc SF.TAR model. In Table 4 we report the results for the ARt I ) relative to the MC method, and focus the discussion t)n the ct)mparison of thc SFTAR and AR models.

Non-zero intercepts can greatly extent,ate or at- tenuate the relative forecast performance of the S F T A R model. The pattern of results in Table 4 can bc tmderstood by running through a number of cases.

C(msider the case of ~.t't,t = - 0 . 8 and tt'_,.t-().6. which yielded a one-step gain to the SETAR of 9% in the absence of intercepts (see Table I). When

thin = u..., =0 .25 the process will be ahnost exclu- sively in thc uppcr regime, land will only move to the lower regime it" e ; - < - ( 0 . 6 y , _ j +0.2511 and the gains to the SETAR model are consequently reduced (and become negative when the intercepts are both equal to 0.5). If instead -oq .~=o¢. , . ,=0 .25 the previous argt, ment is little changed. When. however. aq.. = - oL,.. =0.25, the process will tend to oscillate between regimes, exhibiting a pattern that c;.ul be exploited by the SETAR to record large gains (40%) relative to the AR model. This pattern of movement betwcen regimes is heightened when %. . = - % . , , = 0.5. and the gain at one-step ahead increases to 85%.

As another example, consider the case of ~h.t = 0.6 and az. ~ = - 0 . 8 . When the intercepts are zero there are gains to the SETAR of nearly 18~ at one-step

M.P Clements. J. Smith I International Journal of Forecasting 13 t 10971 403-475

Table 4 MSFE for AR(II relative to MC method (o'~ =o'., =0.5. Gaussian disturbances)

471

Design Forecast oq ,,a:. o horizon

0.25.0.25 -0._5.0._5 0._5. 0._:, 0.5.0.~ -0.5.0.5 0.5.-0.5

a, ~ = -0.8 ~t: T = - 0 . 6

o q , = - 0 . 8

%, =0.6

or, , = - 0.6 a':. I = - 0 . 4

ct, ~ = -0.6 o~:~ =0.8

ct,, = -0.4 (~:.t = - 0 . 6

~'~, t = - 0.4

t r , , = - 0.2

~r: = - 0 . 2

or, t 0 .2 ~ r : , : : 0.2

~r: - O.N

tr., , = --1).4

5 ~r,, =0.6 I ,~,2.1 -: 0.4 2

5

0.99 1.05 1.06 0.98 1.30 1.14 1.00 1.00 1.04 1.00 1.00 1.23

1.00 1.00 1.05 1.01 1.00 1.37

1.02 1.02 1.41 0.95 1.00 1.87 1.01 0.99 1.02 1.00 1.00 I. 13

1.00 1.00 1.00 O . t~ 1.00 1.00 0.98 1.06 1.05 0.97 1.32 1.18

1.00 1.01 1.02 1.130 1.02 1.20

1.00 1.00 1.01 1.00 1.00 I. 15

0.98 1.04 1.41 0.98 1.71 1.00 1.00 1.02 - 0.99 1.07 1.00 I.(X) 1 .00 - 1.01 I .IX) 0.98 1.04 1.06 0.90 1.31 I. 17 1.00 l.OO 1.04 1.00 1.02 I. 16 I .IX) I .IX) 1.01 I .(X) l .IX) I. I0

1.00 1.02 1.24 O.t)S 03)7 1.63

I .(X) 0,99 I .(X) n.qq o,q~ I. I I

I.IX) I ,IX) I .IX) I.IX) 1.02 I .IX)

0.97 1.08 I. I I 11.96 1.23 1.44

I . IX} 1.01 1.112 I .IX) 1.09 1. [ 2 I .IX) I .IX) I .(X) I .IX) 1.01 I .(11

1.00 1.07 I. I0 O.t~S I. lq 1.30

I.IX) I.IX) 1.02 I.IX) 1.05 I. 15

I . IX) 1.00 I .IX) 1.00 1.01 1.01 1.24 l.l)7 1.411 1.211 0.97 I.N(~

I .IXI 1.02 1.01 I .IX) 0.9q I. [ 9

l ,IXI I .IXI 1.00 L.(H) 0.gq I .(HI

I. I I 1.07 1.21 l.OS 0.99 1.57 I .(X) 1.0 [ 0.99 I .IX) 0.99 I. 14

I .IX) 1.00 1.01 1.00 I .IX) I .IX) 0.98 1.112 I. I0 0.96 I. 10 1.30

I .IX} 1.02 1.00 0.99 I. 16 1.0 I

I .IX) I . IX) 1.111 1.1,~) I. I I I .IX)

ahead (sec Table I ) . Unlike in tile case just consid-

crcd (when the signs o f the A R parameters are

reversed), setting tt~.. = m . , = 0.25, (or 0.5), does not

diminish tile gains to the S E T A R model , since m.~ <

(I ensures the process will still be observed ill the

lower regime. As for the previous case. the gains

increase when a l . . = - o ( , . a = 0 . 2 5 , or 0.5, when the

process again oscil lates be tween the regimes. When

%.u = -°C,.o = - 0 . 5 the gains to the S E T A R become

negat ive because the process remains in the lower

regime most o f the time.

Therc can be substantial gains to the S E T A R even

when (x~. I = cq. t provided the intercepts are different

and the process oscil lates be tween regimes.

4. S E T A R a n d A R m o d e l fo recas t s o!" p o s t - w a r

US G N P

In this section some of the forecast ing methods are

used to generate multi-step forecasts froln S E T A R

models of US G N R Potter 11995) littcd a

SETAR(2,5 ,51 model to the change in seasonally-

adjusted real US GNP data for the period 1947.1-

1990.4, with the coeff icients on the third and fourth-

order lags set to zero. Tiao and Tsay 119941 esti-

mated a two-reg ime model as well as a more

elaborate four-regime model , and obtained quali-

tat ively s imilar results to Potter (1995). These

models suggest that US G N P is relat ively robust to

472 M.P. Clements. J. Smith I International Journal o f Forecasting 13 (1997) 463-.175

negative shocks--the economy moves quickly out of recession. Tiao and Tsay (1994) find that their SETAR model forecasts (calculated by the MC method) are only a little better than those from an AR(2) model, but are markedly superior if we only consider forecasts which are made when the economy is in the lower regime. This reflects the ability of the SETAR model to capture the rapid movement out of recession.

In Section 4,1 we redo the Tiao and Tsay forecast exercise with some minor but nevertheless important modifications to evaluate the performance of the SETAR forecasting methods for an empirical exam- ple. In Section 4.2 we analyse a later vintage of data for the period 1960.1-1994.4, and analyse forecasts of the period 1991.1-1994.4.

Only the MC, BS and SK methods of forecasting the SETAR models are considered. Based on the MC results in Section 3, the DE method has little to commend it, and whilst the NFEM generally per- formed well, the MC method is simpler to implement (although computationally more intensive).

4. I. An t'mpirical forecast comparison exercise for the period 197d. I - 1990.4

The Ti:to and Tsay (1994) study compared fore- casts from a SFTAR(2;2,2) to those I'rom a linear AR model for US GNP. Following Tiao and Tsay, forecasts for the SETAR model arc calculated by

estimating the model up to 1972.4, generating multi- step forecasts up to twelve steps ahead (by the MC method), and then re-estimating the model on an extended sample to 1973.1. etc., up to a final estimation period ending in 1987.4. This gives a sample of 60 sequences of one- to twelve-step ahead forecasts. However, certain parameters, namely {d.r, pt,pz}, were selected and held fixed by Tiao and Tsay at their full-sample values, rather than being those which would have been chosen based only on information available at the time the forecasts were made. Hence their study could be misleading as a guide to the genuine ex-ante performance of the SETAR model. Table 5 details the values of {d,r.p~,pz } that would have been chosen on the basis of minimizing AIC for selected sub-samples.

Fig. I plots the MSFEs for multi-step ahead forecasts (one to twelve steps) for the MC method relative to the AR(2), BS and SK methods, when wc follow Tiao and Tsay and use a SETAR(2;2.2) model with d and r lixed at their full-sample wdues. We lind gains to using the SETAR model (MC method) at short horizons. At four-steps ahead the gain over the AR model is maximised at nearly 10%. At longer horizons there arc no clear differences between thc ft)rccast methods for the SETAR, or between the SETAR and AlP, models.

Fig. 2 presents the results of a genuine ex-antc forecast comparison with r, d, p~ and p, all chosen to minimise the Akaikc infomlation Criterion (AIC)

Table 5 Sub-sample es t imates o f S E T A R mode l for US G N P : 1947 to speci l ied end ++late

t'.'lld dale r tit P+ t'rl .tJ err, l u I .., p , U,.o tr.,. i

73:4 -0.0<)8 3

74:4 - 0.0<)8 3

75:4 -0.1)<)8 3

76:4 - 0.(1<)8 3

77:4 -{).0<)8 3

78:4 - 0 . 0 8 7 3

79:4 - 0.008 3

80:4 - 0.0<)8 2

81:4 - 0 . 0 0 8 2

82:4 - 0 .008 2

83:4 - 0.0<)8 2

84:4 - 0 . 0 5 8 2

85:4 - 0.0<)8 2

86:4 - 0.0<)8 2

87:4 - 0.058 2

1.049 0 .329 -

0.931 0 .415 -

0 .986 0 .364 -

0 .986 0 .364 -

0 .986 0 .364 -

1.078 0 .195 -

0 .970 0 .298 -

- 0 .456 0 .410 - 0 .918

- 0.533 I).415 - 0.960

- 0 .497 0 .399 - 0.1t51)

- 0 .479 0 .396 - 0. 852

- 0 .500 {).398 - 0.868

- 0 .479 I).396 - 0 .852

- 0 . 4 5 2 0 .392 - 0 . 8 3 8

- 0 .469 0 .394 - 0 .852

0 .369 (I.443

0 .316 0 .460

0 .256 0 .498

0 .269 0 .492

0 .284 0.485

0.315 0 .494

0.322 0 .468

0.659 0 .328

0.660 0.319

{).61 I I).353

0.627 0 .356

0 .614 0.363

0.625 I).358

0.621 0.351

I).622 {).353

M.P. Clements. J. Smith International Journal of Forecasting 13 ¢ 1997) .103-475 473

1.06 0

1 04 /~xx ." '.. sa,~, , , a

,~_~,.:,,. . + . . / ~ , , ,.. , . .0.,;

o~ o ~ o.m ~ i .°'." t'~

og 2 3 • S § 7 ~ ~ 10 11 1~

Fig. l. Relative MSFEs for US GNP 1973.1-1990.4 for a SETARi2;2.2), r = 0 . d = 2 (the MC meth~,'d MSFE is expressed relative to that for ARI2k BS and SK).

T~ s ~ ~ D . , 104 .,..'" "

; o+ . . .p- . .+o. . . ,~ ~ - - A R I

O~ I1~ "~ , . O , . , s a _ _.~1_ _ ii~ffl~ i

0~,/ "a"

1 ;' 1 I 5 + t + 9 10 II ~;/

Fig. 2. Relative MS|:I-x Ibr US GNP 1~173.1-19()(1.4 for a SI"['ARI2;p~.I;._), r. d ¢stinlalcd (tile MC lllcthtml MSFF is expressed rehltiv¢ h+ that h~r All(2), |IS and SK).

on each sub-santplc (and correspond to the values reported i,t Table 5). We now lind the three reported mcthuds of forecasting the SETAR model yield inferior forecasts to the AR(2) model lk)r horizons up tu three-slops ahead. At three-steps ahead, Ik)r exam- pie, the MC method is approximately 7% worse than the AR(2). The best method at longer horizons appears to be the SK.

I I ~ j t x x

I tl$ x\ / xxx

I "lk --"O--" MG'AK L 1 (J ,j ~ L- "0- - MeT ̂II 13

I .' I 1 ~ ~ 7 I 9 IO II 12

Fig. 3. Relative MSIq's I't)r US GNP 1973.1-19qO.4 for a SI:.TAR(2;/h.p,). r, d estimated (the MC MSFE relative to AR(2) l't~r the hra, cr regime. I+. and the upp,.:r regime. U).

However, a different story emerges if we con- dition on the regime. Fig. 3 plots the relative MSFEs of the MC method to the AR(2) model conditional on the regime at the time the forecast was made. There are gains of 10% or more for the SETAR model at four- and five-steps ahead in the lower regime (MC/AR:L) with nothing thereafter, com- pared with substantial losses at all horizons less than five-steps ahead in the upper regime (MC/AR:U).

These results are broadly in line with those of Clements and Smith (1996). They take the estimated SETAR(2;5,5) model of Potter (1995) and assume that this is the data generating process. In a MC study they find that when the SETAR model is assumed known, then, using the MC method, there are gains of up to 16% at one-step ahead and 21% at two-steps ahcad against an AR(2) model. However, when the AR coeflicients and the lag orders arc unknown (but r and d are known) the potential gains fall to only I% at one-step ahead and 5% at two- steps ahead, when we do n o t condition upon being in a particular regime. Thcsc lindings are also con- sistcnt with tht>sc of Hanson (1996). Hanson is unable to reject the linear model in favour of tile SE'I'AR for tile period 1t)47-1990, suggesting that :uty gains to allowing for SETAR non-linearity arc t, nlikcly to be large.

4.2. An empirical forecast c'otnl~ari.~'tm exercise for the period 1991. I - 1994.4

The second forecasting exercise is based on tile more recent vintage of US GNP data at 1992 prices for the period 1960.1 to 1994.4. We calculate twelve sequences of one- to live-step ahead forecasts for the period 1991.1 to 1994.4, using a SETAR(2:p~,p.,). The first forecast origin is 1990.4, and the last is 1993.3. On each estimation sub-sample {d,r, pa,p., } arc selected to mininaizc AIC in each regime (but not allowing holes in the lag distributions). A SETAR(2;I,I) is sclected for each sub-sample, with r---0.2 and d=2 . Thus, omitting the late 1940s and 1950s from the estimation sample alters the spe- cilication of the SETAR model. This is not a data vintage effect--estimating the model over a similar sample period on the Potter dataset gave similar results, pointing to non-constancy of the SETAR model.

474 M.P. Clements. J. Smith I Internattonal Journal of Forecasrine 13 I 19q7~ 463-475

Table 6 Out-of-sample foreca.sts of US GNP 199 I. 1-1994.4

Steps AR(2) AR(3) MC BS SK

MSFEs relative to the MC method I 1. I 0 1.26 1.00 1.00 2 1.01 1.19 0.91 0.92 3 0.82 0.87 0.93 0.89 4 1.08 1.05 1.58 0.86 5 1.03 I. I I 1.98 0.92 Forecast biases I 0.04 0.07 0.07 0.07 0.07 2 -0+02 0.03 0.04 0.04 0.04 3 -0.03 0.03 0.06 -0.05 0.08 4 -0.07 -0.03 0.07 -0.10 0.08 5 -0.13 -0.10 -0.12 -0.23 0.02 Forecast error variances I 0.17 0.18 0.16 0.16 0.16 2 0.11 0.13 0.10 0.10 0.10 3 0.11 0.13 0.11 0.10 0.10 4 0.10 0,11 0.13 0.11 0.11 5 0.07 (1.07 0,07 0.07 0.07

Nevertheless, our prim:try interest in in comparing thc SFTAR and A R mt)dcls, and the ahcrnative methods of generating n|t, lti-stcp forecasts for the former, The MSFF, s fronl using the IIS alld SK mctht)ds for the SFTAR, a,ld the A R t 2 ) and A R ( 3 )

models, all cxprcsscd rclativc to the MC method, are reported in Table 6. This table shows that the ARt2) model is I(}% worse than the SFTAR MC method at one-step, similar at two-steps, ifnd nearly 20% better at three-steps. The SK ntethod ix generally the bcsl over the horizons considered.

The brcakdown of tile MSFEs into bias and variance components indicates that SETAR MC forecasts are more biased than those of the ARt2) at one- through to four-steps ahead, but have :t smaller variance at one-step ahead.

5. Conclus ions

We have compared a number of methods that have been proposed in the literature for obtaining h-step ahead minimum mean square error forecasts for

SETAR models. The NFEM, BS and MC methods all perl'oml reasonably well fi)r a variety of different SETAR(2; I , I ) models. The MC rnethod is generally at least an good an that of the other methods we

consider, and is simple to implement. The drawback o f this method relative to an approximate analytical

method such as NFEM is of course that it is relatively computer intensive. However, the c o m p u -

tat ional c o s t o f investigating the properties of this method by simulation (as in this paper) were not

prohibitive given the current-generation fast com- puters.

When the SETAR model disturbances come from a highly asymmetric distribution, a BS method is to be preferred to the MC method. But even then. the costs to using the MC method in terms o f forecast accuracy are relatively small, while the NFEM, based on the assumption of Gaussian disturbances, is less robust at long horizons.

The comparison of tile accuracy of the lbrecasts from the SETAR to that of an AR model highlights those instances when a linear model provides a good approximation to the non-linear model, at least for forecasting. The sign and magnitude t)f the intercepts in tile SFTAR model were found It) have a large effect on comparisons of tile SETAR against life AR ntodcl forecasts.

In the empirical study we present multi-step ahead forecasts for US ( ;NI ' from a SI:TAR and a linear AR model for two historical epochs. The SFTAR I'orcc:tsts are citlculalcd using a number of tile methods we study in tile MC. Our investigation casts some doubt on tile robustness t)f SF'] 'AR models of US GNP over tile two slunple periods, and on the

size of the forecilSt ilccuritcy gains that the SETAR models can dcliver. We are able to replicate tile linding in Tiao and Tsay (1994) that the SETAR model produces smaller MSFEs than an ARt2) model over the period 1973.1-1990.4, when certain parameters are set at their Ihll-sample values. These gains disappear if the SETAR model is specilied and estimated solely on tile information available at tile time the fi)rccasts are made (its would be the case in a genuine forecasting exercise), but can bc re-in- stated if forecasts are evaluated conditit)nal t, pon the process being in a particular regime.

Finally, when analysing fi)rccasts of the pcrit)d 1991. I - 1994.4 no clear picturc emerges. There arc unconditional (on the regime) gains to a SETAR model compared to an AR model at one-step ahead, but not at two-steps, and at three-steps the ranking reverses.

M P Clemems. J. Smith I International Journal of f"orecctstin~ 13 (19'47~ 4153-4Z~ 475

Acknowledgements

The first author a c k n o w l e d g e s f inancial support

under ESRC grant L I I 6 2 5 1 0 1 5 . We are grateful to

Jan G. de Goo i j er and to two a n o n y m o u s referees

w h o s e c o m m e n t s have improved the lbcus and

organisat ion o f this paper.

R e f e r e n c e s

AI-Qassam. M.S.. Lane. J.A.. It)89. Forecasting exponential autoregressi',e models of order I. Journal of Time Series Am, ly,d,~ I(), t)5-113.

Andrews. DWK. . It)03. Te,,ts for parameter instability and stnictural change with unknown change point. Econometrica ~)I, 821-856.

Clement,< Me. . tlcndry. D[:.. It)t)6. Muhi-step estimation for l'c, recasting. Oxford Bulletin of | 'conomics and Slatb, lics 5~. ('~57 ~6S4.

Clemcnts. M.I'. and J. Snfilh. 1901), A Monte Carlo study of the Iorccasttng pert,.)rlnal|ce ol empirical SI-TAR models. Xt+trwi+k

I'.<'+,m,mi( Rc~<'arch I'<q,+'r+ No.4t')J. (I)epartn)ent ol |'~con OII|iC~,, [ lnivel'sily o i Warv, ick).

l)e (h)()iict, J.(;., Kumar. K.. 10o2. Some recent deveh)pmenls in m)n linear time seties modelling, leslmg and ('()recasting. Inlclnalumal Journal ()1 F()rccanling 8. I.),5-15ft.

I)e (h)~)iiel..I.(i and I ), I)e Ihuin. lot)7. ()n SI'~'I'AR folecaslmg, ))'t.tt~ticg +rod I'.d,al,ilitv letters. Iollhcolnitlg.

Diebold+ FX., Na>on. J.A., tot)<). Nonparametric exchange rate prediction. Journal of International Economics 28. 315-332.

Granger. C.W.J. and T. Terasvirta. 1993. 3,h)dellin¢ N+mlinear Ecom)mic Re[atumships (Oxford University Press. Oxford).

Hansen. BE., 19<40, |nfcrence "*'hen a nuisance parameter is not identified under the null hypothesis. Econometrica 04, 413- 430.

Lai. T.L.. Zhu. G.. 19t)I. Adaptive prediction in non-linear autoregressive models and control system,:. Stati,,tica Sinica I. 3(D-334.

Potter. S.. It)95. A nonlinear approach it) U.S. GNP. Journal of Applied Econometrics IO. 109-125.

Tiao. G.C.. Tsay. R.S., IOqa, Some advance,~ in non-lmear and adaptive modelling m time-series. Journal of Forecastmg 13. lOt)-131.

Tong. H.. 197,~. On a threshold model, in: C.l-t. Chen. ed.. Patt('rn Recm..nition and Sienal Pro('('.v~ink, (Si jhoff and Nc~,rdol+f. Amsterdam), I(H+- 141.

Tong. t1.. It)~)3, Non-Lin('ar Tim(" S+'ri('x. A /)wt<mti('al Sx~tem A[)l)roa<'h (Clarendon Pre,~s, Oxford).

"F~mg. II.. l+im. K S.. lq~O. Thresh.ld autoregre,,.ion, limit cycles and cyclical data. Journal oI the RL)yal Stati,,tical Socict'. B42. 245 - 2t)2.

lliol~raphies: Michael I'. ( ' I . I !MI iNTN is a Rc:.catch Felh,v.' m L'conoIIIiCS ill lh..+" {Tlli',u'rsily of 'VValwick. and iul Ay,+,t~'lille

Member o l Nuflicld ('ollegc. (),O'~.d.

J~+'l+t+'ll1~ + SMITll i~ a l.(:clttrcr m econonfiu's at the [!IHvL.r,,ID, + ol Wa,v.quk.