interval forecasting - zentraler informatikdienst...

IntroductionPrediction Mean Square Error

Prediction IntervalsEmpirically Based P.I.s

Summary

Interval Forecasting

Based on Chapter 7 of the Time Series Forecasting byChatfield

Econometric Forecasting, January 2008

Pekalski, Swierczyna, Zalewski interval forecasting



Summary

Outline

1 Introduction

2 Prediction Mean Square Error

3 Prediction Intervals

4 Empirically Based P.I.s

5 Summary




Summary

IntroductionTerminologyInterval ForecastsDensity ForecastFan Chart

IntroductionIntroduction

Most forecasters realize the importance of providinginterval forecasts and point forecasts in order to:

asses future uncertainty,enable different strategies to be planned for the range ofpossible outcomes indicated by the interval forecast,compare forecasts from different methods more thoroughly.




Summary


IntroductionTerminology

An interval forecast usually consists of an upper and lower limit.The limits are called

forecast limits,prediction bounds.

The interval is called aconfidence interval,forecast region,prediction interval.




Summary


IntroductionPredictions

Predictions:are not often a prediction intervals,most often are given as a single value.




Summary


IntroductionInterval Forecasts

Reasons for not using interval forecasts:rather neglected in statistical literature,no generally accepted method for calculating predictionintervals (with some exception),theoretical prediction intervals are difficult or impossible toevaluate for many econometric models containing manyequations or which depend on non-linear relationship.




Summary


IntroductionDensity Forecast

By density forecast we mean finding the entire probabilitydistribution of a future value of interest.Linear models with normally distributed innovations:

the density forecast is typically normal with mean equal tothe point forecast and variance equal to that used incomputing a predicion interval.

Linear model without normally distributed innovations:seems to be more prevalent and used e.g. in forecastingvolatility,different percentiles or quantiles of the conditionalprobability distribution of future values are estimated.




Summary


IntroductionFan Chart

Fan charts




Summary

FormulaPotential PitfallExampleKnown vs. Unknown ParametersConditioning Forecast ErrorExample

Prediction Mean Square Error

Preditcion Mean Square Error (PMSE):

E [en(h)2].

Forecast is unbiased that is,when x̂N(h) is the mean of the predictive distriburion,then

E [eN(h)] = 0 and E [eN(h)2] = Var [eN(h)].




Summary


Prediction Mean Square ErrorPotential Pitfall

Assesing forecast uncertainty, remember:

the var of the forecast 6= the var of the forecast error.

Given data up to time N and a particular method or model:the forecast x̂N(h) is not a random variable, it has varianceof zero,XN+h and eN(h) are random variables, condtioned by theobserved data.




Summary


Prediction Mean Square Error

How to evaluate:

E [eN(h)2] or Var [eN(h)]

and what assumptions should be made?




Summary


Prediction Mean Square ErrorExample

Consider the zero-mean AR(1):

Xt = αXt−1 + εt , {εt} ∼ N(0, σ2ε ).

Assume: complete knowledge of the model (α and σ2ε ).

The point forecasts x̂N(h) will be

α̂hxN rather than αhxN .




Summary


Prediction Mean Square ErrorBias of PMSE

Even assuming that the true model is known a priori, there willstill be biases in the usual estimate obtained by substitutingsample estimates of the model parameters and the residualvariance into the true-model PMSE formula.




Summary


Known vs. Unknown Parameters

Consider the case of h = 1 and conditioning on Xn = xn,

eN(1) = XN+1 − x̂N(1).

If the model parameters were known, then

x̂N(1) = αxN ⇒ eN(1) = εN+1

If parameters are not known:

eN(1) = XN+1 − x̂N(1) = αxN + εN+1 − α̂xN = (α− α̂)xN + εN+1

Assume: parameters estimates are obtained by a procedurethat is asymptotically equivalent to maximum likelihood.




Summary


Prediction Mean Square ErrorConditional and Unconditional Errors

Looking once more at equation

eN(1) = (α− α̂)xN + εN+1.

Consider:conditional on xN forecast error: xN is fixed α̂ - biasedestimator of α, then the expected value of eN(1) need notbe zero.unconditional forecast error: If, however, we average overall possible values of xN , as well as over εN , then it can beshown that te expectation will indeed be zero giving anunbiased forecast.




Summary


Prediction Mean Square ErrorComputing PMSE

Important when computing PMSE.To have unconditional PMSE:

average over the distribution of future innovations(e.g. eN+1),average over the distribution of current observed values(e.g. xN ).

eN(1) = (α− α̂)xN + eN+1




Summary


Prediction Mean Square ErrorUnconditional PMSE

Unconditional PMSE can be usefull to assess the ’success’ of aforecasting method on average. This apporach if used tocompute P.I.s, it effectively assumes that te observatoins usedto estimate the model parameters are independent of thoseused to construct the forecasts. This assumption can bejustified asymptotically. Box assesed that the correction termswould generally be of order 1

N (effect of parameter uncertainty).




Summary


Parameters UncertaintyCorrection

Assume: K-variable vector AR(p) process.True model PMSE at lead time one has to be multiplied by thecorrection factor

[1 + K 1N ] + o( 1

N )

to give the corresponding unconditional PMSE allowing forparameters uncertainty.What does it mean?

the more parameters (K ),the shorter the series (N),

⇒ the greater will be the correction term.




Summary


Parameters UncertaintyExample

N = 50, K = 1 and p = 2

the correction for the square root of PMSE is only 2%. for

N = 30, K = 3 and p = 2

the correction for the square root of PMSE rises to 6%.




Summary


Parameters UncertaintyExample

The effect on probabilities.For normal distribution

95% lie in the interval +/− 1.96.

Suppose the s.d. is 6% larger (no correction factor used).Now,

96.2% lie in the interval +/− (1.96× 1.06) = 2.0776.




Summary

Calculating P.I.sProbability ModelNon Formal ModelExample

Calculating P.I.

In general P.I.s are of the form:

100(1− α)%.

P.I. for XN+h is given by:

x̂N(h) + /− zα/2√

Var [eN(h)].




Summary


Formula for P.I.

Properties and assumptions of the formula for P.I.:symmetric about x̂N(h),assumes the forecast is unbiased with PMSEE [eN(h)2] = Var [eN(h)],forecast errors are assumed to be normally distributed.

Note: some authors state that the zα/2 should be replaced bythe precentage point of a t-distribution, with appropriate numberof degrees of freedom (worth making for less than 20 obs).




Summary


Formula for P.I.

The formula

x̂N(h) + /− zα/2√

Var [eN(h)].

is generally used for P.I.s. preferably after checking theassumptions (e.g. forecast errors are approximately normallydistributed) are at least reasonably satisfied. For any givenforecasting method the main problem will then lie withevaluating Var [eN(h)].




Summary


P.I.s derived from a fitted probability modelFormulas for PMSE

PMSE can be derived for:ARMA models (also seasonal and integrated),structural state-space,various regression models (typically allow for parameteruncertainty and are conditional in the sense that theydepend on the particular values of the explanatoryvarialbes from where a prediction is being made).

Cannot be derived:some complicated simultaneous equation,non-linear.




Summary


P.I.s without model identification

What to do when a forecasting method is selected without anyformal model identification procedure ?

assume that the method is optimal (in some sense)apply some empirical procedure




Summary


P.I.s when assumed that method is optimalExample

Exponential smoothing:no obvious trend or seasonality,no attempt to identify the underlying model(i.e. ARIMA(0,1,1)).

PMSE formula:

Var [eN(h)] = [1 + (h − 1)α2]σ2ε .




Summary


P.I.s when assumed that method is optimalWhen it is reasonable ?

When it is reasonable?Observed one-step-ahead forecast errors show no obviousautocorrelation.No other obvious features of the data (e.g. trend) whichneed to be modeled.




Summary


Forecasting methods not based on a probability model

Assume that the method is optimal in the sense that theone-step ahead errors are uncorrelated.Easy to check by looking at the correlogram of theone-step-ahead errors:

if there is correlation we have more structure in the datawhich should improve the forecast.




Summary


Forecasting methods not based on a probability modelExample

Holt-Winters method with

additive and multiplicative seasonality




Summary


The additive case

Properties:results equivalent to SARIMA model for which additiveHolt-Winters is optimal,so complicated that it would never be identified in practice.




Summary


The multiplicative case

No ARIMA model for which the method is optimal.

Assume: one-step-ahead forecast errors are uncorrelated.Results:

Var [eN(h)] not monotonic increase with h,P.I.s are wider near a seasonal peak as would intuitively beexpected.

Remark: Wider P.I.s near a seasonal peak - not captured bymost alternative approaches (except variance-stabilizingtransformation).




Summary

When to use?1st method2nd methodSimulation and Resampling

Empirically based P.I.s

When to use empirically based P.I.s?theoretical formulae not available,doubts about validity of the true model.

Remark: computationally intensive based on:observed distribution of errors,simulation or resample methods.




Summary


Empirically based P.I.s1st method

Apply forecasting method to all the past data.Find within-sample ’forecasts’ at 1, 2, 3, . . . steps ahead(from all available time origins).Find the variance of these errors (at each lead time overthe periods of fit).Assume normality.




Summary


Empirically based P.I.s1st method

Result:approximate empirical 100(1− α)% P.I. for XN+h is given by

x̂N(h) + /− zα/2√

Var [eN(h)].

Problems:if N small - assume t-distribution,long series is needed to get reliable values for se,h,smooth values to make them increase monotonically with hvalues of

√Var [eN(h)] based on in-sample residuals not

on out-of-sample forecast errors,results comparable to theoretical formulae (if available).




Summary


Empirically based P.I.s2nd method

Split data into two parts.Fit on 1st part.Forecast on 2nd part.Get prediction errors.Refit model - move the fitting window.




Summary


Simulation and resampling methods

More computationally intensive approach.Increasingly used for the construction of P.I.s.




Summary


Simulation and resampling methodsSimulation (Monte Carlo approach)

Assumption:Probability time-series model is known (and identifiedcorrectly)Generate random innovationsGenerate possible past and future valuesRepeat many timesFind the interval within which the required percentage offuture values lie




Summary


Simulation and resampling methodsResampling (bootstrapping)

Sample from the empirical not theoretical distribution

⇒ distribution-free approach

The idea (the same as for simulation):use the knowledge about the primary structure of themodelgenerate a sequence of possible future valuesfind a P.I. containing the appropriate percentage of futurevalues




Summary


BootstrappingBrief description

N independent observations,take random sample of size N with replacement.

Result:some values occur twice (or more) some not occur at allIn time-series:makes no sense - correlation over time

Bootstrap by resampling the fitted errors - depend on model.




Summary


BootstrappingProperties

Properties of bootstrap:Bootstrap P.I.s are useful non-parametric alternative to theusual Box-Jenkins intervals.It is difficult to resample correlated data.




Summary


Uncertainty in Forecasts

Sources of uncertainty in forecasts from econometric models:the model innovations,having estimates of model parameters rather than truevalues,having forecasts of exogenous variables rather than truevalues,misspecification of the model.




Summary

SummaryFindings and Recomendations

Summarized main findings and recommendations:Formulate a model, that provides a reasonable apporx forthe process generating a given series, derive PMSE, anduse the formula.Distinction between a forecasting method and a forecastingmodel should be borne in mind. The former may, or maynot, depend (explicitly or implicitly) on the latter.Use not model but method based approach (e.g. theHolt-Winters method).




Summary

SummaryFindings and Recomendations

No theoretical formulae, or there are doubts about modelassumptions, use the empirically based approach.The reason why out-of-sample forecasting ability is worsethan within-sample fit is that the wrong model may havebeen identified or may change through time.The formula x̂N(h) + /− zα/2

√Var [eN(h)] assumes:

model has been correctly identified,innovations are normally distributed,the future will be like the past.

Rather than compute P.I.s based on a single ’best’ model,use Bayesian model averaging, or not model-basedapproach.




Summary

The End

Thank you for yourattention.


interval forecasting - zentraler informatikdienst...

Documents