forecasting and model selection - iitk - indian institute of
TRANSCRIPT
![Page 1: FORECASTING AND MODEL SELECTION - IITK - Indian Institute of](https://reader036.vdocuments.mx/reader036/viewer/2022071601/613d3f40736caf36b75b10d3/html5/thumbnails/1.jpg)
Modeling and ForecastingForecasting Methods
Measuring the Accuracy of Forecast
FORECASTING AND MODEL SELECTION
Anurag Prasad
Department of Mathematics and StatisticsIndian Institute of Technology Kanpur, India
REACH Symposium, March 15-18, 2008
1 Forecasting and Model Selection
![Page 2: FORECASTING AND MODEL SELECTION - IITK - Indian Institute of](https://reader036.vdocuments.mx/reader036/viewer/2022071601/613d3f40736caf36b75b10d3/html5/thumbnails/2.jpg)
Modeling and ForecastingForecasting Methods
Measuring the Accuracy of Forecast
Outline
1 Modeling and Forecasting
2 Forecasting Methods
3 Measuring the Accuracy of Forecast
2 Forecasting and Model Selection
![Page 3: FORECASTING AND MODEL SELECTION - IITK - Indian Institute of](https://reader036.vdocuments.mx/reader036/viewer/2022071601/613d3f40736caf36b75b10d3/html5/thumbnails/3.jpg)
Modeling and ForecastingForecasting Methods
Measuring the Accuracy of Forecast
Assumptions of Forecasting
1 Element of Uncertainty
2 Blind Spots
3 Change in Forecast Accuracy
3 Forecasting and Model Selection
![Page 4: FORECASTING AND MODEL SELECTION - IITK - Indian Institute of](https://reader036.vdocuments.mx/reader036/viewer/2022071601/613d3f40736caf36b75b10d3/html5/thumbnails/4.jpg)
Modeling and ForecastingForecasting Methods
Measuring the Accuracy of Forecast
Framework of a Forecast System
No
Historical Data
Data
SpecificationModel
Checking
Diagnostic Estimation
Model ForecastGeneration
StabilityChecking
ForecastUpdation
NewObservations
Yes Yes
Model−Building Phase Forecasting Phase
No
Theory and/or
4 Forecasting and Model Selection
![Page 5: FORECASTING AND MODEL SELECTION - IITK - Indian Institute of](https://reader036.vdocuments.mx/reader036/viewer/2022071601/613d3f40736caf36b75b10d3/html5/thumbnails/5.jpg)
Modeling and ForecastingForecasting Methods
Measuring the Accuracy of Forecast
Choice of a Particular Forecast Model
1 Degree of Accuracy Required
2 Cost of Producing Forecasts
3 Forecast Horizon
4 Degree of Complexity Required
5 Available Data
5 Forecasting and Model Selection
![Page 6: FORECASTING AND MODEL SELECTION - IITK - Indian Institute of](https://reader036.vdocuments.mx/reader036/viewer/2022071601/613d3f40736caf36b75b10d3/html5/thumbnails/6.jpg)
Modeling and ForecastingForecasting Methods
Measuring the Accuracy of Forecast
Classification of Estimation Methods
1 Time Series Methods
2 Causal Methods
3 Judgemental Methods
6 Forecasting and Model Selection
![Page 7: FORECASTING AND MODEL SELECTION - IITK - Indian Institute of](https://reader036.vdocuments.mx/reader036/viewer/2022071601/613d3f40736caf36b75b10d3/html5/thumbnails/7.jpg)
Modeling and ForecastingForecasting Methods
Measuring the Accuracy of Forecast
Time Series Methods
• Use historical data as a basis• Underlying patterns are fairly stable
1 Autoregressive Moving Average (ARMA)2 Exponential Smoothing3 Extrapolation4 Linear Prediction5 Trend Estimation6 Growth Curve7 Box-Jenkins Approach
7 Forecasting and Model Selection
![Page 8: FORECASTING AND MODEL SELECTION - IITK - Indian Institute of](https://reader036.vdocuments.mx/reader036/viewer/2022071601/613d3f40736caf36b75b10d3/html5/thumbnails/8.jpg)
Modeling and ForecastingForecasting Methods
Measuring the Accuracy of Forecast
Causal Methods
• Belief that some other time series can be useful• Assumption that it is possible to identify the underlying factors
1 Regression Analysis* Linear Regression* Non-Linear Regression
2 Econometrics
8 Forecasting and Model Selection
![Page 9: FORECASTING AND MODEL SELECTION - IITK - Indian Institute of](https://reader036.vdocuments.mx/reader036/viewer/2022071601/613d3f40736caf36b75b10d3/html5/thumbnails/9.jpg)
Modeling and ForecastingForecasting Methods
Measuring the Accuracy of Forecast
Judgemental Methods
• Incorporate intuitive judgements, opinions and probabilityestimates
1 Composite Forecasts2 Surveys3 Delphi Method4 Scenario Building5 Technology Forecasting6 Forecast by Analogy
9 Forecasting and Model Selection
![Page 10: FORECASTING AND MODEL SELECTION - IITK - Indian Institute of](https://reader036.vdocuments.mx/reader036/viewer/2022071601/613d3f40736caf36b75b10d3/html5/thumbnails/10.jpg)
Modeling and ForecastingForecasting Methods
Measuring the Accuracy of Forecast
Forecast Error
For t = 1, . . . , N,y(t) : Actual value at period t ,y(t) : Forecast value at period t ;e(t) : Forecast error at period t ; e(t) = y(t) − y(t)
i
y(t )i
t it
y(t)
y(t )
^
10 Forecasting and Model Selection
![Page 11: FORECASTING AND MODEL SELECTION - IITK - Indian Institute of](https://reader036.vdocuments.mx/reader036/viewer/2022071601/613d3f40736caf36b75b10d3/html5/thumbnails/11.jpg)
Modeling and ForecastingForecasting Methods
Measuring the Accuracy of Forecast
Graphical Measures of Forecast Accuracy
Plot of y(t) versus y(t)
• Keep the same scale for both the axes.• Departure of points from the 450 line through origin indicatesimperfect forecasts.
11 Forecasting and Model Selection
![Page 12: FORECASTING AND MODEL SELECTION - IITK - Indian Institute of](https://reader036.vdocuments.mx/reader036/viewer/2022071601/613d3f40736caf36b75b10d3/html5/thumbnails/12.jpg)
Modeling and ForecastingForecasting Methods
Measuring the Accuracy of Forecast
Correct Model Form
y(t)
y(t)Incorrect Model Form
y(t)
y(t)
12 Forecasting and Model Selection
![Page 13: FORECASTING AND MODEL SELECTION - IITK - Indian Institute of](https://reader036.vdocuments.mx/reader036/viewer/2022071601/613d3f40736caf36b75b10d3/html5/thumbnails/13.jpg)
Modeling and ForecastingForecasting Methods
Measuring the Accuracy of Forecast
Plot of e(t) versus t
• Reveals patterns of variability which the model has failed toexplain.• For a good model, the forecast errors should vary in ahorizontal band around zero.
13 Forecasting and Model Selection
![Page 14: FORECASTING AND MODEL SELECTION - IITK - Indian Institute of](https://reader036.vdocuments.mx/reader036/viewer/2022071601/613d3f40736caf36b75b10d3/html5/thumbnails/14.jpg)
Modeling and ForecastingForecasting Methods
Measuring the Accuracy of Forecast
0
tCorrect Model Form
0
tIncorrect Model Form
0
tIncorrect Model Form
0
e(t) e(t)
e(t) e(t)
tIncorrect Model Form
14 Forecasting and Model Selection
![Page 15: FORECASTING AND MODEL SELECTION - IITK - Indian Institute of](https://reader036.vdocuments.mx/reader036/viewer/2022071601/613d3f40736caf36b75b10d3/html5/thumbnails/15.jpg)
Modeling and ForecastingForecasting Methods
Measuring the Accuracy of Forecast
Descriptive Measures of Forecast Accuracy
Descriptive Measures of Forecast Accuracy are used to...
1 Provide a single, easily interpreted measure of model’sreliability
2 Compare the accuracy of two different models3 Search for an optimal model4 Monitor a model’s performance
15 Forecasting and Model Selection
![Page 16: FORECASTING AND MODEL SELECTION - IITK - Indian Institute of](https://reader036.vdocuments.mx/reader036/viewer/2022071601/613d3f40736caf36b75b10d3/html5/thumbnails/16.jpg)
Modeling and ForecastingForecasting Methods
Measuring the Accuracy of Forecast
Descriptive Measures of Forecast Accuracy
Mean Absolute Deviation (MAD)
MAD =�
|forecast error|number of forecasts =
� Nt=1 |e(t)|
N
Mean Square Error (MSE)
MSE =�
(forecast error)2
number of forecasts =� N
t=1 e(t)2
N
Root Mean Square Error (RMSE)
RMSE =√
MSE
16 Forecasting and Model Selection
![Page 17: FORECASTING AND MODEL SELECTION - IITK - Indian Institute of](https://reader036.vdocuments.mx/reader036/viewer/2022071601/613d3f40736caf36b75b10d3/html5/thumbnails/17.jpg)
Modeling and ForecastingForecasting Methods
Measuring the Accuracy of Forecast
Descriptive Measures of Forecast Accuracy
Mean Absolute Percent Error (MAPE)
MAPE =�
|forecast error/actual value|number of forecasts .100%
=� N
t=1 |e(t)/y(t)|N .100%
Pearson’s Correlation Coefficient (r )between y(t) and y(t)
r =� N
t=1(y(t)−y)( �y(t)−¯�y)� � Nt=1(y(t)−y)2
� � Nt=1( �y(t)−¯�y)2
17 Forecasting and Model Selection
![Page 18: FORECASTING AND MODEL SELECTION - IITK - Indian Institute of](https://reader036.vdocuments.mx/reader036/viewer/2022071601/613d3f40736caf36b75b10d3/html5/thumbnails/18.jpg)
Modeling and ForecastingForecasting Methods
Measuring the Accuracy of Forecast
Descriptive Measures of Forecast Accuracy
"No Change" model is : y(t + 1) = y(t)
Theil’s Inequality Coefficient (U)
U = RMSE("new" model)RMSE("no change" model)
U > 1 ⇒ worse than "no change" model
U = 1 ⇒ as good as "no change" model
U < 1 ⇒ better than "no change" model
18 Forecasting and Model Selection
![Page 19: FORECASTING AND MODEL SELECTION - IITK - Indian Institute of](https://reader036.vdocuments.mx/reader036/viewer/2022071601/613d3f40736caf36b75b10d3/html5/thumbnails/19.jpg)
Modeling and ForecastingForecasting Methods
Measuring the Accuracy of Forecast
Further Readings
Quantitative Forecasting Methods, N.R. Farnum and L.W.Stanton, 1989, PWS-KENT Publishing Co.
Statistical Methods for Forecasting, B. Abraham and J.Ledolter, 1983, John Wiley & Sons
Introduction to Time Series and Forecasting, P.J. Brockwelland R.A. Davis, 2002 (Second Edition) , Springer-Verlag
Time Series Analysis and Forecasting, R. Yaffee, 2000,Academic Press
19 Forecasting and Model Selection
![Page 20: FORECASTING AND MODEL SELECTION - IITK - Indian Institute of](https://reader036.vdocuments.mx/reader036/viewer/2022071601/613d3f40736caf36b75b10d3/html5/thumbnails/20.jpg)
Modeling and ForecastingForecasting Methods
Measuring the Accuracy of Forecast
Thank You
20 Forecasting and Model Selection