forecasting
DESCRIPTION
Forecasting can enable better data-driven decisions. This presentation explores the spectrum of forecasting techniques, including scenario construction and powerful-yet-approachable quantitative methods. See how to match appropriate techniques to decision-support needs and then implement them in ubiquitous productivity software. Learn effective strategies for visualizing and communicating forecast outcomes, uncertainty, and sensitivity. Jeff details his forecasting experience at the Medical School. Examples include financial forecasts informed by operational data and scenario analysis.TRANSCRIPT
ForecastingJeff Horon
25 January 2011
About me [& forecasting]BA Econ / Honors thesis in petroleum price
forecastingMBA [Winter 2011] / Emphases in Finance &
Strategy; Formal training in decision supportSr. Analyst / Medical SchoolDesign responsibility for econometric and
financial modeling describing the Medical School’s $0.5B research enterprise [Ad hoc and standardized reporting, surveys, dashboards]
Fore ▪ castCasten Fore-
“Contrive” “Before [the fact]”
We forecast all the timeCool Kids
Personal Space
Studious
Achievers
You’re doing it right now
Not so bad
Exit Strategy
Why forecast?Decision Support! Decision Support! D-e-c-i-s-i-o-n
S-u-p-p-o-r-t!
Backward-looking: “How did we do?”(it pays to correct your mistakes)
In the present: “How are we doing?”(it pays to not make the mistakes in the first place)
Forward-looking: “Are we headed in the right direction?”
(it pays to be proactive, consistent withreasonable expectations)
Continuum of methodsQualitative --> Quantitative
Subjective Objective
‘Gut feeling’
Casual observation
Extrapolation
Decision trees /Scenario construction
Prediction IntervalsMonte Carlo
Cost [Method]
Cost ~ Complexity ~ Time investment(Skills, effort devoted to creation, maintenance, delivery)
Objectivity
Cost [Scale]
Cost ~ Resources ~ Time and/or capital investment(Skills, effort, capital devoted to implementation and maintenance)
Scale
Expected Value of Information
Expected Value of Information ~ Quality ~ Scale of Decision
Objectivity; Scale
Decision Framework
Marginal analysis: Target Expected Value of Information = Cost of Information
Objectivity; Scale
EV (Info)Cost
Practical Decision FrameworkHigh Impact
-High per-unit stakes-High volume
Repeated
Low Impact-Low per-unit stakes-Low volume
Not Repeated
High EV (Info)
Low EV (Info)
Practical Implementation
Objectivity; Scale
EV (Info)Cost
EV (Info)
Cost
‘Back of Envelope’‘Sketching’‘Low-Fi Prototyping’
WorkflowSTART Identify unmet decision support need
Create
Improve
Match method to need
Is it feasible?
Share
Sketch PrototypeIs it practical?
‘Sell idea’
‘Gut check’ results
Standardize
Build into existing reporting
Method – Gut feelingSubjective
“Well, we usually fall within… ”“I’ve got a good feeling about… ”
Based upon experience
Useful for a check if you are ‘in the ballpark’
Method – Casual observationSubjective
“If we stay on the same growth trajectory as the past few years… ”
Based upon experience and data
However, as our minds rush to think ahead…
Often this is fine…
…. but sometimes it isn’t…
Analogous to extrapolation outside the ‘relevant range’
Method – ExtrapolationLess subjective; ‘Baked-in’ assumptions
Based upon historical data
Method – ExtrapolationExtremely easy to implement in Excel
=FORECAST(); =TREND(); =GROWTH() orRight-click graphed data series, ‘Add Trendline’
Every investment prospectus: “Past performance does not guarantee future results”
Method – Decision treePotential for ‘guesstimation’ in the absence of
historical data
Typically based upon historical data
Method for calculating an expected value across multiple possible outcomes; Branches can be decisions or random events
Example – Decision tree
EV = -$12k
Invest?
Decision Random Event
Meets specs?
Result
No savings
EV = $0
High Savings
Low SavingsEV = $10k
EV = $20k
Yes
No
Yes
No
Key:
50%
50%
Example – Decision tree
EV = -$12k
Invest?
Decision Random Event
Meets specs?
Result
No savings
EV = $0
High Savings
Low SavingsEV = $10k
EV = $20k
Yes
No
Yes
No
Key:
50%
50%
EV = +$3k Invest = Yes
Method – Scenario constructionSome room for subjectivity in assumptions;
Helpful to jog memory regarding important variables, events, etc.
Based upon historical observations or future expectations
Flexible approach depending on decision support need, because you create the scenario
Use case – Effects of legislationSimilar to a marketing ‘conversion rate’ calculation
NIH budgeted extra $10B under ARRA
ARRA dictates internal fund distribution similar to ‘regular appropriation’ funds
U-M Med School tends to attain ‘market share’ of 1% of ‘regular appropriation’ funds
ARRA sets aside $1B for medical school facilities
$1B for medical school facilities $9B proportionally budgetedU-M Med School tends to attain ‘market share’ of 2.7%
of ‘regular appropriation’ funds to medical schools
x 2.7% = $27M x 1% = $90M
Use case – Effects of legislationSimilar to a marketing ‘conversion rate’ calculation
NIH budgeted extra $10B under ARRA
ARRA dictates internal fund distribution similar to ‘regular appropriation’ funds
U-M Med School tends to attain ‘market share’ of 1% of ‘regular appropriation’ funds
ARRA sets aside $1B for medical school facilities
$1B for medical school facilities $9B proportionally budgetedU-M Med School tends to attain ‘market share’ of 2.7%
of ‘regular appropriation’ funds to medical schools
x 2.7% = $27M x 1% = $90M
Proposals submitted, not funded
$82M
Noted sensitivity to market share %
Use case – Revenue projection
Awards
Fiscal Year
Use case – Revenue projection
Awards
Fiscal Year
Use case – Revenue projection
Awards ($)
Fiscal YearCurrent FY
Use case – Revenue projection
Awards
Fiscal Year
Proposals
Use case – Revenue projection
Awards
Fiscal Year
Proposals
Use case – Revenue projection
Awards ($)
Fiscal YearCurrent FY
Use case – Revenue projection
Awards ($)
Fiscal YearCurrent FY
Method – Prediction intervalsFor unknown population mean and variance, the endpoints of a 100p% prediction interval for
Xn + 1 are:
Sample mean Sample standard deviation
Observations
100((1 + p)/2)th percentile of Student's t-distribution with n − 1 degrees of freedom
Method – Prediction intervals
Sample mean
Upper Endpoint
Lower Endpoint
Method – Monte Carlo simulationUse random sampling to work around difficult or
impossible deterministic problems
Variable 1 Variable 2 Variable 3 Result
Best Practices‘Gut check’ (Expectations ~ Results?)Litmus testSensitivity analysis
Adjust for inflation
CommunicationAlways communicate uncertainty, particularly
sensitive outcomes
Source: CBO http://www.cbo.gov/ftpdocs/100xx/doc10014/03-20-PresidentBudget.pdf p34
