the art and science of forecasting financial markets

18
HEADER DATE The Art and Science of Forecasting Financial Markets Ben Jacobsen [email protected] Outline Part 1: Market Efficiency and unpredictability as a benchmark Part 2: Empirical examples of forecastability Part 3: Building quantitative forecasting models 2 Month t t-1 t-2 t-3 Future stock return The Basic Problem Information in the past Academic point of view Market efficiency: prices reflect the available information Consequence: (changes in) stock market prices follow a random walk and are unpredictable The study that led to the EMH Sir Maurice Kendall (1953) seminal study on prices in financial markets: “The series looks like a “wandering” one as if once a week the Demon of Chance drew a random number (...) and added it to the current to determine next week’s price.” “But economists - and I cannot help sympathizing with them - will doubtless resist any such conclusion very strongly. We can at this point suggest only a few conclusions: (a) the interval of observation may be very important. (b) it seems a waste of time to isolate a trend in data such as these; (c) the best estimate of the change in price between now and next week is that there is no change.” Unpredictable? Random? How well does that assumption work?

Upload: aifmrmandacqufrrpresentations

Post on 10-Aug-2015

74 views

Category:

Economy & Finance


0 download

TRANSCRIPT

Page 1: The Art and Science of Forecasting Financial Markets

HEADER DATE

The Art and Science

of

Forecasting Financial Markets

Ben Jacobsen

[email protected]

Outline

Part 1: Market Efficiency and unpredictability as a benchmark

Part 2: Empirical examples of forecastability

Part 3: Building quantitative forecasting models

2

Month t t-1 t-2 t-3

Future stock return

The Basic Problem

Information in the past

Academic point of view

Market efficiency: prices reflect the available information

Consequence: (changes in) stock market prices follow a random walk and are unpredictable

The study that led to the EMH

Sir Maurice Kendall (1953) seminal study on prices in financial markets:

“The series looks like a “wandering” one as if once a week the Demon of Chance drew a random number (...) and added it to the current to determine next week’s price.”

“But economists - and I cannot help sympathizing with them - will doubtless resist any such conclusion very strongly. We can at this point suggest only a few conclusions:

(a) the interval of observation may be very important.(b) it seems a waste of time to isolate a trend in data

such as these;(c) the best estimate of the change in price between

now and next week is that there is no change.”

Unpredictable? Random?

How well does that assumption work?

Page 2: The Art and Science of Forecasting Financial Markets

HEADER DATE

Virtual and Reality Virtual and Reality

We tend to see structure in randomness…

Head and Shoulder

Traditional Approaches

9

• Technical Analysis: • Looking for patterns in historical price information to predict the future

• Fundamental Analysis:• Studying macro economic, sector and company information to derive

a valuation

• Alternative approaches…..

Very fundamental analysis…

The Hemline Indicator Market efficiency may explain a lot

Page 3: The Art and Science of Forecasting Financial Markets

HEADER DATE

Mutual Fund Performance Who is the better investor?

Wall Street Dartboard Competition

16

Wall Street Dart Board Competition Results

30%

40%

50%

60%

70%

80%

90%

100%

1 3 5 7 911 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 43 45 47 49 51 53 55 57 59 61 63 65 67 69 71 73 75 77 79 81 83 85 87 89 91 93 95 97 99

vs Monkey

vs Market

Percentage of contests when analysts beat Monkey and Market (DJI)

Venus versus Mars

Women get higher returns 1) Women are better investors as they tend to trade less

2) This may be caused men being more overconfident

3) Differences are smaller in relationships

(Barber and Odean, “Boys will be Boys”)

Women invest less in stocks 4) Maybe caused by difference in risk tolerance but not likely

5) Women are less optimistic about the economy

6) Women tend to be less optimistic about the future in general

7) Women tend to perceive the stock market as being more risky

(Jacobsen, Lee, Marquering, Zhang, “Gender Differences in Optimismand Asset Allocation”)

Buy a well diversified portfolio of some stocks

If markets are efficient……..

Trade under no circumstances

Avoid any news that might create stimuli

Page 4: The Art and Science of Forecasting Financial Markets

HEADER DATE

• 41% of all net fund flows worldwide went into passive investments in 2012• 18% of total mutual fund and ETF assets worldwide are now passively invested• Passive investing grew at three times the rate of active investing, 9.8% to 3.2%

Source Morningstar

Passive Investing Early 1990’s

1) Assuming market efficiency/rational investors explains a lot- Poor performance of technical and fundamental analysis

2) But some anomalies:- A January effect

- A small firm effect

- Overreaction effects: Winners and Losers.

3) Academics did not like irrationality: degrees of freedom

4) Market efficiency was often not well understood.- Do we need rational investors?

- What if everyone believes markets are efficient?

- What about information?

- How can prices go up?

Market efficiency in detail

E Pt | It1 Pt1

But can this be true?

No! risk and return: nobody would invest:Prices should go up on average

Conditional on all information the best forecast of tomorrows price is today’s price

Pt-1

Pt

Pt Pt1 1

rt Pt Pt1

Pt1

Pt1 1 Pt1

Pt1

Note

rf rm rf time

Prices ‘must’ go up: risk and return

Random walk model

Price already reflect all relevant information.

What is the best forecast of tomorrows stock price given all available information today?

As consequence stock market returns follow a random walk:

t t t tr E r 1[ ]

rt t

Random Walk model

E rt | It1 Note: we assume μ is constant but it mightBe time-varying.

Et1 rt or

Thus if markets are efficient we should not be ableto come with a better forecast given all availableinformation.

How to test this. Note we have a model withExpectations which are difficult to observe.

Page 5: The Art and Science of Forecasting Financial Markets

HEADER DATE

A testable model

rt t

t t t tr E r 1[ ]

Et1 rt Add on rt both sides

rt Et1 rt rtrewrite

rt rt Et1 rt Define:

t t t tr E r 1[ ]

And we have: the random walk model

with

Pt-1

Pt

time

Random deviations caused by unpredictable newsWhich affects value over time

The if’s of market efficiency

If investors are rational and if information is freely available andreaches all investors at the same time

Then

Markets are informationally efficient: prices will refelect all availableinformation and price will equal value

Then

You can trust market prices

Market Efficiency: Rationality

Note that not all investors need to be rational for markets to beefficient.

Markets can be informationally efficient even if not all investorsbehave rational. As long as they do not behave systematicallyirrational.

Systematic irrationality: sentiments or human traits likeoverreaction. This is what behavioral finance studies

Pt-1

Pt

time

Systematic deviations caused by…..

Note: Now the red line is value

Systematic Irrationality Market Efficiency: Information

Note that if information is costly

Or if not all information reaches all investors at the same time

Or, not all investors use the same information

Or, investors have limited information processing capability

Or, if there information is ambiguity

Markets may still be inefficient (partially predicatble)

We call this ‘bounded’ rationality

Page 6: The Art and Science of Forecasting Financial Markets

HEADER DATE

Testing Methodology

rt 1Itn t

t t t tr E r 1[ ]with

rt tRandom walk

Predictability

Past returns: rt 1rt1 t

Oil price changes: rt 1rt1oil t

Note: past returns are observable to investors at t-1

Note: past oil price changes are observable to investors at t-1

rt 1Jant tJanuary effect

Note: we know at time t-1 that t is January

Some Examples

Why regression?:

1) Simple and easy2) Well known properties of estimators3) Allows for control variables4) Allows for heteroscedasticity adjustments and autocorrelation in

the error terms (White s.e./Newey White s.e.): few distributional assumptions

5) Easy to communicate: broader audience6) But beware there are problems: unit roots etc.

rt 1Jant 2rt1 t

Predictable returns….

Over the years anomalies have popped up:

We very often do not know what causes them…..

- Investors being systematically irrational

- Information gradually diffusing

- Data mining, Spurious, Coincidence

- Risk related. Even if we do not know yet how (value and growth)

- Time varying risk

- Frictions (transactions costs or something else)

“It may be a bit more complicated than that” Ben Goldacre

Three Examples

Seasonal Anomalies

Gradual information diffusion

Time varying return predictability

35

The Halloween effectSell in May or go away

The Halloween indicator is a variant of the stock market adage "Sell in May and go away,” the belief that the period from November to April inclusive has significantly stronger growth on average than the other months.

(Source: Wikipedia)

With Sven Bouman; American Economic Review, 2002

Page 7: The Art and Science of Forecasting Financial Markets

HEADER DATE

Month t t-1 t-2 t-3

Future stock return

The Basic Problem

Which month is next?Hon

g Kon

g

South

Afri

ca

Denm

ark

US

Austra

lia

Norway

Sweden

Switzer

land

Canad

a

Nethe

rland

sUK

Spain

Germ

any

Belgium

Japa

n

Austri

a

Franc

e

Singap

ore

Italy

Summer

-4

-2

0

2

4

6

8

10

12

14

16

Returns Figure 1. Average Returns in May-Oct. ('Summer') and Nov.-April ('Winter') in several countries. MSCI re-investment indices 1970-August 1998.

SummerWinter

Sell in May (1970-1998)

Source: Bouman and Jacobsen, American Economic Review, 2002

Test: Methodology

rt 1St t

t t t tr E r 1[ ]

with

Statistical Significance

Halloween strategy in Italy 1970‐1998 Sell in May (1998-2007)

Belgiu

m

South

Afr

ica

Austr

alia

Denm

ark

Cana

da

Singa

pore

Hong

Ko

ng

Norwa

y US

France

Switze

rland UK

Spain

Austr

ia

Swed

en Italy

Japa

n

Nethe

rland

s

Germ

any

Summer

-6%

-4%

-2%

0%

2%

4%

6%

8%

10%

12%

14%

16%returns

country

Winter

Source: Jacobsen and Visaltanachoti, The Financial Review, 2009

Page 8: The Art and Science of Forecasting Financial Markets

HEADER DATE

-15.00

-10.00

-5.00

0.00

5.00

10.00

15.00

20.00

UA

EB

ulga

ria

Mau

ritiu

sS

ri L

anka

Kuw

ait

Cyp

rus

HK

Om

anC

hile

Qat

arP

eru

Aus

tral

iaS

love

nia

Tha

iland US

Indi

aD

enm

ark

Mex

ico

Ven

ezue

laL

atvi

aF

inla

ndS

outh

Afr

ica

UK

Phi

lippi

nes

Can

ada

Japa

nC

hina

Col

ombi

aB

ahra

inN

ZS

wed

enF

ranc

eS

wit

zerl

and

Spa

inN

ethe

rlan

dsIt

aly

Mor

occo

Kor

eaC

zech

Rep

ubli

cN

orw

ayM

alay

sia

Bel

gium

Gre

ece

Mal

taJo

rdan

Pak

ista

nL

ithua

nia

Isra

elS

inga

pore

Arg

entin

aE

gypt

Hun

gary

Ger

man

y

Aus

tria

Lux

embo

urg

Por

tuga

l

Irel

and

Indo

nesi

a

Est

onia

Tur

key

Tai

wan

Rus

sia

Pol

and

Icel

and

Rom

ania

Ret

urn

(%)

Country

Excess Returns in Summer and WinterExcess Returns in 65 Stock MarketsDuring May-October

-35

-30

-25

-20

-15

-10

-5

0

5

10

15

0 10 20 30 40 50 60 70

Me

an

Ex

ce

ss

Re

turn

Standard deviation

Why?

Vacations

Seasonal AffectiveDisorder: SAD

Extreme Temperature

Airline travel

-4.92

-3.28

-1.64

0

1.64

Sw

eden

Phi

lippi

nes

Net

herla

nds UK

Spa

inA

rgen

tina

Bra

zil

Bel

gium

C

hile

Ita

lyP

olan

dE

gypt

Mor

occo

Fra

nce

Jord

anG

erm

any

Aus

tral

iaA

ustr

ia US

Japa

nM

alay

sia

Sw

itzer

land

Hun

gary

Indi

aT

urke

yIr

elan

d

Mex

ico

Por

tuga

lC

olom

bia

Thai

land

Indo

nesi

aS

inga

pore

Pak

ista

nG

reec

eH

ong

Kon

gC

anad

aS

outh

Afr

ica

Den

mar

kV

enez

uela

Finl

and

Rus

sia

Nor

way

New

Zea

land

Cze

ch r

ep.

Kor

eaS

ri La

nka

Isra

elC

hina

Country

Correlation of seasonal variables Market Efficiency: Information

Note that if information is costly

Or if not all information reaches all investors at the same time

Or, not all investors use the same information

Or, investors have limited information processing capability

Or, if there information is ambiguity

Markets may still be inefficient (partially predicatble)

We call this ‘bounded’ rationality

Page 9: The Art and Science of Forecasting Financial Markets

HEADER DATE

Recent insights from theory

Gradual Information Diffusion:

– Hong and Stein (Journal of Finance, 1999)

Limited Attention of Investors

Available important information

Overlooked by investors

Intuition

50

Oil Shares Bank Shares

Oil Price Interest rate

Month t t-1 t-2 t-3

Future Small stock return

The Basic Problem

Information in the past: How large firms have performed

Lo and MacKinlay (1980)

Small follows big

52

rtSmall rt1

Big t

Callaway & Coastcast example Economic Links

Source: Cohen and Frazzini, Journal of Finance, 2008

Page 10: The Art and Science of Forecasting Financial Markets

HEADER DATE

Month t t-1 t-2 t-3

Future stock return:Coastcast

The Basic Problem

Information in the past:Callaway Golf Corporation

Gradual Information Diffusion: US Market leading other markets

56

0.0% 5.0% 10.0% 15.0% 20.0% 25.0% 30.0% 35.0%

Australia

Canada

France

Germany

Italy

Japan

Netherlands

Sweden

Switzerland

UK

Average

US Effect in other markets

Based on International Stock Return Predictability: What is the Role of the United States?Rapach, Strauss and Zhou, Journal of Finance,

Month t t-1 t-2 t-3

Future international market return:

The Basic Problem

Information in the past:US Market return

Month t t-1 t-2 t-3

Gradual Information Diffusion: Oil

Oil Prices and Stock Returns

With Benjamin Maat and Gerben Driesprong

Journal of Financial Economics, 2008

0

5

10

15

20

25

30

35

40

45

D a t e

The ultimate market efficiency test

“Oil is so significant in the international economy that forecasts of economic growth are routinely qualified with 

the caveat: Provided there is no oil shock.” 

Adelman

“It is clear our nation is reliant upon big foreign oil. More and more of our imports come from overseas.”

George W. Bush

Page 11: The Art and Science of Forecasting Financial Markets

HEADER DATE

Purpose of the paper

Do changes in oil prices predict stock  market returns?

0

5

10

15

20

25

30

35

40

45

D a t e

Month t t-1 t-2 t-3

futurestock return

The Basic Problem

Oil Price Change

rt 1rt1oil t

Data

Stock market data: 

MSCI reinvestment indices local currency

48 countries & World Market index: 18 developed markets and 30 ‘emerging markets’

October 1973‐ April 2003

Oil Data:

West Texas, Brent, Dubai

4 oil price series +  2 futures

Arab Light

Oil prices of different types

0

5

1 0

1 5

2 0

2 5

3 0

3 5

4 0

4 5

05/2

9/19

87

05/3

1/19

88

05/3

1/19

89

05/3

1/19

90

05/3

1/19

91

05/2

9/19

92

05/2

8/19

93

05/3

1/19

94

05/3

1/19

95

05/3

1/19

96

05/3

0/19

97

05/2

9/19

98

05/2

8/19

99

05/3

1/20

00

05/3

1/20

01

05/3

1/20

02

Oil

Pri

ce (

US

$/B

arre

l)

West Texas

Dubai

Brent

Arab Light

Results

World Market

1973‐10, 355 observations

Alpha: ‐0.081;  t‐value: ‐2.90

All oil series give significant results

All countries:

Developed markets: 12 out of 18 significant

‘Emerging’markets: 8 out of 30 significant 

Results: The Puzzle

Page 12: The Art and Science of Forecasting Financial Markets

HEADER DATE

Economic significance

Buy and Hold 

Oil strategy

If expected return > risk free: market

If expected return < risk free: deposit  

oilttt rrE 11 08.00.7%

Economic significance

0

1000

2000

3000

4000

5000

6000

Oct

-73

Oct

-75

Oct

-77

Oct

-79

Oct

-81

Oct

-83

Oct

-85

Oct

-87

Oct

-89

Oct

-91

Oct

-93

Oct

-95

Oct

-97

Oct

-99

Oct

-01

Year

En

d o

f p

eri

od

we

alt

h Oil strategy

Buy-and-hold strategy

Sectors: World Market

Sector coefficient t-value

Resources -0.04 -1.56

Utilities -0.02 -0.86

Basic Industries -0.04 -1.05

General Industries -0.06 -1.66

Cyc. Cons. Goods -0.08 -1.83

Non Cyc. Cons. Goods -0.06 -2.23

Cyc. Services -0.08 -2.38

Non Cyc. Services -0.06 -1.78

Information Techn. -0.11 -3.02

Financials -0.06 -1.67

Hypotheses

Initial reaction:negative overall World Market reaction

reaction for countries may depend on import/export

Followed by underreaction: Negative relation

Stronger for countries with high energy consumption

Less strong underreaction in oil related sectors

Conclusions

Oil price changes predict stock returns

Violating market efficiency and not as a result of time varying risk premia– Different Countries, Different Samples, Economically Significant, Robust 

to the inclusion of other variables

– Also significantly predicts negative excess returns

Do not reject Gradual Information Diffusion Hypothesis

Time Varying Return Predictability

Would Industrial Metal forecast stock returns?

Month t t-1 t-2 t-3

futurestock return

Change in Industrial Metals

rt 1rt1IM t

Page 13: The Art and Science of Forecasting Financial Markets

HEADER DATE

t-statistic t-statistic Adj. R2 N

All Data (1977-2010) 0.007 2.771 0.010 0.252 -0.002 399

1977-1990 0.009 2.568 -0.023 -0.713 -0.004 166

1991-2000 0.013 4.321 -0.185 -2.281 0.037 120

2001-2010 -0.003 -0.615 0.187 2.162 0.067 113

All data: NO• First Sub-period: NO• Second Sub-period: YES - NEGATIVELY• Third Sub-period: YES - POSTIVELY

Overall predictability Main Result

The same news, different information

Whether one finds positive, negative or no predictability depends on the number of expansion versus contraction states in the sample.

Price of Copper goes up

Production costs increaseHigher demand

Contraction: Stock market goes up

Expansion: Stock market goes down

Industrial Metal Return

Return on stock market index

Dummy depending on business cycle

MethodologyExpansion Contraction

t-statistic Nt-

statistic N F-test

Panel A: NBER

IM index -0.051 -1.586 332 0.217 1.993 67 -0.2683 5.605Aluminum -0.152 -3.010 193 0.319 2.416 38 -0.4710 10.398Copper -0.045 -1.614 332 0.188 2.554 67 -0.2333 8.879Lead -0.027 -0.703 146 0.120 1.880 37 -0.1468 3.601Nickel -0.045 -1.756 170 0.110 1.741 37 -0.1557 5.517Zinc -0.003 -0.079 193 0.264 2.765 38 -0.2678 6.235

Panel B: CFNAI

IM index -0.054 -1.762 334 0.312 3.257 63 -0.3657 13.459Aluminum -0.166 -3.482 194 0.363 3.177 36 -0.5287 18.409Copper -0.046 -1.769 334 0.255 5.041 63 -0.3010 28.735Lead -0.033 -0.972 151 0.145 2.348 31 -0.1783 6.197Nickel -0.043 -1.735 175 0.123 1.758 31 -0.1659 5.287Zinc -0.013 -0.302 194 0.342 4.166 36 -0.3541 14.450

Predictability across the business cycle

Building A Quantitative Model

An example and lessons learned

My challenge…Can you do it?

- 25 years of experience

- Feel for what might work and what not

- Combine everything that I felt might work

- Longer term (transactions costs)

- Simplicity is the ultimate form of sophistication

- Stock markets- Other markets

Page 14: The Art and Science of Forecasting Financial Markets

HEADER DATE

How does it work?

Start of every month

Data Feeds

Software/ModelPredictions:- Indicators

- % prediction

Datastream Matlab

Directional Forecasts

Creating an Edge

81

A Coin-Flipping Exercise: 50% vs. 58% in USD over 120 months

Hedgefund

What is in the model?

• New insights from recently developed strands in the academic literature: • Cross Asset Return Predictability & Gradual Information Diffusion• Time Varying Return Predictability• Calendar effects

• Studies published in top academic journals and yet new unpublished working papers

• My own published and unpublished academic work

• Insights obtained from studying the predictability of stock markets for over two decades.

• Insights gathered as an academic and practitioner

• Insights from building the models

• So far some 5 years of blood, sweat and tears.

Indicator Selection

What to predict?Monthly directional forecastsWhat window to use

Extensive back testing: statistical significance; historical performance of potential indicators in different periods (more recent periods get more weight);Sign consistency in the different back tests; Robustness of results also during 2008-2009 financial crisis;Robustness across estimation methodsRobustness of measurement intervalsRobustness of rolling window length

Economic reasons to include the variable in the model: the variable itself; as a proxy for underlying fundamentals;

Consistency with economic theory and academic studies; Interaction with other variables in the model; Likelihood that the variable also predicts in the future;Availability of data at the proper time.

Page 15: The Art and Science of Forecasting Financial Markets

HEADER DATE

Markets used

Stock Markets: S&P500, FTSE100, STOXX600, Nikkei, SMI, VIX

FX: Euro, Yen, GBP, Australian Dollar, Swiss Franc

Commodities: Nymex, Heating Oil, Natural Gas, Copper, Platinum, Sugar, Cocoa

Bonds: 2 yr Note, 10 yr Note, 10 yr Japan T, gilt, Bond

European Sectors (Based on Stoxx sector indices)

Too Farfetched: UK consumerservices two months ago

Likely indicators:Japan Industrials, S&P500, Yen

Too Obvious: Last month Nikkei returns

Search Algorithm for Gradual Information Diffusion Indicators

Market: Nikkei

Gradual Information Diffusion Indicators for the CHF/USD based on

CHF/USD

5 Sector Indices- US Utilities- EU Financials

1 Market Variable-Japanese bond

2 Stock Market Indices - Stoxx600

2 Currencies- AUS dollar

2 Commodity Market Indices - GS Industrial Metals

Some backtest results

Monthly S&P FTSE STOXX Euro Nymex Vix2000-2011 67.3% 79.1% 66.4% 70.0% 76.4% 68.2%2010-2011 57.1% 85.7% 64.3% 71.4% 78.6% 71.4%2008-2009 80.0% 92.0% 72.0% 76.0% 68.0% 72.0%2000-2007 65.8% 74.0% 65.8% 68.5% 78.1% 67.1%

Actual versus in sample

Stocks Currencies Commodities Bonds

Out of sample 63.24% 49.73% 54.26% 61.59%

In Sample 62.07% 63.13% 65.22% 64.22%

Correct Predictions Out of Sample

0.00%

10.00%

20.00%

30.00%

40.00%

50.00%

60.00%

70.00%

80.00%

90.00%

Correct: 56.73%

Page 16: The Art and Science of Forecasting Financial Markets

HEADER DATE

Over time (in and out of sample)

0.00%

10.00%

20.00%

30.00%

40.00%

50.00%

60.00%

70.00%

80.00%

90.00%

100.00%

Jan-

00

Jul-0

0

Jan-

01

Jul-0

1

Jan-

02

Jul-0

2

Jan-

03

Jul-0

3

Jan-

04

Jul-0

4

Jan-

05

Jul-0

5

Jan-

06

Jul-0

6

Jan-

07

Jul-0

7

Jan-

08

Jul-0

8

Jan-

09

Jul-0

9

Jan-

10

Jul-1

0

Jan-

11

Jul-1

1

Jan-

12

Jul-1

2

Jan-

13

Jul-1

3

Jan-

14

Jul-1

4Based on 36 month moving average

0.0%

2.0%

4.0%

6.0%

8.0%

10.0%

12.0%

14.0%

16.0%

18.0%

20.0%

Jan-

06

Apr

-06

Jul-0

6

Oct

-06

Jan-

07

Apr

-07

Jul-0

7

Oct

-07

Jan-

08

Apr

-08

Jul-0

8

Oct

-08

Jan-

09

Apr

-09

Jul-0

9

Oct

-09

Jan-

10

Apr

-10

Jul-1

0

Oct

-10

Jan-

11

Apr

-11

Jul-1

1

Oct

-11

Jan-

12

Apr

-12

Jul-1

2

Oct

-12

Jan-

13

Apr

-13

Per

cent

age

over

50%

Gradual Information Diffusion

Seasonals

Attribution of Correct Predictions

Lessons Learned

• Predict what you know

• Your indicators may only work for specific markets

• Simplicity is the ultimate form of sophistication

• Beware: you will datamine!

• Keep a truly out of sample period if you can

• Statistical significance won’t tell you everything

• Common Sense

• Use graphs of parameter estimates over time

• Judgement calls: Do you believe, (Gold…)

• Will the world change

• Trust your model

• Model uncertainty

• You’ll make (silly) mistakes

Lessons learned

Stocks Currencies Commodities Bonds

Out of sample 63.24% 49.73% 54.26% 61.59%

In Sample 62.07% 63.13% 65.22% 64.22%

Some markets are easier than othersWhat may work in one market may not work in others

Currencies: If you are wrong with the dollar…..Commodities: Too many other influences?? Unrelated to economyBonds: Surprising… more than interest rate dependence?

Data mining

In sample results Out of sample results

Performance

Data snooping or mining

Number of variables tested: Data snooping

Number of time periods: sample selection

Combining in sample and out of sample estimation

Different models (choosing the ‘best’ methodology)

Optimisation (choosing the ‘best’ interval)

Number of researchers…

“. . . and the Cross-Section of Expected Returns,” Harvey. Liu, Zhu, 2015

Page 17: The Art and Science of Forecasting Financial Markets

HEADER DATE

While there are all sorts of detailedand specific adjustments…

Bonferroni, Holm, White reality check, etc….

Based on all sorts of assumptions

The problem in the real world goes beyond the statisticalprocedures

A t-stat of 3?

Beware up front

Data snooping and all sorts of biases will enter your equation. No matter how hard you try. If you do not control there are hugeeffects. It may happen in the data you select, the method youchoose.

Judge every decision you make on whether some bias may enter

This is a judgement call.

Keep a true out of sample period that you do not use for anyestimation whatsoever

Nobody benefits from a system that does not work

Economic safeguards

- Consistency with economic theory and academic studies;

- Interaction with other variables in the model;Likelihood that the variable also predicts in the future;

- Availability of data at the proper time.

- Economic reasons to include the variable in the model: - the variable itself;- as a proxy for underlying fundamentals;

Once you have first out of sample results

You can make comparison based on in sample and out of sample results.

In sample correct: 64%

Out of sample: 57%

Bias 7%: What is the impact????

Datamining Adjustment in the Backtest

101

Correct Back Test Predictions:

64.05%

Back Test Return of 39.4%

Correct Real Time Predictions:

58% (Feb. 2012 – April 2012)

Adjusted Back Test Return of 24.6%

100.00%

1000.00%

10000.00%

100000.00%

Jan-

03

Au

g-0

3

Ma

r-0

4

Oct

-04

Ma

y-05

Dec

-05

Jul-0

6

Fe

b-0

7

Se

p-0

7

Ap

r-0

8

Nov

-08

Jun-

09

Jan-

10

Au

g-1

0

Ma

r-1

1

Oct

-11

Ma

y-12

Dec

-12

Jul-1

3

Fe

b-1

4

58.00%

64.05%

Actual

Adjusted

Outline

Part 1: Market Efficiency and unpredictability as a benchmark

Part 2: Some empirical examples of forecastability

Part 3: Building quantitative forecasting models

102

Page 18: The Art and Science of Forecasting Financial Markets

HEADER DATE

Opportunities

• We are moving from the question “whether markets are predictable?” to “how to predict them?”.

• Many people do not seem to realise this yet

• There is a tremendous terra incognita out there both from a practical and an academic perspective

• First mover advantage for every institution that gets ahead of the curve

103

Thank you!

Random or not…..

Correct predictions