algorithmic trading: statistical significance
TRANSCRIPT
IntroductionES 100-day momentum example
Statistical SignificanceClosing Thoughts
Algorithmic Trading: Statistical Significance
Antony Jackson, CFA
University of East Anglia
London Quant Club, July 2016
Antony Jackson, CFA London Quant Club 1/ 19
IntroductionES 100-day momentum example
Statistical SignificanceClosing Thoughts
Overview
Economic Significance versus Statistical Significance.
Antony Jackson, CFA London Quant Club 2/ 19
IntroductionES 100-day momentum example
Statistical SignificanceClosing Thoughts
Overview
Economic Significance versus Statistical Significance.
What is a trading strategy? Binary Signals.
Antony Jackson, CFA London Quant Club 2/ 19
IntroductionES 100-day momentum example
Statistical SignificanceClosing Thoughts
Overview
Economic Significance versus Statistical Significance.
What is a trading strategy? Binary Signals.
100-day ES momentum example.
Antony Jackson, CFA London Quant Club 2/ 19
IntroductionES 100-day momentum example
Statistical SignificanceClosing Thoughts
Overview
Economic Significance versus Statistical Significance.
What is a trading strategy? Binary Signals.
100-day ES momentum example.
Statistical Significance.
Antony Jackson, CFA London Quant Club 2/ 19
IntroductionES 100-day momentum example
Statistical SignificanceClosing Thoughts
Economic Significance versus Statistical Significance
Economic Significance can be thought of as the degree to which a tradingstrategy satisfies the goals of an investor.
Antony Jackson, CFA London Quant Club 3/ 19
IntroductionES 100-day momentum example
Statistical SignificanceClosing Thoughts
Economic Significance versus Statistical Significance
Economic Significance can be thought of as the degree to which a tradingstrategy satisfies the goals of an investor.
What impact does the strategy have on the investor’s portfolio? Do its returnshave a low correlation (diversification) and high stand-alone Sharpe Ratio(risk-adjusted returns)?
Antony Jackson, CFA London Quant Club 3/ 19
IntroductionES 100-day momentum example
Statistical SignificanceClosing Thoughts
Economic Significance versus Statistical Significance
Economic Significance can be thought of as the degree to which a tradingstrategy satisfies the goals of an investor.
What impact does the strategy have on the investor’s portfolio? Do its returnshave a low correlation (diversification) and high stand-alone Sharpe Ratio(risk-adjusted returns)?
Economic Significance rule-of-thumb:
Antony Jackson, CFA London Quant Club 3/ 19
IntroductionES 100-day momentum example
Statistical SignificanceClosing Thoughts
Economic Significance versus Statistical Significance
Economic Significance can be thought of as the degree to which a tradingstrategy satisfies the goals of an investor.
What impact does the strategy have on the investor’s portfolio? Do its returnshave a low correlation (diversification) and high stand-alone Sharpe Ratio(risk-adjusted returns)?
Economic Significance rule-of-thumb:
Accept strategy into existing portfolio if
Antony Jackson, CFA London Quant Club 3/ 19
IntroductionES 100-day momentum example
Statistical SignificanceClosing Thoughts
Economic Significance versus Statistical Significance
Economic Significance can be thought of as the degree to which a tradingstrategy satisfies the goals of an investor.
What impact does the strategy have on the investor’s portfolio? Do its returnshave a low correlation (diversification) and high stand-alone Sharpe Ratio(risk-adjusted returns)?
Economic Significance rule-of-thumb:
Accept strategy into existing portfolio if
Stand-alone SR > Existing portfolio SR× correlation
Antony Jackson, CFA London Quant Club 3/ 19
IntroductionES 100-day momentum example
Statistical SignificanceClosing Thoughts
Economic Significance versus Statistical Significance
Statistical significance is concerned with how likely the results of our tradingstrategy could have been generated by pure chance.
Antony Jackson, CFA London Quant Club 4/ 19
IntroductionES 100-day momentum example
Statistical SignificanceClosing Thoughts
Economic Significance versus Statistical Significance
Statistical significance is concerned with how likely the results of our tradingstrategy could have been generated by pure chance.
If you have enough monkeys and typewriters, how likely is it that one of themwrites the complete works of William Shakespeare?
Antony Jackson, CFA London Quant Club 4/ 19
IntroductionES 100-day momentum example
Statistical SignificanceClosing Thoughts
Economic Significance versus Statistical Significance
Statistical significance is concerned with how likely the results of our tradingstrategy could have been generated by pure chance.
If you have enough monkeys and typewriters, how likely is it that one of themwrites the complete works of William Shakespeare?
It’s possible to have statistical significance without economic significance. Forexample, you design a strategy with a Sharpe Ratio of 0.05, and you are 99%sure this is significantly larger than zero.
Antony Jackson, CFA London Quant Club 4/ 19
IntroductionES 100-day momentum example
Statistical SignificanceClosing Thoughts
Economic Significance versus Statistical Significance
Statistical significance is concerned with how likely the results of our tradingstrategy could have been generated by pure chance.
If you have enough monkeys and typewriters, how likely is it that one of themwrites the complete works of William Shakespeare?
It’s possible to have statistical significance without economic significance. Forexample, you design a strategy with a Sharpe Ratio of 0.05, and you are 99%sure this is significantly larger than zero.
Equally, we can have economic significance without statistical significance. Forexample, a strategy with a Sharpe Ratio of 0.40, but where the buy and holdSharpe Ratio is 0.50.
Antony Jackson, CFA London Quant Club 4/ 19
IntroductionES 100-day momentum example
Statistical SignificanceClosing Thoughts
What is a Trading Strategy? Binary Signals.
I recommend starting algorithmic trading by coding up strategies with simplesignals; e.g., 1 for buy and 0 for out of the market.
Antony Jackson, CFA London Quant Club 5/ 19
IntroductionES 100-day momentum example
Statistical SignificanceClosing Thoughts
What is a Trading Strategy? Binary Signals.
I recommend starting algorithmic trading by coding up strategies with simplesignals; e.g., 1 for buy and 0 for out of the market.
Traditional technical trading systems can be coded up like this. For example,simple moving averages, channel breakouts, filter rules.
Antony Jackson, CFA London Quant Club 5/ 19
IntroductionES 100-day momentum example
Statistical SignificanceClosing Thoughts
What is a Trading Strategy? Binary Signals.
I recommend starting algorithmic trading by coding up strategies with simplesignals; e.g., 1 for buy and 0 for out of the market.
Traditional technical trading systems can be coded up like this. For example,simple moving averages, channel breakouts, filter rules.
Simplifying strategies to a series of ones and zeros helps a lot in establishingthe likelihood of a trading strategy being statistically significant, because purechance strategies can be easily constructed using coin tossing to generatesignals.
Antony Jackson, CFA London Quant Club 5/ 19
IntroductionES 100-day momentum example
Statistical SignificanceClosing Thoughts
100-day ES momentum example
There’s a lot of evidence in the academic literature of momentum in assetprices.
Antony Jackson, CFA London Quant Club 6/ 19
IntroductionES 100-day momentum example
Statistical SignificanceClosing Thoughts
100-day ES momentum example
There’s a lot of evidence in the academic literature of momentum in assetprices.
Cross-sectional momentum involves buying a portfolio of recent winners andfunding the purchase by selling a portfolio of recent losers.
Antony Jackson, CFA London Quant Club 6/ 19
IntroductionES 100-day momentum example
Statistical SignificanceClosing Thoughts
100-day ES momentum example
There’s a lot of evidence in the academic literature of momentum in assetprices.
Cross-sectional momentum involves buying a portfolio of recent winners andfunding the purchase by selling a portfolio of recent losers.
Time-series momentum looks at an asset in isolation. We buy if the asset hasgone up, and sell if it has gone down. This is the simple (but effective!)method I’ll consider today.
Antony Jackson, CFA London Quant Club 6/ 19
IntroductionES 100-day momentum example
Statistical SignificanceClosing Thoughts
100-day ES momentum example
Preliminaries:
The Matlab scripts for this presentation make use of the Quantiacs Toolbox.
Antony Jackson, CFA London Quant Club 7/ 19
IntroductionES 100-day momentum example
Statistical SignificanceClosing Thoughts
100-day ES momentum example
Preliminaries:
The Matlab scripts for this presentation make use of the Quantiacs Toolbox.
You will need to open Matlab and place the files from the presentation in thesame directory as the Toolbox files such as runts and optimize.
Antony Jackson, CFA London Quant Club 7/ 19
IntroductionES 100-day momentum example
Statistical SignificanceClosing Thoughts
100-day ES momentum example
Preliminaries:
The Matlab scripts for this presentation make use of the Quantiacs Toolbox.
You will need to open Matlab and place the files from the presentation in thesame directory as the Toolbox files such as runts and optimize.
The files ESbootstrap.m and ESmomentum.m are Quantiacs trading systemfiles, and the file ESbootstrapCaller.m is a script file used to process theresults of random trading strategies.
Antony Jackson, CFA London Quant Club 7/ 19
IntroductionES 100-day momentum example
Statistical SignificanceClosing Thoughts
100-day ES momentum example
The ESbootstrap.m file contains the following code:
Antony Jackson, CFA London Quant Club 8/ 19
IntroductionES 100-day momentum example
Statistical SignificanceClosing Thoughts
100-day ES momentum example
The ESbootstrap.m file contains the following code:
function [p, settings] = ESbootstrap(CLOSE, settings)
settings.markets = {’F ES’};
settings.lookback = 260;
setting.slippage = 0;
trial = 1; %#[1:1:1000]#
p = 1; % uncomment this to see the performance of buy and hold
p = round(rand()); % uncomment this to generate random strategies
end
Antony Jackson, CFA London Quant Club 8/ 19
IntroductionES 100-day momentum example
Statistical SignificanceClosing Thoughts
100-day ES momentum example
An important line is
Antony Jackson, CFA London Quant Club 9/ 19
IntroductionES 100-day momentum example
Statistical SignificanceClosing Thoughts
100-day ES momentum example
An important line is
settings.slippage = 0;
Antony Jackson, CFA London Quant Club 9/ 19
IntroductionES 100-day momentum example
Statistical SignificanceClosing Thoughts
100-day ES momentum example
An important line is
settings.slippage = 0;
At this stage we are trying to identify a property of the time series of pricesthat offers an opportunity, either in terms of identifying a market anomaly or interms of harvesting a risk premium.
Antony Jackson, CFA London Quant Club 9/ 19
IntroductionES 100-day momentum example
Statistical SignificanceClosing Thoughts
100-day ES momentum example
An important line is
settings.slippage = 0;
At this stage we are trying to identify a property of the time series of pricesthat offers an opportunity, either in terms of identifying a market anomaly or interms of harvesting a risk premium.
We don’t want to distort the results at this stage with transaction costs.
Antony Jackson, CFA London Quant Club 9/ 19
IntroductionES 100-day momentum example
Statistical SignificanceClosing Thoughts
100-day ES momentum example
If we uncomment p = 1;, we simulate a buy-and-hold strategy for ES.
Antony Jackson, CFA London Quant Club 10/ 19
IntroductionES 100-day momentum example
Statistical SignificanceClosing Thoughts
100-day ES momentum example
If we uncomment p = 1;, we simulate a buy-and-hold strategy for ES.
Enter the command runts(’ESbootstrap’) within the Quantiacs Toolbox, andwe obtain the following chart for the buy and hold strategy:
Antony Jackson, CFA London Quant Club 10/ 19
IntroductionES 100-day momentum example
Statistical SignificanceClosing Thoughts
100-day ES momentum example
1000000
1221663
1492461
1823285
2227440
2721182
3324368
4061258
4961490
996368
4943467
Per
form
ance
(lo
g)
Factsheet of fundEquity in ESbootstrap
1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016
0
0.2
0.4
0.6
0.8
1
Long
/Sho
rt
Max DrawdownMax Time Off PeakLongShort
Your Income Estimation($ 1Mio Investment, 10%
performance fee)total $
per anno $324408.4512747.24
Performance Numbers
0.34740.5555
0.06400.1843
0.6292
0.1018
3352
Sharpe RatioSortino Ratio
Performance (prc/y)Volatility (prc/y)
Maximum Drawdown
MAR Ratio
Max Time off peak
Antony Jackson, CFA London Quant Club 11/ 19
IntroductionES 100-day momentum example
Statistical SignificanceClosing Thoughts
100-day ES momentum example
The buy and hold strategy has a Sharpe Ratio of 0.3474.
Antony Jackson, CFA London Quant Club 12/ 19
IntroductionES 100-day momentum example
Statistical SignificanceClosing Thoughts
100-day ES momentum example
The buy and hold strategy has a Sharpe Ratio of 0.3474.
Let’s now have a look at the code for a 100-day momentum strategy, which iscontained in the file ESmomentum.m
Antony Jackson, CFA London Quant Club 12/ 19
IntroductionES 100-day momentum example
Statistical SignificanceClosing Thoughts
100-day ES momentum example
The buy and hold strategy has a Sharpe Ratio of 0.3474.
Let’s now have a look at the code for a 100-day momentum strategy, which iscontained in the file ESmomentum.m
function [p, settings] = ESmomentum(CLOSE, settings)
settings.markets = {’F ES’};
setting.lookback = 260;
setting.slippage = 0;
p = CLOSE(end) > CLOSE(end - 100);
end
Antony Jackson, CFA London Quant Club 12/ 19
IntroductionES 100-day momentum example
Statistical SignificanceClosing Thoughts
100-day ES momentum example
1000000
1158838
1342905
1556209
1803394
2089841
2421787
2806458
3252230
999574
3250845
Per
form
ance
(lo
g)
Factsheet of fundEquity in ESmomentum
1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016
0
0.2
0.4
0.6
0.8
1
Long
/Sho
rt
Max DrawdownMax Time Off PeakLongShort
Your Income Estimation($ 1Mio Investment, 10%
performance fee)total $
per anno $196232.847710.73
Performance Numbers
0.39200.5974
0.04390.1119
0.2584
0.1697
2428
Sharpe RatioSortino Ratio
Performance (prc/y)Volatility (prc/y)
Maximum Drawdown
MAR Ratio
Max Time off peak
Antony Jackson, CFA London Quant Club 13/ 19
IntroductionES 100-day momentum example
Statistical SignificanceClosing Thoughts
100-day ES momentum example
Not bad. The Sharpe Ratio has been boosted from 0.3474 to 0.3920.
Antony Jackson, CFA London Quant Club 14/ 19
IntroductionES 100-day momentum example
Statistical SignificanceClosing Thoughts
100-day ES momentum example
Not bad. The Sharpe Ratio has been boosted from 0.3474 to 0.3920.
So at this stage, we could argue that we have discovered an economicallysignificant trading strategy, because we have boosted the Sharpe Ratio andwe’ve come up with a way of staying out of the worst of the financial crisis,without cheating.
Antony Jackson, CFA London Quant Club 14/ 19
IntroductionES 100-day momentum example
Statistical SignificanceClosing Thoughts
100-day ES momentum example
Not bad. The Sharpe Ratio has been boosted from 0.3474 to 0.3920.
So at this stage, we could argue that we have discovered an economicallysignificant trading strategy, because we have boosted the Sharpe Ratio andwe’ve come up with a way of staying out of the worst of the financial crisis,without cheating.
The next step is to quantify the probability that a “pure chance” strategy couldgenerate a similar (or better) Sharpe ratio.
Antony Jackson, CFA London Quant Club 14/ 19
IntroductionES 100-day momentum example
Statistical SignificanceClosing Thoughts
Statistical Significance
In the ESbootstrap.m file, comment out the p = 1; line, and uncomment thep = round(rand()); line.
Antony Jackson, CFA London Quant Club 15/ 19
IntroductionES 100-day momentum example
Statistical SignificanceClosing Thoughts
Statistical Significance
In the ESbootstrap.m file, comment out the p = 1; line, and uncomment thep = round(rand()); line.
The mysterious %#[1:1:1000]# line is there so that the optimize function inthe Quantiacs Toolbox runs a random trading system 1000 times, collecting theSharpe Ratio each time.
Antony Jackson, CFA London Quant Club 15/ 19
IntroductionES 100-day momentum example
Statistical SignificanceClosing Thoughts
Statistical Significance
In the ESbootstrap.m file, comment out the p = 1; line, and uncomment thep = round(rand()); line.
The mysterious %#[1:1:1000]# line is there so that the optimize function inthe Quantiacs Toolbox runs a random trading system 1000 times, collecting theSharpe Ratio each time.
Essentially, we are going to generate 1000 strategies by tossing a coin every dayto decide whether we are long or out of the market.
Antony Jackson, CFA London Quant Club 15/ 19
IntroductionES 100-day momentum example
Statistical SignificanceClosing Thoughts
Statistical Significance
The ESbootstrapCaller.m script contains the following code:
Antony Jackson, CFA London Quant Club 16/ 19
IntroductionES 100-day momentum example
Statistical SignificanceClosing Thoughts
Statistical Significance
The ESbootstrapCaller.m script contains the following code:
% Script to call the ESbootstrap function
out = optimize(’ESbootstrap’);
hist(out.sharpe)
title(’Histogram of Trading Strategy Sharpe Ratios’)
Antony Jackson, CFA London Quant Club 16/ 19
IntroductionES 100-day momentum example
Statistical SignificanceClosing Thoughts
Statistical Significance
The ESbootstrapCaller.m script contains the following code:
% Script to call the ESbootstrap function
out = optimize(’ESbootstrap’);
hist(out.sharpe)
title(’Histogram of Trading Strategy Sharpe Ratios’)
Enter run ESbootstrapCaller at the Matlab command line, and we obtain ahistogram something like this:
Antony Jackson, CFA London Quant Club 16/ 19
IntroductionES 100-day momentum example
Statistical SignificanceClosing Thoughts
Statistical Significance
−0.2 0 0.2 0.4 0.6 0.80
50
100
150
200
250
300Histogram of Trading Strategy Sharpe Ratios
Antony Jackson, CFA London Quant Club 17/ 19
IntroductionES 100-day momentum example
Statistical SignificanceClosing Thoughts
Statistical Significance
Finally we add the following line to the ESbootstrapCaller script to work outthe proportion of random strategies that obtain a Sharpe Ratio greater thanour “real-life” strategy Sharpe Ratio of 0.3920:
Antony Jackson, CFA London Quant Club 18/ 19
IntroductionES 100-day momentum example
Statistical SignificanceClosing Thoughts
Statistical Significance
Finally we add the following line to the ESbootstrapCaller script to work outthe proportion of random strategies that obtain a Sharpe Ratio greater thanour “real-life” strategy Sharpe Ratio of 0.3920:
pValue = sum(out.sharpe(:) > 0.3920)/1000;
Antony Jackson, CFA London Quant Club 18/ 19
IntroductionES 100-day momentum example
Statistical SignificanceClosing Thoughts
Statistical Significance
Finally we add the following line to the ESbootstrapCaller script to work outthe proportion of random strategies that obtain a Sharpe Ratio greater thanour “real-life” strategy Sharpe Ratio of 0.3920:
pValue = sum(out.sharpe(:) > 0.3920)/1000;
For this particular sequence of coin tosses, we obtain a p-value of 0.1530.
Antony Jackson, CFA London Quant Club 18/ 19
IntroductionES 100-day momentum example
Statistical SignificanceClosing Thoughts
Statistical Significance
Finally we add the following line to the ESbootstrapCaller script to work outthe proportion of random strategies that obtain a Sharpe Ratio greater thanour “real-life” strategy Sharpe Ratio of 0.3920:
pValue = sum(out.sharpe(:) > 0.3920)/1000;
For this particular sequence of coin tosses, we obtain a p-value of 0.1530.
We prefer low p-values, as it’s then less likely that our results could begenerated by pure chance.
Antony Jackson, CFA London Quant Club 18/ 19
IntroductionES 100-day momentum example
Statistical SignificanceClosing Thoughts
Closing Thoughts
The methods I’ve introduced today can be extended to testing multiple ruleson the same time series.
Antony Jackson, CFA London Quant Club 19/ 19
IntroductionES 100-day momentum example
Statistical SignificanceClosing Thoughts
Closing Thoughts
The methods I’ve introduced today can be extended to testing multiple ruleson the same time series.
For instance, if we decide to test both a 100-day and a 250-day momentumrule, we need to take into account that we are now testing multiple strategies,and so the bootstrap methodology needs to be adjusted accordingly.
Antony Jackson, CFA London Quant Club 19/ 19
IntroductionES 100-day momentum example
Statistical SignificanceClosing Thoughts
Closing Thoughts
The methods I’ve introduced today can be extended to testing multiple ruleson the same time series.
For instance, if we decide to test both a 100-day and a 250-day momentumrule, we need to take into account that we are now testing multiple strategies,and so the bootstrap methodology needs to be adjusted accordingly.
In this case, for each trial, we would run two random strategies and pick thehighest Sharpe Ratio each time.
Antony Jackson, CFA London Quant Club 19/ 19
IntroductionES 100-day momentum example
Statistical SignificanceClosing Thoughts
Closing Thoughts
The methods I’ve introduced today can be extended to testing multiple ruleson the same time series.
For instance, if we decide to test both a 100-day and a 250-day momentumrule, we need to take into account that we are now testing multiple strategies,and so the bootstrap methodology needs to be adjusted accordingly.
In this case, for each trial, we would run two random strategies and pick thehighest Sharpe Ratio each time.
We balance the increased likelihood of obtaining a good Sharpe Ratio bytesting multiple models against the rightward shift in the distribution ofrandom Sharpe Ratios that results from picking the best of multiple randomstrategies each time.
Antony Jackson, CFA London Quant Club 19/ 19