Machine learning in stock market analysis

Agenda

• Economic concepts

• Can we predict the future price of a stock?

• Hidden Markov Models

• Building a virtual investor

• Experimental results

• Demo: Ben Investment Assistant

• Conclusions and future work

Economic concepts

• Stock Markets

• Stock price and volume

• Other indicators

Prediction of stock prices

• Random walk and the Efficient Market Hypothesis

• Dow Theory

• Conclusions

Hidden Markov Models

• Intuitive description

• Example:

Building a virtual investor

• He learns from historical financial data

• Based on what he learned he makes decisions (Buy/Sell/Hold)

• What data do we provide?

Preparing data

• We apply the EWMA financial technique to eliminate noise by smoothing the series.

• We consider for each the day the rate of growth by applying the natural logarithm for the daily return

• How do we make use of this data?

Computations

• Modeling observations: Multivariate Gaussian mixtures

• Re-estimations:

– What is the probability of being at state 2 at time 4?

– What is the probability of being at state 2 at time 4 at mixture 3?

– How do we re-estimate the model?

ComputationsForward procedure:

Backward procedure:

Computations

Computations

The algorithm

Experimental results

• Tests conducted for 14 randomly selected companies from different sectors: financial, utilities, technology, services and healthcare.

• We obtained to over 100% in revenues, and we suffered losses only when a company suffered a huge depreciation in its stock price.

• A few examples...

Goldman Sachs (NYSE:GS)

Above is the account evolution for investing in Goldman Sachs during June 07 – June 08 (After a year it generated a 53.3% revenue)

Above is the Goldman Sachs stock price evolution (June 07 – June 08)

Royal Gold (NYSE:RGLD)

Above is the account evolution for investing in Royal Gold (It generated a 50.3% revenue in 97 days)

Above is the Royal Gold stock price evolution for the testing period

An extreme case I (NYSE:MBI)

Above is the account evolution for investing in MBIA. The system does a good job at minimizing losses (only 26.2% loss)

Above is the MBIA stock price evolution for June 07 – June 08

An extreme case II (NYSE:MBI)

Using Auto-regression trees. A 74.2% loss

Above is the MBIA stock price evolution for June 07 – June 08

Demo: Investing in Google

• Ben Investment Assistant was done using:

• Windows Presentation Foundation, Sql Server, Analysis Services, ADOMD.NET, AMO, .NET 3.5, C# 3.0, Linq to SQL on Windows Vista Business.• 3-tier architecture, highly scalable

Conclusions

• Due to our results we can invalidate the assumption that past data has no use.

• Because the algorithm behaves like an investor we can have losses if the company suffers a severe depreciation of value.

Future work

• If we let Ben make decisions on a diversified portfolio we might almost be certain of a profitable outcome.

• We can expand the vector of observations to include more data (for example a news index calculated with text mining and Google search API)

Thank you!