Stock Market Price Prediction Using VariousMachine Learning Approaches

Frank SaldivarDepartment of Computer Science

University of Texas Rio Grande ValleyEdinburg, TX

francisco.saldivar01@utrgv.edu

Mauricio OrtizDepartment of Computer Science

University of Texas Rio Grande ValleyEdinburg, TX

mauricio.ortiz01@utrgv.edu

Abstract—This report will divulge and discuss the researchendeavours of the authors in the field of machine learningapplications in regards to stock market price prediction. Theresults are somewhat favorable and will be discussed towardsthe end of the paper. The approaches used in this experimentare linear regression, Facebook’s Prophet API for time seriespredictions and a LSTM neural network. The experiment utilizes23 years of closing price from the SPDR S&P 500 ExchangeTraded Fund (ETF) which can be thought of as an analog forthe equities market as a whole.

I. INTRODUCTION

Predicting the movements and patterns in a stock market isan impossibly difficult task. There are countless factors thatdrive the price of a stock and its movements. In the financerealm there exist two main approaches to stock valuation thatleads into a form of prediction and alpha, excess returns,gained. There also exists a core hypothesis stating in generalthat it should be impossible to predict a stock’s price, this isknown as the Efficient Market Hypothesis. Despite commoneconomical sense, we believe that a pattern is able to be foundand in our section discussing results, Our goal in this paper isto compare and contrast three popular machine learning meth-ods for time series prediction: single variable linear regression,Facebook’s Prophet algorithm, and the use of long short termmemory units, or LSTMs for short. Our hypothesis for thisexperiment is that LSTMs will demonstrably outperform otherregression techniques and provide deeper insight into thevalidity of technical analysis.

II. BACKGROUND

In this section, we will be will be laying the foundationand groundwork definitions for future use. An overview of therelevant definitions and concepts are as follows: fundamentaland technical analysis, which are non-machine learning meth-ods for stock valuation, efficient market hypothesis and ourmachine learning approaches.

A. Traditional Methods in Finance

1) Fundamental Analysis: Fundamental analysis refers tothe valuation of a stock taking into account company specificinformation, related news, the overall economy and specificeconomy of a company’s sector, among other factors. In-vestopedia offers a more rigrorous definition in, ”Fundamental

analysis is a method of evaluating securities by attempting tomeasure the intrinsic value of a stock. Fundamental analystsstudy everything from the overall economy and industry condi-tions to the financial condition and management of companies.Earnings, expenses, assets, and liabilities are all importantcharacteristics to fundamental analysts”. [1] Fundamental an-alysts typically make a determination if a stock is undervaluedor overvalued, if undervalued, the stock is recommended as abuy, else if overvalued, recommended a sell.

2) Technical Analysis: Technical analysis is nearly the op-posite of fundamental analysis. The only input for a technicalanalysis is previous stock price data; the technical analystbelieves that a previous pattern in the stock is indicative offuture patterns and prices. Referencing Investopedia again,their take on technical analysis is ”The core assumption is thatall known fundamentals are factored into price, thus there is noneed to pay close attention to them. Technical analysts do notattempt to measure a security’s intrinsic value, but, instead, usestock charts to identify patterns and trends that suggest whata stock will do in the future”. [1] Our approach in machinelearning will be an extension of this approach, using previousand historical data to form a model of prediction.

B. Efficient Market Hypothesis

Related to these analytical approaches is the notion of theEfficient Market Hypothesis. The Efficient Market Hypothesis(EMH) was first posited in the 1960’s in a Ph.D. dissertationby Eugene Fama. Morningstar, a renowned investment websiteoffers the following insight on the EMH, ”the efficient markethypothesis states that at any given time and in a liquid market,security prices fully reflect all available information.” [2] Inessence, EMH states that a security’s price is a function of allknown information about it. The EMH comes in three forms.

1) Weak Form: The weak form of the EMH states that astock’s price is reflected by all historical information about therelevant company or commodity.

2) Semi-Strong Form: This form of the EMH postulatesthat any new public information is immediately factored intoa relevant stock’s valuation. This form implies that when newsbreaks of a stock, investors react immediately and the price isadjusted in real time.

3) Strong Form: The strong form of the EMH states that astock’s price is a function of all possible information, knownand unknown to public. Note that includes insider tradingand private company-only known information.

We consider this background knowledge in the EfficientMarket Hypothesis, economics and finance in general tobe fundamental. In classic economic theory and education,students are taught that the Efficient Market Hypothesisis likely true and to try to find alpha (excess returns)through some sort of technical analysis is a fool’s errand.Reading through papers, such as Choudhry and Garg’spaper, ”A Hybrid Machine Learning System for StockMarket Forecasting”, they comment on the Efficient MarketHypothesis, stating, ”If the EMH were true, it would not bepossible to use AI techniques to predict the market. However,due to the success of technical analysts in the financial worldand a number of studies appearing in academic literaturesuccessfully using AI techniques to predict the market, EMHis widely believed to be a null hypothesis now.” [3]

C. Machine Learning Approaches

When we tasked ourselves with this problem, there weremultiple approaches we could have taken. There exist manydifferent machine learning algorithms and approaches andfinding the right method proved to be a challenge. Ouroverarching goal was to provide to the reader and audiencea proper comparison and contrast between multiple machinelearning approaches.

The problem of stock price prediction can be handledas either a time series regression problem where we try topredict and forecast the continuous value of the stock inquestion or it can be thought of as a classification problem,a binary choice between a stock’s price going up or down.We chose to go with making this problem a time seriesproblem, as it seems to be a more difficult problem and offersa nice variety of algorithms available. We had several optionsavailable and chose to compare and contrast the resultsbetween a simple method like linear regression, a moderatelymore rigorous implementation using Facebook’s Prophet APIfor regression and LSTMs, or long short term memory units,a form of recurrent neural networks.

1) Linear Regression: Linear regression is fairly straight-forward. Essentially, we are trying to predict some valuey for an input, x and a slope or gradient. Our goal hereis to try to find an appropriate linear equation that willsomewhat accurately predict a price, y, given some date,x. Towardsdatascience.com reiterates this with a bit morerigor, stating ”Simple linear regression is a type of regressionanalysis where the number of independent variables is one andthere is a linear relationship between the independent(x) anddependent(y) variable.” [4]

2) Prophet: In our search for an algorithm that was morecomplex than linear regression and not as robust as an LSTMimplementation, we found Facebook’s open source project

named Prophet, used for forecasting. According to their githubpage, ”Prophet is a procedure for forecasting time series databased on an additive model where non-linear trends are fit withyearly, weekly, and daily seasonality, plus holiday effects. Itworks best with time series that have strong seasonal effectsand several seasons of historical data. Prophet is robust tomissing data and shifts in the trend, and typically handlesoutliers well.” [5] More information can be found on theirwebsite in the appropriate reference.

3) RNNs and LSTMs: Recurrent Neural Networks (RNNs)are like they sound, they are artifical neural networks thatare capable of remembering data. Techopedia defines RNNsas ”a type of advanced artificial neural network (ANN) thatinvolves directed cycles in memory. One aspect of recurrentneural networks is the ability to build on earlier types ofnetworks with fixed-size input vectors and output vectors.” [6]RNNs have applications in recognizing patterns in sequencesof data, like text, handwriting, and stock price patterns over atime series. The website, Skymind, a company specializing inoffering infrastructure for artificial intelligence applicationselaborates on RNNs, saying that they ”are distinguished fromfeedforward networks by that feedback loop connected totheir past decisions, ingesting their own outputs moment aftermoment as input. It is often said that recurrent networks havememory.” [7]

Long Short Term Memory Units or LSTMs, proposedin the mid 90s by German researchers Sepp Hochreiterand Juergen Schmidhuber to solve a problem of vanishinggradients LSTMs have been said to remember mistakesand adapt accordingly. Skymind also offers some insightinto LSTMs, stating ”LSTMs contain information outsidethe normal flow of the recurrent network in a gated cell.Information can be stored in, written to, or read from acell, much like data in a computers memory. The cell makesdecisions about what to store, and when to allow reads, writesand erasures, via gates that open and close.” [7]

III. DATA AND EXPERIMENT RESULTS

In this section we will discuss our particular data andmotivations for selecting such data and discussing our results,comparing and contrasting with each other method.

A. Data

For our experiment, we thought it would be a simpler andmore straightforward experiment if we used the SPDR S&P500 Exchange Traded Fund (ETF), this ETF trades under theticker of SPY on the Nasdaq stock exchange. The SPY ETFcan be thought of as a reflection of the market as a whole. Itis a blend of the top 500 companies in the US stock exchangeand seeks to emulate the S&P 500 index.

Our motivations for selecting this equity investment tool isthat the SPY ETF is inherently diversified and less volatilethan a singular company’s stock. Unless the market as awhole moves up or down in unison, the SPY ETF is relatively

stable and allows for an easier prediction, countering thedifficulty of predicting a specific price using regression.

B. Linear Regression Results

Our implementation of linear regression was simple andstraightforward, only requiring a handful of lines of code fromsklearn’s linear model package. Calling an instance of themodel and fitting it to our data yielded these results, as seenin figure 1. As expected, our results were less than stellar -we wanted to show linear regression as a base of comparisonand we believe that it served its purpose.

Fig. 1. Linear Regression Results

While with some adjustment, it could be a better fit, linearregression does not account for any dips in stock price andassumes a fairly linear curve. Linear regression was successfulin its service as a baseline for comparison and should not beused in any actual implementation.

C. Prophet Results

Prophet offered nicer results than linear regression. Therewas a moment where our prediction (in green) intersected withthe actual price (in orange/yellow). Prophet was a nice inbetween stepping as a bridge between linear regression andsomething more complex like LSTMs.

Fig. 2. Prophet’s Regression Results

Prophet may be Facebook’s tool for regression and timeseries problems, we suspect that prophet may have betterutilization in other domains for forecasting.

D. LSTM Results

In our experimentation, we found the most success utilizingLSTM recurrent neural networks. This is in accordance withour expectations, seeing that LSTMs can be quite robust andcomplicated. We experienced a lot of variance in our results,much of the variance stemmed from parameters within Keras’input for their LSTM implementation.

One thing of note is that the LSTM occasionally performedbetter with reverse sorted data, rather than normally sorteddata. The reasoning for this is unclear, but some researchyielded no proper answer as to why this is. Included inour findings is normally sorted data and reverse sorted datawith appropriate labeling for each. These following figureswill be displaying validation, or testing data in blue and ourprediction in red. In our experience, training time did not

Fig. 3. LSTM with 5 epochs and normal sorted data

Fig. 4. LSTM with 5 epochs and reverse sorted data

necessarily yield strictly better results. Without tuning ofparameters, more training often yielded in much worse resultswith normally sorted data and no demonstrable increase inaccuracy for reverse sorted data. With specific tuning ofparameters, greater accuracy seemed to be yielded. Figure 7and figure 8 will be with the tuning parameters of 100 epochs,100 batch size, 60 steps, 50 LSTM units and 0.1 dropoutand 100 epochs, 100 batch size, 100 steps, 50 LSTM unitsand 0.2 dropout, respectively. In this section, we showed ourmost accurate and promising results, however, the appendixcontains much more comprehensive testing.

Fig. 5. LSTM with 100 epochs and normal sorted data

Fig. 6. LSTM with 100 epochs and reverse sorted data

Fig. 7. LSTM with 100 epochs, 100 batch size, 60 steps, 50 LSTM unitsand 0.1 dropout with normal sorted data

Fig. 8. LSTM with 100 epochs, 100 batch size, 100 steps, 50 LSTM unitsand 0.2 dropout with reverse sorted data

IV. DISCUSSION

Recalling the ideas of technical analysis in the efforts ofstock price pattern prediction, we can see that with the use of

LSTMs, we can nearly perfectly predict a future stock priceand perhaps more importantly, we can predict movementpatterns in the price, predicting movements up and down. Weconsider these results to be very favorable and can serve as abaseline for future work. Considering that our only data inputwas previous stock prices as training data, to predict threeyears of future stock price movement which high accuracyshows the prowess of LSTMs and recurrent neural networks.

Further insight is required to understand the reasoningof the diversion of prediction and validation data with 100epochs and no tuning of parameters. Further still, it would beinteresting to understand why LSTMs perform just as well orif not, better with reverse sorted data.

V. CONCLUSION AND FUTURE WORK

For as nice as these results are, as computer scientists, webelieve we can further optimize and achieve a better fit. A fewof these optimizations include:

• More granular data, like intraday trading data• Taking into account investor sentiment• Focus on singular stock

Our training set of data was 20 years of historical closingprice data and testing set was 3 years of closing pricedata, which only resulted in a total of 5789 entries of data.Something that can be done would be to only consider thelast five or ten years of data, however take into account astock’s price every two or three hours, this would providea wealth of data and finer detail than just daily changes inprice. One thing to consider as well is that with 20 yearsof data, older data may become stale and not as relevant asfresher data, so to trim that down and get more current datamay yield better results.

One major point of focus as well is to focus on overallinvestor sentiment. Whether investors want to admit it or not,many stock traders trade based on an emotional response,this can stem from current events, news or what Twitter hasto say. An improvement for a more holistic prediction on astock’s price is to take into account what users on Twitter sayabout a particular stock or related stocks. This can be donethrough an API to scrape Twitter for posts about a stock andperhaps filter out posts made by users with less than 1,000followers, as to filter out noise. This approach takes elementsof fundamental analysis by taking a survey of the economythrough the lens of the internet consciousness via Twitter andapplying that as a weight against predictions.

Finally, a way to refine this would be to focus on asingular stock, rather than an index or an ETF. Focusing ona single stock may inherently be more difficult because ofthe volatility of a lone company versus the diversified natureof an ETF, but with taking into account investor sentimentthrough natural language processing (NLP) and finer data,this may offset the difficulty of analyzing a single stock.

ACKNOWLEDGMENT

Many thanks go out to Dr. Dongchul Kim for providingadvice and insight into this project, Adolfo Gonzalez, the TAfor this class, Osvaldo Castellanos, a classmate and the selfassembly and machine learning lab members for their inputand advice on how to approach this problem, without themthis project would have had many more challenges than italready did.

REFERENCES

[1] C. Majaski, “The difference between fundamental vs. technical analysis?,”Apr 2019.

[2][3] R. Choudhry and K. Garg, “A hybrid machine learning system for

stock market forecasting,” World Academy of Science, Engineering andTechnology, vol. 39, no. 3, pp. 315–318, 2008.

[4] R. Gandhi and R. Gandhi, “Introduction to machine learning algorithms:Linear regression,” May 2018.

[5] “Prophet is a forecasting procedure implemented in r and python. it isfast and provides completely automated forecasts that can be tuned byhand by data scientists and analysts..”

[6] “What is a recurrent neural network (rnn)? - definition from techopedia.”[7] “A beginner’s guide to lstms and recurrent neural networks.”

APPENDIX

In this section, we will be displaying all the other LSTMresults of testing with parameters on normal sorted data andreverse sorted data. We will be using the following codesystem to denote tuning parameters:

• E: epochs• B: batch size• S: steps• L: LSTM units• D: dropout

A. Normal Sorted Data Results

Fig. 9. E100B64S60L50D0.2

Fig. 10. E100B100S100L50D0.2

Fig. 11. E100B100S20L50D0.2

Fig. 12. E100B100S60L100D0.2

Fig. 13. E100B100S60L25D0.2

Fig. 14. E100B100S60L50D0.2

Fig. 15. E100B100S60L50D0.15

Fig. 16. E100B100S60L50D0.5

B. Reverse Sorted Data Results

Fig. 17. E100B64S60L50D0.2

Fig. 18. E100B100S20L50D0.2

Fig. 19. E100B100S60L100D0.2

Fig. 20. E100B100S60L25D0.2

Fig. 21. E100B100S60L50D0.1

Fig. 22. E100B100S60L50D0.15

Fig. 23. E100B100S60L50D0.2

Fig. 24. E100B100S60L50D0.5