Institutional Analytics for Retail Investors

Dept. of CIS - Senior Design 2014-2015∗

Chris Holtholtc@seas.upenn.eduUniv. of Pennsylvania

Philadelphia, PA

Eugene Yarovoieyarovoi@seas.upenn.edu

Univ. of PennsylvaniaPhiladelphia, PA

Fahim Abouelfadlfahim@seas.upenn.eduUniv. of Pennsylvania

Philadelphia, PA

ABSTRACTInvestment institutions have sophisticated methods and soft-

ware for the systematic analysis of stocks, but ordinary in-dividuals investing on their own behalf do not have access tothe same tools. We first describe the field of stock analysisand some of the methods used to predict stock movements,in particular three areas: fundamental analysis, which seeksto establish the value of a stock based on the attributes ofthe underlying company; technical analysis, which examinesa stock’s price and volume movements; and sentiment anal-ysis, the systematic examination of attitudes surrounding astock. While investment institutions have spent considerableresources on developing software to automate all of theseanalyses, their complexity means that retail (non-institutional)investors typically lack the knowledge, time, and capital todo the same. This creates an unfair situation in the stockmarket, where institutions are able to more quickly identifyand exploit profit opportunities, while retail investors lag be-hind and never realize the same levels of profit.

We then propose Phinance Analytics, a system to addressthe gap between institutional and retail investors. A sys-tem that performs a hybrid combination of technical, funda-mental, and sentiment analysis, exposed to users through aconvenient web interface, allows individuals access to someportion of the same types of tools that investment institu-tions have at their disposal. The system provides the userwith a score for each analytical component outlined above,combined with a simple natural langugage description of thereasons for the score, and an overall recommendation forwhat the user should do with the stock. This information canbe used as part of a user’s research into a stock that the useris already interested in, or to quickly identify which stocksmay be interesting investment opportunities to consider. Theuser interacts with the system through a simple-to-nagivate,intuitive website.

1. INTRODUCTIONThe field of stock analysis is well explored, and many tools

have been created to gain insight into the true value of astock. Some of these methods are based on a company’s fi-nancials and management, and thus belong to the categoryof fundamental analysis, or the analysis of the company’seconomic fundamentals. Other analysis methods focus onthe price and quantity movements of the stock, and areclassified under the umbrella of technical analysis. More re-cently, many institutions are also developing sentiment anal-

∗Advisor: Aaron Roth (aaroth@cis.upenn.edu).

ysis tools to improve the efficiency of market research. Thesetools programmatically scrape and read text from the Webor other sources and determine its positivity or negativityto capture a picture of investor confidence in a stock.

To implement all of these diverse analysis tools and suc-cessfully identify profitable stocks, a large amount of timeand knowledge is required. These requirements are easilymet by institutional investors, such as large Wall Street firmswith scores of employees and capital, but are mostly outof reach for retail investors, who are individuals investingon their own behalf. Large firms also have the means topurchase analytic tools, such as Bloomberg Terminals [9],which cost tens of thousands of dollars annually and aretherefore once again not accessible to many retail investors.The large imbalance in the quality of the tools available forstock analysis between institutional and retail investors hasallowed institutions to reap the profits of the economic re-covery since 2009 while retail investors, those hardest hitby the recession, have often been unable to realize similargains.

A few companies, such as E-Trade[1] and Scottrade[2] pro-vide retail investors with charts and data to perform theirown analysis and valuations. While the data itself is similarto what is available to institutions, these services leave retailinvestors to process and analyze this data, even though manyretail investors do not have the requisite time or knowledgeto adequately do so. To make sense of the data, retail in-vestors require automated stock analysis tools employingmethods similar to some of those employed by institutions.

We propose a new system, Phinance Analytics, that aggre-gates, analyzes and presents multiple kinds of stock analysesin a simple manner. The user has the ability to view senti-ment, fundamental, and technical analysis scores for stocksindividually or by entire sector from a website, www. phi-nanceanalytics.com. For each score that the system gener-ates, it also presents a simple description of the reasons whythe score was assigned.

Each type of analysis that the system presents is the resultof an algorithm that computes a rating from 0 - 100 indi-cating the strength of a buy or sell recommendation (0 is astrong sell, 100 a strong buy). The technical analysis algo-rithm detects technical chart patterns in the recent price his-tory, the sentiment analysis algorithm classifies tweets fromTwitter according to their positivity to determine overallsentiment about a stock, and the fundamental analysis algo-rithm compares financial metrics of stocks to the remainderof the sector to determine whether a particular stock offershigh value for its price.

The analysis provided by our system is intended to aidusers in performing research on stocks that they alreadyown or are already considering purchasing, as well as a toolto efficiently discover potentially profitable stock picks ofwhich they were previously not aware. It is not intendedfor the user to trade solely based on the generated analyseswithout further research and deeper understanding of thecauses underlying positive analysis scores. However, we dopresent an analysis of the advice provided by the system.

2. BACKGROUND

2.1 Fundamental AnalysisStock analysis has existed as long as stocks themselves,

but no analytic system gained widespread popularity untilBenjamin Graham created a system called value investing,in the early 20th century. Outlined in his books [11] and[12], the value analysis system is based upon the principlethat ownership of a stock is equivalent to ownership in thecompany, and thus the price movements of a stock are notas important as the underlying financial health of the com-pany itself. Information about the finances of a companyis not difficult to find: in the United States, pursuant toregulations set forth by the Securities and Exchange Com-mission (SEC), all publicly traded companies are required tosubmit publicly available financial documents, such as bal-ance sheets, income statements, cash flow statements, andannual reports, so that shareholders can assess the finan-cial state of the company. These documents contain criti-cal financial information such as revenue, expenses, assets,growth forecasts, liabilities, and annual dividends. In ad-dition to these quantitative measures, investors take intoaccount many qualitative features such as company man-agement, company vision, and intellectual property, all ofwhich can affect a company’s profitability.

Graham proposed using such measures to calculate an in-trinsic value, or true value, of the company, which could bedivided by the total number of shares to calculate the in-trinsic value of each share [12]. This value is then comparedagainst the current stock price, and if the margin of safety,or difference between the stock price and the intrinsic value,is large enough, then the stock is a good value, and should bebought; in the long run, the market will realize the intrinsicvalue of the company [12].

2.2 Sentiment AnalysisThe intrinsic value of a stock can be difficult to determine

reliably, in part because one needs to estimate how the prof-its of the company will vary over time. Future growth de-pends on somewhat subjective and difficult-to-judge factorssuch as the quality of the company’s management and busi-ness strategy. Sentiment analysis approaches the problem ofdetermining what stock trades to make from a different an-gle. It is well-understood among investors that the attitudetowards a stock can affect its price. There are several mech-anisms by which this can occur: for example, attitudes couldbe indicative of a growing realization that the company isincorrectly valued by the market, the market could be re-acting to news about the company or data recently cometo light, or prices could simply be affected by the fact thatpositive attitudes increase demand and hence the price forthe stock, while negative attitudes have the opposite effect.Sentiment may thus reflect news, investor confidence, and

the general attitude towards a company in the market, soit stands to reason that analyzing sentiment about a com-pany could be useful for stock analysis. While this type ofanalysis is performed naturally thousands of times daily byinvestors who simply react to news and analyst ratings, anygiven investor can only read a relatively small amount ofarticles in a fixed period of time. A program using naturallanguage processing (NLP) techniques to read text and de-termine its positivity would be able to read many thousandsof times the number of articles and make decisions based onan enormous volume of data.

In general, the goal of a sentiment analysis program is toaccurately label a sentence or document with a label indi-cating its positivity, and thus sentiment analysis is a typeof text classification task. It is typically performed by ex-tracting some set of features from a sentence or document,and then training a classifier on some number of manually-labeled examples to distinguish between instances with dif-ferent positivity. Some of the key differences in sentimentanalysis techniques consist of which features are extractedfor use in the classifier, the type of classifier used, and therange of possible outputs the classifier can produce. Somesentiment analysis algorithms use regression instead of clas-sification, producing a score in a continuous range (e.g. [10]).

2.3 Technical AnalysisAnother popular analytic system is technical analysis, or

the prediction of stock price based on price and volumemovements correlated with past trends and industry indi-cators. A book by Schabacker [19] is an example of earlywork in technical analysis; Bassetti et al. [18] presents amuch more modern treatment. Murphy [17] is studied bymany technical analysts today. Technical analysis followsthe rule of the efficient market hypothesis, or the belief thatstocks always trade at their intrinsic value, and the only wayto profit on the stock market is to play the market swings,which are based on price and volume movements. As tech-nical analysis has progressed, many types of patterns, suchas head and shoulders or rectangle tops, have emerged inaddition to the standard price and volume calculations [18].These patterns help to identify possible trend reversals orcontinuations, and thus may be useful in identifying profitopportunities.

As technical analysis has developed, many institutionshave turned towards automated short-term trading to gainlarge profits. These systems perform technical analysis acrossthe market and automatically buy or sell stocks based ongenerated indicators. These systems can execute thousandsof trades per second, which emphasizes enormous numbersof small-profit trades, rather than occasional, highly prof-itable trades. As these systems need to work as close to real-time as possible, they usually connect directly to the stockexchanges themselves. These types of automated tradingsystems currently account for two-thirds of all stock tradeson major exchanges and will continue to increase as moreinstitutions trend towards high frequency trading.

3. RELATED WORKTehcnical analysis methods have been studied in depth

and textbooks exist on the subject (e.g. [17]). Becausesome technical analysis involves looking at charts and ana-lyzing visual patterns, some of the literature has not alwaysdescribed everything in rigorous, algorithmic form. A good

translation of many of the most common patterns into pre-cise formulas and algorithms is given by Lo et al. [15], whichalso cites a large number of studies and publications thathave provided support for the notion that detecting ”visual”patterns in stock prices can lead to profitable trades. Wefollow the algorithmic definitions set forth by Lo et al. forthe part of our technical analysis that relates to detectingchart patterns commonly used by financial analysts.

In fundamental analysis, the main question is how to de-termine a company’s proper valuation. The literature isvast on this subject, and many textbooks exist, with oneby McKinsey & Company [7] commonly used in businessschools. Comparatively few sources, however, give modelsthat are presented algorithmically and that are easy to im-mediately translate into code. Blakeslee et al. [4] give onesuch presentation. Athanassakos [3] also gives some formu-las in a compact form useful for translating into code. Thesesources give guidance as to the most relevant factors to con-sider when performing fundamental analysis; we have chosena simplified set of factors based on some of this information.

For non-experts in the natural language processing do-main to be able to more easily leverage the state-of-the-artin academic research, the Stanford Natural Language Pro-cessing Group makes available an open source NLP projectwith many libraries to perform NLP tasks, called StanfordCoreNLP [16]. We use several components of this library,most notably the sentiment analysis module, which is an im-plementation of a recently published approach by Socher etal. titled Recursive Deep Models for Semantic Composition-ality Over a Sentiment Treebank [20]. The approach uses theidea of parsing the sentence into a syntax tree and assigninga sentiment to each node in the syntax tree in a bottom-uprecursive manner. The leaf nodes of the tree are individualtokens; the sentiment assigned to a leaf node is the generalsentiment associated with the token when interpreted out-side of any context (for example, ”great” represents a highlypositive sentiment, while ”terrible” represents a highly neg-ative sentiment). Each internal node then holds the senti-ment of the portion of the sentence represented by the sub-tree rooted at that node. The manner in which an internalnode’s sentiment depends on its descendants is learned by aneural network.

The idea of using sentiment analysis specifically for thepurpose of predicting financial markets, as we do in this pa-per, is not new. Bollen et al. [6] used sentiment analysisof Twitter posts (tweets) to evaluate market sentiment andfound that certain mood indicators within tweets were cor-related with the performance of the Dow Jones IndustrialAverage three days later, lending added credibility to thenotion that sentiment analysis can have predictive power.Our sentiment analysis pipeline uses Twitter as the primarysource of data. Das and Chen [8] provide another exampleof extracting market sentiment from text on the Web, thistime from stock discussion boards, lending further credenceto the idea that examining sentiment about a stock can re-veal useful information. While we did not extract sentimentdata from sources other than Twitter, this direction holdspromise for future work.

An important part of our system also involves presentingan elegant user interface. Applications such as ours are par-ticularly susceptible to clutter because they consist of manytools and are prone to feature creep. Since the central con-cept of this project is to present a web application that is

accessible to ordinary investors investing on their own be-half, we should have a very strong bias towards simplicity.For this, we referred to a classic book by Krug [14] on thetopic of creating simple and intuitive interfaces. Johnson[13] provides some additional useful guidance.

4. SYSTEM MODEL & USER INTERFACEOur system is comprised of distinct modules that are each

responsible for one particular type of data processing oranalysis, as well as components to combine the results fromthe individual modules and present them together. Figure 1consists of a diagram outlining the interaction between thedifferent modules, and the major components are summa-rized below.

Figure 1: System Model Diagram

4.1 Data Retrieval ModuleThe data retrieval module uses a combination of several

APIs to retrieve or scrape data and save it in a database forlater use. There are a few kinds of data that are required

by the analysis modules.To enable fundamental analysis, detailed financial state-

ment data by ticker symbol is retrieved from the Web andstored in a database. This includes income statements, cashflow statements, and balance sheets for each stock tickersymbol that will be analyzed by the system. Not all of thenumerous fields that appear on these statements are requiredfor fundamental analysis, but the system captures this in-formation as completely as possible to enable re-processingin case the fundamental analysis module is tweaked at alater time to incorporate more factors. Additionally, dailystock price data is also captured to compute financial ratiossuch as price-to-earnings. Daily stock price data includesthe daily low, high, open, close, and volume of trades.

To enable richer technical analysis, the module downloadsintraday stock data. This information consists of price andvolume information on a fairly granular level (on the order ofminutes) over the course of each day. This information canbe important for generating some types of technical analysistrading signals. However, because our system is geared atpresenting data to users for their own decision making andnot towards automated trading, our current analysis mod-ules make use of only the daily closing price data becausesuch information is all that is needed to identify multi-daytrends. The intraday information could generate additionalsignals, but these would need to be acted upon immediatelyrather than presented to the user for consideration.

Another component extracts recent human-generated con-tent for sentiment analysis. This content could be extractedfrom anywhere where there is a significant amount of con-tent being written by humans on a frequent, ongoing basisthat expresses opinions about stocks. For each supporteddata source, there is a component of the module to acquirethat particular data source, scraping the information fromthe Web in the necessary manner, downloading it via a webservice, or retrieving it from a file or other source. The in-formation obtained is stored in a database and marked withthe stock ticker symbol to which it pertains and the date ofthe content.

More concretely, in our current implementation, we useTwitter as our source of data. As detailed further in the Sys-tem Implementation section, we download tweets that dis-cuss a particular stock and store the tweets in the databasefor use by the sentiment module. However, the implementa-tion could be expanded following the framework describedabove to capture other sources of data such as opinions ex-pressed on stock discussion boards and articles by analystsand writers (e.g. articles on Bloomberg).

Once the raw data is captured, it is stored in a SQLdatabase from where it can be accessed by the analyticalmodules.

4.2 Fundamental Analysis ModuleThis module is responsible for performing the fundamen-

tal analysis calculations. The module depends on informa-tion that is stored in the SQL database – more specificallythe balance sheets, cash flow statements, and income state-ments of companies downloaded by the Data Retrieval Mod-ule. Based on the financial data contained in these docu-ments, the module evaluates the favorability of key financialratios for the stock, both in their own right, and comparedto the rest of the sector. Comparing a stock’s financial ratiosand fundamental metrics to the rest of the sector is impor-

tant because it is often difficult to make judgments aboutthe favorability of financial ratios in isolation. We will seewhy this is so after presenting some of the techniques forfundamental analysis.

With some simplifications, the total intrinsic value of acompany should be the value of all the company’s currentassets, minus the company’s current liabilities, plus the sumof all the future expected discounted cash flows [7]. A dis-counted cash flow consists of future profits that should bemultiplied by a discount factor (a value between 0 and 1)that decreases (becomes a progressively more significant dis-count) with how far away in the future the cash is expectedto be produced by the company. Let V be the intrinsic valueof a company, A denote the company’s current assets, L de-note the company’s current liabilities, d(t) be the discountfactor at time t, c(t) be the amount of cash generated attime t, and T the set of all times at which cash is generated.Then,

V = A− L+∑t∈T

c(t)d(t) (1)

Subject to d(t1) < d(t2) if t1 < t2 and ∀t, 0 ≤ d(t) ≤1. Importantly, the discount factor is a strictly decreasingfunction of time.

A common simplifying assumption for the discount fac-tor is that the discount factor should decrease exponentiallywith the amount of time that elapses from the current timeforwards. It is justified in the medium-term where we donot anticipate huge economic upheavals and assume the eco-nomic climate remains fairly stable. The exponential dis-count represents several different factors, all of which growexponentially over time. It accounts for inflation, the timevalue of money (money now is more valuable than money inthe future, because money possessed now can be reinvested),and the repeated risk each year that some event will happenthat will cause the company to stop producing cash. Thislast factor has an exponential form if we assume that a com-pany will still produce cash some number K years later if ithas produced cash all the years before that, and each yearthere is some probability of failure.

Once we assume an exponentially decreasing discount fac-tor, we can rewrite Equation 1 as follows, where t0 is thecurrent time, at which the discount factor is taken to equal1, and α (0 ≤ α < 1) is the exponential discount factor:

V = A− L+∑t∈T

c(t) · αt−t0 (2)

If we assume we are dealing exclusively with large-capcompanies (i.e. those companies that have a large marketcapitalization, defined as the market value of all shares),these companies control large portions of their respectivemarkets and therefore do not grow at very aggressive ratesin the same manner that is possible for small companies orstartups. Much of retail investing in the stock market doesdeal with very-large cap companies; investors are often notfamiliar with smaller companies, and very small companiesare often not publically traded on the stock exchange. Ifwe make the simplifying assumption that such establishedcompanies have a constant yearly cash flow Y , and we con-sider yearly discounted cash flows out infinitely far into thefuture, then the equation becomes:

V = A− L+ Y · 1

1− α (3)

A constant yearly cash flow may not seem like a reason-able assumption, but note that the equation retains the sameform if Y grows at some exponential rate that is smaller thanthe rate at which the discount factor shrinks; this alterationwould simply cause an adjustment to the α factor. Sincegrowth increases α and hence the valuation, a company ex-hibiting a good quaterly growth rate should receive a morefavorable valuation.

In the case that A and L are small (the company doesnot have excessive assets or liabilities) in comparison to V ,we can approximate the equation by V = 1

1−αY . LettingS be the number of shares and P the price of a share, andE the earnings per share, we can rewrite the equation asP ·S = E ·S · 1

1−α or P/E = 11−α if the company is correctly

valued. Based on that, we may believe the company hasstrong fundamentals if its actual price-to-earnings ratio isbetter.

Economic sectors may vary in their risk-to-reward pro-file. The companies in some sectors, due to the nature ofhow they conduct business, have cash flows that are verysafe for a long period of time, whereas other sectors, such asthe technology sector for example, have large companies riseand fall relatively frequently, making some of these compa-nies less safe investments in relative terms. The choice ofα to properly value a company closely depends on the riskinvolved in each company. Estimating the risk in a particu-lar company is a difficult task and depends on many factors,but one possibility is to use the average risk for the sec-tor under the assumption that companies that compete in asimilar space sometimes have comparable risks. If the cor-rect value of α is represented relatively well by the averageP/E ratio in a sector (that is, the average company in thesector is correctly valued), the stocks that have better P/Ethan the sector average can be considered to have strongfundamentals.

We also take into account debt-to-equity ratio for compa-nies. High debt-to-equity can drive a company into bankruptcy,which increases the risk and therefore lowers α, since weshould discount future cash flows more steeply if the com-pany has higher risk. Since some sectors are more capital-intensive than others, we should also consider the debt-to-equity relative to the remainder of the sector.

Based on all of the above considerations, our fundamentalscoring algorithm assigns a positive rating to stocks depend-ing on how many of the following are met:

We assign buy scores to stocks depending on how manyof the following are met:

1. A P/E ratio of less than 20

2. A P/E that is in the top 25

3. A debt to equity ratio under 20 percent

4. A debt-to-equity ratio in the top 50

5. A quarterly growth rate of 5 percent or more

When the opposite conditions are true, we subtract pointsfrom the stock’s rating.

Such a formula is still relatively primitive because it onlytakes into account simple indicators; future work could ex-pand on the set of indicators considered.

4.3 Technical Analysis ModuleThis module’s role is to generate trading signals based

on technical analysis. The data required for this moduleis the daily price data that was extracted by the Data Re-trieval Module. The high-level workflow of this module isthat it first extracts the necessary raw price data from thedatabase. Then, for each day for which technical analysishas not yet been done, the system examines a sliding win-dow of price history, and applies an algorithm to detect eachtype of technical signal known to the system. If any tech-nical signals were detected, an indicator to buy or sell thestock is generated, depending on which signals were found.The algorithms we implemented for chart signal detectionare the main algorithms discussed in Lo et al. [15]. Whilemany books have been written on technical analysis (e.g.[17]), Lo et al. provides one of the clearest translations intoalgorithms of the criteria for identifying some of the mostpopular chart signals, criteria that other sources often statewith a lack of precision.

Lo et al. does not discuss, however, any methods for howone should trade based on technical chart signals. Their pa-per limits itself to giving algorithms for detecting technicalpatterns and an analysis of whether historical returns arestatistically independent of each of the patterns; evidenceis found for some patterns in favor of rejecting the null hy-pothesis that daily returns are independent of the pattern.In order to implement buy/sell recommendations based ontechnical patterns, we extend the work in Lo et al. with amethod for making trades based on detected chart signals;the Results section provides some support in favor of theapproach’s effectiveness.

We first look at the technical signal detection algorithms.Pursuant to the method of Lo et al., technical signals canbe seen as sequences of extrema (local maxima and min-ima) in price data that satisfy certain conditions. As such,a technical signal occurs over many days, and what is reallybeing detected is the culmination of the signal; that is, theoccurrence of the last extremum necessary to meet the con-ditions of a particular technical signal. Once the technicalpattern is complete, price movements are expected to followdepending on which pattern was observed. Therefore, thelogic to detect technical signals in a given stock’s prices isto examine, for each date, whether it is a local extremum,and if so, whether it completes a sequence of extrema thatconstitutes a technical signal.

One choice that has to be made is the length of price his-tory that must be used. We have chosen to look at slidingwindows of 50 days because many long-term technical signalscan require a timeframe of this duration to develop. How-ever, making the window longer than this results in manyspurious signals being detected. Lo et al. discusses the issueof window size in depth, though they do not use the samevalue for this parameter. Our parameter of 50 days shouldbe similar in intent to Lo et al. ’s 38 days, however, becausetheir analysis only used data from days where the stock ex-change was open. In our rough estimation, 38 trading daysis approximately equal to 50 total days. The analysis is nothighly sensitive to small changes in this parameter, hence weallowed ourselves some rounding. The start of the windowis 46 days before the given date; the end of the window is3 days after the given date. These 3 days are to allow timefor the detection date to be properly smoothed and ensurethat no false extrema are detected; this also means that a

technical signal is only confirmed 3 days after it completes.As Lo et al. discuss, before detecting extrema and scan-

ning the extrema for signals, it is important to first smooththe data to remove local noise. Since stock prices almostalways contain small, short-term price fluctuations whichoften have the appearance of random walks, many highlylocal extrema will be detected if the data is not smoothed.These extrema are in some sense ”false”; they do not reflectlong-term trends. It is therefore important to smooth thedata substantially so that any extrema still remaining aftersmoothing accurately represent the culmination of longer-term price trends.

The smooth price at time t is computed by taking a weightedaverage of the raw prices in a neighborhood close to t. Thatis, we can conceptualize that the raw prices between timet − δ and t + δ for some reasonable δ will be part of thecalculation for the smooth price at time t. Following Loet al. we choose a Gaussian convolution for the weights inthe weighted average calculation. The Gaussian function isgiven by the equation:

G(x) =1

σ√

2π· e−

x2

2σ2 (4)

where σ is the standard deviation of the Gaussian, and x isthe distance from the mean of zero. The choice of σ dependson the amount of smoothing one wishes to perform, whichin turn depends on the noisiness of the data. We discuss ourchoice of smoothing in the System Implementation.

The Gaussian function given above is normalized, but be-cause it is being applied on a discrete sample, the weights inthe weighted average calculation should be summed and di-vided by. Otherwise, there could be a small bias to the pricesmoothing that would adjust all prices slightly upwards ordownwards; this is clearly undesirable because it gives a falseperception of a stock’s value.

Using the Gaussian shown above as a convolution, thesmoothed price s(t) for a time t has the following values,letting the raw (unsmoothed) prices with respect to t berepresented by r(t):

s(t) =

i=N∑i=0

w(i, t) · r(i)

i=N∑i=0

w(i, t)

(5)

where w(i, t) is the weight of the raw price at time i inthe calculation of the smoothed price for time t:

w(i, t) = e− (t−i)2

2σ2 (6)

After the data is smoothed, the algorithm finds all theextrema in the smoothed data. It then determines whetherthe detection date is an extremum. If so, the detection datemay be the culmination of a technical signal and furtherexamination is required. The extrema in the smoothed dataare then checked for the conditions they must meet for eachkind of technical signal. For example, a broadening bottomconsists of five consecutive extrema E1, E2, ..., E5 such thatE1 > E3 > E5, E2 < E4, and E1 is a minimum.

Lo et al. spends considerable space defining all of thetechnical signals that we detect in their paper. We use theirdefinitions for all of the signals. While we detect all the

signals that they did, we use only the following ones to makerecommendations, because the other signals did not generateprofitable trades during back-testing:

1. Head and shoulders - Sell recommendation

2. Inverted head and shoulders - Buy recommendation

3. Double top - Sell recommendation

4. Double bottom - Buy recommendation

5. Broadening Top - Sell recommendation

6. Broadening Bottom - Buy recommendation

7. Triangle Top - Buy recommendation

8. Triangle Bottom - Sell recommendation

We added sell or buy recommendations to the above sig-nals in accordance with the technical analysis textbookscited in Related Work.

Once the technical signals are detected, the technical scoreis computed as follows. The base score starts at 50. It isthen modified by 25 multiplied by a signal multiplier (be-tween 0 and 1) and 25 multiplied by a recent performancerating (between 0 and 1), giving a possible minimum of 0and maximum of 100. There are 3 cases to consider:

1. If a signal is detected that day, it is automatically givena 25 point boost for a buy signal or a 25 point declinefor a sell signal.

2. If there is no signal that day, but there was a signalwithin the last 5 trading periods, the base score is mod-ified by +/- 25 · ((6− d)/6) where the sign depends onwhether the signal is a buy signal or a sell signal, andd is the number of trading periods since the last signal.The score is then modified by a further 25 multipliedby the percent change in price from the last signal.

3. If there was no signal that day, and no recent signal,the base score is modified by 25 multiplied percentchange in price from 10 trading periods earlier.

4.4 Sentiment Analysis ModuleThis module employs sentiment analysis to detect the

sentiment expressed about a stock. It uses the human-generated content captured by the data retrieval module,which is already tagged with information about the stocksymbol and date to which the content pertains. For a givenstock and date on which the sentiment module gauges thesentiment, the relevant data is retrieved from the databaseand passed into the sentiment analysis algorithm.

This logic assumes that sentiment is short-term; morespecifically, the exact assumption is that the best estimateof the sentiment pertaining to a stock on a given date isthe sentiment expressed by content on the same date, with-out taking content from past dates into account. The intu-ition behind this assumption is first that the market reactsfairly quickly to news, and second that if there have beenrecent events that have positively or negatively affected thesentiment about a stock, the sentiment at the present timeshould already capture that information to the extent thatsuch information continues to be relevant; there is thereforeno need to consider content from earlier dates to determine

the sentiment. This assumption seems to be particularly ap-propriate to content that is generated with high frequency,such as the Twitter data that we used in our implementa-tion. While most or all sentiment is short-term, it couldmake sense in future work to consider data from past datesto some extent, for example for articles that are updatedonce every few days.

The sentiment analysis algorithm is designed to have threeparts. First, the sentiment evaluator lexes, splits the datainto sentences, and determines the sentiment of each sen-tence. This component can have any implementation, aslong as it is capable of splitting a file of text into sen-tences and extracting the sentiment of each sentence. Inour case, we use the algorithms implemented in the Stan-ford CoreNLP library [16] to carry out these tasks, as Stan-ford CoreNLP contains implementations of many naturallanguage processing algorithms that have very competitiveperformance. For its sentiment classification module, Stan-ford CoreNLP uses a classifier due to Socher et al. [20].The approach uses the idea of parsing the sentence into asyntax tree and assigning a sentiment to each node in thesyntax tree in a bottom-up recursive manner, inferring howto combine subtree sentiments with a neural network.

The second portion of the sentiment analysis algorithmconsists of a sentiment aggregator that combines the senti-ment results from all the sentences that have been classifiedby the sentiment evaluator into an overall sentiment scorefor the stock symbol and date pair. This task is accom-plished by taking a weighted combination of the sentencescores:

V =

∑s∈S

w(s)v(s)∑s∈S

w(s)(7)

Where V is the overall score, S is the set of all sen-tences pertaining to the same stock and date, w(s) is theweight of a given sentence s, and v(s) is the sentiment valueof a sentence s. The sentiment evaluator returns a senti-ment classification label (for example, StanfordNLP returnsa sentiment in the set ”VeryNegative”, ”Negative”, ”Neutral”,”Positive”, ”VeryPositive”). The sentiment aggregator wouldconvert such scores to a 0-100 scale to match the range usedby all other analysis modules; the exact values to use forthis conversion depend on the sentiment evaluator. For thesentence weights, one possibility is to use a uniform weightof 1 for each sentence. Another possibility is to use a weight-ing scheme that assigns larger weights to longer sentences,under the assumption that they are likely more significant.The best weighting and label-to-score conversion formulasdepend on the data and the properties of the sentiment eval-uator component; we discuss our implementation based onour experience with Stanford CoreNLP in the System Im-plementation section.

During its execution, the sentiment aggregator also calcu-lates some additional information, such as the variance andhistogram of the sentence sentiments it encountered. Thisinformation is stored together with the overall score.

The third and final component of the module, the descrip-tion generator is responsible for generating natural-languagedescriptions for the ratings, based on the information ob-tained from the sentiment aggregator. This component usesa rule-based system to fill in natural-language sentence tem-

plates with values from the analysis. This information servesto permit the user to make better use of the data as a resultof having more detailed information. For example, whensentiment is positive for a stock, the user may wish to knowwhether it is uniformly positive, or whether there is a largeamount of positive sentiment mixed with some negative sen-timent. The former scenario may be the result of a generalpositive perception of the company; the latter could be theresult of a controversy where the company appears on aver-age in a positive light.

Here is an example of a description generated by the de-scription generator: ”The general opinion about this stock isneutral (52/100 sentiment rating) with a moderate standarddeviation (20 rating points). The overall sentiment distri-bution was Positive: 12 %, Neutral: 87 %, Negative: 1 %”.

Once the sentiment scores and overall descriptions havebeen generated, they are stored in the SQL database fromwhich the website backend loads its data for display to users.

4.5 SummarizerThe Fundamental Analysis Module, the Technical Analy-

sis Module, and the Sentiment Extraction Module all storescores on a 0 to 100 scale indicating whether the stock isto be sold or bought. Once all three modules have run, thesummarizer is responsible for determining which of thesescores and trading signals is relevant to the user. Currently,this is done simply by running a database script to aver-age the three ratings and link the data from each moduleto the overall score, which is then stored in the appropriatedatabase table to allow the score to appear on the webiste.An interesting direction for future work would be to weighthe analyses based on the preferences of the user. This way,users who do not believe or do not wish to consider certainkinds of analyses could exclude those analyses from consid-eration when receiving an overall stock recommendation, orusers could have the website weigh their preferred type ofanalysis more heavily.

4.6 WebsiteThe website hosts a web application that presents all of

the relevant data for viewing by the user. It is the onlyway in which the user interacts with our system, since thesystem is intended for lay users. Through the website, theuser can search for stocks and view their analyses, accessingthe information generated by the above modules.

5. SYSTEM IMPLEMENTATION

5.1 Languages and TechnologiesFor writing the code for the modules, the data retrieval,

fundamental analysis, and technical analysis modules areimplemented in Python 2.7 to take advantage of its dynamically-typed nature and ability to easily run scripts. AlthoughPython 3 has been released, many external libraries havenot been updated to use Python 3 yet, so Python 2.7 waschosen due to compatability considerations. Additionally,creating connections from Python code to MySQL is madesimple through the python-mysqldb library, allowing us tostore any data we require. Integrating many of the otherAPIs we wish to use, such as the Twitter API to obtain ourdata for sentiment analysis, also proved easy with Pythonsince Python wrappers already existed. The relatively lowercomputational speed of Python in comparison to compiled

languages like C is not currently a concern because our run-time is dominated by the time taken to load data from,and store data to, the database. If the databases were dis-tributed and scaled out, this operation would become slowerstill in comparison to local calculations.

The sentiment analysis module was implemented in Javabecause Stanford CoreNLP is written as a Java API. It wassimplest to use the same language to interface with it. Sinceour top-level scripts that run all the modules for each day arein Python, a script was created to furnish the inputs to oursentiment analysis module as files, from where the sentimentanalysis module picks up the files, processes them, and thenwrites the outputs to another set of files from where thetop-level scripts read the ratings.

5.2 Data StorageWe chose to store our data in a MySQL database. MySQL

is a simple database solution that has the advantages of be-ing readily understood by developers and achieving a decenttradeoff between expressiveness and performance. The maindisadvantage of MySQL is that it involves all the data beingstored on a single machine. This sufficed for our purposes;however, we did not scale the system to the full range ofstocks available on stock exchanges throughout the world.If the system needed to scale further, it would not be toodifficult to implement a system where the data is shardedbetween multiple machines (and therefore databases). Thesharding could be done based on any number of criteria,with sharding based on the stock symbol being simplest,since data for each individual stock is usually examined sep-arately by our system (and combined by the web applicationin the end).

We chose MySQL Community Server, because it has theadvantage of providing easy server management throughMySQL Workbench. MySQL Community Server makes itsimple to update a database in real-time and then obtaindumps of the schema, so that the final database schemaused in our testing can then be captured and checked in tothe source depot to become the database schema used forinstalling the project from scratch.

5.3 Fundamental AnalysisAt first we used the Yahoo! Query Language (YQL)

API to retrieve detailed quaterly financial information aspublished by publicly traded companies. We used the ya-hoo.finance.balancesheet, yahoo.finance.incomestatement, andyahoo.finance.cashflow tables to obtain financial statements.For price data, we used the yahoo.finance.historicaldata (long-term price data) and yahoo.finance.quotes (shorter-term pricedata) tables. While we continue to use quote and historicalprice data from YQL, we experienced a problem with thefinancial statements when YQL suddenly failed to continueto supply that data. Upon closer examination, it was de-termined that this occurred because, surprisingly, the APIoperated by scraping a Yahoo webpage containing the infor-mation. To rectify the problem, we implemented our ownscraper capable of parsing a portion of the same data. Thistype of problem is somewhat representative of the difficultyof finding reliable and free-of-charge APIs to retrieve finan-cial data.

While we implemented tools to scrape some of the samedata, the difference in data formats has led to our imple-mentation of the fundamental analysis algorithm described

under System Model to be currently incomplete.

5.4 Technical AnalysisThe price data for technical analysis is retrieved from the

YQL API described above. The only data required for thecurrently implemented indicators is end-of-day price dataover a long period of time. While the technical analysismodule examines signals over a relatively short timeframeof 50 days, data from 2007 onwards was downloaded to ad-equately backtest the system.

We implemented the algorithm described for technical anal-ysis in the System Model section and generate technicalanalysis scores based on it, as discussed in the Results sec-tion.

To determine the best smoothing factor to use for thedata, we back-tested the analysis for a variety of potentialsmoothing factors. In general, we found that a smoothingfactor of 1 produces reasonable results. Lo et al. [15] recom-mends using a method called cross-validation to determinethe best smoothing factor. This method involves finding thevalue of σ that would result in the least squared error whenthe smoothed observations are used to predict each pricebased on the weighted averages of the neighboring prices,in accordance with the Gaussian weighed averages. Lo etal. reported that this method produced curves that weretoo smooth for them, and they multiplied the result of thecross-validation by an arbitrary scaling factor to ensure dataretained sufficient granularity. We found that on the con-trary, cross-validation resulted in very little smoothing. Inour final analysis of the results, we consider a variety ofpossible smoothing factors. A larger smoothing factor rep-resents in some sense the extent to which the user wantsto make more long-term trades and detect more long-termsignals. The more smoothing is applied, the fewer extremathe smoothed data has, and the more the detected patternsrepresent long-term trends.

5.5 Sentiment AnalysisOur implementation uses Twitter as the source of sen-

timent data. We reason that Twitter should be effectivein representing sentiment because it has such high trafficand users are constantly engaged in sharing their opinionson the platform. Twitter also has a fairly feature-rich andeasy-to-use API for retrieving Twitter posts (tweets). Formost other sources of news or opinions, the data would needto be scraped from web pages, which is a far more fragileimplementation.

The portion of the data retrieval module that providesTwitter search capability can issue arbitrary queries to theTwitter API. When searching for tweets related to stocks,we issue queries of the form ”$STOCKSYMBOL”. For ex-ample, the query ”$AAPL” would return all tweets with the”$AAPL” tag. On Twitter, this is the equivalent of a ”hash-tag” for stock ticker symbols; investors who frequently dis-cuss stocks on Twitter would know to use such symbols intheir tweets.

Tweets from more reliable and well-established sources area better indicator of public opinion considering the averageperson tends to base their views on people of authority. Themodule inherently takes this into account through the col-lection of retweets when collecting the raw data. A tweetthat has been retweeted several times will contribute moretowards the overall sentiment score in comparison to a lone

tweet written by an individual with a small number of fol-lowers.

Once the raw data has been obtained, automated datacleaning is performed on the tweets to remove portions thatare difficult for sentiment analysis tools to comprehend. Hash-tags (e.g. ”#mystockpick”), stock hash tags (e.g. ”$AAPL”),content containing long lists of stock ticker symbols (e.g.”$MSFT, $AAPL, $GOOG, $AMZN, $INTC”), URLs, andtags that indicate a tweet is in reply to a user (e.g. ”@User-Name”) are removed. Sentiment analysis is still not con-sidered to be a fully solved problem, and even advancedsentiment analysis algorithms such as the one supplied byStanford CoreNLP have significant error rates even on well-formed, concise sentences. The tendency of Twitter poststo contain spam, tags, and URLs, sometimes in seeminglyhaphazard places, make it very difficult for just about anysentiment analysis algorithm to operate properly; even ob-taining each sentence’s parse tree, a key preliminary step ofStanford CoreNLP’s sentiment analysis, may present con-siderable difficulty if the data is not cleaned.

In terms of the best weighting scheme to use in the sen-timent aggregation component, after some testing of oursentiment analysis module on small pieces of sample text(such as a few Yelp reviews for example), we used the for-

mula w(s) =√L(s) where L(s) is the length of a sentence

s in characters. The intuition behind this formula is thatit provides a middle ground between uniform weighing ofsentences, which assigns too much weight to short, some-times one-word fragments or sentences that often containno meaningful sentiment value, and weighing each sentenceproportionally to its length, which causes small sentencesto carry very little weight. The square-root formula waschosen before any significant back-testing on the stock sen-timent data, and not modified afterwards. The approachis admittedly somewhat ad-hoc, since there are other pos-sible functions whose growth rate is strictly between Θ(1)and Θ(n), and refining the choice of weighing scheme is apossible topic for future work.

5.6 WebsiteThe website, www.phinanceanalytics.com, is optimized for

both mobile and desktop platforms to ensure a good viewingexperience on a variety of devices.

The user can interact with the site in a number of differentways. The first way for the user to interact with the websiteis to search for a specific stock symbol and receive a reportthat consists of all three forms of analysis in the trading sig-nals page. Figure 2 is a screenshot from the website of thetrading signals page for AAPL. The row above the graphsprovides a summary of the current recommendation, the in-dividual scores for each form of analysis, as well as the datethe analysis was last performed. The date signals the data’sfreshness to the user. In a production system, the analyseswould run as frequently as possible to keep the ratings asup-to-date as possible.

Each one of the graphs below represent the sentimentscore, the fundamental score, and the technical score as afunction of time so that the user can view the recent trendsassociated with each form of analysis for the stock in ques-tion. Below each graph, a brief explanation is provided bythe summarizer to help the user understand why the stockreceived the score provided. The purpose of providing thissummary is to provide the user with qualititative informa-

Figure 2: The trading signals page

tion to aid the user in understanding the reason for the rat-ing. The user can, and is encouraged to, use the reasons asa point of departure for further research into the stock andto determine whether they will ultimately invest.

For example, if the user learns that the sentiment abouta stock is very positive, the user may do further research tounderstand why the company is garnering so much positiveattention. The user may then judge for themselves whetherthey believe those reasons to be good reasons to invest inthe company, and make their investment decisions accord-ingly. For example, positive sentiment may be due to newsregarding the release of a new product, or may be the resultof a more vague and general positive public perception sur-rounding the company. Regardless of which of those reasonsis the cause, the user can do further research to determinewhether those signals warrant investing in the stock. If thegood news is a new product, the user may consider whetherthe information is already priced into the stock. A positiveperception of the company can be a good sign, but it couldalso represent crowd mentality and hype, which is difficultto judge without human analysis.

The second way for the user to interact with the website isto choose a specific form of analysis out of the three analysesconducted and view the summaries and scores pertaining tothat form of analysis alone. Figure 3 shows an example ofsuch an interaction through a screenshot of the SentimentAnalysis Page. In this view, the user can see the sentimentscore associated with several companies’ stocks. Upon click-ing on a specific company, the website will present the userwith a graph of the company’s score in the selected form ofanalysis as a function of time, along with a summary thatexplains why the company received the calculated score.

Figure 3: Sentiment Analysis Page

The third way for the user to interact with the website is

to go to the Trading Signals Page and select a sector to beanalyzed. The purpose of having this mode of view availableto the user is to provide the user with the top performingstocks according to the system’s metrics within the givensector. Figure 4 demonstrates this type of interaction usingscores calculated for the technology sector. If the user isinterested in investing in the technology sector, this viewprovides a good starting point, as the user can compile ashort list of the top performing stocks within the sector andconduct further research on those specific companies.

Figure 4: Technology Sector

The website is hosted on an Amazon AWS EC2 instance,and pulls the data from a MySQL database hosted on anAmazon AWS RDS instance. It is structured as a modulardesign based on two separate Maven projects, one for ourdatabase access, and one for our web access. The databaseproject, written in Java and encased in the Spring Frame-work, houses the code related to data insertion, retrieval,or modification, using Hibernate for object-relational map-ping. This allows for a very flexible structure enabling thedatabase layer to change, while preserving a public interfacethat allows the web project to query data without needingto know how that data is retrieved. The web layer housesall code necessary to process web requests and retrieve theappropriate data through the public interfaces provided bythe database layer. The web layer, also written in Javaand wrapped in the Spring Framework, allows for simpleauthentication and authorization using the Spring SecurityFramework. In this manner, we have the ability to imple-ment customizations of the web experience to each user’spreferences and saved settings. Additionally, this two-tieredstructure allows us to change the front-end display of thewebsite, without needing to recompile the database layer,saving time and increasing availability. For deployment, theweb layer is compiled into a WAR file, and deployed to aWildfly 8.2 server instance running on EC2, allowing publicaccess to the website.

6. RESULTSTo test the profitabiltiy of our individual modules, as well

as our overall recommendations, we implemented a simplebuy/sell strategy. We buy one share upon our algorithmsindicating that a trade is profitable, and then sell that stockon the next day that our algorithms indicate that the stockis no longer profitable. If we do not have a stock to sell,the sell order is ignored; this simulates the buying habits ofcasual retail investors who mostly buy and sell stocks anddo not short sell or execute more complex trades as often.

The results for our sentiment analysis and fundamentalanalysis were inconclusive due to several factors. Due to

limitations with the Twitter Search API, we were able toget tweets only for a relatively short range of time. Sinceusers cannot buy or sell over the course of a weekend, bothbuy and sell dates would need to be exclusive of weekenddays, which reduced the number of buy and sell transac-tions recommended by the system. Finally, despite all thedata cleansing we performed on tweets, the initial qualityof tweet content was very low. It would seem that, per-haps unlike in the days of older work such as Bollen et al.[6], Twitter has become increasingly full of spam. Inspec-tion of a sample of the tweets downloaded that containedstock ticker hashtags such as ”$AAPL” shows that many ofthe messages are spam, generic messages promoting manystocks, and in some cases are messages only weakly relatedto the stock. More sophisticated methods of filtering tweetsto eliminate spam messages may be required in order to ef-fectively leverage Twitter for sentiment analysis. Anotherapproach would be to leverage sentiment sources other thanTwitter.

For the technical analysis, we back tested our algorithmon data from Jan 1, 2007 until Apr 28, 2015 in order todetermine the performance of our modules. Due to APIlimitations and recent IPO dates, GOOG data runs fromMar 27, 2014, FB from May 18, 2012, and LNKD from May19, 2011.

The results for our chart pattern detector analysis are de-pendent upon the smoothing factor used, and as such, weshow the top ten most profitable smoothing factors per stockin Table 1. Not all stocks have ten profitable smoothingfactors, and some do not have ten smoothing factors thatlead to any trading. Focusing on the top few best perform-ing smoothing factors per stock, the returns are very highacross the board. To show the performance in an economicdownturn, Table 2 displays the data from Table 1 that oc-curs between Jan 1, 2007, and Dec 31, 2009, which spans theofficial timing of the Great Recession. Our returns are stillvery strong against the economic downturn, and far exceedthe market returns during the same period. Another impor-tant factor to consider is the profit per trade, due to the hightrading fees charged by outlets available to retail investors.With every stock, we can find at least one very manageableprofit per trade ratio that would enable even modest retailinvestors to turn a profit in spite of the trading fees. Toillustrate this example, Tables 3 and 4 show our same trad-ing algorithm, but buying 10 shares at once, instead of onlya single share, in the recession, and overall, respectively.With a budget of under $1,500, well within the means ofa retail investor, it is possible to create profits in excess ofthe standard $7-8 trading fees most retail outlets will offer.Additionally, the best few smoothing values per stock alsohave tremendous profitable trade ratios. Looking at solelythe chart pattern performance, independent of stock, Tables5 and 6 show the performance of each chart pattern over-all, and in the recession, respectively. Every chart patternwe detect, except for triangle tops in the recession, generatesprofitable trades. All together, these results present a strongcase that our chart detection algorithms, although simple,are effective at generating profits.

Because some of the referenced tables are large and in-clude data for each of 25 stocks that were analyzed, we havepublished the data for Table N at www. phinanceanalytics.com/data/tableN.

7. FUTURE WORKThe sentiment extraction module can be improved and ex-

tended in three main ways. Although the Stanford CoreNLPAPI is a state-of-the-art tool used to extract and examinesentiment, it has one large drawback that stems from thefact that the training data used is not based on reviews ofstocks. This training data results in the API being particu-larly effective at determining the sentiment related to media,and arts and culture. Due to the nature of linguistics, cer-tain words and combinations of words can have a positivemeaning in one context, and a negative meaning in anothercontext. In order to ensure that the sentiment extraction isas accurate and reliable as possible for inputs pertaining tostocks, it would be useful to feed a set of stock-related inputsas the training data to the CoreNLP API. It can be impor-tant to train sentiment analysis algorithms on training setscontaining words from the domain, because otherwise wordsand phrases like ”bear market”, ”bullish”, or ”rally” mightbe judged as neutral by sentiment classifiers that have notencountered them in the training corpus.

Furthermore, this module could be extended to determinewhich entities are the object of the sentiment when read-ing sentences, in order to produce multifaceted sentimentanalysis that can distinguish between different aspects ofthe sentiment expressed towards a stock. In other words,the module could be used to gauge sentiment about a com-pany’s management, revenues, profits, products, and more.Google Product Search employs this type of sentiment anal-ysis. For example, it shows what reviewers have said about acamera’s lens, resolution, ease of use, etc. Blair-Goldensohnet al. [5] discuss a sentiment analysis system for productreviews with similar capabilities. Such an extension wouldgreatly strengthen the user’s ability to conduct further re-search on specific aspects of a company, enabling them tomake better informed investment decisions.

Moreover, the sentiment extraction module relies on tweetsas a source of input, but does not handle other forms of inputsuch as news articles, journal articles, blog posts, and othermediums. As discussed in the Results section, while Tweetsmay provide some indication of the public’s sentiment to-wards a specific stock, there was generally too much spam onTwitter to produce reliable sentiment analysis results thatwould result in profits. Capturing data from higher-qualitysources would mitigate much of the problem.

In addition to improving the analytical modules associ-ated with the system, the user experience on the website canbe further enhanced in several ways. Users already have theability to create accounts on the website. This customizationby user can be furthered by creating a page that providesthe user with stocks they may be interested in based on theirinvestment or viewing history. One way this could be imple-mented is to provide users with well-performing stocks thathave been viewed by users with a similar viewing or invest-ment history. Another possible direction for improving thewebsite would be to provide more detailed plots in some sec-tions. For example, the technical analysis page could showusers where technical indicators occurred on a stock pricegraph. A page could also be added that would show all theanalysis scores on one graph; this may be useful for compar-ing and contrasting analysis methods.

Finally, the platform as a whole can be extended to takefull advantage of the technical analysis conducted. As itstands, the platform provides users with a tool that helps

users make investment decisions on a day to day basis. Thetrue potential of investing based on technical analysis is un-locked when the user partakes in automated trading. Thesystem currently has the ability to conduct technical anal-ysis that could in principle be used for automated tradingby collecting intra-day price and volume values for stocks.Connecting this website with an automated trading platformwould make the website more useful and valuable, but woulddefinitely raise ethical issues pertaining to financial loss dueto too much dependence on a single source of investmentinformation.

8. ETHICSSince the website is intended to help inform users’ finan-

cial investments, the main ethical issue that arises is thepotential to harm users financially should the recommenda-tions be poor or the analyses unreliable. It is worth notingthat the website does not allow the user to invest directlythrough the platform, nor do we encourage investors to basetheir decisions entirely on the generated indicators. We haveattempted to make clear to users that Phinance Analyticsis meant to merely serve as a starting point for researchinto potential investments, and all trading signals and scoresshould be received by the user with a certain degree of skep-ticism, as there may be several conflicting views over thefuture movement of a stock price.

The purpose of the website is to help users narrow downtheir investment options and find a small number of com-panies to further investigate, rather than being burdenedby the abundance of noise and excess information availableon the web. There are often thousands of companies in asector. While large investment institutions will devote largeamounts of manpower to analyzing all of the notable com-panies in a sector, the average person investing on their ownbehalf cannot reasonably attempt to do the same. As such,Phinance Analytics generates trading signals to inform auser about where interesting profit opportunities may lie;further research and investigation by the user is expectedin order to determine whether the opportunity is in fact asgood as it appears from the available metrics. Experiencedinvestors are aware that no investment strategy, platform,or fund manager can ever be impervious to loss, and theyact accordingly by diversifying their portfolios and conduct-ing ample research before reaching an invesment decision.The true ethical dilemma thus lies with the inexperiencedinvestor who views the platform as a ”get rich quick”scheme,or as a shortcut to investment. Such an investor would prob-ably begin by making small investments that are consistentwith the recommendations provided by the website. If theseinvestments prove to be profitable, the investor might in-correctly assume that they have discovered a tool that isguaranteed to turn a profit, and would therefore make morebold investments that could result in large losses. In order toavoid such an ethical issue and to ensure the financial well-being of the users, users should frequently be encouraged toconduct further research, and the website can connect themto other platforms that can help them in this endeavor. An-other way to help combat this issue is to improve upon thesummary and explanation aspect of the platform to ensurethat the user is not taking the numbers and results at facevalue, but is instead using the tool in the intended man-ner, as an educational and research tool that can help themquickly discover potentially profitable stock picks and find

metrics on stocks they already hold or are already consider-ing purchasing.

A second ethical issue that arises pertains to the senti-ment extraction module. The module relies on the analysisof tweets, which are meant to provide a sample for the gen-eral population’s opinion on a certain company or stock. Itis possible for the tweets and input data to be manipulatedthrough the use of a tweet bot, or spam tweets that areintended to lead people to believe that the sentiment as-sociated with the company or stock is more positive thanthe reality. One way to mitigate such issues is to extendthe sentiment extraction module in the manner described inthe Future Works section; collect input data from outsidesources in addition to Twitter, and provide a breakdown ofthe sentiment score according to the aspects of the company(management, products, etc) that are viewed positively ornegatively.

9. CONCLUSIONSPhinance Analytics is a tool that is intended to empower

the retail investor by providing them with user-friendly anal-yses that are easy to understand and that require minimaltime, effort, and technical knowledge on the part of the user.The platform is geared towards automating much of the datacollection, analysis, and aggregation that would otherwisebe done manually. Despite the platform’s attempt to be ascomprehensive as possible, it is not meant to be used as thesole investment research tool by a retail investor, but ratheras a solid first step that can expose retail investors to themost profitable investment opportunities possible. If usedproperly, the tool can reveal profitable opportunities for itsuser base, while further research and due diligence can helpmitigate any losses that may result due to an overly opti-mistic analysis by the system.

10. REFERENCES[1] https://us.etrade.com//.

[2] https://www.scottrade.com//.

[3] George Athanassakos. The discounted cash flow-basedvaluation methodology as tested by a public markettransaction, 2005.

[4] Vanessa Guo Ben Blakeslee, James Ham. Automatedinvestment analysis, 2011.

[5] Sasha Blair-goldensohn, Tyler Neylon, Kerry Hannan,George A. Reis, Ryan Mcdonald, and Jeff Reynar.Building a sentiment summarizer for local servicereviews. In In NLP in the Information Explosion Era,2008.

[6] Johan Bollen, Huina Mao, and Xiao-Jun Zeng.Twitter mood predicts the stock market. CoRR,abs/1010.3003, 2010.

[7] M.K. Company, T. Koller, M. Goedhart, andD. Wessels. Valuation: Measuring and Managing theValue of Companies. Wiley Finance. Wiley, 2010.

[8] Sanjiv Das and Mike Chen. Yahoo! for amazon:Extracting market sentiment from stock messageboards. In In Asia Pacific Finance AssociationAnnual Conf. (APFA), 2001.

[9] Bloomberg Sales Department.http://www.bloomberg.com/professional/.

[10] A. Drake, E. Ringger, and D. Ventura. Sentimentregression: Using real-valued scores to summarize

overall document sentiment. In Semantic Computing,2008 IEEE International Conference on, pages152–157, Aug 2008.

[11] Benjamin Graham. The Intelligent Investor. Harper &Row, Publishers, Inc., New York, 4th revised edition,1973.

[12] Benjamin Graham and David Dodd. SecurityAnalysis. McGraw-Hill, New York, 5th edition, 1988.

[13] J. Johnson. Designing with the Mind in Mind: SimpleGuide to Understanding User Interface Design Rules.Elsevier Science, 2010.

[14] S. Krug. Don’t Make Me Think: A Common SenseApproach to Web Usability. Voices That Matter.Pearson Education, 2009.

[15] Andrew W. Lo, Harry Mamaysky, and Jiang Wang.Foundations of technical analysis: Computationalalgorithms, statistical inference, and empiricalimplementation. Working Paper 7613, NationalBureau of Economic Research, March 2000.

[16] Christopher D. Manning, Mihai Surdeanu, JohnBauer, Jenny Finkel, Steven J. Bethard, and DavidMcClosky. The Stanford CoreNLP natural languageprocessing toolkit. In Proceedings of 52nd AnnualMeeting of the Association for ComputationalLinguistics: System Demonstrations, pages 55–60,2014.

[17] John J. Murphy. Technical Analysis of the FinancialMarkets: A Comprehensive Guide to Trading Methodsand Applications. New York Institute of Finance, NewYork, 1st edition, 1999.

[18] W.H.C. Bassetti Robert D. Edwards, John Magee.Technical Analysis of Stock Trends. CRC Press, 9thedition, 2007.

[19] Richard Wallace Schabacker. Stock Market Theory andPractice. B.C. Forbes Publishing, New York, 1stedition, 1930.

[20] Richard Socher, Alex Perelygin, Jean Wu, JasonChuang, Christopher D. Manning, Andrew Y. Ng, andChristopher Potts. Recursive deep models for semanticcompositionality over a sentiment treebank. InProceedings of the 2013 Conference on EmpiricalMethods in Natural Language Processing, pages1631–1642, Stroudsburg, PA, October 2013.Association for Computational Linguistics.