financial data mining and algo trading presented at the sas data mining conference in las vegas

Copyright ® 2009, SAS Institute Inc. All right s reserved.


Financial Data Mining with Algorithmic TradingRobert Golan

DBmind Technologies, Inc.Please Note: This is the view of DBmind only which may not

pertain to DBmind’s Client Views


Financial Data Mining with Algorithmic TradingAlgorithmic Trading has changed the world the way the

Traders trade and Trade Support supports. There is a Brave New World happening with the "hands on" Trading evolving into "hands off" Algo Trading. Not all trades need to be made in ultra low latency timing. Future trading will rely on a broader set of data which will be mined for relevance. An important series of XBRL Financial Reporting events are happening throughout the world and especially in the USA. A critical mass of financial data will be ready for mining which will be a boon for transparent "low touch" fundamental style algorithmic trading. Also, "low touch" trading such as program trading & direct market access (DMA) will evolve into advanced Algo Trading strategies. Stock and economic indicators combined with XBRL will add value for Algo Trading. This is about a well thought out strategic high latency trading strategy with data mining discovering the governing rules while adding the expert rules with validation. Yes, the trader is still the key to making this all happen. Both fundamental and technical trading rules need to be combined with the expert rules, the data mined rules, and most importantly the regulatory environment rules. RegNMS in the USA and MiFID in Europe have indirectly helped the adoption of electronic trading and it is important to integrate the GRC related rules in an agile way. Agility is the key and thus the rules need to be placed into a rules engine and managed by the experts for proper compliance, risk management, and governance. Japan, China, and the Netherlands with regards to XBRL are ready to be data mined with Algo Trading now. A XBRL US survey is indicating at least 340 of the estimated 500 public companies that the SEC requires to begin filing in XBRL format in June 2009, have already converted their financial statements into XBRL. XBRL US, is the non-profit XML standard setter that developed and maintains the US GAAP taxonomy used by filers to comply with the SEC mandate. Almost $7 trillion in market capitalization will be represented by this XBRL financial data which is over 50% of the total market cap for all publicly traded companies reporting to the SEC. As this XBRL Financial Data ripens, a wonderful harvest awaits us data miners which will enhance the current Algo Trading strategies which use this data.


Algorithmic trading http://en.wikipedia.org/wiki/Algorithmic_trading

• In electronic financial markets, algorithmic trading, also known as algo, automated, black box, or robo trading, is the use of computer programs for entering trading orders with the computer algorithm deciding on certain aspects of the order such as the timing, price, or type (market vs. limit, or buy vs. sell) of the order. It is widely used by pension funds, mutual funds, and other institutional traders to divide up a large trade into several smaller trades in order to avoid market impact costs or otherwise reduce transaction costs. It is also used by hedge funds and similar traders to make the decision to initiate orders based on information that is received electronically, before human traders are even aware of the information.

• Algorithmic trading may be used in any market strategy, including market making, intermarket spreading, arbitrage, or pure speculation (including trend following) to make the complete decision on entering trades and electronically executing the trade with no human intervention, other than in writing the computer program.

• In 2006 at the London Stock Exchange, over 40% of all orders were entered by algo traders, with 60% predicted for 2007. American markets and equity markets generally have a higher proportion of algo trades than other markets, and estimates for 2008 range as high as an 80% proportion in some markets. Foreign exchange markets also have active algo trading (about 25% of orders in 2006). Futures and options markets are considered to be fairly easily integrated into algorithmic trading, and bond markets are moving toward more access to algorithmic traders.


Citadel Investment Group

Interactive Brokers

Credit Suisse

Deutsche Bank

Goldman Sachs

Lehman Bros.

Morgan Stanley

Susquehanna Investment Group

UBS

Some of the Algo Players


Statistical Arbitrage in the U.S. Equities MarketMarco Avellaneda† and Jeong-Hyun Lee

First draft: July 11, 2008This version: June 15, 2009

AbstractWe study model-driven statistical arbitrage in U.S. equities. The trading signals are generated in two ways: using Principal Component Analysis and using sector ETFs. In both cases, we consider the residuals, or idiosyncratic components of stock returns, and model them as mean-revertingprocesses. This leads naturally to “contrarian” trading signals.The main contribution of the paper is the construction, back-testingand comparison of market-neutral PCA- and ETF- based strategies applied to the broad universe of U.S. stocks. Back-testing shows that, afteraccounting for transaction costs, PCA-based strategies have an average annual Sharpe ratio of 1.44 over the period 1997 to 2007, withmuch stronger performances prior to 2003. During 2003-2007, the averageSharpe ratio of PCA-based strategies was only 0.9. Strategies basedon ETFs achieved a Sharpe ratio of 1.1 from 1997 to 2007, experiencing a similar degradation after 2002. We also introduce a method to account for daily trading volume information in the signals (which is akin to using “trading time” as opposed to calendar time), and observe significant improvement in performance in the case of ETF-based signals. ETF strategies which use volume informationachieve a Sharpe ratio of 1.51 from 2003 to 2007. The paper also relates the performance of mean-reversion statistical arbitrage strategies with the stock market cycle. In particular, we study in detail the performance of the strategies during the liquidity crisis of thesummer of 2007. We obtain results which are consistent with Khandani and Lo (2007) and validate their “unwinding” theory for the quant funddrawdown of August 2007.Courant Institute of Mathematical Sciences, 251 Mercer Street, New York, N.Y. 10012USA†Finance Concepts, 49-51 Avenue Victor-Hugo, 75116 Paris, France.


XBRL Basics

XBRL is XML

It is Extensible

There is an XBRL specification – tells you how to use XBRL

Hinges on taxonomies – the dictionary of terms for business reporting – which includes financial statements


XBRL TaxonomyCreated by XBRL Consortium

Consumed

Rendered

XB

RL

Cre

ati

on

XBRL DocumentCreated by Preparer

TAGGING


Validation

Standardization

CalculationCash = Currency +

Deposits

CalculationCash = Currency +

DepositsFormulas

Cash ≥ 0Formulas

Cash ≥ 0Contexts

US $FY2004

Budgeted

ContextsUS $

FY2004Budgeted

LabelcashCashEquivalentsAndShortTermInvestment

s

LabelcashCashEquivalentsAndShortTermInvestment

s

ReferencesGAAP I.2.(a)Instructions

Ad Hoc disclosures

ReferencesGAAP I.2.(a)Instructions

Ad Hoc disclosures

PresentationCash & Cash Equivalents

PresentationCash & Cash Equivalents

XBRLItem

XBRLItemXMLItemXMLItem

XBRLItem

XBRLItem

PresentationComptant et Comptant

Equivalents

PresentationComptant et Comptant

Equivalents

PresentationGeld & Geld nahe Mittel

PresentationGeld & Geld nahe Mittel

PresentationKas en Geldmiddelen

PresentationKas en Geldmiddelen

Presentation现金与现金等价物

Presentation现金与现金等价物

Presentation現金及び現金等価物

Presentation現金及び現金等価物

PresentationДеньги и их эквиваленты

PresentationДеньги и их эквиваленты

PresentationГроші та їх еквіваленти

PresentationГроші та їх еквіваленти


The Business Reporting Supply Chain

ExternalFinancialReporting

BusinessOperations

InternalFinancialReporting

Investment,Lending, andRegulation

Processes

Participants

AuditorsTradingPartners

Investors

FinancialPublishersand Data

Aggregators

Regulators

Software VendorsSoftware Vendors

ManagementAccountants

Companies

XBRL XBRLXBRL

XBRLFinancial StatementsXBRL-GL

TheJournal

Standard

Transaction Standards

Collaboration is KEY!!!


USUS

UKUK

JPJPESES

SESE

CNCN

ZAZA

AUAU

DEDEDKDK

Financial Banking Regulators PilotPilot Committ

ed

Committed

KRKR

SGSG

FRFR

NZNZ

NLNL

LuXLuXPortug

al

Portugal

BEBEEU CEBSEU CEBS


XBRL Jurisdictions

UKCA

SPUS

AU

NZ

IR

JPKR

BE

VZ

CO

BR

AR

PT

RU

SG

HK

NOSE

PL

FI

IT

CN

IN

LB

CZ

UA

LUIASB

AE

NL

TR

GR

MTCH

FR

SI

HU

AT

Established Jurisdictions

Provisional Jurisdictions

in Construction & in Project

DE

DK

ZA


CACA UKUK

IEIE

AUAU

NONO

JPJP

NZNZ

NLNL DEDE

CNCN

Tax Authorities PilotPilot Committed

Committed

Tax XML Technical Committee recommends use of XBRL (Oasis-OECD)29 Tax authorities


Exchanges & Equity Regulators

Sao PauloSao Paulo

NZSENZSEASXASXJohannesburgJohannesburg

ShenzenShenzen

EuroNextEuroNextKOSDAQKOSDAQ

TokyoTokyo

SingaporeSingapore

SWXSWXLuxLux

Pilot LiveEval

TSXTSX

OBXOBX

LSELSECSECSE

DeutscheBörse

DeutscheBörse

Taipei

SECSEC

Korea

Korea

ShanghaiShanghai


Grant Boyd, [email protected] Technical Manager – XBRL, AICPA•http://www.icgfm.org/XBRLPresentations.htm


Data Mining Approaches

Three Basic Approaches to Data Mining

• Mathematical-based methods,

• Distance-based methods, and

• Logic-based methods

Methods may use supervised or unsupervised variable

• Supervised – induction rules for predefined classifications

• Unsupervised – rules and classifications determined by data mining method


Mathematical-based Methods Neural Network

• Network of nodes modeled after a neuron or neural circuit

• Supervised learning

• Weighted values at different nodes

• Mimics the processing of the human brain

• Form of Artificial Intelligence


Mathematical-based Methods

Discriminant Analysis• Similar to multiple regression analysis uses a non-

continuous dependent variable

• Approach identifies the variables (features or cases) that best explain the classification

• Supervisory learning approach

• Loses effectiveness with large complex data sets


Logic-Based Approach

Tree and Rule Induction • Supervised Learning

− Uses an algorithm to induce a decision tree from a file of individual cases

− Case has set of attributes and the class to which it belongs • Decision tree can be converted to a rule-based view. • Major advantage is ability to communicate and understand

information derived from this approach. • Prior research addressed audit areas of:

− bankruptcy, bank failure, and credit risk


Distance-Based Method

Clustering• Data mining approach that partitions large sets of data

objects into homogeneous groups

• Uses unsupervised classification where little manual pre-screening of data is necessary –

− useful in situations where there is no predefined knowledge of categories

• Classifications based on an object’s attributes

• Most commonly used in field of marketing but could be used in auditing


Selecting Data Mining Approach

Criteria:• Scalability - how well data mining method works

regardless of data set size • Accuracy - how well information extracted remains

stable and constant beyond the boundaries of the data from which it was extracted, or trained

• Robustness - how well the data mining method works in a wide variety of domains

• Interpretability - how well data mining method provides understandable information and valuable insight to user


Continuous Auditing, XBRL and Data Mining

Presenters:

Jennifer Moore, Lumsden & McCormick, LLP

Karina Barton, Canisius College

Dr. Joseph O’Donnell, Canisius College

New York State Society of Certified Public AccountantsTechnology Assurance Committee

June 15, 2004


Prime Services http://en.wikipedia.org/wiki/Prime_brokerage

• Prime Brokerage is the generic name for a bundled package of services offered by investment banks to hedge funds. The business advantage to a hedge fund of using a Prime Broker is that the Prime Broker provides a centralized securities clearing facility for the hedge fund, and the hedge fund's collateral requirements are netted across all deals handled by the Prime Broker. The Prime Broker benefits by earning fees ("spreads") on financing the client's long and short cash and security positions, and by charging, in some cases, fees for clearing and/or other services.

• The following "core services" are typically bundled into the Prime Brokerage package:

− Global custody (including clearing, custody, and asset servicing)− Securities lending− Financing (to facilitate leverage of client assets)− Customized Technology (provide hedge fund managers with

portfolio reporting needed to effectively manage money)− Operational Support (prime brokers act as a hedge fund's primary

operations contact with all other broker dealers)


Hedge Fund

Hedge Fund

Hedge Fund

Exchange Clearing Settlement

Trading

Position

keeping

Clearing

And

Settlement

Prime

Services


Hedge Fund

Mutual Fund

Hedge Fund

Exchange Exchange Exchange

Algorithmic

Trading

System

Prime Service Provider

Ice Berg

VWAP

TWAP


Client

Client

Client

Exchanges Clearing Settlement

Equities

Derivatives

Fixed

IncomeReference

Data

Validation

and

Enrichment

Risk

Management

Financial

Control

Clearing

and

Settlement


Date: 15 May 2007Produced by: Chris Swan


Overview of stock exchanges The main stock exchanges in the world include:

America• American Stock Exchange

• NASDAQ

• New York Stock Exchange

• São Paulo Stock Exchange

Europe• Euronext

• Frankfurt Stock Exchange

• London Stock Exchange

• Madrid Stock Exchange

• Milan Stock Exchange

• Zurich Stock Exchange

• Stockholm Stock Exchange


Listing requirements

LSE — main market has requirements for a minimum market capitalization of £700,000, three years of audited financial statements, minimum public float of 25 % and sufficient working capital for at least 12

months from the date of listing NASDAQ — to be listed a company must have issued at least 1.25

million shares of stock worth at least $70 million and must have earned more than $11 million over the last three years

NYSE — a company must have issued at least a million shares of stock worth $100 million and must have earned more than $10 million over the last three years


Overview of stock exchanges

Australia/Asia/Africa• Australian Stock Exchange

• Bombay Stock Exchange

• Hong Kong Stock Exchange

• Johannesburg Securities Exchange

• Korea Stock Exchange

• Shanghai Stock Exchange

• Taiwan Stock Exchange

• Tokyo Stock Exchange

• Toronto Stock Exchange


Participants Broker — an individual or firm which operates between a buyer and a seller and usually

charge a commission. For most products a licence is required.

Dealer — an individual or firm which buys and sells for its own account.

Broker/dealer — an individual or firm buying and selling for itself and others. A registration is required.

Principal — a role of broker/dealer when buying or selling securities for its own account.

Market maker — a brokerage or bank that maintains a firm bid and ask price in a given security by standing ready, willing, and able to buy or sell at publicly quoted prices (called making a market). These firms display bid and offer prices for specific numbers of specific securities, and if these prices are met, they will immediately buy for or sell from their own accounts.

Specialist — a stock exchange member who makes a market for certain exchange-traded securities, maintaining an inventory of those securities and standing ready to buy and sell shares as necessary to maintain an orderly market for those shares. Can be an individual, partnership, corporation or group of firms.


Prototypical trading systems Call (periodic) auction — selling stocks by bid at intervals throughout

the day. The orders are stored for execution at a single market clearing price.

Continuous auction — buyers enter competitive bids and sellers place competitive offers simultaneously. Continuous, since orders are executed upon arrival.

Dealership market — trading occur between principals buying and selling to their own accounts. Firm price quotations are available prior to order submission.

Auction markets are concentrated and order-driven

Dealership markets are fragmented and quote-driven


Examples

NYSE — opens with a periodic auction market and then

switches to a continuous auction. Same for Tokyo Stock Exchange.

NASDAQ and International Stock Exchange (London) are quote-driven systems (continuous dealership market).


Examples

Euronext Paris — the market is segmented into a number of different groups of stocks based on size and liquidity. The trading mechanisms vary depending on the segment.

Euronext 100, Next 150 ,CAC40 indices and stocks which have more than 2,500 order book transactions per year — continuous auction.

Other stocks — call auction twice a day.


Order types

Market order — immediate execution at the best price available when the order reaches the marketplace

Limit order — to execute a transaction only at a specified price (the limit) or better

Stop order

Good till cancelled

Fill-or-kill

All or None

Day order


Algorithmic Trading: An Overview of Algorithmic Trading: An Overview of Applications And Models.Applications And Models.

Ekaterina Kochieva

Gautam Mitra

Cormac Lucas


Summary Definitions and examples have been given about XBRL &

Financial Data Mining.

Key components to make this work are the global adoption of a Business Reporting language. This includes consistent standards being set by both XBRL US & XBRL International.

All companies need to be held responsible for their reporting with the current Reg rules adjusted. A global certification process is needed with the proper GRC engagement model followed. The good news is that there will be plenty of data to be mined.

How we mine and integrate this into our Trading Strategies is up to us. This is the "special sauce" which will make or break how our Trader trades & Trade Support supports.

financial data mining and algo trading presented at the sas data mining conference in las vegas

Economy & Finance

trading orders

robo trading

future trading

active algo trading

algorithmic trading

technical trading rules

sas institute

right s reserved