rice stock market simulator

Rice Stock Market SimulatorCOMP 415 Final Presentation

RICE

What is Real World Software Engineering?Real projects require real structure and strategyHierarchical team structureDivision of laborPlanning, specifications, and documentation

How does consulting work?Receive project requirementsDevelop specificationsVerify specifications with customerDevelop prototypesProvide demonstrations

Regular customer interaction is necessary

Technologies InvolvedMicrosoft Visual Studio 2005 Team Suite EditionC# .NET 2.0ASP.NET 2.0Microsoft Windows Server 2003Microsoft SQL Server 2005Microsoft Team Foundation Server

Microsoft SharepointFIX 4.2QuickFIX open source implementationLog4Net

Industry Customer HBK InvestmentsCustomer Representative: Matt Jeanes, Rice 06

Who is HBK?HBK is an investment management firm:Employing approximately 325 individuals around the worldBased in Dallas, TX with branches in New York, London, Tokyo and Hong Kong

Established October, 1991$13B in capital under managementRanked in the top tier of large, successful hedge fundsGenerated a compounded annual return of 14.68% since inception to June 2006

What is a Hedge Fund?Similar to mutual fundsRun by an investment management firm (fund manager)Investors pool their money together into one fundThe fund manager invests the money based on one or more strategieswith some key differencesCan invest in broader range of strategies and financial instruments (e.g., short selling, derivatives)Limited in the number and types of investors that can investHBK pursues a multi-strategy trading approach

What comes to mind when you think about working for a financial firm?

This?

What comes to mind when you think about working for a financial firm?

Or this?

22% of HBKs personnel work in technology!

Chart1

85

61

54

16

21

12

Sheet1

Accounting2

Financial Reporting

Management Company Reporting7

Management Company Reporting (Europe)1

Fund Accounting717

Admin SupportDallas8

NYC3

London2

Tokyo114

HR22

Internal Audit11

IR22

Legal & ComplianceLegal8

Compliance19

Operations37.537.5

Quant Research44

Recruiting55

Risk Management2.52.5

Securities Lending1212

Strategic Planning1.51.5

Tax & Upstream44

Treasury2.52.5

IT1515

Development3939

InvestmentsCredit22

Developed Markets FI3

Equities24

Private Placements & Other6

Vol Driven & Quantitative1469

Management Committee1212

249249

Investments, Securities Lending, Quant Research85Front Office (Trading, Quant Research etc)

Operations, Accounting, Treasury, Tax & Upstream61Operations & Accounting

Technology54Technology

Audit, Legal & Compliance, Strategic Planning, IR, Risk Management16Legal & Compliance, Audit, Risk etc.

Admin, HR, Recruiting21Admin, HR & Recruiting

MC12Management Committee

249

0.2168674699

Sheet1

0

0

0

0

0

0

Sheet2

Sheet3

Sheet4

Why does a hedge fund need all these technical folks? Software Development

Decision support tools for trading, risk management, portfolio analysis, accounting, etc.Computationally intensiveVery low error and downtime tolerance

Datafeeds & Databases

Multi-TB databasesMillions of updates every day from hundreds of sources around the worldTools for cleaning, monitoring, management, and reconciliation

Information Technology

Design, build, deploy, and manage global network, server, and workstation infrastructureWork with users to improve productivityAutomate systems management tasks

Quick FactsWe have more servers than we do staff!Our total data capacity is approx 40 terabytes with usage growing at around 10 gigabytes per day.Our databases maintain around 156 million historical prices from over 26,000 issuers. Our systems programmatically price over 170,000 options each day.Real time prices require DS3 (45Mbps)

Our Technology ToolkitMost of our systems and tools are Intranet-awareASP.NET 2.0 , IISApplicationsVisual Studio 2005, C++, C#, .Net FrameworkInfrastructureService-oriented architecture, Web Services, XML / XSLTDatabaseSQL Server 2000 (replicated and clustered), DTSOperating SystemsWindows XP, Windows Server 20033rd party tools and technologies

Technology InitiativesTwo tiers -> n-tiersDirect market accessBetter user experience C# GUIs for tradingExpanding data coverage and capabilityStraight through processingCash management systemPersonalization technology Infrastructure improvementsGigabit networking, WAN optimization, Voice-over-IPBlade servers and Storage Area Networks (SANs)

How has the project evolved?Hedge Funds operate on the worlds stock exchange systemsTesting trading strategies could be costly on the real marketHBK looked to Rices COMP 415 class for an alternative

Introduction to Stock ExchangesExchanges themselves have very simple behaviorTwo actionsBuy (Bids)Sell (Offers)ParametersSymbolVolumePriceTransactions occur when all three parameters of a bid and offer match

Introduction to the Rice Stock Market SimulatorProgrammatically mimic market function as closely as possibleAbility to replay historical market dataStore bid, order, and transaction data in a databaseUser interface for interacting with the market

HBKs New SpecificationsFill ParticipantControlled by an ASP.NET GUIExtend the functionality of the Human ParticipantImplement dynamic button configurationFIX Protocol Communication

Fill ClientWhat is a fill client?What were the requirements?Configurable fill parametersSize of a Fill (Full, Random, Integer)Time between Fills (Random, in ms)Load parameters from configuration fileChange parameters on the fly from a GUISave the modified parameters to a file

Fill Client BehaviorIf an order enters the fill client system, it will be filledOther clients may be acting on these ordersGenerate partial fills after the desired delay for the desired amountWhen to place the next partial fill?Identical orders?Market makes no distinction between orders for the same stock symbol at the same priceBut all identical orders must be filled eventuallySo the fill client needs to make sure it doesnt lose any of them

Fill Client AlgorithmEach new order generates one unit of demand from the fill client for the given stock at the given priceTwo bids for the same stock at the same price -> twice as many partial fillsEach partial fill has a target orderThe market does not guarantee that the partial fill will transact with its target orderFill client should add a target order back to the order queue when the associated partial fill is removed

Financial Information eXchange GatewayWe want to realistically simulate a stock exchangeFIX: an industry standard communications protocolFIX Gateway for any system to connect to RSMS

QuickFIX EngineOpen source implementation of the FIX protocolSeveral valuable features:Object oriented designLoggingError CheckingMessage Checking

How are Orders managed for a client?Listing of orders associated with each clientQuote Server interactionTransaction listener

Processing Node StructureEasy interface for increased functionalityMarket Validation RulesEnhanced simulation of Market behaviorRetrieved from MarketRulesTick SizeLot SizeValid StockNode 1Node 2Node 3

Live Demonstration

ASP.NET ClientFill ConfigurationFill SizeTime between FillsStatisticsActive usersStocksUser ManagementAdd, modify, delete usersOnly people classified as managers can do this

Human ParticipantGUI RedesignOrder form moved from popup to static on left side of the GUIPost bid/Post offer/Create Button forms combined into one

Human ParticipantDynamic ButtonsCreate new buttons given parametersStock SymbolVolume of StockStock PriceAll or some of the fields may be filled outBlank fields filled in when button is clickedButtons may be modified or deleted

Human Participant Demonstration

PerformanceThroughput of Bids/OrdersCapacity for Simultaneous Connections

Fill Client Performance

Chart1

23.2

9.6

20.2

34.2

12.6

1.2

Time

Transactions per Second

100 Orders, 500 Shares

30Apr2007-100ord-500share

Report TimeTransactions in IntervalTransactions/Second

63313563862255500011623.2

633135638672399000489.6

63313563872240000010120.2

63313563877240000017134.2

6331356388224010006312.6

63313563887240200061.2

63313563892240200000

63313563897240300000

63313563902240300000

63313563907240400000

63313563912240500000

63313563917240500000

63313563922240600000

63313563927240700000

63313563932240700000

63313563937240800000

63313563942240900000

63313563947240900000

63313563952241000000

63313563957241100000

63313563962241100000

63313563967241200000

63313563972241200000

63313563977241300000

63313563982241400000

63313563987241400000

63313563992241500000

63313563997241600000

63313564002241600000

30Apr2007-100ord-500share

Time




Chart2

0

9.8

13

9.2

20.2

13.4

16.8

21.2

11.4

12.4

21.8

16.2

9

20.8

13.8

15.4

20.2

11.8

16.4

13.4

19.4

14.6

17.6

18.6

14

19

12.2

18.8

15

21.6

21.4

19

Time



Chart1

0

9.8

13

9.2

20.2

13.4

16.8

21.2

11.4

12.4

21.8

16.2

9

20.8

13.8

15.4

20.2

11.8

16.4

13.4

19.4

14.6

17.6

18.6

14

19

12.2

18.8

15

21.6

21.4

19

Transactions/Second

Time



30Apr2007-500ord-500share-1


63313564350121100000

633135643551211000499.8

6331356436012120006513

633135643651213000469.2

63313564370121300010120.2

6331356437512140006713.4

6331356438012150008416.8

63313564385121500010621.2

6331356439012160005711.4

6331356439512170006212.4

63313564400121700010921.8

6331356440512180008116.2

633135644101218000459

63313564415121900010420.8

6331356442012200006913.8

6331356442512200007715.4

63313564430122100010120.2

6331356443512220005911.8

6331356444012220008216.4

6331356444512230006713.4

6331356445012240009719.4

6331356445512240007314.6

6331356446012250008817.6

6331356446512260009318.6

6331356447012260007014

6331356447512270009519

6331356448012270006112.2

6331356448512280009418.8

6331356449012290007515

63313564495122900010821.6

63313564500123000010721.4

6331356450512310009519

63313564510123100000

63313564515123200000

63313564520123300000

63313564525123300000

63313564530123400000

63313564535123400000

63313564540123500000

63313564545123600000

63313564550123600000

63313564555123700000

63313564560123800000

63313564565123800000

63313564570123900000

63313564575124000000

63313564580124000000

63313564585124100000

63313564590124200000

63313564595124200000

63313564600124300000

63313564605124300000

63313564610124400000

63313564615124500000

63313564620124500000

63313564625124600000

63313564630124700000

63313564635124700000

63313564640124800000

63313564645124900000

63313564650124900000

63313564655125000000

63313564660125000000

63313564665125100000

63313564670125200000

63313564675125200000

63313564680125300000

63313564685125400000

63313564690125400000

63313564695125500000

63313564700125600000

63313564705125600000

63313564710125700000

63313564715125800000

63313564720125800000

63313564725125900000

63313564730125900000

63313564735126000000

63313564740126100000

63313564745126100000

30Apr2007-500ord-500share-1

Time




Chart2

7.8

12.6

10.2

18.2

7

4.6

15.4

14.4

9

9.4

4

18.4

17.8

7.4

4.6

14.8

20.8

3.6

9.6

11

16.4

11.8

8.8

4.6

17.6

18

7

6

11.6

16.6

10.4

9.8

9.4

13.6

15.2

8.4

6

14.4

16

10.6

6

12

13.8

14.6

7.8

6.4

14.2

16.6

8.6

8.2

9.2

17.2

13

7

8.6

14.4

16

8.4

8.8

11.8

15.8

11.8

8.4

11.2

13.8

14.8

8.4

8.4

15.4

16.6

11

8.8

13.2

14

10.8

11.6

10

16

7

2.2

1.4

2

4.6

3

2

0.4

1.2

0.8

0.4

0.6

0.8

0.8

0.4

1

1

Time



Chart1

7.8

12.6

10.2

18.2

7

4.6

15.4

14.4

9

9.4

4

18.4

17.8

7.4

4.6

14.8

20.8

3.6

9.6

11

16.4

11.8

8.8

4.6

17.6

18

7

6

11.6

16.6

10.4

9.8

9.4

13.6

15.2

8.4

6

14.4

16

10.6

6

12

13.8

14.6

7.8

6.4

14.2

16.6

8.6

8.2

9.2

17.2

13

7

8.6

14.4

16

8.4

8.8

11.8

15.8

11.8

8.4

11.2

13.8

14.8

8.4

8.4

15.4

16.6

11

8.8

13.2

14

10.8

11.6

10

16

7

2.2

1.4

2

4.6

3

2

0.4

1.2

0.8

0.4

0.6

0.8

0.8

0.4

1

1

Transactions/Second

Time



30Apr2007-1000ord-500share-1


633135662114574000397.8

6331356621645750006312.6

6331356622145750005110.2

6331356622645760009118.2

633135662314577000357

633135662364577000234.6

6331356624145780007715.4

6331356624645790007214.4

633135662514579000459

633135662564580000479.4

633135662614580000204

6331356626645810009218.4

6331356627145820008917.8

633135662764582000377.4

633135662814583000234.6

6331356628645840007414.8

63313566291458400010420.8

633135662964585000183.6

633135663014586000489.6

6331356630645860005511

6331356631145870008216.4

6331356631645870005911.8

633135663214588000448.8

633135663264589000234.6

6331356633145890008817.6

6331356633645900009018

633135663414591000357

633135663464591000306

6331356635145920005811.6

6331356635645930008316.6

6331356636145930005210.4

633135663664594000499.8

633135663714595000479.4

6331356637645950006813.6

6331356638145960007615.2

633135663864596000428.4

633135663914597000306

6331356639645980007214.4

6331356640145980008016

6331356640645990005310.6

633135664114600000306

6331356641646000006012

6331356642146010006913.8

6331356642646020007314.6

633135664314602000397.8

633135664364603000326.4

6331356644146030007114.2

6331356644646040008316.6

633135664514605000438.6

633135664564605000418.2

633135664614606000469.2

6331356646646070008617.2

6331356647146070006513

633135664764608000357

633135664814609000438.6

6331356648646090007214.4

6331356649146100008016

633135664964611000428.4

633135665014611000448.8

6331356650646120005911.8

6331356651146120007915.8

6331356651646130005911.8

633135665214614000428.4

6331356652646140005611.2

6331356653146150006913.8

6331356653646160007414.8

633135665414616000428.4

633135665464617000428.4

6331356655146180007715.4

6331356655646180008316.6

6331356656146190005511

633135665664619000448.8

6331356657146200006613.2

6331356657646210007014

6331356658146210005410.8

6331356658646220005811.6

6331356659146230005010

6331356659646230008016

633135666014624000357

633135666064625000112.2

63313566611462500071.4

633135666164626000102

633135666214627000234.6

633135666264627000153

633135666314628000102

63313566636462800020.4

63313566641462900061.2

63313566646463000040.8

63313566651463000020.4

63313566656463100030.6

63313566661463200040.8

63313566666463200040.8

63313566671463300020.4

63313566676463400051

63313566681463400051

30Apr2007-1000ord-500share-1

Time



Benefits of the RSMS SystemAllows trading strategies to be tested on historical dataSimulated Market Time allows long histories to be run at a higher velocityFill strategies can be developed using a feedback loop generated by interactions with real data running in syncAllows developers to test trading software under simulated market conditions

Special Thanks to HBK InvestmentsQuestions?

Comp 415 puts students in the drivers seat of a pretend software consulting company working for a real customerTraditional university level computer science class projects typically only involve one or two students per groupIn the real world projects are much larger, and thus take more manpowerWith larger teams, such as the 10 person COMP415 class, a hierarchical structure must be used to keep organizedWorking in a larger team requires much more planning so that a specification can be drawn up and used for development so each team stays on trackAdditionally, documentation of all work is that much more important should others need to understand an unfamiliar piece of the codebaseIn the real world, software consulting companies are hired to solve a particular problemThis involves regular interaction with the customer, first meeting to understand the problem and receive a basic outline of requirementsThen the team will go back and develop a set of specifications along with a timeline of what will get accomplished and whenThese specifications get verified with the customer and if satisfactory, the team begins developing the product, attempting to produce prototypes along the way in order to give the customer an idea of the progress being made 1.) so that the customer can see that something is getting done, and 2.) so that the customer can verify that the team is working in the proper directionFinally, once a solution has been developed to a satisfactory point the final product is packaged, demonstrated and delivered to the customerSo what tools did the class use to accomplish its goals for the semesterIndustry standard tools from Microsoft that are used at many large corporations including HBK

Visual Studio gives us a professional development environment to work in to write code in C# and ASP using the .NET platformWindows Server 2003 gives us a stable platform to run our services onSQL Server gives us an effective means for efficiently storing dataTeam foundation server source control allows for all source code to be deposited into one repository for versioning and redundancy, allowing the team to code such a large project much more easilySharepoint is used for collaboration: posting announcements, specification documents, maintaining calendars and timelines of events, etc.FIX is the Financial Information Exchange protocol which is the industry standard protocol for communicating financial informationLog4Net is an open source logging package that allowed us to maintain a central log system, which is especially useful for bug tracking

So now that Ive given you a little bit better idea of how the class operates, I would like to introduce the representative from our industry customer HBK Investments, Matt Jeanes who graduated from Rice just last yearSo, how did HBK investments and Comp 415 get to where we are today?

A hedge fund buys and sells an astonishing amount of securities and their derivatives on a daily basis. Millions of shares and dollars can be traded on a single day through the worlds stock exchanges: NYSE, Nasdaq, Euronext, etc.

If HBK develops a new trading strategy the current method for testing this strategy is to use it on a live market. This opens the possibility for a strategic flaw costing millions of dollars.

HBK tasked last years Comp 415 class to build an alternative, a stock market simulator that acts like a real stock exchange.

So the first question is how does a real stock exchange work at its core. In essence the exchange facilitates the meeting of buyers and sellers. It provides a black box where someone wanting to buy 1000 shares of GE at $35 can find sellers rapidly.

An electronic stock exchange, like NASDAQ, allows this meeting of buyers and sellers to be done entirely within a digital medium.

In this way the Rice Stock Market Simulator is a simulation of an electronic stock exchange.

The original specification for the RSMS that we inherited consisted of two parts, the actual exchange and its interface.

The goal of the exchange simulation was to mimic the market function as closely as possible. Additionally, HBK requested that the stock market simulator have the ability to play back a recorded day of trading.

The interface was supposed to be extensible, so that multiple different types of participants could connect to the market.

Here is a graphical representation of the original RSMS. The three client applications all interface with the market in the same way.

The Human Client is a GUI application that allows a trader to buy and sell shares interactively as well as sift through information about the history of an individual stock or the current best bids and offers.

The market playback functionality was encapsulated in the concept of an ITCH participant. ITCH is the industry standard for saving order tracing in a stock market, so the playback client loads in a saved set of trades and then recreates them on the stock market.

The C++ Participant is a set of classes that was conceived to provided a programmatic interface between HBKs proprietary trading software and the stock market simulator.

Internals: EXTENSIBLE, EXTENSIBLE

This year HBK approached us with specifications for adding new features to the stock market simulator. Whereas last years class had the luxury of designing their project the way they wanted, this year we had the constraint of being forced to start with an unknown codebase.

First, HBK requested the addition of a Fill Participant which is an automated trading client. Its purpose is to automatically fill orders placed on the market with parameters specified in an ASP.NET webpage interface.

Second, we were to extend the Human GUI client to incorporate some features HBKs proprietary trading software, in particular that the GUI can be customized on the fly by adding new dynamic trading buttons.

Finally, HBK desired that the RSMS implemented a subset of the FIX Protocol, allowing our market to speak the language of the worlds stock exchanges. This is a superior solution to the original C++ class interface to the market for HBKs software.

This is how the RSMS looks after incorporating the new design. The Fill Client and FIX Gateway are new modules added to the internal structure of the market. Just like the quote server there can be multiple gateways or fill clients running on any number of machines.

The FIX Gateway is the new entry point to the market through which orders must go. In addition to translating FIX to the Web Methods used by the Market Web Service the Gateway also validates incoming rules against the stock markets rules. These rules are held by the Market Web Service and distributed to all FIX Gateways

The Fill Client, or Clients are configured using the ASP.NET web page and then listen for new orders from a Quote Server. When it receives notification of new orders, the Fill Client begins placing orders to the Market Web Service as it has been directed.

Now that youve got an overview of our system, Adam is going to explain the Fill Client in more detail.

Fill client represents the actions of the rest of the market, filling the orders that the user of the system placesFirst point is an invariant: if an order leaves the fill client system, it must have been filledCant just blindly submit next partial fill because other clients can act on the orders that youre trying to fillOne unit of fill client demand = one partial fill of the specified amount placed after the specified delay

Each partial fill is generated in order to fill a certain target order placed by a userBut the market has no notion of targetingTo make sure identical orders are filled, the fill client should have this notion of targeting and re-add the target order after its partial fill is goneSuppose that a user places a bid for 500 shares of stock, and the fill client fills 100 at a time

On this slide, explain the whole processOn this slide, emphasize the parts that have changed: looking up in the tableThis process repeats itself until order #1 is no longer in the market, thus ensuring that it is filled before it is removed from the fill client systemEach data point represents an average over a 5 second interval

This first graph is rather shortLonger, shows that the performance hovers in the mid-high teens

Kind of spiky; probably because the buckets dont match up with quote server updates, which come every 2 secondsSpikiness is even more apparent here

Average TPS seems to be dropping; several possible bottlenecks, including the size of the order queue and the load on the quote server/MWS

rice stock market simulator

Documents

financial firm

microsoft sql server

microsoft windows server

real projects

real structure

fundthe fund manager

financial instruments

quant research etcoperations