machine learning and testing in london stock exchange group

Machine Learning and Testing in London Stock Exchange Group

Iosif Itkin, Co-Founder and CEO, Exactpro9 April 2016, AIST Conference

Quality Assurance & Related Software Development for Innovative Trading SystemsTel: +7 495 640 2460, +1 415 830 38 49www.exactpro.com

Exactpro Systems Company Overview

Quality Assurance & Related Software Development for Innovative Trading Systems Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com

Exactpro is: Clients Worldwide:• A specialist firm focused on functional and non functional

testing of systems that process wholesale financial products, particularly market infrastructure

• A UK company with operations in the US and four QA & software development centres in Eastern Europe

• Part of London Stock Exchange Group as of May 29, 2015

• Incorporated in 2009 with 10 people, our company has experienced significant growth as satisfied clients require

more services;

London Stock Exchange Group

• A leading diversified international exchange and infrastructure group with assets across the entire exchange value chain • The Group can trace its history back to 1698• Headquartered in the United Kingdom, with significant global operations, the Group employs approximately 5,500

people• Post trade and risk management services are a significant part of the Group’s business operations• Through FTSE Russell, the Group is a global leader in financial indexing, benchmarking and analytic services with

approximately $10 trillion benchmarked to its indexes

Tools & Methods of Program Analysis

Sailfish: Automated testing Compares intended and received results

ClearTH:• Automated end 2 end clearing system testing

Dolphin:Market surveillance testing

Shsha:• Automated monitoring, analysis and

reporting

Load Injector:• 75K messages / second from a single CPU core• Measures latencies in microsecond range

Minirobots:• A variety of algoes simulating end-clients

will run and see how system performs

Quality Assurance:test planning and

managementProcess audit and testcoverage analysis

Intelligent Managementof Large Data Sets

Test data management

Protocol level testing using FIX/FAST, SOAP, HTTP, ITCH, SWIFT, MQ, SQL, proprietary binary and text based data formats, etc.

Automated regression testing

Gathering requirements and test scenario creation (human, message & reporting interfaces)

Creating and productizing state-of-the-arttest harnesses

Test automation

Intelligent functional andexploratory testing

Latency & capacitytesting

Quality Assurance Trends

FrAgile Process

Crowd-sourced Testing

Formal Verification Methodologies

Cognitive Technology

FrAgile Process

Test is DeadIdeas before ProductsHappy Software DevelopersSoftware Engineer in Test

Crowd-Sourced Testing

Immediate Feedback from Diverse User BaseInstrumentation is Required to Process Data

Formal Methods

Model CheckingTheorem ProvingStatic AnalysisVerification

What’s Next

Can We Learn from Other Industries?

Be Both Faster and Safe

Who Had the Most Severe Problems with Technology?

Proposed Approach

Build Software to Test Software

Test Tool Sophistication should Match Platform’s Complexity

A Good Robot is Required to Survive in Robot Apocalypse

Trading Systems

Testing Instruments, Risk Control and Monitoring Tools should have the Same Priority as the Trading Platform

Testability is Required for Agile Process and Continuous Integration

SET = Building Software to Test Software

No Formal Artefacts from Agile Process. Parallel Stream for Test Harness

Formal Verification can Improve Quality

The Software will Break Anyway

The Absence of Adequate Monitoring and Kill Switches is What Turns a Problem into a Disaster

Passive Testing

Test Tools for Client Onboarding and Certification

Regular Test Automation

14 Quality Assurance & Related Software Development for Innovative Trading Systems Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com

Execute a test regression and make it: as extensive as possibleas efficient as possible

Minirobots

A set of Minirobots Algorithms allowing to perform a complete test regression of a trading system within a few hoursAll Robots work concurrently against the test environmentRobots are independent from each other, and their actions are based on current conditions on the order booksRobots use an abstract Test Library and their main goal is to cover all test cases within the library

Testing Algo-Trading System

16 Exactpro Systems, LLC Quality Assurance & Related Software Development for Innovative Trading Systems Tel: +7 495 640 24 60 , +1 415 830 38 49 exactpro.com

Deploy one or several scalable matching engines to serve as market simulators.

Repeat your tests multiple times to assess the quality and efficiency of your software

Machine Learning

Select efficient limited subset of tests from an infinite number of possibilitiesIdentify test cases that were actually executed and their status

Teach robots to adhere to our values: Integrity, Excellence, Partnership and Innovation

Human Learning

https://ru.linkedin.com/in/iosifitkin http://extentconf.com

machine learning and testing in london stock exchange group

Technology

rules of the london stock exchange derivatives … of the...

sbr - london stock exchange

easyjet plc - london stock exchange

quarterly report - london stock exchange

london stock exchange group

ericsson - london stock exchange

2009 - london stock exchange

orb listing guide - london stock exchange€¦ · orb...

lucky cement - london stock exchange

london stock exchange group - financial stability board ·...

agenda - london stock exchange

exploration report - london stock exchange

minco plc - london stock exchange

london stock exchange plc

margining overview for london stock exchange derivatives ......

disclaimer - london stock exchange

ftse - london stock exchange

london stock exchange group ir... · london stock exchange...

london stock exchange derivatives market

kxisbank - london stock exchange