Iosif ItkinCEOExactpro, LSEG

Machine Learning in Software Testing

Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com 2

Machine Learning inSoftware Testing

Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 2460, +1 415 830 38 49www.exactpro.com

Iosif Itkin, Co-CEO and Co-Founder, ExactproLondon Stock Exchange Group

Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com 3

Advances in Computer Science

Cat on a Screen

Bug in the Code

Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com 4

Technology Trends

Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com 5

Guess Who

Relentlessly Tortures the

Software

Prevents from being Released into the

Wild

Breaks It and Criticises it

Subject to Ruthless Soak and Negative

Testing

Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com 6

Global Trends

Job Loss?Death?

Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com 7

Contents

1) Software development life cycle

2) Key enablers for machine learning in testing

3) An approach to improve the test libraries

4) Challenges with real-time systems

Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com 8

Software Development Life Cycle

Requirements

Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com 9

Software Development Life Cycle

Requirements

Design & Code

Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com 10

Software Development Life Cycle

Requirements

Software Bugs

Design & Code

Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com 11

Software Development Life Cycle

Requirements

Design & Code Software BugsMagic

Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com 12

Software Testing Factory

Requirements & Code

Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com 13

Big Data in Testing

Identify requirements and areas most prone to defects in the next release of software

Predict the probability of a defect to be re-opened Highlight the area of

the code where the bug is most likely located

Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com 14

Dynamic Testing & Learning

Non-Functional Testing Tools

Passive Testing Capability

Build-in Testability in the System

Machine-Readable Specifications

Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com 15

Testing Algo Trading Systems

Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com 16

Machine Learning

Defects Database

Code Coverage

Mutation Testing

Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com 17

Real-Time Systems

Run All the Functional Tests Concurrently

Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com 18

Risk Controls

Google DeepMind published a paper claiming that it is possible to implement "a button" to interrupt undesired behaviour

Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com 19

Risk Controls

Will we have enough time to press the button once?

Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com 20

Thank You

Innovation

Excellence

Partnership

Integrity