How Highly Effective Quality Assurance Techniques Can Make A Positive Difference In Delivering Large Post Trade Systems Initiatives

Post Trade Forum, Vienna 11 September 2014

Exactpro Company & Experience

Exactpro is:

2

• A specialist firm focused on functional and non functional testing of wholesale financial products systems

• An independent company incorporated in

2009 now employing over 240 specialists

Our locations in Russia:

• A US company with four QA & development centres in Russia and sales support in the UK

• Mostly engaged in trading platforms for exchanges and brokers, but also significant post trade experience

Introducing Your Speakers

3

Mark Ryland Strategic Adviser to Exactpro

• Financial Consultancy and QA of complex Risk Management solutions, including calculation algorithms

• BA for Collateral & Risk Management for investment banking;

• QA Delivery Manager for large Post Trade projects

Alyona Lamash Head of Exactpro Risk Management Practice

• IT Manager/CTO • Banking & Market Infrastructure• Trading and Post Trade• Insight and experience of QA

Talk Content

PART A

Challenges that Affect Large Projects - in General

How Great QA Techniques Can Help – Generically

PART B

Challenges that Affect Post Trade Initiatives – Specifically

QA Techniques that Help Post Trade Initiatives - Specifically

4

Testing Making a Positive Contribution

to the Whole

Responding to Issues/ Unknowns

Detailed Functional Understanding

Achieving & then Maintaining Test Automation

Speed of Test Execution

Clarity & Accuracy of Test Results and Defects Reporting

5

Testing a Large System Poses Challenges PART A:

Full Test Coverage

How Can Effective QA Be Part of The Solution, Not Part of The Problem

• GENERIC– Highly capable people with the right behaviours – Efficient and effective test process– Right use of technology – Independent & objective measurement of progress – RESULTS IN GREAT TEAM DYNAMICS ACROSS THE WHOLE PROJECT TEAM

6

Make a Positive Contribution to the Overall Team

7

Conception Initiation Development Test Implement

Project Sponsor

Project Manager

Business Analyst

Architect/Designer

Development Team

QA Team

Production Team

Make a Positive Contribution to the Overall Team

8

Make a Positive Contribution to the Overall Team

9

Pace

mak

er(s

)

1. High Quality Staff

The Elements of Highly Effective QA - 1

10

3. Energy, Enthusiasm &

Proactivity

2. Deep Domain

Knowledge

4. Creativity & Flexibility in Problem

Solving

Highly capable people with the right behaviours

8. Test Execution & Reporting

1. High Quality Staff

3. Energy, Enthusiasm &

Proactivity

5. Planning & Delivering Ability

2. Deep Domain

Knowledge

4. Creativity & Flexibility in Problem

Solving

The Elements of Highly Effective QA - 2

11

7. Defect Reporting & Evidence

Highly capable people with the

right behaviours

Efficient and effective test process

6. Test Case Quality

6. Test Case Quality

8. Test Execution & Reporting

10. Range of Excellent Automation Tools

9. Detailed Management of Test Environment

The Elements of Highly Effective QA - 3

12

7. Defect Reporting & Evidence

Right use of technology 1. High

Quality Staff

3. Energy, Enthusiasm &

Proactivity

2. Deep Domain

Knowledge

4. Creativity & Flexibility in Problem

Solving

Highly capable people with the right behaviours

5. Planning & Delivering Ability

Efficient and effective test process

Some of our Dynamic & Talented Team

13

Independence & Objectivity

14

1. Understanding of Requirements1. Understanding of Requirements

2. Delivery of Solution2. Delivery of Solution

3A. Independent Measurement by

QA Team

3A. Independent Measurement by

QA Team

3B. Independent Measurement by High Quality QA Team3B. Independent Measurement by High Quality QA Team

Developer Designer Business

Analyst Business

Manager End

Users

Team Dynamics When the QA Team is Excellent

15

DEVELOPERSDEVELOPERS

Try harder to get it right in the first placeGet rapid/timely feedback on what’s wrongWork harder to solve defects to keep the project on track

Try harder to get it right in the first placeGet rapid/timely feedback on what’s wrongWork harder to solve defects to keep the project on track

ANALYSTS AND DESIGNERS

ANALYSTS AND DESIGNERS

Have clear problem statements to work from –

with detailed supporting evidence

Have clear problem statements to work from –

with detailed supporting evidence

SPONSORS & PROJECT

MANAGEMENT

SPONSORS & PROJECT

MANAGEMENT

Have (justified) confidence in the system

Fewer problems are going to be found when Live

Project will cost less

Have (justified) confidence in the system

Fewer problems are going to be found when Live

Project will cost less

16

Features of Complex Post Trade Infrastructures PART B:

17

Features of Complex Post Trade Infrastructures

18

Collateral Upload

Risk Run

Margin Call

Risk Run

Trade InputTrade Input

Settlement Run

Trade Date

Settlement Date

Simulate DeliveryFailure

Trade InputTrade Input

Day 1

Examples of Test Scenarios (Before We Get to Specific Problems)

Penalties BUY INCash

Settlement

Risk Run

Trade Input

Day 2

Risk Run

Corporate Action

Trade Input

Day 3

Risk Run

Settlement Run

Trade InputTrade Input

Collateral Deficit

Delivery Default

Corporate Action

Delivery DefaultDelivery Default

6 Resulting Challenges From a QA Perspective

1) Testing of complex multistep scenarios2) Limited test availability of downstream and upstream

systems3) Most of endpoints are accessible via API or File exchange

(no GUI)4) Reference Data setup or migration5) Complexity of Risk Calculation Algorithms6) Regression Cycle for a substantial number of test

scenarios

19

Solving: 1) Testing of complex multistep scenarios

20

Solving: 1) Testing of complex multistep scenarios

21

Solving: 1) Testing of complex multistep scenarios

22

Solving: 2) Limited test availability of downstream and upstream systems

23

Solving: 2) Limited test availability of downstream and upstream systems

24

Solving: 3) Most of endpoints are accessible via API or File exchange (no GUI)

25

REQUEST RESPONSE

Solving: 4) Reference Data setup or migration

26

Reference Data required for Testing

Data Management Tools

Production Data Migration Tasks

Solving: 5) Complexity of Risk Calculation Algorithms

27

Test Model

Solving: 6) Regression Cycle for a substantial number of test scenarios

28

Test Library(100K Test cases)

Test Tools

Automated Test Scripts

Controlled Test Environment

Automated Regression Run(“The Big Button”)

A Holistic Integrated Automated Test Solution

29

Conclusion

30

Gre

at Q

A Te

am &

To

ols

for P

ost T

rade

Questions & Answers

Thank You

31