launch & grow your test automation. complex application client/server financial sector –...

Launch & Grow Your Test Automation

Complex applicationClient/ServerFinancial sector – trading platformVery agile developmentQA

Manually testedOff the shelf automation

NeovestNeovest

Project RealitiesProject RealitiesLimited time to devote to developmentLittle experience in automationLimited budget and no testing engineers were available to work on the projectAutomation needed to maintain quality

Feature set growingContent scope was expandingSoftware release on aggressive schedule

People, Skills, InvestmentPeople, Skills, Investment If we had a larger budget,

we could hire more people

If we had more people or budget, we could quickly increase our skill sets

If we had a more mature skill set, we could save money

Time – major project killer

Our SolutionOur Solution

Controlling our budgetOpen source tools

Proprietary development

Tackling our time and people constraints

Choosing our tool sets

Launch Goal Launch Goal

Proof of Concept Test Prototype Ever Increasing

Capabilities

Automation EvolutionAutomation Evolution• Record and PlaybackRecord and Playback

• Procedural ProgrammingProcedural Programming

• Common LibrariesCommon Libraries

• FrameworkFramework

• AnalysisAnalysis

Record and PlaybackRecord and Playback Good

• Quick development of tests, quick to learn, immediate results• Proof of concept rock star – fastest tool

Record and PlaybackRecord and Playback Bad

Very susceptible to user interface changesNo easy result comparison mechanism

Ugly

Hard to maintain

Our Experience (Work in Progress) • No tests• User interaction log• Robo error reports

Record and PlaybackRecord and Playback

Good• Controlling the sprawl of monolithic

code • Reusable, maintainable, extensible

Bad• Susceptible to application changes• Basic reporting• Limited control for running tests

Procedural ProgrammingProcedural Programming

Our ExperienceBasic Result Reports

o Readable, easily accessedo Notification of test resultso Excel is fast, already built, easy to

import values

Best Diabolical Innovation

Great for prototypes

Procedural ProgrammingProcedural Programming

Prototypes and BeyondPrototypes and Beyond

Basic Rule of Software Prototypes

Simple Compelling Metrics

Common LibrariesCommon Libraries

Proliferation of procedures Solid program design was needed

to improve efficiency

Forward Thinking Required in Program Design

Dragon’s Horde Sloppy program design spreads quickly Control Board


GoodStructure and organization

Increases efficiency and capabilities

Extends flexibility

Reduces development time

BadRequires coordinating groupVulnerable to poor planning


Our ExperiencePattern MatcherSimple Optical Character ReaderReporting Upgrade


Our ExperienceControlling the most dynamic elements ‘If’ statements based on version Object mapping

Version based mouse mapping tableLoose Integration with applicationTight integration with application


FrameworksFrameworks Good

Flexible enough to handle a wide of variety tests

Extensible enough to accommodate future changes

Once created – faster development, keeping common code changes to a minimum

Bad

More complex programming task• Changes are made by a few architects

Requires careful design and planning

FrameworksFrameworks

Our Experience

Slingshot

• Load tests as suites

• Schedule and stop tests

• Direct emailed reports

FrameworksFrameworks

FrameworksFrameworksOur Experience

Shotgun

• Load balancing

• Stop selected tests for replacement or reschedule

• Partitioning of environments

AnalysisAnalysis

Analysis = Brains

Determine cause

Many tests means much analysis

Our ExperienceTest failure patternsDatabase test data pointsPerformance Analysis• Track common events• Identify Trends• Quantify “slowness”

AnalysisAnalysis

Our Automation Our Automation Development Plan TodayDevelopment Plan Today

Our Automation Development Plan Today• Typical release cycle

Test 29,000 data points in 120 tests• Typical Development

3-5 day development cycle for standard tests

Your ExperienceExtensible, FlexibleRapid DevelopmentMinimal Maintenance

Cheating EvolutionCheating Evolution

QUESTIONS?QUESTIONS?

Copyright © 2012 JPMorgan Chase & Co. All rights reserved The views expressed herein do not necessarily represent the views and opinions of JPMorgan Chase & Co. This material is provided for information only and is not intended as a recommendation or an offer or solicitation for the purchase or sale of any security or other financial instrument. In no event shall JP Morgan be liable for any use by any party of, for any decision made or action taken by any party in reliance upon, or for any inaccuracies or errors in, or omissions from, the information contained herein and such information may not be relied upon by you in evaluating the merits of participating in any transaction. JPMorgan and its affiliates may have positions (long or short), effect transactions or make markets in securities or financial instruments mentioned herein, or provide advice or loans to, or participate in the underwriting or restructuring of the obligations of, issuers mentioned herein. Nothing in these materials constitutes a commitment by JP Morgan or any of its affiliates to enter into any transaction. Clients should contact their salesperson at, and execute transactions through, a JP Morgan entity qualified in their home jurisdiction unless governing law permits otherwise. JP Morgan is the marketing name for the investment banking activities of JP Morgan Chase & Co. and its subsidiaries and affiliates worldwide. J.P. Morgan Securities LLC. is a member of FINRA, NYSE and SIPC.

launch & grow your test automation. complex application client/server financial sector –...

Documents

playback slide

tool slide

test automation slide

efficiency slide

automation evolution

shelf automation neovest

tool sets slide

aggressive schedule