balancing test automation techniques

Click here to load reader

Post on 22-Feb-2016




0 download

Embed Size (px)


Agenda Two distinctly different pieces of test automation , both from the same project. What makes them different ? Why have both?. Balancing Test Automation Techniques. Matt Archer @ MattArcherUk {{{ - PowerPoint PPT Presentation


PowerPoint Presentation

Balancing TestAutomationTechniquesMatt Archer

@MattArcherUk{{{[email protected]


Two distinctly different pieces of test automation, both from the same project.

What makes them different?

Why have both?1Behaviour Driven Development (BDD)User Centric


Key Scenario Coverage


A Specification and an example (bond naming)Html file containing both specification and exampleSpecification: Bond Naming ConventionsThe summary name for a bond should be a combination of the issuer, coupon and maturity date in the following format; issuer + coupon (as %) + maturity date. That said, if the bond is perpetual, the maturity date should be replaced by the word "perpetual".

Some examples are shown below.


An Automated test / a fixture (bond naming) public class CheckBondNames : ColumnFixture { public string Issuer; public string Coupon; public string MaturityDate; public Boolean isPerpetual; public string Name() { Bond bond = new Bond();

bond.Issuer = this.Issuer; bond.Coupon = this.Coupon; bond.MaturityDate = this.MaturityDate; bond.isPerpetual = this.isPerpetual; NamingService namingService = new NamingService(); String name = namingService.getName(bond);

return name; } }Class file to connect the example to the software to test

4Test results (bond naming)Test Results are added to a copy of the original html file


6Semi-automated Data Regression TestingSelf-Generated Expected Results

Focused, But Thorough

No Pass / Fail Result

7ABCDEF147135634234882012543452999340491459259105716074432268722190Release CandidateCurrent Live ReleaseABCDEF147135634234882012543452999340491459259105716073432268722190Testing a release for regressions in data validityOops A regression bug!8The compare routineCompare String Live_Url

String Test_Url

String Save_Location Void Compare()

Void Highlight()

Highlighted ReleaseRelease Candidate

Current live ReleaseSelenium

Executed once

For every widget (~10)

On every screen (~25)

For every client configuration (~12)

= (approx. 3000 data tables to compare)9Manual analysis of results

What is a bug? What is a deliberate change?10Oracles / reqs are rubbish, get over itPicture

11Test automation polaritiesTest automation polaritiesChecking key scenarios vs. Checking all scenariosScripted expected results vs. Captured expected resultsPredefined actions vs. Random actionsCode for checking vs. Code for exploringRun on a trigger vs. Run on demandFast vs. SlowUser perspective vs. Technical perspectiveThrow away vs. Continual investmentAtomic vs. End-to-endRecorded vs. WrittenInspired by James Bachs Exploratory Testing 12IntuitionPicture(Pulling a loose thread)

Matt [email protected]

Any Questions?13