Download - Balancing Test Automation Techniques
Balancing TestAutomationTechniques
Matt Archer@MattArcherUk
{{{
[email protected]://mattarcherblog.wordpress.com
Agenda
Two distinctly different pieces of test automation, both from the same project.
What makes them different?
Why have both?
Behaviour Driven Development (BDD)
User Centric
Unmanned
Key Scenario Coverage
A Specification and an example (bond naming)
• Html file containing both specification and example
Specification: Bond Naming Conventions
The summary name for a bond should be a combination of the issuer, coupon and maturity date in the following format; issuer + coupon (as %) + maturity date. That said, if the bond is perpetual, the maturity date should be replaced by the word "perpetual".
Some examples are shown below.
An Automated test / a fixture (bond naming)
public class CheckBondNames : ColumnFixture { public string Issuer; public string Coupon; public string MaturityDate; public Boolean isPerpetual;
public string Name() { Bond bond = new Bond();
bond.Issuer = this.Issuer; bond.Coupon = this.Coupon; bond.MaturityDate = this.MaturityDate; bond.isPerpetual = this.isPerpetual;
NamingService namingService = new NamingService(); String name = namingService.getName(bond);
return name; } }
• Class file to connect the example to the software to test
Test results (bond naming)• Test Results are added to a copy of the original html file
DEMO
Semi-automated Data Regression Testing
Self-Generated Expected Results
Focused, But Thorough
No Pass / Fail Result
A B C D E F
1 4 7 13 56 34
2 34 88 20 12 54
3 45 29 9 93 40
4 91 45 92 59 10
5 71 60 74 43 22
6 87 2 21 9 0
Release Candidate Current Live Release
A B C D E F
1 4 7 13 56 34
2 34 88 20 12 54
3 45 29 9 93 40
4 91 45 92 59 10
5 71 60 73 43 22
6 87 2 21 9 0
Testing a release for regressions in data validity
Oops… A regression bug!
The “compare” routine
Compare
String Live_Url String Test_Url String Save_Location
Void Compare() Void Highlight()
Highlighted ReleaseRelease Candidate
Current live Release
SeleniumExecuted once…
For every widget (~10)
On every screen (~25)
For every client configuration (~12)
= (approx. 3000 data tables to compare)
Manual analysis of results
What is a bug? What is a deliberate change?
Oracles / reqs are rubbish, get over it
• Picture
Test automation polaritiesTest automation polarities
Checking key scenarios vs. Checking all scenariosScripted expected results vs. Captured expected resultsPredefined actions vs. Random actionsCode for checking vs. Code for exploringRun on a trigger vs. Run on demandFast vs. SlowUser perspective vs. Technical perspectiveThrow away vs. Continual investmentAtomic vs. End-to-endRecorded vs. Written
Inspired by James Bach’s Exploratory Testing Polaritieswww.satisfice.com/articles/et-dynamics.pdf
Intuition
• Picture– (Pulling a loose thread)
Matt [email protected]@expresssoftware.co.ukhttp://mattarcherblog.wordpress.com
Any Questions?
