agile designer™: hp alm less test cases, more coverage webinar
DESCRIPTION
These are the slides from the webinar HP ALM Less Test Cases, More Coverage which took place on the 28/05/2014. The slides discuss: • Agile Designer™'s ability to reduce the number of your test cases and improve your functional coverage • How test cases can be created in a fraction of the time • How you can quickly find which test cases need to change when a requirement changes and automatically repair them • Agile Designer™'s ability to quickly find and reserve test data as well as build sophisticated pools of data to drive test cases • How Agile Designer™ can eliminate error prone and manual test case design, reduce testing time and improve quality.TRANSCRIPT
![Page 1: Agile Designer™: HP ALM Less Test Cases, More Coverage Webinar](https://reader035.vdocuments.mx/reader035/viewer/2022062319/5566a534d8b42acd288b4e52/html5/thumbnails/1.jpg)
The science behind software testing
© Grid Tools Ltd 2014
![Page 2: Agile Designer™: HP ALM Less Test Cases, More Coverage Webinar](https://reader035.vdocuments.mx/reader035/viewer/2022062319/5566a534d8b42acd288b4e52/html5/thumbnails/2.jpg)
HP ALM: Less Test Cases more Coverage
Huw PriceMD Grid Tools Ltd
![Page 3: Agile Designer™: HP ALM Less Test Cases, More Coverage Webinar](https://reader035.vdocuments.mx/reader035/viewer/2022062319/5566a534d8b42acd288b4e52/html5/thumbnails/3.jpg)
OR
Huw PriceMD Grid Tools Ltd
How Many Testers does it take to screw in a light bulb?
![Page 4: Agile Designer™: HP ALM Less Test Cases, More Coverage Webinar](https://reader035.vdocuments.mx/reader035/viewer/2022062319/5566a534d8b42acd288b4e52/html5/thumbnails/4.jpg)
The Answer is 3!
![Page 5: Agile Designer™: HP ALM Less Test Cases, More Coverage Webinar](https://reader035.vdocuments.mx/reader035/viewer/2022062319/5566a534d8b42acd288b4e52/html5/thumbnails/5.jpg)
What is Agile Designer?
Requirements
Clear LogicalSpecs
Storyboards
Expected Results
OutsourceWork
Packages
Test Cases
Data-2
DRAW STORY BOARD TEST CASES TEST DATA CLEAR SPECIFICATIONS
Data andVirtualization
Agile
Outsourcing Data
Flow
Logic
![Page 6: Agile Designer™: HP ALM Less Test Cases, More Coverage Webinar](https://reader035.vdocuments.mx/reader035/viewer/2022062319/5566a534d8b42acd288b4e52/html5/thumbnails/6.jpg)
Agenda
• Different Coverage Techniques
• Building the perfect set of test cases from Requirements
• Updating Test Cases when Requirements Change
• Optimizing Existing Test Cases
![Page 7: Agile Designer™: HP ALM Less Test Cases, More Coverage Webinar](https://reader035.vdocuments.mx/reader035/viewer/2022062319/5566a534d8b42acd288b4e52/html5/thumbnails/7.jpg)
Common Challenges
• 56% of software defects can be traced back to ambiguities in requirements1
• Detecting and resolving production defects is over 40x more expensive than in requirements2
• Industry standard for manual test case design coverage is between 10-20%
• Over-testing of some functions by factors of 40 is not uncommon
• Change requests to existing software requirements can take up to as much as 60% of the SDLC
(Stats: 1 and 2 - Bender RBT, Others: Grid-Tools)
![Page 8: Agile Designer™: HP ALM Less Test Cases, More Coverage Webinar](https://reader035.vdocuments.mx/reader035/viewer/2022062319/5566a534d8b42acd288b4e52/html5/thumbnails/8.jpg)
What is Coverage?
It is not:
• Code Covered
• Number of Test Covered – Tests Run
• Percentage of use cases
• All Paired Combinations
It is:
• Designing Sufficient Tests To VERIFY That The Design And Code Correctly Implement The Requirements
• Did you get the right answer for the right reason - Two or more defects may sometimes cancel each other out - Observability
![Page 9: Agile Designer™: HP ALM Less Test Cases, More Coverage Webinar](https://reader035.vdocuments.mx/reader035/viewer/2022062319/5566a534d8b42acd288b4e52/html5/thumbnails/9.jpg)
Copyright 2008 Bender RBT Inc. 9
What is Coverage?
If the customer is a business client or a preferred personal client and they have a checking account, $100,000 or more in deposits, no overdraft protection and fewer than 5 overdrafts in the last 12 months, set up free overdraft protection otherwise do not give overdraft protection.
This function has sixty-four possible combinations of input from which to select test cases:
So which ones to choose?
![Page 10: Agile Designer™: HP ALM Less Test Cases, More Coverage Webinar](https://reader035.vdocuments.mx/reader035/viewer/2022062319/5566a534d8b42acd288b4e52/html5/thumbnails/10.jpg)
Different Coverage Techniques
Combinatorial – All Pairs – Constrained All Pairs
• Does not support Expected Results - You have to work it out for each combination which is very time consuming
• Combinatorial is something that ‘accidentally’ increases functional coverage (and only to a point)
• Combinatorial does not give you any solid metrics on how good your testing actually is – you can only infer that x out of y combinations have been covered, but it has no bearing in actual functional logic.
• You end up with lots of false errors that have to be checked by hand
![Page 11: Agile Designer™: HP ALM Less Test Cases, More Coverage Webinar](https://reader035.vdocuments.mx/reader035/viewer/2022062319/5566a534d8b42acd288b4e52/html5/thumbnails/11.jpg)
Copyright Bender RBT Inc. 2009 11RBT24261
Cause-Effect Graphing Observable Events and Path Sensitizing
• Assume C and F are not observable events.
• Assume A is stuck at FALSE.• Enter as a test case A(T), B(T),
D(T), E(T).• Results should be C(T), F(T)
and G(T).
T
T
TT
T
T
T
Different Coverage Techniques
![Page 12: Agile Designer™: HP ALM Less Test Cases, More Coverage Webinar](https://reader035.vdocuments.mx/reader035/viewer/2022062319/5566a534d8b42acd288b4e52/html5/thumbnails/12.jpg)
Different Coverage Techniques
Optimized Flow Chart Modelling
• Advanced Graph Optimization
• Most projects already have a flow chart
• Similar results to Cause and effect
• Supports Looping
• Supports Constraints
• Not applicable for non sequential based logic
![Page 13: Agile Designer™: HP ALM Less Test Cases, More Coverage Webinar](https://reader035.vdocuments.mx/reader035/viewer/2022062319/5566a534d8b42acd288b4e52/html5/thumbnails/13.jpg)
Agile Designer – Test Case Techniques
Coverage Feature Agile Designer
Combinatorial Yes
Flow Chart Yes
Cause and Effect Yes
Constraints Yes
Observability Yes
Looping Yes
Complexity Metrics Yes
![Page 14: Agile Designer™: HP ALM Less Test Cases, More Coverage Webinar](https://reader035.vdocuments.mx/reader035/viewer/2022062319/5566a534d8b42acd288b4e52/html5/thumbnails/14.jpg)
Typical Examples of Test Cases built Manually from Requirements
Baseline Agile Designer
Test Cases Created 14 17
Time Taken 5 hours 2 hours
Functional Coverage - % 16% 100%
Baseline Agile Designer
Test Cases Created 150 19
Over Testing x18 x2
Time Taken 5 hours 40 mins
Functional Coverage - % 80% ~100%
Baseline Agile Designer
Test Cases Created 3 12
Time Taken 4 hours 30 mins
Functional Coverage - % 5% 100%
![Page 15: Agile Designer™: HP ALM Less Test Cases, More Coverage Webinar](https://reader035.vdocuments.mx/reader035/viewer/2022062319/5566a534d8b42acd288b4e52/html5/thumbnails/15.jpg)
Leg Before Wicket Rule
Demo 1
Visio VersionNode Types
Existing CoverageHP ALM Data Packs
![Page 16: Agile Designer™: HP ALM Less Test Cases, More Coverage Webinar](https://reader035.vdocuments.mx/reader035/viewer/2022062319/5566a534d8b42acd288b4e52/html5/thumbnails/16.jpg)
Screw in a Light Bulb
Demo 2
Different OptimizationLooping
Path HintsTesting Importance, risk based
![Page 17: Agile Designer™: HP ALM Less Test Cases, More Coverage Webinar](https://reader035.vdocuments.mx/reader035/viewer/2022062319/5566a534d8b42acd288b4e52/html5/thumbnails/17.jpg)
Business Analyst
Programmer
Tester
User
Initial Project
Test Cases
Program
Requirements Business Analyst
Programmer
Tester
User
Change Request
Test Cases
Program
Change request
Business Analyst
Programmer
Tester
User
New Process
Requirements
Tester
Test Cases
Program
RED=HARD WORK
Evolving to Greater Efficiency with Agile Designer
![Page 18: Agile Designer™: HP ALM Less Test Cases, More Coverage Webinar](https://reader035.vdocuments.mx/reader035/viewer/2022062319/5566a534d8b42acd288b4e52/html5/thumbnails/18.jpg)
A day at the Races
Demo 3
Changes to RequirementsFix Broken Test Cases
Return to a 100%
![Page 19: Agile Designer™: HP ALM Less Test Cases, More Coverage Webinar](https://reader035.vdocuments.mx/reader035/viewer/2022062319/5566a534d8b42acd288b4e52/html5/thumbnails/19.jpg)
The HP Test Case Factory
![Page 20: Agile Designer™: HP ALM Less Test Cases, More Coverage Webinar](https://reader035.vdocuments.mx/reader035/viewer/2022062319/5566a534d8b42acd288b4e52/html5/thumbnails/20.jpg)
Baseline Optimized Reduction in Testing
Existing Test Cases 8 9
Test Coverage - % 20% 100%
Functional Nodes 78 31 60%
Edges 76 44
Baseline Optimized Reduction in Testing
Existing Test Cases 15 7
Test Coverage - % 30% 100%
Functional Nodes 113 39 65%
Edges 117 50
N.B. Node = design step in test case / Edge = Journey from one design step to the next
Typical Examples of Optimized Test Cases Built with No Requirements
![Page 21: Agile Designer™: HP ALM Less Test Cases, More Coverage Webinar](https://reader035.vdocuments.mx/reader035/viewer/2022062319/5566a534d8b42acd288b4e52/html5/thumbnails/21.jpg)
Leg Before Wicket Rule
Demo 4
Reverse Engineer
![Page 22: Agile Designer™: HP ALM Less Test Cases, More Coverage Webinar](https://reader035.vdocuments.mx/reader035/viewer/2022062319/5566a534d8b42acd288b4e52/html5/thumbnails/22.jpg)
Test cases designed manually by hand usually have quite a static flow.
As you can see by these examples there are a few flows which cover the same functionality.
A->B->C->D
A->B->C->E
R->A->B->C->D
Agile Designer Test Cases
![Page 23: Agile Designer™: HP ALM Less Test Cases, More Coverage Webinar](https://reader035.vdocuments.mx/reader035/viewer/2022062319/5566a534d8b42acd288b4e52/html5/thumbnails/23.jpg)
Agile Designer Test Cases
Each has its individual starting point and set of steps.
G->F->A->B->C
Q->P->R->G
But often cross paths in terms of Testing the same features several times.
We want to avoid this.
![Page 24: Agile Designer™: HP ALM Less Test Cases, More Coverage Webinar](https://reader035.vdocuments.mx/reader035/viewer/2022062319/5566a534d8b42acd288b4e52/html5/thumbnails/24.jpg)
When we have imported these into Agile Designer we can see we have 5 paths but as we mentioned before some cover similar paths.
A occurs 4 timesB occurs 4 timesC occurs 4 timesR occurs 2 timesG occurs 2 times
This gives us the perfect opportunity to optimize and improve the test cases
Agile Designer Test Cases
![Page 25: Agile Designer™: HP ALM Less Test Cases, More Coverage Webinar](https://reader035.vdocuments.mx/reader035/viewer/2022062319/5566a534d8b42acd288b4e52/html5/thumbnails/25.jpg)
From this example we have 5 test cases but have we covered each functional step in the most optimal of fashions?
5 Test Cases24 Nodes (Design Steps)22 In/Out Edges (Journeys)
Agile Designer Test Cases
![Page 26: Agile Designer™: HP ALM Less Test Cases, More Coverage Webinar](https://reader035.vdocuments.mx/reader035/viewer/2022062319/5566a534d8b42acd288b4e52/html5/thumbnails/26.jpg)
Agile Designer Test Cases
So now we optimized the flow and get something like this.
Optimize:- Removed duplicate design
steps and revaluated how the test cases should link together.
During this evaluation process we are able to navigate through the diagram from Start->End without duplicates causing us problems.
![Page 27: Agile Designer™: HP ALM Less Test Cases, More Coverage Webinar](https://reader035.vdocuments.mx/reader035/viewer/2022062319/5566a534d8b42acd288b4e52/html5/thumbnails/27.jpg)
Agile Designer Test Cases
Post Optimize you can see that the number of Nodes/Edges we are testing has reduced significantly.
- 12 Nodes (Design Steps)- 20 Edges (Decisions)
This shows us a 50% reduction in steps we need to take whilst giving us 100% in functional coverage.
We’ve also reduced the number of decisions by 2. This highlights that although a smaller reduction our decision process is much more intelligent and with just 2 fewer we are covering more functions.
![Page 28: Agile Designer™: HP ALM Less Test Cases, More Coverage Webinar](https://reader035.vdocuments.mx/reader035/viewer/2022062319/5566a534d8b42acd288b4e52/html5/thumbnails/28.jpg)
Agile Designer makes Continuous Integration = “Idea to Production”
Optimized Tests
Existing Test Cases
Test Data
Automation
Continuous Development
![Page 29: Agile Designer™: HP ALM Less Test Cases, More Coverage Webinar](https://reader035.vdocuments.mx/reader035/viewer/2022062319/5566a534d8b42acd288b4e52/html5/thumbnails/29.jpg)
Agile Designer - Summary
Think of a Requirement like a circuit board
• The Test Cases will reflect the requirements
Static Testing
• You can Accurately Measure your Coverage
• You can Automatically create the MinimumTest Cases
With Maximum Coverage
• You can reduce and optimize your existing test cases
• You can predict how long software will take to develop
![Page 30: Agile Designer™: HP ALM Less Test Cases, More Coverage Webinar](https://reader035.vdocuments.mx/reader035/viewer/2022062319/5566a534d8b42acd288b4e52/html5/thumbnails/30.jpg)
Agile Designer – Next Steps
• There is a free limited download
• Contact sales for a full copy trial – Linked to HP ALM and Datamaker
• We offer half day training courses (Free T shirt!)http://info.grid-tools.com/agile-designer-training-workshops
• We offer an Agile Designer Certification
• We are looking for partners to roll out the test optimization factory
• We are looking for local partners
![Page 31: Agile Designer™: HP ALM Less Test Cases, More Coverage Webinar](https://reader035.vdocuments.mx/reader035/viewer/2022062319/5566a534d8b42acd288b4e52/html5/thumbnails/31.jpg)
Huw PriceMD Grid Tools Ltd