let the adventure begin the 80-20 testing - ingenuity
TRANSCRIPT
INDIUM SOFTWAREAn Independent Software Testing Firm
Let the Adventure Begin - The 80-20 Testing
“Business Assurance through Quality Assurance”
© 2014 Indium Software
ENHANCING SOFTWARE QUALITY
2
Agenda
Testing – The Need to be Quick
Challenges – With Scripted Tests
Interesting Usage of Intelligence
Case Study – Req & Risks
Exploratory Approach
Metrics and Value Additions
Conclusion
© 2014 Indium Software
ENHANCING SOFTWARE QUALITY
3
Testing - The Need to be Quick
Today’s need:New technology and resources – the environment is unwilling to take its time for the quality achievementsQuick turnaroundCompact lifecycles
The above said:Are we testing enough?Do we have the time to test enough?Are scripted tests complete?What best can we do to ensure coverage?
Let the adventure begin with Exploratory Testing…
© 2014 Indium Software
ENHANCING SOFTWARE QUALITY
4
Challenges – With Scripted Tests
Constant change and Lack of documented requirements
Time constraints and Insufficient test coverage
Early detection of defects
Code breaks & Performance Bottlenecks
Challenges
• What is the effect?
• Why?
• How it affects?
• When?
© 2014 Indium Software
ENHANCING SOFTWARE QUALITY
5
Interesting Usage of Intelligence
Several types of random tests -
When to use what
Situations where
requirement changes
along with an unsettled
process
CurrentA well
groomed approach yielding
maximum test
coverage and high
defect catch rate
Need
Do not validate
unplanned
A complete process with maximum
utilization of minimum resources
Solution
© 2014 Indium Software
ENHANCING SOFTWARE QUALITY
6
Case Study
Requirement
Changing combination of environments
Release Frequency
Weekly (sometimes even lesser)
Content Change Rate
Daily
Testing Life Cycle
3-4 Days
Testing Scope
Compatibility / Functionality
Types of Testing
Scripted & Exploratory
Application Type
Web Application
© 2014 Indium Software
ENHANCING SOFTWARE QUALITY
7
Risk
Story Board
Discussions
Short development cycles
Unrelated Weekly Changes
Support Quick Updates
Platform Dependent Test Cases
© 2014 Indium Software
ENHANCING SOFTWARE QUALITY
8
Exploratory Approach – Team Setup
Team Setup
Planning
Execution
Reporting
Maintenance
• Skilled resources• Knowledge in
productGeneral
• Handy documents• Training is easy• Converts any
resource to handle the process
Derived
© 2014 Indium Software
ENHANCING SOFTWARE QUALITY
9
Exploratory Approach – Planning
Team Setup
Planning
Execution
Reporting
Maintenance
• Minimum planning• Traditional testing
technique used at random
General
• Know your domain and your user
• Data analysis & Areas to focus on
• RCA / Know your environment
• Identify test cases from the suite
Derived
© 2014 Indium Software
ENHANCING SOFTWARE QUALITY
10
Exploratory Approach – Execution
Team Setup
Planning
Execution
Reporting
Maintenance
• Focused testing• Fixed combinationGeneral
• Standardized charter• Soap opera testing• Negative testing
Derived
© 2014 Indium Software
ENHANCING SOFTWARE QUALITY
11
Exploratory Approach – Reporting
Team Setup
Planning
Execution
Reporting
Maintenance
• No formal documentation• Chances of missed
scenario• Redundant testing
General
• Parallel documentation• Exploratory Procedure• Charted• Reviewed• Supervised
Derived
© 2014 Indium Software
ENHANCING SOFTWARE QUALITY
12
Exploratory Approach – Maintenance
Team Setup
Planning
Execution
Reporting
Maintenance
• Testing is considered done
• Script updatesGeneral
• Trackers• Fine tuning scripted
testsDerived
© 2014 Indium Software
ENHANCING SOFTWARE QUALITY
13
Metrics
o This metrics highlights defects found in Scripted vs. Exploratory
o Introduction of derived exploratory testing contributed to 20% more defects in the High and Medium category
o Derived exploratory testing helped to uncover affected component under the UI, Display and behind the scenes functionality
1
2
3
4
5
6
0
10
20
30
40
50
60
Scripted Testing
Generalized Exploratory
Derived Exploratory
IterationsD
efec
ts
© 2014 Indium Software
ENHANCING SOFTWARE QUALITY
14
Metrics
o This chart shows the defects in Test case
o Projects with consecutive iterations need lots of effort in script maintenance, which helps in hands-on testing
o The derived exploratory testing,
helped make documentation more stable therefore reducing time spent in test script fixes
1 2 3 4 5 60
5
10
15
20
25
30
35
40
45
35
40
23
15
8 7
Test Case Defects
Iterations
Defe
cts
© 2014 Indium Software
ENHANCING SOFTWARE QUALITY
15
Metrics
o This pie chat represents the distribution of defects based on the testing type
o Defects with the derived approach is 20% more
o This 20% raises confidence in the application by 80%
o Proves derived approach is a best practice for faster iterations and frequent builds
70%
7%
22%
Defect Distribution
Scprited Testing
Generalized Exploratory
Derived Exploratory
© 2014 Indium Software
ENHANCING SOFTWARE QUALITY
16
Value Additions
Key Levers With Derived Exploratory Testing Value Additions
Test Script maintenance
Reduced by 50% Advanced strategy for bug detection More coverage area Easier to maintain platform dependent scripts Reduced timelines in test script maintenance
TC Defects Reduced by 80% Time saved in test cases fixes
Defect Detection Increased by 25% Defect pattern identified easily Early detection of bugs
Exploratory Issues Increased by 20% Logged high severity issues Increased confidence in product for the business
team
Reporting Better by 40% Proactive monitoring Effective documentation for future reference
© 2014 Indium Software
ENHANCING SOFTWARE QUALITY
17
Conclusion
All paths may or may not lead to the same destination…
… but have all paths been tested?
Try our Derived Exploratory Approach!
© 2014 Indium Software
ENHANCING SOFTWARE QUALITY
18
Thank You
USA: [email protected]
EMEA & APAC: [email protected]@indiumsoft.com
INDIA: [email protected]
SunnyvaleSuite 210, 1250 Oakmead ParkwaySunnyvale, CA – 94085.Phone: +1(408) 501-8844Fax: +1(408) 501-8808
Atlanta Crown Office Suites1870 The Exchange, Suite 100 Atlanta, GA 30339Phone: +1 (678) 608-0809, +1(770) 989-7302
New DelhiF-1/5 Okhla Industrial Area Phase-I, New Delhi – 110 020.Phone: +91-11-6613 0400
BengaluruSRS Plaza, No. 246, 2nd & 3rd Floor13th Cross, Indira Nagar 2nd StageBengaluru – 560 038.Phone: +91-80-4914 7777
ChennaiVDS House41, 2nd Floor, Cathedral RoadChennai – 600 086.Phone: +91-44-2811 6330Fax: +91-44-4210 4033