testing the limits of testing
Post on 05-Dec-2014
362 Views
Preview:
DESCRIPTION
TRANSCRIPT
LonsdaleSystems.com 1
Testing the limits of testing…
Phil Robinson
LonsdaleSystems.com
LonsdaleSystems.com 2
In theory there is no difference between theory and practice. In practice there is.Yogi Berra
Blah blah blah blah…
Teaching software testing…
LonsdaleSystems.com 3
The project…
LonsdaleSystems.com 4
The project objective…
Front End (ASP)
Front End(Oracle ADF)
Transaction Queue
Transaction Queue
(Oracle AQ)
DatabaseBack End (PL/SQL)
LonsdaleSystems.com 5
The project plan…
Testing
Project Deadline
LonsdaleSystems.com 6
The test strategy…
LonsdaleSystems.com 7
The waterfall “mantra”
Analysis
Design
Code
Test
Unit
Integration
System
Acceptance
LonsdaleSystems.com 8
Testing is actually three different activities…
Test Design
Test Execution
Defect Repair
LonsdaleSystems.com 9
Testing is about quality not project phases…
• Fit for intended purpose
• Conform to spec• Absence of Defects and other quality goals
LonsdaleSystems.com 10
Testing objectives…
• Validate that the software is fit for its intended purpose
• Verify that the software conforms to its specification
• Identify failures caused by defects
• Measure non-functional attributes of the software (security, performance, reliability…)
• Build confidence in the software
LonsdaleSystems.com 11
Test to pass…• Validate• Verify• Measure• Build confidence
Test to fail…• Identify failures• Measure (stress)
Testing objectives…
LonsdaleSystems.com 12
Vertical integration…
Unit test
LonsdaleSystems.com 13
Vertical integration…
Integrate
LonsdaleSystems.com
LonsdaleSystems.com 14
Vertical integration…
Unit test
LonsdaleSystems.com
LonsdaleSystems.com 15
Vertical integration…
Integrate
LonsdaleSystems.com
LonsdaleSystems.com 16
Vertical integration…
Unit test
LonsdaleSystems.com
LonsdaleSystems.com 17
Vertical integration…
Integrate
LonsdaleSystems.com
LonsdaleSystems.com 18
Horizontal integration…
Feature Feature Feature Feature
Database
LonsdaleSystems.com 19
Horizontal integration…
Feature Feature Feature Feature
Database
LonsdaleSystems.com 20
Testing should not be “black and white”…
• “Black-box”• Functional• Specification based
• “White-box”• Structural• Code based
LonsdaleSystems.com 21
Designing test cases…
Test Item Test
Cases
Spec
Code
Other Docs
Test Basis
Existing System
LonsdaleSystems.com 22
A Better way to define test strategy…
• Spec• Code• Other docs• Existing
systems
• Fit for intended purpose
• Conform to spec
• Identify failures
• Measure non-functional attributes
• Build Confidence
Test Item Test Basis Objective Who?
• Developer• Technical
specialist• Test
Analyst• Business
Analyst• Subject
matter expert
LonsdaleSystems.com
LonsdaleSystems.com 23
Early and extensive unit testing…
LonsdaleSystems.com 24
Minimal unit testing…
Coding
System
LonsdaleSystems.com 25
Minimal unit testing…
Project Deadline
System Test
Acceptance Test
Quality acceptable for user involvement
LonsdaleSystems.com 26
Extensive unit testing…
Coding
System
Unit Testing
LonsdaleSystems.com 27
Extensive unit testing…
Project Deadline
System Test
Acceptance Test
Quality acceptable for user involvement
Verify conforms to spec
Validate fit for intended purpose
LonsdaleSystems.com 28
One in three developers dedicated to unit testing…
66%coding
full time
33%testing
full time
LonsdaleSystems.com 29
Feature testing…
LonsdaleSystems.com 30
Feature testing to build confidence…
Feature
Feature
Feature
Feature
Feature Testing
LonsdaleSystems.com 31
Feature
Feature
Feature
Feature
Feature
Test Case
Test Case
Test Case
Test Case
Test Case
Modelling features…
LonsdaleSystems.com 32
End-to-end testing…
LonsdaleSystems.com 33
End-to-end verification of business rules…
Feature Feature Feature Feature
LonsdaleSystems.com
LonsdaleSystems.com 34
Feature Feature Feature Feature
End-to-end verification of business rules…
LonsdaleSystems.com
LonsdaleSystems.com 35
Modelling business rules…
State State
StateStateState
Feature
Feature
Feature
Choice
LonsdaleSystems.com 36
Generating scenarios…
Scenario
State
Feature
Feature
Feature
LonsdaleSystems.com 37
Date-based testing…
LonsdaleSystems.com 38
Period 1 Period 2 Period 3Scenario
1
2
3
Feature
Feature
Feature
FeatureFeature
Feature
Feature
Feature
Feature
Feature
Feature Feature
Scenario A
Scenario B
Scenario C
Date-based testing…
LonsdaleSystems.com 39
Exploratory testing…
LonsdaleSystems.com 40
Encouraging testers to get creative…
• Testing skills• Subject matter expertise
• Build confidence• Verify business rules80%
20%
Test Cases
LonsdaleSystems.com
LonsdaleSystems.com 41
Triaging bugs…
LonsdaleSystems.com 42
Triaging bugs…
Spreadsheet
Spreadsheet
Spreadsheet
Test
Execution
JIRA
LonsdaleSystems.com 43
Triaging bugs…
• Duplicates• Vague descriptions• Can’t duplicate• Tester misunderstanding
Bugs
Time
Report
ed
Resolved
LonsdaleSystems.com 45
Coming in 2014
Still teaching software testing…
PracticalSoftwareTesting
LonsdaleSystems.com
LonsdaleSystems.com
LonsdaleSystems.com 46
Questions
LonsdaleSystems.com
top related