software engineering lecture 14: testing techniques and strategies
TRANSCRIPT
![Page 1: Software Engineering Lecture 14: Testing Techniques and Strategies](https://reader035.vdocuments.mx/reader035/viewer/2022062515/56649f4e5503460f94c6f3a6/html5/thumbnails/1.jpg)
Software Engineering
Lecture 14: Testing Techniques and Strategies
![Page 2: Software Engineering Lecture 14: Testing Techniques and Strategies](https://reader035.vdocuments.mx/reader035/viewer/2022062515/56649f4e5503460f94c6f3a6/html5/thumbnails/2.jpg)
Today’s Topics Chapters 17 & 18 in SEPA 5/e Testing Principles & Testability Test Characteristics Black-Box vs. White-Box Testing Flow Graphs & Basis Path Testing Testing & Integration Strategies
![Page 3: Software Engineering Lecture 14: Testing Techniques and Strategies](https://reader035.vdocuments.mx/reader035/viewer/2022062515/56649f4e5503460f94c6f3a6/html5/thumbnails/3.jpg)
Software Testing Opportunities for human error
• Specifications, design, coding
• Communication
“Testing is the ultimate review” Can take 30-40% of total effort For critical apps, can be 3 to 5 times all other
efforts combined!
![Page 4: Software Engineering Lecture 14: Testing Techniques and Strategies](https://reader035.vdocuments.mx/reader035/viewer/2022062515/56649f4e5503460f94c6f3a6/html5/thumbnails/4.jpg)
Testing Objectives Execute a program with the intent of finding
errors Good tests have a high probability of discovering
errors Successful tests uncover errors ‘No errors found’: not a good test! Verifying functionality is a secondary goal
![Page 5: Software Engineering Lecture 14: Testing Techniques and Strategies](https://reader035.vdocuments.mx/reader035/viewer/2022062515/56649f4e5503460f94c6f3a6/html5/thumbnails/5.jpg)
Testing Principles Tests traceable to requirements Tests planned before testing Pareto principle: majority of errors traced to
minority of components Component testing first, then
integrated testing Exhaustive testing is not possible Independent tests: more effective
![Page 6: Software Engineering Lecture 14: Testing Techniques and Strategies](https://reader035.vdocuments.mx/reader035/viewer/2022062515/56649f4e5503460f94c6f3a6/html5/thumbnails/6.jpg)
Software Testability
Operability Observability Controllability Decomposability
Characteristics that lead totestable software:
Simplicity Stability Understandability
![Page 7: Software Engineering Lecture 14: Testing Techniques and Strategies](https://reader035.vdocuments.mx/reader035/viewer/2022062515/56649f4e5503460f94c6f3a6/html5/thumbnails/7.jpg)
Operability
System has few bugs No bugs block execution of tests Product evolves in functional stages
The better it works, the more efficientlyit can be tested
![Page 8: Software Engineering Lecture 14: Testing Techniques and Strategies](https://reader035.vdocuments.mx/reader035/viewer/2022062515/56649f4e5503460f94c6f3a6/html5/thumbnails/8.jpg)
Observability
Distinct output for each input States & variables may be queried Past states are logged Factors affecting output are visible Incorrect output easily identified Internal errors reported Source code accessible
What you see is what you test
![Page 9: Software Engineering Lecture 14: Testing Techniques and Strategies](https://reader035.vdocuments.mx/reader035/viewer/2022062515/56649f4e5503460f94c6f3a6/html5/thumbnails/9.jpg)
Controllability
All possible outputs can be generated by some input
All code executable by some input States, variables directly controlled Input/output consistent, structured Tests are specified, automated, and reproduced
The better we can control the software,the more the testing can be automated
![Page 10: Software Engineering Lecture 14: Testing Techniques and Strategies](https://reader035.vdocuments.mx/reader035/viewer/2022062515/56649f4e5503460f94c6f3a6/html5/thumbnails/10.jpg)
Decomposability
Independent modules Modules can be tested separately
By controlling the scope of testing, wecan more quickly isolate problems andperform smarter retesting
![Page 11: Software Engineering Lecture 14: Testing Techniques and Strategies](https://reader035.vdocuments.mx/reader035/viewer/2022062515/56649f4e5503460f94c6f3a6/html5/thumbnails/11.jpg)
Simplicity
Minimum feature set Minimal architecture Code simplicity
The less there is to test, the morequickly we can test it
![Page 12: Software Engineering Lecture 14: Testing Techniques and Strategies](https://reader035.vdocuments.mx/reader035/viewer/2022062515/56649f4e5503460f94c6f3a6/html5/thumbnails/12.jpg)
Stability
Changes made to system:• are infrequent
• are controlled
• don’t invalidate existing tests
Software recovers from failure
The fewer the changes, the fewer thedisruptions to testing
![Page 13: Software Engineering Lecture 14: Testing Techniques and Strategies](https://reader035.vdocuments.mx/reader035/viewer/2022062515/56649f4e5503460f94c6f3a6/html5/thumbnails/13.jpg)
Understandability
Design is well-understood Dependencies are well understood Design changes are communicated Documentation is:
• accessible
• well-organized
• specific, detailed and accurate
The fewer the changes, the fewer thedisruptions to testing
![Page 14: Software Engineering Lecture 14: Testing Techniques and Strategies](https://reader035.vdocuments.mx/reader035/viewer/2022062515/56649f4e5503460f94c6f3a6/html5/thumbnails/14.jpg)
Test Characteristics
Good test has a high probability of finding an error
Good test is not redundant A good test should be “best of breed” A good test is neither too simple nor too complex
![Page 15: Software Engineering Lecture 14: Testing Techniques and Strategies](https://reader035.vdocuments.mx/reader035/viewer/2022062515/56649f4e5503460f94c6f3a6/html5/thumbnails/15.jpg)
Test Case Design
‘Black Box’ Testing• Consider only inputs and outputs
‘White Box’ or ‘Glass Box’ Testing• Also consider internal logic paths, program states, intermediate
data structures, etc.
![Page 16: Software Engineering Lecture 14: Testing Techniques and Strategies](https://reader035.vdocuments.mx/reader035/viewer/2022062515/56649f4e5503460f94c6f3a6/html5/thumbnails/16.jpg)
White-Box Testing
Guarantee that all independent paths have been tested
Exercise all conditions for ‘true’ and ‘false’ Execute all loops for boundary conditions Exercise internal data structures
![Page 17: Software Engineering Lecture 14: Testing Techniques and Strategies](https://reader035.vdocuments.mx/reader035/viewer/2022062515/56649f4e5503460f94c6f3a6/html5/thumbnails/17.jpg)
Why White-Box Testing?
More errors in ‘special case’ code which is infrequently executed
Control flow can’t be predicted accurately in black-box testing
Typo errors can happen anywhere!
![Page 18: Software Engineering Lecture 14: Testing Techniques and Strategies](https://reader035.vdocuments.mx/reader035/viewer/2022062515/56649f4e5503460f94c6f3a6/html5/thumbnails/18.jpg)
Basis Path Testing
White-box method [McCabe ‘76] Analyze procedural design Define basis set of execution paths Test cases for basis set execute every program
statement at least once
![Page 19: Software Engineering Lecture 14: Testing Techniques and Strategies](https://reader035.vdocuments.mx/reader035/viewer/2022062515/56649f4e5503460f94c6f3a6/html5/thumbnails/19.jpg)
Basis Path Testing [2]
Flow Graph: Representation of Structured Programming Constructs
[From SEPA 5/e]
![Page 20: Software Engineering Lecture 14: Testing Techniques and Strategies](https://reader035.vdocuments.mx/reader035/viewer/2022062515/56649f4e5503460f94c6f3a6/html5/thumbnails/20.jpg)
Cyclomatic Complexity
V(G)=E-N+2 = 4
Independent Paths1: 1,112: 1,2,3,4,5,10,1,113: 1,2,3,6,8,9,10,1,114: 1,2,3,6,7,9,10,1,11
V(G): upper bound on number of teststo ensure all code has been executed
[From SEPA 5/e]
![Page 21: Software Engineering Lecture 14: Testing Techniques and Strategies](https://reader035.vdocuments.mx/reader035/viewer/2022062515/56649f4e5503460f94c6f3a6/html5/thumbnails/21.jpg)
Black Box Testing
Focus on functional requirements Incorrect / missing functions Interface errors Errors in external data access Performance errors Initialization and termination errors
![Page 22: Software Engineering Lecture 14: Testing Techniques and Strategies](https://reader035.vdocuments.mx/reader035/viewer/2022062515/56649f4e5503460f94c6f3a6/html5/thumbnails/22.jpg)
Black Box Testing [2]
How is functional validity tested? What classes of input will make good test cases? Is the system sensitive to certain inputs? How are data boundaries isolated?
![Page 23: Software Engineering Lecture 14: Testing Techniques and Strategies](https://reader035.vdocuments.mx/reader035/viewer/2022062515/56649f4e5503460f94c6f3a6/html5/thumbnails/23.jpg)
Black Box Testing [3]
What data rates and volume can the system tolerate?
What effect will specific combinations of data have on system operation?
![Page 24: Software Engineering Lecture 14: Testing Techniques and Strategies](https://reader035.vdocuments.mx/reader035/viewer/2022062515/56649f4e5503460f94c6f3a6/html5/thumbnails/24.jpg)
Comparison Testing Compare software versions “Regression testing”: finding the outputs that
changed Improvements vs. degradations Net effect depends on frequency and impact of
degradations When error rate is low, a large corpus can be
used
![Page 25: Software Engineering Lecture 14: Testing Techniques and Strategies](https://reader035.vdocuments.mx/reader035/viewer/2022062515/56649f4e5503460f94c6f3a6/html5/thumbnails/25.jpg)
Generic Testing Strategies Testing starts at module level and moves
“outward” Different testing techniques used at different
times Testing by developer(s) and independent testers Testing and debugging are separate activities
![Page 26: Software Engineering Lecture 14: Testing Techniques and Strategies](https://reader035.vdocuments.mx/reader035/viewer/2022062515/56649f4e5503460f94c6f3a6/html5/thumbnails/26.jpg)
Verification and Validation Verification
• “Are we building the product right?”
Validation• “Are we building the right product?”
Achieved by life-cycle SQA activities, assessed by testing
“You can’t create quality by testing”
![Page 27: Software Engineering Lecture 14: Testing Techniques and Strategies](https://reader035.vdocuments.mx/reader035/viewer/2022062515/56649f4e5503460f94c6f3a6/html5/thumbnails/27.jpg)
Organization of Testing
[From SEPA 5/e]
![Page 28: Software Engineering Lecture 14: Testing Techniques and Strategies](https://reader035.vdocuments.mx/reader035/viewer/2022062515/56649f4e5503460f94c6f3a6/html5/thumbnails/28.jpg)
How Much Test Time is Necessary?
Logarithmic Poissonexecution-time model
With sufficient fit,model predicts testingtime required to reachacceptable failure rate
[From SEPA 5/e]
![Page 29: Software Engineering Lecture 14: Testing Techniques and Strategies](https://reader035.vdocuments.mx/reader035/viewer/2022062515/56649f4e5503460f94c6f3a6/html5/thumbnails/29.jpg)
Unit Testing [From SEPA 5/e]
![Page 30: Software Engineering Lecture 14: Testing Techniques and Strategies](https://reader035.vdocuments.mx/reader035/viewer/2022062515/56649f4e5503460f94c6f3a6/html5/thumbnails/30.jpg)
Top-Down Integration
PRO: Higher-level (logic) modules tested earlyCON: Lower-level (reusable) modules tested late
[From SEPA 5/e]
![Page 31: Software Engineering Lecture 14: Testing Techniques and Strategies](https://reader035.vdocuments.mx/reader035/viewer/2022062515/56649f4e5503460f94c6f3a6/html5/thumbnails/31.jpg)
Bottom-Up Integration
PRO: Lower-level (reusable) modules tested earlyCON: Higher-level (logic) modules tested late
[From SEPA 5/e]
![Page 32: Software Engineering Lecture 14: Testing Techniques and Strategies](https://reader035.vdocuments.mx/reader035/viewer/2022062515/56649f4e5503460f94c6f3a6/html5/thumbnails/32.jpg)
Hybrid Approaches Sandwich Integration: combination of top-down
and bottom-up Critical Modules
• address several requirements
• high level of control
• complex or error prone
• definite performance requirements
Test Critical Modules ASAP!
![Page 33: Software Engineering Lecture 14: Testing Techniques and Strategies](https://reader035.vdocuments.mx/reader035/viewer/2022062515/56649f4e5503460f94c6f3a6/html5/thumbnails/33.jpg)
Questions?
![Page 34: Software Engineering Lecture 14: Testing Techniques and Strategies](https://reader035.vdocuments.mx/reader035/viewer/2022062515/56649f4e5503460f94c6f3a6/html5/thumbnails/34.jpg)
Software Engineeringfor Information Technology
Lecture 12: System Design
![Page 35: Software Engineering Lecture 14: Testing Techniques and Strategies](https://reader035.vdocuments.mx/reader035/viewer/2022062515/56649f4e5503460f94c6f3a6/html5/thumbnails/35.jpg)
Today’s Topics Design Elements Principles for Quality Design Modularity & Partitioning Effective Modular Design Architectural Styles Mapping Models to Modules
![Page 36: Software Engineering Lecture 14: Testing Techniques and Strategies](https://reader035.vdocuments.mx/reader035/viewer/2022062515/56649f4e5503460f94c6f3a6/html5/thumbnails/36.jpg)
Design Elements Data Design
data structures for data objects Architectural Design
modular structure of software Interface Design
internal / external communication Component-Level Design
procedural description of modules
![Page 37: Software Engineering Lecture 14: Testing Techniques and Strategies](https://reader035.vdocuments.mx/reader035/viewer/2022062515/56649f4e5503460f94c6f3a6/html5/thumbnails/37.jpg)
[Fro
m S
EPA
5/e
]
IncreasingDetail
Design ElementsLinked to
Analysis Models
![Page 38: Software Engineering Lecture 14: Testing Techniques and Strategies](https://reader035.vdocuments.mx/reader035/viewer/2022062515/56649f4e5503460f94c6f3a6/html5/thumbnails/38.jpg)
Evaluating A Design A design must implement:
• explicit requirements (analysis model)
• customer’s implicit requirements
A design must be readable, understandable by coders & testers
A good design provides a complete view of data, function, and behavior
![Page 39: Software Engineering Lecture 14: Testing Techniques and Strategies](https://reader035.vdocuments.mx/reader035/viewer/2022062515/56649f4e5503460f94c6f3a6/html5/thumbnails/39.jpg)
Design Principles [Davis ‘95]
Consider > 1 design alternative Design traceable to analysis model Use design patterns Design structure should reflect structure of
problem domain Consistent style, well-defined interfaces
![Page 40: Software Engineering Lecture 14: Testing Techniques and Strategies](https://reader035.vdocuments.mx/reader035/viewer/2022062515/56649f4e5503460f94c6f3a6/html5/thumbnails/40.jpg)
Design Principles [2] Structured to accommodate change (easy to
modify & update) Structured to degrade gently “Design is not coding,
coding is not design” Assess quality during creation Review design for semantic errors
![Page 41: Software Engineering Lecture 14: Testing Techniques and Strategies](https://reader035.vdocuments.mx/reader035/viewer/2022062515/56649f4e5503460f94c6f3a6/html5/thumbnails/41.jpg)
Design Process Goals A hierarchical organization making use of the
control characteristics of the software A modular design which logically partitions
software into functional elements Useful abstractions for both data and procedures
![Page 42: Software Engineering Lecture 14: Testing Techniques and Strategies](https://reader035.vdocuments.mx/reader035/viewer/2022062515/56649f4e5503460f94c6f3a6/html5/thumbnails/42.jpg)
Design Goals [2]
Modules should be functionally independent Modular interfaces should have minimal
complexity Explicit linking of design elements to
requirements analysis models
![Page 43: Software Engineering Lecture 14: Testing Techniques and Strategies](https://reader035.vdocuments.mx/reader035/viewer/2022062515/56649f4e5503460f94c6f3a6/html5/thumbnails/43.jpg)
Modularity and Software Cost
[Fro
m S
EPA
5/e
]
![Page 44: Software Engineering Lecture 14: Testing Techniques and Strategies](https://reader035.vdocuments.mx/reader035/viewer/2022062515/56649f4e5503460f94c6f3a6/html5/thumbnails/44.jpg)
Modular Design [Meyer ‘88]
Decomposabilityeffective decomposition reduces complexity
Composabilityenable reuse of existing design elements
Understandabilitymodules that can be understood in isolation are easier to build and change
![Page 45: Software Engineering Lecture 14: Testing Techniques and Strategies](https://reader035.vdocuments.mx/reader035/viewer/2022062515/56649f4e5503460f94c6f3a6/html5/thumbnails/45.jpg)
Modular Design [2] Continuity
changes to requirements should trigger localized changes to specific modules
Protectionerror conditions should be considered on a per-module basis
![Page 46: Software Engineering Lecture 14: Testing Techniques and Strategies](https://reader035.vdocuments.mx/reader035/viewer/2022062515/56649f4e5503460f94c6f3a6/html5/thumbnails/46.jpg)
Architectural Terminology
[Fro
m S
EPA
5/e
]
![Page 47: Software Engineering Lecture 14: Testing Techniques and Strategies](https://reader035.vdocuments.mx/reader035/viewer/2022062515/56649f4e5503460f94c6f3a6/html5/thumbnails/47.jpg)
Partitioning Horizontal
branches for each major function Vertical
control & execution are top-down Increase in horizontal partitioning = increased
number of interfaces Vertically partitioned structures more resilient to
change
![Page 48: Software Engineering Lecture 14: Testing Techniques and Strategies](https://reader035.vdocuments.mx/reader035/viewer/2022062515/56649f4e5503460f94c6f3a6/html5/thumbnails/48.jpg)
[Fro
m S
EPA
5/e
]
PartitioningExamples
![Page 49: Software Engineering Lecture 14: Testing Techniques and Strategies](https://reader035.vdocuments.mx/reader035/viewer/2022062515/56649f4e5503460f94c6f3a6/html5/thumbnails/49.jpg)
ProceduralLayering
[Fro
m S
EPA
5/e
]
![Page 50: Software Engineering Lecture 14: Testing Techniques and Strategies](https://reader035.vdocuments.mx/reader035/viewer/2022062515/56649f4e5503460f94c6f3a6/html5/thumbnails/50.jpg)
Effective Modular Design Functional independence
• maximize cohesion of modules
• minimize coupling between modules
• promote robustness in the design
Cohesionone task per procedure is optimal
Couplingminimize module interconnection
![Page 51: Software Engineering Lecture 14: Testing Techniques and Strategies](https://reader035.vdocuments.mx/reader035/viewer/2022062515/56649f4e5503460f94c6f3a6/html5/thumbnails/51.jpg)
Types of Coupling
[From SEPA 5/e]
![Page 52: Software Engineering Lecture 14: Testing Techniques and Strategies](https://reader035.vdocuments.mx/reader035/viewer/2022062515/56649f4e5503460f94c6f3a6/html5/thumbnails/52.jpg)
Design Heuristics Reduce coupling (implode) Improve cohesion (explode) Minimize fan-out & strive for fan-in Scope of effect = scope of control Reduce interface complexity Predictable “black box” modules Controlled entry (no GOTOs!)
![Page 53: Software Engineering Lecture 14: Testing Techniques and Strategies](https://reader035.vdocuments.mx/reader035/viewer/2022062515/56649f4e5503460f94c6f3a6/html5/thumbnails/53.jpg)
ProgramStructures
[From SEPA 5/e]
![Page 54: Software Engineering Lecture 14: Testing Techniques and Strategies](https://reader035.vdocuments.mx/reader035/viewer/2022062515/56649f4e5503460f94c6f3a6/html5/thumbnails/54.jpg)
Architectural Styles Data-Centered Data-Flow Call-and-Return
• main program / subprogram
• remote procedure call
Layered
![Page 55: Software Engineering Lecture 14: Testing Techniques and Strategies](https://reader035.vdocuments.mx/reader035/viewer/2022062515/56649f4e5503460f94c6f3a6/html5/thumbnails/55.jpg)
Data-Centered Architecture
[From SEPA 5/e]
![Page 56: Software Engineering Lecture 14: Testing Techniques and Strategies](https://reader035.vdocuments.mx/reader035/viewer/2022062515/56649f4e5503460f94c6f3a6/html5/thumbnails/56.jpg)
Data FlowArchitectures
[From SEPA 5/e]
![Page 57: Software Engineering Lecture 14: Testing Techniques and Strategies](https://reader035.vdocuments.mx/reader035/viewer/2022062515/56649f4e5503460f94c6f3a6/html5/thumbnails/57.jpg)
LayeredArchitecture
[From SEPA 5/e]
![Page 58: Software Engineering Lecture 14: Testing Techniques and Strategies](https://reader035.vdocuments.mx/reader035/viewer/2022062515/56649f4e5503460f94c6f3a6/html5/thumbnails/58.jpg)
Mapping Models to Modules Goal: map DFDs to a modular architecture Transform Mapping
data flow is modeled as a series of functions with input / output
Transaction Mapping:data flow is modeled as a chain of events (transactions)
![Page 59: Software Engineering Lecture 14: Testing Techniques and Strategies](https://reader035.vdocuments.mx/reader035/viewer/2022062515/56649f4e5503460f94c6f3a6/html5/thumbnails/59.jpg)
Level 0 DFD for SafeHome
[From SEPA 5/e]
![Page 60: Software Engineering Lecture 14: Testing Techniques and Strategies](https://reader035.vdocuments.mx/reader035/viewer/2022062515/56649f4e5503460f94c6f3a6/html5/thumbnails/60.jpg)
Level 1DFD for SafeHome
[From SEPA 5/e]
![Page 61: Software Engineering Lecture 14: Testing Techniques and Strategies](https://reader035.vdocuments.mx/reader035/viewer/2022062515/56649f4e5503460f94c6f3a6/html5/thumbnails/61.jpg)
Level 2DFD for SafeHome
Refines “monitor sensors” process
[From SEPA 5/e]
![Page 62: Software Engineering Lecture 14: Testing Techniques and Strategies](https://reader035.vdocuments.mx/reader035/viewer/2022062515/56649f4e5503460f94c6f3a6/html5/thumbnails/62.jpg)
Level 3DFD for SafeHome
Refines “monitor sensors”process, with flow boundaries
[From SEPA 5/e]
![Page 63: Software Engineering Lecture 14: Testing Techniques and Strategies](https://reader035.vdocuments.mx/reader035/viewer/2022062515/56649f4e5503460f94c6f3a6/html5/thumbnails/63.jpg)
First-LevelFactoring
• Flow boundaries used to determine program structure and modules
• Additional factoring to introduce more detail
[From SEPA 5/e]
![Page 64: Software Engineering Lecture 14: Testing Techniques and Strategies](https://reader035.vdocuments.mx/reader035/viewer/2022062515/56649f4e5503460f94c6f3a6/html5/thumbnails/64.jpg)
Questions?