software engineering ch-13
Post on 23-Dec-2015
218 Views
Preview:
DESCRIPTION
TRANSCRIPT
3/18/2009 S.Sreenivasa Rao 1
Chapter 13Chapter 13Software Testing StrategiesSoftware Testing Strategies
3/18/2009 S.Sreenivasa Rao 2
Software TestingSoftware Testing
Testing is the process of exercising aTesting is the process of exercising aprogram with the specific intent of findingprogram with the specific intent of findingerrors prior to delivery to the end user.errors prior to delivery to the end user.
3/18/2009 S.Sreenivasa Rao
Test strategy provides a road map;Test strategy provides a road map;Steps to be conductedSteps to be conductedWhenWhenHow much effort, time, resourcesHow much effort, time, resources
Test strategyTest strategyTest planningTest planningTest case designTest case designTest executionTest executionResultant data collection and evaluationResultant data collection and evaluation
3
3/18/2009 S.Sreenivasa Rao
Verification and ValidationVerification and ValidationV&VV&V
Testing is part of V&VTesting is part of V&V
Verification: Are we building the product right?Verification: Are we building the product right?Software correctly implements a specific Software correctly implements a specific functionfunction
Validation: Are we building the right product?Validation: Are we building the right product?Software is traceable to customer Software is traceable to customer requirementsrequirements 4
3/18/2009 S.Sreenivasa Rao 5
What Testing ShowsWhat Testing Showserrorserrors
requirements conformancerequirements conformance
performanceperformance
an indicationan indicationof qualityof quality
3/18/2009 S.Sreenivasa Rao 6
Who Tests the Software?Who Tests the Software?
developerdeveloper independent testerindependent tester
Understands the system Understands the system
but, will test "gently"but, will test "gently"and, is driven by "delivery"and, is driven by "delivery"
Must learn about the system,Must learn about the system,but, will attempt to break itbut, will attempt to break itand, is driven by qualityand, is driven by quality
3/18/2009 S.Sreenivasa Rao 7
Testing StrategyTesting Strategyunit testunit test integrationintegration
testtest
validationvalidationtesttest
systemsystemtesttest
3/18/2009 S.Sreenivasa Rao
Levels of Testing
Client Needs
Requirements
Design
Code
Acceptance Testing
System Testing
Integration Testing
Unit testing
3/18/2009 S.Sreenivasa Rao
Requirement Requirement --> high> high--order tests: order tests: validation test, System testvalidation test, System testDesign Design --> Integration test> Integration testCode Code --> unit test> unit test
9
3/18/2009 S.Sreenivasa Rao 10
Testing StrategyTesting Strategy
We begin by We begin by ‘‘testingtesting--inin--thethe--smallsmall’’ and move toward and move toward ‘‘testingtesting--inin--thethe--largelarge’’For conventional softwareFor conventional software
The module (component) is our initial focusThe module (component) is our initial focusIntegration of modules followsIntegration of modules follows
For OO softwareFor OO softwareour focus when our focus when ““testing in the smalltesting in the small”” changes from an changes from an individual module (the conventional view) to an OO class individual module (the conventional view) to an OO class
that encompasses attributes and operations and implies that encompasses attributes and operations and implies communication and collaborationcommunication and collaboration
3/18/2009 S.Sreenivasa Rao 11
Strategic IssuesStrategic IssuesSpecify product requirements in a quantifiable manner long befoSpecify product requirements in a quantifiable manner long before re testing.testing.State testing objectives explicitly. State testing objectives explicitly. Understand the users of the software and develop a profile for eUnderstand the users of the software and develop a profile for each ach user category.user category.Develop a testing plan that emphasizes Develop a testing plan that emphasizes ““rapid cycle testing.rapid cycle testing.””Build Build ““robustrobust”” software that is designed to test itselfsoftware that is designed to test itselfUse effective formal technical reviews as a filter prior to testUse effective formal technical reviews as a filter prior to testingingConduct formal technical reviews to assess the test strategy andConduct formal technical reviews to assess the test strategy and test test cases themselves. cases themselves. Develop a continuous improvement approach for the testing Develop a continuous improvement approach for the testing process. process.
3/18/2009 S.Sreenivasa Rao 12
Unit TestingUnit Testing
modulemoduleto beto be
testedtested
test casestest cases
resultsresults
softwaresoftwareengineerengineer
3/18/2009 S.Sreenivasa Rao 13
Unit TestingUnit Testing
interface interface local data structureslocal data structuresboundary conditionsboundary conditionsindependent pathsindependent pathserror handling pathserror handling paths
modulemoduleto beto be
testedtested
test casestest cases
3/18/2009 S.Sreenivasa Rao 14
Unit Test EnvironmentUnit Test Environment
ModuleModule
stubstub stubstub
driverdriver
interface interface
local data structureslocal data structures
boundary conditionsboundary conditions
independent pathsindependent paths
error handling pathserror handling paths
RESULTSRESULTS
test casestest cases
3/18/2009 S.Sreenivasa Rao 15
Integration Testing StrategiesIntegration Testing Strategies
Options:Options:•• the the ““big bangbig bang”” approachapproach•• an incremental construction strategyan incremental construction strategy
3/18/2009 S.Sreenivasa Rao 16
Top Down IntegrationTop Down Integration
top module is tested with top module is tested with stubsstubs
stubs are replaced one at stubs are replaced one at a time, "depth first"a time, "depth first"
as new modules are integrated, as new modules are integrated, some subset of tests is resome subset of tests is re--runrun
AA
BB
CC
DD EE
FF GG
3/18/2009 S.Sreenivasa Rao 17
BottomBottom--Up IntegrationUp Integration
drivers are replaced one at a drivers are replaced one at a time, "depth first"time, "depth first"
worker modules are grouped into worker modules are grouped into builds and integratedbuilds and integrated
AA
BB
CC
DD EE
FF GG
clustercluster
3/18/2009 S.Sreenivasa Rao 18
Sandwich TestingSandwich Testing
Top modules areTop modules aretested with stubstested with stubs
Worker modules are grouped into Worker modules are grouped into builds and integratedbuilds and integrated
AA
BB
CC
DD EE
FF GG
clustercluster
3/18/2009 S.Sreenivasa Rao
Regression testingRegression testing
to ensure that changes do not introduce to ensure that changes do not introduce unintended behavior or additional errors.unintended behavior or additional errors.
In context of an integration test: reIn context of an integration test: re--execution of some subset of testsexecution of some subset of testsAny correction Any correction --> change of software> change of software
19
3/18/2009 S.Sreenivasa Rao 20
Smoke TestingSmoke TestingA common approach for creating A common approach for creating ““daily buildsdaily builds”” for product softwarefor product softwareSmoke testing steps:Smoke testing steps:
Software components that have been translated into code are inteSoftware components that have been translated into code are integrated grated into a into a ““build.build.””
A build includes all data files, libraries, reusable modules, anA build includes all data files, libraries, reusable modules, and d engineered components that are required to implement one or moreengineered components that are required to implement one or moreproduct functions.product functions.
A series of tests is designed to expose errors that will keep thA series of tests is designed to expose errors that will keep the build e build from properly performing its function. from properly performing its function.
The intent should be to uncover The intent should be to uncover ““show stoppershow stopper”” errors that have the errors that have the highest likelihood of throwing the software project behind schedhighest likelihood of throwing the software project behind schedule.ule.
The build is integrated with other builds and the entire productThe build is integrated with other builds and the entire product (in its (in its current form) is smoke tested daily. current form) is smoke tested daily.
The integration approach may be top down or bottomThe integration approach may be top down or bottom up.up.
3/18/2009 S.Sreenivasa Rao
Identify critical modulesIdentify critical modules
Address several software requirementsAddress several software requirementsHas a high level of control (high in the program Has a high level of control (high in the program structure)structure)Is complex or error proneIs complex or error proneHas definite performance requirementsHas definite performance requirements
21
3/18/2009 S.Sreenivasa Rao 22
ObjectObject--Oriented TestingOriented Testing
testing strategy changestesting strategy changesthe concept of the the concept of the ‘‘unitunit’’ broadens due to encapsulationbroadens due to encapsulationintegration focuses on classes and their executionintegration focuses on classes and their executionvalidation uses conventional black box methodsvalidation uses conventional black box methods
test case design draws on conventional methods, but test case design draws on conventional methods, but also encompasses special featuresalso encompasses special features
3/18/2009 S.Sreenivasa Rao 23
OOT StrategyOOT Strategy
class testing is the equivalent of unit testingclass testing is the equivalent of unit testingoperations within the class are testedoperations within the class are testedthe state behavior of the class is examinedthe state behavior of the class is examined
integration applied three different strategiesintegration applied three different strategiesthreadthread--based testingbased testing——integrates the set of classes integrates the set of classes required to respond to one input or eventrequired to respond to one input or eventuseuse--based testingbased testing——integrates the set of classes required integrates the set of classes required to respond to one use caseto respond to one use casecluster testingcluster testing——integrates the set of classes required to integrates the set of classes required to demonstrate one collaborationdemonstrate one collaboration
3/18/2009 S.Sreenivasa Rao
Validation TestingValidation Testing
Validation succeeds when software functions in a Validation succeeds when software functions in a manner that can be reasonably expected by the manner that can be reasonably expected by the customer.customer.Reasonable expectation are defined in the Software Reasonable expectation are defined in the Software Requirement SpecificationRequirement Specification
FunctionalFunctionalBehavioralBehavioralPerformancePerformanceDocumentationDocumentationUsability Usability ……. .
Alpha/Beta testingAlpha/Beta testingFocus is on customer usageFocus is on customer usage
24
3/18/2009 S.Sreenivasa Rao 25
High Order Testing: High Order Testing: System testingSystem testing
System testingSystem testingFocus is on system integrationFocus is on system integrationRecovery testingRecovery testing
forces the software to fail in a variety of ways and forces the software to fail in a variety of ways and verifies that recovery is properly performedverifies that recovery is properly performedAutomatic: reAutomatic: re--initialization, check pointing, data initialization, check pointing, data recovery, restartrecovery, restartBy human intervention ; meanBy human intervention ; mean--timetime--toto--repairrepair
Security testingSecurity testingverifies that protection mechanisms built into a system verifies that protection mechanisms built into a system will, in fact, protect it from improper penetrationwill, in fact, protect it from improper penetration
Stress testingStress testingexecutes a system in a manner that demands resources executes a system in a manner that demands resources in abnormal quantity, frequency, or volumein abnormal quantity, frequency, or volume
Performance TestingPerformance Testingtest the runtest the run--time performance of software within the time performance of software within the context of an integrated systemcontext of an integrated system
3/18/2009 S.Sreenivasa Rao 26
Debugging: Debugging: A Diagnostic ProcessA Diagnostic Process
3/18/2009 S.Sreenivasa Rao 27
The Debugging ProcessThe Debugging Processtest casestest cases
resultsresults
DebuggingDebugging
suspectedsuspectedcausescauses
identifiedidentifiedcausescauses
correctionscorrections
regressionregressionteststests
new testnew testcasescases
3/18/2009 S.Sreenivasa Rao 28
Debugging EffortDebugging Effort
time requiredtime requiredto diagnose theto diagnose thesymptom andsymptom anddetermine thedetermine thecausecause
time requiredtime requiredto correct the errorto correct the errorand conductand conductregression testsregression tests
3/18/2009 S.Sreenivasa Rao 29
Symptoms & CausesSymptoms & Causes
symptomsymptomcausecause
symptom and cause may be symptom and cause may be geographically separated geographically separated
symptom may disappear when symptom may disappear when another problem is fixedanother problem is fixed
cause may be due to a cause may be due to a combination of noncombination of non--errors errors
cause may be due to a system cause may be due to a system or compiler erroror compiler error
cause may be due to cause may be due to assumptions that everyone assumptions that everyone believesbelieves
symptom may be intermittentsymptom may be intermittent
3/18/2009 S.Sreenivasa Rao
Debugging tacticsDebugging tactics
Brute forceBrute forceDump the memoryDump the memory
BacktrackingBacktrackingStart from presence of error, go backward in the code manuallyStart from presence of error, go backward in the code manually
Cause eliminationCause eliminationCause hypothesis Cause hypothesis ……
30
3/18/2009 S.Sreenivasa Rao 31
Consequences of BugsConsequences of Bugs
damagedamage
mildmild annoyingannoying
disturbingdisturbingseriousserious
extremeextremecatastrophiccatastrophic
infectiousinfectious
Bug TypeBug Type
Bug Categories:Bug Categories: functionfunction--related bugs, related bugs, systemsystem--related bugs, data bugs, coding bugs, related bugs, data bugs, coding bugs, design bugs, documentation bugs, standards design bugs, documentation bugs, standards violations, etc.violations, etc.
3/18/2009 S.Sreenivasa Rao 32
Debugging: Final ThoughtsDebugging: Final Thoughts
thinkthink about the about the symptom you're seeing.symptom you're seeing.
Use toolsUse tools (e.g., dynamic debugger) to gain (e.g., dynamic debugger) to gain more insight.more insight.
get helpget help from someone else.from someone else.
Be absolutely sure to Be absolutely sure to conduct regression testsconduct regression testswhen you do "fix" the bug.when you do "fix" the bug.
1.1.
2.2.
3.3.
4.4.
top related