chapter 11, testing
DESCRIPTION
Chapter 11, Testing. Terminology. Failure : Any deviation of the observed behavior from the specified behavior Erroneous state ( error ): The system is in a state such that further processing by the system can lead to a failure - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Chapter 11, Testing](https://reader035.vdocuments.mx/reader035/viewer/2022062520/56815c71550346895dca8594/html5/thumbnails/1.jpg)
Usi
ng U
ML,
Pat
tern
s, an
d Ja
vaO
bjec
t-O
rien
ted
Soft
war
e E
ngin
eeri
ng
Chapter 11, Testing
![Page 2: Chapter 11, Testing](https://reader035.vdocuments.mx/reader035/viewer/2022062520/56815c71550346895dca8594/html5/thumbnails/2.jpg)
Bernd Bruegge & Allen H. Dutoit Object-Oriented Software Engineering: Using UML, Patterns, and Java 2
Terminology
• Failure: Any deviation of the observed behavior from the specified behavior
• Erroneous state (error): The system is in a state such that further processing by the system can lead to a failure
• Fault: The mechanical or algorithmic cause of an error (“bug”)
• Validation: Activity of checking for deviations between the observed behavior of a system and its specification.
![Page 3: Chapter 11, Testing](https://reader035.vdocuments.mx/reader035/viewer/2022062520/56815c71550346895dca8594/html5/thumbnails/3.jpg)
Bernd Bruegge & Allen H. Dutoit Object-Oriented Software Engineering: Using UML, Patterns, and Java 3
Examples of Faults and Errors
• Faults in the Interface specification
• Mismatch between what the client needs and what the server offers
• Mismatch between requirements and implementation
• Algorithmic Faults • Missing initialization• Incorrect branching
condition• Missing test for null
• Mechanical Faults (very hard to find)
• Operating temperature outside of equipment specification
• Errors• Null reference errors• Concurrency errors• Exceptions.
![Page 4: Chapter 11, Testing](https://reader035.vdocuments.mx/reader035/viewer/2022062520/56815c71550346895dca8594/html5/thumbnails/4.jpg)
Bernd Bruegge & Allen H. Dutoit Object-Oriented Software Engineering: Using UML, Patterns, and Java 4
Another View on How to Deal with Faults
• Fault avoidance• Use methodology to reduce complexity • Use configuration management to prevent inconsistency• Apply verification to prevent algorithmic faults• Use Reviews
• Fault detection• Testing: Activity to provoke failures in a planned way• Debugging: Find and remove the cause (Faults) of an
observed failure• Monitoring: Deliver information about state => Used
during debugging• Fault tolerance
• Exception handling• Modular redundancy.
![Page 5: Chapter 11, Testing](https://reader035.vdocuments.mx/reader035/viewer/2022062520/56815c71550346895dca8594/html5/thumbnails/5.jpg)
Bernd Bruegge & Allen H. Dutoit Object-Oriented Software Engineering: Using UML, Patterns, and Java 5
Taxonomy for Fault Handling Techniques
Fault Handling
Fault Avoidance
Fault Detection
Fault Tolerance
Verification
ConfigurationManagementMethodology Atomic
TransactionsModular
Redundancy
System Testing
IntegrationTesting
UnitTesting
Testing Debugging
![Page 6: Chapter 11, Testing](https://reader035.vdocuments.mx/reader035/viewer/2022062520/56815c71550346895dca8594/html5/thumbnails/6.jpg)
Bernd Bruegge & Allen H. Dutoit Object-Oriented Software Engineering: Using UML, Patterns, and Java 6
Observations
• It is impossible to completely test any nontrivial module or system
• Practical limitations: Complete testing is prohibitive in time and cost
• Theoretical limitations: e.g. Halting problem• “Testing can only show the presence of bugs,
not their absence” (Dijkstra). • Testing is not for free
=> Define your goals and priorities
![Page 7: Chapter 11, Testing](https://reader035.vdocuments.mx/reader035/viewer/2022062520/56815c71550346895dca8594/html5/thumbnails/7.jpg)
Bernd Bruegge & Allen H. Dutoit Object-Oriented Software Engineering: Using UML, Patterns, and Java 7
Testing takes creativity
• To develop an effective test, one must have:• Detailed understanding of the system• Application and solution domain knowledge • Knowledge of the testing techniques• Skill to apply these techniques
• Testing is done best by independent testers• We often develop a certain mental attitude that the
program should behave in a certain way when in fact it does not
• Programmers often stick to the data set that makes the program work
• A program often does not work when tried by somebody else.
![Page 8: Chapter 11, Testing](https://reader035.vdocuments.mx/reader035/viewer/2022062520/56815c71550346895dca8594/html5/thumbnails/8.jpg)
Bernd Bruegge & Allen H. Dutoit Object-Oriented Software Engineering: Using UML, Patterns, and Java 8
Testing Activities
UnitTesting
AcceptanceTesting
IntegrationTesting
SystemTesting
RequirementsAnalysis
Document
ClientExpectation
SystemDesign
Document
ObjectDesign
Document
Developer Client
UnitTesting
AcceptanceTesting
IntegrationTesting
SystemTesting
![Page 9: Chapter 11, Testing](https://reader035.vdocuments.mx/reader035/viewer/2022062520/56815c71550346895dca8594/html5/thumbnails/9.jpg)
Bernd Bruegge & Allen H. Dutoit Object-Oriented Software Engineering: Using UML, Patterns, and Java 9
Types of Testing
• Unit Testing• Individual component (class or subsystem)• Carried out by developers• Goal: Confirm that the component or subsystem is
correctly coded and carries out the intended functionality
• Integration Testing• Groups of subsystems (collection of subsystems) and
eventually the entire system• Carried out by developers• Goal: Test the interfaces among the subsystems.
UnitTesting
AcceptanceTesting
IntegrationTesting
SystemTesting
![Page 10: Chapter 11, Testing](https://reader035.vdocuments.mx/reader035/viewer/2022062520/56815c71550346895dca8594/html5/thumbnails/10.jpg)
Bernd Bruegge & Allen H. Dutoit Object-Oriented Software Engineering: Using UML, Patterns, and Java 10
Types of Testing continued...
• System Testing• The entire system• Carried out by developers• Goal: Determine if the system meets the requirements
(functional and nonfunctional)• Acceptance Testing
• Evaluates the system delivered by developers• Carried out by the client. May involve executing typical
transactions on site on a trial basis• Goal: Demonstrate that the system meets the
requirements and is ready to use.
UnitTesting
AcceptanceTesting
IntegrationTesting
SystemTesting
![Page 11: Chapter 11, Testing](https://reader035.vdocuments.mx/reader035/viewer/2022062520/56815c71550346895dca8594/html5/thumbnails/11.jpg)
Bernd Bruegge & Allen H. Dutoit Object-Oriented Software Engineering: Using UML, Patterns, and Java 11
When should you write a test?
• Traditionally after the source code is written
• In XP before the source code written• Test-Driven Development Cycle
• Add a test• Run the automated tests
=> see the new one fail• Write some code• Run the automated tests
=> see them succeed• Refactor code.
![Page 12: Chapter 11, Testing](https://reader035.vdocuments.mx/reader035/viewer/2022062520/56815c71550346895dca8594/html5/thumbnails/12.jpg)
Bernd Bruegge & Allen H. Dutoit Object-Oriented Software Engineering: Using UML, Patterns, and Java 12
Unit Testing
• Static Testing (at compile time)• Review
• Walk-through (informal)• Code inspection (formal)
• Dynamic Testing (at run time)• Black-box testing• White-box testing.
UnitTesting
AcceptanceTesting
IntegrationTesting
SystemTesting
![Page 13: Chapter 11, Testing](https://reader035.vdocuments.mx/reader035/viewer/2022062520/56815c71550346895dca8594/html5/thumbnails/13.jpg)
Bernd Bruegge & Allen H. Dutoit Object-Oriented Software Engineering: Using UML, Patterns, and Java 13
Black-box testing
• Focus: I/O behavior • If for any given input, we can predict the output, then
the component passes the test• Goal: Reduce number of test cases by
equivalence partitioning:• Divide input conditions into equivalence classes• Choose test cases for each equivalence class.
UnitTesting
AcceptanceTesting
IntegrationTesting
SystemTesting
![Page 14: Chapter 11, Testing](https://reader035.vdocuments.mx/reader035/viewer/2022062520/56815c71550346895dca8594/html5/thumbnails/14.jpg)
Bernd Bruegge & Allen H. Dutoit Object-Oriented Software Engineering: Using UML, Patterns, and Java 14
Black-box testing: Test case selection
a) Input is valid across range of values • Developer selects test cases from 3 equivalence classes:
• Below the range• Within the range• Above the range
b) Input is only valid if it is a member of a discrete set
• Developer selects test cases from 2 equivalence classes:• Valid discrete values• Invalid discrete values
![Page 15: Chapter 11, Testing](https://reader035.vdocuments.mx/reader035/viewer/2022062520/56815c71550346895dca8594/html5/thumbnails/15.jpg)
Bernd Bruegge & Allen H. Dutoit Object-Oriented Software Engineering: Using UML, Patterns, and Java 15
Black box testing: An examplepublic class MyCalendar {
public int getNumDaysInMonth(int month, int year) throws InvalidMonthException
{ … }}Representation for month:
1: January, 2: February, …., 12: December
Representation for year:1904, … 1999, 2000,…, 2006, …
How many test cases do we need for the black box testing of getNumDaysInMonth()?
![Page 16: Chapter 11, Testing](https://reader035.vdocuments.mx/reader035/viewer/2022062520/56815c71550346895dca8594/html5/thumbnails/16.jpg)
Bernd Bruegge & Allen H. Dutoit Object-Oriented Software Engineering: Using UML, Patterns, and Java 16
Black box testing: Gregorian Calendar• Month parameter equivalence classes
• 30 days• 31 days• February• No month 0, 13, -1 [throw exception]
• Year parameter equivalence classes• Normal year• Leap year:
• /4• /100• /400
![Page 17: Chapter 11, Testing](https://reader035.vdocuments.mx/reader035/viewer/2022062520/56815c71550346895dca8594/html5/thumbnails/17.jpg)
Bernd Bruegge & Allen H. Dutoit Object-Oriented Software Engineering: Using UML, Patterns, and Java 17
White-box testing overview• Code coverage
• Branch coverage
• Condition coverage
• Path coverage
UnitTesting
AcceptanceTesting
IntegrationTesting
SystemTesting
![Page 18: Chapter 11, Testing](https://reader035.vdocuments.mx/reader035/viewer/2022062520/56815c71550346895dca8594/html5/thumbnails/18.jpg)
Bernd Bruegge & Allen H. Dutoit Object-Oriented Software Engineering: Using UML, Patterns, and Java 18
if ( i = TRUE) printf("YES\n"); else printf("NO\n");Test cases: 1) i = TRUE; 2) i = FALSE
White-box Testing (Continued)• Statement Testing (Algebraic Testing): Test single
statements (Choice of operators in polynomials, etc)
• Loop Testing:• Cause execution of the loop to be skipped completely.
(Exception: Repeat loops)• Loop to be executed exactly once• Loop to be executed more than once
• Path testing:• Make sure all paths in the program are executed
• Branch Testing (Conditional Testing): Make sure that each possible outcome from a condition is tested at least once
![Page 19: Chapter 11, Testing](https://reader035.vdocuments.mx/reader035/viewer/2022062520/56815c71550346895dca8594/html5/thumbnails/19.jpg)
Bernd Bruegge & Allen H. Dutoit Object-Oriented Software Engineering: Using UML, Patterns, and Java 19
/*Read in and sum the scores*/
White-box Testing ExampleFindMean(float Mean, FILE ScoreFile) { SumOfScores = 0.0; NumberOfScores = 0; Mean = 0; Read(ScoreFile, Score); while (! EOF(ScoreFile) {
if ( Score > 0.0 ) { SumOfScores = SumOfScores + Score;
NumberOfScores++; }
Read(ScoreFile, Score); } /* Compute the mean and print the result */ if (NumberOfScores > 0 ) { Mean = SumOfScores/NumberOfScores;
printf("The mean score is %f \n", Mean); } else
printf("No scores found in file\n");}
![Page 20: Chapter 11, Testing](https://reader035.vdocuments.mx/reader035/viewer/2022062520/56815c71550346895dca8594/html5/thumbnails/20.jpg)
Bernd Bruegge & Allen H. Dutoit Object-Oriented Software Engineering: Using UML, Patterns, and Java 20
White-box Testing Example: Determining the Paths
FindMean (FILE ScoreFile){ float SumOfScores = 0.0;
int NumberOfScores = 0; float Mean=0.0; float Score;Read(ScoreFile, Score);while (! EOF(ScoreFile) {
if (Score > 0.0 ) {SumOfScores = SumOfScores + Score;NumberOfScores++;}
Read(ScoreFile, Score);}/* Compute the mean and print the result */if (NumberOfScores > 0) {
Mean = SumOfScores / NumberOfScores;printf(“ The mean score is %f\n”, Mean);
} elseprintf (“No scores found in file\n”);
}
1
23
4
5
7
6
8
9
![Page 21: Chapter 11, Testing](https://reader035.vdocuments.mx/reader035/viewer/2022062520/56815c71550346895dca8594/html5/thumbnails/21.jpg)
Bernd Bruegge & Allen H. Dutoit Object-Oriented Software Engineering: Using UML, Patterns, and Java 21
Constructing the Logic Flow DiagramStart
2
3
4 5
6
7
8 9
Exit
1
F
T F
T F
T
![Page 22: Chapter 11, Testing](https://reader035.vdocuments.mx/reader035/viewer/2022062520/56815c71550346895dca8594/html5/thumbnails/22.jpg)
Bernd Bruegge & Allen H. Dutoit Object-Oriented Software Engineering: Using UML, Patterns, and Java 22
Finding the Test CasesStart
2
3
4 5
6
7
8 9
Exit
1
b
d e
gf
i j
hc
k l
a (Covered by any data)
(Data set must
(Data set must contain at least one value)
be empty)
(Total score > 0.0)(Total score < 0.0)
(Positive score) (Negative score)
(Reached if either f or e is reached)
![Page 23: Chapter 11, Testing](https://reader035.vdocuments.mx/reader035/viewer/2022062520/56815c71550346895dca8594/html5/thumbnails/23.jpg)
Bernd Bruegge & Allen H. Dutoit Object-Oriented Software Engineering: Using UML, Patterns, and Java 23
Test Cases
• Test case 1 : ? (To execute loop exactly once)• Test case 2 : ? (To skip loop body)• Test case 3: ?,? (to execute loop more than
once)
These 3 test cases cover all control flow paths
![Page 24: Chapter 11, Testing](https://reader035.vdocuments.mx/reader035/viewer/2022062520/56815c71550346895dca8594/html5/thumbnails/24.jpg)
Bernd Bruegge & Allen H. Dutoit Object-Oriented Software Engineering: Using UML, Patterns, and Java 24
Unit Testing Heuristics
1. Create unit tests when object design is completed
• Black-box test: Test the functional model
• White-box test: Test the dynamic model
2. Develop the test cases • Goal: Find effective num-
ber of test cases3. Cross-check the test cases
to eliminate duplicates• Don't waste your time!
4. Desk check your source code• Sometimes reduces testing
time5. Create a test harness
• Test drivers and test stubs are needed for integration testing
6. Describe the test oracle• Often the result of the first
successfully executed test7. Execute the test cases
• Re-execute test whenever a change is made (“regression testing”)
8. Compare the results of the test with the test oracle
• Automate this if possible.
![Page 25: Chapter 11, Testing](https://reader035.vdocuments.mx/reader035/viewer/2022062520/56815c71550346895dca8594/html5/thumbnails/25.jpg)
Bernd Bruegge & Allen H. Dutoit Object-Oriented Software Engineering: Using UML, Patterns, and Java 25
JUnit: Overview• A Java framework for writing and running unit tests
• Test cases and fixtures • Test suites• Test runner
• Written with “test first” and pattern-based development in mind
• Tests written before code• Allows for regression testing• Facilitates refactoring
• JUnit is Open Source• www.junit.org• JUnit Version 4, released Mar 2006
UnitTesting
AcceptanceTesting
IntegrationTesting
SystemTesting
![Page 26: Chapter 11, Testing](https://reader035.vdocuments.mx/reader035/viewer/2022062520/56815c71550346895dca8594/html5/thumbnails/26.jpg)
Bernd Bruegge & Allen H. Dutoit Object-Oriented Software Engineering: Using UML, Patterns, and Java 26
*
JUnit ClassesTest
run(TestResult)
ConcreteTestCase
setUp()tearDown()runTest()
TestResult
TestCase
run(TestResult)setUp()tearDown()
testName:String
runTest()
TestSuite
run(TestResult)addTest()
UnitToBeTested
Methods under Test
![Page 27: Chapter 11, Testing](https://reader035.vdocuments.mx/reader035/viewer/2022062520/56815c71550346895dca8594/html5/thumbnails/27.jpg)
Bernd Bruegge & Allen H. Dutoit Object-Oriented Software Engineering: Using UML, Patterns, and Java 27
An example: Testing MyList
• Unit to be tested• MyList
• Methods under test• add()• remove()• contains()• size()
• Concrete Test case• MyListTestCase
![Page 28: Chapter 11, Testing](https://reader035.vdocuments.mx/reader035/viewer/2022062520/56815c71550346895dca8594/html5/thumbnails/28.jpg)
Bernd Bruegge & Allen H. Dutoit Object-Oriented Software Engineering: Using UML, Patterns, and Java 28
Test
run(TestResult)
MyListTestCase
setUp()tearDown()runTest()testAdd()testRemove()
TestResult
TestCase
run(TestResult)setUp()tearDown()
testName:String
runTest()
TestSuite
run(TestResult)addTest()
MyList
add()remove()contains()size()
*
![Page 29: Chapter 11, Testing](https://reader035.vdocuments.mx/reader035/viewer/2022062520/56815c71550346895dca8594/html5/thumbnails/29.jpg)
Bernd Bruegge & Allen H. Dutoit Object-Oriented Software Engineering: Using UML, Patterns, and Java 29
Writing TestCases in JUnitpublic class MyListTestCase extends TestCase {
public MyListTestCase(String name) {super(name);
}public void testAdd() { // Set up the test
List aList = new MyList();String anElement = “a string”;
// Perform the testaList.add(anElement);
// Check if test succeededassertTrue(aList.size() == 1);assertTrue(aList.contains(anElement));
}protected void runTest() {
testAdd();}}
Test
run(TestResult)
MyListTestCase
setUp()tearDown()runTest()testAdd()testRemove()
TestResult
TestCase
run(TestResult)setUp()tearDown()
testName:String
runTest()
TestSuite
run(TestResult)addTest()
MyList
add()remove()contains()size()
*
![Page 30: Chapter 11, Testing](https://reader035.vdocuments.mx/reader035/viewer/2022062520/56815c71550346895dca8594/html5/thumbnails/30.jpg)
Bernd Bruegge & Allen H. Dutoit Object-Oriented Software Engineering: Using UML, Patterns, and Java 30
Writing Fixtures and Test Casespublic class MyListTestCase extends TestCase {// …private MyList aList;private String anElement;public void setUp() {
aList = new MyList();anElement = “a string”;
}
public void testAdd() {aList.add(anElement);assertTrue(aList.size() == 1);assertTrue(aList.contains(anElement));
}
public void testRemove() {aList.add(anElement);aList.remove(anElement);assertTrue(aList.size() == 0);assertFalse(aList.contains(anElement));
}
Test Fixture
Test Case
Test Case
![Page 31: Chapter 11, Testing](https://reader035.vdocuments.mx/reader035/viewer/2022062520/56815c71550346895dca8594/html5/thumbnails/31.jpg)
Bernd Bruegge & Allen H. Dutoit Object-Oriented Software Engineering: Using UML, Patterns, and Java 31
Collecting TestCases into TestSuitespublic static Test suite() {
TestSuite suite = new TestSuite();suite.addTest(new MyListTest(“testAdd”));suite.addTest(new MyListTest(“testRemove”));return suite;
}Test
run(TestResult)
TestCase
run(TestResult)setUp()tearDown()
testName:String
runTest()
TestSuite
run(TestResult)addTest()
Composite Pattern!
*
![Page 32: Chapter 11, Testing](https://reader035.vdocuments.mx/reader035/viewer/2022062520/56815c71550346895dca8594/html5/thumbnails/32.jpg)
Bernd Bruegge & Allen H. Dutoit Object-Oriented Software Engineering: Using UML, Patterns, and Java 32
Design patterns in JUnitTest
run(TestResult)
ConcreteTestCase
setUp()tearDown()runTest()
TestResult
TestCase
run(TestResult)setUp()tearDown()
testName:String
runTest()
TestSuite
run(TestResult)addTest()
Command Pattern
CompositePattern
Adapter Pattern
TestedUnit
*
![Page 33: Chapter 11, Testing](https://reader035.vdocuments.mx/reader035/viewer/2022062520/56815c71550346895dca8594/html5/thumbnails/33.jpg)
Bernd Bruegge & Allen H. Dutoit Object-Oriented Software Engineering: Using UML, Patterns, and Java 33
Design patterns in JUnitTest
run(TestResult)
ConcreteTestCase
setUp()tearDown()runTest()
TestResult
TestCase
run(TestResult)setUp()tearDown()
testName:String
runTest()
TestSuite
run(TestResult)addTest()
Command Pattern
CompositePattern
Adapter Pattern
TestedUnit
*
![Page 34: Chapter 11, Testing](https://reader035.vdocuments.mx/reader035/viewer/2022062520/56815c71550346895dca8594/html5/thumbnails/34.jpg)
Usi
ng U
ML,
Pat
tern
s, an
d Ja
vaO
bjec
t-O
rien
ted
Soft
war
e E
ngin
eeri
ngChapter 11, Testing, Part 2: Integration and System Testing
![Page 35: Chapter 11, Testing](https://reader035.vdocuments.mx/reader035/viewer/2022062520/56815c71550346895dca8594/html5/thumbnails/35.jpg)
Bernd Bruegge & Allen H. Dutoit Object-Oriented Software Engineering: Using UML, Patterns, and Java 35
Overview
• Integration testing• Big bang• Bottom up• Top down• Sandwich• Continuous
• System testing• Functional• Performance
• Acceptance testing• Summary
![Page 36: Chapter 11, Testing](https://reader035.vdocuments.mx/reader035/viewer/2022062520/56815c71550346895dca8594/html5/thumbnails/36.jpg)
Bernd Bruegge & Allen H. Dutoit Object-Oriented Software Engineering: Using UML, Patterns, and Java 36
Integration Testing
• The entire system is viewed as a collection of subsystems (sets of classes) determined during the system and object design
• Goal: Test all interfaces between subsystems and the interaction of subsystems
• The Integration testing strategy determines the order in which the subsystems are selected for testing and integration.
![Page 37: Chapter 11, Testing](https://reader035.vdocuments.mx/reader035/viewer/2022062520/56815c71550346895dca8594/html5/thumbnails/37.jpg)
Bernd Bruegge & Allen H. Dutoit Object-Oriented Software Engineering: Using UML, Patterns, and Java 37
Why do we do integration testing?
• Unit tests only test the unit in isolation
• Many failures result from faults in the interaction of subsystems
• Often many Off-the-shelf components are used that cannot be unit tested
• Without integration testing the system test will be very time consuming
• Failures that are not discovered in integration testing will be discovered after the system is deployed and can be very expensive.
![Page 38: Chapter 11, Testing](https://reader035.vdocuments.mx/reader035/viewer/2022062520/56815c71550346895dca8594/html5/thumbnails/38.jpg)
Bernd Bruegge & Allen H. Dutoit Object-Oriented Software Engineering: Using UML, Patterns, and Java 38
Stubs and drivers
• Driver:• A component, that calls the TestedUnit• Controls the test cases
• Stub:• A component, the TestedUnit
depends on• Partial implementation• Returns fake values.
Driver
TestedUnit
Stub
![Page 39: Chapter 11, Testing](https://reader035.vdocuments.mx/reader035/viewer/2022062520/56815c71550346895dca8594/html5/thumbnails/39.jpg)
Bernd Bruegge & Allen H. Dutoit Object-Oriented Software Engineering: Using UML, Patterns, and Java 39
Example: A 3-Layer-Design
Layer I
Layer II
Layer III
SpreadSheetView
A
Calculator
C
BinaryFileStorage
E
XMLFileStorage
F
CurrencyDataBase
G
CurrencyConverter
D
DataModel
B
A
C
E F G
DB
SpreadSheetView
BinaryFileStorage
EntityModel
A
E F
CurrencyDataBase
G
CurrencyConverter
DB
Calculator
C
XMLFileStorage
(Spreadsheet)
![Page 40: Chapter 11, Testing](https://reader035.vdocuments.mx/reader035/viewer/2022062520/56815c71550346895dca8594/html5/thumbnails/40.jpg)
Bernd Bruegge & Allen H. Dutoit Object-Oriented Software Engineering: Using UML, Patterns, and Java 40
A
C
E F G
DB
Big-Bang Approach
Test A
Test B
Test G
Test F
Test E
Test C
Test DTest
A, B, C, D,E, F, G
Unit tests each of the subsystems, and then does one gigantic integration test, in which all the subsystems are immediately tested together.
Not so good. Why?
![Page 41: Chapter 11, Testing](https://reader035.vdocuments.mx/reader035/viewer/2022062520/56815c71550346895dca8594/html5/thumbnails/41.jpg)
Bernd Bruegge & Allen H. Dutoit Object-Oriented Software Engineering: Using UML, Patterns, and Java 41
Big Bang Approach
• Pro• Easy• No additional test drivers and stubs are needed
• Con• Impossible to distinguish failures in the interface from
the failures within a component• Difficult to pinpoint the specific component responsible
for the failure
![Page 42: Chapter 11, Testing](https://reader035.vdocuments.mx/reader035/viewer/2022062520/56815c71550346895dca8594/html5/thumbnails/42.jpg)
Bernd Bruegge & Allen H. Dutoit Object-Oriented Software Engineering: Using UML, Patterns, and Java 42
Bottom-up Testing Strategy• The subsystems in the lowest layer of the call
hierarchy are tested individually• Then the next subsystems are tested that call the
previously tested subsystems• This is repeated until all subsystems are included• Drivers are needed.
![Page 43: Chapter 11, Testing](https://reader035.vdocuments.mx/reader035/viewer/2022062520/56815c71550346895dca8594/html5/thumbnails/43.jpg)
Bernd Bruegge & Allen H. Dutoit Object-Oriented Software Engineering: Using UML, Patterns, and Java 43
A
C
E F G
DB
Bottom-up Integration A
Test A, B, C, D,
E, F, G
ETest E
F
Test F
B
Test B, E, F
C
Test C
D
Test D,G
G
Test G
Test D
![Page 44: Chapter 11, Testing](https://reader035.vdocuments.mx/reader035/viewer/2022062520/56815c71550346895dca8594/html5/thumbnails/44.jpg)
Bernd Bruegge & Allen H. Dutoit Object-Oriented Software Engineering: Using UML, Patterns, and Java 44
Bottom-up
• Pro:• Simple
• Con:• Subsystem interfaces are not tested
• Impossible to distinguish failures in interfaces from failures within a component
• Difficult to pinpoint the specific component responsible for the failure
![Page 45: Chapter 11, Testing](https://reader035.vdocuments.mx/reader035/viewer/2022062520/56815c71550346895dca8594/html5/thumbnails/45.jpg)
Bernd Bruegge & Allen H. Dutoit Object-Oriented Software Engineering: Using UML, Patterns, and Java 45
Pros and Cons of Bottom-Up Integration Testing• Con:
• Tests the most important subsystem (user interface) last
• Drivers needed• Pro
• No stubs needed• Useful for integration testing of the following systems
• Object-oriented systems• Real-time systems• Systems with strict performance requirements.
![Page 46: Chapter 11, Testing](https://reader035.vdocuments.mx/reader035/viewer/2022062520/56815c71550346895dca8594/html5/thumbnails/46.jpg)
Bernd Bruegge & Allen H. Dutoit Object-Oriented Software Engineering: Using UML, Patterns, and Java 46
Top-down Testing Strategy
• Test the top layer or the controlling subsystem first
• Then combine all the subsystems that are called by the tested subsystems and test the resulting collection of subsystems
• Do this until all subsystems are incorporated into the test
• Stubs are needed to do the testing.
![Page 47: Chapter 11, Testing](https://reader035.vdocuments.mx/reader035/viewer/2022062520/56815c71550346895dca8594/html5/thumbnails/47.jpg)
Bernd Bruegge & Allen H. Dutoit Object-Oriented Software Engineering: Using UML, Patterns, and Java 47
Top-down Integration
Test A, B, C, D,
E, F, G
All LayersLayer I + II
Test A, B, C, D
Layer I
Test A
A
E F
B C D
G
![Page 48: Chapter 11, Testing](https://reader035.vdocuments.mx/reader035/viewer/2022062520/56815c71550346895dca8594/html5/thumbnails/48.jpg)
Bernd Bruegge & Allen H. Dutoit Object-Oriented Software Engineering: Using UML, Patterns, and Java 48
Pros and Cons of Top-down Integration TestingPro• Test cases can be defined in terms of the
functionality of the system (functional requirements)
• No drivers needed
Cons• Writing stubs is difficult: Stubs must allow all
possible conditions to be tested.• Large number of stubs may be required,
especially if the lowest level of the system contains many methods.
• Some interfaces are not tested separately.
![Page 49: Chapter 11, Testing](https://reader035.vdocuments.mx/reader035/viewer/2022062520/56815c71550346895dca8594/html5/thumbnails/49.jpg)
Bernd Bruegge & Allen H. Dutoit Object-Oriented Software Engineering: Using UML, Patterns, and Java 49
Sandwich Testing Strategy
• Combines top-down strategy with bottom-up strategy
• The system is viewed as having three layers• A target layer in the middle• A layer above the target• A layer below the target
• Testing converges at the target layer.
![Page 50: Chapter 11, Testing](https://reader035.vdocuments.mx/reader035/viewer/2022062520/56815c71550346895dca8594/html5/thumbnails/50.jpg)
Bernd Bruegge & Allen H. Dutoit Object-Oriented Software Engineering: Using UML, Patterns, and Java 50
Sandwich Testing Strategy
Test A, B, C, D,
E, F, GTest B, E, F
Test D,G
Test A
Test E
Test F
Test G
Test A,B,C, D
A
E F
B C D
G
![Page 51: Chapter 11, Testing](https://reader035.vdocuments.mx/reader035/viewer/2022062520/56815c71550346895dca8594/html5/thumbnails/51.jpg)
Bernd Bruegge & Allen H. Dutoit Object-Oriented Software Engineering: Using UML, Patterns, and Java 51
Pros and Cons of Sandwich Testing
• Top and Bottom Layer Tests can be done in parallel
• Problem: Does not test the individual subsystems and their interfaces thoroughly before integration
• Solution: Modified sandwich testing strategy
![Page 52: Chapter 11, Testing](https://reader035.vdocuments.mx/reader035/viewer/2022062520/56815c71550346895dca8594/html5/thumbnails/52.jpg)
Bernd Bruegge & Allen H. Dutoit Object-Oriented Software Engineering: Using UML, Patterns, and Java 52
Modified Sandwich Testing Strategy
• Test in parallel:• Middle layer with drivers and stubs• Top layer with stubs• Bottom layer with drivers
• Test in parallel:• Top layer accessing middle layer (top layer
replaces drivers)• Bottom accessed by middle layer (bottom
layer replaces stubs).
![Page 53: Chapter 11, Testing](https://reader035.vdocuments.mx/reader035/viewer/2022062520/56815c71550346895dca8594/html5/thumbnails/53.jpg)
Bernd Bruegge & Allen H. Dutoit Object-Oriented Software Engineering: Using UML, Patterns, and Java 53
Modified Sandwich Testing
Test F
Test E
Test B
Test G
Test D
Test A
Test C
Test B, E, F
Test D,G
Test A,C
Test A, B, C, D,
E, F, G
A
E F
B C D
G
![Page 54: Chapter 11, Testing](https://reader035.vdocuments.mx/reader035/viewer/2022062520/56815c71550346895dca8594/html5/thumbnails/54.jpg)
Bernd Bruegge & Allen H. Dutoit Object-Oriented Software Engineering: Using UML, Patterns, and Java 54
Continuous Testing
• Continuous build:• Build from day one• Test from day one• Integrate from day one System is always runnable
• Requires integrated tool support:• Continuous build server• Automated tests with high coverage• Tool supported refactoring• Software configuration management• Issue tracking.
![Page 55: Chapter 11, Testing](https://reader035.vdocuments.mx/reader035/viewer/2022062520/56815c71550346895dca8594/html5/thumbnails/55.jpg)
Bernd Bruegge & Allen H. Dutoit Object-Oriented Software Engineering: Using UML, Patterns, and Java 55
SpreadSheetView
BinaryFileStorage
DataModel
Continuous Testing Strategy
Layer I
Layer II
Layer III
A
E F
CurrencyDataBase
G
CurrencyConverter
DB
Calculator
C
XMLFileStorage
Sheet View + Cells+ Addition + File Storage
![Page 56: Chapter 11, Testing](https://reader035.vdocuments.mx/reader035/viewer/2022062520/56815c71550346895dca8594/html5/thumbnails/56.jpg)
Bernd Bruegge & Allen H. Dutoit Object-Oriented Software Engineering: Using UML, Patterns, and Java 56
Steps in Integration Testing
.
1. Based on the integration strategy, select a component to be tested. Unit test all the classes in the component.
2. Put selected component together; do any preliminary fix-up necessary to make the integration test operational (drivers, stubs)
3. Test functional requirements: Define test cases that exercise all use cases with the selected component
4. Test subsystem decomposition: Define test cases that exercise all dependencies
5. Test non-functional requirements: Execute performance tests
6. Keep records of the test cases and testing activities.
7. Repeat steps 1 to 7 until the full system is tested.
The primary goal of integration testing is to identify failures with the (current) component configuration.
![Page 57: Chapter 11, Testing](https://reader035.vdocuments.mx/reader035/viewer/2022062520/56815c71550346895dca8594/html5/thumbnails/57.jpg)
Bernd Bruegge & Allen H. Dutoit Object-Oriented Software Engineering: Using UML, Patterns, and Java 57
System Testing
• Functional Testing• Validates functional requirements
• Performance Testing• Validates non-functional requirements
• Acceptance Testing• Validates clients expectations
UnitTesting
AcceptanceTesting
IntegrationTesting
SystemTesting
![Page 58: Chapter 11, Testing](https://reader035.vdocuments.mx/reader035/viewer/2022062520/56815c71550346895dca8594/html5/thumbnails/58.jpg)
Bernd Bruegge & Allen H. Dutoit Object-Oriented Software Engineering: Using UML, Patterns, and Java 58
.
Functional Testing
Goal: Test functionality of system• Test cases are designed from the requirements
analysis document (better: user manual) and centered around requirements and key functions (use cases)
• The system is treated as black box• Unit test cases can be reused, but new test
cases have to be developed as well.
![Page 59: Chapter 11, Testing](https://reader035.vdocuments.mx/reader035/viewer/2022062520/56815c71550346895dca8594/html5/thumbnails/59.jpg)
Bernd Bruegge & Allen H. Dutoit Object-Oriented Software Engineering: Using UML, Patterns, and Java 59
Performance Testing
Goal: Try to violate non-functional requirements• Test how the system behaves when overloaded.
• Can bottlenecks be identified? (First candidates for redesign in the next iteration)
• Try unusual orders of execution • Call a receive() before send()
• Check the system’s response to large volumes of data
• If the system is supposed to handle 1000 items, try it with 1001 items.
• What is the amount of time spent in different use cases?
• Are typical cases executed in a timely fashion?
![Page 60: Chapter 11, Testing](https://reader035.vdocuments.mx/reader035/viewer/2022062520/56815c71550346895dca8594/html5/thumbnails/60.jpg)
Bernd Bruegge & Allen H. Dutoit Object-Oriented Software Engineering: Using UML, Patterns, and Java 60
Types of Performance Testing• Stress Testing
• Stress limits of system• Volume testing
• Test what happens if large amounts of data are handled
• Configuration testing• Test the various software and
hardware configurations • Compatibility test
• Test backward compatibility with existing systems
• Timing testing• Evaluate response times and
time to perform a function
• Security testing• Try to violate security
requirements• Environmental test
• Test tolerances for heat, humidity, motion
• Quality testing• Test reliability, maintain-
ability & availability • Recovery testing
• Test system’s response to presence of errors or loss of data
• Human factors testing• Test with end users.
![Page 61: Chapter 11, Testing](https://reader035.vdocuments.mx/reader035/viewer/2022062520/56815c71550346895dca8594/html5/thumbnails/61.jpg)
Bernd Bruegge & Allen H. Dutoit Object-Oriented Software Engineering: Using UML, Patterns, and Java 61
Acceptance Testing
• Goal: Demonstrate system is ready for operational use
• Choice of tests is made by client
• Many tests can be taken from integration testing
• Acceptance test is performed by the client, not by the developer.
• Alpha test:• Client uses the software
at the developer’s environment.
• Software used in a controlled setting, with the developer always ready to fix bugs.
• Beta test:• Conducted at client’s
environment (developer is not present)
• Software gets a realistic workout in target environ- ment
![Page 62: Chapter 11, Testing](https://reader035.vdocuments.mx/reader035/viewer/2022062520/56815c71550346895dca8594/html5/thumbnails/62.jpg)
Bernd Bruegge & Allen H. Dutoit Object-Oriented Software Engineering: Using UML, Patterns, and Java 62
Testing has many activities
Establish the test objectives
Design the test cases
Write the test cases
Test the test cases
Execute the tests
Evaluate the test results
Change the system
Do regression testing
![Page 63: Chapter 11, Testing](https://reader035.vdocuments.mx/reader035/viewer/2022062520/56815c71550346895dca8594/html5/thumbnails/63.jpg)
Bernd Bruegge & Allen H. Dutoit Object-Oriented Software Engineering: Using UML, Patterns, and Java 63
Test Team
Test
Analyst
TeamUser
Programmertoo familiarwith code
ProfessionalTester
Configuration Management
Specialist
System Designer
![Page 64: Chapter 11, Testing](https://reader035.vdocuments.mx/reader035/viewer/2022062520/56815c71550346895dca8594/html5/thumbnails/64.jpg)
Bernd Bruegge & Allen H. Dutoit Object-Oriented Software Engineering: Using UML, Patterns, and Java 64
The 4 Testing Steps
1. Select what has to be tested• Analysis: Completeness of
requirements• Design: Cohesion• Implementation: Source
code2. Decide how the testing is
done• Review or code inspection• Proofs (Design by Contract)• Black-box, white box, • Select integration testing
strategy (big bang, bottom up, top down, sandwich)
3. Develop test cases• A test case is a set of test
data or situations that will be used to exercise the unit (class, subsystem, system) being tested or about the attribute being measured
4. Create the test oracle• An oracle contains the
predicted results for a set of test cases
• The test oracle has to be written down before the actual testing takes place.
![Page 65: Chapter 11, Testing](https://reader035.vdocuments.mx/reader035/viewer/2022062520/56815c71550346895dca8594/html5/thumbnails/65.jpg)
Bernd Bruegge & Allen H. Dutoit Object-Oriented Software Engineering: Using UML, Patterns, and Java 65
Guidance for Test Case Selection• Use analysis knowledge
about functional requirements (black-box testing):
• Use cases• Expected input data• Invalid input data
• Use design knowledge about system structure, algorithms, data structures (white-box testing):
• Control structures• Test branches,
loops, ...• Data structures
• Test records fields, arrays, ...
• Use implementation knowledge about algorithms and datastructures:
• Force a division by zero• If the upper bound of an
array is 10, then use 11 as index.
![Page 66: Chapter 11, Testing](https://reader035.vdocuments.mx/reader035/viewer/2022062520/56815c71550346895dca8594/html5/thumbnails/66.jpg)
Bernd Bruegge & Allen H. Dutoit Object-Oriented Software Engineering: Using UML, Patterns, and Java 66
Summary
• Testing is still a black art, but many rules and heuristics are available
• Testing consists of• Unit testing• Integration testing• System testing• Acceptance testing
• Design patterns can be used for integration testing
• Testing has its own lifecycle