Bottom-Up Integration TestingAfter unit testing of individual components the components are combined together into a system.Bottom-Up Integration: each component at lower hierarchy is tested individually; then the components that rely upon these are tested.Component Driver: a routine that simulates a test call from parent component to child component

Bottom-Up Testing ExampleABCTest B, C individually (using drivers)Test A such that it calls B If an error occurs we know that the problem is in A or in the interface between A and BTest A such that it calls C If an error occurs we know that the problem is in A or in the interface between A and C(-) Top level components are the most important yet tested last.

Top-Down Testing ExampleABCTest A individually (use stubs for B and C)Test A such that it calls B (stub for C) If an error occurs we know that the problem is in B or in the interface between A and BTest A such that it calls C (stub for B) If an error occurs we know that the problem is in C or in the interface between A and C* Stubs are used to simulate the activity of components that are not currently tested; (-) may require many stubs

Big-Bang IntegrationAfter all components are unit testing we may test the entire system with all its components in action. (-) may be impossible to figure out where faults occur unless faults are accompanied by component-specific error messages

Sandwich IntegrationMulti-level component hierarchy is divided into three levels with the test target being in the middle:Top-down approach is used in the top layer;Bottom-down approach used in the lower layer.

Sync & Stabilize Approach

Integration Strategies Comparison

Testing Object Oriented SystemsObjects tend to be small and simple while the complexity is pushed out into the interfaces.Hence unit testing tends to be easy and simple, but the integration testing complex and tedious.(-) Overridden virtual methods require thorough testing just as the base class methods.

Testing Object Oriented Systems

Test-Related ActivitiesDefine test objectives (i.e. what qualities of the system you want to verify)Devise test casesProgram (and verify!) the testsExecute testsAnalyze the results

Test PlanA document that describes the criteria and the activities necessary for showing that the software works correctly.A test plan is well-defined when upon completion of testing every interested party can recognize that the test objectives has been met.

Test Plan ContentsWhat automated test tools are used (if any)Methods for each stage of testing (e.g. code walk-through for unit testing, top-down integration, etc.)Detailed list of test cases for each test stageHow test data is selected or generatedHow output data & state information is to be captured and analyzed

Static & Dynamic AnalysisStatic Analysis: uncovers minor coding violations by analyzing code at compile time (e.g. compiler warnings, FxCop) Dynamic Analysis: runtime monitoring of code (e.g. profiling, variable monitoring, asserts, tracing, exceptions, runtime type information, etc)

Automated TestingStub & driver generationUnit test cases generationKeyboard input simulation / terminal user impersonationWeb request simulation / web user impersonationCode coverage calculation / code path tracingComparing output to the desired

When to Stop Testing?Run out of time, money?Curiously, the more faults we find at the beginning of testing the greater the probability that we will find even more faults if we keep testing further!

Finding and testing all possible path combinations though the code?

Fault SeedingFaults are intentionally inserted into the code. Then the test process is used to locate them.The idea is that:Detected seeded faults/total seeded faults = detected nonseeded faults/total nonseeded faultsThus we can obtain a measure of effectiveness of the test process and use it to adjust test time and thoroughness.

Confidence in the SoftwareThe likelihood that the software is fault-free:C = S/(S-N+1), if n NWhere S the number of seeded faults, N the expected number of indigenous faults, n number of indigenous faults found.

ReadChapter 8 from the white book.