dynamic analysis in software testing
DESCRIPTION
Small description of dynamic analysis in software testing..TRANSCRIPT
The testing of SW is seen as the execution of the test object The test object is provided with test data At the beginning, the test basis is analysed to determine what must be tested It must be determined how the individual requirements & the test cases relate to each other
Three types of systematic techniqueStatic (non-execution)
• examination of documentation,source code listings, etc.
Functional (Black Box)• based on behaviour /
functionality of software
Structural (White Box)• based on structure
of software
Black Box Testing Technique Here the test object is seen as a black box It is the specification-based testing techniques Also known as input/output-driven testing techniques
because they view the software as a black-box with inputs and outputs, but they have no knowledge of how the system or component is structured inside the box
In essence, the tester is concentrating on what the software does, not how it does it
Test cases are derived from the specification of the test object and behavior is observed from outside(PoO is outside the test object)
Not possible to control the operating sequence of the test object(except adequate input test data) as PoC is outside of test object
Black Box Testing Technique The Black-box testing attempts to find errors in :1.To test the modules independently2.To test the functional validity of the software so
that incorrect or missing functions can be recognized
3.To look for interface errors4.To test the system behaviour and check its
performance5.To test the maximum load or stress on the system6.To test the software such that the user/customer
accepts the system within defined acceptable limits
Black Box Testing Technique Both functional and non-functional testing are considered Functional testing is concerned with what the system
does, its features or functions Non-functional testing is concerned with examining how
well the system does something, rather than what it does Non-functional aspects (also known as quality
characteristics or quality attributes) include performance, usability, portability, maintainability, etc
Techniques to test these non-functional aspects are less procedural and less formalized than those of other categories as the actual tests are more dependent on the type of system, what it does and the resources available for the tests
White Box Testing Technique Structure-based testing techniques (which are also dynamic
rather than static) use the internal structure of the software to derive test cases
Also called 'glass-box' techniques (implying we can see into the system) since they require knowledge of how the software is implemented, that is, how it works. For example, a structural technique may be concerned with exercising loops in the software
Here the source code is known and used for test design While executing the test cases, the internal processing of the test
object and the output is analyzed PoC and/or PoO inside the test object Test cases are designed to cover the program structure of the test
object This is called structural testing as structure of the test
object(component hierarchy, flow control, data flow) are considered
Using black box testing, the test object is seen as a black box Test cases are derived from the specification of the test object The behavior of the test object is watched from the outside (PoO – Point of Observation is outside the test object) It is not possible to control the operating sequence of the test object other than choosing the adequate input test data (the PoC – Point of Control is situated outside of test object, too) Test cases are designed by using the specification or the requirements of the test object.
In white box testing, the source code is known and used for test design While executing the test cases, the internal processing of the test object, as well as the output, is analyzed (the Point ofObservation is inside of the test object) Direct intervention in the process of the test object is possible, but should be used only in special situations, e.g. to execute negative testing when the component's interface is not capable of initiating the provoked failure (the Point of Control can be located inside the test object) Test cases are designed to cover the program structure of the test object
White box testing is also called structural testing, because the test designer considers the structure (component hierarchy, flow control, data flow) of the test object The black box testing techniques are known as functional or behavioral testing techniques, because of the observation of the input/output behavior .The functionality of the test object is the center of attention White box testing can be applied at the lower levels of the testing, i.e., component and integration test Black box testing is predominantly used for higher levels of testing even though it is reasonable in component tests Any test design before the code is written (test-first programming, test-driven development) is essentially black box driven
IntegrationIntegration
ComponentComponent
AcceptanceAcceptance
SystemSystem
Black box appropriateat all levels butdominates higherlevels of testing
White box usedpredominatelyat lower levelsto complimentblack box
Black Box test design techniques
Techniques defined : Equivalence partitioningBoundary value analysisState transition testingCause-effect graphingSyntax testingRandom testing
Black Box Testing Technique This test technique is used when the inner
structure & design of the test object is unknown or not considered
A test with all possible input data combinations would be a complete test, but this is unrealistic considering the enormous no. of combinations.
Test design must make a reasonable selection of all possible test cases
Equivalence Class Partitioning• Equivalence Class Partitioning : The domain of possible input data for each input data
element is divided into equivalence class This is a group of data values where the tester assumes
that the test object processes them in the same way These classes should be tested for correct input and
incorrect input as well For every input data element that should be tested the
domain of all possible correct inputs should be determined & then all values outside these domains are wrong inputs
Equivalence classes are refined if the test object’s specification tells that some elements of that class are processed differently ,they are assigned to a new sub-class
Equivalence Class Partitioning• Determination of Equivalence Class : This partitioning is done such that the behavior
of the program is similar for every input data belonging to the same equivalence class
The main idea behind defining the equivalence classes is that testing the code with any one value belonging to an equivalence class is as good as testing the software with any other value belonging to that equivalence class
Equivalence classes for a software can be designed by examining the input data and output data
Equivalence Class Partitioning• Determination of Equivalence Class : The probability of failure detection is highly
dependent upon the quality of the partitioning and which test cases are executed
For the input & output values, identify the restrictions & conditions in the specification
For every restriction or condition, if a continuous numerical domain is specified, then create one valid & two invalid classes
To complete the test cases, the tester must define the preconditions & the expected result for every test case
Equivalence Class PartitioningGeneral guidelines for designing the
equivalenceIf the input data values to a system can be
specified by a range of values, then one valid and two invalid equivalence classes should be defined.
If the input data assumes values from a set of discrete members of some domain, then one equivalence class for valid input values and another equivalence class for invalid input values should be defined.
Equivalence partitioning (EP)
divide (partition) the inputs, outputs, etc. into areas which are the same (equivalent)
assumption: if one value works, all will workone from each partition better than all from
one
1 100 1010
valid invalidinvalid
Equivalence Class PartitioningExample#1:For a software that computes
the square root of an input integer which can assume values in the range of 0 to 5000, how many equivalence classes are there ?
Example#2: Design the black-box test suite for the following program. The program computes the intersection point of two straight lines and displays the result. It reads two integer pairs (m1, c1) and (m2, c2) defining the two straight lines of the form y=mx + c.
Equivalence Class PartitioningRules for test case determinationRestriction of the number of test cases The number of "valid" test cases is the product of
the number of valid equivalence classes per parameter
Because of this multiplicative combination, even a few parameters can generate hundreds of "valid test cases“
Since it is seldom possible to use that many test cases, more rules are necessary to reduce the number of "valid" test cases
Equivalence Class PartitioningRules for test case restriction Combine the test cases and sort them by frequency of
occurrence (typical usage profile)Prioritize the test cases in this order. That way only the
"relevant" test cases (often appearing combinations) are tested
Test cases including boundary values or boundary value combinations are preferred
Ensure as minimum criteria that every representative of an equivalence class appears in at least one test case
Representatives of invalid equivalence classes should not be combined with representatives of other invalid equivalence classes.
Equivalence Class Partitioning• Test Completion Criteria : The Percentage of executed equivalence classes
in comparison to the total no. of specified equivalence classes , i.e.
EC-coverage=(no. of tested EC/total no. of EC)*100%
• Evaluation : It gives a complete test coverage where specified
conditions & restrictions are not overlooked Drawback: only single input or output conditions
are considered while possible dependencies or interactions between conditions are ignored
Boundary Value AnalysisThis checks the “border” of the equivalence classes.
On every boundary, the exact boundary value & both nearest adjacent values (inside & outside the equivalence class)are tested.
Three test cases result from every boundary & if the upper boundary of one equivalence class equals the lower boundary of the adjacent class, then the respective test cases coincide as well
When there is no real boundary value as it belongs to another class, it is sufficient to test the boundary with two values: one which is just inside & another just outside the class
Boundary Value AnalysisA type of programming error frequently occurs at
the boundaries of different equivalence classes of inputs
The reason behind such errors might purely be due to psychological factors
Programmers often fail to see the special processing required by the input values that lie at the boundary of the different equivalence classes
For example, programmers may improperly use < instead of <=, or conversely <= for <. Boundary value analysis leads to selection of test cases at the boundaries of the different equivalence classes
Boundary Value AnalysisWe can also apply equivalence partitioning and boundary
value analysis more than once to the same specification item. For example, if an internal telephone system for a company with 200 telephones has 3-digit extension numbers from 100 to 699, we can identify the following partitions and boundaries:
• digits (characters 0 to 9) with the invalid partition containing non-digits
• number of digits, 3 (so invalid boundary values of 2 digits and 4 digits) • range of extension numbers, 100 to 699 (so invalid
boundary values of 099 and 700) • extensions that are in use and those that are not (two
valid partitions, no boundaries) • the lowest and highest extension numbers that are in use
could also be used as boundary values
Boundary Value AnalysisOne test case could test more than one of these
partitions/boundariesFor example, Extension 409 which is in use would test
four valid partitions: digits, the number of digits, the valid range, and the 'in use' partition. It also tests the boundary values for digits, 0 and 9.
How many test cases would we need to test all of these partitions and boundaries, both valid and invalid? We would need a non-digit, a 2-digit and 4-digit number, the values of 99, 100, 699 and 700, one extension that is not in use, and possibly the lowest and highest extensions in use. This is ten or eleven test cases - the exact number would depend on what we could combine in one test case.
Boundary Value AnalysisExample: ]
For a function that computes the square root of integer values in the range of 0 and 5000, the test cases must include the following values: {0, -1,5000,5001}.
Boundary Value Analysis
Test Completion Criteria : Intended coverage of the boundary values(BV)
can be predefined & calculated as after execution of the tests:
BV-coverage = ( No. of tested BV)/(Total No. of BV)*100%
State Transition TestingIn many cases the current input as well as the history
of execution or events of inputs, influences the outputs & how the test object will behave. To show the dependence on history state diagrams are used
The system or test object starts from an initial state & come to a different state. Events trigger transitions.
It consists of states(nodes),transitions(links),inputs(link weights) & outputs(link weights).
A state transition test should execute all specified functions of a certain state at least once.
Precondition: stack is initialized; state is “empty”Input: Push(“Hello”)Expected result: Stack contains “hello”Post-Condition: State of the stack is “filled”
State Transition Testing• The system or test object starts from an initial state and can then come into different states• Events trigger state transitions where an event normally is a function invocation• State transitions can involve actions• Besides the initial state, the other special state is the end-state• Finite state machines, state diagrams, or state transition tables model this behavior• A finite state machine as follows: "An abstract machine (e.g., program, logic circuit, car's transmission) for which the number of states and input symbols are both finite and fixed. A finite state machine consists of states (nodes), transitions (links), inputs (link weights), and outputs (link weights)
State Transition Testing 'finite state machine’ simply means that the system can be in a (finite) number of different states, and the transitions from one state to another are determined by the rules of the 'machine‘ This is the model on which the system and the tests are based. Any system where a different output is got for the same input, depending on what has happened before, is a finite state system A finite state system is often shown as a state diagram
State Transition Testing A state transition model has four basic parts:
• the states that the software may occupy (open/closed or funded/insufficient funds); • the transitions from one state to another (not all transitions are allowed); • the events that cause a transition (closing a file or withdrawing money); • the actions that result from a transition (an error message or being given your cash)
State Transition Testing
State Transition Testing Deriving tests only from a state graph (also known as a state chart) is very good for seeing the valid transitions, but we may not easily see the negative tests, where we try to generate invalid transitions To see the total number of combinations of states and transitions, both valid and invalid, a state table is useful The state table lists all the states down one side of the table and all the events that cause transitions along the top (or vice versa) Each cell then represents a state-event pair. The content of each cell indicates which state the system will move to, when the corresponding event occurs while in the associated state This will include possible erroneous events - events that are not expected to happen in certain states. These are negative test conditions
State Transition Testing
State Transition Testing
State Transition Testing
Transition tree for the stack
State Transition TestingThe transition tree is built from a transition
diagram in the following way:1.The initial or start state is the root tree2.For every possible transition from the initial state to
a following state in the state transition diagram, the transition tree receives a branch from its root to a node, representing this next state
3.The process for step 2 is repeated foe every leaf in the tree(every newly added node) until one of the following two end-conditions is fulfilled: A) The corresponding state is already included in the tree on the way from the root to the node. This end condition corresponds to one pass of a cycle in the transition diagram. B) The corresponding state is a final state,and therefore has no further transitions to be considered.
State Transition TestingTest Completion Criteria :Criteria for test intensity & Completion can be
defined as:1.Every state has been reached at least once2.Every transition has been executed at least once3.Every transition violating the specification has
been checked4.Percentage can be defined using the proportion
of actually executed test requirements to possible ones
Cause Effect GraphingIt uses the dependencies for identification of the test
cases. The logical relationships between the causes & their effects in a component or a system are displayed here.
It must be possible to find the cause & effects from the specification
Every cause is described as a condition that consists of input conditions or their combinations & they are connected with logical operators
It uses the dependencies for identification of the test cases known as cause-effect graphing
Cause Effect GraphingThe logical relationships between the causes and
their effects in a component or a system are displayed in a so-called cause-effect graph
It must be possible to find the causes and effects from the specification
Every cause is described as a condition that consists of input conditions (or combinations of those). The conditions are connected with logical operators (e.g., AND, OR and NOT)
A condition, and therefore a cause, can be TRue or false
The effects are treated in the same way and noted in the graph
Decision TableThe techniques of equivalence partitioning and
boundary value analysis are often applied to specific situations or inputs
If different combinations results in different actions being taken, this can be more difficult to show using equivalence partitioning and boundary value analysis, which tend to be more focused on the user interface
The other two specification-based techniques, decision tables and state transition testing are more focused on business logic or business rules
A decision table is a good way to deal with combinations of things (e.g. inputs). This technique is sometimes also referred to as a 'cause-effect' table. The reason for this is that there is an associated logic diagramming technique called 'cause-effect graphing' which was sometimes used to help derive the decision table
Decision TableThe graph is transformed into a decision table from which the test cases can be taken.
Criteria for test completion is to execute every column in the decision table by at least one test case. This verifies all sensible combinations & their corresponding effects.
Steps to transform a graph into a table:1. Choose an effect2. Looking into the graph, find combinations of
causes that have this effect and combinations that do not have this effect
3.one column into the table for every one of these cause-combinations and the caused states of the remaining effects
4.Check if decision table entries occur several times and if yes, delete them
Decision TableExample, withdrawing money at an automated
teller machine (ATM) shall illustrate how to prepare a cause-effect graph. In order to get money from the machine, the following conditions must be fulfilled[]:
The bankcard is validThe PIN must be correctly enteredThe maximum number of PIN inputs is threeThere is money in the machine, and in the accountThe following actions are possible at the machine:Reject cardAsk for another PIN input"Eat" the cardAsk for an alternate dollar amountPay the requested amount of money
Decision Table
Cause-effect graph of the ATM
Decision Table
Optimized decision table for the ATM
Decision Table
Empty decision table : Loan repayment Schedule
Decision Table
Decision table with input combinations
Decision Table
Decision table with combinations and outcomes
Decision Table
Decision table with additional outcomes
Decision Table :Example 2• You are a new customer opening a credit card
account, you will get a 15% discount on all your
purchases today. If you are an existing
customer and you hold a loyalty card, you get a
10% discount. If you have a coupon, you can
get 20% off today (but it can't be used with the
'new customer' discount). Discount amounts
are added, if applicable.
Decision Table :Example 2
Decision table for credit card example
Test Completion CriteriaA minimum requirement is to execute every
column in the decision table by at least one test case
This verifies all sensible combinations of conditions and their corresponding effects
Use Case TestingWith the increasing use of object-oriented methods
for software development, the Unified Modeling Language (UML) is used more frequently
There exist quite a number of (research) results and approaches to directly derive test cases from UML diagrams and to generate these tests more or less automatically
In order to detect requirements, use cases or business cases are described
These are then compiled into use case diagrams The diagrams serve the purpose of defining
requirements on a relatively abstract level and describing typical user system interactions
Use Case Testing
Use case diagram for ATM
Use Case TestingThe individual use cases in this example are "Get
money", "PIN query", and "Eat card" Relationships between use cases may be
"include" and "extend" "Include" conditions are always involved, and
"extend" connections can lead to extensions of a use case under certain conditions at a certain point (extension point)
Thus, the "extend" conditions are not always executed as there are alternatives
Use Case TestingUse case diagrams mainly serve to show the external view of a systemIt shall explain the external view of the system from the viewpoint of
the user or the relation to neighboring systemsSuch external connections are shown as lines to "actors“Pre- and post conditions : For every use case, there exist certain
preconditions that must be fulfilled to make it possible to execute the use case
A precondition for getting money at the ATM is, for example, that the bankcard is valid
After executing a use case there exist post-conditionsExample, after successfully entering the correct PIN, it is possible to
get moneyBut first the amount must be entered and it must be confirmed that
the money is available Preconditions and post-conditions are also applicable for the flow of
use cases in a diagram, i.e., the path through the diagram
Use Case TestingUseful for system and acceptance testing : Use
cases and use case diagrams serve as the basis for determining test cases in use case based testing
As the external view is modeled, the technique is useful for both system testing and acceptance testing
If the diagrams are used to model the interactions between different subsystems, test cases can also be derived for integration testing
Use Case Testing
Typical system use is tested : The diagrams show the "normal", "typical", or "probable" flows and often their alternatives
Thus, the use case based test checks typical use of a system
It is especially important for acceptance of a system that it runs relatively stable in "normal" use
Use case based testing has a high relevance for the customer, and therefore for the developer and tester as well
Use Case TestingEvery use case has a purpose and shall achieve a
certain result. Events may occur that lead to further alternatives or activities. After the execution, there are post-conditions
The information necessary for determining the test cases are:
Start situation and preconditionsOther possible conditionsExpected resultsPost-conditions
Use Case Testing : test completion criteriaA possible criterion is that every use case and
every possible sequence of use cases in the diagram is tested at least once by a test case
As alternatives and extensions are use cases too, this criterion also requires their execution
Black Box Techniques
Syntax testIt describes a technique for identification of the test
cases that may be applied if a formal specification of the syntax of the inputs is available
This may be the case for testing interpreters of command languages, compilers, and protocol analyzers
The rules of the syntax definition are used to specify test cases that cover both the compliance to and violation of the syntax rules for the inputs
Black Box Techniques
Random testRandom testing generates values for the test cases by
random selection If a statistical distribution of the input values is given
(e.g., normal distribution), then it can be used for the selection of test values
This ensures the derivation of test cases that are preferably close to reality,making it possible to use statistical models for predicting or certifying system reliability
Black Box TechniquesSmoke testA smoke test is commonly understood as a "quick and dirty“ test that
is primarily aimed at verifying a minimum reliability of the test objectThe test is concentrated on the main functions of the test objectThe output of the test is not evaluated in detail. The main outcome of
interest is a crash or serious misbehavior of the test objectA test oracle is not used, which contributes to making this test
inexpensive and easyThe term "smoke test" is derived from testing electrical circuits in
which short circuits lead to smoke rising A smoke test is often used to decide if the test object is mature
enough to proceed with further testing by the more comprehensive test techniques
Another use of smoke tests is the first test and the fast test of software updates
White Box TestingThe basis for this is the source-code of the test
object.So,the technique is called code-based testing techniques or structural testing techniques.
The generic idea is to execute every part of code of the test object at least once. But expected results should be determined using the requirements or specifications, not the code & this is done to check if execution results in a failure.
Types are: 1. Statement Coverage 2. Branch Coverage 3. Test of conditions (branch condition & condition
determination testing) 4. Path coverage
White Box TestingCoverage-based : It attempts to execute (or cover) certain elements of
a programExamples are Statement Coverage , Branch
Coverage etc.Fault-based :It targets to detect certain types of faultsThese faults that a test strategy focuses on
constitutes the fault model of the strategyExample is mutation testing
White Box Testing White box testing assumes that the tester can take a
look at the code for the application block and create test cases that look for any potential failure scenarios. During white box testing, you analyze the code of the application block and prepare test cases for testing the functionality to ensure that the class is behaving in accordance with the specifications and testing for robustness.
InputThe following input is required for white box testing: RequirementsFunctional specificationsHigh-level design documentsDetailed design documentsApplication block source code
White Box Testing ProcessCreate test plans. Identify all white box test scenarios
and prioritize them.Profile the application block. This step involves
studying the code at run time to understand the resource utilization, time spent by various methods and operations, areas in code that are not accessed, and so on.
Test the internal subroutines. This step ensures that the subroutines or the nonpublic interfaces can handle all types of data appropriately.
Test loops and conditional statements. This step focuses on testing the loops and conditional statements for accuracy and efficiency for different data inputs.
Perform security testing. White box security testing helps you understand possible security loopholes by looking at the way the code handles security.
White Box Testing ProcessOne white-box testing strategy is said to be
stronger than another strategy, if all types of errors detected by the first testing strategy is also detected by the second testing strategy, and the second testing strategy additionally detects some more types of errors
When two testing strategies detect errors that are different at least with respect to some types of errors, then they are called complementary
White Box Testing Process
Stronger and complementary testing strategies
Statement Coverage
The statement coverage strategy aims to design test cases so that every statement in a program is executed at least once
The principal idea governing the statement coverage strategy is that unless a statement is executed, it is very hard to determine if an error exists in that statement
Unless a statement is executed, it is very difficult to observe whether it causes failure due to some illegal memory access, wrong result computation, etc
But executing some statement once and observing that it behaves properly for that input value is no guarantee that it will behave correctly for all input values
Statement Coverage
This analysis focuses on each statement of the test object
The test cases shall execute a predefined minimum quota or even all statements of the test object
The first step is to translate the source code into a control flow graph which makes it easier to specify in detail the control elements that must be covered
In graph, the statements are represented as nodes & the control flow between the statements as edges
If sequence of unconditional statements appear in the program, they are treated as single node, as execution of the first statement of the sequence ensures execution all of the rest
The edges of the graph must be traversed in the following order:a, b, f, g, h, d, e
Statement CoverageThe Value of the TechniqueUnreachable code can be detected If complete coverage of all statements is required, and some
statements cannot be executed by any test cases, then this may be an indication for unreachable source code (dead statements).
Empty ELSE-parts are not considered If a condition statement (IF) has statements only after it is
fulfilled, i.e., after the THEN clause, and there is no ELSE, then the control flow graph has a (THEN-) edge, starting at the condition, with (at least) one node and a second outgoing (ELSE-) edge without nodes
The control flow of both these edges is reunited at the terminating (ENDIF-) node
For statement coverage an empty (ELSE-) edge (between IF and ENDIF) is irrelevant, such as when there is no requirement to cover it during the test
Possible missing statements after ELSE are not detected by a test using this criterion
Statement Coverage : Exampleint compute_gcd(x, y)int x, y;{while (x! = y){if (x>y) then x= x – y;else y= y – x; } return x;}By choosing the test set {(x=3, y=3), (x=4, y=3), (x=3, y=4)}, we
can exercise theprogram such that all statements are executed at least once
Statement CoverageDifferent combinations of edges of the graph can achieve
complete coverageThe cost of testing should always be minimized, which
means reaching the goal with the least possible number of test cases
The expected results and the expected behavior of the test object should be identified in advance from the specification
After execution, the expected and actual results, and the behavior of the test object, must be compared to detect any difference or failure
Test Completion Criteria : It can be defined as Statement Coverage=(No. of
executed statements/total No. of statements)*100%If complete coverage of all statements is required & some
statements can’t be executed by any test cases, that may be unreachable source code(dead statements)
Branch CoverageHere the attention is on the edges of the graphThe execution of each statement is not considered,
rather the execution of decisions. The result of decision determines which statement is executed next.
Testing should make sure every decision is executed with both possible outcome(TRUE & FALSE).That’s decision coverage is another name for it
In the branch coverage-based testing strategy, test cases are designed to make each branch condition to assume true and false values in turn
Branch testing is also known as edge testing as in this testing scheme, each edge of a program’s control flow graph is traversed at least once
Branch CoverageAdditional test cases necessaryIn the previous example additional test cases are
necessary if all branches of the control flow graph must be executed during the test
For 100% statement coverage a test case executing the following order of edges was sufficient:
a, b, f, g, h, d, eThe edges c, i, and k have not been executed in this test
caseThe edges c and k are empty branches of a condition,
while the edge i is the return to the beginning of the loop
Additionally, three further test cases are necessary:a, b, c, d, ea, b, f, g, i, g, h, d, ea, k, e
Branch CoverageConnection (edge) coverage of the control flow graphTogether, all four test cases result in a complete coverage of
the edges of the control flow graphWith that, all possible branches of the control flow in the
source code of the test object have been testedSome edges have been executed more than once. This seems
to be redundant, however, it cannot always be avoided In the example, the edges a and e are executed in every test
case because there is no alternative to these edgesFor each test case, other than the precondition and post-
condition, the expected result and expected behavior must be determined and then compared to the actual result and behavior
It is reasonable to record which of the branches have been executed in which test case
This helps to find faults, especially missing code in empty branches
Branch CoverageIt is obvious that branch testing guarantees
statement coverage and thus is a stronger testing strategy compared to the statement coverage-based testing
In the previous example Euclid’s GCD computation algorithm , the test cases for branch coverage can be {(x=3, y=3), (x=3, y=2), (x=4, y=3), (x=3, y=4)}.
Branch Coverage
Test Completion Criteria : Branch Coverage = (No. of executed
branches/Total No. of branches)*100In contrast to statement coverage it detects
missing statements in empty branchesA branch coverage of 100% should be aimed
forThe test can be categorized as sufficient only
if, in addition to all statements, every possible branch of the control flow is considered during test execution
Test of ConditionsIf a decision is based on several(part) conditions
connected by logical operators, then the complexity of the condition should be considered in the test & there comes the Branch Condition Testing
An atomic part of a condition is a condition that has no logical operators such as AND, OR, and NOT, but at the most, includes relation symbols such as ">" or "=". A condition in the source code of the test object can consist of multiple atomic partial conditions
Branch condition testingThe goal is that each atomic(partial) condition in the
test shall adopt the values TRUE or FALSE.This is a weaker criterion than statement or branch
coverage because it is not required that different logical values for the result of the complete condition are included in the test
Test of ConditionsIt meets the criteria of both statement & branch coverageThis is very comprehensive as it takes into account the
complexity of composed conditions but is expensive due to growing no. of possible combinations
An example for a composed condition is: x > 3 OR y < 5. The condition consists of two atomic partial conditions (x>3; y<5) connected by the logical operator OR
The goal of the branch condition testing is that every atomic part of conditions is evaluated once for each of the logical values
Both parts of the condition have each resulted in both logical values.
The result of the complete condition , however, is equal for both combinations
Branch condition testing is therefore a weaker criterion than statement or branch coverage because it is not required that different logical values for the result of the complete condition are included in the test
Test of Conditions
Branch Condition Combination TestingBranch condition combination testing, also called
multiple-condition coverage requires that all True-False combinations of the atomic partial conditions be exercised at least once
All variations should be built, if possibleBranch condition combination testing includes
statement and branch coverageThe complete condition gives both logical values as
resultsThus, branch condition combination testing meets
the criteria of statement, as well as branch coverage
Branch Condition Combination TestingIt is a more comprehensive criterion that also takes
into account the complexity of composed conditions But it is a very expensive technique due to the
growing number of atomic conditions which make the number of possible combinations grow exponentially (to 2n with n atomic parts of conditions)
A problem results from the fact that all combinations cannot always be implemented by test data
For the combined condition of 3<=x AND x<5 not all combinations with the according values for the variable x can be produced because the parts of conditions depend on each other
Condition Determination TestingHere all combinations are not considered &
consideration is given to every possible combination of logical values where the modification of the logical value of an atomic condition can change the logical value of the whole
For every logical combination of the conditions, it must be decided which test cases are sensitive to faults & for which combinations faults can be masked & those are not considered
Stated in another way, for a test case, every atomic condition has a meaningful impact on the result
Test cases in which the result does not depend on a change of an atomic condition need not be designed
So less no. of test cases need execution. For n no. of Boolean operands of the conditions amount lies between n+1 to 2n
Condition Determination TestingCombination of logical expressions are specially error-
prone & so, a comprehensive test is very important.Combined complex conditions can be split into a tree-
structure and then execute branch coverage to avoid this expensive technique
This intensive testing can be avoided if they have been subjected to review in which the correctness is verified
The example with the two atomic condition parts (x>3, y<5).
Four combinations are possible (2*2):1) x=6 (T), y=3 (T), x>3 OR y<5 (T)2) x=6 (T), y=8 (F), x>3 OR y<5 (T)3) x=2 (F), y=3 (T), x>3 OR y<5 (T)4) x=2 (F), y=8 (F), x>3 OR y<5 (F)
Condition Determination TestingFor the first combination the following applies: If the logical
value is calculated wrong for the first condition part (i.e., an incorrect condition is implemented) then the fault can change the logical value of the first condition part from true (T) to false (F)
But the result of the complete condition stays unchanged (T)The same applies for the second condition partFor the first combination incorrect results of each condition
part are masked because they have no effect on the result of the complete condition and thus failures will not become visible
Consequently the test with the first combination can be left out
If the logical value of the first condition part in the second test case is calculated wrongly as false, then the result value of the total condition changes from true (T) to false (F)
A failure then becomes visible because the value of the complete condition has also changed
Condition Determination TestingBecause of the weak significance, branch condition testing
should be abandoned for complex conditionsFor complex conditions, condition determination testing should
be applied for test case design, because the complexity of the conditional expression is taken into account
The method also leads to statement and branch coverage, which means they need not be used additionally
Test Completion Criteria: Analogous to the previous techniques, the proportion between the executed and all the required logical values of the condition (parts) can be calculated
This can serve as criteria for termination of the tests For the techniques, which concentrate attention to the
complexity of the conditions in the source code, it is reasonable to try to achieve a complete verification (100% coverage)
If there are no complex condition expressions, branch coverage can be seen as sufficient
Path CoverageUntil now, test case determination focused on the
statements or branches of the control flow, as well as the complexity of conditions
If the test object includes loops or repetitions, the previous deliberations are not sufficient for an adequate test
Path coverage requires the execution of all different paths through the test object
If the test object includes loops or repetitions ,this method only requires the execution of all different paths through the test object. Both statement & branch coverage focuses on statement & branch of control flow & complexity of the conditions
A path describes the possible order of single program parts in a program fragment. Paths consider dependencies between branches but branch coverage are independent type.
Path Coverage
Path CoverageThe program fragment represented by the graph includes a
loopThis DO-WHILE loop is executed at least onceIn the WHILE-condition it is decided at the end of the loop
whether the loop must be repeated, i.e., if a jump to the start of the loop is necessary
When using branch coverage for test design the loop has been considered in two test cases:
Loop without repetition: a, b, f, g, h, d, eLoop with single return (i) and a single repetition: a, b, f, g, i, g, h, d, e
Path CoverageUsually a loop is repeated more than once. Further possible
sequences of branches through the graph of the program are
a, b, f, g, i, g, i, g, h, d, e a, b, f, g, i, g, i, g, i, g, h, d, e a, b, f, g, i, g, i, g, i, g, i, g, h, d, e etc.This shows that there is an indefinite number of paths
in the control flow graphEven with restrictions on the number of loop
repetitions, the number of paths increases indefinitely
Path Coveragedouble calculate_price (double baseprice, double specialprice,double extraprice, int extras, double discount){double addon_discount;double result;if (extras >= 3) addon_discount = 10;else if (extras >= 5) addon_discount = 15;else addon_discount = 0;if (discount > addon_discount)addon_discount = discount;result = baseprice /100.0*(100-discount) + specialprice +
extraprice/100.0*(100-addon_discount);return (result);}
Path Coverage In this example the test object consists of altogether three IF-
statements whereof two are nested and the third is placed separately from the others
All statements (nodes) are reached by the following sequence of edges in the graph:
a, b, c, j, k, l, n a, d, e, f, i, j, k, l, n a, d, g, h, i, j, k, l, nThese sequences are sufficient to achieve 100% statement
coverage.But not all branches (edges) have been covered yetThe edge m is still missing. A sequence might look as follows: a, b, c, j, m, nThis additional sequence should have replaced the first
sequence (test case) above and with the resulting three test cases, branch coverage of 100% is achieved
Path CoverageBut, even for this simple program fragment, there are
still possibilities to traverse the graph differently, and thus consider all paths of the graph
Until now, the following paths have not been executed: a, d, e, f, i, j, m, n a, d, g, h, i, j, m, nAltogether, six different paths through the source code
result (the three possible paths through the graph before edge j multiplied with the two possible paths after edge j)
There is the precondition that the conditions are independent from each other and the edges can be combined freely
Intuitive testing Basis of this method is the skill, experience, and knowledge of the testerThe tester selects test cases to uncover expected problems and their
symptoms A more systematic approach for this cannot be described The test cases are based on the experience of where faults have occurred in
the past or the tester's assumptions where faults might occur in the futureThis type of test case design is also called "error guessing" and is used very
often in practiceKnowledge in developing similar applications and using similar technologies
should also be used when designing test cases, in addition to experience in testing
If, for example, there exist experiences with a new programming language in previous projects, it is reasonable to apply the failures found, as well as their cause in using the programming language for designing the tests in the actual project
Exploratory testing If the documents, which form the basis for test design, are of
very low quality or do not exist at all , so-called "exploratory testing" may help
In the extreme case only the program exists The technique is also applicable when time is severely
restricted because it uses much less time than other techniques
The approach is mainly based on the intuition and experience of the tester
Here minimum planning and maximum test execution is doneThe planning involves the creation of a test charter, a short
declaration of the scope of a short (1 to 2 hour) time-bound test effort, the objectives and possible approaches to be used
Exploratory testing The test design and test execution activities are
performed in parallel typically without formally documenting the test conditions, test cases or test scripts
This does not mean that other, more formal testing techniques will not be used
For example, the tester may decide to use boundary value analysis but will think through and test the most important boundary values without necessarily writing them down
Some notes will be written during the exploratory-testing session, so that a report can be produced afterwards
Exploratory testing Test logging is undertaken as test execution is performed,
documenting the key aspects of what is tested, any defects found and any thoughts about possible further testing
A key aspect of exploratory testing is learning: learning by the tester about the software, its use, its strengths and its weaknesses
As its name implies, exploratory testing is about exploring, finding out about the software, what it does, what it doesn't do, what works and what doesn't work
The tester is constantly making decisions about what to test next and where to spend the (limited) time
Exploratory testing It makes sense to restrict exploratory testing to
certain elements of the program (certain tasks or functions)
The elements are further broken down: The term "test charter" is used for such smaller parts
The test of a "charter" should not take more than one or two hours of uninterrupted test time