lecture-6 testing slide 4 print

8/16/2019 Lecture-6 Testing Slide 4 Print

1/12

Testing the Programs

Testing is part of SWE Process – irrespective any model

Testing the programs

In this part we look at

• classification of faults

• the purpose of testing

• unit testing

• integration testing strategies

• when to stop testing

Concept change!!

• Many programmers view testing as ademonstration that their program performsproperly.

• The idea of demonstrating correctness isreally the reverse of what testing is all about.

• We test a program to demonstrate theexistence of a fault! Because our goal is todiscover faults, we consider a test successful

only when a fault is discovered or a failureoccurs as a result of our testing procedures.

Classification of faults

• In an ideal world, we produce programswhere everything works flawlessly every time.Unfortunately this is not the case!

• We say that our software has failed, usuallywhen its behaviour deviates from the onedescribed in the requirements.

• First we identify the fault, i.e. determine whatfault or faults caused the failure. Next we

correct the fault by making changes to thesystem so that the fault is removed.


2/12

Classification of faults

• Why do we classify faults?

– In order to improve our development

process!

• We would like to match a fault to a

specific area of our development

process.

– In other words, we would like our

classification scheme to be orthogonal.

IBM Orthogonal Defect

ClassificationFault type Meaning

Function Fault that affects capability, end-user interfaces,

product interfaces, interface with hardware

architecture, or global data structure

Interface Fault in interacting with other components or

drivers via calls, macros,control blocks or parameter lists

Checking Fault in program logic that fails to validate data

and values properly before

they are used

Assignment Fault in data structure or code block initialization.

IBM Orthogonal Defect

ClassificationFault type Meaning

Timing/serialization Fault that involves timing of shared and real-

time resources

Build/package/merge Fault that occurs because of problems in

repositories, management changes,

or version control

Documentation Fault that affects publications and

maintenance notes

AlgorithmFault involving efficiency or correctness of

algorithm or data structure butnot design

Hewlett-Packard fault

classification


3/12

Fault Classification for HP (1970)

Computation

18%

Hardware

4%

Requirements

5%

Documentation

19%

Data Handling

6%

Other Code

11%

Logic

32%

Process/

Interprocess

5%

Testing steps

• Black (opaque) box

– In this type of testing, the test object is

viewed from the outside and its contents

are unknown.

– Testing consists of feeding input to the

object and noting what output is produced.

– The test's goal is to be sure that every kind

of input is submitted and that the observed

output matches the expected output.

Views of Test Objects

• Suppose we have a component that acceptsas input the three numbers a, b, c andoutputs the two roots of the equation ax2 + bx+ c = 0 or the message “no real roots”.

• It is impossible to test the component bysubmitting every possible triple of numbers(a,b,c).

• Representative cases may be chosen so thatwe have all combinations of positive, negativeand zero for each of a, b, and c.

• Additionally we may select values that ensurethat the discriminant, (b2 – 4ac) is positive,zero, or negative.

Black Box Example


4/12

• If the tests reveal no faults, we have no

guarantee that the component is fault-

free!

• There are other reasons why failure

may occur.

• For some components, it is impossible

to generate a set of test cases to

demonstrate correct functionality for all

cases.

Black Box Example

• White (transparent) box

– In this type of testing, we use the structure

of the test object to test in different ways.

– For example, we can devise test cases that

execute all the statements or all the control

paths within the component(s).

– Sometimes with many branches and loops

it may be impractical to use this kind of

approach.

White Box Testing

Our goal is to find faults in components.

There are several ways to do this:

• Examining the code

– Code walkthroughs

– Code inspections

• Proving code correct

• Testing components

Unit Testing


5/12

• Code walkthroughs are an informal type

of review.

• Your code and documentation is

presented to a review team and the

team comments on their correctness.

• You lead and control the discussion.

The focus is on the code not the

programmer

Examining the Code

• A code inspection is similar to a walkthroughbut is more formal.

• Here the review team checks the code anddocumentation against a prepared list ofconcerns. – For example, the team may examine the

definition and use of data types and structures tosee if their use is consistent with the design andwith system standards and procedures.

• The team may review algorithms andcomputations for their correctness andefficiency

Examining the Code

• Proof techniques are not widely used. It is difficult

to create proofs, these can sometimes be longer

than the program itself!

• Additionally customers require demonstration that

the program is working correctly.

• Whereas a proof tells us how a program will work

in a hypothetical environment described by the

design and requirements, testing gives us

information on how the program works in its

actual operating environment.

Correcting the Code Testing components

• Choosing test cases: To test a component,we select input data and conditions andobserve the output.

• A test point or case is a particular choice oftest data.

• A test is a finite collection of test cases.

• Create tests that can convince ourselves andour customers that the program workscorrectly, not only for the test cases but for allinput.

– We start by defining test objectives and definetests designed to meet a specific objective.

– One objective can be that all statements shouldexecute correctly another can be that everyfunction performed by the code is done correctly.


6/12

• As seen before we view the component as

either a “white” or “black” box.

– If we use “black” box testing, we supply all

possible input and compare the output with what

was expected.

– For example, with the quadratic equation seen

earlier we can choose values for the coefficients

that range over combinations of positive, zero and

negative numbers.

– Or select combinations based on the relative sizes

e.g. a > b > c, b > c > a, c > b > a, ...etc

Testing Components

• We can go further and select valuesbased upon the discriminant.

• We even supply non-numeric input todetermine the program's response.

• In total we have four mutually exclusivetypes of test input.

• We thus use the test objective to helpus separate the input into equivalenceclasses.

Testing Components

• Every possible input belongs to one of the

classes. That is, the classes cover the entire

set of input data.

• No input datum belongs to more than one

class. That is, the classes are disjoint.

• If the executing code demonstrates a fault

when using a particular class member is used

as input, then the same fault can be detected

using any other member of the class as input.

That is , any element of the class represents all

elements of that class.

Equivalence Classes

• It is not always easy of feasible to tell if

the third restriction can be met and it is

usually rewritten to say:

– if a class member is used to detect a fault

then the probability is high that the other

elements in the class will reveal the same

fault.

Equivalence Classes


7/12

Common Practice

• Usually 'white' box and 'black' box testing arecombined.

• Suppose we have a component expects apositive input value. Then, using 'black' boxtesting, we can have a test case for each ofthe following: – a very large positive integer

– a positive integer

– a positive, fixed point decimal

– a number greater than 0 but less than 1

– a negative number

– a non numeric character

Common Practice

• Using 'white' box testing we can chose one or

more of the following:

– Statement testing: Every statement in the

component is executed at least once in some test.

– Branch testing: For every decision point in the the

code, each branch is chosen at least once in

some test.

– Path testing: Every distinct path through the code

is executed at least once in some test.

Check how many

paths are possible?

White box testing• Statement testing

– choose X > K that produces a +ve result

– 1-2-3-4-5-6-7

• Branch testing – choose two test cases to traverse each branch of

the decision points

– 1-2-3-4-5-6-7

– 1-2-4-5-6-1

• Path testing – four test cases needed

– 1-2-3-4-5-6-7 – 1-2-3-4-5-6-1

– 1-2-4-5-6-7

– 1-2-4-5-6-1


8/12

four test cases needed

1-2-3-4-5-6-7

1-2-3-4-5-6-1

1-2-4-5-6-7

1-2-4-5-6-1

a = 5;

for (b=1; b


9/12

Integration testing

• When each component has been completed

and tested, we can then combine them into a

working system.

• This integration must be planned and

coordinated so that in the case of a failure,

we would be able to determine what may

have caused it.

• Suppose we view the system as a hierarchy

of components (shown on the following slide).

Integration testing

Bottom-up integration Top-down integration


10/12

Big-bang integration Sandwich integration

Comparison of integration

strategies

When to Stop Testing?


11/12

Fault Seeding• We intentionally insert or “seed” a know

number of faults in a program.

• Then another member of the team locate as

many faults as possible.

• The number of undiscovered seeded faults

act as an indicator of the total number of

faults(unseeded and seeded) remaining in

the program.

• We say:

Fault Seeding

Problems:

• It is assumed that the seeded faults are of thesame kind and complexity as the actual faultsin the program. – This is difficult to do since we do not know what

are the typical faults until we have found them.

• We can attempt to overcome this by basingthe seeded faults on historical data aboutprevious faults. – This, however requires that we have built similar

systems before.

Fault Seeding

Solution

• Use two independent groups, Test Group 1and Test Group 2.

• Let x be the number detected by Group 1 andy the number detected by Group 2.

• Some faults will be detected by both groupssay q, such that q


12/12

Confidence in the Software

• If we seeded a program with S faultsand we claim that the code has only Nactual faults.

• Suppose we tested until all S faultshave been found as well as n non-seeded faults, then a confidence levelcan be calculated as

1 , if n > N

C =

S/(S – N + 1) , if n N

Confidence in the Software

• With that approach we cannot predict the

level of confidence until all the seeded faults

are detected.

• Richards (1974) suggests a modification,

where the confidence level can be estimated

whether or not all the seeded faults have

located.

1 , if n > N

• C= S S + N + 1 ,if n

lecture-6 testing slide 4 print

Documents