software testing day 2: functional testing

126
Software Testing Day 2: Functional Testing Aditya P. Mathur Purdue University August 12-16 @ Guidant Corporation Minneapolis/St Paul, MN Graduate Assistants: Baskar Sridharan Ramkumar Natarajan Last update: July 23, 2002

Upload: fleta

Post on 11-Jan-2016

48 views

Category:

Documents


3 download

DESCRIPTION

Graduate Assistants :. Baskar Sridharan Ramkumar Natarajan. Software Testing Day 2: Functional Testing. Aditya P. Mathur Purdue University August 12-16 @Guidant Corporation Minneapolis/St Paul, MN. Last update: July 23, 2002. Course Organization. Part I: MondayPreliminaries - PowerPoint PPT Presentation

TRANSCRIPT

Page 1: Software Testing Day 2: Functional Testing

Software TestingDay 2: Functional Testing

Aditya P. MathurPurdue UniversityAugust 12-16

@ Guidant CorporationMinneapolis/St Paul, MN

Graduate Assistants: Baskar SridharanRamkumar Natarajan

Last update: July 23, 2002

Page 2: Software Testing Day 2: Functional Testing

Functional testing 2

Course OrganizationPart I: Monday Preliminaries

Part II: TuesdayFunctional Testing

Part III:Wednesday Test Assessment and improvement

Part IV: Thursday Robustness, performance, and GUI testing

Part V: Friday Reliability and test process

Page 3: Software Testing Day 2: Functional Testing

Functional testing 3

Part II: Functional testing Learning objectives-

What is functional testing? How to perform functional testing?

What are clues, test requirements, and test specifications?

How to generate test inputs? What are equivalence partitioning, boundary value

testing, domain testing, state testing, and decision table testing?

Page 4: Software Testing Day 2: Functional Testing

Functional testing 4

What is functional testing? When test inputs are generated using

program specifications, we say that we are doing functional testing.

Functional testing tests how well a program meets the functionality requirements.

Page 5: Software Testing Day 2: Functional Testing

Functional testing 5

The methodology The derivation of test inputs is based on

program specifications. Clues are obtained from the specifications. Clues lead to test requirements. Test requirements lead to test specifications. Test specifications are then used to actually

execute the program under test.

Page 6: Software Testing Day 2: Functional Testing

Functional testing 6

Test methodologySpecifications

Clues

Test requirements

Test specifications

Test driver

Program

Oracle

Expected behavior

Actual behavior

Program output is correct

Program hasfailed; make a note and proceedwith testing orget into the debugmode.

or

Until specs.

Exhausted.

Page 7: Software Testing Day 2: Functional Testing

Functional testing 7

Specifications Inputs and tasks:

Given inputs

Perform tasks

0,,....,, 21 ≥nIII n

0,,....,, 21 ≥mTTT m

Page 8: Software Testing Day 2: Functional Testing

Functional testing 8

Specifications-continued Input properties

Input

must satisfy

Function f is a pre-condition on input

kI

),..,,..,( 21 nk IIIIf

kI

Page 9: Software Testing Day 2: Functional Testing

Functional testing 9

Specifications-continued Two types of pre-conditions are considered:

Validated: those that are required to be validated by the program under test and an error action is required to be performed if the condition is not true.

Assumed: those that are assumed to be true and not checked by the program under test.

Page 10: Software Testing Day 2: Functional Testing

Functional testing 10

Specification: example For the sort program:

Inputs are: N pointer to a sequence of length N pointer to an area in memory where the output

sequence is to be placed.

Page 11: Software Testing Day 2: Functional Testing

Functional testing 11

Specification: example..continued

Tasks to be performed: Sort the sequence in ascending order Return the sorted sequence in an area

provided. Return 1 if sorting is successful, -1 otherwise.

Page 12: Software Testing Day 2: Functional Testing

Functional testing 12

Preconditions for sort Validated:

N>0 On failure return -1; sorting considered

unsuccessful. Assumed:

The input sequence contains N integers. The output area has space for at least N

integers.

Page 13: Software Testing Day 2: Functional Testing

Functional testing 13

Deriving pre-conditions Pre-conditions result from properties of

inputs. Example:

alpha_sequence(name)

alpha_sequence is the string obtained from name by removing all characters other then A-Z, and a-z. Thus, if name is “A12C” then alpha_name is “AC”.

Page 14: Software Testing Day 2: Functional Testing

Functional testing 14

Deriving pre-conditions-continued This leads to the following pre-condition:

Validated: the string alpha_sequence(name) is shorter than name.

On failure: print “invalid name”. This property could also lead to the pre-

condition: Assumed: the string alpha_

sequence(name) is shorter than name.

Page 15: Software Testing Day 2: Functional Testing

Functional testing 15

Post-conditions A post-condition specifies a property of the

output of a program. The general format of a post-condition is:

if condition then effect-1 {else effect-2} Example:

For the sort program a post-condition is: if N>0 then {the output sequence has the same

elements as in the input sequence and in ascending order.}

Page 16: Software Testing Day 2: Functional Testing

Functional testing 16

Post-condition-continued This could be stated more formally as:

if N>0 then{

and each is a member of the input sequence and sort returns 1.

} else{ the output sequence is undefined and

sort returns -1.}

NAAA ≤≤≤ ....21

Ni1Ai ≤≤,

Page 17: Software Testing Day 2: Functional Testing

Functional testing 17

Post-condition-continued Another example:

if (A=B) and (B=C) then return “equilateral”; Can you complete the above post-condition

for a program that is required to classify a triangle given the length of three sides?

Convention: We will not nest if-then-else statements while specifying a post-condition.

Page 18: Software Testing Day 2: Functional Testing

Functional testing 18

Incompleteness of specifications Specifications may be incomplete or

ambiguous. Example post-condition:

if user places cursor on the name field then read a string

This post-condition does not specify any limit on the length of the input string hence is incomplete.

Page 19: Software Testing Day 2: Functional Testing

Functional testing 19

Ambiguous specifications It also does not make it clear as to

whether a string should be input only after the user has placed the cursor on the name field and clicked the mouse or simply placed the cursor on the name field.

and hence is ambiguous.

Page 20: Software Testing Day 2: Functional Testing

Functional testing 20

Clues: summary Clues are:

Pre-conditions Post-conditions Variables, e.g. A is a length implying

thereby that its value cannot be negative. Operations, e.g. “search a list of names” or

“find the average of total scores” Definitions, e.g. “filename(name) is a name

is no spaces.”

Page 21: Software Testing Day 2: Functional Testing

Functional testing 21

Clues-continued Ideally variables, operations and definitions

should be a part of at least one pre- or post-condition.

However, this may not be the case as specifications are not always written formally.

Hence look out for variables, operations, and definitions within a specification!

Page 22: Software Testing Day 2: Functional Testing

Functional testing 22

Test requirements A test requirement is a description of how to

test the program that is under test. Here is a sample test requirement for a

program that classifies a triangle given the length of three sides. A, B, C are non-zero and positive. One of A, B, C is negative; error condition. One of A, B, C is zero; error condition.

Page 23: Software Testing Day 2: Functional Testing

Functional testing 23

Test requirements-derivation Test requirements are derived from clues. For example, consider the following pre-

conditions (clues): Assumed: A, B, and C are lengths Validated: A>0, B>0, C>0

These pre-conditions on A, B, and C lead to the test requirement given above.

Page 24: Software Testing Day 2: Functional Testing

Functional testing 24

Test requirements-derivation Note that we have clumped pre-

condition for each input variable into one condition. This is being done only for inconvenience.

It is recommended that pre-conditions be separated for each variable.

Page 25: Software Testing Day 2: Functional Testing

Functional testing 25

Test requirements-derivation Note also that each validated pre-

condition results in at least two requirements: one for the validated part and the other for the failure part.

In our example above we did not list all requirements. For example, we are content with testing “one of A, B, C is negative; error condition.”

Page 26: Software Testing Day 2: Functional Testing

Functional testing 26

Test requirements-derivation Post-conditions also lead to test

requirements. For example, the partial post-condition:

if (A=B) and (B=C) then return “equilateral”

leads to the following test requirement: A=B and B=C.

Page 27: Software Testing Day 2: Functional Testing

Functional testing 27

Compound validated pre-conditions

Compound pre-conditions are ones that use the and or or connectors.

Examples: validated compound pre-conditions: Pre-condition: A and B Pre-condition: user places the mouse over

the name field and clicks it.

Page 28: Software Testing Day 2: Functional Testing

Functional testing 28

Compound validated pre-conditions

The first of the above pre-conditions leads to four requirements:

A true, B true (This is the validated part) A false, B true (This and the rest are failures) A true, B false A false, B false

You may work out the requirements for compound pre-condition with the or connector.

Page 29: Software Testing Day 2: Functional Testing

Functional testing 29

Compound validated pre-conditions

Compound validated pre-conditions could become quite complex.

Example: (A and (B or C)) Brute force method will lead to 8 test

requirements.

Page 30: Software Testing Day 2: Functional Testing

Functional testing 30

Compound validated pre-conditions

In general this will lead to too many test requirements.

We can prune them by leaving out those requirements that are unlikely to reveal a program error.

For example, consider the validated pre-condition: A or B.

Page 31: Software Testing Day 2: Functional Testing

Functional testing 31

Pruning test requirements There are four possible test requirements:

A true, B true A false, B true A true, B false A false, B false

Consider a correct C implementation:if (!(A || B))

exit_with_error(“Error: A is %d, B is %d”, A, B);

else.. {/* the validated code comes here.*/}

Page 32: Software Testing Day 2: Functional Testing

Functional testing 32

Possible errors Programmer forgets to check for one of the

two cases resulting in the code: if (!A)

exit_with_error(“Error: A is %d, B is %d”, A, B);

or if (!B)

exit_with_error(“Error: A is %d, B is %d”, A, B);

Page 33: Software Testing Day 2: Functional Testing

Functional testing 33

Possible errors-continued Or use a wrong logical operator as in:

if (!(A && B))

exit_with_error(“Error: A is %d, B is %d”, A, B);

Let us analyze how the four different tests will perform in each of the four implementations: one correct, and three incorrect ones.

Page 34: Software Testing Day 2: Functional Testing

Functional testing 34

Truth table: or condition

A B !(A || B) !(A&&B) !A !B

T F F T F T

F T F T T F

F F T T T T

T T F F F F

Inputs Correctimplementation

Incorrectimplementations

Notice this one: will it help find any of the three possible errors?

Page 35: Software Testing Day 2: Functional Testing

Functional testing 35

Truth table analysis

Case 1: A test input with A=true and B=false will

cause the correct program to evaluate the condition to false.

The two incorrect implementations, !(A&&B) and (!B) will evaluate the condition to true.

Page 36: Software Testing Day 2: Functional Testing

Functional testing 36

Truth table analysis-continued Both incorrect implementations will print

the error message. The oracle will observe that the correct and

the incorrect implementations behave differently.

It will therefore announce failure for each incorrect implementation thereby pointing to an error.

End of Case 1.

Page 37: Software Testing Day 2: Functional Testing

Functional testing 37

Truth table analysis-continued

Case 2: Test input A=false and B=true will reveal

the error in the two incorrect implementations, !(A&&B) and (!A).

Case 3: Test input A=false and B=false might find a

fault in the then branch of the if condition.

Page 38: Software Testing Day 2: Functional Testing

Functional testing 38

Truth table analysis-continued

Case 4: Test input A=true and B=true might find a

fault in the else branch of the if condition. Thus, all four test inputs are likely to be

useful.

Page 39: Software Testing Day 2: Functional Testing

Functional testing 39

Truth table analysis-continued However, if we were to check for the correct

implementation of the condition A or B, then only the first two inputs are necessary.

In this example, reducing the number of test specifications from 4 to 2 does not lead to any significant savings. When will the savings be significant?

Page 40: Software Testing Day 2: Functional Testing

Functional testing 40

Assumed pre-conditions Each assumed pre-condition is likely to

result in a test requirement. Example:

Assumed: MODE is “on ground” or “flying” This leads to two requirements:

MODE is “on ground” , MODE is not “flying” MODE is not “on ground” , MODE is “flying”

Page 41: Software Testing Day 2: Functional Testing

Functional testing 41

Assumed pre-conditions These can be simplified to:

MODE is “on ground” MODE is “flying”

Page 42: Software Testing Day 2: Functional Testing

Functional testing 42

Clues from code? Yes, clues can also be derived by

scanning the code. However, such clues might be

redundant and incomplete if coverage measurement and use is planned.

In the absence of coverage measurement, it is a good idea to scan the code and find clues.

Page 43: Software Testing Day 2: Functional Testing

Functional testing 43

Clues from code-continued Examine internal variables. These may

lead to new test requirements. Example: Suppose that variable length

is input and denotes the length of an array. In the code we find:

int last_index=length+1;

This leads to the test requirements:

Page 44: Software Testing Day 2: Functional Testing

Functional testing 44

Clues from code-continued an array of length zero array of length 1 array with more than one element

Later we will see how these clues and requirements might be derived, with certainty, using boundary-value analysis.

Another example: Consider the sort program for which we have seen the specifications.

Page 45: Software Testing Day 2: Functional Testing

Functional testing 45

Clues from code-continued The specifications do not indicate what

algorithm is to be used for sorting the input array.

A programmer might decide to use different algorithms for different array sizes. When scanning the code we may see:

if (scan<min_length) simple_sort();else quicksort();

Page 46: Software Testing Day 2: Functional Testing

Functional testing 46

Clues from code-continued Variable size and the check against

min_length give us a clue for new test requirements. These are: size is equal to or greater than min_length

Later we will see how this clue and requirements might be derived, with certainty, using branch coverage.

Page 47: Software Testing Day 2: Functional Testing

Functional testing 47

Test requirements checklist Obtaining clues and deriving test

requirements can become a tedious task. To keep it from overwhelming us it is a good

idea to make a checklist of clues. This checklist is then transformed into a

checklist of test requirements by going through each clue and deriving test requirements from it.

Page 48: Software Testing Day 2: Functional Testing

Functional testing 48

Test specifications A test requirements indicates “how” to

test a program. But it does not provide exact values of inputs.

A test requirement is used to derive test specification, which is the exact specification of values of input and environment variables.

Page 49: Software Testing Day 2: Functional Testing

Functional testing 49

Test specifications-continued There may not be a one-to-one

correspondence between test requirements and test specifications.

A test requirement checklist might contain 50 entries. These might result in only 22 test specifications.

The fewer the tests the better but only if these tests are of good quality!

Page 50: Software Testing Day 2: Functional Testing

Functional testing 50

Test specifications-continued We will discuss test quality when

discussing test assessment. A test specification looks like this:

Test 2: global variable all_files is initially false. next_record is set to 1.

Upon return expect: all_files to be true next_record is last_record+1

Page 51: Software Testing Day 2: Functional Testing

Functional testing 51

Test specifications-continued Notice the format of a test specification:

Each test is given a number which serves as its identifier.

There is a set of input values. There is a set of expected values upon return from

execution. Any side effects on files or networks must also be specified here. In essence, all observable effects must be specified in the “Expect” part of a test specification.

Page 52: Software Testing Day 2: Functional Testing

Functional testing 52

Test specifications-continued Any side effects on files or networks must

also be specified. In essence, all observable effects must be specified in the “Expect” part of a test specification.

Similarly, values of all input variables, global or otherwise, must also be specified.

Page 53: Software Testing Day 2: Functional Testing

Functional testing 53

Test requirements to specifications The test requirements checklist guides

the process of deriving test specifications.

Initially all entries in the checklist are unmarked or set to 0.

Each time a test is generated from a requirement it is marked or the count incremented by 1.

Page 54: Software Testing Day 2: Functional Testing

Functional testing 54

Test requirements to specifications

Thus, at any point in time, one could assess the progress made towards the generation of test specifications.

One could also determine how many tests have been generated using any test requirement.

Page 55: Software Testing Day 2: Functional Testing

Functional testing 55

Test requirements to specifications

Once a test requirement has been marked or its count is more than 0 we say that it has been satisfied.

Some rules of thumb to use while designing tests: Try to satisfy multiple requirements using

only one test. Satisfy all test requirements.

Page 56: Software Testing Day 2: Functional Testing

Functional testing 56

Test requirements to specifications

Avoid reuse of same values of a variable in different tests. Generating new tests by varying an existing one is likely to lead to tests that test the same part of the code as the previous one.

In testing, variety helps! Though we try to combine several test

requirements to generate one test case, this is not advisable when considering error conditions.

Page 57: Software Testing Day 2: Functional Testing

Functional testing 57

Test requirements to specifications For example, consider the following:

speed_dial, an interval speed_dial<0 ,error speed_dial>120, error

zones, an interval zones <5, error zones>10, error

Page 58: Software Testing Day 2: Functional Testing

Functional testing 58

Test requirements to specifications

One test specification obtained by combining the two requirements above is:

speed_dial=-1 zone=3

Now, if the code to handle these error conditions is:

Page 59: Software Testing Day 2: Functional Testing

Functional testing 59

Test requirements to specifications

if (speed_dial<0 || speed_dial>120)error_exit(“Incorrect speed_dial”);

if (zone<6 ||zone>10)error_exit(“Incorrect zone”);

For our test, the program will exit before it reaches the second if statement. Thus, it will miss detecting the error in coding the test for zone.

error

Page 60: Software Testing Day 2: Functional Testing

Functional testing 60

Test requirements to specifications Also, do not assume an error test to satisfy

any other test requirement. Example:

Consider the function: myfunction(int X, int Y);

A test for the erroneous value of X might not test the code that uses Y.

Page 61: Software Testing Day 2: Functional Testing

Functional testing 61

Test requirements to specifications Test specifications must not mention internal

variables. Remember, a test specification aids in setting input variables to suitable values before the test begins. Values of internal variables are computed during program execution.

However, there are exceptions to the above rule. Can you think of one?

Page 62: Software Testing Day 2: Functional Testing

Functional testing 62

A peek into today’s laboratory The laboratory is based on the example

given in your text by Marick. The example is based on a C program

sreadhex. Given the specifications and code of

sreadhex you will be required to find clues, develop test requirements, and test specifications.

Page 63: Software Testing Day 2: Functional Testing

Functional testing 63

A peek into today’s laboratory You will then test sreadhex using tests

derived by you. To test sreadhex you will write a test

driver. More details will be provided during the

laboratory session. Let us examine sreadhex!

Page 64: Software Testing Day 2: Functional Testing

Functional testing 64

The sreadhex program Inputs:

A string of characters (s). The characters in the string represent

hexadecimal digits. s is null terminated. Maximum number of bytes (rlen) in the

output array (str). A pointer to an integer (odd-digit).

Page 65: Software Testing Day 2: Functional Testing

Functional testing 65

The sreadhex program Outputs:

A packed array of bytes (str) containing a maximum of rlen bytes.

The hexadecimal characters in the input string are converted to their 4-bit binary equivalent. Two successive hexadecimal digits are packed into one byte of str.

if the number of digits in str is odd, then the 4-bit value of the last hexadecimal digit in s is placed in odd-digit else odd-digit is set to -1.

Page 66: Software Testing Day 2: Functional Testing

Functional testing 66

The sreadhex program The number of bytes filled in str is returned

in nread. Any non-hexadecimal characters in s are

ignored. sreadhex returns 1 if all hexadecimal

characters filled in the output array, 0 if not; -1 if there was an error.

Page 67: Software Testing Day 2: Functional Testing

Functional testing 67

The sreadhex program Examples:

Input: s=“4EDIT” odd_digit=-1 rlen=2

Output: str byte 1: 01001110 odd_digit: 1101 nread: 1

“T” and “I” ignored

Odd digit “D”

Page 68: Software Testing Day 2: Functional Testing

Functional testing 68

The sreadhex program Input:

s=“3A” odd_digit=5 rlen=2

Output: str byte 1: 01010011 odd_digit: 1010 nread: 1

Odd digit from previous call

Input odd_digit placedin the first byte.

Page 69: Software Testing Day 2: Functional Testing

Functional testing 69

The sreadhex program Input:

s=“1F” odd_digit=-1 rlen=2

Output: str byte 1: 00011111 odd_digit: -1 nread: 1

Page 70: Software Testing Day 2: Functional Testing

Functional testing 70

The sreadhex program Input:

s=“1F2A3” odd_digit=-1 rlen=2

Output: str byte 1: 00011111 str byte 2: 00101010 odd_digit: -1 nread: 2

“3” ignored as it is the fifth digit whereas only 4 are allowed in str.

Page 71: Software Testing Day 2: Functional Testing

Functional testing 71

Sample pre- and post- conditions

Pre-conditions: Assumed: str is a non-null pointer to an

array that can hold rlen bytes. Validated: rlen is not 0; on failure *nread is

0 and return value is 0. Post-conditions:

An even number of digits and not enough to fill str; use all of them, return value is 1.

Page 72: Software Testing Day 2: Functional Testing

Functional testing 72

Sample pre- and post- conditions

An odd number of digits and not enough to fill str; use all of them, set odd-digit to the value of the last hexadecimal digit in str, return 1.

More than enough digits to fill str, ignore excess, return 0.

Page 73: Software Testing Day 2: Functional Testing

Functional testing 73

Sample definitions START:

Location in str where the value of the first hexadecimal character in s is placed.

if (*odd-digit = = -1)

then

START is 0

else

START is 1.

Page 74: Software Testing Day 2: Functional Testing

Functional testing 74

Using definitions Definitions help formulate and specify

pre- and post-conditions. Example:

if START+NUM_HEX_CHARS>=2*rlen

then ……

where NUM_HEX_CHARS is the number of hex digits in s.

Page 75: Software Testing Day 2: Functional Testing

Functional testing 75

Understanding specs and code Look at the specifications of sreadhex:

see section 2.2 of the text. Look at the code on page 28. Understand the specifications and the

code before you begin the laboratory assignments.

Page 76: Software Testing Day 2: Functional Testing

Functional testing 76

Equivalence partitioning The input domain is usually too large for

exhaustive testing. It is therefore partitioned into a finite

number of sub-domains for the selection of test inputs.

Each sub-domain is known as an equivalence class and serves as a source of at least one test input.

Page 77: Software Testing Day 2: Functional Testing

Functional testing 77

Equivalence partitioning

12

3

4

Input domain Input domain partitioned into four sub-domains.

Too manytest inputs.

Four test inputs, one selected from each sub-domain.

Page 78: Software Testing Day 2: Functional Testing

Functional testing 78

How to partition? Inputs to a program provide clues to

partitioning. Example 1:

Suppose that program P takes an input X, X being an integer.

For X<0 the program is required to perform task T1 and for X>=0 task T2.

Page 79: Software Testing Day 2: Functional Testing

Functional testing 79

How to partition?-continued The input domain is prohibitively large

because X can assume a large number of values.

However, we expect P to behave the same way for all X<0.

Similarly, we expect P to perform the same way for all values of X>=0.

We therefore partition the input domain of P into two sub-domains.

Page 80: Software Testing Day 2: Functional Testing

Functional testing 80

Two sub-domains

X<0 X>=0

One test case:X=-3

Another test case:X=-15

All test inputs in the X<0 sub-domain are considered equivalent.The assumption is that if one test input in this sub-domain revealsan error in the program, so will the others.

This is true of the test inputs in the X>=0 sub-domain also.

Equivalence class

Equivalence class

Page 81: Software Testing Day 2: Functional Testing

Functional testing 81

Non-overlapping partitions In the previous example, the two

equivalence classes are non-overlapping. In other words the two sub-domains are disjoint.

When the sub-domains are disjoint, it is sufficient to pick one test input from each equivalence class to test the program.

Page 82: Software Testing Day 2: Functional Testing

Functional testing 82

Non-overlapping partitions An equivalence class is considered

covered when at least one test has been selected from it.

In partition testing our goal is to cover all equivalence classes.

Page 83: Software Testing Day 2: Functional Testing

Functional testing 83

Overlapping partitions Example 2:

Suppose that program P takes three integers X, Y and Z. It is known that:

X<Y Z>Y

Page 84: Software Testing Day 2: Functional Testing

Functional testing 84

Overlapping partitions

X<Y

X>=Y

Z>Y Z<=Y

X<Y, Z>YX=3, Y=4, Z=7

X<Y, Z<=YX=2, Y=3, Z=1

X>=Y, Z<=YX=15, Y=4, Z=1

X>=Y, Z>YX=15, Y=4, Z=7

Page 85: Software Testing Day 2: Functional Testing

Functional testing 85

Overlapping partition-test selection

In this example, we could select 4 test cases as: X=4, Y=7, Z=1 satisfies X<Y X=4, Y=2, Z=1 satisfies X>=Y X=1, Y=7, Z=9 satisfies Z>Y X=1, Y=7, Z=2 satisfies Z<=Y

Thus, we have one test case from each equivalence class.

Page 86: Software Testing Day 2: Functional Testing

Functional testing 86

Overlapping partition-test selection

However, we may also select only 2 test inputs and satisfy all four equivalence classes: X=4, Y=7, Z=1 satisfies X<Y and Z<=Y X=4, Y=2, Z=3 satisfies X>=Y and Z>Y

Thus, we have reduced the number of test cases from 4 to 2 while covering each equivalence class.

Page 87: Software Testing Day 2: Functional Testing

Functional testing 87

Partitioning using non-numeric data In the previous two examples the inputs were

integers. One can derive equivalence classes for other types of data also.

Example 3: Suppose that program P takes one character X

and one string Y as inputs. P performs task T1 for all lower case characters and T2 for upper case characters. Also, it performs task T3 for the null string and T4 for all other strings.

Page 88: Software Testing Day 2: Functional Testing

Functional testing 88

Partitioning using non-numeric data

X: LC

X:UC

Y: null Y: not nullX: LC, Y: null

X: LC, Y: not null

X: UC, Y: not null

X: UC, Y: null LC: Lower case characterUC: Upper case characternull: null string.

Page 89: Software Testing Day 2: Functional Testing

Functional testing 89

Non-numeric data Once again we have overlapping

partitions. We can select only 2 test inputs to

cover all four equivalence classes. These are: X: lower case, Y: null string X: upper case, Y: not a null string

Page 90: Software Testing Day 2: Functional Testing

Functional testing 90

Guidelines for equivalence partitioning

Input condition specifies a range: create one for the valid case and two for the invalid cases. e.g. for a<=X<=b the classes are

a<=X<=b (valid case) X<a and X>b (the invalid cases)

Page 91: Software Testing Day 2: Functional Testing

Functional testing 91

Guidelines-continued

Input condition specifies a value: create one for the valid value and two for incorrect values (below and above the valid value). This may not be possible for certain data types, e.g. for boolean.

Input condition specifies a member of a set: create one for the valid value and one for the invalid (not in the set) value.

Page 92: Software Testing Day 2: Functional Testing

Functional testing 92

Sufficiency of partitions In the previous examples we derived

equivalence classes based on the conditions satisfied by input data.

Then we selected just enough tests to cover each partition.

Think of the advantages and disadvantages of this approach!

Page 93: Software Testing Day 2: Functional Testing

Functional testing 93

Boundary value analysis (BVA) Another way to generate test cases is to look

for boundary values. Suppose a program takes an integer X as

input. In the absence of any information, we

assume that X=0 is a boundary. Inputs to the program might lie on the boundary or on either side of the boundary.

Page 94: Software Testing Day 2: Functional Testing

Functional testing 94

BVA: continued This gives us 3 test inputs:

X=0, X=-20, and X=14. Note that the values -20 and 14 are on either side

of the boundary and are chosen arbitrarily.

Notice that using BVA we get 3 equivalence classes. One of these three classes contains only one value (X=0), the other two are large!

Page 95: Software Testing Day 2: Functional Testing

Functional testing 95

BVA: continued Now suppose that a program takes two

integers X and Y and that x1<=X<=x2 and y1<=Y<=y2.

x1 x2

y2

y1

1 2

34

5

6

7

8 9

1011

1213

14

Page 96: Software Testing Day 2: Functional Testing

Functional testing 96

BVA-continued In this case the four sides of the

rectangle represent the boundary. The heuristic for test selection in this

case is: Select one test at each corner (1, 2, 3, 4). Select one test just outside of each of the

four sides of the boundary (5, 6, 7, 8)

Page 97: Software Testing Day 2: Functional Testing

Functional testing 97

BVA-continued Select one test just inside of each of the

four sides of the boundary (10, 11, 12, 13). Select one test case inside of the bounded

region (9). Select one test case outside of the

bounded region (14). How many equivalence classes do we

get?

Page 98: Software Testing Day 2: Functional Testing

Functional testing 98

BVA -continued In the previous examples we considered only

numeric data. BVA can be done on any type of data. For example, suppose that a program takes a

string S and an integer X as inputs. The constraints on inputs are: length(S)<=100 and a<=X<=b

Can you derive the test cases using BVA?

Page 99: Software Testing Day 2: Functional Testing

Functional testing 99

BVA applied to output variables Just as we applied BVA to input data,

we can apply it to output data. Doing so gives us equivalence classes

for the output domain. We then try to find test inputs that will

cover each output equivalence class.

Page 100: Software Testing Day 2: Functional Testing

Functional testing 100

BVA-continued Example: each student to construct one!

Page 101: Software Testing Day 2: Functional Testing

Functional testing 101

Finite State Machines (FSMs) A state machine is an abstract

representation of actions taken by a program or anything else that functions!

It is specified as a quintuple: A: a finite input alphabet Q: a finite set of states q0: initial state which is a member of Q.

Page 102: Software Testing Day 2: Functional Testing

Functional testing 102

FSMs-continued T: state transitions which is a mapping

Q x A--> Q F: A finite set of final states, F is a subset of Q.

Example: Here is a finite state machine that recognizes integers ending with a carriage return character.

A={0,1,2,3,4,5,6,7,8,9, CR} Q={q0,q1,q2} q0: initial state

Page 103: Software Testing Day 2: Functional Testing

Functional testing 103

FSMs-continued T: {((q0,d),q1),(q1,d),q1), (q1,CR),q2)} F: {q2}

A state diagram is an easier to understand specification of a state machine. For the above machine, the state diagram appears on the next page.

Page 104: Software Testing Day 2: Functional Testing

Functional testing 104

State diagram

q0 q1d

d

CRq2

Final state indicatedby concentric circles.

States indicated by circles.

State transitions indicatedby labeled arrows from one statethe another (which could be the same). Each label must be from the alphabet. It is also known asan event.

d: denotes a digit

Page 105: Software Testing Day 2: Functional Testing

Functional testing 105

State diagram-actions

q0 q1 q2d/set i to d

d/add 10*d to i

CR/output i

i is initialized to d when the machine moves from state q0 to q1.i is incremented by 10*d when the machine moves from q1 to q1.The current value of i is output when a CR is encountered.

Can you describe what this machine computes?Can you construct a regular expression that describes all strings recognized by this state machine?

x/y: x is an element ofthe alphabet and y is an action.

Page 106: Software Testing Day 2: Functional Testing

Functional testing 106

State machine: languages Each state machine recognizes a language. The language recognized by a state machine

is the set S of all strings such that: when any string s in S is input to the state

machine the machine goes through a sequence of transitions and ends up in the final state after having scanned all elements of s.

Page 107: Software Testing Day 2: Functional Testing

Functional testing 107

State diagram-errors

q0 q1 q2d/set I to d

d/add 10*d to I

CR/output I

q4 has been added to the set of states. It represents an error state. Notice that reset is a new member added to the alphabet.

CR/output error q4reset

Page 108: Software Testing Day 2: Functional Testing

Functional testing 108

State diagram-program A state diagram can be transformed into

a program using case analysis. Here is a C program fragment that embodies logic represented by the previous state diagram.

There is one function for each action. digit is assumed to be provided by the

lexical analyzer.

Page 109: Software Testing Day 2: Functional Testing

Functional testing 109

Program for “integer” state machine

case q0:i=digit; /* perform action. */state=q1; /* set next state. */break; /* event digit is done. */

case q1:i=i+10*digit; /* Add the next digit. */state=q1;break;

/*…complete the program. */

/* state is global, with values q0, q1, q2. i is also global.*/

switch (state)

void event_digit(){

Page 110: Software Testing Day 2: Functional Testing

Functional testing 110

More examples Let us go over figures 19.4 and 19.5 on

pages 308-309 of the text.

Page 111: Software Testing Day 2: Functional Testing

Functional testing 111

Checking state diagrams Unreachable state: One that cannot be

reached from q0 using any sequence of transitions.

Dead state: One that cannot be left once it is reached.

Page 112: Software Testing Day 2: Functional Testing

Functional testing 112

Test requirements Every state must be reached at least once,

Obtain 100% state coverage. Every transition must be exercised at least

once.Obtain 100% transition coverage. The textbook talks about duplicate transitions.

No transitions are duplicate if the state machine definition we have given is used.

Page 113: Software Testing Day 2: Functional Testing

Functional testing 113

Example test requirements For the “integer” state machine:

state machine transitions: event digit in state q0 event CR in state q0 event digit in state q1 event CR in state q1 event reset in state q4

Page 114: Software Testing Day 2: Functional Testing

Functional testing 114

More testing of state machines?

Yes, it is possible! When we learn about path coverage we

will discuss how more test requirements can be derived from a state diagram.

Page 115: Software Testing Day 2: Functional Testing

Functional testing 115

Test specifications As before, test specifications are derived from

test requirements. In the absence of dead states, all states and

transitions can be reached by one test. It is advisable not to test the entire machine

with one test case. Develop test specifications for our “integer”

state machine.

Page 116: Software Testing Day 2: Functional Testing

Functional testing 116

Decision tables Requirements of certain programs are

specified by decision tables. Such tables can be used for deriving

test requirements and specifications. A decision table is useful when

specifying complex decision logic

Page 117: Software Testing Day 2: Functional Testing

Functional testing 117

Decision tables A decision table has two parts:

condition part action part

The two together specify under what condition will an action be performed.

Page 118: Software Testing Day 2: Functional Testing

Functional testing 118

Decision table-nomenclature

C: denotes a condition A: denotes an action Y: denotes true N:denotes false X: denotes action to be taken. Blank in condition: denotes “don’t care” Blank in action: denotes “do not take the action”

Page 119: Software Testing Day 2: Functional Testing

Functional testing 119

Bank example Consider a bank software responsible for

debiting from an account. The relevant conditions and actions are:

C1: The account number is correct C2: The signature matches C3: There is enough money in the account A1: Give money A2: Give statement indicating insufficient funds A3: Call vigilance to check for fraud!

Page 120: Software Testing Day 2: Functional Testing

Functional testing 120

Decision tables

12345C1NNYYYC2NNYYC3NYNA1XA2XXA3X

Page 121: Software Testing Day 2: Functional Testing

Functional testing 121

Example-continued A1 is to be performed when C1, C2,

and C3 are true. A2 is to be performed when C1 is true

and C2 and C3 are false or when C1 and C2 are true and C3 is false.

A3 is to be performed when C2 and C3 are false.

Page 122: Software Testing Day 2: Functional Testing

Functional testing 122

Default rules Are all possible combinations of

conditions covered? No! Which ones are not covered? We need a default action for the

uncovered combinations. A default action could be an error report or a reset.

Page 123: Software Testing Day 2: Functional Testing

Functional testing 123

Example-test requirements Each column is a rule and corresponds

to at least one test requirement. If there are n columns then there are at

least n test requirements. What is the maximum number of test

requirements?

Page 124: Software Testing Day 2: Functional Testing

Functional testing 124

Example-test specifications For each test requirement find a set of

input values of variables such that the selected rule is satisfied.

When this test is input to the program the output must correspond to the action specified in the decision table.

Should the testing depend on the order in which the conditions are evaluated?

Page 125: Software Testing Day 2: Functional Testing

Functional testing 125

Summary Specifications, pre-conditions, and post-

conditions. Clues, test requirements, and test

specifications. Clues from code. Test requirements catalog. Equivalence partitioning and boundary value

analysis.

Page 126: Software Testing Day 2: Functional Testing

Functional testing 126

Summary-continued Finite state machine State diagram Events and actions Unreachable and dead states Test requirements and specifications for

state machines Decision tables, rules, actions