CS 217 Software Verification and

Validation

Week 8, Summer 2014

Instructor: Dong Sihttp://www.cs.odu.edu/~dsi

REVIEW OF BB-testing

Black-box Testing

1. Input Space Partitioning

2. Boundary Value Analysis

3

Example 1: compare two numbers – p50 of week3

Function ‘Compare (x, y)’

Inputs: Two numbers – x and y Outputs: A larger number between x and y

4

Compare (x, y) = z

(x, y) z

– p51 of week3

5

• Equivalence Classes:

{ (x, y) | x < y }

{ (x, y) | x > y }

{ (x, y) | x = y }

{ input other than a pair of numbers,

“as&%dfget^$(&w” }

Valid inputs

Invalid inputs

6

Valid (x, y) Input Space

x = y

x < y

x > y

Three test cases:

(1, 2) --- 2

(8, 8) --- 8

(100, 30) --- 100

Plus one test cases:

(^&%*) --- ERROR

– p52 of week3

Example 2: Loan application- p53 of week3

7

Customer Name

Account number

Loan amount requested

Term of loan

Monthly repayment

Term:

Repayment:

Interest rate:

Total paid back:

6 digits, 1stnon-zero

$500 to $9000

1 to 30 years

Minimum $10

2-64 chars.

Choosing (or defining) partitions seems easy, but is easy to get wrong…

8

Customer name

Number of characters:

2 64 65invalid valid invalid

1

Conditions ValidPartitions

InvalidPartitions

ValidBoundaries

InvalidBoundaries

Customername

2 to 64 charsvalid chars

< 2 chars> 64 charsinvalid chars

2 chars64 chars

1 chars65 chars0 chars

Valid characters:Anyother

A-Za-z-’

space

9

Loan amount

500 9000 9001

invalid valid invalid

499

Conditions ValidPartitions

InvalidPartitions

ValidBoundaries

InvalidBoundaries

Loanamount

500 - 9000 < 500>90000non-numericnull

5009000

4999001

Black-box Testing

1. Input Space Partitioning

2. Boundary Value Analysis

10

Example 3 - Bonus

Consider a program IsAnOddNumber(x) which output “Yes” or “No” to the user to tell if the input integer x is a odd number or not.

If you are planning to classify all possible user inputs into two classes - valid and invalid inputs

1) How do you partition the valid input space (the integer number space)?

2) Can you give an example of invalid test data?

11

Example 3 - Bonus

If your software manager asks the program MUST ONLY take input real number x as: -100<x<100

1) How do you partition all possible user inputs into two classes - valid and invalid inputs now?

2) If the software manager further asks you to do some Boundary Value Analysis, then what do you plan to do for the BVA analysis?

12

Example 4 - Bonus

Consider a program Absolute(x) which calculate the absolute value of a input real number x.

If you are planning to classify all possible user inputs into two classes - valid and invalid inputs

1) How do you partition the valid input space (the real number space)?

2) Can you give an example of invalid test data?

13

Example 4 - Bonus

If your software manager asks the program MUST ONLY take input real number x within the range of:

-321<x<-123, 123<x<321

1) How do you partition all possible user inputs into two classes - valid and invalid inputs now?

2) If the software manager further asks you to do some Boundary Value Analysis, then what do you plan to do for the BVA analysis?

14

Today’s topic

WHITE-BOX TESTING

Styles of Testing Testing traditionally can be conducted in two

styles

Black-Box testingTry to choose "smart" tests based on the requirements,without looking at the code.

White-Box testingTry to choose "smart" tests based on the structure of thecode, with minimal reference to the requirements.

16

Definition of White-Box Testing

Testing based on analysis of internal logic (design, code, etc.). (But expected results still come from requirements.)

Use different techniques to check every visible path of the source code to minimize errors.

It is the ability to know which line of the code is being executed and being able to identify what the correct output should be.

17

Characters of WB testing Exploits the structure, control or data flow of

programs in order to design test cases.

Typically performed during coding.

Allows to test parts of the program, since you know the structure. (with black‐box is not possible)

Allows to cover systematically every part of the program.

18

WB Testing Techniques

Logic coverage: (learned in previous classes) Statement coverage Branch coverage Condition coverage …

Dataflow based testing / Path coverage: all program paths have been traversed at least once

19

Pseudo code & Control/Dataflow Graphs

20

“nodes”

“edges”Input

output

Absolute (x){ IF (x>=0) THEN y = x;

ELSE IF (x<0) THEN y = -x;

Output y;}

x

IF (x>=0) ELSE IF (x<0)

y = x; y = -x;

y

Can you draw a graph?

21

Absolute (x){ IF (x>0) THEN y = x; ELSE IF (x==0) THEN y = x;

ELSE IF (x<0) THEN y = -x;

Output y;}

Example

22

Fun (x, y){ z = 0;

IF ((x>z) or (y>z)) THEN z = 2;

ELSE z = 3;

Output z; }

Test cases:1. Fun (0, 0) 2. Fun (2, 0) 3. Fun (0, 2) 4. Fun (2, 2) 5. Fun (8, 9)

Control/Dataflow of the example

23

Fun (x, y){ z = 1;

IF ((x>z) or (y>z)) THEN z = 2;

ELSE z = 3;

Output z; }

Input x, y

z = 1;

IF ((x>z) or (y>z))

z = 2;

ELSE

z = 3;

Output z;

Test cases:1. Fun (0, 0) 2. Fun (2, 0) 3. Fun (0, 2) 4. Fun (2, 2) 5. Fun (8, 9)

Statement coverage

Has each statement in the program been executed?

24

√ √

√

Branch coverage

Has each branch of each control structure been executed?

For example, given an if statement, have both the T and F branches been executed?

Another way of saying this is, has every edge in the Control/Dataflow graph been executed?

25

Condition coverage

Has each Boolean sub-expression evaluated both to true (T) and false (F) ?

26

Test cases:1. Fun (0, 0) 2. Fun (2, 0) 3. Fun (0, 2) 4. Fun (2, 2) 5. Fun (8, 9)

X>1 Y>1

T , F ? T , F ?

Fun (x, y){ z = 1;

IF ((x>z) or (y>z)) THEN z = 2;

ELSE z = 3;

Output z; }

Exercise Consider a program ‘Compare’ which compare

the value between two numbers. The inputs are two real numbers x and y, the program outputs the larger number between x and y.

27

Compare (x, y){ IF (x>y) THEN z = x;

ELSE IF (x==y) THEN z = x;

ELSE IF (x<y) THEN z = y;

Output z;}

Exercise - Control/Data Flow Graphs

I have provided you the Pseudo code,

Can you draw a Control/Dataflow Graphs?

Try it! This is a bonus question with +5 points towards to FINAL!!

28

Exercise - Statement Coverage

Statement Coverage requires that each statement will have been executed at least once.

What is the minimum number of test cases required to achieve statement coverage for the program Compare (x, y) ?

29

Exercise - Branch Coverage

Branch Coverage requires that each branch will have been traversed at least once.

What is the minimum number of test cases required to achieve branch coverage for the program Compare (x, y) ?

30

Condition Coverage

Condition Coverage requires that each condition (sub-expression) will have been True at least once and False at least once.

What is the relationship between Branch and Condition Coverage?

Condition / branch coverage

IF ( and )

THEN …

ELSE …

32

X>0 Y>0

T F

T , F ? T , F ?

Condition Coverage example – Combination !

IF (A or B) THEN s1;

ELSE s2;

A B Branch

test 1 T F true

test 2 F F false

test 3 T T true

test 4 F T true

Combinations of condition values: TT, TF, FT, FF

Dataflow based testing -> Path coverage

All program paths have been traversed at least once.

3434

Two paths !

Input x, y

z = 1;

IF ((x>z) OR (y>z))

z = 2;

ELSE

z = 3;

Output z;

Path coverage - Example 1

35

if a then s1else if b then s2 else if c then s3 else s4 end_if_then_else end_if_then_elseend_if_then_else

# Paths: ___

# Branches: ___

WB Testing Techniques

Logic coverage: (learned in previous class) Statement coverage Branch coverage Condition coverage …

Dataflow based testing / Path coverage: all program paths have been traversed at least once

36

Coming Up Next week…

Lab 2: White-box Testing

Preparation: Go over the slides of WB testing Understand “Code coverage” & “Data flow /

Path”