1 ece 453 – cs 447 – se 465 software testing & quality assurance lecture 6 instructor paulo...

1

ECE 453 – CS 447 – SE 465 Software Testing & Quality Assurance

Lecture 6

InstructorPaulo Alencar

2

OverviewFunctional Testing

Boundary Value Testing (BVT)Boundary Value AnalysisRobustness Testing(Robust) Worst Case TestingSpecial Value Testing

Equivalence Class TestingWeak Equivalence Class TestingStrong Equivalence Class TestingTraditional Equivalence Class testing

Decision Table Based testing

3

Equivalence Class Testing

• The use of equivalence class testing has two motivations:

– Sense of complete testing– Avoid redundancy

• Equivalence classes form a partition of a set that is a collection of mutually disjoint subsets whose union is the entire set.

• Two important implications for testing:1. The fact that the entire set is represented provides a

form of completeness2. The disjointness assures a form of non-redundancy

4

Equivalence Classes

• The idea of equivalence class testing is to identify test cases by using one element from each equivalence class.

• If the equivalence classes are chosen wisely this greatly reduces the potential redundancy among test cases.

• The key point in equivalence class testing is the choice of the equivalence relation that determines the classes (partitions).

• We will differentiate below, between weak and strong equivalence class testing.

5

Equivalence Class Selection

• It tends to be a “craft”:– no dependence on knowledge of code, only the

specification– needs knowledge of input domain that usually

goes beyond what an interface design specification provides

» familiarity with specifications

– must understand how inputs are mutually dependent.

6

Example

• Let us consider a program P with 3 inputs: a, b and c and the corresponding input domains are A, B, and C.– define the partition:

A = A1 U A2 U A3B = B1 U B2 U B3 U B4C = C1 U C2

7

• define a1, a2 and a3 as:

– let ai be a “representative” or “typical” value within its respective equivalence class (e.g. the midpoint in a linear equivalence class).

– similarly define bi and ci.

• test cases can be stated for the inputs <a,b,c> in terms of the representative points.

• the basic idea behind the techniques is that one point within an equivalence class is just as good as any other point within the same class.

3,2,1 321 AaAaAa

8

Weak Equivalence Class Testing• Weak equivalence class testing is accomplished by using one

variable from each equivalence class in a test case.

• The minimum number of test cases is equal to the number of classes in the partition with the largest number of subsets.

• From the previous example, we have:

Test Case Variable a Variable b Variable c

WE1 a1 b1 c1

WE2 a2 b2 c2

WE3 a3 b3 c1

WE4 a1 b4 c2

9

Strong Equivalence Class Testing

• Strong equivalence class testing is based on the Cartesian Product of the partition subsets.

• From the previous example, this would generate:3 * 4 * 2 = 24 test cases

• Generates more test cases which test for any interaction between the representative values from each of the subsets.

• For either method, it may be possible to define equivalence relations for the program output, then test cases can also be based on these.

10

Test Case # a b cSE1 a1 b1 c1

SE2 a1 b1 c2

SE3 a1 b2 c1

SE4 a1 b2 c1

SE5 a1 b3 c1

SE6 a1 b3 c2

SE7 a1 b4 c1

SE8 a1 b4 c2

SE9 a2 b1 c1

SE10 a2 b1 c2

SE11 a2 b2 c1

SE12 a2 b2 c2

SE13 a2 b3 c1

SE14 a2 b3 c2

SE15 a2 b4 c1

SE16 a2 b4 c2

SE17 a3 b1 c1

SE18 a3 b1 c2

SE19 a3 b2 c1

SE20 a3 b2 c2

SE21 a3 b3 c1

SE22 a3 b3 c2

SE23 a3 b4 c1

SE24 a3 b4 c2

11

Traditional View• The traditional view of equivalence class testing

defines equivalence in terms of validity that is, test cases determined from the valid and invalid values for each input variable.

• For each input variable there are valid and invalid values.

• In the traditional approach, these are identified and numbered, and then incorporated into test cases in the weak sense as presented above.

12

Traditional View

• Given valid and invalid sets of inputs, the traditional equivalence testing strategy identifies test cases as follows:– For valid inputs, use one value from each valid class (as in what

we have called weak equivalence class testing). In this context, each input in these test cases will be valid.

– For invalid inputs, a test case will have one invalid value and the remaining values will be valid. In this context, a “single failure” should cause the test case to fail.

• If the input variables have defined ranges, then the test cases from traditional equivalence class testing will always be a subset of those that would be generated by robustness testing.

13

Issues with Traditional View

• There are two problems with the traditional equivalence testing:

– The first is that, very often, the specification does not define what the expected value for an invalid test case should be.

– The second problem is that strongly typed languages eliminate the need for the consideration of invalid inputs.

14

Example

• For example consider a program with two input variables size and weight:– valid ranges:

S1: 0 < size < 200

W1: 0 < weight < 1500

– corresponding invalid ranges might be:

0 weight :W3

1500 weight :W2

0 size :S3

200 size :S2

15

Test Cases Example (Traditional View)

Test Case size weight Expected Output

TE1 100 750 whatever it should be

TE2 100 -1 invalid input

TE3 100 1500 invalid input

TE4 -1 750 invalid input

TE5 200 750 invalid input

16

Equivalence Test Cases for the Triangle Problem (Output Domain)

• In the problem statement we note that there are four possible outputs: – Not a Triangle– Isosceles– Equilateral– Scalene

• We can use these to identify output (range) equivalence classes:

R1= {<a, b, c> | the triangle with sides a, b, c, is equilateral} R2= {<a, b, c> | the triangle with sides a, b, c, is isosceles} R3= {< a, b, c> | the triangle with sides a, b, c, is scalene} R4= {a, b, c> | sides a, b, c do not form a triangle}

• These classes yield the following simple set of test cases:

17

Sample Test Cases based on Output Domain

Test Case a b c Expected

Output

OE1 5 5 5 Equilateral

OE2 2 2 3 Isosceles

OE3 3 4 5 Scalene

OE4 4 1 2 Not a Triangle

18

Equivalence Test Cases for the Triangle Problem (Input Domain)

• If we base the equivalence classes on the input domain, we will obtain a larger set of test cases. We can define the sets:

D1= {<a,b,c> | a=b=c}D2= {<a,b,c> | a=b, a≠c}D3= {<a,b,c> | a=c, a≠b}D4= {<a,b,c> | b=c, a≠b}D5= {<a,b,c> | a≠b, a≠c, b≠c}

• As a separate property we can apply the triangle property to see even if the input constitutes a triangle

D6= {<a, b, c> | a ≥ b+c}D7= {<a, b, c> | b ≥ a+c}D8= {<a, b, c> | c ≥ a+b}

• If we wanted also we could split D6 into D6’={<a, b, c> | a = b+c} andD6’’= {<a, b, c> | a > b+c}

19

Equivalence Test Cases for the NextDate Problem (Input Domain)

• Nextdate is a function of three variables, month, day, and year and these have ranges defined as:

1 ≤ month ≤ 121 ≤ day ≤ 31

1812 ≤ year ≤ 2012

• We will examine below the valid, invalid equivalence classes, strong, and weak equivalence class testing.

20

Traditional Test Cases

• The valid equivalence classes are: M1= {month | 1 ≤ month ≤ 12} D1= {day | 1 ≤ day ≤ 31} Y1= {year | 1812 ≤ year ≤ 2012}

The invalid equivalence classes are:M2= {month | month < 1}M3= {month | month > 12}D2= {day | day < 1}D3= {day | day > 31}Y2= {year | year < 1812}Y3= {year | year > 2012}

These classes yield the following test cases, where the valid inputs are mechanically selected from the approximate middle of the valid range:

21

Traditional Test Cases

Case ID Month Day Year Expected

Output

TE1 6 15 1912 6/16/1912

TE2 -1 15 1912 Invalid

TE3 13 15 1912 Invalid

TE4 6 -1 1912 Invalid




22

Choice of Equivalence Classes

• If we more carefully chose the equivalence relation, the resulting equivalence classes will be more useful

M1= {month | month has 30 days} M2= {month | month has 31 days} M3= {month | month is February} D1= {day | 1 ≤ day ≤ 28} D2= {day | day = 29} D3= {day | day = 30} D4= {day | day=31} Y1= {year | year = 1900} Y2= {year | 1812 ≤ year ≤ 2012 AND year ≠ 1900 AND (0 = year mod 4} Y3= {year | 1812 ≤ year ≤ 2012 AND 0 ≠ year mod 4}

23

Weak Equivalence Class Test Cases

Case ID Month Day Year Expected Output

WE1 6 14 1900 6/15/1900

WE2 7 29 1912 7/30/1912

WE3 2 30 1913 Invalid

WE4 6 31 1900 Invalid

24

Strong Equivalence Test CasesCASE ID Month Day Year Output

SE1 6 14 1900 6/15/1900

SE2 6 14 1912 6/15/1912

SE3 6 14 1913 6/15/1913

SE4 6 29 1900 6/30/1900

SE5 6 29 1912 6/30/1912

SE6 6 29 1913 6/30/1913

SE7 6 30 1900 7/1/1900

SE8 6 30 1912 7/1/1912

SE9 6 30 1913 7/1/1913

SE10 6 31 1900 ERROR

SE11 6 31 1912 ERROR

SE12 6 31 1913 ERROR

SE13 7 14 1900 7/15/1900

SE14 7 14 1912 7/15/1912

SE15 7 14 1913 7/15/1913

SE16 7 29 1900 7/30/1900

SE17 7 29 1912 7/30/1912

SE18 7 29 1913 7/30/1913

25

Strong Equivalence Test ClassesCASE ID Month Day Year Output

SE19 7 30 1900 7/31/1900

SE20 7 30 1912 7/31/1912

SE21 7 30 1913 7/31/1913

SE22 7 31 1900 8/1/1900

SE23 7 31 1912 8/1/1912

SE24 7 31 1913 8/1/1913

SE25 2 14 1900 2/15/1900

SE26 2 14 1912 2/15/1912

SE27 2 14 1913 2/15/1913

SE28 2 29 1900 ERROR

SE29 2 29 1912 3/1/1912

SE30 2 29 1913 ERROR

SE31 2 30 1900 ERROR

SE132 2 30 1912 ERROR

SE33 2 30 1913 ERROR

SE34 2 31 1900 ERROR

SE35 2 31 1912 ERROR

SE36 2 31 1913 ERROR

26

Guidelines and Considerations

• The traditional form of equivalence testing is generally not as thorough as weak equivalence testing, and in its turn, not as thorough as strong equivalence testing

• If error conditions is a priority we can extend strong equivalence testing to include invalid classes

• Equivalence class testing is appropriate when input data is defined in terms of ranges and sets of discrete values.

• Logic of functionality of the program can help define the equivalence classes

• Strong equivalence takes the presumption that variables are independent, otherwise it generates some “error” test cases

• Can be strengthened by using it with domain testing (boundary value):– Reuse the work to define the ranges– Does not consider elements at equivalence class boundaries – Need to expand ECT to include BVT-like requirements (domain testing)

1 ece 453 – cs 447 – se 465 software testing & quality assurance lecture 6 instructor paulo...

Documents