Issues in Testing OO Programs(7.1, 2.2.2, 2.4.2)

Course Software Testing & Verification2013/14

Wishnu Prasetya

Plan

• The goal is to discuss how to test OO programs; to that end we also need some concepts from:

• Data flow perspective to testing• Integration testing

2

Basic idea

• The two paths of P above will carry the same value of x “defined” at the beginning to the return statement.

• Do we need to test both paths? Purely with respect to the influence on x probably not.

• Control flow graph cannot make the distinction between which paths are equivalent with respect to their effect on a selected set of variables.

3

P(x,y) { if (y<x) y = x+1 ; return x*y }

Data-flow based approach(sec. 2.2.2)

• Data flow graph : CFG, where the nodes and edges are decorated with information about the variables used and defined by each node/edge.

• In my examples I avoid decorating on the edges.• Some subtlety on the granularity of the nodes/edges later.

4

P1(x,y) { if (y<x) y = x+1 ; return x*y } n1

n2

n3

(then)

use(n1) = {x,y}

use{n2) = {x}def(n2) = {y}

use(n3) = {x,y}

n0 def (n0) = { x,y }

Terminologies

• Def/use graph: see above. In general, def/use labels are allowed on the edges as well.

• A path p = [i,...,k] is def-clear wrt v iff v def(j) and v def(e) for all nodes j and edges e in p between i and k.

• It (def-clear) is a du-path wrt v iff vdef(i) and vuse(k), and furthermore it is a simple path.

5

n1

n2

n3

use = {x,y}def = { x,y }

use = {y}def = {y}

n4

n5

use = {x,y}def =

use = {x,y}def =

use = {x,y}def ={x} du-paths wrt x:

13, 134, 135 12, 1234, 1235, 45Not du-path wrt x: 1345 (not clear)

Block’s granularity

• When a node both defines and uses the same variable x, we will assume that it uses x first, then assigns to it, as in x = ... x ...

• If this assumption would be broken split the node. Else this will conflict the intention of your definition of du-path.

• Check 2.3.2 on how to map a source code to a def/use graph.6

x = x+y x = x+1y = y+1 ;

x = 0x = x+1

nuse(n) = {x,y}def (n) = { x }

nuse(n) = {x,y}def (n) = { x,y }

n1use(n1) = {}def (n1) = { x}

n2use(n2) = {x}def (n2) = { x}

Data-flow based coverage(Rapps-Weyuker 1985, Frankl-Weyuker 1988)

• Def-path set du(i,v): the set of all du-paths wrt v, that starts at i.• Def-pair set du(i,j,v): the set of all du-paths wrt v, that starts at i,

and ends at j.• Same concept of touring.• (C 2.9, All Defs Covrg, ADC) For each def-path set S = du(i,v), TR

includes at least one member of S.7

n1 n2 n3

use = {x}def = { x,y }

use = {x}def ={x}

n4

use = {x}def =

use ={x}def =

Data-flow based coverage

• (C 2.10, All Uses Covrg, AUC) For each def-pair set S = du(i,j,v), TR includes at least one member of S.

• (C2.11, All du-paths Covrg, ADUPC) For each def-pair set S = du(i,j,v), TR includes all members of S.

8

ADC TR: 12, 234

AUC TR: 12 23 , 232, 234

ADUC TR: 12 23 , 232, 234, 24

n1 n2 n3

use = {x}def = { x,y }

use = {x}def ={x}

n4

use = {x}def =

use ={x}def =

Overview of subsumption relations

• PPC ADUPC, because every simple path is a subpath of some prime path.

• AUC EC ... the argument is a bit contrived. AnO assume:– every use (of v) is preceded by a def.– every def reaches at least one use.– arrows are labelled; arrows that leave the node i must use at least one var,

and they all must use the same set of vars. 9

complete path covrg

prime path covrg

edge-pair covrg

edge covrg

node covrg

ADUPC

AUC

ADC

Summary

• Data-flow based testing ia an alternative to prime-path testing, if the latter becomes too expensive.

• Also a useful concept for integration testing and OO testing.

10

Integration Test

• To test if a component P that is a composition of smaller “components” works correctly.

• When have we tested this enough?– Problem : the combined CFG is complex, leads to many

paths to cover.

• Using the “connections” between P and its components to define coverage.

11

AO’s discussion on Integration Test

• “Hidden” in Section 2.4 about using design elements (e.g. your UML models) as your coverage criterion. AO see this as a trend, but 2.4.1 is a bit vague. But do say: “testing software based on design elements is usually associated with integration testing”.

• 2.4.2 is actually more about integration testing rather than design-element-based testing.

12

Data flow approach to Integration Test (2.4.2)

• Imagine method f from module A calls method/API g from module B.

• Idea 1: construct the combined CFG of both, then use prime path coverage criteria. Blow up... some paths are perhaps “equivalent” with respect to the interaction between f and g.

• Idea 2: focus only on the flow of the “coupling data” around the call point. 13

f (a) { ... g(e) ... }

g(x) { ... } B

A

Coupling

• caller, callee, actual parameters, formal parameters• parameter coupling, shared variable coupling,

external device coupling• More general concept: coupling variable a

‘variable’ defined in one ‘unit’, and used in the other. 14

(1) a = ... ;b = ...

(3) a = ...(2) a>b

(4) b = g(b)

(1) g(x)

(2) x<=a

(3) x==a

(4) return ...

(5) return ...

Coupling du-path(Jin-Offutt, 1998)

• (Def 2.40) Last-def of v : set of nodes defining v, from which there is a def-clear path (wrt v) through the call-site, to a use in the other unit.

• Can be at the caller or callee! Example: f1, f3, g4, g5.• (Def 2.41) First-use (of v) E.g.: g2, f4• Coupling du-path of v is a path from v’s last-def to its first-use.

15

(1) a = ... ;b = ...

(3) a = ...(2) a>b

(4) b = g(b)

(1) g(x)

(2) x<=a

(3) x==a

(4) return ...

(5) return ...

Now you have your Integration Test-Requirement

• ADC All Coupling Def Cov. : for every last-def d of each coupling var v, TR includes one coupling du-path starting from d.

• AUC All Coupling Use Cov. : for every last-def d and first-use u of v, TR includes one couple du-path from d to u.

16

Coupling vars: a, b, g.returnCoupling du-paths : a: [f1,f2,g1,g2] [f3,f4,g1,g2] b: [f1,f2,g1,g2] [f1,f2,f3,g1,g2] g.return: [g4,f4] [g5,f4]

(1) a = ... ;b = ...

(3) a = ...(2) a>b

(4) b = g(b)

(1) g(x)

(2) x<=a

(3) x==a

(4) return ...

(5) return ...

First use at the callee ..

• When you just pass a variable as in g(b), it makes sense to define g’s first use to be the first time g uses the parameter (location g.3 in the previous example).

• But when you pass a complex expression like g(a+b) what are g’s first uses of a and b?

in this case it make sense to define g’s first uses (of a and b) to be at the call location itself.

17

Sources of problems in OO programs(in addition to the traditional ones)

• A function behaves “cleanly”; a procedure does not. An OO program: even worse:– non-functional interaction through instance

variables and static variables– interaction through inheritance– dynamic binding – access control (priv, pub, default, protected, ...)

18

Anomalies

• Simply mean non-standard situations. Not necessarily errors, but they tend to be error prone. AO list 9 anomalies related to inheritance and dynamic binding:– Inconsistent Type Use (ITU)– State definition anomaly (SDA) , ... 7 more

• AO also implicitly discuss two more: deep yoyo and data-flow anomaly.

19

Inconsistent Type Use (ITU) anomaly

20

List +insertElemAt(i:int)+remElemAt(i:int)

Stack +push(x)+pop()

Obviously it is ok to use a Stack as a Stack. However, if the programmer sometimes uses a Stack also as a List this is error prone. Calling insertElemAt or remElemAt on a Stack which you intend to use as a stack is obviously dangerous.

(Not the structure is anomalous, but its use might be)

State Definition Anomaly (SDA)

• A subclass A1’s state is partially made of its superclass A’s states. Through extension or overriding, the A1 may directly manipulate its super-state part rather than via A’s methods; this may break A’s assumptions on that part.

• It would be useful if a class invariant for A is defined.21

ScalableItemsetScale(s) { scale = s }

UnitItemscale = 1price() { return 1/scale }

MultiItemquantityMultiItem() { quantity=1 ; setScale(2) }setQuantity(q)

(OK...)(error prone)

State definition inconsistency due to state variable hiding (SDIH)

• A subclass A1 of A shadows an instance variable x of A, while some methods m of A uses x. So, when m is called from A1, it will use the super-x, rather than the new x.

22

ItempricetaxnetPrice() { return price * (1 + tax) }setPrice(p) { price = p }

OnSaleItempricesale(p) { price = 0.8 p }

State Visibility Anomaly (SVA)

23

A - x+ incr()

B C + incr()

C’s needs to implements its own incr. However it can’t get to x, because it is private to A. The only way to do it is through A’s incr. This can be done simply by calling super.incr(). So far it is ok. Now imagine someone else changes B by overriding incr. This suddenly changes the behavior of C; as it now calls B’s incr instead.

Anomalous Construction Behavior 1 (ACB1)

• A1 is a subclass of A. A’s constructor calls r(). A1 overrides this r. If A1’s constructor calls super, it will then calls the new r instead (notice the Yoyo there). In the above example, it breaks because reset() would use an uninitialized baseprice (can btw be recognized with a Yoyo+dataflow graph, as shown later). 24

Itempricetax = 0 Item() { reset() }reset() { price = 1 }

ExpensiveItembasepriceExpensiveItem() { super() ; baseprice=10 }reset() { price = baseprice }

Need concept: call graph

• A graph describing which program “units” call which. .• However, in OO, due to overriding, the call graph of the same

method in the subclass may be different. While this is not surprising, we can create a “yoyo” that makes things error prone.

25

f

g h

We’ll define f g means f contains a call to g in its body (note that it does not mean that in the runtime f will actually call g; it may depend on its parameters) The two branches from f in the example says nothing about the order in which the children will be called by f.

The call graph of d() in ABCyoyo graph, Alexander-Offut 2000

26

C +j () { k() }

B +h () { i() ...} + i() { super.i() ...}+k() { }

A +d () { g() }+g () { h() }+h () { i() }+i () { j() }+j () {}

C j()

B h() i()

A d() j()g() h() i()

A d() j()g() h() i()

B h() i() k()

A d() j()g() h() i()

k()

Data flow anomaly

27

B +h ()+i ()

A +d ()+g ()+h ()+i ()+j ()

B h() i()

A d() j()g() h() i()

A d() j()g() h() i()

def x use x

def y use y

It is an anomaly; not necessarily an error. E.g. if at the runtime the use of x in j() when it is called from B’s i() is actually unfeasible.

Check the remaining anomalies yourself

• State Defined Incorrectly (SDI)• Indirect Inconsistent State Definition (IISD)• Anomalous Construction Behavior 2 (ACB2)• Incomplete Construction (IC)

28

Testing your OO programs

• Intra-method testing (as in traditional unit-testing)• inter-method testing and intra-class testing. The

latter is similar to the ADT approach.• Inter-class testing; this is perhaps closer to

integration testing.• (were proposed by Harold – Rothermel, 1994)• The first three levels are usually not enough to

expose errors related to access control, interaction through inheritance, and dynamic binding.

29

Testing your OO programs

• Due to dynamic binding the exact m called in f depends on the type of actual type of o

• We can treat this as an integration testing problem you do need to quantify over subclasses.

30

f(x) { A o = FactoryA(x ) ; ... o.m() ; ... }

Dynamic binding headache

• It can get more complicated: m and n above may interact . Note that either m or n can be also be of the superclass of the actual class of o, if they are not overriden in o.

• Again, just testing all possible (inter-methods) control flow paths will bound to explode.

• see this as some form of integration that is additionally also further affected by dynamic binding.

• Previous integration testing integration between m and f. Next concept integration between m and n. 31

f(x) { A o = FactoryA(x ) ; // (1) o.m() ; // (2) ... o.n() } // (3)

Antecedent-consequent method pairs

• o is defined when one of its field is updated. The field o.x is “indirectly” defined (i-def) if the field is updated. Analogously, use and i-use.

• Focus on coupling due to variables v (indirectly) def. by m, and used by n. antecedent and consequent.

• f itself is here called the coupling method.• We will limit to the cases where m and n target the same o

(called context var), and only to v’s which are fields of o. 32

o.x = epxr

expr = ... o.x ...

f(x) { (1) A o = FactoryA(x ) ; (2) o.m() ; ... (3) o.n() }

Antecedent-consequent method pairs

• Coupling sequence C: a pair of antecedent m, and consequence n, such that there is at least 1x def-clear path from m to n with respect to some field o.x defined in m and used in n.

• o.x is called a coupling var (of C)• The set of all coupling vars of C coupling set.

33

f(x) { (1) A o = FactoryA(x ) ; (2) o.m() ; ... (3) o.n() }

34

With polymorphism the situation will now look like this...

Notice how different types of the context o may give different coupling variables as well as coupling paths.

Binding triple

• Each coupling sequence induce one or more binding triples; each give you a test-requirement vs the polymorphism of the context o.

• Each row above is called binding triple. Each may have more than one coupling vars, and each coupling var may have multiple coupling paths.

35

A

B

C

type(o) A/C pair coupling vars

A m() n() { o.v }

B m() n() { o.u }

C m() n() { o.u }

Now, complementary testing criteria for OO

• (Def 7.53) All-Coupling-Sequences (ACS): for every coupling sequence c in f , TR includes at least one coupling path of c. ignore polymorphism.

• (Def 7.54) All-Poly-Classes (APC): for every binding triple b of c, TR includes at least one coupling path of b. ignore that c may have multiple coupling vars.

36

A

B

C

type(o) A/C pair coupling vars coupling paths

A m() n() { o.v } p1

B m() n() { o.u } q1, q2

C m() n() { o.u } r1

Complementary testing criteria for OO

• (Def 7.55) All-Coupling-Defs-Uses (ACDU): for every coupling var v of c, TR includes at least one of v’s coupling path.

• (Def 7.56) All-Poly-Coupling-Defs-Uses (APCDU): for every coupling var v of every binding triple of c, TR includes at least one of v’s coupling path.

37

A

B

C

type(o) A/C pair coupling vars coupling paths

A m() n() { o.v } p1

B m() n() { o.u } q1, q2

C m() n() { o.u } r1