Precisely Selecting Regression Test for Aspect-Oriented ProgramsGuoqing Xu

The Ohio State Universityxug@cse.ohio-state.edu

Outline Big picture and background Problem statement and motivation Our analysis Implementation status Related work

Big Picture AOSD improves separation of concerns Challenges for analysis of AO programs

Control flow in bytecode is not equivalent to the program logic in source code

Compiler-specific weaving Dynamic/static parts of pointcut designators No analysis framework implementation available

Challenges for testing of AO programs Coverage criteria; test generation for exercising of

aspects; regression test selection

Big Picture Long-term goals

Framework that supports static analyses similar to popular analyses for OO program

Develop AO-specific techniques for analysis and testing

Topic of this talk Regression test selection for AO programs Work in progress

Regression Test Selection Testing after software modifications Re-running the entire regression test suite is

expensive Select a subset of tests to run Safe test selection chooses every test case

that may reveal a fault

Related Previous Work Rothermel and

Harrold, TOSEM 97 Graph traversal

algorithms Harrold et al.,

OOPSLA 01 Java interclass

graph (JIG) Copied from [Harrold et al., OOPSLA 01]

Execute P/Record

Coverage

IdentifyDangerous

Entities

Select Tests

Program P

Program P’

Program P

Program P’

P’s Edge Coverage Matrix

Dangerous Entities

containing edges in

P

Regression Testing for AOSD When AO features are added/modified, the

program needs to be regression tested Case 1: P is an OO program and P’ is an AO

version of P Case 2: Both P and P’ are AO programs How should regression test selection be How should regression test selection be

performed for AO software?performed for AO software?

Outline Big picture and background Problem statement and motivation Our analysis Implementation status Related work

Existing Work Applied to AO Programs How to compare two JIGs?

The JIG of the woven code includes redundant nodes and edges and does not correspond to the logical control flow as presented in the source

Need new representations How to recover CFG edges from the

execution trace when computing edge-coverage matrix? (when P is AO program) The execution trace is compiler-specific Need instrumentation before/during weaving

Existing Work Applied to AO Programs1. class C {

2. public void m(int i) {…}3. }4. 5. void bar(C p, int k) {6. p.m(k);7. }Java version P

CFG edge Call edge

Java Interclass Graph (JIG)

bar()

p.m()

return

C.m()

…

exit exit

Example1. class C {2. public void m(int i){…};3. }4. 5. void bar(C p, int k) {6. p.m(k);7. }8. 9. aspect Sample{10. void around(C c, int i):11. call(C.m(int)) && 12. target(c) &&13. args(i) {14. proceed(p , x);15. }

bar()

p.m()

Return

C.m()

…

exit

JIG for P AspectJ Version P’

C.m ()

…

exit

bar()

..around$0

return

…

c.m()

return

JIG for P’ (from woven code)

..Sample

.around$0

Some Results When we applied the [Harrold et.al. 01]

algorithm to several subjects:

Subjects V1 (java version)Num of tests

V2 (AspectJ version)Num of selected tests

bean 10 10

introduction 40 40

observer 10 10

telecom 30 30

tjp 10 10

tracing 40 40

testReadFile 20 19

Possible Approach Create “clean” CFGs in which the wrapper

code inserted during weaving is removed Graph traversal and comparison corresponds to

the “logical” structure of the code, not the compiler-specific woven code

New representation: AJIG AspectJ Inter-module Graph – more later

For regression test selection, need to consider additional issues

A More Complex Problem1. class C {2. public void m(int i){…};3. }4. 5. void bar(C p, int k) {6. C.m(k);7. }8. 9. aspect Sample{10. void around(C c, int

i):11. call(C.m(int)) && 12. target(c) &&13. args(i) {14. System.out.println(i);15. proceed(p , x);15. }

C.m ()

bar()

return

m()

AJIG for P’

…

exit exit

System.out….

Do we need to select all the tests that go through the edge marked in red?

Why it needs to be addressed This is an issue not only for AO software, but

also for procedural and OO software Advices are often free of side effects

Study in [Rinard et al. FSE 04] reported 6 “observer” advices out of ten inspected advices

Recommended for “safe” AO programming Adding side-effect-free advices should not

result in overly conservative regression test selection

Approach: use side-effect analysis

Outline Big picture and background Problem statement and motivation Our analysis Implementation status Related work

Our Work Consider both situations:

Case 1: P is an OO version, P’ is an AO version Case 2: both P and P’ are AO versions

Analysis to select regression tests Build a new control flow representation: AJIG Apply existing graph-traversal algorithm on AJIG Side-effect analysis when comparing AJIGs

AJIG AspectJ Inter-module Graph (recent work)

For the Java parts, same as JIG. Shadow node A shadow node is associated with

a set of JIGs of advices the precedence of these advices an integrated shadow advice JIG

AJIG supports all static AspectJ join point types Conservatively approximate the dynamic part of pointcut

designators.

Side Effect Side effect node

Mutate the objects that have existed before the method/advice is invoked

For an advice, All the nodes in the CFG path that does not contain a

proceed node The proceed node has side effect if its actual

parameters and parameters of the advice don’t point to same objects.

Mutates the object returned by the proceed node The return node has side effect if the return value of

the advice and that of the proceed node don’t point to the same object

Side Effect (Cont’d) Side effect related node in AJIG

Has side effects Has some dependency on the nodes that have side

effects Safe edge in AJIG

An edge is safe edge, if the sink node of this edge is not side effect related.

New Test Selection criterion New Test Selection criterion:

Dangerous set S computation: for each edge e in P, and its counter part e’ in P’ e is not equivalent to e’ and Neither e nor e’ is a safe edge.

Select a test that execute one or more edges in S

Selection

Computing dangerous set S by comparing AJIGs.

We plan to use some form of side-effect analysis Large body of existing work

Selecting tests

Implementation progress

The implementation of algorithms described in [Harrold et.al. 01] √

Building AspectJ Inter-module Graph √ Make an extension for the abc compiler that

generates the Jimple based CFGs for aspects between the weaving of inter-type declarations and advices.

Instrument advices at different phases. Side effect analysis ongoing work Evaluation

Evaluation Plan Benchmarks: ten to twenty moderate size (50-100

classes) AO projects taken from tmbenches, used by the abc compiler.

Experiments collect data.

The number of tests selected over versions. The comparison between enabling side-effect

analysis and disabling it.

Related Work Static/Dynamic Analysis for AO programs.

abc compiler [AOSD 05] [PLDI 05] [TR 04]. Static analysis of aspects [Sereni and Moor,

AOSD 02]. Zhao’s work on the analysis and testing of AO

programs [COMPSAC 03] [WPC 02][AOSD 06]. Classification system for AO programs [Rinard et.

al. FSE 04].

Related Work (Cont’d) Regression Test Selection

[LW ICSM 91], [CRV ISCE 94], [RH TOSEM 97], [Ball ISSTA 98], [Harrold+ OOPSLA 01], [OSH FSE 04]…

Change Impact Analysis [KGH+ ICSM 94], [RT PASTE 01], [OAH FSE 03],

[OAL+ ICSE 04], [RST+ OOPSLA 04], …

Thank you!!

Questions??