research article mutation analysis approach to develop...

Research ArticleMutation Analysis Approach to Develop ReliableObject-Oriented Software

Monalisa Sarma

Indian Institute of Technology Kharagpur Kharagpur 721302 India

Correspondence should be addressed to Monalisa Sarma monalisaiitkgpacin

Received 4 April 2014 Revised 19 August 2014 Accepted 22 September 2014 Published 25 December 2014

Academic Editor Henry Muccini

Copyright copy 2014 Monalisa Sarma This is an open access article distributed under the Creative Commons Attribution Licensewhich permits unrestricted use distribution and reproduction in any medium provided the original work is properly cited

In general modern programs are large and complex and it is essential that they should be highly reliable in applications In orderto develop highly reliable software Java programming language developer provides a rich set of exceptions and exception handlingmechanisms Exception handling mechanisms are intended to help developers build robust programs Given a program withexception handling constructs for an effective testing we are to detect whether all possible exceptions are raised and caught ornot However complex exception handling constructs make it tedious to trace which exceptions are handled and where and whichexceptions are passed on In this paper we address this problem and propose a mutation analysis approach to develop reliableobject-oriented programs We have applied a number of mutation operators to create a large set of mutant programs with differenttype of faults We then generate test cases and test data to uncover exception related faults The test suite so obtained is applied tothe mutant programs measuring the mutation score and hence verifying whether mutant programs are effective or not We havetested our approach with a number of case studies to substantiate the efficacy of the proposed mutation analysis technique

1 Introduction

Of late computer applications have permeated heavily intoevery sphere of the daily life of an average person Manyof these applications are large complex and safety criticalrequiring the software to be extremely reliable An exceptionoccurrence during a program execution is an abnormal com-putation state [1] and thus is a threat to the reliability of soft-ware It is advocated that software should be tested adequatelyto trace if there is any exception in intended behavior [2]To realize this software developers prefer exception handlingmechanisms to be embedded inside code so that softwarecan arrest and raise alarm whenever there is any exceptionand then cause of exception can be eliminated Many reasonsmay be attributed to causes of exceptions such as erroneousinput hardware faults logical errors in the code semanticviolation linking errors and limitation of system resourcesIt is reported that more than 50 of the operational failuresthat may occur in a system are due to faults in exceptionhandling and recovery [3] To cope with the system failuresdue to exceptions most modern programming languagessuch as Ada C++ and Java incorporate explicit mechanisms

for exception handling Syntactically an exception handlingmechanism consists of a means to explicitly raise an excep-tional condition at a programpoint and ameans of expressinga block of code to handle one ormore exceptional conditionsIn other words an exception handling mechanism providesa way to separate code that deals with unusual situations andabnormal program termination

Exception handling constructs are responsible for thedetection and handling of system conditions that couldpotentially lead to failure Recent studies of Java programsindicate that exception handling constructs occur frequentlyand approximately 15ndash25 of the classes contained excep-tion handling constructs [4 5] Despite the frequency of theiroccurrences the behavior of exception handling constructs isoften the least understood andpoorly tested part of a program[6] Testing strategy to test object-oriented programs ingeneral and exception handling constructs in particular isscarcely reported [7] For testing exception handling con-structs some works have been reported [4 6 8] Sinha andHarrold [9] proposed a class of adequacy criteria that can beused to test the behavior of exception handling constructsSinha and Harrold [9] described a methodology for applying

Hindawi Publishing CorporationAdvances in Soware EngineeringVolume 2014 Article ID 197983 16 pageshttpdxdoiorg1011552014197983

2 Advances in Software Engineering

the criteria to unit and integration testing of programs thatcontain exception handling constructs However to the bestof our literature survey no approach has been reported so farto generate exception handling related faulty programs andthen to test the adequacy of test suites Adequacy testing is animportant issue because a test suite may not locate all excep-tion points For example in Java an exception can be raisedimplicitly In that case it is not possible to drawproper controlflow graph [4 10 11] Further even if all exception paths areknown from the control flow graph [4] and satisfy exception-coverage criteria [9] it needs to design test data manually forraising all possible potential exceptionsThismanual test casegeneration is not only tedious [6] but raising exceptions is notalways guaranteed As a consequence we need to determinethe adequacy of test cases to raise exceptions In generaldetermining the path of exceptions is a difficult problem andmanual synthesis of test data may not be sufficient to raiseall exceptions In this context efficiency of a test suite isnecessary to be evaluated in order to assure the reliability of asystemunder test In summary given a programwe are to testwhether the exception handling constructs are sufficient tothrow and catch all possible exceptions or not

Mutation analysis is a practical method to evaluate thequality of test suites [12ndash14] In this work we propose amutation analysis method to evaluate test suites for testingexception handling constructs We consider a set of mutationoperators with which different types of faults can be injectedto the original programsThemutation operators are adoptedfrom the work of Ji et al [15] The mutant programs arethen analyzed with some test suites We consider test suitesgenerated according to the control flow based structuralanalysis approach proposed in [4 9] for a given Java programWe also analyzed the program in JUnit 46 [16] and Jumble 10[17] testing environment and their mutant programs Ourexperimental results substantiate the efficiency of the pro-posed mutation analysis approach

The rest of the paper is organized as follows We discussbasic concepts related terminologies and tools in Section 2In Section 3 we discuss the mutation operators followed inthis work Mutation analysis is discussed in Section 4 Ourexperimental results are presented in Section 5 Related workand comparison of our work with others are discussed inSection 6 Finally Section 7 concludes the paper

2 Basic Concepts

In this section we discuss some basic concepts and termi-nologies which are referred to in our subsequent discussionsWe also briefly introduce the tools used in our work

21 Exception Handling Constructs in Java Java [18] pro-gramming language is considered as a model language in thiswork In this subsection we provide an overview of exceptionhandling constructs in Java Details of exception handlingconstructs in Java can be found in [19 20]

In Java exceptions are first-class objects Each exceptionis an instance of a class that is derived from the classjavalangThrowable which is defined in the stan-dard Java API [19] An exception can be raised at any

point in the program through a throw statement A throwstatement is associated with an expression denoting anexception object which may be a variable (eg throw e)a method call (eg throw m()) or a new instantiation(eg throw new E()) and so forth A throw statement canappear anywhere in the program Java exception handlingconstruct includes three blocks try catch and finally(see Figure 1(a)) Code can be guarded for exceptionswithin a try block Exceptions raised through execution ofcode (implicitly) or a throw statement (explicitly) withina try block may be caught in one or more catch clause(s)declared immediately following the try block The finallyblock is executed independently of what happens in thetry block It may be noted that a try statement consists ofa try block and optionally one or more catch blocks andfinally block (see Figure 1(a)) The valid instances of atry statement are try catch ( ) [catch ( ) ][ ] [finally ] where the blocks in square bracketsindicate optional Here a try block contains statementwhose execution is monitored for exception occurrencesA catch block specifies the type of exception it handlesand exception handler that is it contains a block of codethat is executed when an exception of that type is raised inthe associated try block If a finally block is associatedwith a try block then the finally block is always executedirrespective of how control transfers out of the try block

22 Java Exception Hierarchy Similar to other Java objectsexceptions are typed and types are organized into a hierarchyAll exceptions inherit from the class type Throwable definedin javalang API The exception type hierarchy definesthree different groups of exceptions errors runtime excep-tions and checked exception (defined in JavalangException) (also see Figure 1(b)) Note that errors andruntime exceptions are unchecked exceptions Uncheckedexceptions can be thrown at any point in a Java programand if uncaught may propagate back to the programentry point causing the Java Virtual Machine [20] toterminate The class Error is responsible for giving errorsin some catastrophic failures The exceptions of typeRuntimeException are typically created automaticallyduring runtime in response to some execution errors Theexceptions of the type Exception are checked exceptionsand are used for exceptional conditions that user programscan catch A list of exceptions that a Java programmer cancatch in programs is available in [19] There are 11 runtimeexceptions 17 errors and 7 checked exceptions defined inJavalangAPI [19] as per the release of Java 6 Programmerscan define and extend existing exception type in Java API tocustomize their needs Some new exception types can alsobe defined [20]

23 Call Exception Hierarchy In this subsection we discussthe flow of exceptions within the context of the structure ofa given programThis flow of exceptions we specially term itas call exception hierarchy (CEH) is useful to understand andevaluate how exceptions are handled within a method whichexceptions might arise during the execution of a method

Advances in Software Engineering 3

Guarded block

where exception can

be raised

Exception handler for



Must executable code

try

finally

exception type Exceptionm

exception type Exception2


catch (Exception1 e1)


catch (Exceptionm em )

(a) Exception handling constructs in Java

JavalangObject

JavalangThrowable

JavalangError JavalangException

JavalangRuntimeException

(b) Exception class hierarchy in Java

Figure 1 Exception handling constructs and exceptions in Java [19]

which exceptions are handled and where and which excep-tions are passed on Alternatively a call exception hierarchyspecifies which exceptions may propagate to its caller Inessence the call exception hierarchy is precisely specifyingthe behavior of the program as far as the exceptions andhandling exceptions are concerned [21]

We identify three basic control flow constructs to obtainthe call exception hierarchy in any program These aremethod calls nesting andmulticatch Any complex and arbi-trary call exception hierarchy can be resolved with the help ofthe above three basic constructs We discuss the three basicconstructs as follows

231 Method Calls A method say Main can call a method119898119894 which in turn may call another method say 119898

119895and

so on Any method can throw and catch exceptions Thecall exception hierarchy in this case implies the parent-child relationship among exceptions An exception 119864

2is a

child of another exception say 1198641 if 1198642occurs in a method

that is called by the method where 1198641may occur The call

exception hierarchy with method calls is illustrated with anexample in Figure 2(a) In Figure 2(a) the method Main callsthe method 119872

1() and is supposed to catch an exception 119864

Now themethod1198721() callsmethod119872

2() andmay throw two

exceptions namely 11986411

and 11986412 The method119872

2() in turns

may throw exception 11986421and 119864

22 In this case the exception

119864 should be at the root of the hierarchy because119864 is supposedto represent all instances of exceptions when the invocationof the method119872

1occurs The exceptions 119864

11and 119864

12are 119864rsquos

children because they are instances which may propagate to119864 Similarly 119864

1is a child of 119864 and 119864

21 11986422are the children of

1198641 Note that here the exceptions 119864

21and 119864

22are supposed to

be caught through the exception 1198641 if it fails then through 119864

The call exception hierarchy for a situation of method calls inFigures 2(a)(i)ndash2(a)(iii) is shown in Figure 2(a)(iv)

232 Nested try Blocks In Java a try block can be nestedwithin another try block and so on In this case the callexception hierarchy can be obtained as discussed below Allexceptions at outer try block are at the root All exceptionsat the next inner try blocks are the children of the rootAny exceptions in a try block at the next level are thechildren of exceptions at its upper try level and so on Weillustrate such a situation with an example We consider atwo-level nested try structure as shown in Figure 2(b) InFigure 2(b)(i) the method Main is supposed to catch theexception 119864 which is at the outermost try block Now Maincalls method119872

1() at the innermost try block The method

1198721()may raise two exceptions namely 119864

11and 119864

12 11986411and

11986412would be the successor of 119864

1 which is supposed to catch

these two exceptions In turn 1198641would be the successor of

119864 The call exception hierarchy of the program structure ofFigures 2(b)(i) and 2(b)(ii) is shown in Figure 2(b)(iii)

233 Multiple catch Blocks There may be multiple catchblocks corresponding to a try block [19 20] In this caseall exceptions which are supposed to be caught wouldbe placed as a single node and all exceptions which areraised by a called method inside the try block would bethe children of this node As an example let us consider asimple try with multiple catch as shown in Figure 2(c) InFigure 2(c)(i) the method Main contains a try block andthis try block is associated with three parallel catch blocksThese three catch blocks are corresponding to handle threedifferent exceptions namely 119864

1 1198642 and 119864

3 Further the

method Main() calls the method1198721() which is supposed to

throw the exceptions 11986411and 119864

12(see Figure 2(c)(ii)) In this

case the exceptions 1198641 1198642 and 119864

3of the callermethod Main

is at the root and the exceptions 11986411

and 11986412

of the calledmethod119872

1() are the children of the exceptions 119864

1 1198642 and


(i)

Main

middot middot middot






(iii)

(ii)

(iv)

M1

middot middot middotmiddot middot middot


try

try





catch E1

catch E

middot middot middotM2




throw E21

throw E11

throw E12



throw E22

E

E11E1

E21 E22

E12

if ⟨condition 2⟩


M1( )

M2( )

(a) Call exception hierarchy with method calls

(i)

(ii)

(iii)

Main







M1middot middot middot





E

E11

E1

E12

if ⟨condition 1⟩throw E11

throw E12


try

try

catch E1

catch E

M1( )

(b) Call exception hierarchy with nested try blocksMain





(i)

M1






(ii)

E1E2E3

E11 E12

(iii)

try

catch E1catch E2catch E3


throw E12


M1( )

(c) Call exception hierarchy with multiple catch blocks

Figure 2 Basic constructs of call exception hierarchy

1198643 The call exception hierarchy for this case (Figures 2(c)(i)

and 2(c)(ii)) is shown in Figure 2(c)(iii)Related to the call exception hierarchy we define the

following terminologies

Caller Exception An exception 119864 in a CEH for a program119875 is called the caller exception if it is a parent of any otherexception 119864

119894isin 119862119864119867 In other words all exceptions except at

the leaf of call exception hierarchy are the caller exceptions

Callee Exception An exception 119864 in a CEH for a program119875 is called the callee exception if it is a child of any other

exception 119864119894isin 119862119864119867 In other words all exceptions except

the exception(s) at the root of a call exception hierarchy arecallee exceptions

Direct Caller Exception In CEH if an exception 119864119894is the

immediate predecessor of another exception 119864119895 then the

exception 119864119894is called the direct caller exception

Direct Callee Exception In CEH if an exception 119864119894is the

immediate successor of another exception119864119895 then the excep-

tion 119864119894is called the direct callee exception


Inputprogram P

Input testprogram Pn

Create Input testsuite T

Modify testsuite

Fix program

No

No YesYesProgram testcorrect

Run testcases oneach livemutant

All mutantsdead Quit

Run testcases on P

mutants 120601(P)

120601(P)

Figure 3 Mutation testing framework

24 Mutation Testing Mutation testing [22] is a widelyaccepted and practical method to evaluate the quality of a testsuite to test programsMutation testing is a fault-based testingadequacy criterion proposed by DeMillo et al [23] initiallywith the name mutation analysis Given a program 119875 aset of alternative programs say 120601(119875) is considered in order tomeasure the test adequacy of a test suite 119879 A test set 119879 is ade-quate to 119875 in relation to 120601(119875) if for each program 119876 isin 120601(119875)either 119876 is equivalent to 119875 or 119876 differs from 119875 on at least onetest case 119905 isin 119879 [24] Amutation testing framework is shown inFigure 3 In the context of mutation analysis some commonlyknown terminologies are mentioned below

241 Mutant A set of alternative programs 120601(119875) is calledmutants of 119875 A mutant program 119876 isin 120601(119875) is different from119875 only on some simple syntactic changesThis is also alterna-tively called fault injection to 119875

242 Mutation Operator Mutation operator is a set of rulesto decide syntactic changes in the program 119875 to obtainmutants 120601(119875)

243 Mutant Dead To assess the adequacy of a test set 119879each mutant 119876 isin 120601(119875) as well as the program 119875 has to beexecuted against the test cases in 119879 If the observed output ofthe mutant 119876 is the same as that of 119875 for all test cases in 119879then 119876 is considered alive otherwise it is considered dead ordistinguished

244 Equivalent Mutant Let 119863 be the input domain of thetest suite119879 Amutant119876 is called an equivalentmutant if forall119909 isin119863119875 and119876 produce the same results Note that an equivalentmutant cannot be distinguished and should be discardedfrom the mutant set as it does not contribute to the adequacyof 119879

245 Mutation Score Test adequacy is measured by themutation score 120583(119875 119879) as defined in

120583 (119875 119879) =DM

TM minus EM (1)

where DMdenotes the number of deadmutants of119875 for a testsuite 119879 and TM and EM are the total number of mutants in120601(119875) and equivalent mutants respectively In summary a lowvalue of the mutation score implies either test suite unable tocover sufficient faults or there is less faults and vice versa

25 Java Program Testing Tools Of late Java programminghas been extensively practiced in industries A number ofJava tools are available to support the Java practitioners Twocommonly used tools are JUnit [16 25] and Jumble [17 26]These two tools are used in this work We introduce thesetools briefly in the following

251 JUnit JUnit [16] is an API framework and is used toautomate unit and regression testing Using JUnit an entireclass part of a class that is a method or some interactingmethods and interactions among several objects can betested To test using JUnit a tester class is requiredThe testerclass contains more than one test case where each test caseis written into one test method [25] The tester class alsoincludes a test executor to run the test cases and methods toset up the state(s) of object(s) before and update the state(s)after the execution of each test case

The advantages with JUnit are that it allows assertions fortesting expected results it has test features for sharing com-mon data and it helps to isolate and localize errorsThe limi-tation with JUnit is that developers have to write test methodsthemselves which requires a substantial amount of codingeffort Further JUnit is designed to call methods and comparethe results they return against expected resultsThis works formethods that return some results but many methods insteadof returning values have side effects such as displaying


output modifying states of constituent objects and modify-ing attributesrsquo values Also JUnit cannot directly test privatemethods and not suitable for system or integration testing

252 Jumble Jumble [17] is a mutation testing tool for Javaprograms which interoperates with JUnit Jumble takes aclass to be mutation tested and a set of JUnit test classesas parameters The output is the overall score of how manymutations are successfully caught as well as details aboutthosemutationswhich are not Jumble performs itsmutationsby directly modifying Java byte code using BCEL package[26] This allows the mutation testing to be done with nocompilations

The advantages with Jumble are that it mutates a programat the byte code level hence Jumble is faster and it can be usedto test the programs even when the source code is not avail-able Moreover Jumble uses a heuristic to improve its perfor-mance Using the heuristic it determines the order in whichto run the test cases so that a test fails at a faster rate It ensuresthat the longer test cases run only when there is no shortertest case to cover mutations

The major limitation with Jumble is that it only providesmutations operators for conditional arithmetic initializationand assignment expressions return values and switch state-ments [26] Jumble cannot discriminate boolean shortand char data types because all these data types in byte codeis represented as 32 bits integer Further Jumble uses strongmutation [26] Weak mutation is not possible with Jumblebecause if JUnit test case uses a value that causes a mutatedexpression to return a different value there is no guaranteethat the JUnit test case will detect that different value

3 Mutation Operators

We consider eight mutation operators to create mutantprograms [15] These mutation operators are listed in Table 1There are two types of mutation operators related to catchblock and try block These mutation operators are discussedin the following subsections First we discuss the proposedmutation operators to change in catch blocks We thendiscuss the mutation operators related to alteration in tryblocks We refer to CEH as the call exception hierarchy of aJava program 119875 in our discussions

31 BD Deletion of catch Block The mutation operator BDis proposed to delete a catch block say CB following a tryblock say TB Deleting a catch block lt 1198782 gt using themuta-tion operator BD can be formally stated as

119861119863 rArr [119879119861 ]try [119862119861 ]catch(119864) rarr [119879119861 ]try

Figure 4 explains the operation of the mutation operatorBD If there are more than one catch block then it candelete in an arbitrary way one or more catch block(s) Theobjective of this mutation operator is to alter the exceptionpropagation path When an exception say 119864 is thrown inTB the mutant program with BD should not handle theexception 119864 appropriately because of the absence of CB As

Table 1 Exception handling related mutation operators

Operator DescriptionBD Catch block deletionBR Catch block replacementBI Catch block insertionRE Catch and rethrow exceptionTD Deletion in try blockTR Replacement in try blockTI Insertion in try blockTA Alter try block

middot middot middotmiddot middot middotmiddot middot middot





BD


try try

catch (Exception e) middot middot middot

Figure 4 Mutation operator BD

a consequence the exception raised at TB should be backpropagated to the caller exception if any in the CEH Theexception 119864 then may be handled at an upper level if theexception type of 119864 matches with some other exception atupper level It may be noted that the original program anda mutant version may cause same exception meeting somereachability at the try block TB In that case deleting thewhole catch block CB would not produce any distinctionsbetween the two programs for a test case Hence this operatormay produce equivalent mutants

32 BR Replacement of catch Block Themutation operatorBR is proposed to cover faults such that a program doesnot handle exception correctly Let CB be the catch blockto be altered The mutation operator BR attempts to deviatefrom the exception propagation path The mutation operatorBR creates mutant program(s) depending on the level of theoccurrence of the catch block CB Also see Figure 5 whichstates the three different levels of CBs and their possiblemutants Three rules are to be followed

Rule 1 If the CB is at the root node in the CEH then BRreplaces the exception parameter type ofCBwith that ofDCEwhereDCE denotes the direct callee exception of the exceptionof CB

Rule 2 If the CB is at any leaf node in the CEH then BRreplaces the exception parameter type ofCBwith that ofDPEwhereDPE denotes the direct caller exception of the exceptionof CB

Rule 3 If the CB is at an intermediate level that is neither atthe root nor at leaf of theCEH thenCBR replicates the catchblock CB into two blocks CB1 and CB2 such that BR replaces


R1 replace by direct calleeexception parameter type

R3 replace by directcaller + direct callee

R2 replace bydirect caller

E

E2E1

E12E23

Figure 5 Mutation operator BR

the exception parameter type ofCBwith that ofDCE andDPEas CB1 and CB2 respectively Here DPE andDCE denote thedirect caller and direct callee exceptions of the exception typeof CB in CEH respectively

The mutation operator BR can be formally stated asfollows

Rule 1 If CB is at the root of CEH

119861119877 rArr [119879119861 ]try [119862119861 ]catch(119864) rarr [119879119861 ]try [119862119861 ]catch(1198641015840)

where 1198641015840 is the DCE of 119864

Rule 2 If CB is any leaf node in CEH


where 1198641015840 is the DPE of 119864

Rule 3 If CB is any node other than root and leaf node inCEH

119861119877 rArr [119879119861 ]try [119862119861 ]catch(119864) rarr [119879119861 ]try [1198621198611]catch(1198641015840) [1198621198612 ]catch(11986410158401015840)

where 1198641015840 and 11986410158401015840 are the DCE and DPE of 119864 respectively

33 BI Insertion in catch Block The purpose of the muta-tion operator catch block insertion BI is to catch all possibleexceptions that might arise at a try block TB In other wordsif CB is the catch block associated with the try block TBthen BI tries to create mutant programs which hide allexceptions in the try block TB The mutation operator BIinserts catch blocks 119862119861

1 1198621198612 119862119861

119899for all the exception

parameter types 1198641 1198642 119864

119899where 119864

1 1198642 119864

119899are the

callee exceptions in the CEH for the exception at the tryblock TB

Table 2 Mutation operator TD

Operator Description LevelAOD Arithmetic operator deletion

Method-levelCOD Conditional operator deletionLOD Logical operator deletionIHD Information hiding deletion

Class-level

IOD Overriding method deletionISD Super keyword deletionOMD Overloading method deletionJTD This keyword deletionJSD Static modifier deletionJID Member variable initialization deletion

It can be formally stated as

119861119868 rArr [119879119861 ]try [119862119861 ]catch(119864) rarr [119879119861 ]try [119862119861

119894]catch(119864

119894)

where 119862119861119894is a catch block with exception 119864

119894for each 119864

119894isin

119862119864119867The operation of BI mutation operator is depicted in

Figure 6 Since the mutation operator BI hides some excep-tions to propagate upward it may produce equivalentmutants

34 RE Catch and Rethrow Exception A situation that mayarise in a program is that exception raised in a try blockmay not be propagated down the exception handling stackTo cover such a fault the mutation operator called catchand rethrow exception and denoted by RE is proposed Themutation operator RE for a catch block say CB associatedwith a try block say TB catches the exception type E of CBand rethrows the exception E inside the CB (see Figure 7 theoperation of RE)

Rethrowing an exception using the mutation operator REcan be formally stated as

119877119864 rArr [119879119861 ]try [119862119861 ]catch(119864) rarr [119879119861 ]try [119862119861 ]catch(119864) 119905ℎ119903119900119908119864

119894

for all 119864119894isin 119862119864119867

Let 119864 be the exception type at the catch block CB Alsolet 1198641 1198642 119864

119899be the callee exceptions of the exception

E according to call exception hierarchy CEH The mutationoperator RE creates the mutant programs by inserting athrow statement to throw the exception 119864

119894for each 119864

119894isin

119864 1198641 1198642 119864

119899 Note the difference between BI and RE In

BI a catch block is inserted with an exception whereas inRE it inserts a throw statement in the catch block to throwexception of callee exception type of the exception at catchblock including itself

35 TD Deletion in try Blocks The mutation operator TDconsists of two types of operators method-level and class-level mutation operators Concepts of these operators aretaken from [27] These operators are listed in Table 2

The first three mutation operators in Table 2 are themethod-level mutation operators and they modify Java









E1 E2

E21 E22

E

(i) Original program with CEH (ii) Mutant program with BI

try

try

E) catch (Exception middot middot middot


1 1)Ecatch (Exception middot middot middot




Figure 6 Mutation operator BI








E1 E2

E11 E12

E

throw elowasti

(i) Original program with CEH (ii) Mutant program with RE

try

try



lowastFor each ei 120576 E E1 E2 E11 E12

Figure 7 Mutation operator RE

expressions by deleting primitive operators in Java program-ming language [19] The rest of the mutation operators inTable 2 are the class-level mutation operators dealing withobject-oriented features such as inheritance polymorphismsand dynamic binding These operators are to modify dec-larations modifiers initializations and so forth A detaileddescription of these mutant operators can be found in [28]

The mutant operator TD can be mathematically stated asfollows

119879119863 rArr [119879119861 ]try 119878 [119862119861 ]catch(119864) rarr[119879119861 ]try 1198781015840 [119862119861 ]catch(119864)

where 1198781015840 is an altered statement of 119878 after deleting one ormoreoperator(s) or declaration(s)

36 TR Replacement in try Blocks Like the mutation oper-ator TD the mutation operator TR consists of method-leveland class-levelmutation operators [27 28] which are listed inTable 3 As shown in Table 3 the first six mutation operatorsare to modify expressions by replacing some compatible Javaprimitive operators The remaining mutation operators inTable 3 are to replace modifiers assignments initializationand so forth [28]

Themutant operator TR can be formally stated as follows119879119877 rArr [119879119861 ]try 119878 [119862119861 ]catch(119864) rarr[119879119861 ]try 1198781015840 [119862119861 ]catch(119864)

where 1198781015840 is an altered statement of 119878 after replacing one ormore operator(s) or declaration(s)

37 TI Insertion in try Blocks Different mutation operatorsbelonging to the proposedmutation operatorTI are shown inTable 4 In Table 4 the mutation operators in first three rowsare to modify expressions by inserting some compatible Javaprimitive operators The rest of the operators in Table 4 arerelated to insertions ofmodifiers assignments initializationsand so forth [28]

The mutant operator TD can be mathematically stated asfollows119879119868 rArr [119879119861 ]try 119878 [119862119861 ]catch(119864) rarr[119879119861 ]try 1198781015840 [119862119861 ]catch(119864)

where 1198781015840 is an altered statement of 119878 after inserting one ormore operator(s) or declaration(s)

38 TA Alter try Block The mutation operators discussedin the previous subsections are proposed to modify some


Table 3 Mutation operator TR

Operator Description LevelAOR Arithmetic operator replacement

Method-level

ROR Relation operator replacement

COR Conditional operatorreplacement

LOR Logical operator replacement

ASR Assignment operatorreplacement

SOR Shift operator replacement

IOP Overridden method callingposition change

Class-level

IOR Overridden method renamePCC Cast type change

OMR Overloading method contentchange

EOA Reference and assignmentreplacement

EOC Reference and contentreplacement

EAM Accessor method changeEMM Modifier method change

Table 4 Mutation operator TI

Operator Description LevelAOI Arithmetic operator insertion

Method-levelCOI Conditional operator insertionLOI Logical operator insertionIHI Hiding variable information

Class-levelISI Super keyword insertionPCI Type cast operation insertionJTI This keyword insertionJSI Static modifier insertion

statements within try blocks In the mutation operatorTA we propose to include some statements which areoutside try blocks or move some statements outside tryblocks Figure 8(ii) shows the working of the mutation oper-ator TA In Figure 8(i) 119875

1is a Java program and there is a try

block such that a variable say 119894119899 is initialized in the try blockand it is referenced later outside the try block Suppose that119894119899 is initialized with a null pointer In that case it may thrownull pointer exception [29] We mutate the program 119875

1to 11987510158401

by including the statement of reference within the try blockwith the objective that whether the catch block correspond-ing to try block is able to catch the exception type whichmight raise due to the reference of 119894119899 Here the variable 119894119899 isinitialized and referencedwithin the try block and exceptionraised say due to null pointer assignment in statementmarkedas (119860) may be caught in the catch block Alternatively letus consider the case of Java program 119875 where a variable 119909is initialized and referenced in a try block We apply themutation operator TA to 119875 to have a mutant program 1198751015840 suchthat the reference statement is moved from the try block and

place it outside the try-catch block The objective of thistype ofmutation is to checkwhether the exception raisedmaybe propagated and catch in any caller exception or not In themutant program it is now placed outside the try block andwhich is now beyond the scope of the catch block of the tryblock

The mutant operator TA can be mathematically stated asfollows


where 1198781015840 is an altered statement of 119878 obtained after eitherkeeping both declaration and reference of an object withinthe same try block if they were not together or declarationin the try block but its reference outside the try block whenthey were originally together within a try block

4 Mutation Analysis

In the last section we have discussed the proposed mutationoperators to test the effectiveness of test cases for testingexception handling constructs in Java In this section we dis-cuss our analysis to evaluate the effectiveness of the proposedmutation operators To do this we have considered a numberof Java programsWe then apply mutation operators to createa number of mutant programs for each Java program Anumber of test cases which are used in our analysis is thendiscussed All these steps are discussed in the followingsubsections

41 Subject Programs In order to verify the effectiveness ofour approach we have carried out a number of experimentsIn our experiments we have considered five different typesof subject programs from different application domains TheJava programs considered are

(i) library information systems (LIS) automating theregular routines in library management

(ii) cell phone system (CPS) designing hand-set servicesfor mobile communications

(iii) trading house automation system (TAS) automatingbusiness processes of trading houses

(iv) conference management systems (CMS) automatingvarious activities in conference organization

(v) vending automation system (VAS) automating vend-ing processes

The designs of LIS and CPS are followed from [30] and[31] respectivelyThe applications namelyTAS andCMS aredesigned in UML 20 by the students of postgraduate course(software engineering) inComputer Science and EngineeringDepartment Indian Institute of Technology Kharagpur [3233] The UML models of the application VAS are takenfrom [34] We follow the designs from the above-mentionedsources and implemented them in Java The implementationcharacteristics are summarized in Table 5

The Java programs we have considered are neither verysmall nor very large but of moderate sizes In column 3 of


Variable say x

is initialized ina try-block andlater x isreferenced inoutside the try-block

Variable x isinitialized andreferenced withina try-block

TA

(i)

P9984001P1









public void openFile (String fname)

inread (readData 0 255)

in = new FileInputStream (frame) (A)

catch (IOException ioe)

(ii) Original program P1

throw FileNotFoundException (ldquofile not foundrdquo)









byte readData [ ] = NULL

(B)


(iii) Muatnt program P9984001 of P1


try


in = new FileInputStream (frame)try


Figure 8 Mutation operator TA

Table 5 the numerator indicates the number of entity andboundary classes and the denominator indicates the numberof modal classes also called controller classes (classes whichexhibit distinct behavior in different states) The numbers oftotal try blocks catch blocks are shown in columns 4 and 5of Table 5 respectively The number of exceptions thrown in

Java methods which are not within the scope of try block isshown in column 6 of Table 5

42 Mutant Programs In our mutation analysis we haveconsidered fault-based mutation analysis technique For thiswe have generated the mutant programs of the considered


Table 5 Characteristics of subject programs

Subject program Size (LOC) Number of classes Exception handling constructsTry blocks Catch blocks Outside exceptions

LIS 3356 1604 26 33 18TAS 2448 1606 30 38 29CPS 2257 2107 24 46 21CMS 2146 2509 19 32 27VAS 1864 1004 11 25 31

programs (in Table 5) The mutant programs are generatedby seeding faults randomly to the programs based on a faultmodelThe faultmodel we have followed in our work is statedbelow

(1) Catch block misses exception the exception thrownin a try block does not match with the exceptionparameter type of the catch block associated withthe try block

(2) Catch block handles exceptions incorrectly the bodyof a catch block that is the exception handlingroutine is not proper to handle the exceptions

(3) Catch block rethrows exceptions catch block handlesan exception and rethrows the same or anotherexception(s)

(4) Catch block catches exceptions exception thrownmatches with the exception parameter type ina catch block and does not perform any routine butavoids abnormal exit

(5) Try block throws exceptions implicitly due to faultsin expression keyword declaration initializationand assignment some exceptions may be generatedimplicitly

(6) Try block misses or throws explicit exceptions someexceptions are missed and thrown outside try blocksome exceptions are thrown deliberately inside tryblock

Based on the above fault model we have applied themutation operators as discussed in Section 3 and the numberof mutant programs created for each programs is shown inTable 6

We have also generated a number of mutant programswith Jumble [17 26] The mutant programs with Jumble foreach sample program is shown in the last row of Table 6According to the proposed mutation operators we create149 mutant programs for LIS (see column 2 Table 6) 152mutant programs for TAS (column 3 Table 6) 163 mutantprograms for CPS (column 4 Table 6) 162 mutant programsfor CMS (column 5 Table 6) and 131 mutant programs forVAS (column 6 Table 6)

43 Generating Test Cases To test the subject programs andtheir mutant versions we have considered several test casesfor each subject program Generation of test cases is not thefocus of this work We have studied the existing test case

Table 6 Mutant programs

Mutation operator Subject programsLIS TAS CPS CMS VAS

BR 16 14 19 22 14BD 17 19 20 26 18BI 12 18 21 22 15RE 8 11 16 10 13TR 34 27 29 26 21TD 23 26 21 22 20TI 30 26 25 24 22TA 9 11 12 10 8Jumble 98 105 102 118 91

generation approaches and followed an appropriate approachas discussed in the following

Structural testing techniques [35] develop test cases tocover various structural elements of a program such ascontrol-flow data-flow branch and path Control-flow basedstructural testing criteria [36] use the flow of control in a pro-gram to generate test cases Data-flow-based testing criteria[37ndash40] use the data-flow relationships to select the test casesIn branch testing [36 41] test cases are developed by consid-ering inputs that cause certain branches in the programundertest to be executed Similarly in path testing [42 43] test casesare generated to execute certain paths in programs All theexisting structural testing techniques however not necessar-ily cover new structural element such as exception handlingconstructs all statements that raise exceptions and those thatcatch exceptions Please note that these criteriamay be similarto the traditional coverage criteria [36] that require thecoverage of statements or branches Nevertheless the criteriasuch as branch testing [41] have weak fault detection capa-bilities [44] As a consequence all-throw all-catch coveragecriteria are not necessarily strong coverage criteria In factthis coverage criterion does not cover the various exceptionalcontrol flow paths [4] Sinha andHarrold [9] proposed a fam-ily of exception testing criteria to adequately test the behaviorof exception handling constructs Based on the proposedcriteria [9] Sinha and Harrold [4] proposed a testing mech-anism dealing with exception handling constructs In [4] atechnique has been proposed to construct control-flow graphrepresentation of programs with explicit exception occur-rences that is exceptions that are raised explicitly throughthrow statements and exception handling constructs This


Table 7 Results of mutation analysis

Program under test Number of test cases Mutant killed Equivalent mutants Mutation scoreJumble Our approach Jumble Our approach Jumble Our approach

LIS 35 18 116 32 16 27 87TAS 48 14 129 28 12 18 92CPS 56 23 112 26 15 30 76CMS 44 29 117 29 23 33 84VAS 32 20 108 23 16 29 94Average 21 116 28 16 27 87

control-flow representation is useful to trace various excep-tional control flow paths and cover different types of excep-tion that can be raised at a statement or the complex controland data interactions both intra- and intermethods thatmayhave in the presence of exception handling constructs In ourwork we propose the test case generation scheme followingthe control flow representation of programs proposed in [49]

From the control-flow representation we can specify thepaths to be tested however test data to exercise a path is achallenging task particularly for complex and large numberof paths The existing approaches of test data generation canbe divided into three classes random static and dynamicRandom test data generation is easy to automate but problem-atic [45] First it produces a statistically insignificant sampleof the possible paths through the programunder test Secondit may be expensive to generate the expected output datafor the large amount of input data produced Finally giventhat exceptions occur only rarely the input domain whichcauses an exception is likely to be small Random test datagenerations may never hit on this small area of the inputdomain

Static approaches to test data generation generally usesymbolic execution Many test data generation approachespresented in the literature use symbolic execution to obtainstructural test data [46ndash50] Symbolic execution works bytraversing a control flow graph of the program under testand building up symbolic representations of the internalvariables in terms of the input variables for the desiredpath Branches within the code introduce constraints on thevariables Solutions to these constraints represent the desiredtest data A number of problems exist with this approachUsing symbolic execution it is difficult to analyze recursionarray and indices which depend on input data and some loopstructures Also the problem of solving arbitrary constructsis known to be undecidable

Dynamic test-data generation involves execution of theprogram under test and a directed search for test data thatmeets the desired criterion [51] Local search techniquesonly work effectively for linear continuous functions Conse-quently these techniques are likely to become stuck at a localoptimum [52] DeMillo and Offutt [53] and Offutt et al [54]proposed approaches for test data generation for mutationtesting The technique proposed in [53] is based on mutationanalysis to create test data approximating the test adequacyThe technique is to automatically generate constraints and

solve them to create test cases for unit testing Another tech-nique proposed in [54] is an extension of previous research[53] in constraint-based testing This later method takes aninitial set of values for each input and then refine the valuesthrough the control-flow graph of the program and resolvethe path constraint dynamically In this work to generate thetest data for test cases we follow the approach proposed in[54]

5 Results and Discussion

Test suites for each program considered in our work weregenerated as discussed in Section 4 Mutant programs foreach program is also developed using the mutation opera-tors discussed in Section 3 In this section we discuss ourexperiments and result is obtained Let 119879 be the test suite fora program 119875 and 120601(119875) its set of mutant programs For eachtest case 119905 isin 119879 we execute the program 119875 and all mutantprograms 119876 isin 120601(119875) If the observed output of the mutant119876 is different than that of 119875 then we increase the count ofthe number of mutants killedThis way we calculate the totalnumbers of mutants killed by a given test suite 119879 Further wealso calculate the equivalent mutants such that two mutantprograms 119876

1isin 120601(119875) and 119876

2isin 120601(119875) are not distinguishable

as far as the output of the two is concerned We determinewhether a mutant 119876 isin 120601(119875) is an equivalent mutant or notTo do this we look through the semantic exception hierarchy(as discussed in Section 22) If the exception parametertype in any original catch block is a super class of all thederived types (according to Java exception hierarchy) in themutant catch block then we judge the mutant 119876 to be anequivalent mutant This is because the base type of exceptioncan also accommodate all its subclass types We also check ifsome mutants due to the mutation operators BD BI TD TRand TI are equivalent or not

Our experimental results of mutation analysis with fiveconsidered subject programs are presented in Table 7 Thenumber of mutants killed equivalent mutants detected andmutation score for each program and its number of test casesin test suite are shown in Table 7 It is observed that the num-ber of mutants killed by Jumble with the mutant programscreated by Jumble is on average 21 On the other hand thenumber of mutants killed with mutant programs created byour proposed mutant operators is on average 87 It is alsoobserved that mutant killed by Jumble is mainly relatedto logical arithmetic and conditional expression whereas


180

160

140

120

100

80

60

40

20

0LIS TAS CPS CMS VAS

JumbleOur approach

Num

ber o

f mut

ants

(a) Mutant programs

140

120

100

80

60

40

20

0

LIS TAS CPS CMS VAS

JumbleOur approach

Mut

ant k

illed

(b) Mutants killed

180

160

140

120

100

80

60

40

20

0

LIS TAS CPS CMS VAS

JumbleOur approach

Mut

atio

n po

ints

(c) Mutation points

Mut

atio

n sc

ore

100

90

80

70

60

50

40

30

20

10

0

LIS TAS CPS CMS VAS

JumbleOur approach

(d) Mutation scores

Figure 9 Mutation analysis Jumble versus our approach

mutant killedwith our approach is largely related to exceptionhandling constructs The performance of our mutation anal-ysis is attributed in Figure 9 Figure 9 compares the mutantprograms created mutant killed mutation points with excep-tion handling constructs and mutation score according toJumble and our approach

Itmay be noted that JUnit considers onemodule at a timeIn contrast we have followed the system testing approachThis is because the unit testing approach is not so effective totest an object-oriented program asmultiple objects are in col-laboration to solve a problem [55] Moreover our hierarchyof call exception is not necessarily limited to a single class butrather distributed across several classes As a consequence weneed to have access to all the relevant modules so that fromthe callee class we can determine the actual location fromwhere an exception has been thrown and caught Withmodular approach it cannot be determined Hence in ourapproach we have considered the whole program that is

a nonmodular analysis technique We admit that modularapproach is faster but the nonmodular analysis at the use caselevel is quite practical and usually followed in object-orientedsoftware engineering development environment [56]

Time Complexity The running time complexities of bothJumble and our approach are 119874(119899) where 119899 is the numberof exceptional handling constructs in a program

6 Related Work and Comparison

Gallagher andNarasimhan [8] proposed an approach for test-ing exception handling constructs They proposed structuralanalysis to select the effective test cases from a given set of testcases to test exception handling constructs in Java programsIn their structural analysis the whole structure of a programunder test is analyzed to retrieve all possible execution pathsThey define a fitness function tomeasure the performances of


test cases All test cases are executed and then evaluated oneby one and the high quality test cases are selected based ontheir fitness measures

Ji et al [15] proposed four mutation operators for Javaexception handling constructs They aimed to develop effec-tivemutant programs based on thosemutation operatorsThemajor difference with the work of Ji et al [15] and our workis that they proposed the mutation operators to test whetherthe Java exceptional handling constructs will be able to catchthe exceptions or not whereas objective of our work is to testthe adequacy of test suites through mutation analysis on Javaexceptions Further in Ji et al [15] work they consideredexceptions which are raised implicitly that is calling Javalibrary routines On the other hand in addition to implicitraise of exceptions we consider the generation of exceptionsexplicitly using arithmetic and logical control statementsThat is why we consider many more conventional mutationoperators other than the mutation operators as proposed byJi et al [15] Moreover to ensure that there is a sufficientnumber of exceptions raised we have given extra effort todevelop test cases accordingly which is grossly ignored in thework of Ji et al [15] More significantly Ji et al [15] adopteda trivial experiment with JUnit to test the effectiveness oftheir proposedmutation operators that is whether their pro-posed mutation operators really capable of causing mutantprograms or not It may be noted that JUnit tests a programat a unit level (ie one class at a time) which in fact can catchexceptions raised by the methods in that class only But ingeneral an exception can be raised from method(s) locatednot necessarily in the same class it is actually caught Indeedit is a difficult task to catch the exceptions unless we have afull tracking of all paths of exceptions from the point of causeof exception to the caller routine (and hence main class)This requires entire program to be taken into account ratherthan only a particular unit Furthermore in our approachemphasize has been given to locate the cause of exception atany point on a path of execution

Sinha and Harrold [9] described a class of test adequacycriteria that can be used to test the behavior of exceptionhandling constructs They present a subsumption hierarchyof the criteria and point out the relationship of the criteria tothose found in traditional subsumption hierarchiesThey alsoproposed techniques for generating the testing requirementsfor the criteria using some control flow representations

Buhr [21] proposed some exception handling mecha-nisms in the context of developing rich language features fordeveloping robust programs They also discussed the excep-tion propagation mechanism and proposed propagationmodels such as static dynamic and propagation

Sinha andHarrold [4] extended their work reported in [9]to analyze and test programs with exception handling con-structs They proposed techniques to construct control-flowrepresentations for Java programs with explicit exceptionsoccurrencesThey used these representation to perform someanalysis such as static and dynamic slicing structural andregression testing and dynamic execution profiling

Tracey et al [6] proposed an optimization technique forautomatically generating test data to test exceptions Theyconsidered Ada as the model language in their approachThe

proposed approach is based on the application of a dynamicglobal optimization based search for the required test data

Adamopoulos et al [57] reported on the cause of equiv-alent mutants Offutt and Pan [58] proposed a technique todetect equivalent mutants and infeasible paths

Andrews et al [14] made a thorough investigation of afundamental assumption that has been underlying much ofexperimental software testing research that whether faultsgenerated by hand or from mutation operators is represen-tative of real faults They observed that the use of mutationoperators is able to yield trustworthy results that is generatedmutants are similar to real faults Mutants appear however tobe different from hand seeded faults that seem to be harder todetect than real faults

Ma et al [27] introduced a number of mutation operatorsto create mutant programs in the context of Java program-ming language They propose a tool for entire mutation pro-cess and testing [28] The mutant operators proposed in [27]only modify expressions by replacing deleting and insertingprimitive features of Java programming language at method-level and object-oriented specific features such as inheritancepolymorphisms and dynamic binding at class-level How-ever Ma et al did not consider any mutation operators tohandle exceptions related faults

Work of Gallagher and Narasimhan [8] and Sinha andHarrold [4] followed static analysis and their approachesnot necessarily locate all exception points particularly thoseexceptions raised implicitly For example an exception can beraised implicitly from calling Java library routine As a conse-quence it is very difficult to draw control flow representationsor path identification to cover all problems Hence test caseselection according to these approach may not be adequateThe proposedmethod on the other hand does not determineexception paths explicitly We attempt to change exceptionpropagation path according to the characteristic of Javaexception handling constructs

Jumble provides mutation operators for arithmeticexpression conditional predicate initialization assignmentreturn values and control constructs such as if-then if-then-else switch and loop It is to be noted that the mutationoperators according to Jumble are not sufficient enough forexception handling constructs

7 Conclusions

Exception handling is an important mechanism to developrobust and reliable programs It is a difficult task to testwhether all exception points are covered and all exceptionhandler routines are arranged appropriately or not It is alsonot possible to identify all exception paths precisely becausethere are some exceptions which can be raised implicitlyThispaper proposes a number of mutation operators to createmutant programs With these mutant programs we verifythe effectiveness of a given test suite particularly to verifythe exception handling constructs In other words mutationoperators used in this paper can generate effective mutantprograms which can raise and handle different type of excep-tions The test suite which can kill more mutant programs iswith a higher mutation score Effectiveness of the proposed


mutation operators is comparable to that of Jumble Jumblealso reports the quantity ofmutants killed as the score of a testsuite Jumble provides mutations operators which are mainlylimited to arithmetic expressions conditions return valuesand control statements such as decision and loop The mainlimitation in Jumble is that there is no explicit mutation oper-ators in Jumble that can take care exception handling con-structs Our experiments with five different programs sub-stantiate that the proposed mutation operators are 69 (onaverage mutation score in Jumble and our approach is 27 and87 resp) more effective to build mutants and then kill themutant programs than the mutation analysis technique inJumble Nevertheless this result is based on experimentswithmoderate sized Java programs from different applicationdomains and can be taken as indicative To have conclusivedecision we should go for more experiments with moresubjects

Using the proposed mutation operators and mutationanalysis method our work can be further extended toguide automatic synthesis of test cases for testing exceptionhandling constructs Further themutation operators are pro-posed considering Java as themodel languageWe can extendthe approach for other advanced programming languages

Conflict of Interests

The author declares that there is no conflict of interestsregarding the publication of this paper

Acknowledgments

This work was supported by Department of Science andTechnology Government of India under Women ScientistAward (WOS-A) granted to the authorThe author is thankfulto the postgraduate students who helped in designing andimplementing the applications used in this work

References

[1] J B Goodenough ldquoException handling issues and proposednotationsrdquoCommunications of the ACM vol 18 no 12 pp 683ndash696 1975

[2] J D Musa Software Reliability Engineering McGraw-HillLondon UK 1998

[3] W N Toy ldquoFault tolerant design of local ESS processorrdquo inTheTheory and Practice of Reliable System Design 1981

[4] S Sinha and M J Harrold ldquoAnalysis and testing of programswith exception handling constructsrdquo IEEE Transactions onSoftware Engineering vol 26 no 9 pp 849ndash871 2000

[5] B G Ryder D Smith U Kremer M Gordon and N Shah ldquoAstatic study of java exceptions using JSEPrdquo Tech Rep DCS-TR-403 Rutgers University November 1999

[6] N Tracey J Clark K Mander and J McDermid ldquoAutomatedtest-data generation for exception conditionsrdquo Software Prac-tice and Experience vol 30 pp 61ndash79 2000

[7] A Bertolino ldquoSoftware testing research achievements chal-lenges dreamsrdquo in Proceedings of the Future of Software Engi-neering (FoSE rsquo07) pp 85ndash103 May 2007

[8] M J Gallagher and V L Narasimhan ldquoAdtest a test datageneration suite for ada software systemsrdquo IEEE Transactionson Software Engineering vol 23 no 8 pp 473ndash484 1997

[9] S Sinha and M J Harrold ldquoCriteria for testing exception-handling constructs in Java programsrdquo in Proceedings of theIEEE International Conference on Software Maintenance (ICSMrsquo99) pp 265ndash275 September 1999

[10] M P Robillard and G C Murphy ldquoAnalyzing exception flowin java TM programsrdquo in Proceedings of the Joint 7 th EuropeanSoftware Engineering Conference and the 7 th ACM SIGSOFTInternation al Symposium on the Foundations of Software Engi-neering (LNCS rsquo99) vol 1687 pp 322ndash337 Toulouse FranceSeptembe 1999

[11] J-D Choi D Grove M Hind and V Sarkar ldquoEfficient and pre-cisemodeling of exceptions for the analysis of Java programsrdquo inProceedings of the ACM SIGPLAN-SIGSOFT Workshop on Pro-gram Analysis for Software Tools and Engineering (PASTE rsquo99)pp 21ndash31 1999

[12] T A Budd and D Angluin ldquoTwo notions of correctness andtheir relation to testingrdquo Acta Informatica vol 18 no 1 pp 31ndash45 1982

[13] S-W Kim J A Clark and J A McDermid ldquoInvestigatingthe effectiveness of object-oriented testing strategies using themutation methodrdquo Software Testing Verification and Reliabilityvol 11 no 4 pp 207ndash225 2001

[14] J H Andrews L C Briand and Y Labiche ldquoIs mutation anappropriate tool for testing experimentsrdquo in Proceedings of the27th International Conference on Software Engineering (ICSErsquo05) pp 402ndash411 IEEE May 2005

[15] C Ji Z Chen B Xu and Z Wang ldquoA new mutation analysismethod for testing java exception handlingrdquo in Proceedings ofthe 33rd Annual IEEE International Computer Software andApplications Conference (COMPSAC rsquo09) vol 2 pp 556ndash561IEEE Computer Society Press July 2009

[16] httpjunitorg[17] httpwwwjumblesourceforgenet[18] httpwwwjavacom[19] D Samanta Object-Oriented Programming with C++ and Java

Prentice Hall of India New Delhi India 2003[20] J Gosling B Joy and G SteeleThe Java Language Specification

Addison-Wesley Reading Mass USA 1996[21] P A Buhr ldquoAdvanced exception handling mechanismsrdquo IEEE

Transactions on Software Engineering vol 26 no 9 pp 820ndash836 2000

[22] G J Myers The Art of Software Testing John Wiley amp SonsHoboken NJ USA 2nd edition 2004

[23] R A DeMillo R J Lipton and F G Sayward ldquoHints ontest data selection help for the practicing programmerrdquo IEEEComputer vol 11 no 4 pp 34ndash41 1978

[24] M E Delamaro J C Maldonado and A P Mathur ldquoInterfacemutation an approach for integration testingrdquo IEEE Transac-tions on Software Engineering vol 27 no 3 pp 228ndash247 2001

[25] J B Rainsberger and S Stirling JUnit Recipes PracticalMethodsfor Programmer Testing Manning Greenwich UK 2005

[26] S A Irvine T Pavlinic L Trigg J G Cleary S Inglis andM Utting ldquoJumble java byte code to measure the effectivenessof unit testsrdquo in Proceedings of the IEEE Proceedings of TestingAcademic and Industrial ConferencemdashPractice and ResearchTechniques pp 169ndash175 September 2007

[27] Y-S Ma J Offutt and Y-R Kwon ldquoMuJava a mutation systemfor javardquo in Proceedings of the 28th International Conference onSoftware Engineering (ICSE rsquo06) pp 827ndash830 ACM May 2006


[28] httpcsgmuedusimoffuttmujava[29] K Dobolyi andWWeimer ldquoChanging javarsquos semantics for han-

dling null pointer exceptionsrdquo in Proceedings of the 19th Inter-national Symposium on Software Reliability Engineering (ISSRErsquo08) pp 47ndash56 November 2008

[30] L Briand and Y Labiche ldquoA UML-based approach to systemtestingrdquo Tech Rep TR SCE-01-01 Version 4 Carleton Univer-sity 2002

[31] A Abdurazik J Offutt and A Baldini ldquoA comparative evalua-tion of tests generated from different UML diagrams diagramsand datardquo Tech Rep ISE-TR-05-04 George Mason UniversityFairfax Va USA 2005

[32] M SarmaAutomatic test specification generation for state-basedsystem testing [PhD thesis] Indian Institute of TechnologyKharagpur India 2008

[33] M Sarma ldquoSystem statemodel generation fromUML20rdquo TechRep CSE-TR-04-07 Indian Institute of Technology KharagpurIndia April 2007

[34] L C Briand J Cui and Y Labiche ldquoTowards automatedsupport for deriving test data from UML statechartsrdquo inProceedings of the Unified Modeling Language Conference (UMLrsquo03) vol 2863 of Lecture Notes in Computer Science pp 249ndash264 Springer San Francisco Calif USA October 2003

[35] S C Ntafos ldquoA comparison of some structural testing strate-giesrdquo IEEE Transactions on Software Engineering vol 14 no 6pp 868ndash874 1988

[36] J CHuang ldquoAn approach to program testingrdquoACMComputingSurveys vol 7 no 3 pp 114ndash128 1975

[37] P G Frankl and E JWeyuker ldquoAn applicable family of data flowtesting criteriardquo IEEE Transactions on Software Engineering vol14 no 10 pp 1483ndash1498 1988

[38] M J Harrold and M L Soffa ldquoInterprocedural data flowtestingrdquo in Proceedings of the ACM SIGSOFT of 3rd Symposiumof Software Testing Analysis and Verification (SIGSOFT rsquo89) pp158ndash167 December 1989

[39] J W Laski and B Korel ldquoA data flow oriented program testingstrategyrdquo IEEE Transactions on Software Engineering vol 9 no3 pp 347ndash354 1983

[40] S Rapps and E J Weyuker ldquoSelecting software test datausing data flow informationrdquo IEEE Transactions on SoftwareEngineering vol 11 no 4 pp 367ndash375 1985

[41] P G Frankl and S N Weiss ldquoExperimental comparison ofthe effectiveness of branch testing and data flow testingrdquo IEEETransactions on Software Engineering vol 19 no 8 pp 774ndash7871993

[42] W EHowden ldquoMethodology for the generation of program testdatardquo IEEE Transactions on Computers vol 24 no 5 pp 554ndash560 1975

[43] T J McCabe ldquoA complexity measurerdquo IEEE Transactions onSoftware Engineering vol 2 no 4 pp 308ndash320 1976

[44] P G Frankl and E J Weyuker ldquoProvable improvements onbranch testingrdquo IEEE Transactions on Software Engineering vol19 no 10 pp 962ndash975 1993

[45] B Beizer Software Testing Techniques Thomson ComputerPress New York NY USA 2nd edition 1990

[46] R Boyer B Elspas and K Levitt ldquoSELECTmdasha formal systemfor testing and debugging programs by symbolic executionrdquo inProceedings of International Conference on Reliable Software pp234ndash245 1975

[47] L A Clarke ldquoA system to generate test data and symbolicallyexecute programsrdquo IEEE Transactions on Software Engineeringvol 2 no 3 pp 215ndash222 1976

[48] J C King ldquoSymbolic execution and program testingrdquo Commu-nications of the Association for ComputingMachinery vol 19 no7 pp 385ndash394 1976

[49] C V Ramamoorthy S-B F Ho and W T Chen ldquoOn theautomated generation of program test datardquo IEEE Transactionson Software Engineering vol 2 no 4 pp 293ndash300 1976

[50] R A DeMillo and A J Offutt ldquoExperimental results from anautomatic test case generatorrdquo ACM Transactions on SoftwareEngineering and Methodology vol 2 no 2 pp 109ndash127 1993

[51] B Korel ldquoAutomated software test data generationrdquo IEEETrans-actions on Software Engineering vol 16 no 8 pp 870ndash879 1990

[52] B F Jones H-H Sthamer and D E Eyres ldquoAutomatic struc-tural testing using genetic algorithmsrdquo Software EngineeringJournal vol 11 no 5 pp 299ndash306 1996

[53] R A DeMillo and A J Offutt ldquoConstraint-based automatic testdata generationrdquo IEEE Transactions on Software Engineeringvol 17 no 9 pp 900ndash910 1991

[54] A J Offutt Z Jin and J Pan ldquoThe dynamic domain reductionprocedure for test data generationrdquo SoftwaremdashPractice andExperience vol 29 no 2 pp 167ndash193 1999

[55] R V Binder Testing Object Oriented Systems Models Patternsand Tools The Object Technology Addison-Wesley 1999

[56] J Z Gao H-S J Tsao and YWu Testing and Quality Assurancefor Component-Based Software Artech House Norwood MassUSA 2003

[57] K Adamopoulos M Harman and R M Hierons ldquoHow toovercome the equivalent mutant problem and achieve tailoredselective mutation using co-evolutionrdquo in Proceedings of theGenetic and Evolutionary Computation Conference (LNCS rsquo04)vol 3103 pp 1338ndash1349 Seattle Wash USA June 2004

[58] A J Offutt and J Pan ldquoAutomatically detecting equivalentmutants and infeasible pathsrdquo Software Testing Verification andReliability vol 7 no 3 pp 165ndash192 1997

Submit your manuscripts athttpwwwhindawicom

Computer Games Technology

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014


Distributed Sensor Networks


Advances in

FuzzySystems

Hindawi Publishing Corporationhttpwwwhindawicom

Volume 2014


ReconfigurableComputing

Hindawi Publishing Corporation httpwwwhindawicom Volume 2014


Applied Computational Intelligence and Soft Computing

thinspAdvancesthinspinthinsp

Artificial Intelligence

HindawithinspPublishingthinspCorporationhttpwwwhindawicom Volumethinsp2014

Advances inSoftware EngineeringHindawi Publishing Corporationhttpwwwhindawicom Volume 2014


Electrical and Computer Engineering

Journal of

Journal of

Computer Networks and Communications


Hindawi Publishing Corporation

httpwwwhindawicom Volume 2014

Advances in

Multimedia


Biomedical Imaging


ArtificialNeural Systems

Advances in


RoboticsJournal of



Computational Intelligence and Neuroscience

Industrial EngineeringJournal of


Modelling amp Simulation in EngineeringHindawi Publishing Corporation httpwwwhindawicom Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014


Human-ComputerInteraction

Advances in

Computer EngineeringAdvances in



the criteria to unit and integration testing of programs thatcontain exception handling constructs However to the bestof our literature survey no approach has been reported so farto generate exception handling related faulty programs andthen to test the adequacy of test suites Adequacy testing is animportant issue because a test suite may not locate all excep-tion points For example in Java an exception can be raisedimplicitly In that case it is not possible to drawproper controlflow graph [4 10 11] Further even if all exception paths areknown from the control flow graph [4] and satisfy exception-coverage criteria [9] it needs to design test data manually forraising all possible potential exceptionsThismanual test casegeneration is not only tedious [6] but raising exceptions is notalways guaranteed As a consequence we need to determinethe adequacy of test cases to raise exceptions In generaldetermining the path of exceptions is a difficult problem andmanual synthesis of test data may not be sufficient to raiseall exceptions In this context efficiency of a test suite isnecessary to be evaluated in order to assure the reliability of asystemunder test In summary given a programwe are to testwhether the exception handling constructs are sufficient tothrow and catch all possible exceptions or not

Mutation analysis is a practical method to evaluate thequality of test suites [12ndash14] In this work we propose amutation analysis method to evaluate test suites for testingexception handling constructs We consider a set of mutationoperators with which different types of faults can be injectedto the original programsThemutation operators are adoptedfrom the work of Ji et al [15] The mutant programs arethen analyzed with some test suites We consider test suitesgenerated according to the control flow based structuralanalysis approach proposed in [4 9] for a given Java programWe also analyzed the program in JUnit 46 [16] and Jumble 10[17] testing environment and their mutant programs Ourexperimental results substantiate the efficiency of the pro-posed mutation analysis approach

The rest of the paper is organized as follows We discussbasic concepts related terminologies and tools in Section 2In Section 3 we discuss the mutation operators followed inthis work Mutation analysis is discussed in Section 4 Ourexperimental results are presented in Section 5 Related workand comparison of our work with others are discussed inSection 6 Finally Section 7 concludes the paper

2 Basic Concepts

In this section we discuss some basic concepts and termi-nologies which are referred to in our subsequent discussionsWe also briefly introduce the tools used in our work

21 Exception Handling Constructs in Java Java [18] pro-gramming language is considered as a model language in thiswork In this subsection we provide an overview of exceptionhandling constructs in Java Details of exception handlingconstructs in Java can be found in [19 20]

In Java exceptions are first-class objects Each exceptionis an instance of a class that is derived from the classjavalangThrowable which is defined in the stan-dard Java API [19] An exception can be raised at any

point in the program through a throw statement A throwstatement is associated with an expression denoting anexception object which may be a variable (eg throw e)a method call (eg throw m()) or a new instantiation(eg throw new E()) and so forth A throw statement canappear anywhere in the program Java exception handlingconstruct includes three blocks try catch and finally(see Figure 1(a)) Code can be guarded for exceptionswithin a try block Exceptions raised through execution ofcode (implicitly) or a throw statement (explicitly) withina try block may be caught in one or more catch clause(s)declared immediately following the try block The finallyblock is executed independently of what happens in thetry block It may be noted that a try statement consists ofa try block and optionally one or more catch blocks andfinally block (see Figure 1(a)) The valid instances of atry statement are try catch ( ) [catch ( ) ][ ] [finally ] where the blocks in square bracketsindicate optional Here a try block contains statementwhose execution is monitored for exception occurrencesA catch block specifies the type of exception it handlesand exception handler that is it contains a block of codethat is executed when an exception of that type is raised inthe associated try block If a finally block is associatedwith a try block then the finally block is always executedirrespective of how control transfers out of the try block

22 Java Exception Hierarchy Similar to other Java objectsexceptions are typed and types are organized into a hierarchyAll exceptions inherit from the class type Throwable definedin javalang API The exception type hierarchy definesthree different groups of exceptions errors runtime excep-tions and checked exception (defined in JavalangException) (also see Figure 1(b)) Note that errors andruntime exceptions are unchecked exceptions Uncheckedexceptions can be thrown at any point in a Java programand if uncaught may propagate back to the programentry point causing the Java Virtual Machine [20] toterminate The class Error is responsible for giving errorsin some catastrophic failures The exceptions of typeRuntimeException are typically created automaticallyduring runtime in response to some execution errors Theexceptions of the type Exception are checked exceptionsand are used for exceptional conditions that user programscan catch A list of exceptions that a Java programmer cancatch in programs is available in [19] There are 11 runtimeexceptions 17 errors and 7 checked exceptions defined inJavalangAPI [19] as per the release of Java 6 Programmerscan define and extend existing exception type in Java API tocustomize their needs Some new exception types can alsobe defined [20]

23 Call Exception Hierarchy In this subsection we discussthe flow of exceptions within the context of the structure ofa given programThis flow of exceptions we specially term itas call exception hierarchy (CEH) is useful to understand andevaluate how exceptions are handled within a method whichexceptions might arise during the execution of a method


Guarded block

where exception can

be raised





try

finally








JavalangObject

JavalangThrowable








119895and


2is a









2() in turns





11and 119864

12are 119864rsquos





21and 119864

22are supposed to






11and 119864

12 11986411and






1 1198642 and 119864

3 Further the







and 11986412



1 1198642 and


(i)

Main







(iii)

(ii)

(iv)

M1



try

try





catch E1

catch E





throw E21

throw E11

throw E12



throw E22

E

E11E1

E21 E22

E12



M1( )

M2( )


(i)

(ii)

(iii)

Main












E

E11

E1

E12


throw E12


try

try

catch E1

catch E

M1( )






(i)

M1






(ii)

E1E2E3

E11 E12

(iii)

try



throw E12


M1( )



















Inputprogram P



Modify testsuite

Fix program

No




Run testcases on P

mutants 120601(P)

120601(P)








120583 (119875 119879) =DM

TM minus EM (1)






















BD


try try












E

E2E1

E12E23














1 1198621198612 119862119861


parameter types 1198641 1198642 119864

119899where 119864

1 1198642 119864

119899are the





Class-level




119894]catch(119864

119894)


119894for each 119864

119894isin






119894

for all 119864119894isin 119862119864119867




119894for each 119864

119894isin

119864 1198641 1198642 119864













E1 E2

E21 E22

E


try

try















E1 E2

E11 E12

E

throw elowasti


try

try



















Method-level







Class-level













1to 11987510158401






4 Mutation Analysis











Variable say x



TA

(i)

P9984001P1
























(B)




try






































1isin 120601(119875) and 119876





180

160

140

120

100

80

60

40

20


JumbleOur approach

Num

ber o

f mut

ants

(a) Mutant programs

140

120

100

80

60

40

20

0

LIS TAS CPS CMS VAS

JumbleOur approach

Mut

ant k

illed

(b) Mutants killed

180

160

140

120

100

80

60

40

20

0

LIS TAS CPS CMS VAS

JumbleOur approach

Mut

atio

n po

ints

(c) Mutation points

Mut

atio

n sc

ore

100

90

80

70

60

50

40

30

20

10

0

LIS TAS CPS CMS VAS

JumbleOur approach

(d) Mutation scores





















7 Conclusions







Acknowledgments


References

































































Advances in

FuzzySystems


Volume 2014












Journal of

Journal of





Advances in

Multimedia


Biomedical Imaging



Advances in


RoboticsJournal of










Advances in




Guarded block

where exception can

be raised





try

finally








JavalangObject

JavalangThrowable








119895and


2is a









2() in turns





11and 119864

12are 119864rsquos





21and 119864

22are supposed to






11and 119864

12 11986411and






1 1198642 and 119864

3 Further the







and 11986412



1 1198642 and


(i)

Main







(iii)

(ii)

(iv)

M1



try

try





catch E1

catch E





throw E21

throw E11

throw E12



throw E22

E

E11E1

E21 E22

E12



M1( )

M2( )


(i)

(ii)

(iii)

Main












E

E11

E1

E12


throw E12


try

try

catch E1

catch E

M1( )






(i)

M1






(ii)

E1E2E3

E11 E12

(iii)

try



throw E12


M1( )



















Inputprogram P



Modify testsuite

Fix program

No




Run testcases on P

mutants 120601(P)

120601(P)








120583 (119875 119879) =DM

TM minus EM (1)






















BD


try try












E

E2E1

E12E23














1 1198621198612 119862119861


parameter types 1198641 1198642 119864

119899where 119864

1 1198642 119864

119899are the





Class-level




119894]catch(119864

119894)


119894for each 119864

119894isin






119894

for all 119864119894isin 119862119864119867




119894for each 119864

119894isin

119864 1198641 1198642 119864













E1 E2

E21 E22

E


try

try















E1 E2

E11 E12

E

throw elowasti


try

try



















Method-level







Class-level













1to 11987510158401






4 Mutation Analysis











Variable say x



TA

(i)

P9984001P1
























(B)




try






































1isin 120601(119875) and 119876





180

160

140

120

100

80

60

40

20


JumbleOur approach

Num

ber o

f mut

ants

(a) Mutant programs

140

120

100

80

60

40

20

0

LIS TAS CPS CMS VAS

JumbleOur approach

Mut

ant k

illed

(b) Mutants killed

180

160

140

120

100

80

60

40

20

0

LIS TAS CPS CMS VAS

JumbleOur approach

Mut

atio

n po

ints

(c) Mutation points

Mut

atio

n sc

ore

100

90

80

70

60

50

40

30

20

10

0

LIS TAS CPS CMS VAS

JumbleOur approach

(d) Mutation scores





















7 Conclusions







Acknowledgments


References

































































Advances in

FuzzySystems


Volume 2014












Journal of

Journal of





Advances in

Multimedia


Biomedical Imaging



Advances in


RoboticsJournal of










Advances in




(i)

Main







(iii)

(ii)

(iv)

M1



try

try





catch E1

catch E





throw E21

throw E11

throw E12



throw E22

E

E11E1

E21 E22

E12



M1( )

M2( )


(i)

(ii)

(iii)

Main












E

E11

E1

E12


throw E12


try

try

catch E1

catch E

M1( )






(i)

M1






(ii)

E1E2E3

E11 E12

(iii)

try



throw E12


M1( )



















Inputprogram P



Modify testsuite

Fix program

No




Run testcases on P

mutants 120601(P)

120601(P)








120583 (119875 119879) =DM

TM minus EM (1)






















BD


try try












E

E2E1

E12E23














1 1198621198612 119862119861


parameter types 1198641 1198642 119864

119899where 119864

1 1198642 119864

119899are the





Class-level




119894]catch(119864

119894)


119894for each 119864

119894isin






119894

for all 119864119894isin 119862119864119867




119894for each 119864

119894isin

119864 1198641 1198642 119864













E1 E2

E21 E22

E


try

try















E1 E2

E11 E12

E

throw elowasti


try

try



















Method-level







Class-level













1to 11987510158401






4 Mutation Analysis











Variable say x



TA

(i)

P9984001P1
























(B)




try






































1isin 120601(119875) and 119876





180

160

140

120

100

80

60

40

20


JumbleOur approach

Num

ber o

f mut

ants

(a) Mutant programs

140

120

100

80

60

40

20

0

LIS TAS CPS CMS VAS

JumbleOur approach

Mut

ant k

illed

(b) Mutants killed

180

160

140

120

100

80

60

40

20

0

LIS TAS CPS CMS VAS

JumbleOur approach

Mut

atio

n po

ints

(c) Mutation points

Mut

atio

n sc

ore

100

90

80

70

60

50

40

30

20

10

0

LIS TAS CPS CMS VAS

JumbleOur approach

(d) Mutation scores





















7 Conclusions







Acknowledgments


References

































































Advances in

FuzzySystems


Volume 2014












Journal of

Journal of





Advances in

Multimedia


Biomedical Imaging



Advances in


RoboticsJournal of










Advances in




Inputprogram P



Modify testsuite

Fix program

No




Run testcases on P

mutants 120601(P)

120601(P)








120583 (119875 119879) =DM

TM minus EM (1)






















BD


try try












E

E2E1

E12E23














1 1198621198612 119862119861


parameter types 1198641 1198642 119864

119899where 119864

1 1198642 119864

119899are the





Class-level




119894]catch(119864

119894)


119894for each 119864

119894isin






119894

for all 119864119894isin 119862119864119867




119894for each 119864

119894isin

119864 1198641 1198642 119864













E1 E2

E21 E22

E


try

try















E1 E2

E11 E12

E

throw elowasti


try

try



















Method-level







Class-level













1to 11987510158401






4 Mutation Analysis











Variable say x



TA

(i)

P9984001P1
























(B)




try






































1isin 120601(119875) and 119876





180

160

140

120

100

80

60

40

20


JumbleOur approach

Num

ber o

f mut

ants

(a) Mutant programs

140

120

100

80

60

40

20

0

LIS TAS CPS CMS VAS

JumbleOur approach

Mut

ant k

illed

(b) Mutants killed

180

160

140

120

100

80

60

40

20

0

LIS TAS CPS CMS VAS

JumbleOur approach

Mut

atio

n po

ints

(c) Mutation points

Mut

atio

n sc

ore

100

90

80

70

60

50

40

30

20

10

0

LIS TAS CPS CMS VAS

JumbleOur approach

(d) Mutation scores





















7 Conclusions







Acknowledgments


References

































































Advances in

FuzzySystems


Volume 2014












Journal of

Journal of





Advances in

Multimedia


Biomedical Imaging



Advances in


RoboticsJournal of










Advances in




















BD


try try












E

E2E1

E12E23














1 1198621198612 119862119861


parameter types 1198641 1198642 119864

119899where 119864

1 1198642 119864

119899are the





Class-level




119894]catch(119864

119894)


119894for each 119864

119894isin






119894

for all 119864119894isin 119862119864119867




119894for each 119864

119894isin

119864 1198641 1198642 119864













E1 E2

E21 E22

E


try

try















E1 E2

E11 E12

E

throw elowasti


try

try



















Method-level







Class-level













1to 11987510158401






4 Mutation Analysis











Variable say x



TA

(i)

P9984001P1
























(B)




try






































1isin 120601(119875) and 119876





180

160

140

120

100

80

60

40

20


JumbleOur approach

Num

ber o

f mut

ants

(a) Mutant programs

140

120

100

80

60

40

20

0

LIS TAS CPS CMS VAS

JumbleOur approach

Mut

ant k

illed

(b) Mutants killed

180

160

140

120

100

80

60

40

20

0

LIS TAS CPS CMS VAS

JumbleOur approach

Mut

atio

n po

ints

(c) Mutation points

Mut

atio

n sc

ore

100

90

80

70

60

50

40

30

20

10

0

LIS TAS CPS CMS VAS

JumbleOur approach

(d) Mutation scores





















7 Conclusions







Acknowledgments


References

































































Advances in

FuzzySystems


Volume 2014












Journal of

Journal of





Advances in

Multimedia


Biomedical Imaging



Advances in


RoboticsJournal of










Advances in







E

E2E1

E12E23














1 1198621198612 119862119861


parameter types 1198641 1198642 119864

119899where 119864

1 1198642 119864

119899are the





Class-level




119894]catch(119864

119894)


119894for each 119864

119894isin






119894

for all 119864119894isin 119862119864119867




119894for each 119864

119894isin

119864 1198641 1198642 119864













E1 E2

E21 E22

E


try

try















E1 E2

E11 E12

E

throw elowasti


try

try



















Method-level







Class-level













1to 11987510158401






4 Mutation Analysis











Variable say x



TA

(i)

P9984001P1
























(B)




try






































1isin 120601(119875) and 119876





180

160

140

120

100

80

60

40

20


JumbleOur approach

Num

ber o

f mut

ants

(a) Mutant programs

140

120

100

80

60

40

20

0

LIS TAS CPS CMS VAS

JumbleOur approach

Mut

ant k

illed

(b) Mutants killed

180

160

140

120

100

80

60

40

20

0

LIS TAS CPS CMS VAS

JumbleOur approach

Mut

atio

n po

ints

(c) Mutation points

Mut

atio

n sc

ore

100

90

80

70

60

50

40

30

20

10

0

LIS TAS CPS CMS VAS

JumbleOur approach

(d) Mutation scores





















7 Conclusions







Acknowledgments


References

































































Advances in

FuzzySystems


Volume 2014












Journal of

Journal of





Advances in

Multimedia


Biomedical Imaging



Advances in


RoboticsJournal of










Advances in











E1 E2

E21 E22

E


try

try















E1 E2

E11 E12

E

throw elowasti


try

try



















Method-level







Class-level













1to 11987510158401






4 Mutation Analysis











Variable say x



TA

(i)

P9984001P1
























(B)




try






































1isin 120601(119875) and 119876





180

160

140

120

100

80

60

40

20


JumbleOur approach

Num

ber o

f mut

ants

(a) Mutant programs

140

120

100

80

60

40

20

0

LIS TAS CPS CMS VAS

JumbleOur approach

Mut

ant k

illed

(b) Mutants killed

180

160

140

120

100

80

60

40

20

0

LIS TAS CPS CMS VAS

JumbleOur approach

Mut

atio

n po

ints

(c) Mutation points

Mut

atio

n sc

ore

100

90

80

70

60

50

40

30

20

10

0

LIS TAS CPS CMS VAS

JumbleOur approach

(d) Mutation scores





















7 Conclusions







Acknowledgments


References

































































Advances in

FuzzySystems


Volume 2014












Journal of

Journal of





Advances in

Multimedia


Biomedical Imaging



Advances in


RoboticsJournal of










Advances in






Method-level







Class-level













1to 11987510158401






4 Mutation Analysis











Variable say x



TA

(i)

P9984001P1
























(B)




try






































1isin 120601(119875) and 119876





180

160

140

120

100

80

60

40

20


JumbleOur approach

Num

ber o

f mut

ants

(a) Mutant programs

140

120

100

80

60

40

20

0

LIS TAS CPS CMS VAS

JumbleOur approach

Mut

ant k

illed

(b) Mutants killed

180

160

140

120

100

80

60

40

20

0

LIS TAS CPS CMS VAS

JumbleOur approach

Mut

atio

n po

ints

(c) Mutation points

Mut

atio

n sc

ore

100

90

80

70

60

50

40

30

20

10

0

LIS TAS CPS CMS VAS

JumbleOur approach

(d) Mutation scores





















7 Conclusions







Acknowledgments


References

































































Advances in

FuzzySystems


Volume 2014












Journal of

Journal of





Advances in

Multimedia


Biomedical Imaging



Advances in


RoboticsJournal of










Advances in




Variable say x



TA

(i)

P9984001P1
























(B)




try






































1isin 120601(119875) and 119876





180

160

140

120

100

80

60

40

20


JumbleOur approach

Num

ber o

f mut

ants

(a) Mutant programs

140

120

100

80

60

40

20

0

LIS TAS CPS CMS VAS

JumbleOur approach

Mut

ant k

illed

(b) Mutants killed

180

160

140

120

100

80

60

40

20

0

LIS TAS CPS CMS VAS

JumbleOur approach

Mut

atio

n po

ints

(c) Mutation points

Mut

atio

n sc

ore

100

90

80

70

60

50

40

30

20

10

0

LIS TAS CPS CMS VAS

JumbleOur approach

(d) Mutation scores





















7 Conclusions







Acknowledgments


References

































































Advances in

FuzzySystems


Volume 2014












Journal of

Journal of





Advances in

Multimedia


Biomedical Imaging



Advances in


RoboticsJournal of










Advances in

































1isin 120601(119875) and 119876





180

160

140

120

100

80

60

40

20


JumbleOur approach

Num

ber o

f mut

ants

(a) Mutant programs

140

120

100

80

60

40

20

0

LIS TAS CPS CMS VAS

JumbleOur approach

Mut

ant k

illed

(b) Mutants killed

180

160

140

120

100

80

60

40

20

0

LIS TAS CPS CMS VAS

JumbleOur approach

Mut

atio

n po

ints

(c) Mutation points

Mut

atio

n sc

ore

100

90

80

70

60

50

40

30

20

10

0

LIS TAS CPS CMS VAS

JumbleOur approach

(d) Mutation scores





















7 Conclusions







Acknowledgments


References

































































Advances in

FuzzySystems


Volume 2014












Journal of

Journal of





Advances in

Multimedia


Biomedical Imaging



Advances in


RoboticsJournal of










Advances in




180

160

140

120

100

80

60

40

20


JumbleOur approach

Num

ber o

f mut

ants

(a) Mutant programs

140

120

100

80

60

40

20

0

LIS TAS CPS CMS VAS

JumbleOur approach

Mut

ant k

illed

(b) Mutants killed

180

160

140

120

100

80

60

40

20

0

LIS TAS CPS CMS VAS

JumbleOur approach

Mut

atio

n po

ints

(c) Mutation points

Mut

atio

n sc

ore

100

90

80

70

60

50

40

30

20

10

0

LIS TAS CPS CMS VAS

JumbleOur approach

(d) Mutation scores





















7 Conclusions







Acknowledgments


References

































































Advances in

FuzzySystems


Volume 2014












Journal of

Journal of





Advances in

Multimedia


Biomedical Imaging



Advances in


RoboticsJournal of










Advances in
















7 Conclusions







Acknowledgments


References

































































Advances in

FuzzySystems


Volume 2014












Journal of

Journal of





Advances in

Multimedia


Biomedical Imaging



Advances in


RoboticsJournal of










Advances in








Acknowledgments


References

































































Advances in

FuzzySystems


Volume 2014












Journal of

Journal of





Advances in

Multimedia


Biomedical Imaging



Advances in


RoboticsJournal of










Advances in










































Advances in

FuzzySystems


Volume 2014












Journal of

Journal of





Advances in

Multimedia


Biomedical Imaging



Advances in


RoboticsJournal of










Advances in



research article mutation analysis approach to develop...

Documents