Effectiveness of Pair-wise Testing on Logical Expressions

Presented byWilliam Ballance

Overview

• Introduction• Fault Types• Fault Evaluator• Experiment Results

Introduction

• Effectiveness of pair-wise testing to reveal fault in logical expressions

• Only one previous paper that investigate this Non-specification-based approaches to logic testing for software

Fault Types

• VNF (Variable Negation Fault)• ORF (Operator Reference Fault)• VRF (Variable Reference Fault)• ENF (Expression Negation Fault)• ASF (Associative Shift Fault)

VNF

• Fault type where Boolean variable x is replaced by !x.

Original expression:a ∧ (!b ∨ !c) ∧ d ∨ e

Expression with VNF:!a ∧ (!b ∨ !c) ∧ d ∨ e

ORF

• Fault type where Boolean AND operator is incorrectly used instead of Boolean OR operator or vice versa.

Original expression:a ∧ (!b ∨ !c) ∧ d ∨ e

Expression with ORF:a ∨ (!b ∨ !c) ∧ d ∨ e

VRF

• Fault type where Boolean variable x is replaced by another variable y.

Original expression:a ∧ (!b ∨ !c) ∧ d ∨ e

Expression with VRF:b ∧ (!b ∨ !c) ∧ d ∨ e

ENF

• Fault type where Boolean expression p is replaced by !p.

Original expression:a ∧ (!b ∨ !c) ∧ d ∨ e

Expression with ENF:a ∧ !(!b ∨ !c) ∧ d ∨ e

ASF

• Fault type where parentheses in Boolean expression is misused.

Original expression:a ∧ (!b ∨ !c) ∧ d ∨ e

Expression with ENF:a ∧ !(b ∨ !c) ∧ d ∨ e

Fault Evaluator

• A software tool that can generate faulty Boolean expressions and evaluate those faulty expressions with test sets that are loaded in the software.

Fault Evaluator Steps

1. Load in the correct Boolean expressions2. Load in the test cases3. Generate faults (specify which faults or all faults)4. Evaluate– Fault Evaluator applies the test sets to the original

expressions and the faulty expressions.– If the result from applying the test case to the faulty

expression was different from the result obtained from the original expression, then the fault was revealed.

Comparison of Our Results and Kobayashi’s Results

Number of Faults for each Expression Set

Effectiveness of Pair-wise testing on 5 different sets of expressions

• Percentages of pair-wise testing to reveal faults for each expression set.

Comparison of Pair-wise testing and random testing

• Percentages of Pair-wise and random testing

Pair-wise testing depending on size of expression

• Effectiveness of Pair-wise testing for each expression set

Pair-wise testing depending on size of expression

• Effectiveness of Pair-wise testing for each fault type