from symbolic execution to concolic testing - tu · pdf fileexecute the program in symbolic...
Post on 06-Feb-2018
231 Views
Preview:
TRANSCRIPT
From Symbolic Execution to
Concolic Testing
Daniel Paqué
Structure
Symbolic Execution
Concolic Testing
Execution Generated Testing
Concurrency in Concolic Testing
From Symbolic Execution to Concolic Testing 2 28.11.2014
Motivation
28.11.2014 From Symbolic Execution to Concolic Testing 3
Software Testing “usually accounts for 50% of software
development cost” [Source: “The economic impacts of inadequate infrastructure for software testing”, NIST]
complex and large Software Systems complicate finding
small test suites with high coverage
Symbolic Execution
automic test case generation
high code coverage
Symbolic Execution - Idea
28.11.2014 From Symbolic Execution to Concolic Testing 4
execute the program in symbolic domain
explore all possible execution paths
for each path the constraints of the branching points are
collected
generate test input based on the constraints
Symbolic Execution - Example
28.11.2014 From Symbolic Execution to Concolic Testing 5
Symbolic Execution - Example
28.11.2014 From Symbolic Execution to Concolic Testing 6
symbolic domain:
symbolic state path condition
Symbolic Execution - Example
28.11.2014 From Symbolic Execution to Concolic Testing 7
symbolic domain:
Symbolic Execution - Example
28.11.2014 From Symbolic Execution to Concolic Testing 8
symbolic domain:
Symbolic Execution - Example
28.11.2014 From Symbolic Execution to Concolic Testing 9
symbolic domain:
1
2
Symbolic Execution - Example
28.11.2014 From Symbolic Execution to Concolic Testing 10
1
2
3
symbolic domain:
Symbolic Execution - Example
28.11.2014 From Symbolic Execution to Concolic Testing 11
Limits of Symbolic Execution
28.11.2014 From Symbolic Execution to Concolic Testing 12
1
2
symbolic domain:
Solution
28.11.2014 From Symbolic Execution to Concolic Testing 13
Mix Symbolic Execution with Concrete Execution
Concolic Testing
Execution Generated Testing
28.11.2014 From Symbolic Execution to Concolic Testing 14
2005,
Godefroid, Sen
Symbolic Execution
Concolic Testing Execution Generated
Testing (EGT)
mix concrete with
symbolic execution
2006
Cadar et. al
1979
J.C. King
+ improvements in
constraint solving
Concolic Testing
28.11.2014 From Symbolic Execution to Concolic Testing 15
execute program with concrete values and collect
symbolic constraints during execution
explore paths sequentially instead of forking
infer input for next execution
use concrete values to solve problematic constraints
Concolic Testing - Example
28.11.2014 From Symbolic Execution to Concolic Testing 16
symbolic state:
Concolic Testing - Example
28.11.2014 From Symbolic Execution to Concolic Testing 17
symbolic state:
false
symbolic domain:
Concolic Testing - Example
28.11.2014 From Symbolic Execution to Concolic Testing 18
new input:
Concolic Testing - Example
28.11.2014 From Symbolic Execution to Concolic Testing 19
Concolic Testing - Example
28.11.2014 From Symbolic Execution to Concolic Testing 20
symbolic domain:
evaluate
condition in
concrete
true
Concolic Testing - Example
28.11.2014 From Symbolic Execution to Concolic Testing 21
symbolic state:
evaluate bar()
in concrete
28.11.2014 From Symbolic Execution to Concolic Testing 22
sequential path exploration
guided by concrete input
Symbolic Execution
Concolic Testing Execution Generated
Testing (EGT)
mix concrete with
symbolic execution
fork execution for each path
guided by symbolic execution
1979
J.C. King
Execution Generated Testing
28.11.2014 From Symbolic Execution to Concolic Testing 23
further differences to Concolic Testing:
EGT dynamically checks if all operands are concrete
if so the operation can be executed in concrete
elsewise the operation is executed symbolical
concrete
symbolic
How to deal with
concurrent programs?
28.11.2014 From Symbolic Execution to Concolic Testing 24
Main Challenge
28.11.2014 From Symbolic Execution to Concolic Testing 25
Thread 0 Thread 1
…
Problem
Main Challenge
28.11.2014 From Symbolic Execution to Concolic Testing 26
Thread 0 Thread 1
…
Idea
summarize redundant
interleavings
28.11.2014 From Symbolic Execution to Concolic Testing 27
Koushik Sen & Gul Agha:
(2006)
„race-detection and flipping algorithm“
minimize redundant executions in concurrent
programs
uses vector clocks to identify races
jCUTE
Redundant Executions
28.11.2014 From Symbolic Execution to Concolic Testing 28
Thread t0: Thread t1:
Execution 1:
{4, 0, 16}
Execution 2:
{4, 0, 16}
Execution 3:
{3, 0, 15}
Execution 4:
{3, 0, 16} result:
Redundant Executions – Race Detection
28.11.2014 From Symbolic Execution to Concolic Testing 29
two events are in a race if… they stem from different threads
both access the same memory location (without locking)
the order both events can be permuted by changing the schedule
Execution 1:
{4, 0, 16}
Execution 2:
{4, 0, 16}
Execution 3:
{3, 0, 15}
Execution 4:
{3, 0, 16}
result:
races:
The Race-Detection and Flipping Algorithm
28.11.2014 From Symbolic Execution to Concolic Testing 30
generate a random input and a schedule
execute code with the generated input and schedule
compute the race conditions and symbolic constraints
generate a new schedule or a new input
continue until all possible distinct execution paths have
been explored (depth-first search strategy)
init:
loop:
Generating new inputs/schedules
28.11.2014 From Symbolic Execution to Concolic Testing 31
new input: concolic testing
new schedule:
pick two events which are in a race
delay the first event as much as possbile
schedule1: schedule 2:
How to identify
races?
28.11.2014 From Symbolic Execution to Concolic Testing 32
How to identify races?
28.11.2014 From Symbolic Execution to Concolic Testing 33
vector clocks
- each thread t gets it‘s own vector clock Vt
- each memory location gets another two
Vector Clocks - Example
28.11.2014 From Symbolic Execution to Concolic Testing 34
Vector Clocks - Algorithm
28.11.2014 From Symbolic Execution to Concolic Testing 35
Vector Clocks – Example
28.11.2014 From Symbolic Execution to Concolic Testing 36
Vector Clocks – Example
28.11.2014 From Symbolic Execution to Concolic Testing 37
Vector Clocks – Example
28.11.2014 From Symbolic Execution to Concolic Testing 38
Vector Clocks – Example
28.11.2014 From Symbolic Execution to Concolic Testing 39
Vector Clocks – Example
28.11.2014 From Symbolic Execution to Concolic Testing 40
Vector Clock Theorem
28.11.2014 From Symbolic Execution to Concolic Testing 41
Questions?
28.11.2014 From Symbolic Execution to Concolic Testing 42
Precise Definitions
(just in case)
28.11.2014 From Symbolic Execution to Concolic Testing 43
Race Relation – Simple Defintion:
28.11.2014 From Symbolic Execution to Concolic Testing 44
sequentially related
28.11.2014 From Symbolic Execution to Concolic Testing 45
access precedence related
28.11.2014 From Symbolic Execution to Concolic Testing 46
causally related
28.11.2014 From Symbolic Execution to Concolic Testing 47
race related
28.11.2014 From Symbolic Execution to Concolic Testing 48
Race-Detection and Flipping Algorithm
Detailled Example
28.11.2014 From Symbolic Execution to Concolic Testing 49
top related