jFuzz – Java based Whitebox Fuzzing
David HarvisonAdam Kiezun
2
Summary
Problem Generating interesting test inputs for file
reading programs takes time.
Approach Create a smart fuzzer to generate inputs
that cause programs to crash.
Results jFuzz generates a many input files and
creates a base for others to expand upon.
3
Problem
Bugs in a program may cause crashes for specific input files.
A compiler with buggy code, a media player with a corrupt file, etc.
Creating input files by hand takes time. Some of the files may exercise the same code.
Want a way to automatically generate input files that crash the program.
4
Idea
Generate inputs that cause crashes by generating many inputs that exercise unique execution paths.
5
Program Exampleif (car0 == '-') {
neg = true;
} else {
neg = false;
cnt++;
}
if (car1 >= '0' && car1 <= '5') {
val = car1 - '0';
} else {
val = car1;
}
File reading code.
6
Program Example File reading code. Want to generate
files which exercise different program paths.
if (car0 == '-') {
neg = true;
} else {
neg = false;
cnt++;
}
if (car1 >= '0' && car1 <= '5') {
val = car1 - '0';
} else {
val = car1;
}
7
Program Example File reading code. Want to generate
files which exercise different program paths.
car0 == '-'car1 == '5'
if (car0 == '-') {
neg = true;
} else {
neg = false;
cnt++;
}
if (car1 >= '0' && car1 <= '5') {
val = car1 - '0';
} else {
val = car1;
}
8
Program Example File reading code. Want to generate
files which exercise different program paths.
car0 == '-'car1 == '7'
if (car0 == '-') {
neg = true;
} else {
neg = false;
cnt++;
}
if (car1 >= '0' && car1 <= '5') {
val = car1 - '0';
} else {
val = car1;
}
9
Program Example File reading code. Want to generate
files which exercise different program paths.
car0 == '+'car1 == '3'
if (car0 == '-') {
neg = true;
} else {
neg = false;
cnt++;
}
if (car1 >= '0' && car1 <= '5') {
val = car1 - '0';
} else {
val = car1;
}
10
Program Example File reading code. Want to generate
files which exercise different program paths.
car0 == '+'car1 == '9'
if (car0 == '-') {
neg = true;
} else {
neg = false;
cnt++;
}
if (car1 >= '0' && car1 <= '5') {
val = car1 - '0';
} else {
val = car1;
}
11
Related Tools
Cute, EXE, SAGE, catchconv, Apollo Smart fuzzers - programs that generate
interesting new inputs for programs. Not for Java.
JCute Smart fuzzer for Java. Reinstruments code – Requires source files. Has problems with the JDK.
12
Overall Idea
Input A compiled (into bytecode) Java program. A valid input file.
Output New input files which exercise unique control paths.
Run the subject program in a modified JVM.
A logic predicate, the Path Condition, is formed as the program executes.
Describes control flow of execution.
New inputs are created by manipulating the path condition.
13
Example
public void top(char[] input) { int cnt = 0; if (input[0] == ‘b’) cnt++; if (input[1] == ‘a’) cnt++; if (input[2] == ‘d’) cnt++; if (input[3] == ‘!’) cnt++; if (cnt > 3) crash();}
good
14
Example
public void top(char[] input) { int cnt = 0; if (input[0] == ‘b’) cnt++; if (input[1] == ‘a’) cnt++; if (input[2] == ‘d’) cnt++; if (input[3] == ‘!’) cnt++; if (cnt > 3) crash();}
good
I0 != 'b'
I1 != 'a'
I2 != 'd'I3 != '!'
path condition
Negate constraints in path condition.Solve the new path condition to create new inputs.
15
Example
public void top(char[] input) { int cnt = 0; if (input[0] == ‘b’) cnt++; if (input[1] == ‘a’) cnt++; if (input[2] == ‘d’) cnt++; if (input[3] == ‘!’) cnt++; if (cnt > 3) crash();}
good
I0 != 'b'
I1 != 'a'
I2 != 'd'I3 == '!'
goo!
16
Example
public void top(char[] input) { int cnt = 0; if (input[0] == ‘b’) cnt++; if (input[1] == ‘a’) cnt++; if (input[2] == ‘d’) cnt++; if (input[3] == ‘!’) cnt++; if (cnt > 3) crash();}
good
I0 != 'b'
I1 != 'a'
I2 == 'd'godd
17
Example
public void top(char[] input) { int cnt = 0; if (input[0] == ‘b’) cnt++; if (input[1] == ‘a’) cnt++; if (input[2] == ‘d’) cnt++; if (input[3] == ‘!’) cnt++; if (cnt > 3) crash();}
good
godd
goo!
gaod
bood
All paths are explored systematically.
18
Tool Used
NASA Java PathFinder Dynamic analysis framework for Java
implemented as a JVM. Allows backtracking including saving and restoring
the whole state of the VM. Can execute all thread interleavings. Can execute a program on all possible inputs.
19
Attributes
Additional state-stored information. Associated with runtime values. JPF propagates attributes across
assignment, method calls, etc. Allows us to keep track of how the
variables relate to the input using symbolic expressions.
20
Concrete v. Concolic
3 4
+
7
Normal
+
3 4
7
With Attributes
exp0 exp1
Sum(exp0, exp1)
Concolic execution is both concrete and symbolic.
Concrete
Symbolic
21
Concrete v. Concolic
public class IADD extends Instruction {
public Instruction execute (... ThreadInfo th) {
[1] int v1 = th.pop();
[2] int v2 = th.pop();
[3] th.push(v1 + v2, ...);
[4] return getNext(th);
}
}
public class IADD extends ...bytecode.IADD {
public Instruction execute (... ThreadInfo th) {
[1] int v1 = th.pop();
[2] int v2 = th.pop();
[3] th.push(v1 + v2, ...);
[4] StackFrame sf = th.getTopFrame();
[5] IntExpr sym_v1 = sf.getOperandAttr();
[6] IntExpr sym_v2 = sf.getOperandAttr();
[7] if (sym_v1 == null && sym_v2 == null)
return getNext(th);
[7] IntExpr result = sym_v1._plus(sym_v2);
[8] sf.setOperandAttr(result);
[9] return getNext(th);
}
}
Concrete
Symbolic
22
jFuzz Architecture
Runs JPF many times on the subject program and input files.
Each run: Collects the Path Condition
(PC). Negates each constraint,
reduces, and solves. Uses new PCs to generate new
input files.
Keeps track of inputs which caused exceptions to be thrown.
jFuzz
JPF
Subjectand Input
PC
Solver
NegatedPC
NewInput
Subjectand
OriginalInput
Inputswhichcause
crashes
23
Creating New Inputs
For a given execution some parts of the input may not be read.
24
Creating New Inputs
For a given execution some parts of the input may not be read.
25
Creating New Inputs
For a given execution some parts of the input may not be read.
When the path condition is solved, only the read parts will have new values.
26
Creating New Inputs
For a given execution some parts of the input may not be read.
When the path condition is solved, only the read parts will have new values.
The changes are written over the original input, preserving the unused parts.
27
Reducing the Path Condition
Path Conditions can be very long.
Not all constraints are effected by negating the PC.
Constraints not effected can be removed from the PC.
jFuzz
JPF
Subjectand Input
PC
Solver
NegatedPC
NewInput
Subjectand
OriginalInput
Inputswhichcause
crashes
PCMinimiz
er
28
Example Reduction
Path Condition:
[1] a + b < 10
[2] b > 6
[3] c < 15
[4] a < 3
[5] c + d > 7
[6] e != 1
[7] c – e = 5
[8] a == 2
29
Example Reduction
Start with fuzzing the last constraint.
Path Condition:
[1] a + b < 10
[2] b > 6
[3] c < 15
[4] a < 3
[5] c + d > 7
[6] e != 1
[7] c – e = 5
[8] a != 2
30
Example Reduction
Start with fuzzing the last constraint.
Select all constraints which contain variables in that constraint.
Path Condition:
[1] a + b < 10
[2] b > 6
[3] c < 15
[4] a < 3
[5] c + d > 7
[6] e != 1
[7] c – e = 5
[8] a != 2
31
Example Reduction
Start with fuzzing the last constraint.
Select all constraints which contain variables in that constraint.
If one of the constraints contains multiple variables, select all constraints which contain those variables.
Path Condition:
[1] a + b < 10
[2] b > 6
[3] c < 15
[4] a < 3
[5] c + d > 7
[6] e != 1
[7] c – e = 5
[8] a != 2
32
Example Reduction
Start with fuzzing the last constraint.
Select all constraints which contain variables in that constraint.
If one of the constraints contains multiple variables, select all constraints which contain those variables.
All other constraints can be removed.
Path Condition:
[1] a + b < 10
[2] b > 6
[4] a < 3
[8] a != 2
33
Example Reduction
Start with fuzzing the last constraint.
Select all constraints which contain variables in that constraint.
If one of the constraints contains multiple variables, select all constraints which contain those variables.
All other constraints can be removed.
Variables not in the new PC are left unchanged.
Path Condition:
[1] a + b < 10
[2] b > 6
[3] a < 3
[4] a != 2
34
Reducing the Path Condition
Reductions are performed for every constraint that is negated.
jFuzz uses a UnionFind data structure to find which variables are connected to each other.
In our case study, the average reduction was from around 250 constraints to about 5 constraints.
35
Case Study
Subject: Sat4J SAT solver written in Java. Takes inputs in dimacs files. ~10 kloc.
Goals: Create inputs that crash
Sat4J. Create a set of good inputs.
test1.dimacs
c test 3 single clauses and 2
c binary clauses
p cnf 4 5
1 0
2 0
3 0
-2 4 0
-3 4 0
36
Results
After 30 minutes of execution: 12,000 input files were created. 70 crashes where found.
The crashes are actually normal for SAT4J 38 Invalid DIMACS files. 27 Contradictions. 4 Assertion Errors.
A Java compiler would be more compelling. Any crash is due to a bug in the compiler. Much larger program.
37
Performance
Sat4J was run 100 times in each VM. The times are average runtime. Simplifying the Path Condition reduces
the solving time by 30%.
Time (s) Ratio PC Solving (s)Sun JVM 0.21 1 - 0.15JPF 2.5 11 - 1.8
5.8 28 2.5 1.85.3 25 1.8 1.8
Init (s)
jFuzz--jFuzz
38
Conclusions
jFuzz is the first concolic tester for Java which will work for any bytecode.
This opens the door for more advanced fuzzing techniques, such as grammar based fuzzing.
