jfuzz – java based whitebox fuzzing david harvison adam kiezun

Click here to load reader

Post on 14-Jan-2016

214 views

Category:

Documents

1 download

Embed Size (px)

TRANSCRIPT

  • jFuzz Java based Whitebox FuzzingDavid HarvisonAdam Kiezun

  • SummaryProblemGenerating interesting test inputs for file reading programs takes time.ApproachCreate a smart fuzzer to generate inputs that cause programs to crash.ResultsjFuzz generates a many input files and creates a base for others to expand upon.

  • ProblemBugs in a program may cause crashes for specific input files.A compiler with buggy code, a media player with a corrupt file, etc.Creating input files by hand takes time.Some of the files may exercise the same code.Want a way to automatically generate input files that crash the program.

  • IdeaGenerate inputs that cause crashes by generating many inputs that exercise unique execution paths.

  • Program Exampleif (car0 == '-') { neg = true;} else { neg = false; cnt++;} if (car1 >= '0' && car1
  • Program ExampleFile reading code.Want to generate files which exercise different program paths.if (car0 == '-') { neg = true;} else { neg = false; cnt++;} if (car1 >= '0' && car1
  • Program ExampleFile reading code.Want to generate files which exercise different program paths.car0 == '-'car1 == '5'if (car0 == '-') { neg = true;} else { neg = false; cnt++;} if (car1 >= '0' && car1
  • Program ExampleFile reading code.Want to generate files which exercise different program paths.car0 == '-'car1 == '7'if (car0 == '-') { neg = true;} else { neg = false; cnt++;} if (car1 >= '0' && car1
  • Program ExampleFile reading code.Want to generate files which exercise different program paths.car0 == '+'car1 == '3'if (car0 == '-') { neg = true;} else { neg = false; cnt++;} if (car1 >= '0' && car1
  • Program ExampleFile reading code.Want to generate files which exercise different program paths.car0 == '+'car1 == '9'if (car0 == '-') { neg = true;} else { neg = false; cnt++;} if (car1 >= '0' && car1
  • Related ToolsCute, EXE, SAGE, catchconv, Apollo Smart fuzzers - programs that generate interesting new inputs for programs. Not for Java.JCute Smart fuzzer for Java.Reinstruments code Requires source files.Has problems with the JDK.

  • Overall IdeaInputA compiled (into bytecode) Java program.A valid input file.OutputNew input files which exercise unique control paths.Run the subject program in a modified JVM.A logic predicate, the Path Condition, is formed as the program executes.Describes control flow of execution.New inputs are created by manipulating the path condition.

  • Examplepublic void top(char[] input) { int cnt = 0; if (input[0] == b) cnt++; if (input[1] == a) cnt++; if (input[2] == d) cnt++; if (input[3] == !) cnt++; if (cnt > 3) crash();}good

  • Examplepublic void top(char[] input) { int cnt = 0; if (input[0] == b) cnt++; if (input[1] == a) cnt++; if (input[2] == d) cnt++; if (input[3] == !) cnt++; if (cnt > 3) crash();}goodI0 != 'b'I1 != 'a'I2 != 'd'I3 != '!'path conditionNegate constraints in path condition.Solve the new path condition to create new inputs.

  • Examplepublic void top(char[] input) { int cnt = 0; if (input[0] == b) cnt++; if (input[1] == a) cnt++; if (input[2] == d) cnt++; if (input[3] == !) cnt++; if (cnt > 3) crash();}goodI0 != 'b'I1 != 'a'I2 != 'd'I3 == '!'goo!

  • Examplepublic void top(char[] input) { int cnt = 0; if (input[0] == b) cnt++; if (input[1] == a) cnt++; if (input[2] == d) cnt++; if (input[3] == !) cnt++; if (cnt > 3) crash();}goodI0 != 'b'I1 != 'a'I2 == 'd'godd

  • Examplepublic void top(char[] input) { int cnt = 0; if (input[0] == b) cnt++; if (input[1] == a) cnt++; if (input[2] == d) cnt++; if (input[3] == !) cnt++; if (cnt > 3) crash();}goodgoddgoo!gaodboodAll paths are explored systematically.

  • Tool UsedNASA Java PathFinderDynamic analysis framework for Java implemented as a JVM.Allows backtracking including saving and restoring the whole state of the VM.Can execute all thread interleavings.Can execute a program on all possible inputs.

  • AttributesAdditional state-stored information.Associated with runtime values.JPF propagates attributes across assignment, method calls, etc.Allows us to keep track of how the variables relate to the input using symbolic expressions.

  • Concrete v. Concolic34+7Normal+347With Attributesexp0exp1Sum(exp0, exp1)Concolic execution is both concrete and symbolic.ConcreteSymbolic

  • Concrete v. Concolicpublic class IADD extends Instruction { public Instruction execute (... ThreadInfo th) {[1] int v1 = th.pop();[2] int v2 = th.pop();[3] th.push(v1 + v2, ...);[4] return getNext(th); }}public class IADD extends ...bytecode.IADD { public Instruction execute (... ThreadInfo th) {[1] int v1 = th.pop();[2] int v2 = th.pop();[3] th.push(v1 + v2, ...);[4] StackFrame sf = th.getTopFrame();[5] IntExpr sym_v1 = sf.getOperandAttr(); [6] IntExpr sym_v2 = sf.getOperandAttr();[7] if (sym_v1 == null && sym_v2 == null) return getNext(th);[7] IntExpr result = sym_v1._plus(sym_v2);[8] sf.setOperandAttr(result);[9] return getNext(th); }}ConcreteSymbolic

  • jFuzz ArchitectureRuns JPF many times on the subject program and input files.Each run:Collects the Path Condition (PC).Negates each constraint, reduces, and solves.Uses new PCs to generate new input files.Keeps track of inputs which caused exceptions to be thrown.jFuzzJPFSubjectand InputPCSolverNegatedPCNewInputSubjectand OriginalInputInputswhichcausecrashes

  • Creating New InputsFor a given execution some parts of the input may not be read.

  • Creating New InputsFor a given execution some parts of the input may not be read.

  • Creating New InputsFor a given execution some parts of the input may not be read.When the path condition is solved, only the read parts will have new values.

  • Creating New InputsFor a given execution some parts of the input may not be read.When the path condition is solved, only the read parts will have new values.The changes are written over the original input, preserving the unused parts.

  • Reducing the Path ConditionPath Conditions can be very long.Not all constraints are effected by negating the PC.Constraints not effected can be removed from the PC.jFuzzJPFSubjectand InputPCSolverNegatedPCNewInputSubjectand OriginalInputInputswhichcausecrashesPCMinimizer

  • Example ReductionPath Condition:[1] a + b < 10[2] b > 6[3] c < 15[4] a < 3[5] c + d > 7[6] e != 1[7] c e = 5[8] a == 2

  • Example ReductionStart with fuzzing the last constraint.Path Condition:[1] a + b < 10[2] b > 6[3] c < 15[4] a < 3[5] c + d > 7[6] e != 1[7] c e = 5[8] a != 2

  • Example ReductionStart with fuzzing the last constraint.Select all constraints which contain variables in that constraint.Path Condition:[1] a + b < 10[2] b > 6[3] c < 15[4] a < 3[5] c + d > 7[6] e != 1[7] c e = 5[8] a != 2

  • Example ReductionStart with fuzzing the last constraint.Select all constraints which contain variables in that constraint.If one of the constraints contains multiple variables, select all constraints which contain those variables.Path Condition:[1] a + b < 10[2] b > 6[3] c < 15[4] a < 3[5] c + d > 7[6] e != 1[7] c e = 5[8] a != 2

  • Example ReductionStart with fuzzing the last constraint.Select all constraints which contain variables in that constraint.If one of the constraints contains multiple variables, select all constraints which contain those variables.All other constraints can be removed.Path Condition:[1] a + b < 10[2] b > 6

    [4] a < 3

    [8] a != 2

  • Example ReductionStart with fuzzing the last constraint.Select all constraints which contain variables in that constraint.If one of the constraints contains multiple variables, select all constraints which contain those variables.All other constraints can be removed.Variables not in the new PC are left unchanged.Path Condition:[1] a + b < 10[2] b > 6[3] a < 3[4] a != 2

  • Reducing the Path ConditionReductions are performed for every constraint that is negated.jFuzz uses a UnionFind data structure to find which variables are connected to each other.In our case study, the average reduction was from around 250 constraints to about 5 constraints.

  • Case StudySubject: Sat4JSAT solver written in Java.Takes inputs in dimacs files.~10 kloc.Goals:Create inputs that crash Sat4J.Create a set of good inputs.test1.dimacs

    c test 3 single clauses and 2c binary clausesp cnf 4 51 02 03 0-2 4 0-3 4 0

  • ResultsAfter 30 minutes of execution:12,000 input files were created.70 crashes where found.The crashes are actually normal for SAT4J38 Invalid DIMACS files.27 Contradictions.4 Assertion Errors.A Java compiler would be more compelling.Any crash is due to a bug in the compiler.Much larger program.

  • PerformanceSat4J was run 100 times in each VM.The times are average runtime.Simplifying the Path Condition reduces the solving time by 30%.

    Sheet1

    Time (s)RatioPC Solving (s)Init (s)

    Sun JVM0.211-0.15

    JPF2.511-1.8

    jFuzz--5.8282.51.8

    jFuzz5.3251.81.8

    &C&"Bitstream Vera Sans,Roman"&A

    &C&"Bitstream Vera Sans,Roman"Page &P

  • ConclusionsjFuzz is the first concolic tester for Java which will work for any bytecode.This opens the door for more advanced fuzzing techniques, such as grammar based fuzzing.

  • Fuzzing the InputabcdePath Condition:[1] a + b < 10[2] b > 6[3] c < 15[4] a < 3[5] c + d > 7[6] e != 1[7] c e = 5[8] a != 2

  • Fuzzing the InputThe reduced PC does not effect all parts of the input.The solver will only return values for variables in the PC.abcdePath Condition:[1] a + b < 10[2] b > 6[3] a < 3[4] a != 2

  • Fuzzing the InputThe new values are written over the original input.The new input only differs from the original in the values in the PC.a'b'cdePath Condition:[1] a + b < 10[2] b > 6[3] a < 3[4] a != 2