dynamorio rpioss-aug2011

The DynamoRIODynamic Tool Platform

Derek Bruening

Typical Modern Application: IIS

2

Runtime Interposition Layer

3

underlying platform (stock OS,

commodity hardware)

DynamoRIO:

manipulate every instruction in

running application

running application

Outline

System Overview

Example Tools

• Security

• Debugging

Open Source Project

4

Direct Code Modification

5

Kernel32!TerminateProcess:

7d4d1028 7c 05 jl 7d4d102f

7d4d102a 33 c0 xor %eax,%eax

7d4d102c 40 inc %eax

7d4d102d eb 08 jmp 7d4d1037

7d4d102f 50 push %eax

7d4d1030 e8 ed 7c 00 00 call 7d4d8d22

e9 37 6f 48 92 jmp <callout>

Entry Point Complications

6

Kernel32!TerminateProcess:

7d4d1028 7c 05 jl 7d4d102f

7d4d102a 33 c0 xor %eax,%eax

7d4d102c 40 inc %eax

7d4d102d eb 08 jmp 7d4d1037

7d4d102f 50 push %eax

7d4d1030 e8 ed 7c 00 00 call 7d4d8d22

e9 37 6f 48 92 jmp <callout>

application code

bar()foo()

Basic Interpreter

7

A

B C

D

E

F

interpreter

Slowdown: ~300x

fetch decode execute

application code

bar()foo()

Improvement #1: Basic Block Cache

8

A

B C

D

E

F

DynamoRIO

software

code

cache

A

C

D

E

F

Slowdown: 300x 25x

application code

bar()foo()

Improvement # 2: Linking Direct Branches

9

A

B C

D

E

F

DynamoRIO

software

code

cache

A

C

D

E

F

Slowdown: 300x 25x 3x

application code

bar()foo()

Improvement # 3: Linking Indirect Branches

10

A

B C

D

E

F

DynamoRIO

software

code

cache

A

C

D

E

F

Slowdown: 300x 25x 3x 1.2x

indirect

branch

lookup

application code

bar()foo()

Improvement # 4: Building Traces

11

A

B C

D

E

F

DynamoRIO

software

code

cache

A

C

D

E

F

indirect

branch

lookup

Slowdown: 300x 26x 3x 1.2x 1.1x

cmp

application code

bar()foo()

Tool Platform

12

A

B C

D

E

F

DynamoRIO

software

code

cache

C

X

D

F

indirect

branch

lookup

cmp

tool codeA

E

Transparency

Do not want to interfere with the semantics of the program

Dangerous to make any assumptions about:

• Register usage

• Calling conventions

• Stack layout

• Memory/heap usage

• I/O and other system call use

13

Painful, But Necessary

Difficult and costly to handle corner cases

Many applications will not notice…

…but some will!

• Microsoft Office: Visual Basic generated code, stack convention

violations

• COM, Star Office, MMC: trampolines

• Adobe Premiere: self-modifying code

• VirtualDub: UPX-packed executable

• etc.

14

Avoid Resource Conflicts

15

Linux Windows

DynamoRIO Demo

Inserts counters into every basic block

Counters are visible via shared memory

16

Outline

System Overview

Example Tools

• Security

• Debugging

Open Source Project

17

Anatomy of an Attack

system and

application memory

kernel

network

ENTER

CORRUPT DATA

HIJACK PROGRAM COUNTER

COMPROMISE

Critical Data: Control Flow Indirection

Subroutine calls

• Return address and activation records on visible stack

Dynamic library linking

• Function exports and imports

Object oriented polymorphism: dynamic dispatch

• Vtables

Callbacks – registered function pointers

• Event dispatch, atexit

Exception handling

Any problem in computer science can be solved with another layer

of indirection.

- David Wheeler

Critical Data: Control Flow Exploits

Return address overwrite

• Classic buffer overflow

GOT overwrite

Object pointer overwrite or uninitialized use

Function pointer overwrite

• Heap, stack, data, PEB

Exception handler overwrites

• SEH exploits

Any problem in computer science can be solved with another layer

of indirection. But that usually will create another problem.

- David Wheeler

Preventing Data Corruption Is Difficult

Stored program addresses legitimately manipulated by

many different entities

• Dynamic linker, language runtime

Intermingled with regular data

• Return addresses on stack

• Vtables in heap

Even if could distinguish a good write from a bad write, too

expensive to monitor all data writes

Insight: Hijack Violates Execution Model

HardwareInterface

Typical Application

Execution ModelSecurity Attack

Goal: Shrink Hardware Interface

Typical Application

Execution ModelSecurity Attack

Constrained Hardware Interface

Program Shepherding

Monitor all control-flow transfers during program execution

• DynamoRIO is in perfect position to do this

Validate that each transfer satisfies security policy based

on execution model

• Application Binary Interface (ABI): calling convention, library

invocation

The application may be damaged by data corruption, but

the system will not be compromised by hijacking control

flow

Outline

System Overview

Example Tools

• Security

• Debugging

Open Source Project

25

Memory Bugs

Memory bugs are challenging to detect and fix

• Memory corruption, reading uninitialized memory, memory leaks

Observable symptoms resulting from memory bugs are

often delayed and non-deterministic

• Errors are difficult to discover during regular testing

• Testing usually relies on randomly happening to hit visible symptoms

• The sources of these bugs are painful and time-consuming to track

down from observed crashes

Memory bugs often remain in shipped products and can

show up in customer usage

26

Dr. Memory

Detects unaddressable memory

accesses

• Wild access to invalid address

• Use-after-free

• Buffer and array overflow and underflow

• Read beyond top of stack

• Invalid free, double free

Detects uninitialized memory reads

Detects memory leaks

27

Implementation Strategy

Track the state of application memory using shadow

memory

• Track whether allocated and whether defined

Monitor every memory-related action by the application:

• System call

• Malloc, realloc, calloc, free, mmap, mumap, mremap

• Memory read or write

• Stack adjustment

At exit or on request, scan memory to check for leaks

28

Shadow each byte of memory with one of 3 states:

Shadow Metadata

definedunaddressable uninitialized

allocate:

malloc, stack

deallocate

write

deallocate

allocate: mmap, calloc

29

Shadow Memory

30

defined

unaddr

uninit

defined

Shadow StackStack

Shadow HeapHeap

header

malloc

header

padding

unaddr

unaddr

unaddr

defined

uninit

defined

freed

unaddr

Performance Comparison

31

Valgrindfailed

Valgrindfailed

Outline

System Overview

Example Tools

• Security

• Debugging

Open Source Project

32

Dynamo

@HP Labs

on x86

DynamoRIO History

33

Dynamo + RIO

DynamoRIO

1999

Dynamo

@HP Labs

on PA-RISC

RIO @MIT

(Runtime Introspection

and Optimization)

2001

late 1990’s 2000

Google

sponsors

Dr. Memory

DynamoRIO History Cont’d

34

2001

VMware

acquires

Determina

Determina

security startup

open-sourced

BSD licensebinary releases

DynamoRIO

@MIT

2003 2007 2010

2002 2009

Google

sponsors

Dr. Memory

DynamoRIO Team

35

VMwareDetermina

security startup

DynamoRIO

@MIT

DynamoRIO Open Source Project

Google Code

• BSD license

• Subversion repository

300 KLOC

Mostly C, some assembly

• Issue tracker

Google Groups

• User discussion forum/mailing list

• Developer mailing list

36

http://dynamorio.org

Dr. Memory Open Source Project

Google Code

• http://code.google.com/p/drmemory

• LGPL 2.1 license

• Subversion repository

67 KLOC

Mostly C

• Issue tracker

Google Groups

• User discussion forum/mailing list

• Developer mailing list

37

Potential Projects

Build a New Tool

• Code coverage

• Fuzzer

• Profiler: basic block, edge, function, etc.

• Malware sandbox

• Reverse engineering

Contribute to an Existing Tool

• Dr. Memory or Dr. Heapstat

• Revive PiPA or UMI

38

Potential Projects Cont’d

Build a Tool Library

• Control flow, call graph, data dependence analysis

• Symbol table access

Contribute to Platform

• Buffer filling API

• Probe API

• Port to MacOS

• Port to ARM

• Debugger integration

39