1

Vulnerability-Specific Execution Filtering for Exploit Prevention on Commodity Software

James NewsomeDavid Brumley, Dawn Song

Carnegie Mellon University

2

Danger: Zero-day exploits

First host attacked

All vulnerable hosts attacked

Reaction timeSlammer: 10 mins

Future worms: < 1 minute [Staniford et. al. 2002]

Any reaction must be automatic

3

Sting: End-to-End Automatic Worm Defense

ExploitDetected!

4

Antibody Requirement: Vulnerability-Specific

• Signatures are typically exploit-specific– Honeycomb [Kreibich et. al. 2003],

Earlybird [Singh et. al. 2003], Autograph [Kim et. al. 2004]

• Not robust against polymorphism– Use encryption and code obfuscation to automatically

rewrite exploit code each time

• Antibodies should be vulnerability-specific

5

Related Work: Vulnerability-Specific Input Filters

• Vigilante [Costa et. al. 2005]• Vulnerability-Based Signatures [Brumley et. al. 2006]• “Correct” filter can be excessively large or expensive• Do not automatically take process state into account

– Current decryption key (needed for encrypted protocols)– Configuration (vulnerable feature enabled?)

6

Our Approach: Vulnerability Specific Execution Filtering (VSEF)

• Execution filtering instead of input-based filtering– Greater accuracy– Works with encrypted protocols

• Approach:– Assuming monitoring points detect new attacks – Generate execution filter specifying detection code needed to

detect attacks against that vulnerability

• Efficiency:– Only monitor execution for vulnerability-related segments– In contrast to traditional full protection:

» Rewrite binary to insert checks at every potential vulnerability» Problem: Too slow to use everywhere all the time

7

VSEF Example• buf[j] = x• …• anotherbuf[k] = y• …• yetanother[m] = z

Vulnerable Program

• buf[j] = x• …• if (k>KMAX)…• anotherbuf[k] = y• …• yetanother[m] = z

Vulnerability-Specific Execution Filtering

Full Execution Filtering

• if (j>JMAX)…• buf[j] = x• …• if (k>KMAX)…• anotherbuf[k] = y• …• if (m>MMAX)…• yetanother[m] = z

8

VSEF

VSEF Architecture

SampleExploit

ExploitDetector

Exploit execution

trace VSEF FilterGeneration

VSEF Binary Instrumentation

Engine

VSEF Filter

HardenedBinary

Program

VulnerableBinary

Program

9

VSEF Properties

• Vulnerability-specific, not exploit-specific– Robust to polymorphism

• Low risk of false positives• Low performance overhead• Works on encrypted protocols• Works on commodity software

10

Outline

• Overview• Taint-based VSEF• Destination-based VSEF• Distributed protection• Conclusion

11

Exploit Detection: Dynamic Taint Analysis

• Detects overwrite attacks– Buffer overflows– Format string attacks– Double free attacks

• Dynamic Taint Analysis: – Keep track of tainted data from untrusted sources– Detect when tainted data is used in a sensitive way

» e.g., as return address or function pointer

• We use TaintCheck [Newsome et. al. 2005]• Others:

– [Suh et. al. 2004], [Costa et. al. 2004], [Crandall et. al. 2004]

12

Full Dynamic Taint Analysis

1. recv(socketfd, buf1, …)NewTaint(buf1, …)

2. memcpy(buf2, buf1, …)PropTaint(buf2, buf1, …)

3. y = xPropTaint(&y, &x, …)

4. strcpy(dummy->buf, buf1)PropTaint(dummy->buf, buf1, …)

5. dummy->fnptr()TaintAssert(&fnptr, …)

MISUSE!

x

y

buf1

buf2

dummy->buf

dummy->fnptr

Memory Map

13

Taint-Based VSEF

• Full taint analysis: most instructions instrumented– 3 to 40x slowdown

• Only a few are relevant to a particular vulnerability• Strategy:

– Detection: Track which instructions propagate or misuse taint– VSEF generation: Identify relevant instructions– VSEF instrumentation: Instrument only relevant instructions

14

Taint-Based VSEF filter generation

1. recv(socketfd, buf1, …)NewTaint(buf1, …)

2. memcpy(buf2, buf1, …)PropTaint(buf2, buf1, …)

3. y = xPropTaint(&y, &x, …)

4. strcpy(dummy->buf, buf1)PropTaint(dummy->buf, buf1)

5. dummy->fnptr()TaintAssert(&fnptr)

MISUSE!

x

y

1buf1

buf2 2

dummy->buf

dummy->fnptr4

Memory Map

VSEF filterTaint misuse at:

5Taint propagation at:

41

15

Taint-Based VSEF binary hardening

1. recv(socketfd, buf1, …)NewTaint(buf1, …)

2. memcpy(buf2, buf1, …)

3. y = x

4. strcpy(dummy->buf, buf1)PropTaint(dummy->buf, buf1)

5. dummy->fnptr()TaintAssert(&fnptr)

MISUSE!

1buf1

dummy->buf

dummy->fnptr4

Memory Map

VSEF filterTaint misuse at:

5Taint propagation at:

41

16

False positive analysis: untainting

• Challenge: instruction not in filter overwrites tainted data with clean data

• Solution 1: untaint heuristic– Record value of tainted memory– Value changes untaint– Drawback: uninstrumented instruction may overwrite tainted

memory with same value it already had

• Solution 2: hardware assistance– Use hardware assistance to detect writes to tainted memory– E.g., memory watch points, write-protected pages– Drawback: not always available, trap on write to tainted

memory

17

Path 2: 1,5,6

False negative analysis: polymorphism

• Immune to polymorphism (code encryption)• False negative if exploited via alternate execution path

– Not many such paths possible– Attacker has to manually find alternate execution paths– Can merge VSEFs when we observe another execution path

1. recv(buf, socketfd, …)2. if (buf[0] == 1)3. while((dummy->buf[j] = buf[j]) != ‘\0’));4. else if (buf[1] == 2)5. while((dummy->buf[j] = buf[j]) != ‘\0’));6. dummy->fnptr();

Union: 1,3,5,6

Path 1: 1,3,6

18

Evaluation

• Experiment 1: Valgrind implementation (Linux)– ATPhttpd web server– Invalid URL buffer overflow– Use Flood tool to fetch 1 KB pages

• Experiment 2: DynamoRIO implementation (Windows)– SQL Server– SQL Slammer exploit– Simple SELECT query

19

Performance Evaluation

• ATPhttpd (Valgrind on Linux)– 186 us to generate VSEF from execution trace– 195 ms to harden binary– Hardened binary

» 6% overhead over Valgrind » 140% over native

• SQL Server (DynamoRIO on Windows)– Hardened binary

» 3% over DynamoRIO» 14% over native

20

Accuracy Evaluation

• No false positives during performance analysis• Successfully detects original attack• Successfully detects polymorphic variants

– Replaced attack-code with random byte values

21

Properties of Taint-based VSEF

• Accurate– Zero or low false positives– Low false negatives, effective for polymorphic worms

• Fast generation– Negligible time from execution trace

• No side-effect on normal execution– Does not change normal program execution behavior– No extra testing needed

• Efficient– Low performance overhead (10% over Valgrind)

22

Outline

• Overview• Taint-based VSEF• Destination-based VSEF• Distributed protection• Conclusion

23

Destination-Based VSEF

• One of the propagation instructions is overwrite point– e.g., the instruction that overwrote the function pointer

• Overwrite-point wrote to unintended destination• Strategy:

– Filter generation: identify overwrite point, and destination– Binary hardening: instrument overwrite point

• Advantages:– Only one instruction to instrument– Detect attack at time-of-overwrite instead of time-of-misuse

24

Destination-Based VSEF filter generation

1. recv(socketfd, buf1, …)NewTaint(buf1, …)

2. memcpy(buf2, buf1, …)PropTaint(buf2, buf1, …)

3. y = xPropTaint(&y, &x, …)

4. strcpy(dummy->buf, buf1)PropTaint(dummy->buf, buf1)

5. dummy->fnptr()TaintAssert(&fnptr)

MISUSE!

x

y

1buf1

buf2 2

dummy->buf

dummy->fnptr4

Memory Map

VSEF:Overwrite Point:

main(), vuln(), strcpy(), 0x.…Overwrite destination:

dummy->fnptr

25

Destination-Based VSEF binary hardening

1. recv(socketfd, buf1, …)

2. memcpy(buf2, buf1, …)

3. y = x

4. strcpy(dummy->buf, buf1)WriteAssert(&dummy->fnptr, …)

5. dummy->fnptr()

VSEF:Overwrite Point:

main(), vuln(), strcpy(), 0x.…Overwrite destination:

dummy->fnptr

26

False positive analysis

• Wrong instruction as overwrite point• Overwrite instruction can legitimately write there

– Example: C union of a buffer and a function pointer

27

False negative analysis

• Conventional polymorphism is no problem• Polymorphic execution path

– Different overwrite instruction– Different context– Different destination

28

Implementation

• VSEF generation– Use execution trace from TaintCheck– Overwrite point = last write to a dynamically calculated

address

• VSEF hardening– Use Dyninst to insert check at overwrite point

29

Evaluation

• Used ATPhttpd vulnerable web server• Performance

– 3% over native execution

• No false positives• No false negatives

– Including polymorphic variants

30

Comparison to Taint-Based VSEF

• More efficient: only 3% overhead over native– (vs. Taint-Based was 6% over Valgrind, 140% over native)– Fewer instructions instrumented– Pseudo-static rewriting using Dyninst

• Detects at time-of-write instead of time-of-use– Could be an important distinction in future work

31

Outline

• Overview• Taint-based VSEF• Destination-based VSEF• Distributed protection• Conclusion

32

Application: Reactive Antibody Defense

ExploitDetected!

33

Distribution & Verification

• Distribution speed– Need exponential distribution to match worm speed– Use a robust P2P architecture

• Verification– Poor \ Malicious Taint-Based VSEF only hurts performance– Generate hardened binary candidate using VSEF– Replay exploit against hardened candidate– Exploit detected: optimize and accept VSEF– No exploit detected: reject VSEF

34

Related Work

• Syntax pattern-extraction signature generation– Honeycomb [Kreibech2003]– Autograph [Kim2004]– Earlybird [Singh2004]– Polygraph [Newsome2005]

• Vulnerability-based signature generation– Vigilante [Costa2005]– Vulnerability-based Signatures [Brumley2006]

• Host-based antibody generation– Shadow honeypots [Anagnostakis2005]– STEM [Sidiroglou2005]– COVERS [Liang2005]– [Xu2005]

35

Conclusion

• We propose a new form of exploit antibody: VSEF• Advantages of VSEF approach:

– Only run heavy detector on some fraction of all traffic– Only need 1 exploit sample to automatically generate VSEF filter– VSEF filter is light-weight and accurate– Works on commodity software (no access to source code)

• VSEF enables automated, fast, accurate response to zero-day exploits

36

DynInst vs. Valgrind experience

• DynInst is an excellent tool+ Nice, well documented API+ Good performance (for “surgical” modifications)+ Works on Linux and Windows+ Good support (Thanks Drew!)

• Valgrind, compared to DynInst+ Easier to develop (abstraction)+ More suitable for “total rewrite” applications?– Linux only– Poor performance

• Ideas to consider ‘borrowing’ from Valgrind– System call callbacks, and abstraction– Instruction set abstraction (DynInst does this a little)– Shadow memory support

37

Thanks

• jnewsome@ece.cmu.edu• dbrumley@cs.cmu.edu• dawnsong@cmu.edu