2006 2 nd joint workshop between security research labs in japan and korea an enhanced buffer...

2006 2nd Joint Workshop between Security Research Labs in JAPAN and KOREA

An Enhanced Buffer Separation Scheme to Protect Security

Sensitive Data against Buffer Overflow Attacks

2006. 2. 20.

Researcher : Lee, Seung Min(Presenter : Lee, Seung Ick)

High Performance Computing Laboratory at POSTECH

2006 2nd Joint Workshop between Security Research Labs in JAPAN and KOREA 2/14

Contents

Introduction

Related Works

Motivation

Problem Definition

Proposed Idea

Performance Analysis

Conclusion and Future Works


Introduction Buffer overflow

Occur when a program or process stores more data in buffer than the buffer size

Security sensitive data Data to be a target of buffer overflow attack for a

successful change of control flow Basic steps of buffer overflow attack

The first step To find, discover and identify vulnerability of buffer overflow

The second step To overflow and overwrite security sensitive data near buffer

The third step To change control flow of process

The fourth step To execute intended instruction or process


Protection Methods are classified depending on the step the prevention method is applied.

The first step To eliminate buffer overflow vulnerability

The second step To prevent overflow or overwrite data

The third step To prevent no intended control flow of a program or process

The fourth step To apply that data of buffer can’t be executed through

hardware support

Protection Methods Related Works ( 1/2 )


Problems of Previous Works

Problem of the prevention method on the first step Generate too many false warnings and miss errors in the code

Problem of the prevention method on the second step Performance degradation through array bounds checking

Not to provide complete protection against vulnerabilities in user defined or non standard library code

Problem of prevention method on the third and fourth steps Occur an exception or termination because of attack failure

Related Works ( 2/2 )


Motivation ( 1/2 )

The prevention methods applied in these steps but the second have the problem of process availability, since the process stops its execution because of false warnings, exceptions or termination.

Our focus is to provide a reliable process availability and more secure protection method.

Buffer separation approach is the method applied on the second step for prevention. Remove buffers from stack Allow the occurrence of buffer overflow but prevent

security sensitive data from being overwritten Gemini and DYBOC


Shortcoming of previous works Gemini

Can generate heap overflow because of buffer using heap area instead of stack

DYBOC Has a memory overhead problem because of using

write- protected page

Motivation ( 2/2 )

Buffer 1

Meta Data

Heap Struture

Low Address

High Address

Buffer 2

Meta Data

Case of Gemini

Write-protected Page

Meta Data

Buffer 3

Case of DYBOC


Heap overflowoccurs

Heap overflowcan’t occurs


Problem Definition

To prevent stack and heap overflow for using

enhanced buffer separation approach

To have minimal performance penalty


Basic Idea

Assumption We can know the size of arrays at compile time.

Enhanced buffer separation schemes Buffer stack

Separate buffer from the security sensitive data on the stack area.

Separated meta data Separate meta data from buffer on the heap

area.

Proposed Idea ( 1/3 )


Buffer stack is to prevent against stack overflow. Buffer stack will be determined as using heap or

stack at the compile time.

Buffer Stack Architecture Proposed Idea ( 2/3 )

Original Stack Structure

Low Address

High Address

Return Address

Old EBP

Buffer 1

Local Variables

Buffer 2

Return addresscan be changed

Return Address

Old EBP

Address of EXP

Local Variables

ESP

EBP

Write-protected Page Write-protected Page

ESP

EBP

High Address

Low Address

Stack Area

Buffer 1

Buffer 2

Heap Area

EXP

Return addresscan’t be changed

Buffer Stack Structure for using Heap

OR

Return Address

Old EBP

Local Variables


ESP

EBP

Buffer 1

Buffer 2

High Address

Low Address Stack Area Stack Area

Middle of Stack

Return addresscan’t be changed

Buffer Stack Structure for using Stack


Separated Meta Data Architecture

Separated meta data is to prevent against heap overflow.

Proposed Idea ( 3/3 )

Buffer 1

Meta Data

Buffer 2

Meta Data

Heap Struture

Parameter

Return Address

Old Frame Pointer

Variables

Pointer 1

Stack StrutureLow Address

High Address

ESP

EBP

Pointer 2point

Previous Heap Architecture

Meta data canbe broken

Buffer 1

Buffer 2

Parameter

Return Address

Old Frame Pointer

Variables

Pointer 1

Pointer 1'

Pointer 2

Pointer 2'

Heap Struture Stack Struture

EBP

ESP

point

High Address

Low Address

Meta Data


Meta Data

pointMeta data can’t

be broken

Separated Meta Data Architecture


Performance Analysis Performance comparison with Gemini, DYBOC and

our solutions

Limitation If the security sensitive data exists inside buffers, it is

very hard to split them from the buffers. Our solutions can’t prevent data from pointer operations

that use the primitive type variables. Compiler has to know the size of stack.

Data(Stack) Data(Heap) Time Memory

Original Overwrite Overwrite - -

Gemini Prevent Overwrite Increase Increase

DYBOC Prevent Overwrite Poor Poor

OurSolution

Prevent Prevent A littleIncrease

Poor


Experiments We chose two vulnerabilities which is similar to the source of finger

demon and PCT SSL vulnerability. We applied single thread program but it can be applied in multi-

threaded environment easily. In both cases, the security sensitive data of the original program is

overwritten and make an exception. But, the program applied our approaches correctly executes because

it protects the security sensitive data even though the buffer data is tainted.


Conclusion and Future Works

Conclusion Buffer stack prevents the stack overflow attack. Separated meta data prevents the heap overflow attack.

Future works We must find trade-off between performance and

memory overhead on case 1 and 2 of the Buffer stack architecture, and then will choose one of cases.

Throughout the implementation, we can provide more exact performance analysis.

We can provide more powerful solutions to change the main assumption: security sensitive data is located near the buffer and also in the buffer.


Thank you!Questions?


Backup slide


References [1] Hiroaki Etoh and Kunikazu Yoda. Protecting from Stack-Smashing Attacks. Published on

World-Wide Web at URL http://www.trl.ibm.com/projets/security/ssp/main.html, June 2000

[2] Donaldson, Mark E. Inside The Buffer Overflow Attack: Mechanism, Method, & Prevention. April 3, 2002. URL:http://www.sans.org/rr/paper.php?id=386

[3] Bharath Madhusudan, John Lockwood. Design of a System for Real-Time Worm [4] H. Wang, C. Guo, D. Simon, and A. Zugenmaier. Shield: Vulnerability-driven network filt

ers for preventing known vulnerability exploits. In Proceedings of ACM SIGCOMM, Portland, OR, Aug. 2004 Detection, 12th Annual Proceedings of IEEE Hot Interconnects 2004

[5] J. Newsome and D. Song. Dynamic taint analysis for automatic detection, analysis, and signature generation of exploits on commodity software. In Proceedings of the 12th Annual Network and Distributed System Security Symposium(NDSS05), Feb. 2005

[6] Rinard. M., Cada. C., Dumitran. D., Roy. D., Leu.T. A Dynamic Technique for Eliminating Buffer Overflow Vulnerabilities (and Other Memory Errors). In: Proceedings 20th Annual Computer Security Applications Conference (ACSAC), 2004

[7] StackShield. http://www.angelfire.com/sk/stackshield [8] A. Baratloo, T. Tsai, and N. Singh. Transparent Run-Time Defense Against Stack Smash

ing Attacks. In Proceedings. of the USENIX Annual Technical Conference, June 2000


Reference (Cont.) [9] Crispin Cowan, Steve Beattie, John Johansen, and Perry Wagle. Pointguard: Pro

tecting pointers from buffer overflow vulnerabilities. In Proceedings of the 12th USENIX Security Symposium, Washington, D.C., August 2003

[10] BB. Madan, S. Phoha, G. NIST, KS Trivedi, StackOfence: A Technique for Defending Against Buffer Overflow Attacks, In Proceedings of the International COnference on Information Technology: Coding and Computing(ITCC05), 2005

[11] J. Xu, Z. Kalbarczyk, S. Patel, and R. K. Iyer. Architecture support for defending against buffer overflow attacks. In 2nd Workshop on Evaluating and Architecting Systems for Dependability, 2002

[12] S Bhatkar, DC DuVarney, R Sekar, Address obfuscation: An efficient approach to combat a broad range of memory error exploits, In Proceedings of the 12th USENIX Security Symposium, 2003

[13] Wilander, J. and M. Kamkar, Comparison of Publicly Available Tools for Dynamic Buffer Overflow Prevention, 10th Network and Distributed System Security Symposium, 2003

[14] Stelios Sidiroglou, Giannis Giovanidis, and Angelos D. Keromytis, A Dynamic Mechanism for Recovering from Buffer Overflow Attacks

[15] R Hieb, RK Dybvig, C Bruggeman, L Hall, Representing Control in the Presence of First-Class Continuations, In Proceedings of the ACM SIGPLAN 1990 conference on 1990 [16] EG Barrantes, DH Ackley, S Forrest, D Stefanovic, Randomized Instruction Set Emulation, ACM Transactions on Information and System Security, 2005

[16] Christopher Dahn, Spiros Mancoridis, Using Program Transformation to Secure C Programs Against Buffer Overflows, in Proceedings of the 10 th Working Conference on Reverse Engineering (WCRE03)


Scenario

void f(char *a) { char c[10]; strcpy(c,a);}void main() { char a[20]; int i; a[5] = 4; f1(a);}

Original Code push ebpmov ebp, espsub esp, 10

push [ebp+8]push [ebp-10]call strcpy

add esp,10leaveret

push ebpmov ebp, espsub esp, 24

mov [ebp-15],4push ebp-20call f1

add esp,24leaveret

Original Pseudo Assembly Code

Prologue

Epilogue

Return Address of main

Old EBP

a

i

c

Return Address of fOld EBP

EBP

ESP

address of a

push ebpmov ebp, espsub exp, 10

push [ebp+4]push expcall strcpy

add exp,10leaveret

push ebpmov ebp, espsub esp, 4sub exp, 20

mov [exp+5],4push expcall f1

add exp,20add esp,4leaveret

Modified Pseudo Assembly Code


Old EBP

a

i

c

Return Address of f

Old EBP

EBP

ESPaddress of a

EXP


Scenario (Cont.)

void f(char *a) { char c[10]; strcpy(c,a);}void main() { char a[20]; int i; a[5] = 4; f1(a);}

Original Code push ebpmov ebp, espsub esp, 10

push [ebp+4+S]push ebp-10+Scall strcpy

add esp,10leaveret

push ebpmov ebp, espsub esp, 24

mov [exp-15+S],4push expcall f1

add esp,24leaveret

Modified Pseudo Assembly Code

Prologue

Epilogue


Old EBP

a

i

c

Return Address of f

Old EBP

EBP

ESP

address of a

S : Size of Stack


function(char *packet, unsigned int N){ char buf[32]; unsigned int register i; if(N < 32) { memcpy(buf,packet,N); for(i = 0; i < N; i++) buf[i+N] = ~buf[i]; }}

Vulnerable code of PCT SSL vulnerability

2006 2 nd joint workshop between security research labs in japan and korea an enhanced buffer...

Documents