Detection and Prevention Detection and Prevention of Buffer Overflow Exploitof Buffer Overflow Exploit

Cai JunCai JunAnti-Virus Section Manager Anti-Virus Section Manager R&D DepartmentR&D DepartmentBeijing Rising Tech. Corp. LTD.Beijing Rising Tech. Corp. LTD.

Review of Buffer Overflow Review of Buffer Overflow ExploitExploit

TimeTime Virus NameVirus Name Financial LossFinancial Loss

19891989 Morris Morris WormWorm

$96,000,000$96,000,000

2001-62001-6 CodeRed (I/ICodeRed (I/II)I)

$2,600,000,0$2,600,000,00000

2003-12003-1 SQL SQL SlammerSlammer

$1,200,000,0$1,200,000,00000

2003-82003-8 Worm.BlasteWorm.Blasterr

$1,200,000,0$1,200,000,00000

2004-72004-7 Worm.SasserWorm.Sasser $500,000,000$500,000,000

…… …… ……

What is Buffer Overflow What is Buffer Overflow ExploitExploit• Definition of a BufferDefinition of a Buffer

• How Buffers Are ExploitedHow Buffers Are Exploited

• How to Exceed Program SpaceHow to Exceed Program Space

• Overflow the StackOverflow the Stack

• What Follows a Buffer What Follows a Buffer OverflowOverflow

An Example of Buffer An Example of Buffer OverflowOverflow

int login(){

int count;int *okay;char username[16];char password[8];strcpy(username, get_name());strcpy(password, get_psw());*okay = 1;return *okay;

} A Process Stack

Subrouti ne parametersReturn addressFrame poi nter

countokay

usernamepassword

. . .

. . .

Address

Hi gh

Low

How to Detect and Prevent How to Detect and Prevent Buffer Overflow ExploitBuffer Overflow Exploit

• Static DetectionStatic Detection

• Compile Time DetectionCompile Time Detection

• Network-based DetectionNetwork-based Detection

• Host-based DetectionHost-based Detection

Static Code Analysis (Part I)Static Code Analysis (Part I)

• How it works?How it works?

Source code level analysisSource code level analysis

Static Code Analysis (Part II)Static Code Analysis (Part II)

• AdvantagesAdvantages

Help to improve an Help to improve an applicationapplication

• DisadvantagesDisadvantages– Program analysis is Program analysis is

inadequateinadequate– Modification and Modification and

recompiling of source code are recompiling of source code are needed needed

Compile Time Detection Compile Time Detection (Part I)(Part I)• How it works?How it works?

Stack-smashing protectionStack-smashing protection

A Process Stack

Subrouti ne parametersReturn addressFrame poi nter

countokay

usernamepassword

. . .

. . .

Address

Hi gh

Low Modi fi ed Process Stack

Subrouti ne parametersReturn addressFrame poi nter

caneryusernamepassword

countokay

. . .

. . .

Address

Hi gh

Low

Compile Time Detection Compile Time Detection (Part II)(Part II)• AdvantagesAdvantages

Nearly 100% protection of Nearly 100% protection of “simple function calls”“simple function calls”

• DisadvantagesDisadvantages– Recompiling is neededRecompiling is needed– No sane way to protect No sane way to protect

“complex function calls” “complex function calls”

Network based Detection Network based Detection (Part I)(Part I)• How it works?How it works?

Analyze network data for Analyze network data for attack codeattack code

Network-based Detection Network-based Detection (Part II)(Part II)• AdvantagesAdvantages

Detect exploit code by ruleDetect exploit code by rule

• DisadvantagesDisadvantages

Either high number of false Either high number of false positive alert or low positive alert or low number of true positive number of true positive alertalert

Host-based Detection (Part Host-based Detection (Part I)I)• How it works?How it works?

Executable space Executable space protectionprotection– Hardware solution (CPU)Hardware solution (CPU)– Software solutionSoftware solution

NX TechnologyNX Technology

• What is NX?What is NX?NX stands for ‘No Execute’NX stands for ‘No Execute’

• CPUs which support NXCPUs which support NXSun's Sparc, Transmeta's Efficeon,Sun's Sparc, Transmeta's Efficeon,newer 64-bit x86 processorsnewer 64-bit x86 processors::AMD64, IA-64, etc.AMD64, IA-64, etc.

• OSs implement NXOSs implement NXWindows XP SP2, Windows LonghornWindows XP SP2, Windows LonghornLinux with NX patchLinux with NX patch

Software Solution From Rising Software Solution From Rising Tech.Tech.(Part I)(Part I)Solution 1: TDI driver (only for Solution 1: TDI driver (only for

Windows)Windows)

• How it works?How it works?

use TDI driver to detect known use TDI driver to detect known buffer overflow exploitbuffer overflow exploit

TDI Cl i ents

Transport Provi derNI C Dri ver(s) and NI C(s)

Appl i cati on

Ri si ng Anti -BOE Dri ver

Software Solution From Rising Software Solution From Rising Tech.Tech.(Part II)(Part II)Solution 1:TDI driverSolution 1:TDI driver

• AdvantagesAdvantages

Detect viruses which exploit Detect viruses which exploit known vulnerabilities known vulnerabilities

• DisadvantagesDisadvantages

Fail to protect unknown Fail to protect unknown vulnerabilities vulnerabilities

Software Solution From Rising Software Solution From Rising Tech.Tech. (Part III) (Part III)Solution 2: StackCheckerSolution 2: StackChecker (Only for Windows)(Only for Windows)• How it works?How it works?

Install kernel driver to inspect system callInstall kernel driver to inspect system calls and detect invalid user calls from stack s and detect invalid user calls from stack or heapor heap

Software Solution From Rising Software Solution From Rising Tech.Tech. (Part IV) (Part IV)

API ParametersReturn address

. . .System Cal l Parameters

User-Mode Stack

Ol d Regi stersCopy of System Cal l

Parameters

Kernel -Mode StackCheck thi s address

Software Solution From Rising Software Solution From Rising Tech.Tech.(Part IV)(Part IV)Solution 2: StackCheckerSolution 2: StackChecker • AdvantagesAdvantages

Detect viruses which exploit buffer Detect viruses which exploit buffer overflowoverflow

• DisadvantagesDisadvantagesVictim program will eventually crasVictim program will eventually crash despite of the warning h despite of the warning

Summary (Part I)Summary (Part I)

If you are a programmer If you are a programmer • Check your source code manuallyCheck your source code manually• Use aid tools to find hidden bugsUse aid tools to find hidden bugs• Compile with StackGuard or other tools Compile with StackGuard or other tools

to avoid buffer overflowto avoid buffer overflow

Summary (Part II)Summary (Part II)

If you are a network administratorIf you are a network administrator• Apply NIDS productApply NIDS product• Update it promptlyUpdate it promptlyIf you are a userIf you are a user• Apply latest updates of your operate systemApply latest updates of your operate system• Try StackChecker to detect real-time buffer Try StackChecker to detect real-time buffer

overflow exploitoverflow exploit

The EndThe End