1

Introduction to Honeypot, Denial-of-Service, and Rootkit

Cliff C. ZouCAP6135

Spring, 2011

Acknowledgement

Some contents on honeypot are from http://staff.washington.edu/dittrich/talks/ar

o-honeynets.ppt

Some figures on DDoS are from http://www.cisco.com/web/IT/events/pdf/iin

2005/distributed_denial.pdf

2

3

What Is a Honeypot?

Abstract definition: “A honeypot is an information system resource whose value lies in unauthorized or illicit use of that resource.” (Lance Spitzner)

Concrete definition: “A honeypot is a faked vulnerable system used for the purpose of being attacked, probed, exploited and compromised.”

4

Example of a Simple Honeypot

Install vulnerable OS and software on a machine

Install monitor or IDS software Connect to the Internet (with global IP) Wait & monitor being scanned,

attacked, compromised Finish analysis, clean the machine

5

Benefit of Deploying Honeypots

Risk mitigation: Lure an attacker away from the real

production systems (“easy target“).

IDS-like functionality: Since no legitimate traffic should take

place to or from the honeypot, any traffic appearing is evil and can initiate further actions.

6

Benefit of Deploying Honeypots

Attack analysis: Find out reasons, and strategies why and

how you are attacked. Binary and behavior analysis of capture

malicious code Evidence:

Once the attacker is identified, all data captured may be used in a legal procedure.

Increased knowledge

7

Honeypot Classification

High-interaction honeypots A full and working OS is provided for being

attacked VMware virtual environment

Several VMware virtual hosts in one physical machine

Low-interaction honeypots Only emulate specific network services No real interaction or OS

Honeyd

Honeynet/honeyfarm A network of honeypots

8

Low-Interaction Honeypots

Pros: Easy to install (simple program) No risk (no vulnerable software to be attacked) One machine supports hundreds of honeypots,

covers hundreds of IP addresses Can distinguish most attacks on the same port

Cons: No real interaction to be captured

Limited logging/monitor function Hard to detect unknown attacks; hard to generate filters

Easily detectable by attackers

Emulation of Services

9

QUIT* )

echo -e "221 Goodbye.\r"

exit 0;;

SYST* )

echo -e "215 UNIX Type: L8\r"

;;

HELP* )

echo -e "214-The following commands are recognized (* =>'s unimplemented).\r"

echo -e " USER PORT STOR MSAM* RNTO NLST MKD CDUP\r"

echo -e " PASS PASV APPE MRSQ* ABOR SITE XMKD XCUP\r"

echo -e " ACCT* TYPE MLFL* MRCP* DELE SYST RMD STOU\r"

echo -e " SMNT* STRU MAIL* ALLO CWD STAT XRMD SIZE\r"

echo -e " REIN* MODE MSND* REST XCWD HELP PWD MDTM\r"

echo -e " QUIT RETR MSOM* RNFR LIST NOOP XPWD\r"

echo -e "214 Direct comments to ftp@$domain.\r"

;;

USER* )

10

High-Interaction Honeypots

Pros: Real OS, capture all attack traffic/actions Can discover unknown attacks/vulnerabilites Can capture and anlayze code behavior

Cons: Time-consuming to build/maintain Time-consuming to analysis attack Risk of being used as stepping stone High computer resource requirement

11

Honeynet A network of honeypots High-interaction honeynet

A distributed network composing many honeypots Low-interaction honeynet

Emulate a virtual network in one physical machine

Example: honeyd

Gen II Honeynet

12

Data Control

13

Internet

Honeywall

Honeypot

Honeypot

No Restrictions

Connections Limited Packet Scrubbed

Prevent a honeypot being used by attackers to attack others (legal/ethnical issues)

14

Honeypot-Aware Botnet [Zou’07]

Honeypot is widely used by defenders Ability to detect unknown attacks Ability to monitor attacker actions (e.g.,

botnet C&C)

Botnet attackers will adapt to honeypot defense When they feel the real threat from

honeypot We need to think one step ahead

15

Honeypot Detection Principles

Hardware/software specific honeypot detection Detect virtual environment via specific code

E.g., time response, memory address Detect faculty honeypot program Case by case detection

Detection based on fundamental difference Honeypot defenders are liable for attacks sending out

Liability law will become mature It’s a moral issue as well

Real attackers bear no liability Check whether a bot can send out malicious traffic or not

16

Detection of Honeypot Bot

Infection traffic Real liability to defenders No exposure issue: a bot needs to do this regardless

Other honeypot detection traffic Port scanning, email spam, web request (DoS?)

C&C

bot Sensor (secret)1 malicious traffic

2 Inform

bot’s IP3 Authorize

17

Two-stage Reconnaissance to Detect Honeypot in Constructing P2P Botnets

Fully distributed No central sensor is used Could be fooled by double-honeypot

Counterattack is presented in our paper

Lightweighted spearhead code Infect + honeypot detection Speedup UDP-based infection

Host A spearhead Host B

request

main-force

spearheadHost C

1

3

2

18

Defense against Honeypot-Aware Attacks

Permit dedicated honeypot detection systems to send out malicious traffic Need law and strict policy

Redirect outgoing traffic to a second honeypot Not effective for sensor-based honeypot detection

Figure out what outgoing traffic is for honeypot detection, and then allow it It could be very hard

Neverthless, honeypot is still a valuable monitoring and detection/defense tool

19

Distributed Denial of Service (DDoS) Attack

Send large amount of traffic to a server so that the server has no resource to serve normal users

Attacking format: Consume target memory/CPU resource

SYN flood (backscatter paper presented before) Database query…

Congest target Internet connection Many sources attack traffic overwhelm target link Very hard to defend

20

Why hard to defined DDoS attack?

Internet IP protocol has no built-in security No authentication of source IP

SYN flood with faked source IP However, IP is true after connection is setup

Servers are supposed to accept unsolicited service requests

Lack of collaboration ways among Internet community How can you ask an ISP in another country to

block certain traffic for you?

21

DDoS Defenses

Increase servers capacity Cluster of machine, Multi-CPUs, larger

Internet access Use Internet web caching service

E.g., Akamai Defense Methods (many in research

stage) SYN cookies (http://en.wikipedia.org/wiki/SYN_cookies)

SOS IP traceback

22

23

24

SYN Cookies

SYN flood attack Fill up server’s SYN queue Property: attacker does not respond to

SYN/ACK from victim. Defense

Fact: normal client responds to SYN/ACK Remove initial SYN queue Server encode info in TCP seq. number

Use it to reconstruct the initial SYN

25

DoS spoofed attack defense: IP traceback

Suppose a victim can call ISPs upstream to block certain traffic

SYN flood: which traffic to block? IP traceback:

Find out the real attacking host for SYN flood Based on large amount of attacking packets Need a little help from routers (packet

marking)

26

SOS: Secure Overlay Service

Central Idea: Use many TCP connection respondent machines Only setup connections relay to server Identity of server is secrete

The Evolution of Malware

Malware, including spyware, adware and viruses want to be hard to detect and/or hard to remove

Rootkits are a fast evolving technology to achieve these goals

Cloaking technology applied to malware Not malware by itself Example rootkit-based viruses: W32.Maslan.A@mm,

W32.Opasa@mm

Rootkit history Appeared as stealth viruses

One of the first known PC viruses, Brain, was stealth First “rootkit” appeared on SunOS in 1994

Replacement of core system utilities (ls, ps, etc.) to hide malware processes

Cloaking

Modern rootkits can cloak: Processes Services TCP/IP ports Files Registry keys User accounts

Several major rootkit technologies User-mode API filtering Kernel-mode API filtering Kernel-mode data structure manipulation Process hijacking

Visit www.rootkit.com for tools and information

Attack user-mode system query APIs

Con: can be bypassed by going directly to kernel-mode APIs

Pro: can infect unprivileged user accounts Examples: HackerDefender, Afx

Taskmgr.exeTaskmgr.exe Ntdll.dllNtdll.dll

Explorer.exe, Explorer.exe, Malware.exeMalware.exe, Winlogon.exe, Winlogon.exe

RootkitRootkit

Explorer.exe,Explorer.exe,Winlogon.exeWinlogon.exe

user modeuser mode

kernel modekernel mode

User-Mode API Filtering

Attack kernel-mode system query APIs

Cons: Requires admin privilege to install Difficult to write

Pro: very thorough cloak Example: NT Rootkit

Taskmgr.exeTaskmgr.exe Ntdll.dllNtdll.dll

user modeuser modekernel modekernel mode

RootkitRootkit

Explorer.exe,Explorer.exe,Winlogon.exeWinlogon.exe

Explorer.exe,Explorer.exe,Winlogon.exeWinlogon.exe

Explorer.exe, Explorer.exe, Malware.exe,Malware.exe, Winlogon.exeWinlogon.exe

Kernel-Mode API Filtering

Also called Direct Kernel Object Manipulation (DKOM)

Attacks active process data structure Query API doesn’t see the process Kernel still schedules process’ threads

Cons: Requires admin privilege to install Can cause crashes Detection already developed

Pro: more advanced variations possible Example: FU

Explorer.exeExplorer.exe Malware.exeMalware.exe Winlogon.exeWinlogon.exeActiveActive

ProcessesProcesses

Kernel-Mode Data Structure Manipulation

Hide inside a legitimate process

Con: doesn’t survive reboot Pro: extremely hard to detect Example: Code Red

Explorer.exeExplorer.exeMalwareMalware

Process Hijacking

Detecting Rootkits

All cloaks have holes Leave some APIs unfiltered Have detectable side effects Can’t cloak when OS is offline

Rootkit detection attacks holes Cat-and-mouse game Several examples

Microsoft Research Strider/Ghostbuster RKDetect Sysinternals RootkitRevealer F-Secure BlackLight

Perform a directory listing online and compare with secure alternate OS boot (see http://research.microsoft.com/rootkit/ ) Offline OS is Windows PE, ERD Commander, BartPEdir /s /ah * > dirscan.txt

windiff dirscanon.txt dirscanoff.txt

This won’t detect non-persistent rootkits that save to disk during shutdown

Simple Rootkit Detection

RootkitRevealerRootkitRevealer

RootkitRootkit

Windows APIWindows API

Raw file system, Raw file system, Raw Registry hiveRaw Registry hive

Filtered Windows API Filtered Windows API

omits malware files and keysomits malware files and keys Malware files and keys Malware files and keys are visible in raw scanare visible in raw scan

RootkitRevealer

RootkitRevealer (RKR) runs online RKR tries to bypass rootkit to uncover cloaked

objects All detectors listed do the same RKR scans HKLM\Software, HKLM\System and the file

system Performs Windows API scan and compares with raw

data structure scan

Demo

HackerDefender HackerDefender before and after view of file

system Detecting HackerDefender with

RootkitRevealer

Unless you have specific uninstall instructions from an authoritative source:

Don’t rely on “rename” functionality offered by some rootkit detectors It might not have detected all a rootkit’s

components The rename might not be effective

Reformat the system and reinstall Windows!Reformat the system and reinstall Windows!

Dealing with Rootkits