simon brooks 100042660 - dissertation - 2010-2011

126
University of Derby School of Computing and Mathematics A Project Completed as part of the Requirements for the BSc (Hons) in Computer Networks Entitled To determine efficiency of automated signature creation compared with human crafted signature creation By Simon Brooks In the Years 2010 -2011

Upload: simon-brooks

Post on 10-Feb-2017

172 views

Category:

Documents


12 download

TRANSCRIPT

Page 1: Simon Brooks 100042660 - Dissertation - 2010-2011

To determine efficiency of automated signature creation compared with human crafted signature creation 03 May 2011

Simon Brooks Page 1 of 126

University of Derby

School of Computing and Mathematics

A Project Completed as part of the Requirements for the

BSc (Hons) in Computer Networks

Entitled

To determine efficiency of automated signature creation

compared with human crafted signature creation

By

Simon Brooks

In the Years 2010 -2011

Page 2: Simon Brooks 100042660 - Dissertation - 2010-2011

To determine efficiency of automated signature creation compared with human crafted signature creation 3 May 2011

Simon Brooks Page 2 of 126

Table of Contents

Abstract .......................................................................................................................................7

Acknowledgements .....................................................................................................................7

1 Chapter One – Introduction .................................................................................................8

2 Chapter Two – Literature Review .......................................................................................9

2.1 The Threats Computer Systems Face in Today‟s World ............................................9

2.1.1 The Growth of the Public Internet .......................................................................9

2.2 What threats do computer systems face? ..................................................................10

2.2.1 What defines a Hacker? .....................................................................................10

2.2.2 Attacks on Networks .........................................................................................10

2.3 Network Intrusion Detection Systems VS Traditional Systems ...............................11

2.3.1 What is an Intrusion Detection System? ...........................................................11

2.3.2 What is the overall goal of an Intrusion Detection System? .............................12

2.3.3 What types of Intrusion Detection Systems exist? ............................................12

2.3.4 Which Detection Method is Better Anomaly-Based or Signature-Based? .......16

2.3.5 Signature Writing for Vulnerabilities and Exploits ...........................................16

2.3.6 What is a Traditional Based Network Security System?...................................19

2.3.7 An Overview of Firewall Systems ....................................................................19

2.3.8 An Overview of Anti-Virus and Malware Scanners .........................................21

2.3.9 Are Intrusion Detection Systems Better than Traditional Systems? .................22

2.4 Honeypot Technology ...............................................................................................23

2.4.1 What is a Honeypot? .........................................................................................23

2.4.2 Honeypots and Automated IDS Signature Generation ......................................24

2.5 Review of Research and Literature ...........................................................................25

3 Chapter Three – Research Methods ..................................................................................26

3.1 Introduction ...............................................................................................................26

3.2 Action Research ........................................................................................................27

3.3 Survey Based Research .............................................................................................27

3.4 Case Study Based Research ......................................................................................28

3.5 Experiment Based Research ......................................................................................28

3.6 Internet-based research ..............................................................................................29

3.7 Chosen Research Method ..........................................................................................30

4 Chapter Four – Conceptual Model of Problem Domain ...................................................31

4.1 Introduction ...............................................................................................................31

4.2 Choice of Honeypot and Automated Signature .........................................................32

4.2.1 Automated Signature Software .........................................................................32

Page 3: Simon Brooks 100042660 - Dissertation - 2010-2011

To determine efficiency of automated signature creation compared with human crafted signature creation 3 May 2011

Simon Brooks Page 3 of 126

4.2.2 Honeypots .........................................................................................................39

4.2.3 Conclusion .........................................................................................................41

4.3 Experiment Test bed Environment ............................................................................42

4.3.1 System Specification .........................................................................................42

4.3.2 Virtualization .....................................................................................................42

4.3.3 Conclusion .........................................................................................................45

4.4 Testing Environment Configuration ..........................................................................46

4.4.1 Honeyd ..............................................................................................................46

4.4.2 Attacker Machine System Configuration and Requirements ............................48

4.4.3 Intrusion Detection System Configuration and Requirements ..........................49

4.5 Experiment Test Bed .................................................................................................53

4.5.1 Experiment Part One .........................................................................................53

4.5.2 Experiment Part One Network Setup ................................................................53

4.5.3 Experiment Part Two Method ...........................................................................54

4.5.4 Experiment Part Two Network Setup ...............................................................55

4.6 Metasploit Attacks ....................................................................................................56

4.6.1 Windows 2000 Attacks .....................................................................................56

4.6.2 Windows XP Attacks ........................................................................................57

4.6.3 Metasploit Attack Overview .............................................................................59

4.6.4 Overview of the Experiment .............................................................................60

5 Chapter 5 –Analysis of Data Collected .............................................................................61

5.1 Introduction ...............................................................................................................61

5.2 Experiment One Results ............................................................................................61

5.2.1 Honeyd Pre-Tests ..............................................................................................61

5.2.2 Attack Results for Windows 2000 SP4 Attacks ................................................62

5.2.3 Attack Results for Windows XP Service Pack 2 attacks ...................................65

5.2.4 Withdrawal of Experiment Two ........................................................................68

5.3 Critical Evaluation of the Results ..............................................................................68

6 Chapter 6 – Conclusions and Recommendations ..............................................................69

6.1 Report Summary .......................................................................................................69

6.2 Aims and Objectives .................................................................................................70

6.3 Critique and Limitations............................................................................................70

6.4 Future work ...............................................................................................................71

7 Chapter 7 – Critical Evaluation .........................................................................................72

7.1 Time Management ....................................................................................................72

7.2 Research Skills ..........................................................................................................72

Page 4: Simon Brooks 100042660 - Dissertation - 2010-2011

To determine efficiency of automated signature creation compared with human crafted signature creation 3 May 2011

Simon Brooks Page 4 of 126

7.3 Practical and Technical Skills ...................................................................................72

7.4 Conclusion.................................................................................................................72

8 Works Cited ......................................................................................................................74

9 Appendices ........................................................................................................................82

9.1 Experiment Part One - Network Topology Diagram ................................................82

9.2 Experiment Part Two - Network Topology Diagram ................................................83

9.3 Overall Virtualization and Attack Topology Diagram ..............................................84

9.4 Metasploit Website Module Reference list ...............................................................85

9.5 Remaining Four Windows XP SP2 Attack Results...................................................86

9.5.1 ms08_067_netapi ..............................................................................................86

9.5.2 ms06_040_netapi ..............................................................................................86

9.5.3 ms04_031_netdde..............................................................................................87

9.5.4 MS04_011_lsass ...............................................................................................88

9.6 Honeyd and Honeycomb Installation Instructions ....................................................89

9.7 Honeyd + Honeycomb Default Configuration (Provos, 2003) (Andrade, 2009) ......91

9.8 Honeyd and Honeycomb Used in the Experiment ....................................................94

9.9 CentOS Screenshots ................................................................................................103

9.10 4Ethical Approval Form ..........................................................................................106

9.11 Dissertation Plan – February 2011 ..........................................................................115

9.12 Preliminary Structure – November 2010.................................................................116

9.13 Progress Sheets ........................................................................................................119

Page 5: Simon Brooks 100042660 - Dissertation - 2010-2011

To determine efficiency of automated signature creation compared with human crafted signature creation 3 May 2011

Simon Brooks Page 5 of 126

Table of Figures

Figure 1 - Growth in number of internet hosts (Internet Systems Consortium, 2010) ................9

Figure 2 - Attack Sophistication vs. Intruder Technical Knowledge (Carnegie Mellon

University, 2010).......................................................................................................................11

Figure 3 – NIDS Example (Magalhaes, 2006) ..........................................................................13

Figure 4 - HIDS Example (Magalhaes, 2006) ...........................................................................14

Figure 5 - Prelude Hybrid IDS Architecture (Yasm, 2009) ......................................................14

Figure 6 - Distributed Intrusion Detection System (Symantec , 2007) .....................................15

Figure 7 – Know the Pattern - Comparing samples of network traffic .....................................18

Figure 8 - OSI 7 Layer Reference Model (CISCO, 2003) ........................................................19

Figure 9 - Packet Filtering Firewall ..........................................................................................20

Figure 10 -Application Layer Firewall......................................................................................20

Figure 11 - Stateful Firewall .....................................................................................................21

Figure 12 - Outline of the Proposed Experiment ......................................................................31

Figure 13 - SweetBait Architecture Overview (Portokalidis et al., 2006) ................................32

Figure 14 – Overview of the Honeycyber architecture (Mohammed et al., 2009) ...................34

Figure 15 - Honeycyber - Signature Generation architecture (Mohammed et al., 2009) ..........34

Figure 16 - Overview of Honeycomb Architecture (Kreibich & Crowcroft, 2004) .................35

Figure 17 - Overview of Honeycomb signature generation algorithm (Kreibich & Crowcroft,

2004) .........................................................................................................................................36

Figure 18 - Honeycomb Horizontal Detection (Kreibich & Crowcroft, 2003) .........................37

Figure 19 - Honeycomb Vertical Detection (Kreibich & Crowcroft, 2003) .............................38

Figure 20 - HiHAT Overview Mode (HiHAT, 2007) ...............................................................39

Figure 21 - LaBrea installed on a Linux machine (Softpedia, 2006) ........................................40

Figure 22 - Honeyd Administration Interface running on CentOS ...........................................41

Figure 23 - Windows 7 - Laptop Specification .........................................................................42

Figure 24 - VirtualBox on running Ubuntu 10.10 VM on Windows 7 (Oracle, 2011) .............43

Figure 25 - Citrix XenServer 5.6.1 Home Screen (Softpedia, 2011) ........................................44

Figure 26 - VMware Player on Ubuntu Linux (TheTechJournal, 2010) ...................................44

Figure 27 - VMware Workstation 7 and Virtual Network Editor .............................................45

Figure 28 - netVigilance WinHoneyd Configurator (negVigilance, Inc., 2009) .......................46

Figure 29 - Ubuntu 10.04 LTS - Synaptic Package Manager - Listing Honeyd package details

...................................................................................................................................................47

Figure 30 - Honeyd Configuration File Sample ........................................................................47

Figure 31 - Exploit Database Archive (Offensive Security, 2011) ...........................................48

Figure 32 - Backtrack 4 R1 - Metasploit Console Mode ..........................................................49

Figure 33 - Metasploit Install on Windows XP Virtual Machine .............................................49

Figure 34 - Network Security Toolkit v2.13.0 - Snort Setup Page ...........................................50

Figure 35 - Experiment Part One Overview ..............................................................................53

Figure 36 - Experiment Part One - Network Topology Diagram ..............................................54

Figure 37 - Experiment Part Two Overview .............................................................................55

Figure 38 - Experiment Part Two - Network Topology Diagram .............................................55

Figure 39 - Overall Virtualization and Attack Topology Diagram ...........................................56

Figure 40 - Meterpreter Shell on Backtrack 4 (Makker, 2011) .................................................59

Figure 41 - Virtualization View and Attack Diagram ...............................................................60

Figure 42 – CentOS Honeyd start-up ........................................................................................61

Figure 43 - Attacker Ping to Honeyd Win2K Machine on CentOS ..........................................62

Figure 44 - Attacker Ping to Honeyd WinXP Machine on CentOS ..........................................62

Page 6: Simon Brooks 100042660 - Dissertation - 2010-2011

To determine efficiency of automated signature creation compared with human crafted signature creation 3 May 2011

Simon Brooks Page 6 of 126

Figure 45 - Honeyd Ping ICMP Echo Replies to Victim Machine ...........................................62

Figure 46 - ms06_040_netapi – Metasploit Results ..................................................................63

Figure 47 - ms06_040_netapi - Honeyd/Honeycomb CentOS Bash Console Results ..............63

Figure 48 - Windows 2000 SP4 - ms06_040_netapi - Honeycomb Signature..........................64

Figure 49 – ms05_039_pnp – Metasploit Results .....................................................................64

Figure 50 – Windows 2000 SP4 – ms05_039_pnp ...................................................................65

Figure 51 - ms03_026_dcom - Metasploit Results ...................................................................66

Figure 52 – Windows XP SP2 – ms03_026_dcom ...................................................................66

Figure 53 - ms03_049_netapi - Metasploit Results ..................................................................67

Figure 54 – Windows XP SP2 – ms03_049_netapi ..................................................................67

Figure 55 - MS08_067_netapi - Metasploit Results .................................................................86

Figure 56 – Windows XP SP2 – MS08_067_netapi .................................................................86

Figure 57 - MS06_040_netapi (XP) - Metasploit Results .........................................................86

Figure 58 – Windows XP SP2 – MS06_040_netapi .................................................................87

Figure 59 - MS04_031_netdde – Metasploit Results ................................................................87

Figure 60 – Windows XP SP2 – MS04_031_netdde ................................................................87

Figure 61 - MS04_011_lsass – Metasploit Results ...................................................................88

Figure 62 – Windows XP SP2 – MS04_011_lsass ...................................................................88

Figure 63 - CentOS Install Screen...........................................................................................103

Figure 64 - CentOS Loading Screen .......................................................................................103

Figure 65 - Honeycomb Configuration Overview Page ..........................................................104

Figure 66 - Honeycomb Error .................................................................................................104

Figure 67 - SNORT IDS on Network Security Toolkit1.........................................................105

Table of Table Figures

Table 1 - URL-encoding example .............................................................................................36

Table 2 - A basic Snort Rule Outline ........................................................................................51

Table 3 - A Hand Written Snort Rule for Slammer SQL Worm (Reid, 2003) ..........................51

Table 4 - Signature Honeycomb created for the Slammer Worm (Kreibich & Crowcroft, 2003)

...................................................................................................................................................52

Table 5 - Windows 2000 Exploits in Metasploit .......................................................................57

Table 6 - Successful Windows 2000 Exploits in Metasploit .....................................................57

Table 7 - Windows XP Exploits in Metasploit .........................................................................58

Table 8 – Successful Windows XP Exploits in Metasploit .......................................................59

Table 9- Testbed Network Configuration .................................................................................60

Page 7: Simon Brooks 100042660 - Dissertation - 2010-2011

To determine efficiency of automated signature creation compared with human crafted signature creation 3 May 2011

Simon Brooks Page 7 of 126

Abstract

Network intrusion detection systems are designed to identify network attacks that may occur

from the public internet or from inside the network infrastructure. Signature-based intrusion

detection uses signature files that identify a particular attack by comparing a set of rules

against traffic on a network. This research project aims to determine the efficiency of an

automated IDS signature against a human crafted signature, and is supported by a review of

literature that provides useful background information about this topic. An applicable research

method is used for this project with clear presentation of results. The final results are critically

analysed and an evaluation of the project is discussed.

Acknowledgements

I would like to thank my personal tutor David Day for providing help and support throughout

this project, for both his intellectual support and guidance.

A big thank you to Michael Hilton, IT Manager at Saint Benedict School, where I undertook

my 3rd year work placement and where the ideas and a passion for Linux, honeypots and

particularly network security, became the catalyst for this dissertation.

Congratulations and many thanks to the creator of Honeyd, Niels Provos, and architect of

Honeycomb, the Honeyd plug –in, Christian Kreibich. Both have created remarkable pieces of

software and I have learnt a great deal from their work. Thank you both for making your

software freely available, enabling me to undertake this project.

Heartfelt thanks to my parents for all their love and support over the years, without them I

would not have come this far.

Page 8: Simon Brooks 100042660 - Dissertation - 2010-2011

To determine efficiency of automated signature creation compared with human crafted signature creation 3 May 2011

Simon Brooks Page 8 of 126

1 Chapter One – Introduction

This research project aims to determine the efficiency of automated signature creation

compared with human crafted signature creation. This will be achieved through research

covering a range of topics from intrusion detection systems, through to honeypots. To

sufficiently meet the required aim, a number of objectives have been designed to measure how

well this project achieves its aims. The proposed objectives and how they fit into the chapters

of this research project are listed below.

Objectives 1.1 – Evaluate different methods of automated signature creation

Objectives 1.2 – Evaluate different philosophies of signature writing

Objectives 1.3 – Decide which methods/systems to compare

Objectives 1.4 – Design and implement a test bed

Objectives 1.5 – Analysis of results in accordance with Aim

Chapter 2 – Literature Review – This chapter identifies a range of literature

covering current trends in network threats, intrusion detection systems vs traditional

based systems, intrusion detection signature writing and honeypot technology. This

aims to cover objectives 1.1 and 1.2

Chapter 3- Research methods covers – This chapter identifies a range of research

methods available, based on academic literature and provides a discussion of why a

specific research method has been chosen over other research methods listed in this

chapter.

Chapter 4 – Conceptual Model of Problem Domain – This chapter highlights the

project‟s methodology in sufficient detail, critically identifies the reasons of choice

surrounding any methods used to create a test bed and aims to complete objectives 1.3

and 1.4

Chapter 5 – Analysis of Results – This chapter provides an overview of the data and

results that are generated from the experiment. It will provide a detailed analysis in

accordance with the main aim of this project, thus completing objective 1.5

Page 9: Simon Brooks 100042660 - Dissertation - 2010-2011

To determine efficiency of automated signature creation compared with human crafted signature creation 3 May 2011

Simon Brooks Page 9 of 126

2 Chapter Two – Literature Review

2.1 The Threats Computer Systems Face in Today’s World

In today‟s modern world, many people across the globe have access to a networked computer.

Businesses rely on the structure and security of their own computer networks and seek

protection against threats. Increasingly more and more people are communicating via the

internet, to share information and knowledge, as a result the world is becoming increasingly

reliant on computer systems, th1is truly is the digital age.

With the majority of the world connected to the public internet, there is a potential risk

involving the security of information both from the internet and on the inside from business

and home user computer networks. Malware, viruses, software vulnerabilities and network

attacks are just some of the security threats computers are currently facing. This chapter

briefly discusses the growth of the public internet and a momentary overview of overview of

what threats face our computer systems today.

2.1.1 The Growth of the Public Internet

(Stringer, 1999) describes the internet as a network of networks that communicate via the two

main core communication protocols; TCP and IP. These protocols have developed over time

to produce an entire suite of protocols, named the TCP/IP suite. The internet contains many

networks and amongst these networks sit servers running services such as, DNS, HTTP

(Web), SMTP (E-Mail) and others. Businesses allow access to their servers either through

public or private networks and home users can connect to this global internetwork of networks

through an Internet Service Provider (ISP).

Figure 1 - Growth in number of internet hosts (Internet Systems Consortium, 2010)

At the last count of internet hosts by the Internet Systems Consortium (ISC) in January 2010,

there were an estimated 769 million hosts connected to the internet.

Figure 1 shows the rise in internet hosts since January 1994. However as stated by (Stringer,

Page 10: Simon Brooks 100042660 - Dissertation - 2010-2011

To determine efficiency of automated signature creation compared with human crafted signature creation 3 May 2011

Simon Brooks Page 10 of 126

1999) “most internet-connected computers act only as clients; they access the data that is

stored on other servers, but don‟t store or share any information themselves”. Furthermore

(Stringer, 1999), adds that it is difficult to identify how many client computers actually exist

in cyberspace, because many clients connect dynamically or are hidden behind firewalls.

According to statistics from (Internet Usage Statistics, 2010), there were over 6,845,609,960

billion computers connected to the global internet. This is a dramatic change when compared

to the population of internet users as of December 31st 2000, which was only 360,985,492

million.

2.2 What threats do computer systems face?

Every day companies face the threat of possible severe damage to their computerized systems.

Many of these rely on computer networks to store credit card information, personal

information about their customers, and send electronic messages across the internet. As stated

by (Wolfe, 2007), “the goal of today‟s hackers is financial gain”, stealing valuable personal

data such as credit card information and financial account information has become very

popular amongst today‟s „hackers‟. This is in stark contrast when compared to historical

hackers that focused on gaining access and defacing websites such as the U.S Department of

Justice and U.S Air force (Trigaux, 2000).

2.2.1 What defines a Hacker?

The term “hacking” as quoted by (Sterling, 1992) is “the free-wheeling intellectual exploration

of the highest and deepest potential of computer systems.” Persons that carry out “Hacking”

are called hackers. As defined by (Crystal, 2010), the term hacker has two meanings; an expert

computer programmer who creates complex software and hardware, and someone who breaks

into computer security networks for his/her own purpose. These two types of “hackers” are

identified by (Clarke & Tetz, 2010) as White-hat hackers and Black-hat hackers. The Black-

hat hackers are the ones who break into systems for financial or personal gain, and the White-

hat hackers aim to “hack” the software and hardware in order to protect the systems from the

Black-hat hackers.

2.2.2 Attacks on Networks

Attacks have evolved over the past twenty years. Figure 2 is a graph depicting the decline in

the average knowledge of an intruder, and a rise in the sophistication of attacks over the past

twenty years. The increase in the availability of public knowledge could account for why the

level of attack sophistication has increased. As Figure 2 shows, the knowledge required to

carry out an attack has reduced but the amount of sophistication in how they are implemented

has increased.

Page 11: Simon Brooks 100042660 - Dissertation - 2010-2011

To determine efficiency of automated signature creation compared with human crafted signature creation 3 May 2011

Simon Brooks Page 11 of 126

Figure 2 - Attack Sophistication vs. Intruder Technical Knowledge (Carnegie Mellon University, 2010)

Attackers are using more sophisticated technology to attack systems, importantly, attacks are

moving towards malware, coordinated cyber-physical attacks and adaptive, high-impact,

target attacks on critical infrastructures.

2.3 Network Intrusion Detection Systems VS Traditional Systems

In the previous section 2.2, the nature of attacks has been identified as increasing in

sophistication; Black-hat Hackers have worked towards generating financial gain when

deploying network attacks. The main topic of this next section is to identify the concept of

Intrusion Detection systems and how they compare with the traditional systems that exist

currently. This section poses the question; Are Intrusion Detection Systems Better than

Traditional Systems?

2.3.1 What is an Intrusion Detection System?

As described by (Scarfone & Mell, 2007), Intrusion detection is the process of monitoring the

events occurring in a computer system or network and analysing them for signs of possible

incidents. When referring to intrusion detection as a system (Rexworthy, 2009), states that an

intrusion detection system is a system that is designed to detect malicious activity such as DoS

(Denial of Service) attacks, port scans and attempts to crack into computers.

Before intrusion detections existed, system administrators would monitor the audit logs of

computer systems and look for suspicious behaviour. Searching through these audit logs

proved to be too time consuming and once audit logs became available online, researchers

developed programs to analyse the data. (Kemmerer & Giovanni, 2002). In 1987, (Denning,

Page 12: Simon Brooks 100042660 - Dissertation - 2010-2011

To determine efficiency of automated signature creation compared with human crafted signature creation 3 May 2011

Simon Brooks Page 12 of 126

1987) created a model for a real-time intrusion-detection system that would be capable of

detecting break-ins, penetrations, and other forms of computer abuse.

2.3.2 What is the overall goal of an Intrusion Detection System?

As defined by (Mukherjee et al., 2002), the goal of intrusion detection is to identify, preferably

in real time, unauthorized use, misuse and abuse of computer systems by both system insiders

and external penetrators. The basic components that structure IDS are; a sensor, a traffic

analyser and a user interface. Intrusion detection systems can have multiple sensors to detect

and collect multiple points of data.

An intrusion detection system with one or more sensors would be suitable for monitoring two

or more separate network subnets. The analyser component receives input from one or more of

the sensors, or can receive information from another analyser. It is the job of the analyser to

determine if an intrusion has occurred and notify the user. Finally, the user interface produces

notification alarms and displays a list of intrusions that have been identified. Administrators

can also use the user interface to control the behaviour of the system (Allen et al., 2000).

2.3.2.1 Notification Alarms

As previously described, IDSs produce notification alarms when malicious behaviour is

detected. There are four types of notification alarms as defined by (DeLaet & Schauwers,

2004), and they are:

False-positive alarm – a false-positive alarm occurs when an IDS identifies traffic as

malicious when the nature of the traffic is benign. This results in the administrator

trying to locate the malicious traffic, when it is in fact not malicious at all.

False-negative alarm – a false-negative alarm is dangerous because the IDS does not

report the attack as malicious, therefore allowing the attack to take place.

True-positive alarm – the opposite of a false-positive alarm and an important

notification for an administrator

True-negative alarm – this is not an actual alarm but a situation whereby the IDS

does not trigger an alarm for activity permitted within a network

2.3.3 What types of Intrusion Detection Systems exist?

As highlighted in section 2.1, there is a rising threat in network based attacks towards systems,

these attacks can be aimed towards the outside of the network by using network worms, or

infected e-mails, or even by internal users that may attempt to gain unauthorised access

towards precious file servers. The following points describe the types of IDS systems in which

intrusion detection systems are classified.

Page 13: Simon Brooks 100042660 - Dissertation - 2010-2011

To determine efficiency of automated signature creation compared with human crafted signature creation 3 May 2011

Simon Brooks Page 13 of 126

2.3.3.1 Active and Passive Intrusion Detection

Intrusion detection systems are designed to identify intrusions that may harm a network host,

the way in which they deter the intrusion differs. Below are two types of approach used by

IDS systems to deal with intrusions:

Passive Intrusion Detection – A passive intrusion detection system will monitor and

analyse any traffic activity and alert the administrator of attacks or potential

vulnerabilities. They offer no automated management for blocking any attempts.

Passive IDS systems are considered easier to deploy and less likely to get

compromised by an attacker because they are less susceptible to such attacks (Miller

& Gregory, 2009).

Active Intrusion Detection – An active IDS differs greatly from a passive IDS and is

more commonly known as an intrusion prevention system or IPS. Like the passive IDS

it can monitor network traffic but will automatically block any intrusions against the

network. This reduces the need for action to take place but heightens the risk of the

system blocking important services. Unlike passive systems, active IDS systems are

susceptible to attacks (deleted this is untrue) (Miller & Gregory, 2009).

These two types of detection systems offer automated and non-automated ways of dealing

with intrusions that affect network hosts. Essentially passive IDS systems will only detect the

intrusion whereby an active IDS or IPS will physically stop the intrusion from occurring. Both

types of systems offer advantages and disadvantages, and are suited for different types of

networks.

2.3.3.2 Network-Based Intrusion Detection (NIDS)

Figure 3 – NIDS Example (Magalhaes, 2006)

A network based intrusion detection system, focuses on monitoring the current situation of the

network. Figure 3 illustrates a typical NIDS layout, monitoring traffic that flows from each of

the hosts (PC, Workstation and Laptop) listed on the network diagram. Essentially, an IDS,

provides a type of surveillance across the network, ensuring that any notifications are made if

Page 14: Simon Brooks 100042660 - Dissertation - 2010-2011

To determine efficiency of automated signature creation compared with human crafted signature creation 3 May 2011

Simon Brooks Page 14 of 126

an attack or intrusion is detected on the network wire (Allen et al., 2000). Snort, is a popular

open source example of a network intrusion detection system. This system supports signature,

protocol and anomaly-based inspection, currently having more than 300,000 registered users

in the Snort community (Sourcefire, 2011).

2.3.3.3 Host-Based Intrusion Detection (HIDS)

Figure 4 - HIDS Example (Magalhaes, 2006)

A host based intrusion detection system, concentrates on protecting the file system and system

kernel of the operating system. The system kernel of an operating system, acts as a bridge

between an application and data processing that is carried out with the hardware (Wulf et al.,

1974). A malicious piece of software will focus on distributing corruptive data processing to

the system kernel that may alter the functions of the operating system. HIDS aim to identify

such anomalies by reducing the risk and increasing system integrity (Kozushko, 2003).

Figure 1Figure 4 is a modified network diagram of NIDS, with HIDS hosts displayed in gold

A notable HIDS product is Tripwire Enterprise, an integrity file monitoring solution that is

suitable for providing features such as file integrity monitoring across an IT Enterprise

infrastructure (Tripwire, 2011).

2.3.3.4 Hybrid Intrusion Detection (HIDS)

Figure 5 - Prelude Hybrid IDS Architecture (Yasm, 2009)

A hybrid IDS system is not one physical system altogether but a collection of network IDS

sensors and host IDS sensors. These sensors are controlled via a centralised console where

Page 15: Simon Brooks 100042660 - Dissertation - 2010-2011

To determine efficiency of automated signature creation compared with human crafted signature creation 3 May 2011

Simon Brooks Page 15 of 126

administrators can add sensors and monitor any alerts that may occur. As described by (Yasm,

2009), Prelude is a centralised console that can support many different types of network

devices and „third party sensors‟ including Samhain. An open source host-based intrusion

detection system, that supports file integrity checking and log file monitoring/analysis

(Wichmann, 2006), and the open source intrusion detection system, Snort IDS. The diagram in

Figure 5, displays an installation of Prelude‟s management server console and three different

sensors: firewall, Snort and Syslog, a system logging service for Linux (linux.die.net, 2011).

2.3.3.5 Distributed Intrusion Detection (DIDS)

Figure 6 - Distributed Intrusion Detection System (Symantec , 2007)

A distributed intrusion detection system is made up of a collection of intrusion detection

systems that communicate with one another. Additionally the DIDS can communicate to a

centralized server that facilitates advanced network monitoring, incident analysis, and instant

attack data (Einwechter, 2001). Problems can arise with this method of intrusion detection.

The central analyser, (depicted as DIDS control centre agent in Figure 6), is a single point of

failure, this becomes the main vulnerability of the system. If an intruder can take advantage of

this vulnerability, the outcome would be that the system would cease to work properly,

therefore putting the whole network at risk and with no protection (Balasubramaniyan et al.,

1998).

Page 16: Simon Brooks 100042660 - Dissertation - 2010-2011

To determine efficiency of automated signature creation compared with human crafted signature creation 3 May 2011

Simon Brooks Page 16 of 126

2.3.4 Which Detection Method is Better Anomaly-Based or Signature-Based?

Both network-intrusion detection systems and host-intrusion detection systems can operate in

one of two detection modes:

Anomaly-based detection – An Anomaly-based IDS uses a baseline of everyday

traffic considered to be non-threatening, to compare any irregularities that may be

considered to be an intrusion (Depren et al., 2005). This method of detection is

considered an advantage when compared with the constant updates Signature-based

detection requires. However, well known attacks may not be detected if they do not fit

the profile of „strange traffic (Chebrolu et al., 2004).

Signature-Based detection – A signature-based IDS, attempts to identify attacks

based on defined patterns or „signatures‟. Typically these signatures are located in a

signature database and require a regular daily update to ensure that the IDS is up to

date with the latest threats. A disadvantage of signature based-detection is it requires

expert knowledge to create bespoke signatures for specific intrusions (Ghorbani et al.,

2009).

2.3.5 Signature Writing for Vulnerabilities and Exploits

Signature-based intrusion detection systems originally used exploit signatures. Writing for an

exploit involves identifying a specific attack for a vulnerable piece of software or service.

There are often many exploits that exist for a single vulnerability (Schear et al., 2008).

However, recently an alternative is to write a signature based on the actual vulnerability of the

software application or system in place. Writing for the vulnerability can reduce the amount of

signatures that need to be written for the piece of software or security risk (Brumley et al.,

2007).

2.3.5.1 What is a Vulnerability?

A vulnerability is a piece of software or computing method that is at risk from exploitation

from a bug or network attack (SecPoint, 2011). An example of a vulnerability is a password

containing all lowercase letters e.g. “hello”, having all lowercase letters would make the

password weak in security and would be vulnerable to a brute force attack. A brute force

attack is an attempt to try every combination of letters and potentially guess a password e.g.

aaaaa, aaaab, aaaac, aaaad – until reaching the final combination.

The weak password may also be vulnerable to a dictionary attack, this involves trying to guess

the password against a text file containing common words found in the dictionary. Providing

the word “hello” was located in the dictionary file, there would be a high chance the two

words would match. To fix this vulnerability the use of strong password combinations of

Page 17: Simon Brooks 100042660 - Dissertation - 2010-2011

To determine efficiency of automated signature creation compared with human crafted signature creation 3 May 2011

Simon Brooks Page 17 of 126

numbers letters and symbols would reduce the risk and lengthen the amount of time taken to

guess the password (Leggett, 2005).

Additionally vulnerabilities could be found in a software bugs, an example of a software bug

is a buffer overflow weakness. A buffer is a piece of temporary storage, located in computer

memory and used by software to store a finite amount of data. When too much data is sent to

this area of memory, the extra data can overflow into another buffer potentially overwriting

the information stored. To exploit this vulnerability an attacker can send malicious code to the

buffer overflow and execute the malicious code (Kramer, 2001).

Other vulnerabilities exist within the scope of computer security, a notable one being a SQL

injection attack, which is a type of buffer overflow that affects the SQL database query

language. The attack is attempted by using methods to trick an SQL application to give out

information located in tables and bypassing the use of login credentials. Information should

not be gained without username and password access to the database if the database is secured

(Friedl, 2007). SQL injection attacks are a risk within website logins that use an SQL database

as a back end. By ensuring that these web applications are properly tested against SQL

injection attacks and hardened by security, the risk of exploitation can be reduced.

2.3.5.2 What is an exploit?

In contrast to vulnerabilities, exploits are attacks that take advantage of the specific

vulnerability that may or may not exist on a target system. To take advantage of a

vulnerability an attacker may need a great deal of preparation to find out information that may

be useful in attempting to launch an exploit (Secpoint, 2011). For example, if a victim

machine is running a particular operating system such as Windows XP SP2, it may be useful

for an attacker to scan the system to identify the identity of the operating system. A popular

scanning tool called Nmap has the ability to map out the identity of an operating system and

any network ports TCP or UDP that may be open on a system. This concept is known as port

scanning and OS detection (Lyon, 2011).

From this information the attacker is now aware of the operating system that the victim

machine is running and is also aware of all of the open ports that are running on that machine.

This technique can be adapted to identify certain services that may be running such as a

HTTP web server (port 80) or mail server SMTP (port 25) and from there identify the versions

of these services for example Apache Tomcat, a popular webserver (Apache, 2011). With this

information the attacker can then potentially search the internet for vulnerabilities within that

software, service, or operating system. They are then able via the internet or an exploit

database such as Offensive Security (Offensive Security, 2011) to find a suitable exploit to

launch at the potential victim machine.

Page 18: Simon Brooks 100042660 - Dissertation - 2010-2011

To determine efficiency of automated signature creation compared with human crafted signature creation 3 May 2011

Simon Brooks Page 18 of 126

2.3.5.3 Writing Signatures for Exploits and Vulnerabilities

Signature based intrusion detection can identify exploits and vulnerabilities and there are two

methods of signature writing referred to by (Trost, 2010), the “Know the pattern” and “Know

the Vulnerability” schools of thought. The “Know the Pattern” focuses on writing signatures

based on the data found within them. An exploit may be notated by a specific string of

characters such as “ABCD”£$”£RRDW”, or by another characteristic such as size of the data

packet in bytes.

Figure 7 – Know the Pattern - Comparing samples of network traffic

To create a signature for a specific exploit analyst using the “Know the pattern” route, two

samples of network traffic flowing to that particular piece of exploited software are compared,

Figure 7 depicts this method used by an analyst. Traffic samples of “normal” data are

collected (i.e. traffic that does not contain any abnormalities concerning application layer

protocols and is expected to be sent to that piece of software), and compared against samples

of data that are known to be malicious in content towards that piece of software. On

comparison of the two types of traffic a process of elimination can be taken to figure out the

main specifics that will identify that particular exploit. This is usually a process that has to be

refined over a period of time and it is this that makes signature writing a time consuming

process.

The second method the “Know the Vulnerability” defined by (Trost, 2010), focuses on

identifying how the vulnerability is triggered. Writing signatures for vulnerabilities is

considered more effective than exploit based signatures because fewer signatures will need to

be written, one vulnerability signature may prevent against unknown exploit attacks. In

contrast exploit signatures are still useful for detecting exploits that may be for an unknown

vulnerability (Snort.org, 2009).

Page 19: Simon Brooks 100042660 - Dissertation - 2010-2011

To determine efficiency of automated signature creation compared with human crafted signature creation 3 May 2011

Simon Brooks Page 19 of 126

2.3.6 What is a Traditional Based Network Security System?

As explained in 2.3.1, Intrusion Detection Systems focus on monitoring the networks and

alerting network administrator to any intrusions that exist. This section identifies network

security systems that are considered traditional such as firewalls, malware and anti-virus

scanners. There are a number of risks associated with being connected to the internet such as

viruses, worms, trojans, phishing attacks and malicious software (Daya, 2009).

Figure 8 - OSI 7 Layer Reference Model (CISCO, 2003)

The seven layer OSI model is a network communications framework whereby protocols are

implemented among these layers (Webopedia, 2010). Figure 8 displays the full OSI Model.

The important layers in regard to traditional systems are: Layer 3 the Network layer and layer

7 the application layer. The Network layer provides a means of communication between two

systems to establish maintain and terminate a network connection. Packet Filter Firewalls

operate at this level of the OSI Model. Layer 7, the Application layer is where end user

communication occurs and where protocols such as telnet and ftp operate. Application layer

firewalls can view data at this level of the OSI mode (Briscoe, 2000).

2.3.7 An Overview of Firewall Systems

As explained by (Bautts et al., 2005), a firewall is a physical device that is located typically at

the point of entry to the network. The purpose of a firewall is to control the flow of traffic

from one network, typically a public network such as the internet, to the private network.

Traffic is either accepted into the network or dropped based on certain rules. There are three

types of firewall that are defined in this section, Packet Filters, Application Layer Firewalls

and „Stateful‟ firewalls. The main difference between these firewalls is how they make their

decisions to accept or deny packets of information. Firewalls that operate at the lower levels of

the OSI model are less likely to identify application level attacks due to their lack of

knowledge concerning higher level protocols.

Page 20: Simon Brooks 100042660 - Dissertation - 2010-2011

To determine efficiency of automated signature creation compared with human crafted signature creation 3 May 2011

Simon Brooks Page 20 of 126

2.3.7.1 Packet Filtering Firewall

Figure 9 - Packet Filtering Firewall

A packet filtering firewall operates at layer 3 of the OSI model and searches through an IP

packet header to determine if a packet is safe to pass through. An IP packet header contains

information about the source address, destination address, source, and destination port. The

header also holds information about the protocol the packet represents such as TCP, UDP or

ICMP (Morton, 1997). To filter the traffic special rules are created to block the traffic, Figure

9 depicts an example of a packet filtering firewall. A rule has been created to allow a specific

source address of 80.21.1.78 into the network but blocks any packet using the UDP protocol.

When a packet reaches this firewall, the rule is taken into effect and the packet is dropped.

2.3.7.2 Application Layer Firewall

Figure 10 -Application Layer Firewall

The next type of firewall is an application layer firewalls. An application layer gateway or

proxy is a firewall that operates up to layer 7 of the OSI model. At this layer of the OSI model

the firewall is capable of reading IP packet headers but also the data that is stored in the

packet. Proxy firewalls are useful for monitoring certain ports such as port 80/443

HTTP/HTTPS.

One way of avoiding the application layer firewall identifying malicious traffic is to

masquerade it as HTTP traffic. This can be done using a method known as URL-encoding. By

encoding traffic into code that looks like HTTP traffic, an attempt can be made to trick the

firewall analyser. Providing the HTTP traffic looks genuine. Other methods of avoiding the

firewall including IP fragmentation, a method used to break down packets for ease of

Page 21: Simon Brooks 100042660 - Dissertation - 2010-2011

To determine efficiency of automated signature creation compared with human crafted signature creation 3 May 2011

Simon Brooks Page 21 of 126

transmission. Essentially these techniques of attack can cause denial of service attacks against

the firewalls. The benefit of an application layer firewall is that it can detect certain attacks

that occur at the higher layer 7 layer (Eubanks, 2005). Figure 10 depicts an example of an

application layer firewall.

2.3.7.3 Stateful Firewall

Figure 11 - Stateful Firewall

The last type of firewall, stateful, is a relatively new type of firewall technology compared to

packet filtering and proxy. A stateful firewall is considered as having the functionality of a

packet filter and the knowledge of an application-level firewall. When monitoring

connections, the firewall is aware of the state of the connection and can make decisions based

on layers 3-4 and up to layer 7. Figure 11 is an example of a stateful firewall, when a request

is made to the web server google.com, the server response is allowed through the firewall.

When a suspicious server makes a request to the host, the connection is refused because a

connection to that server has not been initiated (Northcutt et al., 2005).

2.3.8 An Overview of Anti-Virus and Malware Scanners

As defined by (Solomon, 1995), a virus is a written program that replicates itself across a

computer hard drive, either over a network or a form of portable media. Anti-virus scanners

are designed to search through the computer host to find these infections. Similarly to

signature-based intrusion detection systems, anti-virus scanners use a database of signatures to

identify current threats. Anti-virus scanners are designed for host systems and tend not to be

dedicated systems. An example of a popular Virus Scanner is Kaspersky Anti-Virus 2011

(Kaspersky Lab, 2011).

Malware is short for malicious software and includes viruses, worms, trojan horses, spyware

and rootkits (Security4web, 2011). The intent of malware is usually to produce harm to the

target machine such as: delete sensitive files, infect other computers, monitor keyboard strokes

and execute exploit attacks. Malware scanners operate like anti-virus scanners and scan the

machine to try to detect any malicious software that is located on the host machine (Skoudis,

2003).

Page 22: Simon Brooks 100042660 - Dissertation - 2010-2011

To determine efficiency of automated signature creation compared with human crafted signature creation 3 May 2011

Simon Brooks Page 22 of 126

The disadvantage of both of these systems is that they rely on signatures to provide them with

the intelligence of any new malware. If the database is not updated, the host is at risk from

being infected. Anti-virus software is typically combined with anti-malware software to

produce „total-protection‟ systems aimed at protecting the home user against internet threats.

2.3.9 Are Intrusion Detection Systems Better than Traditional Systems?

When implementing security wide protection across an entire network, it is important that

every aspect of the network is secure (Daya, 2009). As explained in section 2.1.5, intrusion

detection systems are designed to identify any unauthorised use and misuse of computer

systems, particularly inside a network infrastructure.

Firewalls are designed to prevent the misuse of computer systems such as viruses and worm

programs but are limited depending on the type of firewall that is selected. When compared to

a packet filtering firewalls, intrusion detection systems can make intelligent decisions on what

is bad traffic because IDS can operate at layer 7. Application layer firewalls can identify the

same layer of traffic but do not offer the same amount of coverage across a network as

intrusion detection systems.

Anti-virus and malware scanners are designed to keep the host protected from security

breaches such as viruses and malware. Intrusion detection systems and firewalls are also

available for the host. Just like signature-based intrusion detection systems, anti-virus and

malware scanners require up to date signatures in order to identify the latest threats.

In conclusion, traditional systems offer a realistic amount of protection against threats.

Firewalls are designed to protect the boundaries between networks and can offer the same

level of application protection as an intrusion detection system when a layer 7 capable firewall

is used. Nevertheless, intrusion detection systems provide that extra inside protection against

attacks that may occur inside the network. Anti-virus and malware scanners provide an

adequate level of protection providing that the signatures are updated regularly but are limited

to host protection. Traditional systems should be used in conjunction with intrusion detection

systems to provide a “layered” approach to security (Daya, 2009).

Page 23: Simon Brooks 100042660 - Dissertation - 2010-2011

To determine efficiency of automated signature creation compared with human crafted signature creation 3 May 2011

Simon Brooks Page 23 of 126

2.4 Honeypot Technology

As discussed in the previous section, traditional systems such as anti-virus and firewalls aim

to stop attacks and infections from accessing and potentially ruining data. Intrusion detection

systems provide detection and an identification of the threats that have occurred on a network

or a host. Both systems offer information to identify the attack but provide little explanation as

to:

Where the attacker originated from? – The IP address of a security breach can be

found in IDS/Firewall logs, but identifying the true geographical nature of the

attacker is increasingly difficult due to Internet Anonymity (Bassi, 2005).

How the attacker attempted/gained access to the network? – Little information is

offered from IDS logs as to how an attack has taken place and what tools an attacker

may have used because they are designed to identify and prevent attacks (Balzarotti,

2006).

An attacker in this context of this section refers to “Black Hat” hackers as described in section

2.2.1 Honeypot technology is an area of computer security that focuses on the idea of learning

the tools and motives of “Black Hat” hackers, to aid in the defence against them (Veysset &

Laurent, 2006). The impression behind this is that, if more is known about the attacker, such

as the methods he or she uses, the better prepared a network administrator is likely to be.

2.4.1 What is a Honeypot?

The aim of the honeypot is to entice a “Hacker” in towards a trap, the honeypot system is

designed to look and act like a real machine with potential vulnerabilities but has the

capabilities of logging any activity that occurs. The benefit of collecting this information is

that it can be used to learn how intruders gain access to systems and gain insight into

intruder‟s attack methodologies. Once this knowledge has been obtained it can be used to

better protect and harden the systems that exist in the network (Even, 2000).

Honeypot systems can be divided into two separate types that offer the same logging

capabilities but differ in regards to functionality from a “Hackers” perspective. The systems

are divided into:

High-Interaction Honeypots – High-interaction honeypots provide the attacker with

full functionality and match that of a physical system. Virtualization is one of the

desired methods used to create high-interaction honeypots due to the simplicity of

creating virtualized systems. However hardware limitations can affect the volume of

honeypots that can be generated (Chazarain et al., 2008).

Page 24: Simon Brooks 100042660 - Dissertation - 2010-2011

To determine efficiency of automated signature creation compared with human crafted signature creation 3 May 2011

Simon Brooks Page 24 of 126

Low-Interaction Honeypots – Low-interaction honeypots do not have the full

operating system functionality that high-interaction honeypots have. They simulate

services and offer methods of fooling the attacker into believing “real” system exists.

Due to the reduction in computing resources low-interaction honeypots can be

replicated to mimic entire networks consisting of network devices, operating systems

and services (Provos, 2007).

A high-interaction honeypot will provide a full blown system for an attacker to compromise

and potentially destroy by means of attack. The risk of running a high-interaction honeypot is

higher than that of a low-interaction honeypot. Once a high-interaction honeypot is

compromised an attacker has the potential to move the attack towards other systems that may

not be honeypots, putting the rest of the network at risk (Provos & Holz, 2007).

Due to the nature of low-interaction honeypots only simulating network services and

protocols, the risk of fully compromising the system is reduced. However this can be

considered a disadvantage because an attacker may expect certain functionalities such as a

Command Shell and could arouse suspicion as to the authenticity of the honeypot he/she is

attacking (Provos & Thorsten, 2007).

2.4.2 Honeypots and Automated IDS Signature Generation

Signature writing for intrusion detection systems requires knowledge of the

vulnerability/exploit but more importantly human input to compose a signature with the least

amount of false-positives and false-negative alerts. Honeypot technology has been used in

conjunction with intrusion detection systems to create security systems that can automate the

signature-writing process. By using the concept of a honeypot to gather attacker information,

these systems can automatically generate working IDS signatures based on the traffic from the

honeypot.

The following are some examples of automated signature based systems:

Honeycyber – Honeycyber is an automated signature system designed to create

signatures for zero-day polymorphic worms. A “double Honeynet” system is used and

can automatically detect new worms and isolate the attack traffic at the same time.

The systems is said to create signatures for most polymorphic worms with low false-

positives and low false-negatives (Mohammed et al., 2009).

Honeycomb – Honeycomb uses a combination of protocol conformance and pattern

matching techniques to automatically generate attack signatures. This program has

been designed to “plug-in” to the Honeypot software Honeyd and processes any

traffic flowing to a honeypot created by the software. Honeycomb‟s attack signatures

Page 25: Simon Brooks 100042660 - Dissertation - 2010-2011

To determine efficiency of automated signature creation compared with human crafted signature creation 3 May 2011

Simon Brooks Page 25 of 126

are compatible with the Snort and Bro signature languages (Kreibich & Crowcroft,

2004).

SweetBait – SweetBait is an “automated protection system” that uses low-interaction

and high-interaction honeypots to collect potential malicious traffic. Patterns of

traffic are compared against a neutral white list. If these patterns do not match any in

the white list, a signature is automatically generated. This system is currently

deployed on medium sized educational networks and focuses on generating signatures

for zero day worms (Portokalidis & Bos, 2005).

A worm is a malicious program that spreads automatically among hosts by exploiting a

vulnerability that may appear on the targeted host. Polymorphic worms are able to mutate their

code by using techniques such as encryption and code transformations as they spread (Kruegel

et al., 2005). By changing the variation of the code, polymorphic worms aim to fool intrusion

detection systems. Generating signatures via human methods can take up an increasing

amount of time. Zero-day worms are malicious programs that are not yet identifiable by an

intrusion detection system due to them being relatively unknown (Akritidis et al., 2005).

These three automated systems make use of honeypot technology in order to trick malicious

worms into “attacking” the systems. Each system deals with a different type of worm and is

perceived to automatically generate a signature that supposedly produces accurate results.

2.5 Review of Research and Literature

This chapter has focused on the security of networks with particular reference to intrusion

detection systems and how they compare with traditional systems such as firewalls and anti-

virus technology. From the research undertaken it can be said that traditional systems provide

good defence for a network host and provide protection at the edge of the network. Intrusion

detection technology provides solid protection across the inside of the network providing that

the anomaly-based baseline is accurate and that signatures are correctly written and updated to

identify attacks and threats.

This chapter acknowledged the use of honeypots to recognise malicious traffic and

automatically generate signatures for signature-based intrusion detection. From the research it

was assumed that creating signatures by hand was a difficult and time consuming process and

these systems could automatically generate signatures to speed up the hand written process

and provide protection against the more sophisticated worm programs such as polymorphic

and zero-day worms. This report aims to determine the efficiency of automated signature

creation compared with human crafted signature creation. The next stage of the report will

focus on providing a methodology that is suitable for achieving this aim.

Page 26: Simon Brooks 100042660 - Dissertation - 2010-2011

To determine efficiency of automated signature creation compared with human crafted signature creation 3 May 2011

Simon Brooks Page 26 of 126

3 Chapter Three – Research Methods

3.1 Introduction

The previous chapter identified a type of intrusion detection method called signature-based

detection. From the research it was recognized that signatures required professional analysts to

create accurate signatures. Often this process could take a long time and result in a period of

time when a system was not protected against zero-day attacks. The use of honeypot

technology has seen the creation of automated signature generation systems.

The subsequent section identifies and analyses a selection of research methods available to

answer this research question: Do these automated systems provide better efficiency when

compared with human crafted signatures? After identifying and analysing each method, a

suitable choice will be made to carry out the research needed to answer this question.

(Oates, 2005), discusses six research strategies; survey, design and creation, experiment, case

study, action research and ethnography. As stated by (Cornford & Smithson, 2005), there are

three “broad styles” of research that exist. These three styles of research are; constructive

research methods, nomothetic research methods and idiographic research methods. Within

these styles of research methods sit additional styles associated with them, they are as follows:

Constructive research methods

Technical development

Conceptual development

Nomothetic research methods

Formal-mathematical analysis

Experimentation

Surveys

Idiographic research methods

Case studies

Action research

The following methods will be looked at to determine if they are suitable to be used to answer

the project question: action research, surveys, case studies, experimentation and internet based

research.

Page 27: Simon Brooks 100042660 - Dissertation - 2010-2011

To determine efficiency of automated signature creation compared with human crafted signature creation 3 May 2011

Simon Brooks Page 27 of 126

3.2 Action Research

In relation to the three “broad” styles as defined by (Cornford & Smithson, 2005) in the

introduction, action research falls under the idiographic method of research. The research

involves studying a situation and allowing an effect of change at the same time. As defined by

(Oates, 2005), Action Research is characterized by:

Concentration on practical issues – Instead of following experiments in a lab

environment this characterization of action research focuses towards the concerns and

complex problems expressed by people living in the real world.

Iterative cycle of plan-act-reflect – The researchers plan an action, carry out the action

and then reflect upon what has happened. Following this, the researchers then repeat the

cycle.

Emphasis on change – Using Action research, the researchers are concerned with doing

things that make a change and learning about how they have made the change.

Collaboration with practitioners – The persons living and working who is part of the

study participate in the active research.

Multiple data generation methods – There are no restrictions on the type of data used in

Action research, both Quantitative and Qualitative data can be used.

Action outcomes plus research outcomes – The outcomes of action research preferably

relate to both action and research. However some projects may not be practically

successful but if reasons of failure are stated may be judged as successful. Therefore both

aspects (action and research) do not always need to occur.

(Cherry, 2005), identifies quantitative data as data that can be counted using numbers. It is the

main type of data that experiments and surveys generate. In contrast, qualitative data includes

anything that is not numerical data, such as words, images and pictures. It is the main type of

data that is generated in case study research and action research (Trochlm, 2006).

3.3 Survey Based Research

Surveys do not have to be limited to questionnaire based research, there are other ways of

generating data. These include methods such as; interviews, observations and documents

(Oates, 2005). Single surveys offer a cross-sectional picture of affairs at a point in time. When

repeating the basic technique, overtime surveys can provide longitudinal data (Cornford &

Smithson, 2005), 2005). In addition surveys, reviews and interviews produce qualitative and

quantitative data based on the layout of the questions that are asked. Researchers have the

benefit of defining the types of data they wish to question with (Oates, 2005).

Page 28: Simon Brooks 100042660 - Dissertation - 2010-2011

To determine efficiency of automated signature creation compared with human crafted signature creation 3 May 2011

Simon Brooks Page 28 of 126

3.4 Case Study Based Research

The main focus of a case study is a “thing” to be investigated. (Oates, 2005), States that such

subject matter for a case study in the context of information systems could be an organization,

a department, an information system, a discussion forum or a systems developer. Furthermore,

(Oates, 2005) explains that the overall aim of a case study is to acquire a detailed insight into

the “life” of that case and determine its complex relationships and processes.

(Oates, 2005), lists the characteristics of a case study to be the following:

Focus on depth rather than breadth – The researcher finds out as much as is possible,

in detail about one instance of the experience under investigation.

A Natural setting – There is no laboratory or artificial situation, so the case is examined

in its natural setting. The case scenario exists prior to the researcher visiting and will still

exist after a researcher leaves. Upon departure, the researcher attempts to disturb the

setting as little as possible.

Supply Holistic study leaving – The researcher focuses on the complexity of

relationships and how these are unified and consistent, instead of trying to segregate

individual factors.

Take into consideration multiple sources and methods – The researcher must use a

broad collection of sources when carrying out research based on a case study.

The positive side to case study-based research is that it becomes suitable when the researcher

is lacking the control needed with events. Furthermore, the results that case studies produce

are close to people‟s experiences and appeal more to people rather than a highly numerical

based study. However the negative side to case study based research is that it can be

sometimes alleged to be lacking strictness within the results and can often lead to

generalizations. In addition there are no set rules to follow, so there is often ambiguity faced

with how to approach this type of research.

3.5 Experiment Based Research

An experiment is an investigation that is under controlled conditions and is observational in

nature. Experiments are designed to examine specific factors by their properties or the

relationships between factors. By conducting experiments an individual factor can be isolated

and the effect observed in detail (Denscombe, 2007).

The typical characteristics of experimentation are defined by (Oates, 2005) as:

Observation and measurement – precise and detailed observations are made by

researchers of the outcomes and changes that occur when particular factors are

either removed or introduced.

Page 29: Simon Brooks 100042660 - Dissertation - 2010-2011

To determine efficiency of automated signature creation compared with human crafted signature creation 3 May 2011

Simon Brooks Page 29 of 126

Identification of casual factors – The researchers refrain from identifying that two

factors appear to be linked. The researchers aim to determine which factor is the

cause (labelled the “independent variable”) and which is the effect (labelled the

“dependent variable(s)”).

Explanation and prediction – Researchers are able to explain the causal link

between two factors by the methods of theory from which their hypothesis was

derived. In some cases this may be a new theory that the researcher proposes.

Furthermore, from this theory they are able to predict future events providing the

experiment proves that a factor will always generate a specific outcome.

Repetition - Experiments are repeated many times and under diverse settings. This

ensures that the observed/measured outcomes are not caused by any other factors,

such as faulty equipment.

3.6 Internet-based research

The internet is a network of networks that has grown tenfold over the past two decades. Since

1994 the internet has advanced both technologically and by how much information and ideas

have been added. This has made it a powerful tool for any type of research and methodology.

As a research tool, internet research is suitable for use with any of the covered research

methods in this chapter. Internet research can allow a researcher to have access to a vast

amount of subjects and sources that would not normally be available. However in terms of

limitations, internet research cannot always aid with direct study using experiments. Internet

based research should be considered a tool amongst a researcher and not a method of research

in its own right (Oates, 2005).

During the literature review stage of this project, internet based research has been a key source

of information regarding; signatures for intrusion detection systems including automated

signature generation, honeypots and traditional systems. In the next stage of this project,

internet based research may not be suitable to provide answers, therefore a different research

method will be chosen to ensure that sufficient answers and results will be generated to the

topic of this project.

Page 30: Simon Brooks 100042660 - Dissertation - 2010-2011

To determine efficiency of automated signature creation compared with human crafted signature creation 3 May 2011

Simon Brooks Page 30 of 126

3.7 Chosen Research Method

This chapter has identified five research methods to aid in providing an answer to the report

question. Out of the five research methods discussed, creating an experiment is the best

solution. An experiment will provide empirical and observational based research and help

determine the efficiency of automated signature creation in comparison to human crafted

signatures. The following explains why the other methods of research that are mentioned in

this chapter have not been selected:

Action based research – Action research can provide a fast generation of results and

allow a researcher to change the environment that exists to provide solutions to a

problem. However the goal associated with this research method is to obtain answers

from a comparison rather than provide a solution.

Survey based research – Survey based research can provide a good range of results

from a wide range of sources. Unfortunately this method of research will not provide

the results that are expected for this project

Case study based research – Case studies are useful to provide information that is

appropriate for the literature review chapter but is not suitable to define strong

answers about the problem. Case studies have individual influences and one may not

be as trustworthy as the other

Internet based research - Internet based research may not be suitable to provide

answers for this project question. However this method of research may be useful

later on in the project to identify internet based materials to support findings.

Experiment based research can provide data results by using a controlled environment. Each

feature of the situation can be controlled directly and prevent any outside impact affect the

data gathered. Using automated signature software to generate signatures is going to involve

using network attacks to change the situation.

The next chapter focuses on deriving an experiment that is suitable to test an automated

signature generation tool and compare the results with human crafted signatures based on

false-positives and false-negatives.

Page 31: Simon Brooks 100042660 - Dissertation - 2010-2011

To determine efficiency of automated signature creation compared with human crafted signature creation 3 May 2011

Simon Brooks Page 31 of 126

4 Chapter Four – Conceptual Model of Problem Domain

4.1 Introduction

The previous chapter highlighted several research methods available to a researcher in

achieving results regarding a research topic. As stated in the conclusion, experimentation

methods have been chosen as the appropriate means of research.

The research topic of this project is that of identifying the efficiency of automated intrusion

detection signatures compared to that of human crafted signatures. An experiment is required

to automatically generate signatures based on current network attacks and compare these with

the equivalent that has been written by hand.

Figure 12 - Outline of the Proposed Experiment

Figure 12 shows a basic outline of the content of the proposed experiment. An automated

signature system chosen from section 2.1.13 will be tested to generate automated signatures

for a range of network attacks. Once these signatures are successfully generated they will be

placed in the intrusion detection system and tested by means of false-positive and false-

negative results.

In contrast the same attacks will be run against two vulnerable virtual machines. Default hand

written rules found in the IDS build will be used to compare against the automated signatures

that are generated with the chosen solution. This chapter identifies the methods used for the

experimentation process and provides reasoning for the choices of systems chosen for this

experiment.

Attacker Machine

Launch attacks agains the systems

Two Virtual Machines matching Honeypot System by

OS

Vulnerable to Network Attacks

Intrusion Detection System

Test Automated Signatures

Test Pre-Written Signatures

Compare the Two Signatures

Automated Signature Generation System

Vulnerable to Network Attacks

Page 32: Simon Brooks 100042660 - Dissertation - 2010-2011

To determine efficiency of automated signature creation compared with human crafted signature creation 3 May 2011

Simon Brooks Page 32 of 126

4.2 Choice of Honeypot and Automated Signature

Section 2.4, identified honeypot technology as a means of fooling attackers and malicious

traffic into assuming that honeypot systems were “real” and could be attacked. However the

honeypots were either simulations of underlying OS (low-interaction) or fully operation

systems (high-interaction) with no purpose except to designate all traffic to them as

suspicious. The traffic sent to these honeypots could be used to identify threats and by using

special systems could effectively create IDS signatures.

The following is a critical comparison and discussion surrounding three automated signature

generation systems as briefly stated in section 2.1.13 of the report. Out of the three products, a

suitable choice will be selected to form the basis of the experiment and produce some

automated signatures. This section also provides an overview of three Honeypot solutions that

are currently available today.

4.2.1 Automated Signature Software

4.2.1.1 SweetBait

SweetBait is an automated protection solution that uses a console based control centre to

control and monitor a series of network intrusion detection, network intrusion prevention and

host-based intrusion prevention systems. The solution is focused on identifying zero-day

worms and providing up to date signatures to support these new worms. To monitor traffic,

SweetBait uses a combination of two signature generators: SweetSpot and SweetBait

(Portokalidis & Bos, 2005). SweetSpot is a low-interaction honeypot system based on Niels

Provos‟ system, Honeyd. The second signature generator is Argos, an experimental system

that is built up of a high-interaction honeypot and is installed over the top of an x86 processor

emulator (Portokalidis et al., 2006).

Figure 13 - SweetBait Architecture Overview (Portokalidis et al., 2006)

Page 33: Simon Brooks 100042660 - Dissertation - 2010-2011

To determine efficiency of automated signature creation compared with human crafted signature creation 3 May 2011

Simon Brooks Page 33 of 126

Figure 13 identifies an overview of the SweetBait system architecture, a complex collection of

components that are separated into sensors and control elements. An IDS sensor detects

network attacks and a honeypot signature generation system called Honeycomb is capable of

creates signatures for attacks. In order to control these systems a control centre is used to

organise the two sensor systems. This control centre is similar to that of the centralized control

system used in hybrid IDS systems (section 2.3.3.4).

The SweetSpot sensor uses Honeyd, a virtual honeypot framework, to simulate multiple low-

interaction honeypots, capable of producing large scale virtualised networks. A network

identity profile can be added to the honeypot to give the illusion to network scanners such as

the Nmap Security Scanner (Lyon, 2011), that there is a real system in place. Automated

signature creation is possible by using a plug-in for the Honeyd program called Honeycomb.

Signatures that are generated using this plug-in are suitable for use with the popular intrusion

detection system, Snort IDS (Sourcefire, 2011). To counteract the bad traffic with the good

traffic another plug-in called Honeybounce acts as a filter to add any benign patters to a

whitelist (Portokalidis et al., 2006).

Overall the system is as a complete solution that can aid in the detection of network worms

rather than a single tool that can generate signatures. The honeypot sensors provide an alluring

trap to zero-day worms and the inbuilt signature generation tool Honeycomb, can generate

signatures by comparing patterns of traffic against the white list generated by the

Honeybounce plug-in. The Honeyd and Honeycomb tools are available to compile as separate

products, Honeycomb is covered in more detail in section 4.2.1.3.

4.2.1.2 Honeycyber

Honeycyber is a solution designed to create signatures that identify polymorphic worms for

signature-based IDS solutions. According to (Mohammed et al., 2009), systems such as

Honeycomb are unsuitable to create sufficient signatures for polymorphic worms due to the

lack of information gathered by the honeypot and because of the pattern-based analysis

techniques that are used. A single signature is deemed unsuitable to match all of the worm

instances of a polymorphic worm and provide low false-positive and low false-negative results

at the same time.

To gather enough information to generate signatures suitable to identify polymorphic worms,

Honeycyber uses a Honeynet to gather extensive amounts of information about the subject. As

defined by (Mohammed et al., 2009), a Honeynet consists of a network of production systems

that are located behind a traditional access control device such as a firewall (see section 2.1.9

An Overview of Firewall Systems). Single honeypot systems require experts to search through

and analyse the data that is logged by these systems. By the time the correct data has been

Page 34: Simon Brooks 100042660 - Dissertation - 2010-2011

To determine efficiency of automated signature creation compared with human crafted signature creation 3 May 2011

Simon Brooks Page 34 of 126

analysed to generate a signature manually, a new polymorphic worm may have already spread

and infected a system.

Figure 14 – Overview of the Honeycyber architecture (Mohammed et al., 2009)

Figure 14 depicts an overview of the Honeycyber architecture. As explained by (Mohammed

et al., 2009), the goal of the Honeycyber system is to attract worm traffic towards Honeynet

(1) before the traffic compromises the local server. Once the attack traffic has compromised

the first Honeynet, it is sent to an Internal Translator (1), which redirects the traffic to the

second Honeynet (2), followed by the second internal translator (2). Worm traffic attempts to

create an outbound connection back to the internet. This process effectively performs a loop,

containing the worm traffic and preventing it from leaving the network.

Figure 15 - Honeycyber - Signature Generation architecture (Mohammed et al., 2009)

Figure 15 is a diagram of the signature generation architecture of Honeycyber. The

architecture differs from the single substring signature generation techniques as used by

programs such as Honeycomb. As mentioned, these single substring techniques are proven to

be inadequate to create signatures for polymorphic worms (Mohammed et al., 2009). The

architecture uses several complex signature generation algorithms that are explained by

(Gusfield, 1997).

To sum up Honeycyber, the system is used as part of a wide solution to create automated

signatures, these signatures are designed to tackle polymorphic worms and reduce the time

constrains involved with writing manual signatures. A double Honeynet system lures

Page 35: Simon Brooks 100042660 - Dissertation - 2010-2011

To determine efficiency of automated signature creation compared with human crafted signature creation 3 May 2011

Simon Brooks Page 35 of 126

suspicious traffic inside and prevents the traffic from escaping back to an outbound

connection. The aim of Honeycyber is to produce effective and accurate signatures using

complex signature generation algorithms that are superior to pattern based techniques used to

identify standard and zero day worms.

4.2.1.3 Honeycomb

Honeycomb is an automated signature generation plug-in for designed for Honeypot system,

Honeyd. Both these tools are used in the automated protection system SweetSpot. This section

aims to identify Honeycomb in more detail. Honeycomb is a system designed to automatically

generate IDS signatures for malicious traffic. The signatures are automatically generated by

analysing and performance tests on Honeyd honeypot traffic (Kreibich & Crowcroft, 2004).

In order to identify the traffic as malicious, certain techniques are used to generate signatures.

Unlike Honeycyber which uses advanced algorithms to identify polymorphic worm traffic,

Honeycomb uses a simplistic form of pattern-detection techniques and performs packet header

conformance tests on any traffic that is captured via the Honeyd honeypot. Currently the

system can examine the IP, TCP and UDP headers and any relevant payload data. This is

achieved by extending the open-source code supplied by the Honeyd system (Kreibich &

Crowcroft, 2004).

Figure 16 - Overview of Honeycomb Architecture (Kreibich & Crowcroft, 2004)

Figure 16 displays an overview of Honeycomb‟s architecture. The diagram displays a typical

Honeyd setup that emulates a Linux, BSD, Windows 98 and Cisco environment and some

extra services. These run using Honeyd‟s scripting language and personality file to perceive

the appearance of a working machine. Honeycomb does not duplicate any of the network

traffic that flows to the simulated honeypots, instead it uses the Libpcap library (McCanne et

al., 2010), included in Honeyd to inspect the traffic before forwarding the relevant packet

information to the honeypots. This is considered an advantage of Honeyd, due the reduction of

effort made to analyse the malicious traffic as defined by (Kreibich & Crowcroft, 2004).

Page 36: Simon Brooks 100042660 - Dissertation - 2010-2011

To determine efficiency of automated signature creation compared with human crafted signature creation 3 May 2011

Simon Brooks Page 36 of 126

Traffic normalizers are designed to reduce the risk of an intruder evading the detection of their

attacks. There are ways in which an attacker can try to “evade” detection from an intrusion

detection system. For example, URL-encoding is a technique used to hide information in a

URL web address and can also be used to evade the scanning techniques of application

firewalls. A URL-encoded text string consists of a “%” followed by converting the letter to a

hexadecimal value. For example the uppercase letter “B” converted to hexadecimal is “41”.

When using URL-encoding a “%” is added before the hexadecimal conversion, which would

make “B” turn to “%41”. If the string of text “bad traffic” was to be encoded it would be the

following:

%42 %61 %64 %20 %54 %72 %61 %66 %66 %69 %63

B a d *space* T r a f f i c

Table 1 - URL-encoding example

Malicious code could be converted using this method. When an intrusion detection system

scans the traffic, a signature that was written to detect “Bad Traffic” would not match

“%42%61%64%20%54%72%61%66%66%69%63” so the traffic could slip by and

potentially cause damage. In the early days of intrusion detection, these types of evasion

techniques were common. To help make intrusion detection systems smarter at detecting these

attacks an extra module known as a “traffic normalizer” has since been added to improve these

systems.

A traditional packet normalizer ensures that the packets put back to a way in which the

intrusion detection system will understand. Traffic normalizers operate at layer 3, the network

layer, and layer 4 the transport layer, therefore they cannot identify any of the upper layers of

the OSI model protocols including layer 7, the application layer (Handley et al., 2002). The

SNORT intrusion detection system uses a more sophisticated traffic normalizer, capable of

identifying the higher layers and is known as a “pre-processor” (Niem, 2008).

Figure 17 - Overview of Honeycomb signature generation algorithm (Kreibich & Crowcroft, 2004)

Page 37: Simon Brooks 100042660 - Dissertation - 2010-2011

To determine efficiency of automated signature creation compared with human crafted signature creation 3 May 2011

Simon Brooks Page 37 of 126

Figure 17 demonstrates an overview of Honeycomb‟s signature generation algorithm.

Honeycomb is designed to focus on identifying the protocols found at layer 3, and layer 4 of

the OSI model but not the higher application protocols. When a packet reaches Honeycomb

the system first checks to see if there is already a TCP or UDP connection state for the packet.

If the connection already exists, the state of the connection is updated, for example a TCP

state may be “LISTNING” (ttcplinux, 2000).

When the status of the connection is updated Honeycomb uses protocol analysis that is similar

to the header-walking technique used in packet normalizers (Handley et al., 2002). The

algorithm operates like traffic normalizers but cannot identify application protocols and does

not modify modifications of packets. When inconsistencies such as an illegal TCP or UDP

packet are detected, abnormal behaviour of these protocols within the packet header is

recorded as a signature (Kreibich & Crowcroft, 2003).

When performing payload analysis, honeycomb uses a slightly different approach. A generic

algorithm has been created by the authors of Honeycomb called O(n) longest-common-

substring (LCS) to analyse the payload of an IP packet. The payload analysis uses an

algorithm by (Ukkonen, 1995). This algorithm is applied to binary strings that are built from

the exchanged messages, the flow of the messages are analysed horizontally (Figure 18), by

means of sent message to received message and vertically based on every message that has

been received by the host and honeypot (Figure 19).

Figure 18 - Honeycomb Horizontal Detection (Kreibich & Crowcroft, 2003)

Page 38: Simon Brooks 100042660 - Dissertation - 2010-2011

To determine efficiency of automated signature creation compared with human crafted signature creation 3 May 2011

Simon Brooks Page 38 of 126

Figure 19 - Honeycomb Vertical Detection (Kreibich & Crowcroft, 2003)

Every time a signature is generated based on the protcol analysis or payload flow analysis the

result is stored in a “Signature Pool”, a log file that stores signatures in formats suitable for the

Snort and Bro intrusion detection systems (Kreibich & Crowcroft, 2003).

In summary Honeycomb is an advanced signature generation system that has been built to

compliment the Honeyd honeypot system. Honeyd can be used to emulate large networks that

are designed to be eye-catching towards malicious attackers and traffic. Honeycomb takes

advantage of this traffic and uses a method known as protocol analysis based on traffic

normalization to identify abnormalities within the TCP and UDP protocols. To sum up the

performance of Honeycomb, the system has been tested to generate signature for the popular

worms: CodeRed II and Slammer, and some popular port scanning techniques. On the whole,

the system performed well during the initial tests and retained good response times throughout

the system (Kreibich & Crowcroft, 2003).

Page 39: Simon Brooks 100042660 - Dissertation - 2010-2011

To determine efficiency of automated signature creation compared with human crafted signature creation 3 May 2011

Simon Brooks Page 39 of 126

4.2.2 Honeypots

4.2.2.1 High Interaction Honeypot Analysis Toolkit

HiHAT is an open source, high interaction honeypot analysis tool that can transform arbitrary

PHP applications into web-based high-interaction honeypots. The tool can be installed on

PHPNuke, PHPMyAdmin or OSCommerce systems and turns the system into a fully

functional honeypot with a comprehensive logging and monitoring system (HiHAT, 2007).

Figure 20 - HiHAT Overview Mode (HiHAT, 2007)

HiHAT logs IP information about the attacker that can be mapped geographically. However IP

information can easily be spoofed and so the actual location of an attack may not be accurate.

In addition, the honeypot can detect major web application attacks such as SQL injection. The

system can also capture any malicious tools that are used against the honeypot. These tools are

collected stored on the system. This feature can aid administrators with the analysis of these

tools and figure out ways to prevent reoccurring events. HiHAT can generate a wide range of

graphical statistics about the traffic that has been collected from the system (HiHAT, 2007).

An Example of the statistics page is demonstrated in Figure 20, the overview mode of

HiHAT.

Page 40: Simon Brooks 100042660 - Dissertation - 2010-2011

To determine efficiency of automated signature creation compared with human crafted signature creation 3 May 2011

Simon Brooks Page 40 of 126

4.2.2.2 LaBrea

Figure 21 - LaBrea installed on a Linux machine (Softpedia, 2006)

LaBrea is an open-source tool that creates a “Sticky Honeypot” known as a Tarpit. LaBrea

takes over unused IP addresses on a network and creates “virtual servers” that seem attractive

to network worms. The system has been tested on Linux, FreeBSD, Solaris and Windows

98/2000 sytstems. LaBrea responds in a way that slows down attacker‟s connections attempts,

resulting in the attacker at the other end getting “stuck”, sometimes for a very long time

(Softpedia, 2006). The name La Brea is from the La Brea Tar pits, located in Los Angeles,

America. LaBrea logs bandwidth usage from attackers to the “virtual machines”, this

information can be viewed directly from the server. Figure 21 displays a console display of

LaBrea in function, note that LaBrea is a command line based tool.

Page 41: Simon Brooks 100042660 - Dissertation - 2010-2011

To determine efficiency of automated signature creation compared with human crafted signature creation 3 May 2011

Simon Brooks Page 41 of 126

4.2.2.3 Honeyd

Honeyd is a small network daemon that is used to create multiple virtual honeypots on a single

machine, these virtual honeypots can be configured to emulate services (such as HTTP, SMTP

or FTP) via a script. The scripting language is powerful enough to fool network scanners such

as NMAP (Lyon, 2011) and give the impression that “real” machine is active (Provos, 2004).

The daemon runs on Unix/Linux and the code has also been ported for use on windows

systems.

Figure 22 - Honeyd Administration Interface running on CentOS

When running the Unix/Linux version of Honeyd, logging can be turned on using the dash L

(-l) command. This causes Honeyd to log all received packets in a human readable format

(Provos, 2002). A log entry is made up of several types of information; time and date, protocol

(tcp/udp/icmp), an S to indicate the start of the connection or an E to indicate the end of a

connection. Other useful pieces of information logged include the source port and IP and

destination port and IP (Provos, 2004). Figure 22 displays the Web Based administration

interface, and offers graphs of traffic, interface information and other statistics.

4.2.3 Conclusion

This section has identified three automated signature generation systems and three honeypot

solutions. Each system has been identified to generate signatures based on many different

versions of attacks. SweetBait identifies zero-days worms, whereas Honeycyber focuses on

polymorphic worms. Both of these systems incorporate other third party aspects such as IDS

and control centres. As a whole these systems appear to perform well but seem rather complex

in the way they are setup. Therefore as a choice of automated signature system, Honeycomb

seems simplistic in its setup and only requires a Host, Honeyd installation files, pre-requisite

files and the Honeycomb packages.

Page 42: Simon Brooks 100042660 - Dissertation - 2010-2011

To determine efficiency of automated signature creation compared with human crafted signature creation 3 May 2011

Simon Brooks Page 42 of 126

4.3 Experiment Test bed Environment

4.3.1 System Specification

The test bed is going to be built on a Dell Vostro 3700 3700 laptop system, the specification

of the laptop includes a 64-bit version of Windows 7 Business and an Intel i5 Quad Core

2.5GHz processor with 4.00GB of DDR3 memory. The i5 processor is equipped with Intel

Virtualization Technology (VT-x), that is suitable enough to run and support the use of Virtual

Machines (Intel Corporation, 2011). Figure 23 is a screenshot from the laptop showing the

laptop specification and system rating noted in Windows 7.

Figure 23 - Windows 7 - Laptop Specification

4.3.2 Virtualization

Virtualization is a technology that effectively, enables a single computer to run multiple

operating systems. These extra operating systems are known as “Guest OS” machines and

make use of the host‟s hardware, which is managed by a process known as a hypervisor

(RedHat, 2010). Virtualization is considered a use of Green IT because fewer servers are

required and multiple instances of systems and hardware can be run on a single piece of

hardware. Another useful aspect of virtualization is the capacity to install and test new

operating systems, without the need to install over the original host machine, making

virtualization ideal for this experiment.

Many virtualization solutions exist in the market today. Some are available via an open-source

based licence such as Oracle‟s Virtual Box. Others are proprietary based software and require

payment such as VMware ESX, an enterprise-level product (VMware Inc., 2009). The

virtualization software will need to support the use of multiple operating systems such as

Linux/Unix and Windows. In addition the software should be able to run several virtual

machines simultaneously and offer sufficient virtual network functionality.

Page 43: Simon Brooks 100042660 - Dissertation - 2010-2011

To determine efficiency of automated signature creation compared with human crafted signature creation 3 May 2011

Simon Brooks Page 43 of 126

4.3.2.1 VirtualBox

VirtualBox is a virtualization tool for virtualizing x86 (32-bit) hardware systems. The product

is designed by Oracle and supports installation on Solaris/Intel Macs/Linux and Windows

setups. VirtualBox is an open-source product that is free to distribute and edit the source code

under the GPL Version 2 licence (GNU, 1991). The product supports a wider range of

Windows/Linux/Mac/Solaris and UNIX operating systems.

Figure 24 - VirtualBox on running Ubuntu 10.10 VM on Windows 7 (Oracle, 2011)

Virtual networking can emulate a range of fast Ethernet and Gigabit network cards.. These

cards can be configure in a range of modes including; NAT mode, to share the IP of the host,

bridged mode, that creates direct connection from the virtual card to the physical network and

internal mode, a network that is visible only to VMs (Oracle, 2011). Figure 24 displays an

example of an instance of VirtualBox running an Ubuntu virtual machine.

4.3.2.2 XenServer

XenServer is a free virtualization tool offered by Citrix Systems, a company that specializes in

server and desktop virtualization. The software is available in other flavours, including

Enterprise and Platinum editions. XenServer Free is designed for Windows based machines

and can support Windows and Linux OS systems (Citrix Systems, Inc., 2011). Unfortunately

Virtual networking is not supported in the free version on XenServer. Figure 25 displays the

home screen of Citrix XenServer.

Page 44: Simon Brooks 100042660 - Dissertation - 2010-2011

To determine efficiency of automated signature creation compared with human crafted signature creation 3 May 2011

Simon Brooks Page 44 of 126

Figure 25 - Citrix XenServer 5.6.1 Home Screen (Softpedia, 2011)

4.3.2.3 VMware

VMware offers a wide range of virtualization solutions suited for Data Centre environments

right through to desktop and end-user computing. The end user computing products are the

ones that are most suited for this experiment. VMware Player is a solution designed for

running a virtual machine that has been created by a VMware product. The software supports

Linux and Windows host operating systems in both 32-bit and 64-bit flavours. Unfortunately

the VMware player cannot create virtual machines but only run them, making it an unsuitable

product to use in this experiment. Figure 26 is a screenshot showing VMware player installed

on an Ubuntu Linux machine.

Figure 26 - VMware Player on Ubuntu Linux (TheTechJournal, 2010)

VMware Workstation is another product by VMware that is designed to run multiple virtual

machines in a non-server environment. The software is compatible with 32bit and 64bit

processers and offers support for 64-bit guest operating systems with Intel VT CPUs. VMware

workstation is a commercial product but offers a 30-day trial and also sells at Academic

prices. Full support for virtualized network is included, by an implementation of a virtual

switch. Figure 27 is a screenshot of the VMware workstation in action and the Virtual

Page 45: Simon Brooks 100042660 - Dissertation - 2010-2011

To determine efficiency of automated signature creation compared with human crafted signature creation 3 May 2011

Simon Brooks Page 45 of 126

Network Editor. The Virtual Network Editor offers similar functionality to that of Oracle

VirtualBox.

Figure 27 - VMware Workstation 7 and Virtual Network Editor

4.3.3 Conclusion

This section has identified three different virtualization solutions that are all offering the

ability to virtualize an x86 or x64 machine. VirtualBox offers virtualized network support and

can create Windows/Unix/Mac virtual machines. VMware workstation has similar desired

functionality to Virtualbox but VMware is more commercial in the way it distributes its

software. Support is more likely to be easier to obtain with the VMware product than it is with

VIrtualBox. Citrix XenServer lacks the desired network support in the free edition and costs

too much to warrant it for this experiment. In conclusion, VMware Workstation can create

operating systems for a variety of different operating systems including Windows and Linux,

the software does charge a fee but the trial version should be suitable enough for this

experiment. Therefore VMware Workstation will be the software used to virtualise the attack,

honeypot, Honeycomb and intrusion detection systems.

Page 46: Simon Brooks 100042660 - Dissertation - 2010-2011

To determine efficiency of automated signature creation compared with human crafted signature creation 3 May 2011

Simon Brooks Page 46 of 126

4.4 Testing Environment Configuration

The previous section identified three virtualization products and chose VMware as the main

choice to virtualize the systems required for this project. Therefore several virtual machines

will need to be prepared for this experiment.

4.4.1 Honeyd

Honeyd operates as a small daemon program and is primarily designed for UNIX-like

platforms, but can be run on Windows platforms as well (Provos, 2007). WinHoneyd is a

commercial solution that offers Windows GUI over the top of their licenced WinHoneyd

configurator tool. A Windows tool seems like an easier route to follow, however the licence

fee of $99.00 is rather high priced and compatibility with the signature generation tool

Honeycomb seems unlikely. Figure 28 is a screenshot of the WinHoneyd configuration file

editor used to create honeypot templates. In the UNIX/Linux version, configuration files are

created using a textfile.

Figure 28 - netVigilance WinHoneyd Configurator (negVigilance, Inc., 2009)

The UNIX-like code version is covered under the GNU General Public Licence, version 2

(GNU, 1991), making it free to distribute the code and not incur any download costs. Since

Honeyd can be run on any UNIX-like operating system, Ubuntu is the preferred OS of choice,

due to its simplicity to install software packages using the synaptic package manager

(Rijckenberg, 2010). A Ubuntu 10.04 virtual machine has been prepared with a stable

installation of Ubuntu 10.04 Lucid Lynx At the time of writing Ubuntu is currently in version

10.04.02 (Stewart, 2011). Figure 29 is a screenshot of the Synaptic Package Manager listing

the Honeyd package. The version of Ubuntu listed here is 10.0.4 LTS, codenamed Lucid

Lynx.

Page 47: Simon Brooks 100042660 - Dissertation - 2010-2011

To determine efficiency of automated signature creation compared with human crafted signature creation 3 May 2011

Simon Brooks Page 47 of 126

Figure 29 - Ubuntu 10.04 LTS - Synaptic Package Manager - Listing Honeyd package details

4.4.1.1 Brief Overview of Honeyd’s Configuration Language

Honeyd uses a text-based configuration file to specify the IP addresses and services of the

honeypots. Figure 30 is an example template containing information about the operating

system, ports and IP information. Honeyd can also simulate services such as HTTP, SMTP

and TELNET, these services can be written in the python programming language or C based

languages.

###Honeyd Config

### Windows computers

1.create windows

2.set windows personality “Microsoft Windows NT

4.0 SP5-SP6″

3.set windows default tcp action reset

3.set windows default udp action reset

4.add windows tcp port 80

“scripts/iisemulator/iisemul8.pl”

4.add windows tcp port 139 open

4.add windows tcp port 137 open

4.add windows udp port 137 open

4.add windows udp port 135 open

4.add windows tcp port 110 “sh scripts/pop3.sh”

4.add windows tcp port 25 “sh scripts/smtp.sh”

4.add windows tcp port 21 “sh scripts/ftp.sh”

5.set windows uptime 3284460

6.bind 192.168.1.11 windows

Key

1. Creates a template called

windows

2. Sets the honeypot personality

3. Refers to an external default

configuration

4. This adds information port

details and any other service

emulation scripts such as

“iisemul8.pl”, a Microsoft

Internet Information Services

script.

5. Sets the uptime of the system

6. Binds the Honeypot to an IP

address

Figure 30 - Honeyd Configuration File Sample

Page 48: Simon Brooks 100042660 - Dissertation - 2010-2011

To determine efficiency of automated signature creation compared with human crafted signature creation 3 May 2011

Simon Brooks Page 48 of 126

4.4.1.2 Honeycomb

Honeycomb is a system for automated generation of signatures for network intrusion detection

systems, including Snort IDS. It applies protocol analysis and pattern-detection techniques to

traffic captured on Honeyd honeypots, operating as a pluggable piece of software. The website

states that Honeycomb should build on Linux, FreeBSD and OpenBSD systems but does not

specify any specific flavours of Linux. Ubuntu 10.04 is built on the Linux 2.6.32 kernel, so an

attempt at installing and configuring Honeycomb with Honeyd on this version of Ubuntu

Linux will be made.

4.4.2 Attacker Machine System Configuration and Requirements

The Metasploit Framework is an open source penetration testing kit that is free to download,

as a framework the tool aids in the development and execution of exploit code against remote

machines. Many of the exploits support a range of operating systems, including Windows and

Linux, and also include vulnerabilities for well-known software products (Rapid7, 2011).

Figure 31 is a screenshot of the Exploit Database website.

Figure 31 - Exploit Database Archive (Offensive Security, 2011)

Open Source is supported choice because of the freedom that occurs with the distribution of

code and no excessive licencing fee. The Metasploit Framework has the ability to add in

current and Zero Day exploits. Vulnerable software can also be supported from the Exploit

Database archive (Offensive Security, 2011). For this experiment exploits that are included

with the default Metasploit install will be used. Figure 32 is a screenshot of the Metasploit

welcome screen installed on Backtrack 4, a Linux Security Distribution (BackTrack, 2011).

Listed on the screen is the number of exploits and payloads installed, followed by the last time

of update. The toolkit requires a connection to the internet to ensure the exploits are fully up to

date.

Page 49: Simon Brooks 100042660 - Dissertation - 2010-2011

To determine efficiency of automated signature creation compared with human crafted signature creation 3 May 2011

Simon Brooks Page 49 of 126

Figure 32 - Backtrack 4 R1 - Metasploit Console Mode

The framework is available via the Metasploit website for Windows, Linux and UNIX

operating systems. The Windows package offers the same functionality as the Linux

Backtrack version. Figure 33 shows a typical install of the Metasploit framework on a

Windows XP machine, due to the simplicity of the XP installation this version will be the

choice of install for this experiment.

Figure 33 - Metasploit Install on Windows XP Virtual Machine

4.4.3 Intrusion Detection System Configuration and Requirements

Honeycomb can output signature for Bro IDS and Snort IDS. Bro is an Intrusion Detection

System targeted at high-speed Gbps networks that use high-volume amounts of traffic

(Lawrence Berkeley National Laboratory, 2011). Snort IDS, is an open source network

intrusion prevention and detection system, and can be installed on any UNIX based system or

Windows based machine (Sourcefire, 2011). Both these systems support intrusion detection

and use open-source technology. However a creating a Gbps network seems rather ambitious

Page 50: Simon Brooks 100042660 - Dissertation - 2010-2011

To determine efficiency of automated signature creation compared with human crafted signature creation 3 May 2011

Simon Brooks Page 50 of 126

to setup via a virtual environment and would seem more suited if this project was based on a

large corporate environment.

Figure 34 - Network Security Toolkit v2.13.0 - Snort Setup Page

A pre-built Linux environment called the Network Security Toolkit is available with a

complete install of the Snort intrusion detection system. The Network Security Toolkit is a

bootable live CD/DVD environment, consisting of a complete set of open source network

security tools. Among these tools sits a full SNORT IDS installation, including BASE a log

analysis tool for snort, ready for use. The NST is available as a VMware virtual machine

application, a VM appliance that is ready for download without the need to install

(networksecuritytoolkit.org, 2011). Figure 34 displays a screen shot of the Snort setup page

for the Network Security Toolkit.

4.4.3.1 Brief Overview of SNORT Signatures vs. Honeycomb Signatures

Snort offers up-to date rule sets designed by the SourceFire Vulnerability Research Team

(VRT). Rules are available via a free registered user subscription, or a fee paid subscription

(Sourcefire, 2011). Another signature provider is Emerging Threats, an open source

community project that is free to use by any user or organisation (Emerging Threats, 2011).

They offer new rule sets several times a day, 7 days a week for Suricata, another open-source

IDS/IPS (Open Information Security Foundation, 2011) and the SNORT IDS/IPS.

When writing a SNORT rule by hand, the rule should be simple and accurate, if the rule is too

complex, it can become too much for SNORT to process and may result in false-positives or

false-negatives. A snort rule is built up of the following components (Bianco, 2006):

Header – The header contains an instruction of whether to log or alert the rule. More

information such as the Protocol (ip,tcp,udp,icmp,any), the source and Destination IP

address and Ports, and the directional operators (“->”,”<>”) are also included in

Page 51: Simon Brooks 100042660 - Dissertation - 2010-2011

To determine efficiency of automated signature creation compared with human crafted signature creation 3 May 2011

Simon Brooks Page 51 of 126

the header section . Any information to identify a host and its protocol is defined in

this area of the snort rule (Bianco, 2006).

Body - The body of the snort rule is the complex part of the snort rule and can affect

the accuracy and efficiency of the rule if it becomes too complex. Contained in the

body are options such as: Metadata, Payload Detection, Non-Payload Detection,

Post-Detection, and Thresholding and suppression (Bianco, 2006).

Table 2 displays the basic outline of a snort rule. In this example the header identifies an alert

for the TCP protocol from any source IP, towards any destination on any port. The body

section includes a sample alert and this can be customised to include sufficient payload data to

identify a specific set or pattern of traffic.

alert tcp any any -> any any (msg:"Sample alert";)

Header Body

Table 2 - A basic Snort Rule Outline

Table 3 is an example of a hand written snort rule by (Reid, 2003). In the header of the rule

the alert instruction is followed by $EXTERNAL_NET any port, to $HOME_NET port 1434.

These are two default SNORT variables and are used to define the source and destination

network addresses. The body of the snort rule identifies the alert by the message “HELL-SQL

Worm Scan”, followed by some detailed information about the packet data.

alert udp $EXTERNAL_NET any ->

$HOME_NET 1434

(msg:"HELL-SQL Worm

Scan";

flow:to_server,from_server;

content:"|684765745466b96c6c|";classtype

:attempted-admin)

Header Body

Table 3 - A Hand Written Snort Rule for Slammer SQL Worm (Reid, 2003)

Page 52: Simon Brooks 100042660 - Dissertation - 2010-2011

To determine efficiency of automated signature creation compared with human crafted signature creation 3 May 2011

Simon Brooks Page 52 of 126

Table 4 depicts an example of a Snort signature generated by Honeycomb. The signature

follows the layout of a snort signature with a header and body field. Note that in the header,

the UDP protocol has been selected and the port number 1434, this is the SQL Server

Resolution port the SQL Slammer Worm operates on by sending 376 bytes of data (Knowles,

2003).

alert udp any any ->

192.168.169.2/32 1434

(msg: "Honeycomb Fri Jul 18 11h46m33 2003 "; content: "|04 01 01 01 01 01

01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01

01 01 01 01 0101 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01

01 01 01 01 01 01 01 01 01 01 01 01 01 01 0101 01 01 01 01 01 01 01 01

01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 DC C9 B0|B|EB

0E 01 01 01 01 01 01 01|p|AE|B |01|p|AE|B|90 90 90 90 90 90 90 90|h|DC C9

B0|B|B8 01 01 01 01|1|C9 B1 18|P|E2 FD|5 |01 01 01 05|P|89

E5|Qh.dllhel32hkernQhounthickChGetTf|B9|llQh32.dhws2_f|B9|etQhsockf|B9|

toQhsend|BE 18 10 AE|B|8D|E|D4|P|FF 16|P|8D|E|E0|P|8D|E|F0|P|FF 16|P|BE 10

10 AE|B|8B 1E 8B 03|=U|8BEC|Qt|05 BE 1C 10 AE|B|FF 16 FF

D0|1|C9|QQP|81 F1 03 01 04 9B 81 F1 01 01 01 01|Q|8D|E|CC|P|8B|E|C0|P|FF

16|j|11|j|02|j|02 FF D0|P|8D|E|C4|P|8B|E|C0|P|FF 16 89 C6 09 DB 81 F3|<a|D9

FF 8B|E|B4 8D 0C|@|8D

14 88 C1 E2 04 01 C2 C1 E2 08|)|C2 8D 04 90 01 D8 89|E|B4|j|10

8D|E|B0|P1|C9|Qf|81 F1|x|01|Q|8D|E|03|P|8B|E|AC|P|FF D6 EB|" ; )

Header Body

Table 4 - Signature Honeycomb created for the Slammer Worm (Kreibich & Crowcroft, 2003)

Overall, Honeycomb follows the outline of a Snort rule by creating header and body

information. When comparing the signature in Table 3 and Table 4, there is a considerable

increase in the amount of information located in the body. However Table 3 is not an official

Snort signature written by the VRT and is provided in this context as general snort rule

overview.

Both of the VRT and emerging threats signature types are useful for the SNORT IDS solution,

in helping defend a production network, however because Snort is going to be used in a non-

production testing environment, up to date signatures are not needed. The Network Security

Toolkit provides pre-built signatures included in the Snort IDS system installation that are

suitable enough to identify attacks for the operating systems, Windows XP and Windows

2000.

Page 53: Simon Brooks 100042660 - Dissertation - 2010-2011

To determine efficiency of automated signature creation compared with human crafted signature creation 3 May 2011

Simon Brooks Page 53 of 126

4.5 Experiment Test Bed

4.5.1 Experiment Part One

The experiment is separated into two parts. The first experiment will attempt to send a set of

exploits one by one, to a Honeyd XP and Windows 2000 machine. An attempt will be made to

generate signatures based on these exploits using honeycomb. The outcome expected from

these experiments is that:

Honeycomb will generate some signatures that will be good enough to place into the

Snort IDS

Or

Honeycomb will generate some signatures that are not good enough to place into the

Snort IDS

Once the results are collected, the Honeycomb signatures will be placed into the Snort IDS.

Exactly the same network attacks will be run against full versions of the Operating systems,

and the IDS will be running on the network medium to detecting the attacks using the

Honeycomb generated signatures. Figure 35 is an overview of this first experiment.

Figure 35 - Experiment Part One Overview

4.5.2 Experiment Part One Network Setup

Figure 36 is a diagram of network topology used in part one of the experiment. The

Metasploit Attacker Machine is connected to the virtual VMware switch. Honeyd and

Honeycomb are installed on a CentOS virtual machine that is also connected to the virtual

VMware switch. Honeyd and Honeycomb were originally tested on Ubuntu 10.04 Lucid

Lynx, during the testing phase there were significant problems compiling Honeycomb and

Honeyd with this version of Ubuntu Linux. Further research was undertaken to figure out a

way to install both of the products successfully. Appendix 9.6 is a modified version of the

Portuguese installation instructions adapted for the install of CentOS and Honeyd and

Honeycomb for this experiment.

Run Windows 2000 SP4 exploits from attacker

machine using Metasploit

Honeyd Victim #1 automatically generates

signatures using Honeycomb

Run Windows XP SP2 exploits from attacker

machine using Metasploit

Honeyd Victim #2 automatically generates

signatures using Honeycomb

Place the generated Honeycomb signatures

into Snort IDS

Replace Honeyd Victim# 1 and Honeyd Victim# 2

with High-Interaction Virtual Machines

Repeat the same Windows 2000 SP4 and Windows XP SP2 attacks

against the High-Interaction VMs

Snort IDS should detect Exploits based on

Honeycomb signatures

Document results - Move to Experiment 2

Page 54: Simon Brooks 100042660 - Dissertation - 2010-2011

To determine efficiency of automated signature creation compared with human crafted signature creation 3 May 2011

Simon Brooks Page 54 of 126

Figure 36 - Experiment Part One - Network Topology Diagram

The Honeyd environment pictured in Figure 36 is the emulated network that is running from

Honeyd. It is built up of two emulated Cisco Routers that provide connectivity from the

10.0.0.0 network to the 10.0.1.0 network. The 10.0.0.0/16 network has been chosen for this

experiment because of its existence and recommendation in documentation surrounding

Honeyd. A different subnet could have been selected but due to the complexities of this

product 10.0.0.0/16 is the subnet used for both Honeyd and Virtual machines. Connected to

the Cisco Routers are two Honeyd Personalities, impersonating a Windows 2000 Service Pack

4 and Windows XP Service Pack 2 machine. Appendix 9.7 is a combination of the Honeyd

network configuration file written by Niels Provos (Provos, 2007) a Honeycomb configuration

from (Andrade, 2009). This file has been adapted to emulate hosts that are suited for this

experiment and can be found in Appendix 0.

4.5.3 Experiment Part Two Method

The second experiment will attempt to recreate the exact same attacks but against full versions

of the XP Service Pack 2 and Windows 2000 Service Pack 4 operating systems. Snort IDS

will be running on the network wire to detect the attacks that are being run against these

systems using the default pre-configured signatures installed with the Snort package.

The outcome expected from these experiments is that on comparison:

Honeycomb signatures will have less false-positives and negatives than human

created Snort signatures

Page 55: Simon Brooks 100042660 - Dissertation - 2010-2011

To determine efficiency of automated signature creation compared with human crafted signature creation 3 May 2011

Simon Brooks Page 55 of 126

Or

Human created Snort signatures will have less false-positives and negatives than

Honeycomb created Snort signatures

Once the results are collected the signatures will then be compared by how many false-

positives and false-negatives are produced. Figure 37 provides an overview of part two of the

experiment.

Figure 37 - Experiment Part Two Overview

4.5.4 Experiment Part Two Network Setup

Figure 38 is a diagram of network topology used in part two of the experiment. The

Metasploit Attacker Machine is still used and connected to the virtual VMware switch.

Honeyd and Honeycomb are replaced with virtual machines of full versions of Windows XP

and Windows 2000. The extra addition to the network is the Snort IDS, running on the

Network Security Toolkit.

Figure 38 - Experiment Part Two - Network Topology Diagram

Run Windows 2000 SP4 exploits from attacker

machine using Metasploit

Windows 2000 VMware Virtual Machine will be

exploited

Snort IDS Default Human Crafted Signatures should

detect exploits and generate alerts

Run Windows XP SP2 exploits from attacker

machine using Metasploit

Windows XP VMware Virtual Machine will be

exploited

Snort IDS default human crafted signatures should

detect exploits and generate alerts

Document Results

Compare Snort alerts with honeycomb rules based on

False Positives and True Postitives

End Experiment

Page 56: Simon Brooks 100042660 - Dissertation - 2010-2011

To determine efficiency of automated signature creation compared with human crafted signature creation 3 May 2011

Simon Brooks Page 56 of 126

4.6 Metasploit Attacks

As explained in section 4.4.2 the exploit framework known as Metasploit was chosen to send

attacks. This section identifies the attacks available for the two operating systems, Windows

XP and Windows 2000. Each exploit was tested before use to ensure that these attacks could

be used in the experiment. Appendix 0 provides a list of online documentation for the majority

of attacks listed in this section.

4.6.1 Windows 2000 Attacks

A search was run for “Windows 2000” exploits in the Metasploit Framework and Table 5

shows the result of this search.

Exploit Name General Description

Ie_iscomponentinstalled Internet Explorer isComponentInstalled Overflow

ms06_055_vml_method Internet Explorer VML Fill Method Code Execution

ms06_057_webview_setslice Internet Explorer WebviewFolderIcon setSlice()

Overvlow

ms06_071_xml_core Internet Explorer XML Core Services HTTP Request

Handling

ms03_026_dcom Microsoft RPC DCOM Interface Overflow

ms01_023_printer Microsoft IIS 5.0 Printer Host Header Overflow

ms03_007_ntdll_webdav Microsoft IIS 5.0 WebDav ntdll.dll Path Overflow

ms00_094_pbserver Microsoft IIS Phone Book Service Overflow

ms03_051_fp30reg_chunked Microsoft IIS ISAPI Frontpage fp30reg.dll Chunked

Overflow

W3who_query Microsoft IIS ISAPI w3who.dll Query String Overflow

Imail_thc Imail LDAP Service Buffer Overflow

ms10_025_wmss_connect_funnel Windows Media Services ConnectFunnel Stack Buffer

Overflow

ms05_039_pnp Microsoft Plug and Play Service Overflow

ms06_025_rasmans_reg Microsoft RRAS Service RASMAN Registry Overflow

ms06_025_rras Microsoft RRAS Service Overflow

ms06_040_netapi Microsoft Server Service NetpwPathCanonicalize

Overflow

ms06_070_wkssvc Microsoft Workstation Service NetpManageIPCConnect

Overflow

Figure 39 - Overall

Virtualization and Attack

Topology Diagram

Page 57: Simon Brooks 100042660 - Dissertation - 2010-2011

To determine efficiency of automated signature creation compared with human crafted signature creation 3 May 2011

Simon Brooks Page 57 of 126

ms04_011_pct Microsoft Private Communications Transport Overflow

Table 5 - Windows 2000 Exploits in Metasploit

Each of these exploits was tested first against the Windows 2000 operating system to see if

they were successful. Table 6 is a list of the successful exploits that ran.

Exploit Name Details

ms05_039_pnp This module exploits a stack overflow in the Windows Plug and Play

service. This vulnerability can be exploited on Windows 2000 without

a valid user account. Since the PnP service runs inside the service.exe

process, a failed exploit attempt will cause the system to automatically

reboot.

ms06_040_netapi This module exploits a stack overflow in the NetApi32

CanonicalizePathName() function using the NetpwPathCanonicalize

RPC call in the Server Service. It is likely that other RPC calls could

be used to exploit this service. This exploit will result in a denial of

service on Windows XP SP2 or Windows 2003 SP1. A failed exploit

attempt will likely result in a complete reboot on Windows 2000 and

the termination of all SMB-related services on Windows XP. The

default target for this exploit should succeed on Windows NT 4.0,

Windows 2000 SP0-SP4+, Windows XP SP0-SP1 and Windows 2003

SP0.

Table 6 - Successful Windows 2000 Exploits in Metasploit

4.6.2 Windows XP Attacks

A search was run for “Windows XP” exploits in the Metasploit Framework. Table 7 is the list

of Windows XP XP results, some of the these such as Apple_quicktime_rtsp require extra

software. These exploits will not be used in this experiment.

Exploit Name General Description

Apple_quicktime_rtsp Apple QuickTime 7.1.3 RTSP URI Buffer Overflow

awingsoft_winds3d_sceneurl AwingSoft Winds3D Player 3.5 SceneURL Download and

Execute

Dxstudio_player_exec Worldweaver DX Studio Player <= 3.0.29 shell.execute()

Command Execution

Ie_iscomponentinstalled Internet Explorer isComponentInstalled Overflow

ms06_001_wmf_setabortproc Windows XP/2003/Vista Metafile Escape() SetAbortProc

Code Execution

ms06_013_createtextrange Internet Explorer createTextRange() Code Execution

ms06_055_vml_method Internet Explorer VML Fill Method Code Execution

ms06_057_webview_setslice Internet Explorer WebviewFolderIcon setSlice() Overvlow

ms06_071_xml_core Internet Explorer XML Core Services HTTP Request

Handling

ms03_026_dcom Microsoft RPC DCOM Interface Overflow

Page 58: Simon Brooks 100042660 - Dissertation - 2010-2011

To determine efficiency of automated signature creation compared with human crafted signature creation 3 May 2011

Simon Brooks Page 58 of 126

W3who_query Microsoft IIS ISAPI w3who.dll Query String Overflow

ms03_049_netapi Microsoft Workstation Service

NetAddAlternateComputerName Overflow

ms04_011_lsass Microsoft LSASS Service

DsRolerUpgradeDownlevelServer Overflow

ms04_031_netdde Microsoft NetDDE Service Overflow

ms06_040_netapi Microsoft Server Service NetpwPathCanonicalize Overflow

ms06_070_wkssvc Microsoft Workstation Service NetpManageIPCConnect

Overflow

ms08_067_netapi Microsoft Server Service Relative Path Stack Corruption

ms04_011_pct Microsoft Private Communications Transport Overflow

Table 7 - Windows XP Exploits in Metasploit

Each of these exploits was tested first against the Windows 2000 operating system to see if

they were successful. Table 8 is a list of the successful exploits that ran.

Exploit Name

Details

ms03_026_dcom

This module exploits a stack buffer overflow in the RPCSS

service, this vulnerability was originally found by the Last

Stage of Delirium research group and has been widely

exploited ever since. This module can exploit the English

versions of Windows NT 4.0 SP3-6a, Windows 2000,

Windows XP, and Windows 2003 all in one request :)

ms03_049_netapi

This module exploits a stack buffer overflow in the NetApi32

NetAddAlternateComputerName function using the

Workstation service in Windows XP.

ms04_011_lsass

This module exploits a stack buffer overflow in the LSASS

service, this vulnerability was originally found by eEye.

When re-exploiting a Windows XP system, you will need

need to run this module twice. DCERPC request

fragmentation can be performed by setting 'FragSize'

parameter.

ms04_031_netdde

This module exploits a stack buffer overflow in the NetDDE

service, which is the precursor to the DCOM interface. This

exploit effects only operating systems released prior to

Windows XP SP1 (2000 SP4, XP SP0). Despite Microsoft's

claim that this vulnerability can be exploited without

authentication, the NDDEAPI pipe is only accessible after

successful authentication.

ms06_040_netapi

This module exploits a stack buffer overflow in the NetApi32

CanonicalizePathName() function using the

NetpwPathCanonicalize RPC call in the Server Service. It is

likely that other RPC calls could be used to exploit this

service. This exploit will result in a denial of service on

Windows XP SP2 or Windows 2003 SP1. A failed exploit

attempt will likely result in a complete reboot on Windows

2000 and the termination of all SMB-related services on

Page 59: Simon Brooks 100042660 - Dissertation - 2010-2011

To determine efficiency of automated signature creation compared with human crafted signature creation 3 May 2011

Simon Brooks Page 59 of 126

Windows XP. The default target for this exploit should

succeed on Windows NT 4.0, Windows 2000 SP0-SP4+,

Windows XP SP0-SP1 and Windows 2003 SP0.

ms08_067_netapi

This module exploits a parsing flaw in the path

canonicalization code of NetAPI32.dll through the Server

Service. This module is capable of bypassing NX on some

operating systems and service packs. The correct target must

be used to prevent the Server Service (along with a dozen

others in the same process) from crashing. Windows XP

targets seem to handle multiple successful

exploitation events, but 2003 targets will often crash or hang

on subsequent attempts. This is just the first version of this

module, full support for NX bypass on 2003, along with other

platforms, is still in development.

Table 8 – Successful Windows XP Exploits in Metasploit

4.6.3 Metasploit Attack Overview

The exploits are the attack methods that are to be used against the Honeyd honeypots and XP

and Windows 2000 virtual machines. As mentioned in section 2.3.5, exploits are malicious

code samples that make use of vulnerabilities that may appear in an operating system. For

example MS06_040_netapi exploits a stack buffer overflow in netapi32.dll, a Microsoft

Windows network module.

Metasploit offers a range of payloads designed to run commands against the machine, to

enable access to the files system and upload and download files. One of the popular payloads

of Metasploit is the Meterpreter payload.

Figure 40 - Meterpreter Shell on Backtrack 4 (Makker, 2011)

The payload creates a shell on the target machine whereby commands can be run remotely

such as listing the file directory, logging the keyboard strokes and taking screenshots of the

victim‟s desktop. A payload is sent after an exploit is successfully completed.

Page 60: Simon Brooks 100042660 - Dissertation - 2010-2011

To determine efficiency of automated signature creation compared with human crafted signature creation 3 May 2011

Simon Brooks Page 60 of 126

4.6.4 Overview of the Experiment

Figure 41 is a diagram showing an overview of the experiment network and virtualization

sections of the experiment. VMware has been installed on the physical hardware of the laptop

and is labelled as the “Physical Environment” The VMware Environment lists all of the virtual

machines in this experiment, followed by the directions of the attacks. Finally the Honeyd

environment shows the layout of the Honeyd network as explained in section 4.5.2. Table 9

lists all of the IP addresses and networks used in this experiment, to enable the network to

connect and ping packets to the honeyd environment, the default gateway should be set to the

Honeyd Host CentOS machine. Larger versions of all the network diagrams can be found at

appendix 9.1, 0, and 9.3.

Figure 41 - Virtualization View and Attack Diagram

Virtual Machine IP Address Subnet Default Gateway

VMnet3 (Virtual Network) 10.0.0.0 255.255.255.0 N/A

Metasploit Attacker 10.0.0.7 255.255.255.0 10.0.0.1

Honeyd and Honeycomb

Host

10.0.0.1 255.255.255.0 N/A

Honeyd Entry Router 10.0.0.1 255.255.255.0 N/A

Honeyd Secondary Router 10.0.1.1 255.255.255.0 N/A

Honeyd Victim #1 Windows

20000

10.0.1.51 255.255.255.0 N/A

Honeyd Victim #2 Windows

XP

10.0.1.53 255.255.255.0 N/A

VMware Victim #1 Windows

2000

10.0.0.2 255.255.255.0 10.0.0.1

VMware Victim #2 Windows

XP

10.0.0.3 255.255.255.0 10.0.0.1

Snort IDS 10.0.0.201 255.255.255.0 10.0.0.1

Table 9- Testbed Network Configuration

Page 61: Simon Brooks 100042660 - Dissertation - 2010-2011

To determine efficiency of automated signature creation compared with human crafted signature creation 3 May 2011

Simon Brooks Page 61 of 126

5 Chapter 5 –Analysis of Data Collected

5.1 Introduction

Chapter 4 discussed in detail three automated signature generation tools and chose

Honeycomb as the main software product to use. The documented experiment was split into

two parts, part one gathers the automated signatures for a range of network exploits performed

by the Penetration Testing Toolkit known as Metasploit.

Each attack was fired against two low-interaction honeypots, emulating Windows 2000 SP4

and Windows XP SP2. These honeypots were configured based on a Honeyd network

configuration by Neils Provos (Provos, 2003) , which was re-written for this experiment. A

clean version of Metasploit has been installed on a Windows XP machine to launch the attacks

against the systems.

Part two of the experiment focused on testing the performance of pre-written Snort rules,

written for the chosen Metasploit attacks. This chapter focuses on the analysis of both results

obtained from the two parts of the experiment.

5.2 Experiment One Results

5.2.1 Honeyd Pre-Tests

The following command ran on CentOS “sudo ./honeyd –d –I eth0 –p nmap.prints –f

config.sample 10.0.0.0/8” started Honeyd on the Ethernet interface 10.0.0.1. On start-up,

Honeyd and Honeycomb ran successfully and evidence can be seen below in Figure 42.

Figure 42 – CentOS Honeyd start-up

Two PING tests were sent from the attack virtual machine to each of the Honeyd virtual

machines to ensure a connection was present to both the Honeyd Honeypots on the CentOS

host. Figure 43 and Figure 44 show that the PING request was successful towards both

machines and four replies were given with no signs of any packet loss.

Honeycomb Successfully Registered as a plug-in

Honeyd is listening on Eth0 – 10.0.0.1

Page 62: Simon Brooks 100042660 - Dissertation - 2010-2011

To determine efficiency of automated signature creation compared with human crafted signature creation 3 May 2011

Simon Brooks Page 62 of 126

Figure 43 - Attacker Ping to Honeyd Win2K Machine on CentOS

Figure 44 - Attacker Ping to Honeyd WinXP Machine on CentOS

Figure 45 - Honeyd Ping ICMP Echo Replies to Victim Machine

Figure 45 shows the ICMP ping echo replies logged and sent by Honeyd on behalf of the

Honeyd virtual machines.

5.2.2 Attack Results for Windows 2000 SP4 Attacks

During the testing phase, two Metasploit attacks were tested against full versions of the

operating system Windows 2000 Service Pack 4. The following are the results of the two

Windows 2000 SP4 Metasploit attacks launched against the Windows 2000 SP4 Honeyd

Honeypot.

Page 63: Simon Brooks 100042660 - Dissertation - 2010-2011

To determine efficiency of automated signature creation compared with human crafted signature creation 3 May 2011

Simon Brooks Page 63 of 126

5.2.2.1 ms06_040_netapi

Figure 46 - ms06_040_netapi – Metasploit Results

Figure 46 depicts a screenshot of the MS06_040_netapi attack in action. Noted in the exploit

list an “exploit exception” was made and the connection was “refused by the remote host on

port 445”. When looking at the Logfile from the CentOS Bash command line Figure 47, line 3

shows that Honeycomb has created a state for a new TCP connection and increased the

connections to 1. Line 6 states that an attempted connection from 10.0.0.7 on port 2143

(Attacker) to 10.0.1.51:4444 (Victim) has been killed.

Figure 47 - ms06_040_netapi - Honeyd/Honeycomb CentOS Bash Console Results

Line 19 denotes that a new signature has been added and the history size is now one.

Honeycomb stores any signatures that have been generated in a spate file called

honeycomb.log. Line 32 shows a signature duplicate has been thrown away and Line 33

shows that the connection attempt from the attacker 10.0.0.7:2144 to 10.0.1.51:445 has been

killed. The killed connection attempt by Honeycomb must have made the Metasploit attack

3

19

32

33

Page 64: Simon Brooks 100042660 - Dissertation - 2010-2011

To determine efficiency of automated signature creation compared with human crafted signature creation 3 May 2011

Simon Brooks Page 64 of 126

fail. The log file continued to record blocking attempts for port 4444 and logged signature

duplicates.

Figure 48 shows the generated signatures for the ms06_040_netapi attack for Windows 2000.

When looking at the signatures they all seem to follow a similar pattern. The header shows

alerts for the TCP protocol but does not specify a source or destination address except “any”

or “0.0.0.0” A port is not identified either, only “0,1034”.

The Body contains a descriptive message noting the time and date stamp of creation, followed

by ip_proto “ip”, a variety of results for the “flags” command, TCP Acknowledgement

numbers and the flow of connection which all seems to be recorded as stateless. There has

been no “content:” in the signature this could be because the exploit connection was refused

by honeycomb, resulting in no exploit data being sent.

Figure 48 - Windows 2000 SP4 - ms06_040_netapi - Honeycomb Signature

5.2.2.2 ms05_039_pnp

The second attack, ms05_039_pnp was run against the Windows 2000 Honeyd honeypot

using the Metasploit toolkit. The results were similar to that of the ms06_040_netapi attack,

the connection was refused and Metasploit outputted “Exploit completed, but no session was

created”. Figure 49 shows the results from the Metasploit Console.

Figure 49 – ms05_039_pnp – Metasploit Results

Page 65: Simon Brooks 100042660 - Dissertation - 2010-2011

To determine efficiency of automated signature creation compared with human crafted signature creation 3 May 2011

Simon Brooks Page 65 of 126

When looking at the Honeyd and Honeycomb console a large amount of information was

displayed inside the console, the format of the log was similar to that of ms06_040_netapi and

highlighted signature duplicates. Honeycomb killed connection attempts from 10.0.0.1:2171

and also 10.0.0.1:2180 to 10.0.1.51:4444 on several occasions. Also noted in the log was the

creation of several signatures and signature duplicates noted below.

“Hc_signature_hist.c/210: Signature duplicate – throwing away. (1)”

“Hc_signature_hist.c/256: Adding new signature, history size now 1”

Figure 50 shows the signatures generated for the ms05_039_pnp attack, there are slight

differences compared with the signatures generated for ms06_040_netapi. Line 2 identifies the

source IP Address as 140.173.29.8/32 on any port to the destination IP of 10.0.0.7 (Metasploit

attacker). Line 4 identifies the same source and destination IP addresses 140.173.29.8 and

10.0.0.7. Line 8 has an alert for the source address 224.0.0.0, a multicast Class D network

address (Computer Hope, 2011). The body of the signature does not contain any “payload”

data because Honeycomb closed the connections and the attack failed to send any data.

Figure 50 – Windows 2000 SP4 – ms05_039_pnp

Two attacks have been attempted against the Honeycomb Windows 2000 SP4 honeypot. By

looking at the log files of Honeycomb, it can be assumed that Honeycomb has managed to

identify the attempts to connect to the Honeypot using the TCP protocol and disable the

connection.

The next section displays the results from a collection of the attempted Windows XP Service

Pack 2 attacks against the Victim #2 Windows XP SP2 Honeyd honeypot.

5.2.3 Attack Results for Windows XP Service Pack 2 attacks

During the testing phase, six Metasploit attacks were tested against full versions of the

operating system Windows XP Service Pack 2. The following are the results of the six

Windows 2000 SP4 Metasploit attacks launched against the Windows 2000 SP4 Honeyd

honeypot.

2

4

8

Page 66: Simon Brooks 100042660 - Dissertation - 2010-2011

To determine efficiency of automated signature creation compared with human crafted signature creation 3 May 2011

Simon Brooks Page 66 of 126

5.2.3.1 ms03_026_dcom

Figure 51 - ms03_026_dcom - Metasploit Results

Figure 51 shows a screenshot of the ms03_026_dcom attack being launched against the

Honeyd Windows XP SP2 honeypot. The exploit failed to launch a session because the

connection was refused by 10.0.0.1.53:135 on port 135. Like the previous exploits for

Windows 2000, there is an emerging trend towards Honeycomb attempting to close the

connection from the attacker, before the “Payload” data can be sent.

Figure 52 – Windows XP SP2 – ms03_026_dcom

Figure 52 is a sample of the signature data generated for the ms03_026_dcom, A noticeable

reduction in the amount of signatures can be seen compared with the Windows 2000 SP4

signatures. Only two signatures have been generated whereas ms06_040_netapi and

ms05_039_pnp generated between eight and ten signatures. The content found inside the

ms03_026_dcom is generally the same: Alert TCP SOURCE IP 0.0.0.0/8/ 0 to “Any”

destination.

Page 67: Simon Brooks 100042660 - Dissertation - 2010-2011

To determine efficiency of automated signature creation compared with human crafted signature creation 3 May 2011

Simon Brooks Page 67 of 126

5.2.3.2 ms03_049_netapi

Figure 53 - ms03_049_netapi - Metasploit Results

The second attack was run against the Windows XP SP2 Honeyd honeypot, the exploit failed

to complete its execution with the message “Exploit exception Login Failed: execution

expired” and “Exploit completed, but no session was created”.

Figure 54 shows the results of the signatures created for the ms03_049_netapi exploit.

Compared with the previous attack ms03_026_dcom, there has been an increase in the amount

of signatures generated. These signatures are very similar to the Windows 2000 signatures and

convey no information about the exploit data due to Honeycomb aborting the connection flow.

Figure 54 – Windows XP SP2 – ms03_049_netapi

After testing the remaining four attacks, the majority of the results are similar to the Windows

2000 signatures . All of the exploits failed to send their payload to the attacker machine and

Honeycomb stopped all connections on ports such as 4444 and 445. Section 9.5 contains

screenshots for the remaining Metasploit attacks and signatures that were generated by

Honeycomb.

Page 68: Simon Brooks 100042660 - Dissertation - 2010-2011

To determine efficiency of automated signature creation compared with human crafted signature creation 3 May 2011

Simon Brooks Page 68 of 126

5.2.4 Withdrawal of Experiment Two

Honeycomb has produced a selection of signatures for each attack that was generated for both

the Windows 2000 and Windows XP Honeyd honeypots. During the launch of the attacks on

the systems, Honeycomb blocked the TCP connections on a variety of ports needed to send

the “Payload” data to the victim machines.

Due to the results found in part one of the experiment, the signatures created by Honeycomb

are unsuitable for a comparison against SNORT signatures that are written for these exploits,

therefore no data has been collected for part two of the experiment.

5.3 Critical Evaluation of the Results

The Windows 2000 and XP exploits were tested against VMware Victim #1 and VMware

Victim #2 to ensure they were successful. When each of the exploits was executed against

each honeypot victim, Honeycomb blocked each connection attempt. Due to the attacker‟s

connection attempts being blocked, each exploit ultimately failed.

Honeycomb still managed to generate some SNORT rules but none of the rules recorded the

attacker‟s IP address 10.0.0.7 or any of the specific TCP ports Honeycomb had identified

when blocking the connections. Instead Honeycomb specified the “any” destination or “any”

port notations and bizarre destination addresses such as 224.0.0.0 and 140.173.29.8/32. Due to

these rules being ambiguous the Honeycomb rules did not get placed into SNORT for testing.

Part two of the experiment did not go ahead as planned because the Metasploit attacks failed

in the first experiment. Testing the Honeycomb signatures against SNORT crafted signatures

that were created for successful attacks would invalidate the fairness of the experiment.

An experiment in a different project involved installing Honeyd and Honeycomb on an

unsecured cable modem internet line over a period of 24-hours. A collection of data was

processed and Honeycomb was able to generate precise signatures for the Slammer and

CodeRed II worms (Kreibich & Crowcroft, 2004). The experiment in this project used exploits

and generated attacks in a closed environment.

Honeycomb‟s signature creation algorithm is limited to identifying protocol violations found

at the network and transport layer protocols (Kreibich & Crowcroft, 2004). Metasploit exploits

work at the application layer of the OSI model. From the results generated in this experiment

it can be concluded that Honeycomb is unsuccessful at generating signatures for exploit based

attacks and is generally better at generating signatures for lower-level attacks.

Page 69: Simon Brooks 100042660 - Dissertation - 2010-2011

To determine efficiency of automated signature creation compared with human crafted signature creation 3 May 2011

Simon Brooks Page 69 of 126

6 Chapter 6 – Conclusions and Recommendations

6.1 Report Summary

The report has covered ground surrounding the subject area of signature-based intrusion

detection systems and automated signature generation tools. Primarily the research presented

in Chapter Two – Literature Review, identified a growing threat towards computer systems

from network attacks growing in sophistication.

Research found firstly that intrusion detection systems offer a reasonable method of securing

against network attacks but only attacks aimed at the inside of the network. Secondly the

methods of identifying these attacks relied on user input by means of a “baseline” or a

“signature”. Research found that signature-based detection could be time consuming to write

signatures for network attacks.

Finally the research identified a means of automating the signature process by analysing

network traffic sent to Honeypots. The issue raised from automating the signature process was

how efficient these signatures would be in comparison to those written by hand. A selection of

research methods identified in Chapter Three – Research Methods, were analysed and

experimentation was the most suited method to explore this issue.

In Chapter Four – Conceptual Model of Problem Domain, three automated signature solutions

were critically analysed based on their suitability for this project. Out of the three products

Honeycomb was selected as the product of choice because unlike the other systems critiqued

only one piece of hardware was needed for installation. At the end of the section, two

experiments were proposed to both automatically generate exploit signatures and test their

efficiency against human crafted SNORT rules of the same nature.

From the research collected in Chapter 5 –Analysis of Data Collected, it was identified that

Honeycomb interfered with the Metaspoit exploit attacks and effectively caused them to fail.

This result invalidated the next part of the experiment because the Honeycomb signatures

would ultimately be inferior to SNORT rules. Honeycomb generated signatures for each of the

Metasploit attacks, however research found that Honeyd is cannot properly identify

application layer exploits because the protocol analyser design is based on traffic normalizers

which operate at the lower protocol layers, network and transport layer. Sufficient evidence

has shown that Honeyd is more likely to create signatures for Worm traffic.

Page 70: Simon Brooks 100042660 - Dissertation - 2010-2011

To determine efficiency of automated signature creation compared with human crafted signature creation 3 May 2011

Simon Brooks Page 70 of 126

6.2 Aims and Objectives

This report has been designed with the aim of determining the efficiency of automated

signature creation compared with human crafted signatures. A result to this aim remains

unclear because the signatures generated by Honeycomb were insufficient enough to provide

an accurate enough comparison. It can be assumed that from the research Honeycomb is better

at generating signatures based on worm attacks. To achieve the aim the experiment would

require a slight modification of attack traffic similar to “live” traffic used in the experiment by

(Kreibich & Crowcroft, 2004).

Objective 1 – Evaluate different methods of automated signature creation – This

objective has been met by means of evaluating three automated signature systems,

SweetBait, Honeycyber and Honeycomb. Section 4.2.1 Automated Signature Software

provides a critical overview of each of the products.

Objective 2 – Evaluate different philosophies of signature writing – This objective

has been met by identifying the exploit and vulnerability attack and how these two

attacks can be written for using “Know the Pattern” and “Know the Vulnerability” as

defined by (Trost, 2010)

Objective 3 – Decide which methods/systems to compare – This objective has been

met by deciding to compare Honeycomb signatures to SNORT signatures and is

documented in Chapter Four – Conceptual Model of Problem Domain

Objective 4 – Design and implement test bed – This objective has been met by

providing an experiment design and implementation of a test bed documented in

section 4.5 Experiment Test Bed

Objective 5 – Analysis of results in accordance with Aim – This objective has been

met in section 5.3 Critical Evaluation of the Results

6.3 Critique and Limitations

From the research devised in this report it has been found that Honeycomb is incapable of

generating signatures that fully reflect the network exploit. This is due to the signature

generation algorithm being based on intrusion detection system packet normalisers that can

only read layers three to four of the seven layer OSI model. Research suggests that the product

is more inclined to identify worm traffic rather than exploits, therefore making this more of a

tool to detect anomalous traffic, rather like anomaly-based detection.

Page 71: Simon Brooks 100042660 - Dissertation - 2010-2011

To determine efficiency of automated signature creation compared with human crafted signature creation 3 May 2011

Simon Brooks Page 71 of 126

Originally when the project began it was focused on testing worm programs as the basis of the

attack method. When researching the internet it was proved to be too difficult to simulate a

worm in a non-live environment, therefore the project was changed to use exploit attacks.

6.4 Future work

Overall this project has covered great detail into the nature of network attacks and intrusion

detection systems. The expected outcomes of this project including those of Honeycomb have

differed from what has been anticipated. Honeycomb is clearly not capable of identifying or

dealing with application layer exploits, but this does not necessarily prove that the software is

unsuitable at generating signatures. There is a great deal of room open for discussion as to

how this program may aid in generating IDS signatures for lower layer attacks such as

Worms. By making slight changes to the experiment method, or by changing the automated

signature system, future research could lead to a better understanding of just how efficient

automated signatures really are compared with human crafted signatures.

Page 72: Simon Brooks 100042660 - Dissertation - 2010-2011

To determine efficiency of automated signature creation compared with human crafted signature creation 3 May 2011

Simon Brooks Page 72 of 126

7 Chapter 7 – Critical Evaluation

7.1 Time Management

In order to make this project a success, a great deal of time was spent preparing an overall plan

of how the project was to be laid out. Appendix 9.11 and appendix 9.12 show two plans used

in this project. The use of my project meetings with tutor David Day proved to be an essential

aid appendix 9.13 contains a write up of all the project meetings. During the first semester

time was spent on preparing the information and researching very deeply into the areas

surrounding intrusion detection systems. In addition large amounts of work took place through

researching and practicing the installation and configuration of Honeyd and Honeycomb.

Unfortunately a lot of time was eaten away with the practical side of the project and estimated

completion dates of the literature review and subsequent chapters kept getting moved back

until after the Christmas period. Once the practical parts were completed, more time could be

allocated towards writing up the results of the project. From the experience of managing my

own project I have learned that precise planning do not always pan out, but it is necessary to

have a plan in place.

7.2 Research Skills

The literary review was the most intensive chapter, regarding research. A lot of effort was

made researching into the area of intrusion detection, honeypots and signature generation

tools. Finding papers that covered these areas was particularly difficult because a lot of the

research was found online. To aid in my research I visited the University library and booked

an appointment with the librarian Chris Martindale who aided me in the search of “Intrusion

Detection” papers. My research and writing skills have certainly improved over the course of

this project.

7.3 Practical and Technical Skills

The most challenging aspect of this project has been setting up and installing the project

tested. Large amounts of hours have been spent trying to get Honeyd and Honeycomb to fully

work. There is little documentation surrounding the installation of these two products and I

had to search the internet and Honeyd forums to try and figure out a solution. Eventually I

found instructions that I had to adapt to because they were in Portuguese. However I have

picked up some extensive and useful knowledge about both these products and my Linux

skills have increased dramatically.

7.4 Conclusion

Overall this final year research project has been a challenging and rewarding project. My skill

set has increased in both research and technical areas. I hope that my area of research will help

Page 73: Simon Brooks 100042660 - Dissertation - 2010-2011

To determine efficiency of automated signature creation compared with human crafted signature creation 3 May 2011

Simon Brooks Page 73 of 126

to benefit areas of future work and I will be able to put all my experiences to use when

applying for future jobs after graduation.

Page 74: Simon Brooks 100042660 - Dissertation - 2010-2011

To determine efficiency of automated signature creation compared with human crafted signature creation 3 May 2011

Simon Brooks Page 74 of 126

8 Works Cited Akritidis, P., Anagnostakis, K. & Markatos, E.P., 2005. Efficient Content-Based Detection of

Zero-Day Worms. [Online] Available at:

http://ieeexplore.ieee.org/xpl/freeabs_all.jsp?arnumber=1494469 [Accessed 4 April 2011].

Allen, J. et al., 2000. State of the Practice of Intrusion Detection Technologies 1.4.2 ID

Systems "Hierachy".

Allen, J. et al., 2000. State of the Practice of Intrusion Detection Technologies 1.4.2 ID

Systems Components.

Andrade, L., 2009. Instalação do honeyd 1.5c com honeycomb 0.7 no CentOS 5.2 via

compilação. [Online] Available at: http://aaaleonardo.blogspot.com/2009/02/instalacao-do-

honeyd-15c-com-honeycomb.html [Accessed 2 April 2011].

Apache, 2011. Apache Tomcat. [Online] Available at: http://tomcat.apache.org/ [Accessed 3

April 2011].

BackTrack, 2011. BackTrack Linux - Penetration Testing Distribution. [Online] Available at:

http://www.backtrack-linux.org/ [Accessed 2 April 2011].

Balasubramaniyan, J.S. et al., 1998. An Architecture for Intrusion Detection using

Autonomous Agents. [Online] Available at:

https://www.cerias.purdue.edu/assets/pdf/bibtex_archive/98-05.pdf [Accessed 16 February

2011].

Balzarotti, D., 2006. Testing Network Intrusion Detection Systems. [Online] Available at:

http://www.cs.ucsb.edu/~seclab/projects/sploit/dbalzarotti_thesis.pdf [Accessed 1 April 2011].

Bassi, S., 2005. Tracking the Attacker - Conquering the Bastion of Internet Anonymity.

[Online] Available at:

http://www.cse.scu.edu/~jholliday/COEN150W05/Projects/Tracing%20Attackers.pdf

[Accessed 1 April 2011].

Bautts, T., Dawson, Dawson, T. & Purdy, G.N., 2005. TCP/IP Firewall. In Linux Network

Administrator's Guide. 3rd ed. O'REILLY. pp.122-23.

Bianco, D.J., 2006. Ez Snort Rules - Find the Turffles, Leave the Dirt. [Online] Available at:

http://www.vorant.com/files/EZ_Snort_Rules.pdf [Accessed 3 April 2011].

Briscoe, N., 2000. PC Network Advisor - Understanding The OSI 7-Layer Model. [Online]

Available at: http://www.techsupportalert.com/pdf/t04124.pdf [Accessed 4 February 2011].

Brumley, D., Wang, H., Jha, S. & Song, D., 2007. Creating Vulnerability Signatures Using

Weakest Preconditions. [Online] Available at:

http://ieeexplore.ieee.org/xpl/freeabs_all.jsp?arnumber=4271657 [Accessed 20 April 2011].

Carnegie Mellon University, 2010. Trusted Computing in Embedded Systems. [Online]

Available at: www.cert.org/tces/pdf/archie%20andrews.pdf [Accessed 19 March 2011].

Chazarain, G., Vallette d'Osia, B., Nobelis, N. & Boudaoud, K., 2008. A Virtual High-

Interaction Honeypot. [Online] Available at: http://guichaz.free.fr/writings/hpovua05-

poster.pdf [Accessed 1 April 2011].

Page 75: Simon Brooks 100042660 - Dissertation - 2010-2011

To determine efficiency of automated signature creation compared with human crafted signature creation 3 May 2011

Simon Brooks Page 75 of 126

Chebrolu, S., Abraham, A. & Thomas, J.P., 2004. Feature deduction and ensemble design of

intrusion detection systems. [Online] [Accessed 8 January 2011].

Cherry, K., 2005. What is Quantitative Data? [Online] Available at:

http://psychology.about.com/od/qindex/g/quant_data.htm [Accessed 10 March 2011].

CISCO, 2003. OSI Model 7 Layers. [Online] Available at:

http://aaronmcclintock.com/am_wiki/img/wiki_up/osi-model-7-layers.png [Accessed 3 March

2011].

Citrix Systems, Inc., 2011. XenServer Tech Specs. [Online] Available at:

http://www.citrix.com/English/ps2/products/subfeature.asp?contentID=1681139 [Accessed 2

April 2011].

Clarke, G.E. & Tetz, E., 2010. Book IX: Security Systems Chatper 1: Fundamentals of

Security. In Comp TIA A+ Certification All-In-One For Dummies. 2nd ed. Wiley Publishing,

Inc. p.1040.

Computer Hope, 2011. IP. [Online] Available at:

http://www.computerhope.com/jargon/i/ip.htm [Accessed 4 April 2011].

Cornford, T. & Smithson, S., 2005. Project Research in Information Systems: A Student's

Guide. 2nd ed. Palgrave Macmillan.

Crystal, G., 2010. What is a Hacker? [Online] Available at: http://www.wisegeek.com/what-

is-a-hacker.htm [Accessed 18 March 2011].

Daya, B., 2009. Network Security: History, Importance, and Future. [Online] Available at:

http://web.mit.edu/~bdaya/www/Network%20Security.pdf [Accessed 7 April 2011].

DeLaet, G. & Schauwers, G., 2004. Intrusion Detection System Concepts. In Network Security

Fundamentals: An introduction to the key tools and technologies used to secure network

access. Cisco Press. p.195.

Denning, D.E., 1987. An Intrusion-Detection Model. [Online] Available at:

http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.102.5127&rep=rep1&type=pdf

[Accessed 4 March 2011].

Denscombe, M., 2007. The Good Research Guide. 3rd ed. McGraw-Hill.

Depren, O., Topallar, M., Anarim, E. & Ciliz, M.K., 2005. An intelligent intrusion detection

system (IDS) for anomaly and misuse detection in computer networks. [Online] Available at:

http://www.ft.unicamp.br/RedesComplexas/downloads/An_intelligent_intrusion_detection_sy

stem_for_anomaly_and_misuse_detection_in_computer_networks.pdf [Accessed 1 March

2011].

Einwechter, N., 2001. An Introduction To Distributed Intrusion Detection Systems. [Online]

Available at: http://www.symantec.com/connect/articles/introduction-distributed-intrusion-

detection-systems [Accessed 1 February 2011].

Emerging Threats, 2011. Emerging Threats. [Online] Available at:

http://www.emergingthreats.net/ [Accessed 2 April 2011].

Page 76: Simon Brooks 100042660 - Dissertation - 2010-2011

To determine efficiency of automated signature creation compared with human crafted signature creation 3 May 2011

Simon Brooks Page 76 of 126

Eubanks, R., 2005. Application Firewalls: Don't Forget About Layer 7. [Online] Available at:

http://www.sans.org/reading_room/whitepapers/application/application-firewalls-forget-

about-layer-7_1632 [Accessed 3 April 2011].

Even, L.R., 2000. Intrusion Detection FAQ: What is a Honeypot. [Online] Available at:

http://www.sans.org/security-resources/idfaq/honeypot3.php [Accessed 1 April 2011].

Friedl, S., 2007. SQL Injection Attacks by Example. [Online] Available at:

http://www.unixwiz.net/techtips/sql-injection.html [Accessed 3 April 2010].

Ghorbani, A.A., Lu, W. & Tavallaee, M., 2009. Network Intrusion Detection and Prevention:

Concepts and Techniques. Springer.

GNU, 1991. GNU General Public Licence, Version 2. [Online] Available at:

http://www.gnu.org/licenses/old-licenses/gpl-2.0.html [Accessed 1 April 2011].

Gusfield, D., 1997. Algorithms on strings, trees and sequences: Computer Science and and

Computational Biology. Cambridge University Press.

Handley, M., Paxson, V. & Kreibich, C., 2002. Network Intrusion Detection: Evasion, Traffic

Normalization, and End-to-end protcol semantics. [Online] Available at:

http://www.icir.org/vern/papers/norm-usenix-sec-01.pdf [Accessed 3 April 2011].

HiHAT, 2007. Hihat - Home Page. [Online] Available at: http://hihat.sourceforge.net/

[Accessed 17 February 2011].

Intel Corporation, 2011. Intel Core i4-450M Processor. [Online] Available at:

http://ark.intel.com/Product.aspx?id=49022 [Accessed 2 April 2011].

Internet Systems Consortium, 2010. Internet Domain Survey Host Count January 2010.

[Online] Available at: http://ftp.isc.org/www/survey/reports/hosts.png [Accessed 18 March

2011].

Internet Usage Statistics, 2010. Internet Usage Statistics. [Online] Available at:

http://www.internetworldstats.com/stats.htm [Accessed 17 March 2011].

Kaspersky Lab, 2011. Kaspersky Anti-Virus. [Online] Available at:

http://www.kaspersky.co.uk/virusscanner [Accessed 3 April 2011].

Kemmerer, R.A. & Giovanni, V., 2002. Intrusion Detection: A Brief History and Overview.

[PDF] Available at: www.computer.org/comp/mags/co/2002/04/r4s27.pdf [Accessed 18

February 2011].

Knowles, D., 2003. W32.SQLExp.Worm. [Online] Available at:

http://www.symantec.com/security_response/writeup.jsp?docid=2003-012502-3306-99

[Accessed 4 April 2011].

Kozushko, H., 2003. Intrusion Detection: Host-Based and Network-Based Intrusion Detection

Systems. [Online] Available at:

http://infohost.nmt.edu/~sfs/Students/HarleyKozushko/Papers/IntrusionDetectionPaper.pdf

[Accessed 1 March 2011].

Page 77: Simon Brooks 100042660 - Dissertation - 2010-2011

To determine efficiency of automated signature creation compared with human crafted signature creation 3 May 2011

Simon Brooks Page 77 of 126

Kramer, D., 2001. buffer overflow. [Online] Available at:

http://searchsecurity.techtarget.com/definition/buffer-overflow [Accessed 3 April 2011].

Kreibich, C., 2009. Honeycomb. [Online] Available at:

http://www.icir.org/christian/honeycomb/ [Accessed 10 April 2011].

Kreibich, C. & Crowcroft, J., 2003. Automated NIDS Signature Creation using Honeypots.

[Online] Available at: http://www.icir.org/christian/publications/honeycomb-poster-paper-

sc2003.pdf [Accessed 4 April 2011].

Kreibich, C. & Crowcroft, J., 2004. Honeycomb - Creating Intrusion Detection Signatures

Using Honeypots. [Online] Available at:

http://www.icir.org/christian/publications/honeycomb-hotnetsII.pdf [Accessed 3 December

2011].

Kruegel, C. et al., 2005. Polymorphic Worm Detection. [Online] Available at:

http://www.cs.ucsb.edu/~seclab/projects/polyworms/index.html [Accessed 4 April 2011].

Lawrence Berkeley National Laboratory, 2011. Bro Intrusion Detection System. [Online]

Available at: http://bro-ids.org/Overview.html [Accessed 1 April 2011].

Leggett, S., 2005. Preventing Brute Force Attacks. [Online] Available at:

http://www.webhostgear.com/240.html [Accessed 4 April 2011].

linux.die.net, 2011. Syslog(3) - Linux man page. [Online] Available at:

http://linux.die.net/man/3/syslog [Accessed 4 April 2011].

Lyon, G., 2011. Nmap Security Scanner. [Online] Available at: http://insecure.org/fyodor/

[Accessed 4 April 2011].

Magalhaes, R.M., 2006. Host-Based IDS vs Network-Based IDS (Part 1). [Online] Available

at: http://www.windowsecurity.com/articles/Hids_vs_Nids_Part1.html [Accessed 3 November

2010].

Makker, A.M., 2011. Metasploit Tutorial - With an Example | Exploiting the Vulnerabilities.

[Online] Available at: http://3.bp.blogspot.com/-Xqc5CQ2eTO0/TX-

wNlyz7mI/AAAAAAAAAnI/apgB3aeKHdI/s1600/keylogger%2Bstart.JPG [Accessed 23

April 2011].

McCanne, S., Leres, C. & Jacobson, V., 2010. TCPDump/Libpcap. [Online] Available at:

http://www.tcpdump.org/ [Accessed 3 April 2011].

Miller, L. & Gregory, P.H., 2009. Part II: Domains. In CCISP for Dummies. 3rd ed. John

Wiley & Sons. p.406.

Mohammed, M.M.Z.E., Chan, A.H. & Ventura, N., 2009. Honeycyber: Automated Signature

Generation For Zero-Day Polymorphic Worms. [Online] Available at:

202.194.20.8/proc/milcom08/milcom08/pdfs/1386.pdf [Accessed 4 November 2010].

Morton, D., 1997. PC Network Advisor - Understanding Firewalls. [Online] Available at:

http://www.techsupportalert.com/pdf/s0499.pdf [Accessed 3 March 2011].

Page 78: Simon Brooks 100042660 - Dissertation - 2010-2011

To determine efficiency of automated signature creation compared with human crafted signature creation 3 May 2011

Simon Brooks Page 78 of 126

Mukherjee, B., Levitt, K.N. & Herberlein, T.L., 2002. Network Intrusion Detection. [Online]

Available at: http://www.cc.gatech.edu/~wenke/ids-readings/network_id.pdf [Accessed 3

January 2011].

negVigilance, Inc., 2009. winhoneydconfigurator gif image. [Online] Available at:

http://www.netvigilance.com/images/winhoneydconfigurator-big.gif [Accessed 1 April 2011].

networksecuritytoolkit.org, 2011. Network Security Toolkit (NST v2.13.0). [Online] Available

at: http://networksecuritytoolkit.org/nst/index.html [Accessed 4 April 2011].

Niem, J., 2008. Developing a Snort Dynamic Preprocessor. [Online] Available at:

http://www.sans.org/reading_room/whitepapers/tools/developing-snort-dynamic-

preprocessor_32874 [Accessed 4 April 2011].

Northcutt, S. et al., 2005. Chapter 3 - Stateful Firewalls. In Insider Network Perimeter

Security, 2/E. Sams Publishing. pp.55-57.

Oates, B.J., 2005. Researching Information systems and computing. Sage Publications Ltd.

Offensive Security, 2011. Exploit Database. [Online] Available at: http://www.exploit-

db.com/ [Accessed 2 April 2011].

Open Information Security Foundation, 2011. Suricata Downloads. [Online] Available at:

http://www.openinfosecfoundation.org/index.php/downloads [Accessed 1 April 2011].

Oracle, 2011. Chapter 6. Virtual Networking. [Online] Available at:

http://www.virtualbox.org/manual/ch06.html [Accessed 8 April 2011].

Oracle, 2011. Windows 7 running a Ubuntu 10.10 VM. [Online] Available at:

http://www.virtualbox.org/attachment/wiki/Screenshots/win7.png [Accessed 1 April 2011].

Portokalidis, G. & Bos, H., 2005. SweetBait: Zero-Hour Worm Detection and Containment

Using Honeypots. [Online] Available at:

citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.80.815 [Accessed 4 January 2011].

Portokalidis, G., Slowinska, A. & Bos, H., 2006. Argos: an Emulator for Fingerprinting Zero-

Day Attacks. [Online] Available at: portal.acm.org/citation.cfm?id=1217938 [Accessed 3

April 2011].

Provos, N., 2002. Honeyd(8) - Linux Man Page. [Online] Available at:

http://linux.die.net/man/8/honeyd [Accessed 22 February 2011].

Provos, N., 2003. Sample Network Template Ver 0.7. [Online] Available at:

http://www.honeyd.org/config/honeyd.conf.networks [Accessed 5 April 2011].

Provos, N., 2004. Honeyd - How do I interpret the fields in Honeyd's packet log? [Online]

Available at: http://honeyd.org/faq.php#logformat [Accessed 13 February 2011].

Provos, N., 2007. Honeyd Frequently Asked Questions - What is Honeyd? [Online] Available

at: http://www.honeyd.org/faq.php [Accessed 1 April 2011].

Provos, N. & Holz, T., 2007. Chapter 2. High-Interaction Honeypots. In Virtual Honeypots:

From Botnet Tracking to Intrusion Detection. 1st ed. Addison Wesley Professional. pp.19-21.

Page 79: Simon Brooks 100042660 - Dissertation - 2010-2011

To determine efficiency of automated signature creation compared with human crafted signature creation 3 May 2011

Simon Brooks Page 79 of 126

Provos, N. & Thorsten, H., 2007. Chapter 3. Low-Interaction Honeypots. In Virtual

Honeypots: From Botnet Tracking to Intrusion Detection. 1st ed. Addison Wesley

Professional. pp.71-73.

Rapid7, 2011. What are the Metasploit Framework and the Metasploit Project. [Online]

Available at: http://www.metasploit.com/learn-more/what-is-it/ [Accessed 2 April 2011].

RedHat, 2010. What is Virtualization. [Online] Available at:

http://www.redhat.com/f/pdf/virtualization/gunner_virtual_paper2.pdf [Accessed 3 April

2011].

Reid, F., 2003. MS-SQL Worm Signature. [Online] Available at: http://lists.virus.org/snort-

users-0301/msg00807.html [Accessed 3 April 2011].

Rexworthy, B., 2009. Intrusion detections systems - an outmoded network protection model.

Network Security, June. pp.17-19. Available at:

http://www.sciencedirect.com/science/article/B6VJG-4WNRC2G-

9/2/9b9179dc128bdc92756815cc45c6a358 [Accessed 10 November 2010].

Rijckenberg, M., 2010. SynapticHowto. [Online] Available at:

https://help.ubuntu.com/community/SynapticHowto [Accessed 1 April 2011].

Scarfone, K. & Mell, P., 2007. Guide to Intrusion Detection and Prevention Systems (IDPS).

[Online] Available at: http://csrc.nist.gov/publications/nistpubs/800-94/SP800-94.pdf

[Accessed 7 March 2011].

Schear, N., Albrecht, D.R. & Borisov, N., 2008. High-speed Matching of Vulnerability

Signatures. [Online] Available at: http://hatswitch.org/~nikita/papers/vespa-raid08.pdf

[Accessed 7 April 2011].

SearchSecurity, 2000. Payload. [Online] Available at:

http://searchsecurity.techtarget.com/definition/payload [Accessed 4 December 2010].

Secpoint, 2011. What is a security exploit? [Online] Available at:

http://www.secpoint.com/what-is-real-exploits.html [Accessed 3 April 2011].

SecPoint, 2011. What is a vulnerability? [Online] Available at:

http://www.secpoint.com/what-is-a-vulnerability.html [Accessed 22 April 2011].

Security4web, 2011. What is Malware? [Online] Available at:

http://www.security4web.org/page.php?id=10 [Accessed 3 April 2011].

Skoudis, E., 2003. Defining the Problem. In Malware: Fighting Malicious Code. 1st ed.

Prentice Hall. p.3.

Snort.org, 2009. Exploit-based signature is dead? or not. [Online] Available at:

https://forums.snort.org/forums/rules/topics/exploit-based-signature-is-dead-or-not [Accessed

5 April March].

Softpedia, 2006. Labrea Description. [Online] Available at:

http://linux.softpedia.com/get/System/Monitoring/labrea-14507.shtml [Accessed 3 April

2011].

Page 80: Simon Brooks 100042660 - Dissertation - 2010-2011

To determine efficiency of automated signature creation compared with human crafted signature creation 3 May 2011

Simon Brooks Page 80 of 126

Softpedia, 2011. Citrix XenServer Screenshots. [Online] Available at: http://i1-win.softpedia-

static.com/screenshots/Citrix-XenServer_2.png [Accessed 2 April 2011].

Solomon, A., 1995. All About Viruses. [Online] Available at:

http://vx.netlux.org/lib/aas10.html [Accessed 4 April 2011].

Sourcefire, 2011. Snort Home Page. [Online] Available at: http://www.snort.org/ [Accessed 1

April 2011].

Sterling, B., 1992. The Hacker Crackdown - Law and Disorder on the Electronic Fronteir.

Bantam Books.

Stewart, K., 2011. Ubuntu 10.0.4.2 Released. [Online] Available at:

https://lists.ubuntu.com/archives/ubuntu-announce/2011-February/000141.html [Accessed 2

April 2011].

Stringer, G., 1999. The Internet. [Online] Available at:

services.exeter.ac.uk/cmit/modules/the_internet/MITxx14-notes.pdf [Accessed 15 March

2011].

Symantec , 2007. Symantec Distributed Intrusion Detection System. [Online] Available at:

http://www.symantec.com/connect/sites/default/files/infocus/dids.gif [Accessed 29 March

2011].

TheTechJournal, 2010. How To: Run Windows in Ubuntu with VMware Player. [Online]

Available at: http://cdn.thetechjournal.com/wp-

content/uploads/vmware_player_linux_xp_1.jpg [Accessed 3 April 2011].

Trigaux, R., 2000. A history of hacking. [Online] Available at:

http://www.sptimes.com/Hackers/history.hacking.html [Accessed 7 April 2011].

Tripwire, 2011. Tripwire Enterprise. [Online] Available at: http://www.tripwire.com/it-

compliance-products/te/ [Accessed 1 April 2011].

Trochlm, W.M., 2006. Qualitative Data. [Online] Available at:

http://www.socialresearchmethods.net/kb/qualdata.php [Accessed 10 March 2011].

Trost, R., 2010. Two Detection Philosophies: Signature and Anomaly Based. In Trost, R.

Practical Intrusion Analysis. Pearson Education, Inc.

ttcplinux, 2000. TCP State Transitions. [Online] Available at:

http://ttcplinux.sourceforge.net/documents/one/tcpstate/tcpstate.html [Accessed 3 April 2011].

Ukkonen, E., 1995. On-line construction of suffix trees. [Online] Available at:

citeseer.ist.psu.edu/viewdoc/summary?doi=10.1.1.10.751 [Accessed 4 April 2011].

Veysset, F. & Laurent, B., 2006. Honeypot technologies - 2006 First Conference / tutorial.

[Online] Available at: http://www.first.org/conference/2006/papers/veysset-franck-slides.pdf

[Accessed 1 April 2011].

VMware Inc., 2009. VMware ESX and VMware ESXi. [Online] Available at:

http://www.vmware.com/files/pdf/VMware-ESX-and-VMware-ESXi-DS-EN.pdf [Accessed 2

April 2011].

Page 81: Simon Brooks 100042660 - Dissertation - 2010-2011

To determine efficiency of automated signature creation compared with human crafted signature creation 3 May 2011

Simon Brooks Page 81 of 126

Webopedia, 2010. The 7 Layers of the OSI Model. [Online] Available at:

http://www.webopedia.com/quick_ref/OSI_Layers.asp [Accessed 3 April 2011].

Wichmann, R., 2006. The SAMHAIN file integrity / host-based intrusion detection system.

[Online] Available at: http://www.la-samhna.de/samhain/ [Accessed 12 April 2011].

Wolfe, M.M., 2007. Facing Down Computer Security Threats. [Online] Available at:

http://www.dicksteinshapiro.com/files/Publication/cb195616-52fa-450c-be73-

00df1e396a96/Presentation/PublicationAttachment/e7400025-001b-4b72-b9a1-

0604dffc1b70/NYLJ_Wolfe_byline.pdf [Accessed 18 March 2011].

Wulf, W. et al., 1974. HYDRA: The Kernel of a Multiprocessor Operating System. [Online]

Available at:

http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.62.8610&rep=rep1&type=pdf

[Accessed 7 January 2011].

Yasm, C., 2009. Prelude as a Hybrid IDS Framework. [Online] Available at:

http://www.sans.org/reading_room/whitepapers/awareness/prelude-hybrid-ids-

framework_33048 [Accessed 1 April 2011].

Page 82: Simon Brooks 100042660 - Dissertation - 2010-2011

To determine efficiency of automated signature creation compared with human crafted signature creation 3 May 2011

Simon Brooks Page 82 of 126

9 Appendices

9.1 Experiment Part One - Network Topology Diagram

Page 83: Simon Brooks 100042660 - Dissertation - 2010-2011

To determine efficiency of automated signature creation compared with human crafted signature creation 3 May 2011

Simon Brooks Page 83 of 126

9.2 Experiment Part Two - Network Topology Diagram

Page 84: Simon Brooks 100042660 - Dissertation - 2010-2011

To determine efficiency of automated signature creation compared with human crafted signature creation 3 May 2011

Simon Brooks Page 84 of 126

9.3 Overall Virtualization and Attack Topology Diagram

Page 85: Simon Brooks 100042660 - Dissertation - 2010-2011

To determine efficiency of automated signature creation compared with human crafted signature creation 3 May 2011

Simon Brooks Page 85 of 126

9.4 Metasploit Website Module Reference list

http://www.metasploit.com/modules/exploit/windows/browser/apple_quicktime_rtsp

http://www.metasploit.com/modules/exploit/windows/browser/awingsoft_winds3d_sceneurl

http://www.metasploit.com/modules/exploit/windows/browser/dxstudio_player_exec

http://www.metasploit.com/modules/exploit/windows/browser/ie_iscomponentinstalled

http://www.metasploit.com/modules/exploit/windows/browser/ms06_001_wmf_setabortproc

http://www.metasploit.com/modules/exploit/windows/dcerpc/ms03_026_dcom

http://www.metasploit.com/modules/exploit/windows/isapi/w3who_query

http://www.metasploit.com/modules/exploit/windows/smb/ms03_049_netapi

http://www.metasploit.com/modules/exploit/windows/smb/ms04_011_lsass

http://www.metasploit.com/modules/exploit/windows/smb/ms04_031_netdde

http://www.metasploit.com/modules/exploit/windows/smb/ms06_040_netapi

http://www.metasploit.com/modules/exploit/windows/smb/ms06_070_wkssvc

http://www.metasploit.com/modules/exploit/windows/smb/ms08_067_netapi

Page 86: Simon Brooks 100042660 - Dissertation - 2010-2011

To determine efficiency of automated signature creation compared with human crafted signature creation 3 May 2011

Simon Brooks Page 86 of 126

9.5 Remaining Four Windows XP SP2 Attack Results

The following are the remaining results for the Windows XP SP2 Honeyd Honeypot. These

results were similar to what has been noted in the Body of the report.

9.5.1 ms08_067_netapi

Figure 55 - MS08_067_netapi - Metasploit Results

Figure 56 – Windows XP SP2 – MS08_067_netapi

9.5.2 ms06_040_netapi

Figure 57 - MS06_040_netapi (XP) - Metasploit Results

Page 87: Simon Brooks 100042660 - Dissertation - 2010-2011

To determine efficiency of automated signature creation compared with human crafted signature creation 3 May 2011

Simon Brooks Page 87 of 126

Figure 58 – Windows XP SP2 – MS06_040_netapi

9.5.3 ms04_031_netdde

Figure 59 - MS04_031_netdde – Metasploit Results

Figure 60 – Windows XP SP2 – MS04_031_netdde

Page 88: Simon Brooks 100042660 - Dissertation - 2010-2011

To determine efficiency of automated signature creation compared with human crafted signature creation 3 May 2011

Simon Brooks Page 88 of 126

9.5.4 MS04_011_lsass

Figure 61 - MS04_011_lsass – Metasploit Results

Figure 62 – Windows XP SP2 – MS04_011_lsass

Page 89: Simon Brooks 100042660 - Dissertation - 2010-2011

To determine efficiency of automated signature creation compared with human crafted signature creation 3 May 2011

Simon Brooks Page 89 of 126

9.6 Honeyd and Honeycomb Installation Instructions

These installation instructions have been modified from (Andrade, 2009)

# instala honeyd 1.5c

# install honeyd 1.5c

wget http://www.citi.umich.edu/u/provos/honeyd/honeyd-1.5c.tar.gz

tar -zxvf honeyd-1.5c.tar.gz

yum install pcre pcre-devel libpcap libpcap-devel

wget http://monkey.org/~provos/libevent-1.4.8-stable.tar.gz

tar -zxvf libevent-1.4.8-stable.tar.gz

yum install gcc

cd libevent-1.4.8-stable

./configure --prefix=/usr/local/libevent

make

make install

cd ..

wget http://prdownloads.sourceforge.net/libdnet/libdnet-1.11.tar.gz?download

tar -zxvf libdnet-1.11.tar.gz

cd libdnet-1.11

yum install gcc-c++

./configure --prefix=/usr/local/libdnet

make

make install

cd ..

cd honeyd-1.5c

yum install libtool readline-devel zlib-devel python-devel

./configure --prefix=/usr/local/honeyd --with-libevent=/usr/local/libevent --with-

libdnet=/usr/local/libdnet --with-python

make

make install

cp -r scripts/ /usr/local/honeyd/

cd ..

# instala-se o honeycomb 0.7

# install the honeycomb 0.7

wget http://www.icir.org/christian/downloads/honeycomb-0.7.tar.gz

wget http://www.icir.org/christian/downloads/libstree-0.4.2.tar.gz

# instala libstree (pré-requisito para honeycomb)

# install libstree (pre-requisite for honeycomb)

tar -zxvf libstree-0.4.2.tar.gz

cd libstree-0.4.2

./configure

make

make install

cd ..

tar -zxvf honeycomb-0.7.tar.gz

cd honeycomb-0.7

./configure --with-honeyd=/usr/local/honeyd/bin/honeyd --with-libdnet=/usr/local/libdnet/bin --with-

libevent=/usr/local/libevent --enable-debugging

cp -R ../honeyd-1.5c honeyd/

Page 90: Simon Brooks 100042660 - Dissertation - 2010-2011

To determine efficiency of automated signature creation compared with human crafted signature creation 3 May 2011

Simon Brooks Page 90 of 126

[root@localhost honeycomb-0.7]# cp -R /home/monsi/honeyd-1.5c honeyd/

make

make install

# reinstala honeyd1.5c com suporte ao honeycomb

# reinstall honeyd1.5c with honeycomb support

cd honeyd

./configure --prefix=/usr/local/honeyd --with-libevent=/usr/local/libevent --with-

libdnet=/usr/local/libdnet --with-python --with-plugins=honeycomb

make clean

make

make install

ln -s /usr/local/lib/libhoneycomb.so /usr/lib/libhoneycomb.so

ln -s /usr/local/lib/libstree.so.0 /usr/lib/libstree.so.0

chmod -R 766 /usr/local/honeyd/share/honeyd/webserver

chmod -R 766 /usr/local/honeyd/share/honeyd/webserver/htdocs/styles/

For the honeycomb configuration, see:

# reference: http://jsfyp.wordpress.com/2007/03/27/running-honeycomb

cd ..Honeycomb getting work!

root@jason-desktop:/usr/local/share/honeyd# honeyd -df test1.conf -p nmap.prints -x xprobe2.conf -a

nmap.assoc -l /var/log/honeyd -i eth0 192.168.1.0/24

Honeyd V1.5b Copyright (c) 2002-2004 Niels Provos

honeyd[645]: started with -df test1.conf -p nmap.prints -x xprobe2.conf -a nmap.assoc -l

/var/log/honeyd -i eth0 192.168.1.0/24

Warning: Impossible SI range in Class fingerprint “IBM OS/400 V4R2M0″

Warning: Impossible SI range in Class fingerprint “Microsoft Windows NT 4.0 SP3″honeyd[645]:

listening promiscuously on eth0: (arp or ip proto 47 or (udp and src port 67 and dst port 68) or (ip and

(net 192.168.1.0/24))) and not ether src 00:17:31:b6:9a:a1

honeyd[645]: registering plugin „Honeycomb‟ (0.7)

honeyd[645]: Demoting process privileges to uid 65534, gid 65534

honeyd[645]: Killing unknown connection: tcp (208.65.153.253:80 – 192.168.1.11:52826)

Copy Honeycomb Configuration to the sample.conf

Sudo echo 0 > /proc/sys/net/ipv4/ip_forward

/sbin/route –n add –net 10.0.0.0/8 gw 127.0.0.1

Sudo honeyd –d –I lo –p nmap.prints –f config.sample 10.0.0.0/8

Error: Webserver: require read access to /usr/local/honeyd//share/honeyd/webserver/htdocs/styles:

Permission denied

To fix this : http://www.linuxquestions.org/questions/linux-server-73/honeyd-set-up-817138/

[root@localhost honeyd]# ./honeyd --fix-webserver-permissions

http://www.linuxquestions.org/questions/linux-server-73/honeyd-set-up-817138/

didn‟t work

./honeyd -df config.sample -p nmap.prints -x xprobe2.conf -a nmap.assoc -l /var/log/honeyd -i eth0

10.0.0.0/8

Page 91: Simon Brooks 100042660 - Dissertation - 2010-2011

To determine efficiency of automated signature creation compared with human crafted signature creation 3 May 2011

Simon Brooks Page 91 of 126

9.7 Honeyd + Honeycomb Default Configuration (Provos, 2003)

(Andrade, 2009)

###Honeyd Config

### Windows computers

create windows

set windows personality “Microsoft Windows NT 4.0 SP5-SP6″

set windows default tcp action reset

set windows default udp action reset

add windows tcp port 80 “scripts/iisemulator/iisemul8.pl”

add windows tcp port 139 open

add windows tcp port 137 open

add windows udp port 137 open

add windows udp port 135 open

add windows tcp port 110 “sh scripts/pop3.sh”

add windows tcp port 25 “sh scripts/smtp.sh”

add windows tcp port 21 “sh scripts/ftp.sh”

set windows uptime 3284460

bind 192.168.1.11 windows

# Honeycomb plugin configuration

#

# Add this to your honeyd configuration file and tweak as you see fit!

# ____________________________________________________________________

# Whether to run the plugin (1) or not (0)

option honeycomb enable 1

# What Snort alert category we use for our signatures

option honeycomb snort_alert_class alert

# The name of the output log file to which we log generated signatures

option honeycomb sig_output_file /tmp/honeycomb.log

# How many IP packets we keep in mind and search

# for matching data.

option honeycomb ip_backlog 100

# How many attempted UDP connections we maintain state for at any one time

option honeycomb udp_conns_max 1000

# How many answered UDP connections we maintain state for at any

# one time. Once a connection is answered, it is moved to a different

# hashtable. We therefore keep state for udp_conns_max attempted

# connections PLUS udp_dataconns_max answered ones.

option honeycomb udp_dataconns_max 1000

# The maximum number of bytes flowing in a single direction without

# any payload coming the other way during the UDP dialog that we

# store. More data going in one direction without any real data

# going the other way is not stored, as we‟re currently not looking

# for data there.

#

# This is also the maximum string size the longest common substring

# algorithm in libstree needs to deal with, so we don‟t make this

# too high to avoid performance hits.

option honeycomb udp_max_msg_size 5000

Page 92: Simon Brooks 100042660 - Dissertation - 2010-2011

To determine efficiency of automated signature creation compared with human crafted signature creation 3 May 2011

Simon Brooks Page 92 of 126

# We stop hunting for patterns at some point into a UDP exchange.

# The following defines the number of total bytes inbound before

# we stop caring.

option honeycomb udp_max_bytes 10000

# The minimum pattern length we require before we consider

# a string match in UDP payload meaningful:

option honeycomb udp_pattern_minlen 5

# How many initiated TCP connections we maintain state for at any one time.

option honeycomb tcp_conns_max 65000

# How many established TCP connections we maintain state for at any

# one time. Once a connection is established, it is moved to a different

# hashtable. We therefore keep state for tcp_conns_max unestablished

# connections PLUS tcp_dataconns_max established ones.

option honeycomb tcp_dataconns_max 1000

# The maximum number of bytes flowing in a single direction without

# any payload coming the other way during the TCP dialog that we

# store. More data going in one direction without any real data

# going the other way is not stored, as we‟re currently not looking

# for data there.

#

# This is also the maximum string size the longest common substring

# algorithm in libstree needs to deal with, so we don‟t make this

# too high to avoid performance hits.

option honeycomb tcp_max_msg_size 5000

# We stop hunting for patterns at some point into a TCP dialogue.

# The following defines the number of total bytes inbound before

# we stop caring.

option honeycomb tcp_max_bytes 10000

# For TCP, we also buffer the incoming payloads in one single buffer

# directly. This defines the size of that buffer.

option honeycomb tcp_max_buffering_in 1000

# The minimum pattern length we require before we consider

# a string match in TCP payload meaningful:

option honeycomb tcp_pattern_minlen 5

# The number of slots in the hashtables:

option honeycomb conns_hash_slots 199

# The connection hashtables are periodically checked for dead connections

# we‟re no longer interested in (this doesn‟t automatically mean terminated

# connections, as we need to keep connections around in order to be able to

# have something to compare new ones against!). This setting defines

# the interval in seconds between cleanups.

option honeycomb conns_hash_cleanup_interval 10

# How many generated signatures we keep around before we

# start to forget some.

option honeycomb sighist_max_size 200

Page 93: Simon Brooks 100042660 - Dissertation - 2010-2011

To determine efficiency of automated signature creation compared with human crafted signature creation 3 May 2011

Simon Brooks Page 93 of 126

# Detected signatures are kept in a history structure and reported

# periodically. This settings defines how long to wait between those

# reports. During the waiting period, existing signatures can be

# improved upon through new traffic flows.

option honeycomb sighist_interval 10

Page 94: Simon Brooks 100042660 - Dissertation - 2010-2011

To determine efficiency of automated signature creation compared with human crafted signature creation 3 May 2011

Simon Brooks Page 94 of 126

9.8 Honeyd and Honeycomb Used in the Experiment

### Honeyd Configuration File ###

### Sample Network Template Ver 0.7 ###

###Modified by Simon Brooks from

http://www.honeyd.org/config/honeyd.conf.networks###

# Last Updated: 20 April 2011 #

#####################################################################

### ###

### This sample network configuration template builds a virtual ###

### network step-by-step. The network we simulate has multiple ###

### hops, two entry points, a GRE tunnel to a remote location ###

### and integrates external physical hosts to the virtual network.###

### The template builds the network in the accompanying paper: ###

### "Simulating Networks with Honeyd". The latest version of the ###

### paper is available at: ###

### www.paladion.net/papers/simulating_networks_with_honeyd.pdf ###

### ###

### ###

### Authors: Roshen Chandran, Sangita Pakala ###

### Paladion Networks [http://www.paladion.net] ###

### ###

### Thanks to: Niels Provos, Lance Spitzner, Ed Balas, ###

### Laurent Oudot ###

#####################################################################

#####################################################################

### Start by creating an entry router for the network. Then add ###

### some IP addresses that are directly reachable from the router.###

### Then add a new router connected to the first, and the IPs ###

### directly reachable from that. This is the essential strategy ###

Page 95: Simon Brooks 100042660 - Dissertation - 2010-2011

To determine efficiency of automated signature creation compared with human crafted signature creation 3 May 2011

Simon Brooks Page 95 of 126

### of building a virtual network. ###

### ###

### On the desktops in the LAN, point the default gateway to the ###

### entry router, or add a route to the virtual network via the ###

### entry router. Run arpd to respond to requests for 10.0.0.0/24 ###

### network. ###

### ###

#####################################################################

### To create the router at the entry point, use the

### route entry command and specify the IP address of

### the router and the network reachable through it.

route entry 10.0.0.1 network 10.0.0.0/24

### To specify the IP addresses directly reachable from

### a router, use the route link configuration. In the

### example below, we specify that the 10.0.1.0/24

### network is directly reachable from the 10.0.0.100 router.

route 10.0.0.1 link 10.0.1.0/24

### Add a new router connected to an existing router

### in the network by using the route add net

### directive. Specify the network range that can be

### reached by the new router and the IP address of the

### new router. In the example below, we add

### 10.0.1.100 as a new router that serves the

### 10.1.0.0/16 network and connected to the first

### router 10.0.0.100

Page 96: Simon Brooks 100042660 - Dissertation - 2010-2011

To determine efficiency of automated signature creation compared with human crafted signature creation 3 May 2011

Simon Brooks Page 96 of 126

#route 10.0.0.1 add net 10.1.0.0/16 10.0.1.1

### Specify the range of IP addresses that are directly

### reachable from the new router with the route link

### configuration. Here, we indicate that 10.1.0.0/16

### is directly accessible from the router 10.0.1.100 we

### newly added

#route 10.0.1.1 link 10.1.0.0/16

#####################################################################

### Here we add another router connected to 10.0.1.100 ###

### that can reach the 10.1.1.0/24 network. The new ###

### router takes the IP 10.1.0.100. Additionally, we ###

### also specify the network characteristics of that ###

### link using the latency, loss and bandwidth keywords. ###

#####################################################################

#route 10.0.1.1 add net 10.1.1.0/24 10.1.0.1 latency 50ms loss 0.1 bandwidth 1Mbps

### With the route link configuration, we next

### specify that the 10.1.1.0/24 network is directly

### accessible from the 10.1.0.100 router.

#route 10.1.0.1 link 10.1.1.0/24

#####################################################################

### External physical machines can be integrated into the ###

Page 97: Simon Brooks 100042660 - Dissertation - 2010-2011

To determine efficiency of automated signature creation compared with human crafted signature creation 3 May 2011

Simon Brooks Page 97 of 126

### virtual network topology of the honeynet. The bind ###

### to interface configuration is used to attach external ###

### machines into the network. In our example here, ###

### the external machine at 10.1.1.53 is integrated ###

### into the virtual network through eth0. ###

#####################################################################

#bind 10.1.1.53 to eth0

#####################################################################

### IP addresses are assigned to virtual hosts that we ###

### want to simulate within Honeyd with the bind ###

### configuration. Here, we bind the honeypot IPs ###

### to a template called windows that we have defined. ###

#####################################################################

### Windows XP SP1

create windowsxpsp1

set windowsxpsp1 personality "Microsoft Windows XP Professional SP1"

#add windows tcp port 80 "perl scripts/iis-0.95/iisemul8.pl"

add windowsxpsp1 tcp port 139 open

add windowsxpsp1 udp port 135 open

add windowsxpsp1 udp port 445 open

add windowsxpsp1 udp port 1025 open

set windowsxpsp1 default tcp action reset

set windowsxpsp1 default udp action reset

### Windows 2000 SP4

create windows2000

Page 98: Simon Brooks 100042660 - Dissertation - 2010-2011

To determine efficiency of automated signature creation compared with human crafted signature creation 3 May 2011

Simon Brooks Page 98 of 126

set windows2000 personality "Microsoft Windows 2000 SP4"

#add windows tcp port 80 "perl scripts/iis-0.95/iisemul8.pl"

add windows2000 tcp port 139 open

add windows2000 udp port 135 open

add windows2000 udp port 445 open

add windows2000 udp port 1025 open

set windows2000 default tcp action reset

set windows2000 default udp action reset

### Windows XP SP2

create windowsxp

set windowsxp personality "Microsoft Windows XP SP2"

add windowsxp tcp port 80 "perl scripts/iis-0.95/iisemul8.pl"

add windowsxp tcp port 139 open

add windowsxp tcp port 137 open

add windowsxp tcp port 443 open

add windowsxp tcp port 445 open

add windowsxp udp port 137 open

add windowsxp udp port 135 open

set windowsxp default tcp action reset

set windowsxp default udp action reset

bind 10.0.1.51 windows2000

bind 10.0.1.53 windowsxp

#bind 10.0.0.61 windowsxpsp1

#bind 10.0.0.60 windows2000

#####################################################################

### The routers we have created in the virtual network ###

### also need to be bound to templates to model their ###

Page 99: Simon Brooks 100042660 - Dissertation - 2010-2011

To determine efficiency of automated signature creation compared with human crafted signature creation 3 May 2011

Simon Brooks Page 99 of 126

### behavior. We have created a template called router ###

### and bound the router IP addresses to that template. ###

#####################################################################

### Cisco Router

create router

set router personality "Cisco IOS 11.3 - 12.0(11)"

set router default tcp action reset

set router default udp action reset

add router tcp port 23 "/usr/bin/perl scripts/router-telnet.pl"

set router uid 32767 gid 32767

set router uptime 1327650

bind 10.0.0.1 router

bind 10.0.1.1 router

#bind 10.1.0.1 router

#bind 10.2.0.100 router

# Honeycomb plugin configuration

#

# Add this to your honeyd configuration file and tweak as you see fit!

# ____________________________________________________________________

# Whether to run the plugin (1) or not (0)

option honeycomb enable 1

# What Snort alert category we use for our signatures

option honeycomb snort_alert_class alert

Page 100: Simon Brooks 100042660 - Dissertation - 2010-2011

To determine efficiency of automated signature creation compared with human crafted signature creation 3 May 2011

Simon Brooks Page 100 of 126

# The name of the output log file to which we log generated signatures

option honeycomb sig_output_file /tmp/honeycomb.log

# How many IP packets we keep in mind and search

# for matching data.

option honeycomb ip_backlog 100

# How many attempted UDP connections we maintain state for at any one time

option honeycomb udp_conns_max 1000

# How many answered UDP connections we maintain state for at any

# one time. Once a connection is answered, it is moved to a different

# hashtable. We therefore keep state for udp_conns_max attempted

# connections PLUS udp_dataconns_max answered ones.

option honeycomb udp_dataconns_max 1000

# The maximum number of bytes flowing in a single direction without

# any payload coming the other way during the UDP dialog that we

# store. More data going in one direction without any real data

# going the other way is not stored, as we‟re currently not looking

# for data there.

#

# This is also the maximum string size the longest common substring

# algorithm in libstree needs to deal with, so we don‟t make this

# too high to avoid performance hits.

option honeycomb udp_max_msg_size 5000

# We stop hunting for patterns at some point into a UDP exchange.

# The following defines the number of total bytes inbound before

# we stop caring.

option honeycomb udp_max_bytes 10000

Page 101: Simon Brooks 100042660 - Dissertation - 2010-2011

To determine efficiency of automated signature creation compared with human crafted signature creation 3 May 2011

Simon Brooks Page 101 of 126

# The minimum pattern length we require before we consider

# a string match in UDP payload meaningful:

option honeycomb udp_pattern_minlen 5

# How many initiated TCP connections we maintain state for at any one time.

option honeycomb tcp_conns_max 65000

# How many established TCP connections we maintain state for at any

# one time. Once a connection is established, it is moved to a different

# hashtable. We therefore keep state for tcp_conns_max unestablished

# connections PLUS tcp_dataconns_max established ones.

option honeycomb tcp_dataconns_max 1000

# The maximum number of bytes flowing in a single direction without

# any payload coming the other way during the TCP dialog that we

# store. More data going in one direction without any real data

# going the other way is not stored, as we‟re currently not looking

# for data there.

#

# This is also the maximum string size the longest common substring

# algorithm in libstree needs to deal with, so we don‟t make this

# too high to avoid performance hits.

option honeycomb tcp_max_msg_size 5000

# We stop hunting for patterns at some point into a TCP dialogue.

# The following defines the number of total bytes inbound before

# we stop caring.

option honeycomb tcp_max_bytes 10000

# For TCP, we also buffer the incoming payloads in one single buffer

Page 102: Simon Brooks 100042660 - Dissertation - 2010-2011

To determine efficiency of automated signature creation compared with human crafted signature creation 3 May 2011

Simon Brooks Page 102 of 126

# directly. This defines the size of that buffer.

option honeycomb tcp_max_buffering_in 1000

# The minimum pattern length we require before we consider

# a string match in TCP payload meaningful:

option honeycomb tcp_pattern_minlen 5

# The number of slots in the hashtables:

option honeycomb conns_hash_slots 199

# The connection hashtables are periodically checked for dead connections

# we‟re no longer interested in (this doesn‟t automatically mean terminated

# connections, as we need to keep connections around in order to be able to

# have something to compare new ones against!). This setting defines

# the interval in seconds between cleanups.

option honeycomb conns_hash_cleanup_interval 10

# How many generated signatures we keep around before we

# start to forget some.

option honeycomb sighist_max_size 200

# Detected signatures are kept in a history structure and reported

# periodically. This settings defines how long to wait between those

# reports. During the waiting period, existing signatures can be

# improved upon through new traffic flows.

option honeycomb sighist_interval 10 10

Page 103: Simon Brooks 100042660 - Dissertation - 2010-2011

To determine efficiency of automated signature creation compared with human crafted signature creation 3 May 2011

Simon Brooks Page 103 of 126

9.9 CentOS Screenshots

Figure 63 - CentOS Install Screen

Figure 64 - CentOS Loading Screen

Page 104: Simon Brooks 100042660 - Dissertation - 2010-2011

To determine efficiency of automated signature creation compared with human crafted signature creation 3 May 2011

Simon Brooks Page 104 of 126

Figure 65 - Honeycomb Configuration Overview Page

Figure 66 - Honeycomb Error

Page 105: Simon Brooks 100042660 - Dissertation - 2010-2011

To determine efficiency of automated signature creation compared with human crafted signature creation 3 May 2011

Simon Brooks Page 105 of 126

Figure 67 - SNORT IDS on Network Security Toolkit1

Page 106: Simon Brooks 100042660 - Dissertation - 2010-2011

To determine efficiency of automated signature creation compared with human crafted signature creation 3 May 2011

Simon Brooks Page 106 of 126

9.10 4Ethical Approval Form

Appendix E – ARP Project Request form

Request for Ethical Approval for Individual Study / Programme of Research

by University Students

Students conducting PG Independent Scholarship (PG IS), UG Applied Research

Project (UG ARP), UG Maths Projects (UG MP) or Learning-through-Work (LTW)

projects must complete this form and submit to their project supervisors for approval.

After initial approval, project supervisors need to submit these forms to PG IS, UG

ARP, UG MP or LTW coordinator who would then submit these to the Chair of the

Computing Ethics Committee (CREC) for further consideration.

Students conducting PG Research projects (eg MPhil, PhD etc) must also complete this

form and submit to their research supervisors for approval who would then submit these

to the Chair of the CREC for further consideration.

Feedback on your application will be via the Project/Research Supervisor.

Your Name: Simon Brooks

2a. Programme name and code: Computer Networks G406

2b. Your student ID: 100042660

2a. Programme name and code: Computer Networks G406

3. Contact Info Email: [email protected]

Tel No. 07999616341

Address: Flat 13C Flat 5 Princess Alice Court Bridge Street Derby DE13LD

4. Module name and code: Applied Research Project 6CC039

5. Name of project/research supervisor David Day

6. Title or topic area of proposed study

The Efficiency of automated Signature Writing Techniques. Automated Signature Generation

Systems.

Honeypots and Intrusion Detection Systems.

7. What is the aim and objectives of your study?

Aim – To determine efficiency of automated signature creation compared with human crafted signature creation Objectives 1.1 – Evaluate different methods of automated signature creation Objectives 1.2 – Evaluate different philosophies of signature writing Objectives 1.3 – Decide which methods/systems to compare Objectives 1.4 – Design and implement test bed

School of Computing and Mathematics

Faculty of BCL

Page 107: Simon Brooks 100042660 - Dissertation - 2010-2011

To determine efficiency of automated signature creation compared with human crafted signature creation 3 May 2011

Simon Brooks Page 107 of 126

Objectives 1.5 – Analysis of results in accordance with Aim

8. Brief review of relevant literature and rationale for study (attach on a separate sheet references of approximately 6 key publications, it is not necessary to attach copies of the publications)

I will be performing research and tests into the efficiency of automated signature creation compared with human crafted signatures. My research is focused on Signature Writing techniques and the philosophies of signature writing. Using this research I will write SNORT signatures for a variety of network worms. These signatures will be tested against automated signatures and compared by False Negatives and True Positives. I am undertaking this study to gain a greater knowledge of both human and automated Signature Creation. My research will be useful for organisations that wish to deploy Intrusion Detection Systems, or for organisations that use IDS already and want to find out more about automated signature generation. This study will also be a useful overview of how Intrusion Detection Systems work. Griffin, K. Schneider,S. Hu, X. Chieueh, T. (2009) Automatic Generation of String Signatures for Malware Detection [Online]. Available at: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.153.3065&rep=rep1&type=pdf - This paper discusses exponential growth in Malware signatures and looks at using string signatures as a different solution. Kreibich, C. Crowcroft, J. (2003) Automated NIDS Signature Creation using Honeypots [Online]. Available at: http://www.foo.be/cours/dess-20072008/papers/honeycomb-poster-paper-sc2003.pdf - This paper describes Honeycomb, a system for generating automated signatures. Honeycomb is run on an unprotected cable modem for 24 hours. This paper discusses the results gathered. Newsome, J. Song, D. (2005) Dynamic Taint Analysis for Automatic Detection, Analysis, and Signature Generation of Exploits on Commodity Software [Online]. Available at: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.83.2141&rep=rep1&type=pdf - This paper looks at dynamic taint analysis for automated detection of overwrite attacks, which include most types of exploits. The authors discuss TaintCheck a mechanism that can perform dynamic taint analysis.

Al Daoud, E. Jebril, H. I. Zaqaibeh, B. (2008) Computer Virus Strategies and Detection Methods [Online]. http://www.emis.ams.org/journals/IJOPCM/files/IJOPCM(vol.1.2.3.S.08).pdf - This paper shows that to develop new reliable antivirus software some problems must be solved. The authors discuss various methods in this paper, such as a new method to detect all metamorphic virus copies, new reliable monitoring techniques to discover the new viruses or attaching a digital signature and a certificate to each piece of new software. Singh, S. Estan, C. Varghese, G. Savage, S. (2004) Automated Worm Fingerprinting [Online]. http://portal.acm.org/citation.cfm?id=1251254.1251258 - In this paper the authors propose an automated approach for quickly detecting previously unknown worms and viruses based on two key behavioural characteristics.

Page 108: Simon Brooks 100042660 - Dissertation - 2010-2011

To determine efficiency of automated signature creation compared with human crafted signature creation 3 May 2011

Simon Brooks Page 108 of 126

Chandran, R. Pakala, S. (2003) Simulating Networks with Honeyd [Online]. http://www.hynesim.org/files/652/simulatingNetworksWithHoneyd_RoshenChandran.pdf - A guide explaining how to use the honeyd network daemon. Sample configuration files are shown in this document.

9. Outline of study design and methods

My outline of study involves document based research in the Literature Review to determine the above. Using empirical and observational research in a practical environment to test hypothesis formed from the document based research. My study design requires the use of a Dell Laptop with VMware.This laptop will be running four separate virtual machines with different operating systems. VM One will have Honeyd and Honeycomb configured. Honeyd is honeypot software that can be configured to emulate hosts on a network. Honeycomb is an add-on for the Honeyd software that can generate automated signatures for BRO and SNORT Intrusion Detection Systems. Using research into various types of network worms I will configure Honeyd hosts so that they can be compromised by the worms and trigger honeycomb to generate an automated signature. These worms will be launched using an infected Virtual machine (VM Two). VM Three will be running SNORT, the Intrusion Detection System. I will test the automated signatures that have been generated in honeycomb in the SNORT Intrusion Detection System. Once I have completed this test I will replace the Honeyd virtual machine with a real virtual machine running an operating system that is vulnerable to the Worms. I will re-launch the worms targeting the vulnerable virtual machine (VM4). Finally I will use a pre-written SNORT VRT rule to detect these worms that are in the test, launch the worm sequence once more and compare the honeycomb automated rules against the VRT rules in terms of False Positives and True Positives. The final Virtual Machine will be running a full version of Microsoft XP Service Pack 2 OS. Comparisons will be made against automated honeycomb rules and snort VRT rules in terms of False Positives and True Positives.

Virtual Machines

VM 1 – Malicious Worm Host

VM 2 – CentOS with Honeyd and Honeycomb configured

VM 3 – SNORT IDS

VM 4 – Microsoft XP with Service Pack 2

10. Research Ethics

PROPOSALS INVOLVING HUMAN PARTICIPANTS (eg collecting data using questionnaires, interviews etc) MUST ADDRESS QUESTIONS 10 - 14.

Does the proposed study entail ethical considerations Yes / No (please circle as

appropriate) If ‘No’ provide a statement below to support this position and skip Questions 11-14. I will not be gathering any personal data therefore I do not require any ethical considerations for this project. My data will be gathered in-house so I do not need to obtain any information from the general public.

Page 109: Simon Brooks 100042660 - Dissertation - 2010-2011

To determine efficiency of automated signature creation compared with human crafted signature creation 3 May 2011

Simon Brooks Page 109 of 126

If ‘Yes’ move on to Question 11.

Page 110: Simon Brooks 100042660 - Dissertation - 2010-2011

To determine efficiency of automated signature creation compared with human crafted signature creation 3 May 2011

Simon Brooks Page 110 of 126

11. Ethical Considerations: Please indicate how you intend to address each of the following in your study. Points a - i relate particularly to projects involving human participants.

Guidance to completing this section of the form is provided at the end of the document.

a. Consent

b. Deception

c. Debriefing

d. Withdrawal from the investigation

e. Confidentiality

f. Protection of participants

g. Observation research [complete if applicable]

h. Giving advice

i. Research undertaken in public places [complete if applicable]

j. Data protection

k. Animal Rights [complete if applicable]

l. Environmental protection [complete if applicable]

Page 111: Simon Brooks 100042660 - Dissertation - 2010-2011

To determine efficiency of automated signature creation compared with human crafted signature creation 3 May 2011

Simon Brooks Page 111 of 126

12. Sample: Please provide a detailed description of the study sample, covering selection, number, age, and if appropriate, inclusion and exclusion criteria.

N/A

13. Are payments or rewards/incentives going to be made to the participants? If so, please give details below.

N/A

14. What study materials will you use? (Please give full details here of validated scales, bespoke questionnaires, interview schedules, focus group schedules etc and attach all materials to the application)

N/A

15. What resources will you require? (e.g. psychometric scales, equipment, such as video camera, specialised software, access to specialist facilities, such as specialist laboratories).

Software Requirements VMware Workstation 7.1.1 CentOS Virtual Machine configured with Honeyd and Honeycomb SNORT Intrusion Detection System Clean install of Windows XP Service Pack 2 Hardware Requirements The hardware I require includes: Laptop 1 - Dell Vostro

Intel® Core™ i5 m450 @ CPU 2.40GHz Dual Core 4.00GB RAM

All of this hardware and software is available to me and I have the resources I need to carry out this project.

Page 112: Simon Brooks 100042660 - Dissertation - 2010-2011

To determine efficiency of automated signature creation compared with human crafted signature creation 3 May 2011

Simon Brooks Page 112 of 126

16. Have/Do you intend to request ethical approval from any other body/organisation?

Yes / No (please circle as appropriate)

If ‘Yes’ – please give details below.

17. Declaration: The information supplied is, to the best of my knowledge and belief, accurate. I clearly understand my obligations and the rights of the participants. I agree to act at all times in accordance with University of Derby Code of Practice on Research Ethics http://www.derby.ac.uk/research/ethics/policy-document Date of submission……………………………….. Signature of applicant…………………………………………… Signature of project supervisor …………………………………………… Signature of PG IS, UG ARP or UG MP Coordinator (and comments, if any) ………………………

For CREC Committee Use Reference Number (Subject area initials/year/ID number)………………….

Date received……………… Date approved ……………. Signed……………………… Comments

Page 113: Simon Brooks 100042660 - Dissertation - 2010-2011

To determine efficiency of automated signature creation compared with human crafted signature creation 03 May 2011

iii

PLEASE ALSO SUBMIT THE FOLLOWING DOCUMENTATION WHERE APPROPRIATE (please tick to indicate the material that has been included or provide information as to why it is not available):

Questionnaires/Interview schedules

Covering letters/Information sheets

Briefing and debriefing material

Consent forms for participants

Advice on completing the ethical considerations aspects of a programme of research Consent

Informed consent must be obtained for all participants before they take part in your project. The form

should clearly state what they will be doing, drawing attention to anything they could conceivably

object to subsequently. It should be in language that the person signing it will understand. It should

also state that they can withdraw from the study at any time and the measures you are taking to

ensure the confidentiality of data. If children are recruited from schools you will require the

permission, depending on the school, of the head teacher, and of parents. Children over 14 years

should also sign an individual consent form themselves. If conducting research on children you will

normally also require Criminal Records Bureau clearance. You will need to check with the school if

they require you to obtain one of these. It is usually necessary if working alone with children,

however, some schools may request you have CRB clearance for any type of research you want to

conduct within the school. Research to be carried out in any institution (prison, hospital, etc.) will

require permission from the appropriate authority.

Covert or Deceptive Research

Research involving any form of deception can be particularly problematical, and you should provide a

full explanation of why a covert or deceptive approach is necessary, why there are no acceptable

alternative approaches not involving deception, and the scientific justification for deception.

Debriefing

How will participants be debriefed (written or oral)? If they will not be debriefed, give reasons. Please

attach the written debrief or transcript for the oral debrief. This can be particularly important if covert

or deceptive research methods are used.

Withdrawal from investigation

Participants should be told explicitly that they are free to leave the study at any time without jeopardy.

It is important that you clarify exactly how and when this will be explained to participants. Participants

also have the right to withdraw their data in retrospect, after you have received it. You will need to

clarify how they will do this and at what point they will not be able to withdraw (i.e. after the data has

been analysed and disseminated).

Protection of participants

Are the participants at risk of physical, psychological or emotional harm greater than encountered

ordinary life? If yes, describe the nature of the risk and steps taken to minimise it.

Observational research

Page 114: Simon Brooks 100042660 - Dissertation - 2010-2011

To determine efficiency of automated signature creation compared with human crafted signature creation 3 May 2011

Simon Brooks Page 114 of 126

If observational research is to be conducted without prior consent, please describe the situations in

which observations will take place and say how local cultural values and privacy of individuals and/or

institutions will be taken into account.

Giving advice

Students should not put themselves in a position of authority from which to provide advice and should

in all cases refer participants to suitably qualified and appropriate professionals.

Research in public places

You should pay particular attention to the implications of research undertaken in public places. The

impact on the social environment will be a key issue. You must observe the laws of obscenity and

public decency. You should also have due regard to religious and cultural sensitivities.

Confidentiality/Data Protection

You must comply with the Data Protection Act and the University's Good Scientific Practice

http://www.derby.ac.uk/research/policy-and-strategy This means:

It is very important that the Participant Information Sheet includes information on what the research is for, who will conduct the research, how the personal information will be used, who will have access to the information and how long the information will be kept for. This is known as a 'fair processing statement.'

You must not do anything with the personal information you collect over and above that for which you have consent.

You can only make audio or visual recordings of participants with their consent (this should be stated on the Participant Information sheet)

Identifiable personal information should only be conveyed to others within the framework of the act and with the participant's permission.

You must store data securely. Consent forms and data should be stored separately and securely.

You should only collect data that is relevant to the study being undertaken.

Data may be kept indefinitely providing its sole use is for research purposes and meets the following conditions:

The data is not being used to take decisions in respect of any living individual.

The data is not being used in any which is, or is likely to, cause damage and/or distress to any living individual.

You should always protect a participant's anonymity unless they have given their permission to be identified (if they do so, this should be stated on the Informed Consent Form).

All data should be returned to participants or destroyed if consent is not given after the fact, or if a participant withdraws.

Animal rights.

Research which might involve the study of animals at the University is not likely to involve intrusive or invasive procedures. However, you should avoid animal suffering of any kind and should ensure that proper animal husbandry practices are followed. You should show respect for animals as fellow sentient beings.

Environmental protection

The negative impacts of your research on the natural environment and animal welfare, must be minimised and must be compliant to current legislation. Your research should appropriately weigh longer-term research benefit against short-term environmental harm needed to achieve research goals.

Page 115: Simon Brooks 100042660 - Dissertation - 2010-2011

To determine efficiency of automated signature creation compared with human crafted signature creation 3 May 2011

Simon Brooks Page 115 of 126

9.11 Dissertation Plan – February 2011

To determine efficiency of automated signature creation compared with

human crafted signature creation.

Chapter 1 - Introduction – 1000 Words – Write at the end April 3rd week

Chapter 2 - Literature Review – 3,500 Words – Aim to finish December 2010

Review the case or network intrusion detection systems vs. traditional defences e.g. Antivirus,

Firewalls. They are not so good at detecting Application Layer attacks, payloads. Headers trailers in

the lower layers of the OSI model. Put forward the case for NIDS.

Signature writing techniques for Intrusion Detection Systems and Honeypots – Know the pattern vs.

Know the vulnerability Ryan Trost

An Overview of Honeypot technology

How worms work – general information

Automated Signature generation Technology/brief collection of products – focused on WORMS

Chapter 3 - Conceptual Model of Problem Domain - 5,000 Words – Aim to finish February 2011

An overview of Honeyd and the plug-in honeycomb

Why honeycomb- Alternative Signature Generation products – February 2011

Why honeyd – honeypots justify selection over the alternatives –outcomes should create a provable

hypothesis – be wary of over generalisation – February/March 2011

The experiment design – 1st draft done - Complete by March 2011

How you are going to model it. - Complete by March 2011

Other types of automated signature based programs – Compare snort VRT Rules vs. Honeycomb rules

Worms – what worms I have selected and why – bit more detail specifically how they work

Chapter 4 - Research Methods/Experimentation – 2,000 Words – Aim to finish March 2011

Justify why I have used experimentation and imperial research

Chapter 5 – Analysis –500 words – Aim to finish april 2nd

week

How effective the rules

What impacts findings will have

Potential outcomes

Chapter 6 - Conclusions - 1,000 Words April 2nd

week

Chapter 7 – Critical Evaluation – April 3rd

Week

Page 116: Simon Brooks 100042660 - Dissertation - 2010-2011

To determine efficiency of automated signature creation compared with human crafted signature creation 3 May 2011

Simon Brooks Page 116 of 126

9.12 Preliminary Structure – November 2010

1. Table of Contents

2. Table of Figures

3. Acknowledgements

4. Introduction

5. Chapter 1 – Literary Review

a. The Threats Computer Systems Face in the Digital age

1. The Growth of the Public Internet

2. What Threats do Computers face?

a. What defines a hacker

b. Attacks on networks

b. Network Intrusion Detection Systems VS Traditional Systems

2. What is an Intrusion Detection System?

3. What is the overall goal of an Intrusion Detection System?

a. Notification Alarms

4. IDS detection methods – Anomaly and Signature Based Detection

5. Advantages and Disadvantages of Signature-Based Detection

6. Advantages and Disadvantages of Anomaly-Based Detection

7. What types of Intrusion Detection Systems exist today?

i. Network Based Intrusion Detection (NIDS)

ii. Distributed Intrusion Detection (dIDS)

iii. Host Based Intrusion Detection (HIDS)

iv. Hybrid Intrusion Detection Systems

v. Intrusion Prevention System

8. Traditional Systems

a. 2.2 Firewall

i. 1st Generation – Packet Filters

ii. 2nd

Generation – Application Layer Gateway (Proxy)

iii. 3rd

Generation – Stateful Inspection

iv. 4th Generation – Hybrid Systems – ALG + Packet Filter

v. Argument

b. 2.3 Anti-Virus

c. 2.4 Malware Detection

d. 2.5 Conclusion -NIDS

9. Signature Writing Techniques for Intrusion Detection Systems++

i. Know the Pattern

ii. Know the Vulnerability

b. Section C: What is a Honeypot?

i. Honeypot Technology

1. High-Interaction Honeypots

a. VM-Ware

b. User-mode Linux

c. Argos

2. Low-Interaction Honeypots

a. Deception Toolkit

b. LaBrea

c. Honeyd

Page 117: Simon Brooks 100042660 - Dissertation - 2010-2011

To determine efficiency of automated signature creation compared with human crafted signature creation 3 May 2011

Simon Brooks Page 117 of 126

c. Section E: Automated signature generation

products/technologies/types

i. Dynamic Taint Analysis

1. TaintCheck

2. Autograph

3. Honeycomb

4. SweetBait

5.

ii. Low and High Interaction honeypot

iii. Or

iv. Worms

1. Autograph

2. EarlyBird

v. Polymorphic

1. PolyGraph

2. HoneyCyber

10. Chapter 2 – Conceptual Model of Problem Domain

a. Section A: The Experiment Design

i. Honeypot Technology

1. Choices

2. Chosen

3. Reason

ii. Virtualization

1. Choices

a. VMware

b. Oracle Virtual Box

2. Chosen

3. Reason

iii. OS

1. Choices

a. Windows NT

b. Linux Flavours

2. Chosen

a. Windows XP

b. CentOS

3. Reason

iv. Automated Signature Generation

1. Choices

2. Chosen

3. Reason

4. Snort VRT Rules VS Honeycomb Rules

v. Worms

1. Choices

2. Chosen

3. Reason

b. Section B: How the experiment will be modelled

i. Virtualization

1. Build VM‟s

Page 118: Simon Brooks 100042660 - Dissertation - 2010-2011

To determine efficiency of automated signature creation compared with human crafted signature creation 3 May 2011

Simon Brooks Page 118 of 126

a. Honeypot – CentOS

b. Infected Host

c. Non-Honeypot Host

d. Intrusion Detection System

2. Configure Honeypot

a. Honeyd Scripts

3. Configure Honeycomb

a. Honeycomb Logs

b. Mapping/re-compiling Honeyd to

honeycomb

4. Launching the Worms at Honeypot

5. Generating Automated Signatures

6. Placing These Automated Signatures into

SNORT

7. Testing Automated Rules

8. Re-Launch the worms, targeting the non-

honeypot host

9. Create a pre-written snort VRT rules to detect

the launched worms

10. Launch the worms one more time

11. Compare SNORT alerts with honeycomb rules

in terms of False Positives and True Positives

11. Chapter 3 – Research Methods

a. Section A: Justify why I have used experimentation and imperial

research

12. Chapter 4 – Analysis

a. Section A How effective are the rules

b. Section B: What Impacts do the findings have?

c. Section C:Potential Outcomes

13. Chapter 5 – Conclusions

a. x

b. x

c. x

14. Bibliography

15. Works Cited

16. Appendix

Page 119: Simon Brooks 100042660 - Dissertation - 2010-2011

To determine efficiency of automated signature creation compared with human crafted signature creation 3 May 2011

Simon Brooks Page 119 of 126

9.13 Progress Sheets

University of Derby - School of Computing & Mathematics

UG Computing Programmes

Applied Research Project – Project Progress Sheet

Student Name: Simon Brooks Date of Meeting: 1.10.2010

Number of Meeting 1

Work Done Since Last Meeting:

Started working on the Proposal document. Installed VMware workstation. Installed a clean install of

Ubuntu 10.04. Obtained the packages for Honeyd and subsequent pre-requisites. Downloaded and compiled

the honeycomb add-on. In addition downloaded the Network Security Toolkit Virtual Appliance that

contains a copy of snort. Have not actually tested the snort application or looked in much detail.

I also have a Windows XP Service Pack 2 Virtual Machine that should be susceptible to some SP2 attacks

using the Metasploit framework.

Looked briefly at some documents other people have done on study of the Honeyd software when looking

up troubleshooting.

Problems encountered and suggested solutions:

Following the readme and compiling Honeyd with the debug option, the notification to say that honeycomb

was running did not appear. Unsure if this has actually worked. I think first I need to plan out the actual

design of the honeypot and the Honeyd scripting first before including the Honeyd with the add-on.

Work proposed during the next period:

Collect more background reading on the honeypot subject, particularly the documents I found about

honeycomb research and Honeyd.

Start to design the honeypot, find out what I will need to test.

Project supervisor‟s comments (incl. comments on project progress, student‟s commitment/attitude, time

management etc.):

Identified Aim and Objectives today.

Project supervisor‟s Signature: Date of next meeting: 19.10.2010

Page 120: Simon Brooks 100042660 - Dissertation - 2010-2011

To determine efficiency of automated signature creation compared with human crafted signature creation 3 May 2011

Simon Brooks Page 120 of 126

University of Derby - School of Computing & Mathematics

UG Computing Programmes

Applied Research Project – Project Progress Sheet

Student Name: Simon Brooks Date of Meeting: 19.10.2010

Number of Meeting 2

Work Done Since Last Meeting:

Attempted more work on compiling Honeyd and honeycomb – Successfully compiled honeycomb and ran

the test configuration files.

Some brief research into papers on the topics of Automated NIDS Signature creation, Automated Detection,

Detection methods.

Problems encountered and suggested solutions:

Problems arose during the re-compile with honeycomb. Several errors arose including missing packages.

These were fixed and downloaded however there were still problems.

To make these issues a little easier I have saved VMWARE states of the Honeyd box so that I can go back

to the beginning and start fresh without having to re-install the OS.

Looking across the internet for some more information on how to compile honeycomb.

Work proposed during the next period:

Work on the literary review aiming for completion late November/ early December

In addition, work towards getting honeycomb to work within the same time constraints.

Project supervisor‟s comments (incl. comments on project progress, student‟s commitment/attitude, time

management etc.):

Discussed the project plan and test bed

Discussed Literary review and time constraints

Project supervisor‟s Signature: Date of next meeting:

Page 121: Simon Brooks 100042660 - Dissertation - 2010-2011

To determine efficiency of automated signature creation compared with human crafted signature creation 3 May 2011

Simon Brooks Page 121 of 126

University of Derby - School of Computing & Mathematics

UG Computing Programmes

Applied Research Project – Project Progress Sheet

Student Name: Simon Brooks Date of Meeting: 17.11.2010

Number of Meeting 3

Work Done Since Last Meeting:

Literary Review

o Ryan Trost Book from Library

o Met with Chris Martindale for help on finding Journals on Firewalls/IDS/

o Researching relevant Literature

o Writing up parts of the Literature Review

ARP Form Hand In

Practical Work – Moved from Ubuntu to CentOS, have installed Honeyd successfully and Honeycomb add-

on following an online tutorial.

Problems encountered and suggested solutions:

Ubuntu 10.04 was causing too many problems when re-compiling Honeyd with honeycomb. I decided to

start again with a new Linux operating system, CentOS 5.0 because the tutorial I was following used the

same operating system.

Work proposed during the next period:

Interim Report – Plan of what to do – All Lit Review Information that I have done so far

Changes to the Dissertation plan

Project supervisor‟s comments (incl. comments on project progress, student‟s commitment/attitude, time

management etc.):

Skype Meeting – Send Dissertation Plan through via E-Mail

Send Literature Review Work so far

Project supervisor‟s Signature: Date of next meeting: 8.2.2011

Page 122: Simon Brooks 100042660 - Dissertation - 2010-2011

To determine efficiency of automated signature creation compared with human crafted signature creation 3 May 2011

Simon Brooks Page 122 of 126

University of Derby - School of Computing & Mathematics

UG Computing Programmes

Applied Research Project – Project Progress Sheet

Student Name: Simon Brooks Date of Meeting: 8.2.2011

Number of Meeting 4

Work Done Since Last Meeting:

Interim Report – Emailed in with copy of Sun, Nov 28, 2010 at 2:06 AM

Literary Review – Signature Writing techniques, Honeyd and Honeycomb Overview and Other types of

automated signature based programs. Research into these areas and writing up about these areas for the

Literary Review.

Research into Honeycomb, how it creates signatures, some research into the TCP/IP model, differences

between OSI 7 layers and how this relates to Wom propagation.

Problems encountered and suggested solutions:

Require some guidance with the literary review to ensure that I am on the correct lines.

Work proposed during the next period:

Complete the Literature Review/progress further

Start Chapter 3, work on the experiment design.

Project supervisor‟s comments (incl. comments on project progress, student‟s commitment/attitude, time

management etc.):

Discussed Literature review topics, with reference to Intrusion Detection Systems Vs Traditional Systems

Project supervisor‟s Signature: Date of next meeting: 24/01/11

Page 123: Simon Brooks 100042660 - Dissertation - 2010-2011

To determine efficiency of automated signature creation compared with human crafted signature creation 3 May 2011

Simon Brooks Page 123 of 126

University of Derby - School of Computing & Mathematics

UG Computing Programmes

Applied Research Project – Project Progress Sheet

Student Name: Simon Brooks Date of Meeting: 24.1.2011

Number of Meeting 5

Work Done Since Last Meeting:

Worked solidly on the structure of the Dissertation report. Made a thorough plan as to what chapters and

sections are to be in chapters 1-6

Researched into finding some binary Worms – Found source code for top 60 Worms – Problems

encountered with the compiling stage

Found some Malware binaries – Tested them on Server 2003 box – Wiped the HDD

Focused time on getting Honeyd to integrate with the other virtual machines – Currently honeypots can be

pinged via the CentOS host – not from other Virtual Hosts

Problems encountered and suggested solutions:

Honeyd – Cannot get Honeyd to integrate with the other virtual machines

Looked at the structure of the honeypot – Configuration and loopback address

Suggestions: Changed the loopback address of CentOS to 192.168.0.4/24 – Could not ping the XP virtual

machine – Checked VMnet details – same virtual subnet

This stopped the web server from running and had to add the –disable-webserver command when running

Honeyd

Also, when pinging 192.168.0.200/24 (XP) virtual host, could ping out but not back from the XP machine.

Weirdly however, the whole entire 192.168.0.0/24 network could be contacted – does this have something

to do with the Loopback address being changed?

Work proposed during the next period:

Complete all areas of the Literature Review.

Re-install Honeyd and honeycomb using ARPD instead of a static route and integrate the virtual machines

into the 10.0.0.0/8 network

Project supervisor‟s comments (incl. comments on project progress, student‟s commitment/attitude, time

management etc.):

Recommended to change the local address of the Loopback adapter to try and connect Honeyd to the

network

Project supervisor‟s Signature: Date of next meeting:14/2/2011

Page 124: Simon Brooks 100042660 - Dissertation - 2010-2011

To determine efficiency of automated signature creation compared with human crafted signature creation 3 May 2011

Simon Brooks Page 124 of 126

University of Derby - School of Computing & Mathematics

UG Computing Programmes

Applied Research Project – Project Progress Sheet

Student Name: Simon Brooks Date of Meeting: 14.2.2011

Number of Meeting 6

Work Done Since Last Meeting:

Tried changing the loopback address to the 192.168.0.0/24 subnet, this caused weird results and did not

enable an external machine to contact a Honeyd honeypot.

Tried to install ARPD on CentOS but ran into many difficult compiling issues

Went back to Ubuntu Virtual Machine, re-configured the Honeyd configuration file using a modified

version of http://www.honeyd.org/config/honeyd.conf.networks

Using FARPD was able to contact the virtual honeypots from an external virtual machine

Tried re-installing Honeyd and honeycomb on Ubuntu but failed miserably.

On CentOS with Honeyd and honeycomb installed the working Honeyd configuration file, ensured that all

of the Virtual Machines were located on the correct subnet 10.0.0.0/24 as of the Honeyd server (CentOS).

Successful connections to Honeyd honeypots took place.

Finally, ran an Nmap Scan to test honeycomb, successfully generated some signatures

in/tmp/honeycomb.log

Problems encountered and suggested solutions:

Discussed honeycomb, how does create the signatures? What are they based on? Answer not too sure as of

yet but uses traffic normalization and some tasteful inspection.

Discussed the following steps now that the test bed has been built and is stable

Work proposed during the next period:

Use Metasploit to generate some attacks against a virtual machine Win2k, XP (Honeyd)

Use the Generated honeycomb signatures in Snort

Look at the statistics in Pgraph – Speed – True Positives – True Negatives

Compare these to the VST versions of these attacks

Continue Writing up the Literary Review and Conceptual Model of the Problem Domain

Project supervisor‟s comments (incl. comments on project progress, student‟s commitment/attitude, time

management etc.):

Install Metasploit and record the statistics in Pgraph

Project supervisor‟s Signature: Date of next meeting: 4.4.2011

Page 125: Simon Brooks 100042660 - Dissertation - 2010-2011

To determine efficiency of automated signature creation compared with human crafted signature creation 3 May 2011

Simon Brooks Page 125 of 126

University of Derby - School of Computing & Mathematics

UG Computing Programmes

Applied Research Project – Project Progress Sheet

Student Name: Simon Brooks Date of Meeting: 4.4.2011

Number of Meeting 7

Work Done Since Last Meeting:

Fired several Metasploit attacks against the honeyd/honeycomb Virtual machine, expecting some

detailed/varied signatures. Instead got some quite poor results

Written up Chapter 2 – Research Methods – Why have I chosen experimentation to be completed

Added another section to literature review – Background reading on the Internet and threats that are faced in

today‟s digital age

Partly Documented experiments due to unexpected results

Problems encountered and suggested solutions:

Following the experiment with Honeyd and Honeycomb, no acceptable results were gained from firing

attacks from Metasploit against the Honeyd/Honeycomb VM machine

Continue with the same experiment against SNORT – looking for pre-processor alerts

Report back for another meeting with documented results

Work proposed during the next period:

SNORT experiment – Fire same Metasploit attacks that were sent to Honeycomb to snort and record the

results and document

Provide Documentation of previous experiments

Continue with written work – Research methods, Conceptual Problem of the domain etc.

Project supervisor‟s comments (incl. comments on project progress, student‟s commitment/attitude, time

management etc.):

Continue with the experiment project and try to find out why the Honeycomb results are unsuccessful

Project supervisor‟s Signature: Date of next meeting: 26.4.2011

Page 126: Simon Brooks 100042660 - Dissertation - 2010-2011

To determine efficiency of automated signature creation compared with human crafted signature creation 3 May 2011

Simon Brooks Page 126 of 126

University of Derby - School of Computing & Mathematics

UG Computing Programmes

Applied Research Project – Project Progress Sheet

Student Name: Simon Brooks Date of Meeting: 26.4.2011

Number of Meeting 8

Work Done Since Last Meeting:

Completed Written work for Chapters 1-4

Designed appropriate Experiment network diagrams

Started writing up Analysis and thinking about conclusions

Gathering all Appendices together

Problems encountered and suggested solutions:

Unsure of how to organise the analysis results due to honeycomb signatures being rather poor

Work proposed during the next period:

Finish up the Conclusions, Introduction and critical analysis of performance

Read through and make any corrections

Project supervisor‟s comments (incl. comments on project progress, student‟s commitment/attitude, time

management etc.):

Project supervisor‟s Signature: Date of next meeting: N/A