1

IC3 - Network Security

An Introduction to Intrusion Detection and Vulnerability Assessment

RHUL, 8-Dec-2003

Andreas Fuchsberger & Robert Christian, F.A.C.T.S. Group

2

Agenda

• Basics & Definitions

• Why Intrusion Detection and Vulnerability Assessment– Attack Development– Vulnerability Development– Hacker Strategy– Anatomy of a Hack

• VA– Software– Services ( Audits)– Web-Based Services

• IDS– Host based IDS– Network Based IDS

• Demo of VA and IDS• Current technological Approaches

– “Honey Pots”– Appliances

• Summary– Critical Issues

3

Basic and Definitions

• Perimeter security devices (e.g. firewalls) and computer security mechanisms (e.g. application and OS security) can only prevent attacks by outsiders.

• They may fail to do so: a firewall may be misconfigured, a password may be sniffed off the network, a new attack type may emerge.

• They do not detect when an attack is underway or has taken place.

• And they do not react to attacks.

4

Basics and Definitions

• Example:– Imagine continuous inspection of a Unix system by hand (similar

examples for NT, W2K):– The following checklist is from CERT

(http://www.cert.org/tech_tips/intruder_detection_checklist.html):

1. Examine log files for connections from unusual locations or other unusual activity. For example, look at your 'last' log, process accounting, all logs created by syslog, and other security logs.

2. Look for setuid and setgid files (especially setuid root files) everywhere on your system. Intruders often leave setuid copies of /bin/sh or /bin/time around to allow them root access at a later time.

5

Ad Hoc Intrusion Detection

• Imagine the complexity and degree of expertise needed to carry out the tasks in this checklist for every host and every sensitive network link on a network every single day.

• The ad hoc approach is not recommended!

• Automated systems are needed:– monitor multiple hosts and network links for

suspicious behaviour;– report this behaviour, possibly react to it.

• Hence: Intrusion Detection Systems (IDS).

7

Intrusion Detection Systems

• Popular second layer of technical Information Security enforcement

• Passive supervision of exiting network, analogues to intruder alarms– Creates more work for personal

• There exist 2 different approaches to the implementation of Intrusion Detection Systems (IDS)– Knowledge-based IDS

• Network based

• Host based

– Behaviour-based IDS• Statistical anomaly detection

8

Why Intrusion Detection and Vulnerability Assessment

Intruder Knowledge

High

Low

1980 1985 1990 1995 2000

Attack Sophistication

AttackSophistication

Cross site scripting

password guessing

self-replicating code

password cracking

exploiting known vulnerabilities

disabling audits

back doors

hijacking sessions

sweepers

sniffers

packet spoofing

GUIautomated probes/scans

denial of service

www attacks

“stealth” / advanced scanning techniques

burglaries

network mgmt. diagnostics

distributedattack tools

Staged

Auto Coordinated

Source: Carnegie Mellon University

9

Vulnerability Development

0

100

200

300

400

500

600

700

1997 1998 1999 2000 (Cum.)

Linux (aggr.)

Solaris

Windows NT

Gesamt

Source: SecurityFocus

Why Intrusion Detection and Vulnerability Assessment

10

Advisory Release

Widespread Awareness

Vulnerability Scannersadding detection signature

Selective AwarenessFirst

Discovery

Vulnerability & Exploit Lifecycle

Why Intrusion Detection and Vulnerability Assessment

11

Unauthorized Access to Networks

Why Intrusion Detectionand Vulnerability Assessment

12

Origin of the Attack

Why Intrusion Detection and Vulnerability Assessment

13

Source of the Attack

Why Intrusion Detection and Vulnerability Assessment

14

Which Type of Attacks ?

2001 CSI/FBI - Computer Crime and Security Survey

Why Intrusion Detection and Vulnerability Assessment

15

Types of Attacks

Why Intrusion Detection and Vulnerability Assessment

16

Reactions to attacks

Why Intrusion Detection and Vulnerability Assessment

17

Why Intrusion Detection and Vulnerability Assessment

“Classic”

Hacker Strategy

18

Primary Target Identification - Identify Hosts ( ) with external visibility

denotes internal hosts with high value data but no external view

CORP

NETWORK

PING

SWEEPInternet

Why Intrusion Detection and Vulnerability Assessment

19

Primary Target Analysis - Identify services running on visible hoststo prioritize further probing activities

PORT

SWEEP

CORP

NETWORK

DNS

WEB

NFS

Why Intrusion Detection and Vulnerability Assessment

20

Primary Target Selection - Determine vulnerability state of weakest pointand concentrate further activities against this system

FINGER

NFS CORP

NETWORK

Why Intrusion Detection and Vulnerability Assessment

21

Primary Target Exploitation - Gain privileges & control of primary target- attacker now controls a ‘trusted’ corporate system !

Rlogin Root

NFS CORP

NETWORK

Why Intrusion Detection and Vulnerability Assessment

22

Secondary Target Identification - Probing for high value information or systems which are then compromised and data stolen or trojan horses planted, etc.

NFS CORP

NETWORK

HR

R&D

$

Why Intrusion Detection and Vulnerability Assessment

23

Big Widget’s Network

Firewall

E-Mail Server

Web Server

Router

Unix

Clients & Workstations

Network

imap

NT NTUnixcrack netbus

Summary / Schematic

24

Denial of Service

• Denial of Service attacks (DoS)

In contrast to unauthorised access attacks a DoS attack does not need to contain method for communicating back to the attacker

• Distributed Denial of Service (DDoS) attacks– Trin00/Stacheldraht (Feb 2000)

• Attacks on ebay, amazon.com and etrade.com

– MS.Blaster (August 2003)

• Problem of lack of metrics to measure the impact of Denial of Service attacks – more research required

25

Vulnerability Assessment

• Vulnerability Assessment Methods– Software solutions (ISS Scanner, Stat, Nessus etc.)– Audit Services (manual Penetration tests etc)– Web based commercial (Qualys, Security Point etc)

• Keep up-to-date with security (and other) patches– Form Microsoft OS www.windowsupdate.com

• Enterprise version available

– Microsoft Baseline Security Advisor• Includes hfnetcheck.exe (from Shavlik)

– Similar for SUN, HP, IBM, CISCO etc. OS

26

Vulnerability Assessment (VA)

Vulnerability Assessment

DEMO

27

Intrusion Detection

• Intrusion Detection Systems (IDS)

• Intrusion Prevention Systems (IPS)

28

Knowledge-based IDS

• ALL commercial IDS look for attack signatures:– specific patterns of network traffic or activity in log

files that indicate suspicious behaviour.

• Called a knowledge-based or misuse detection IDS

• Example signatures might include:– a number of recent failed login attempts on a

sensitive host;– a certain pattern of bits in an IP packet, indicating a

buffer overflow attack;– certain types of TCP SYN packets, indicating a SYN

flood DoS attack.

29

Knowledge-based IDS

• Knowledge-based IDS uses information such as:– Security policy;– Known vulnerabilities of particular OS and applications;– Known attacks on systems.

• They are only as good as the information in the database of attack signatures:– new vulnerabilities not in the database are constantly being

discovered and exploited;– vendors need to keep up to date with latest attacks and issue

database updates; customers need to install these;– large number of vulnerabilities and different exploitation

methods, so effective database difficult to build;– large database makes IDS slow to use.

30

Behaviour-based IDS

• Statistical Anomaly Detection (or behaviour-based detection) is a methodology where statistical techniques are used to detect penetrations and attacks.

• Begin by establishing base-line statistical behaviour: what is normal for this system?

• Then gather new statistical data and measure the deviation from the base-line.

• If a threshold is exceeded, issue an alarm.

31

Behaviour-based IDS

• Example: monitor the number of failed login attempts at a sensitive host over a period; – if a burst of failures occurs, an attack may be under

way;

– or maybe the admin just forgot his password?

• This raises the issue of false positives (an attack is flagged when one was not taking place – a false alarm) and false negatives (an attack was missed because it fell within the bounds of normal behaviour).

• This issue does also apply to knowledge-based systems.

32

Behaviour-based IDS

• IDS does not need to know about security vulnerabilities in a particular system – the base-line defines normality;– don’t need to know the details of the construction of a buffer

overflow packet.

• Normal behaviour may overlap with forbidden behaviour.– Legitimate users may deviate from the baseline, causing false

positives (e.g. user goes on holiday, or works late in the office, or forgets password, or starts to use new application).

– If the base-line is adjusted dynamically and automatically, a patient attacker may be able to gradually shift the base-line over time so that his attack does not generate an alarm.

33

Host-based and Network-based IDS

• When an IDS looks for attack signatures in network traffic, it is called a network-based IDS (NIDS).

• When an IDS looks for attack signatures in log files of hosts, it is called a host-based IDS (HIDS).

• Naturally, the most effective Intrusion Detection System will make use of both kinds of information.

34

IDS Architecture

• Distributed set of sensors – either located on hosts or on network – to gather data.

• Centralised console to manage sensor network, analyze data, report and react.

• Ideally:– Protected communications between sensors and

console;– Protected storage for signature database/logs;– Secure console configuration;– Secured signature updates from vendor;– Otherwise, the IDS itself can be attacked and

manipulated.

38

Placement of Network-based IDS

InternetInternet

FirewallMail server

Web server

Protected Network

Sensor

Sensor

Sensor

Console

Perimeter Network

39

Host-based IDS

• Typically monitors system, event, and security logs on Windows and syslog in Unix environments.

• Checks key system files and executables via checksums at regular intervals for unexpected changes.

• Some products can use regular-expressions to refine attack signatures (e.g. passwd program executed AND .rhosts file changed).

• Some products listen to port activity and alert when specific ports are accessed – limited NIDS capability.

42

Placement of Host-based IDS

InternetInternet

FirewallMail server

Web server

Sensor

Console

Perimeter Network

Sensor

Sensor

Human Resources Network

43

IDS as a Response Tool

• Given the (near) real-time nature of IDS alerts, an IDS can be used as a response tool as well as for detection.

• NIDS and HIDS have different response capabilities – because they detect different attacks, or the same attacks but in different ways.

44

HIDS and NIDS

• There are attack types that a HIDS can detect but a NIDS cannot:– SYN flood, Land, Smurf and Teardrop attacks, BackOrifice,…

• And vice-versa:– Trojan login script, walk up to unattended keyboard attack,

encrypted traffic,…

• For more reliable detection, combine both types of IDS.

45

IDS Response Options

Network-based Host-based

Notification Alarm to console Alarm to console

E-Mail notification E-Mail notification

SNMP trap SNMP trap

View active session

Storage Log summary Log summary

Log raw network data

Active Kill connection (TCP Reset)

Terminate user login

Re-configure firewall Disable user account

Restore index.html

46

IDS Response Options

• Dangers of automated response:– Attacker tricks IDS to respond, but response aimed

at innocent target (say, by spoofing source IP address);

– Users locked out of their accounts because of false positives;

– Repeated e-mail notification becomes a denial of service attack on sysadmin’s e-mail account;

– Repeated restoration of index.html from CD reduces website availability.

47

Intrusion Detection

Intrusion Detection DEMO

48

What is Snort?

• Snort is a fast, flexible, small-footprint, open-source NIDS developed by the security community and a “benevolent dictator”

• Lead coder: Marty Roesch, now founder of Sourcefire (www.sourcefire.com)

• Initially developed in late 1998 as a sniffer with consistent output, unlike protocol-dependent output of TCPDump

• Licensed under GPL, but version 2.0 may change to a different license

49

Snort Rules

• Snort rules are extremely flexible and are easy to modify, unlike many commercial NIDS

• Sample rule to detect SubSeven trojan:

alert tcp $EXTERNAL_NET 27374 -> $HOME_NET any (msg:"BACKDOOR subseven 22"; flags: A+; content: "|0d0a5b52504c5d3030320d0a|"; reference:arachnids,485; reference:url,www.hackfix.org/subseven/; sid:103; classtype:misc-activity; rev:4;)

• Elements before parentheses comprise ‘rule header’

• Elements in parentheses are ‘rule options’

50

Third-Party Enhancements

• Analysis Console for Intrusion Databases (ACID)– http://acidlab.sourceforge.net/– PHP-based analysis engine to search and process a

database of security events generated by various IDSes, firewalls, and network monitoring tools

– Query-builder and search interface, packet viewer (decoder), alert management, chart and statistics generation

– Description and screenshots taken from ACID web

53

Third-Party Enhancements

• Demarc– www.demarc.com – NIDS management console, integrating Snort with

the convenience and power of a centralized interface for all network sensors

– Monitor all servers / hosts to make sure network services such as a mail or web servers remain accessible at all times

– Monitor system logs for anomalous log entries that may indicate intruders or system malfunctions

– Description and screenshots taken from demarc web

57

Intrusion Prevention System - IPS

• Relatively new (marketing) term

• Essentially a combination of access control (firewall/router) and intrusion detection systems– Often shared technologies between stateful

inspection and signature recognition (“looking deep into the packet”)

– Inline network IDS allows for instant access control policy modification

• Recent Gartner study claims by 2005 only integrated firewalls with IDS (i.e. IPS) will survive

• Most success to-date with “flood” attacks

58

Honeypots

• Technology used to track, learn and gather evidence of hacker activities

• Definition– “… a resource whose value is being attacked or compromised”

Laurence Spitzner, “The value of honeypots”, SecurityFocus, October 2001

• Strategically placed systems designed to mimic production systems, but not reveal “real” data

• Modes of operation– Baiting– Waiting– Collating– Disseminating

59

Honeypot types of implementation

• Level of Involvement– Low Involvement: Port Listeners– Mid Involvement: Fake Daemons– High Involvement: Real Services

• Risk increases with level of involvement

60

Honeynet

• Network of honeypots

• Supplemented by firewalls and intrusion detection systems - Honeywall

• Advantages:– “More realistic” environment– Improved possibilities to collect data

61

Honeynet

62

Sebek

• Sebek is a data capture tool designed to capture all of the attackers activities on a honeypot, without the attacker knowing it.

• 2 components. – Client that runs on the honeypots, its purpose is to

capture all of the attackers activities (keystrokes, file uploads, passwords) then covertly send the data to the server.

– Server which collects the data from the honeypots. The server normally runs on the Honeywall gateway.

• Since the Sebek client runs as a kernel module on the honeypots, it can capture all activity, including encrypted, such as SSH, IPSec

63

Honeynet using a Honeywall

64

Lecture Summary

• Threats are both internal and external.

• Prevention, detection and reaction are needed in combination.

• Intrusion detection systems are a very useful second line of defence (in addition to firewalls and other safeguards).

• IDS deployment, customisation and management is generally not straightforward.

65

Lecture Summary

• Critical Issues

• Why detect, if it cannot be prevented ?

• Technical limitations

• What defines the quality of any IDS

• Reliability (False Positives / False Negatives)

• Reliabilty

• Managebility

• Implementation

• “Is a Patch really a Patch ?”

• What other means exist ?

66

Lecture Summary

• What do you absolutely need to know:

• What is IDS / VA ?

• Different Types

• How do they function

• What are issues to be observed ?

• What are limitations to IDS / VA

• … and if you really want to be good:

• What are critical issues and how could they be overcome ?

67

IDS Further Reading

• Stallings Chapter 9, pp.292-303 (possibly too much emphasis on statistical approach; research-focussed rather than commercially focussed).

• An article: “The future of IDS” by Matthew Tanase at SecurityFocus.com:– http://online.securityfocus.com/infocus/1518

• An evaluation of IDS products by Kathleen A. Jackson:– http://www.sekure.net/ids/00416750.pdf

68

Thank You !