Survey of Intrusion Detection and Attack Description Languages

Based on

Monitoring Program Behavior

Erbil YILMAZArea Survey Exam

Summer 2003

Intrusion Detection Approaches Anomaly Detection

Detect deviations from normal behavior Misuse Detection

Looks for pattern or predefined sequences of events defining a known attack

Specification Based Describes the intended behavior of

programs

Monitoring Program Behavior Regardless of the nature of an attack, damage

can ultimately be affected only via system calls made by processes running on the target system

By monitoring and restricting system calls, possible attacks can be prevented

Increase efficiency of IDS System call interception is usually achieved by

wrappers

Early Systems Using Program Behavior Patterns

JANUS Restricts access of untrusted helper

applications to the operating system “Sense of Self” for Unix Processes

Normal behavior is defined by short-range correlations in a process’s system calls

DPEM Describes a security-critical application’s

intended behavior through specifications

JANUS

Netscape and ghostview example Uses “/proc virtual file system” Modular approach: framework + dynamic

policy modules First configuration file is read, which contains

lines like: path allow read, write /tmp/*

JANUS Modules Basic module: Supplies defaults for system

calls that are easiest to analyze and takes no configuration parameters.

Putenv module: Allows one to specify environment variable settings for the traced application via its parameters. The ones that are not mentioned are unset.

Tcpconnect module: Allows restricting TCP connections by host and port.

Path module: Allows or denies path access according to patterns.

Sense of Self for Unix Processes

Based on an analogy between the human immune system and intrusion detection

Normal behavior is defined by short-range correlations in a process’s system calls

Unix’s strace is used to intercept system calls A file of short sequences of system calls is

created

Short Sequence of System Calls Creation Procedure

Application runs and all its system calls are recorded

These sequences then decomposed into short sequences using sliding window

Unique ones are stored in a database and called N-grams

Comparatively small collection of short sequences derived

DPEM

Describes a security-critical application’s intended behavior through specifications

Monitors its behavior for violations of these specifications also called trace policies

Audit trails are used to check for violations of these trace policies

Security Relevant Characteristics of Programs

Access: The set of system objects being accessed through the program’s execution.

Sequencing: The order of operations performed by a program.

Synchronization: Refers to improper synchronizations of programs.

Race Conditions: A special case of the synchronization problem.

Example 1. SPEC <(?, binmail, U, H)> 2. ENV int CREATTMP = 0; 3. ENV int PID = getpid(); 4. SE: <binmail> || <other> 5. <binmail> -> <init> <mktemp> <rest>. 6. <init> -> <not_mktemp> <init> | Nil. 7. <rest> -> any_op <rest> | Nil. 8. <mktemp> -> open_tmpfile-PID { CREATTMP = 1; }. 9. <not_mktemp> -> not_open_tmpfile-PID 10. <other> -> <vop, CREATTMP> <other> | Nil. 11. <vop, 0> -> not_chgtmp. 12. <vop, 1> -> any_op. 13. END;

Attack Description and Specification Languages STATL

Extensible, domain-independent, state-transition-based attack description language

ASL Specifies normal and abnormal behaviors of

a process as logical assertions about an application program’s sequence of system calls and their argument values

BMSL Monitors incoming network packets Also monitors system calls requested by

executing processes

STATL

Transition Types and Example Non-consuming transition Consuming transition Unwinding transition

ASL

Supports automatic defensive actions Normal and abnormal behaviors of a process as

logical assertions Local correctness assertions:

execve(f) | f != “usr/ucb/finger” -> exit(-1) Non-interference assertions:

nonatomic(f) in (access(f, mode) .. open(f)->exit(-1))

ASL

BMSL Efficient real-time intrusion detection Supports temporally related event sequences

Some Related Software Engineering Concepts

Threat Model A way to introduce security into design Helps list vulnerabilities of a software system Helps choose appropriate solutions

Aspect-Oriented Programming Isolates different aspects of programming Helps ease of comprehension and maintenance

Threat Modeling Process

1. Brainstorm the known threats to the system.2. Rank the threats by decreasing risk. 3. Choose how to respond to the threats.4. Choose techniques to mitigate the threats.

5. Choose the appropriate technologies from the identified techniques.

STRIDE Threat Model Spoofing identity, Tampering with data, Repudiation,

Information disclosure, Denial of service, Elevation of privileges

Aspect Oriented Programming Some aspects of a program’s code

Data Storage Security Performance

Components Connectors Tangling of the code (aspect weavers)

Conclusions Monitoring system calls is effective Which system call sequence is normal? Program behavior spec’s and monitoring system are

highly coupled Integrate software engineering practices to get better

specifications

Specifications should be a part of dev. effort Automated generation of monitoring system based on

spec’s