detection of attacks on cognitive channels
DESCRIPTION
DETECTION OF ATTACKS ON COGNITIVE CHANNELS. Annarita Giani Institute for Security Technology Studies Thayer School of Engineering Dartmouth College Hanover, NH. Berkeley, CA October 12, 2006. Outline. Motivation and Terminology Process Query System (PQS) Approach - PowerPoint PPT PresentationTRANSCRIPT
DETECTION OF ATTACKS ON COGNITIVE CHANNELS
Annarita Giani
Institute for Security Technology StudiesThayer School of Engineering
Dartmouth CollegeHanover, NH
Berkeley, CAOctober 12, 2006
Outline
1.Motivation and Terminology
2.Process Query System (PQS) Approach
3. Implementation of a PQS detecting
a.Phishing
b.Data Exfiltration
c. Covert Channel
4.Flow Attribution and Aggregation
5.Conclusion and Acknowledgments
Outline
1.Motivation and Terminology
2.Process Query System (PQS) Approach
3. Implementation of a PQS detecting
a.Phishing
b.Data Exfiltration
c. Covert Channel
4.Flow Attribution and Aggregation
5.Conclusion and Acknowledgments
4
1945
Grace Hopper. MIT - First Computer Bug
1940Von Neumann studied self reproducing mathematical automata
1951Von Neumann demonstrated how to create self- reproducing automata
1959
Penrose: Self- reproducing machines
~1960Stahl reproduces Penrose idea in machine code on an IBM 650
1970
Computer viruses on ARPANET
1982
First virus in the wild
1990s - First Commercial Antivirus
1988
Phishing Attacks
1990s
Malicious programs exploit vulnerabilities in applications and operating systems
1991 – Norton Antivirus released by Symantec
1999
Melissa virus, damage = $80 M
2001
Code Red worm, damage = $2 B
Morris worm
now
Misinformation
Exfiltration of information
OUR FOCUS
70s. System Admins directly monitor user activities
Late 70 - early 80s. System Admins review audit logs for evidence of unusual behavior.
Programs analyze audit log, usually at night.
90s. Real time IDS.
Malware and Detection
1972: J.P. Anderson, Computer Security Technology Planning Study, ESD-TR-73-51, ESD/AFSC, Bedford, MA1984: D. Denning, An Intrusion Detection Model, IEEE Transaction on Software Engineering, VolSE-13(2)1988: M. Crosby, Haystack Project, Lawrence Livermore Laboratories1989: from the Haystack Project. Stalker, a Commercial Product First HIDS1990: L. Heberlein et al, A Network Security Monitor, Symposium on Research Security and Privacy First NIDS1994: from ASIM (Air Force) Netranger First Commercial NIDS.
Web Defacements
Covert Channel
FOCUS OF MOST SECURITY WORK
Intrusion Detection System (IDS) are mainly based on signature matching and anomaly detection.
THEORETICAL WORK
Covert Channel Multi Stage Attacks
Cognitive Channels
SERVER CLIENT USER
Network Channel
Cognitive Channel
Focus of the current protection and detection approaches
A cognitive channel is a communication channel between the user and the technology being used. It conveys what the user sees, reads, hears, types, etc.
The cognitive channel is the weakest link in the whole framework. Little investigation has been done on detecting attacks on this channel.
6
Cognitive Attacks
Cognitive attacks are computer attacks over a cognitive channel. They exploit the attention of the user to manipulate her perception of reality and/or gain advantages.
Cognitive attacks are computer attacks over a cognitive channel. They exploit the attention of the user to manipulate her perception of reality and/or gain advantages.
COGNITIVE HACKING. The user’s attention is focused on the channel. The attacker exploits this fact and uses malicious information to mislead her.
COVERT CHANNELS. The user is unaware of the channel. The attacker uses a medium not perceived as a communication channel to transfer information.
PHISHING. The user's attention is attracted by the exploit. The information is used to lure the victim into using a new channel and then to create a false perception of reality with the goal of exploiting the user’s behavior.
Our definition is from an engineering point of view.
7
Cognitive Hacking
The user's attention is focused on the channel. The attacker exploits this fact and uses malicious information in the channel to mislead her.
Misleading information from a web site
Misleading information from a web site
2
1
Victim: Acts on the information from the web site
3Attacker: Obtains advantages from user actions
Attacker: Makes a fake web site
4
8
Covert ChannelsThe user's attention is unaware of the channel. The attacker uses a medium not perceived as a communication channel to transfer information.
User: does not see inter-packet delay as a communication channel and does not notice any communication.
User: does not see inter-packet delay as a communication channel and does not notice any communication.
Attacker: Codes data into inter-packet delays, taking care to avoid drawing the attention of the user.
data
1
2
9
PhishingThe user's attention is attracted by the exploit. The information is used to lure the victim into using a new channel and then to create a false perception of reality with the goal of exploiting the user’s behavior.
Visit http://www.cit1zensbank.com
First name,Last nameAccount #SSN
Bogus web site
First name,Last nameAccount NumberSSN
1
3
2
Misleading email to get user attention
Misleading email to get user attention
Send a fake email
4
10
Why current IDS cannot be applied to attacks on cognitive channels
• Sophistication of attack approaches.
• Increasing frequency and changing nature of attacks.
• Inherent limits of network-based IDS.
• Inability to identify attackers’ goals.
• Inability to identify new attack strategies.
• No guidance for response.
• Often simplistic analysis.
Outline
1.Motivation and Terminology
2.PQS Approach
3. Implementation of a PQS detecting
a.Phishing
b.Data Exfiltration
c. Covert Channel
4.Flow Attribution and Aggregation
5.Conclusion and Acknowledgments
12
Process Query System
Observable events coming from sensorsObservable events coming from sensors
ModelsModels
Tracking Algorithms
Tracking Algorithms
PQSENGINE
HypothesisHypothesis
13
Framework for Process Detection
Multiple Processes
router failure
wormscan
Events
…….
Time
An Environment
consists of
that produce
Unlabelled Sensor Reports
…….
Time
thatare
seenas
Track 1
Track 2
Track 3Hypothesis 1
Track 1
Track 2
Track 3
Hypothesis 2
that PQS resolves into
that detect complex attacks and anticipate the next steps
129.170.46.3 is at high risk129.170.46.33 is a stepping stone......
that are
usedfor
control 1
2
3
4
5
6Indictors and Warnings
Real World Process Detection (PQS)
Hypotheses
TrackScores
SampleConsole
0
0.2
0.4
0.6
0.8
1
0 100 200 300 400 500 600Time (s)
Tra
ck
Sc
ore
Service Degradation Model Process Execution Model
FO
RW
AR
D P
RO
BLE
M
INV
ER
SE
PR
OB
LEM
14
Flow and Covert Channel Sensor
Samba
SnortTripwire
SnortIP Tables
Exfiltration
Data Access
Scanning
Infection
PQS
PQS
PQS
PQS
PQS
TIER 1
TIER 1 Models
TIER 1 Observations
TIER 1 Hypothesis
TIER 2
TIER 2 Models
TIER 2 Observations
TIER 2 Hypothesis
Hierarchical PQS Architecture
Events
Events
Events
Events
More ComplexModels
RESULTS
15
Causal - next state depends only on the pastHidden – states are not directly observedObservable - observations conditioned on hidden state are independent of previous states
Example. Hidden Markov Model
N StatesM Observation symbolsState transition Probability Matrix, AObservation Symbols Distribution, BInitial State Distribution
HDESM models are general
Hidden Discrete Event System Models
Dynamical systems with discrete state spaces that are:
16
HDESM Process Detection Problem
Identifying and tracking several (casual discrete state) stochastic processes (HDESM’s) that are only partially observable.
Discrete Sources Separation: :Determine the “most likely” process-to-observation association
Hidden State Estimation: Determine the “best” hidden states sequence of a particular process that accounts for a given sequence of observations.
TWO MAIN CLASSES OF PROBLEMS
17
Discrete Source Separation Problem
3 states + transition probabilitiesn observable events: a,b,c,d,e,…Pr( state | observable event ) given/known
Observed event sequence:
….abcbbbaaaababbabcccbdddbebdbabcbabe….
Catalog ofProcesses
Which combination of which process models “best” accounts for the observations?
HDESM Example (HMM):
Events not associated with a known process are “ANOMALIES”.
18
An analogy....
What does
hbeolnjouolor
mean?
Events are: h b e o l n j o u o l o r
Models = French + English words (+ grammars!)
hbeolnjoulor = hello + bonjour
Intermediate hypotheses include tracks: ho + be
19
PQS applications
• Vehicle tracking • Worm propagation detection • Plume detection• Dynamic Social Network Analysis• Cyber Situational Awareness• Fish Tracking• Autonomic Computing• Border and Perimeter Monitoring• First Responder Sensor Network• Protein Folding
TRAFEN (TRacking and Fusion ENgine): Software implementation of a PQS
20
Example – vehicle tracking(Valentino Crespi, Diego Hernando)
T T+1 T+2
Continuous Kinematic ModelLinear Model with Gaussian noise
21
T T+1 T+2
Multiple Hypothesis Tracking
Track = process instanceHypothesis = consistent tracks Given a set of “hypotheses” for an event stream of length k-1, update the hypotheses to length k to explain the new event (based on model description).
PredictionsHypotheses
D. Reid. An algorithm for Tracking Multiple Targets – IEEE Transaction on Automatic Control,1979
Use Kalman Filter
22
Model vehicle Kinematics)()()()( kkkk twtxttx 1
)( ktx)( kt
)( ktw
State of target at timePrediction MatrixPrecision MatrixSequence of normal r.v. with Zero mean and covariance:
States:
)( ktQ
kt
)()()( kkk ttHxtz
Model MeasurementObserve State of target through a noisy measurement:
)( ktz
H
)( kt
Measure (observation)“Observable” Matrix: extracts observable information from state.Sequence of normal r.v. with Zero mean and covariance R
)()()( kkk ttHxtz
State EstimationKalman filters are used for predictions.
23
Kalman Filters
)(),( 1 kk tPtx
)(),( kk tPtx
)( ktz
)(ˆ),(ˆ kk tPtx
)( 1ktz
)(ˆ),(ˆ 11 kk tPtx
Noisy observation
Estimate state
Prediction
ErrorCovariance Prediction
ErrorCovariance Estimation
)(),( 11 kk tPtx
)( 1ktz
)(),( kk tPtxPx ˆ,ˆ
)( ktz
Prediction
Estimation
KF
KF
output
Estimation given obs before tk
Correct the estimation given the new obs
24
Kalman Equations
))(),((~|)( 1kk
kk tPtxNztx
(Normal Multivariate)
)(ˆ)()( kkk txttx 1
Tk
Tkkkk tQttPttP )()()(ˆ)()( 1
)]()([)()()(ˆ kkkkk txHtztKtxtx
)())(()()()(ˆ 1k
Tk
Tkkk tPHRHtPHHtPtPtP
1ˆ RHPK T
K is the Kalman Gain: minimizes updated error covariance matrix (mean-square error)
])ˆ)(ˆ[( Tkkkk xxxxE
Estimation
New Prediction
System’s state:
(output)
kk ztx |)(
)(ˆ ktx
)(ˆ2ktP
25
Real time Fish Tracking (Alex Jordan )
• Track the fish in the fish tank
• Very strong example of the power of PQS
– Fish swim very quickly and erratically
– Lots of missed observations
– Lots of noise
– Classical Kalman filters don’t work (non-linear
movement and acceleration)
– “Easier” than getting permission to track people (we
mistakenly thought)
26
Fish Tracking Details
• 5 Gallon tank with 2 red Platys named Bubble and Squeak
• Camera generates a stream of “centroids”:
For each frame a series of (X,Y) pairs is generated.
• Model describes the kinematics of a fish:
The model evaluates if new (X,Y) pairs could belong to the same fish, based on measured position, momentum, and predicted next position. This way, multiple “tracks” are formed. One for each object.
• Model was built in under 3 days!!!
Cybenko
Detect and differentiatepeople by behavior not
appearance
Infrared Camera
27
Autonomic Server Monitoring (Chris Roblee)
• Objective: Detect and predict deteriorating service situations• Hundreds of servers and services• Various non-intrusive sensors check for:
– CPU load– Memory footprint– Process table (forking behavior)– Disk I/O– Network I/O– Service query response times– Suspicious network activities (i.e.. Snort)
• Models describe the kinematics of failures and attacks:
The model evaluates load balancing problems, memory leaks, suspicious forking behavior (like /bin/sh), service hiccups correlated with network attacks…
Cybenko
28t0 t1 t2 t3 t4
Server Compromise Model: Integration of host CPU load sensors and IDS sensor allows
detection of attacks not possible with different sensors
Observations Response
o1
Snort NIDS sensor output
...Nov 21 20:57:16 [10.0.0.6] snort: [1:613:7]SCAN myscan [Classification: attempted-recon] [Priority: 2]:{TCP} 212.175.64.248-> 10.0.0.24...
1.o1 o2 o3
Current system record for host 10.0.0.24 (10 records): Average memory over previous 10 samples: 251.000Average CPU over previous 10 samples: 0.970| time | mem used | CPU load | num procs | flag |----------------------------------------------------------------------------------| 1101094903 | 251 | 0.970 | 64 | || 1101094911 | 252 | 0.820 | 64 | || 1101094920 | 251 | 0.920 | 64 | || 1101094928 | 251 | 0.930 | 64 | || 1101094937 | 251 | 0.870 | 65 | || 1101094946 | 251 | 0.970 | 65 | || 1101094955 | 251 | 0.820 | 65 | || 1101094964 | 253 | 1.220 | 65 | ! || 1101094973 | 255 | 1.810 | 65 | ! || 1101094982 | 258 | 2.470 | 65 | ! |
Monitored host sensor output (system level)2.PQS Tracker Output
Last Modified: Mon Nov 21 21:01:03 Model Name: server_compromise1Likelihood: 0.9182Target: 10.0.0.24Optimal Response: SIGKILL proc 6992
SIGKILL
3.
Cybenko
29
Airborne Plume Tracking (Glenn Nofsinger)
159.4
0.012.324.737.049.361.774.0
98.7111.0
182
0
10
20
30
40
50
60
70
80
90
100
110
120
130
140
150
160
170
1820 10 20 30 40 50 60 70 80 90 100 110 120 130 140 150 160 170
Forward Problem - drift and diffusion
Inverse Problem - locate sources andtypes of releases
Airborneagentsensor onDC Mall
30
Dynamic Social Network Analysis (Wayne Chung)
“Static” Analysis
A
BB
A
A asks B to join a project
A adds B to a list of recipientsAB, C, …
A
B
B accepts
“Dynamic” Analysis
invitequestion/accept
not join
join
New member activeintroducing others
Largegroupjoining
Detect "business" and "social"processes, not static artifacts.Sensors...communication eventsModels...social processes
31
Internet
DMZ
WS
BRIDGE
WinXP LINUX
WWW Mail
DIB:s
BGP
IPTables
Snort
Tripwire
SaMBa
Worm
ExfiltrationPhishing
PQS in Computer Security (Alex Barsamian, Vincent Berk, Ian De Souza, Annarita Giani)
5
87
12
12
PQSENGINE
observationsobservations
32
Sensors and Models
Noisy Internet Worm Propagation – fast scanning
Email Virus Propagation – hosts aggressively send emails
Low&Slow Stealthy Scans – of our entire network
Unauthorized Insider Document Access – insider information theft
Multistage Attack – several penetrations, inside our network
DATA movement
TIER 2 models
1
7
6
5
4
3
2
DIB:s Dartmouth ICMP-T3 Bcc: System1
ClamAV Virus scanner6
Flow sensor Network analysis 5
Samba SMB server - file access reporting4
IPtables Linux Netfilter firewall, log based3
Snort, Dragon Signature Matching IDS2
Tripwire Host filesystem integrity checker7
Outline
1.Motivation and Terminology
2.PQS Approach
3. Implementation of a PQS detecting
a.Phishing
b.Data Exfiltration
c. Covert Channel
4.Flow Attribution and Aggregation
5.Conclusion and Acknowledgments
34
Phishing Attack
The act of sending an e-mail to a user falsely claiming to be an established legitimate enterprise in an attempt to scam the user into surrendering private information.
The e-mail directs the user to visit a web site where they are asked to update personal information.
Bogus web site
First name,Last nameAccount NumberSSN
First name,Last nameAccount numberSSN
2
3
Visit http://www.cit1zensbank.com
1
35
Complex Phishing Attack Steps
attacks the victim
100.10.20.9
Victim
100.20.3.127
Attacker
165.17.8.126
Web page, Madame X
up
loa
ds
som
e co
de
downloads some data
Stepping stone
51.251.22.183
records username and password
… as usual browses the web and …
…. visits a web page. inserts username and password.(the same used to access his machine)
acc
esse
s u
ser
mac
hin
e u
sin
g u
sern
ame
and
pas
sw
ord
1
5
43
2
6
36
Complex Phishing Attack Observables
SOURCE
4. ATTEMPT (ATTACK RESPONSE)
SNORT POTENTIAL BAD TRAFFIC
100.10.20.9
Victim
100.20.3.127
Attacker
165.17.8.126
Web Server used- Madame XAttacker
2. A
TT
EM
PT
S
NO
RT
SS
H (
Po
licy
Vio
lati
on
)N
ON
-ST
AN
DA
RD
-PR
OT
OC
OL
3. D
AT
A U
PL
OA
D
FL
OW
SE
NS
OR
5. DATA DOWNLOAD FLOW SENSOR
1. RECON SNORT: KICKASS_PORNDRAGON: PORN HARDCORE
SOURCEDEST
SOURCE
SOURCE
SOURCE
DEST
DEST
DEST DEST
Stepping stone
51.251.22.183
Username password
Sept 29 11:17:09
Sept 29 11:24:07
Sept 29 11:24:06
Sep
t 29
11:
23:5
6
Sep
t 29
11:
23:5
6
Flow Sensor• Based on the libpcap interface for packet capturing.
• Packets with the same source IP, destination IP, source port, destination
port, protocol are aggregated into the same flow.
We did not use Netflow only because it does not have all the fields that we need.
• Timestamp of the last packet• # packets from Source to Destination• # packets from Destination to Source• # bytes from Source to Destination• # bytes from Destination to Source• Array containing delays in microseconds between packets in the flow
Two Models Based on the Flow Sensor
Volume Packets Duration Balance Percentage
Tiny: 1-128b
Small: 128b-1Kb
4:10-99
5: 100-999 6: > 1000
4: 1000-10000 s 5: 10000-100000 s
6: > 100000 s
Out >80
Low and Slow UPLOAD
Volume Packets Duration Balance Percentage
Tiny: 1-128b
Small: 128b-1Kb
Medium: 1Kb-100Kb
Large: > 100Kb
1: one packet 2: two pckts 3: 3-9 4: 10-99 5: 100-999 6: > 1000
0: < 1 s 1: 1-10 s 2: 10-100 s 3: 100-1000 s 4: 1000-10000 s 5: 10000-100000 s
6: > 100000 s
Out >80
UPLOAD
39
1
2
3
4
5
6 7RECON
ATTEMPT
ATTEMPT
ATTEMPT ATTEMPT
ATTEMPT
ATTEMPT
UPLOAD
UPLOAD
UPLOADDOWNLOAD
DOWNLOAD
UPLOAD
RECON ATTEMPT
Phishing Attack Model 1 – very specific
UPLOAD
40
1
2
3
4
5
6 7
ATTEMPT dst,src
ATTEMPT dst,A
ATTEMPT dst, src
ATTEMPT dst,src
ATTEMPT dst, ! src
UPLOAD dst, src
UPLOAD dst
UPLOAD dst, src DOWNLOAD
src
DOWNLOAD src
UPLOAD dst,src
ATTEMPT dst, !src
Phishing Attack Model 2 – less specific
UPLOAD dst,src
ATTEMPT dst, !src
RECON or ATTEMPT or COMPROMISE
RECON or ATTEMPT or COMPROMISE dst
RECON or ATTEMPT or COMPROMISE
41
1
2
3
4
5
6 7
UPLOAD dst, src
UPLOAD dst
UPLOAD dst, src DOWNLOAD
src
DOWNLOAD src
UPLOAD dst,src
Phishing Attack Model 3 – more general
UPLOAD dst,src
RECON or ATTEMPT or COMPROMISE
RECON or ATTEMPT or COMPROMISE dst
RECON or ATTEMPT or COMPROMISE dst, src
RECON or ATTEMPT or COMP dst, src
RECON or ATTEMPT or COMP dst, src
RECON or ATTEMPT or COMP dst
RECON or ATTEMPT or COMP dst, !src
RECON or ATTEMPT or COMP dst,! src
RECON or ATTEMPT or COMP dst, ! src
RECON or ATTEMPT or COMPROMISE
42
1 2 3 4RECON
ATTEMPT
ATTEMPT orUPLOAD
DOWNLOAD
RECON
Phishing Attack Model 3 – Most general
ATTEMPT orUPLOAD
ATTEMPT DOWNLOAD
Stricter models reduce false positives, but less strict models can detect unknown attack sequences
43
Air Force Rome Lab Blind Test
• Valuable feedback on performance and design• Strengths:
– Number of sensors integrated– Number of models– Easy of sensor integration– Ease of model building
• Drawback:– System is real-time (results time-out)
The collected data is an anonymized stream of network traffic, collected using tcpdump. It resulted in hundreds of gigabytes of raw network traffic.
December 12-14, 2005
44
Complex Phishing Attack Results
Attack steps 5 of 5
Background attackers 10 of 15
Background scanners 23 of 55
Stepping stones 1 of 1
False alarms 1
Attack steps 0 of 5
Background attackers 9 of 15
Background scanners 25 of 55
Stepping stones 0 of 1
No observations coming from Dragon sensor and Flow sensor
Using Dragon and Flow observations
45
0.0Threshold Values: 0.5 0.75
0
10
20
30
40
50
60
70
80
90
100
4s1 4s3 4s4 4s13 4s14 4s5 4s6 4s8 4s16 4s17
Precision
GO
AL
: >
AV
ER
AG
E
0
10
20
30
40
50
60
70
80
90
100
4s1 4s3 4s4 4s13 4s14 4s5 4s6 4s8 4s16 4s17
Mis-Associations
GO
AL
: <
AV
ER
AG
E
0
10
20
30
40
50
60
70
80
90
100
4s1 4s3 4s4 4s13 4s14 4s5 4s6 4s8 4s16 4s17
Fragmentation
GO
AL
: <
AV
ER
AG
E
Summary of Results
Scenario 4s14: Phishing attack
Outline
1.Motivation and Terminology
2.PQS Approach
3. Implementation of a PQS detecting
a.Phishing
b.Data Exfiltration
c. Covert Channel
4.Flow Attribution and Aggregation
5.Conclusion and Acknowledgments
47
Data Exfiltration
Tier 1 models monitor outbound data. They are based on flow analysis.
Tier 2 models correlate outbound data within a context to infer if it is a normal systems and user behavior or ongoing attacks
Tier 1 models monitor outbound data. They are based on flow analysis.
Tier 2 models correlate outbound data within a context to infer if it is a normal systems and user behavior or ongoing attacks
CNN.COMSunday, June 19, 2005 Posted: 0238 GMT (1038 HKT)
NEW YORK (AP) -- The names, banks and account numbers of up to 40 million credit card holders may have been accessed by an unauthorized user, MasterCard International Inc. said.
CNN.COMSunday, June 19, 2005 Posted: 0238 GMT (1038 HKT)
NEW YORK (AP) -- The names, banks and account numbers of up to 40 million credit card holders may have been accessed by an unauthorized user, MasterCard International Inc. said.
The Problem:
PQS Approach:
48
Exfiltration modes:
• SSH• HTTP• FTP• Email• Covert channel• Phishing• Spyware• Pharming• Writing to media
• paper• drives
• etc
Basic Ideas: An Example
Normal activity
ScanningInfectionData Access
by
tes
Time x 15 sec50 100 150 200 250 300 350
Low Likelihood ofMalicious Exfiltration
High Likelihood of Malicious Exfiltration
Increased outbound data
nfs2.pqsnet.net
100000
200000
300000
400000
500000
600000INOUT
49
Flow and Covert Channel Sensor
Samba
SnortTripwire
SnortIP Tables
Exfiltration
Data Access
Scanning
Infection
PQS
PQS
PQS
PQS
PQS
TIER 1
TIER 1 Models
TIER 1 Observations
TIER 1 Hypothesis
TIER 2
TIER 2 Models
TIER 2 Observations
TIER 2 Hypothesis
Hierarchical PQS Architecture
Events
Events
Events
Events
More ComplexModels
RESULTS
50
1
2
3
4TIER 1 VIRUS
Balanced Flow
Data Exfiltration
RECON
Example PQS model: Macro in word document for exfiltration
Balanced Flow
Data Exfiltration
Data Exfiltration
Balanced Flowor
Data Exfiltration
Balanced Flow
Word virus opens up a ftp connection with a server and upload documents.
Outline
1.Motivation and Terminology
2.PQS Approach
3. Implementation of a PQS detecting
a.Phishing
b.Data Exfiltration
c. Covert Channel
4.Flow Attribution and Aggregation
5.Conclusion and Acknowledgments
52
• A communication channel is covert if it is neither designed nor intended to transfer information at all. (Lampson
1973)
• Covert channels are those that use entities not normally viewed as data objects to transfer information from one subject to another. (Kemmerer 1983)
Covert Channel
STORAGE TIMING
Information is leaked by hiding data packet header fields: IP identification, Offset, Option, TCP Checksum, TCP Sequence Numbers.
Information is leaked by triggering or delaying events at specific time intervals.
53
Covert Channel in Interpacket Delays
SENDER RECEIVERWe shall not
spend a large expense
of timeBefore we
reckon with your
several loves,And make us
even with you. My thanes
and kinsmen,
…
We shall not spend
a large expense of time
Before we reckon
with your several loves,And make us
even with you. My thanes
and kinsmen,
…
010001010
010001010
INTERNET
54
Binary Asymmetric Channel
01 110 0 0 0 0 00 0
ERROR: it should be a 1
1 1
0 0
Noisy Channel
eP
eP
eP
eP
55
Binary Asymmetric Channel Capacity
24 hops.
Capacity: Highest amount of information per symbol that can be
transmitted with arbitrarily small error probability.
Received Sent
Bit/symbols
Error Probability
56
Statistical Detection
maxN
)(N
sample mean
# of packets with delay
max # of packets with the same delay
1)(
max
N
N Covert channel
Threshold used in the PQS experiments
max
1N
N
Level of confidence:
Nu
mb
er
of
pa
ck
ets
max
)(
N
N
bits
Delay – tenth of a sec
delays
For every traffic flow it registers the time delays between consecutive packets.
source ip: 129.170.248.33dest ip: 208.253.154.210source port: 44806dest port: 23164
Protocol: TotalSize: #Delays[20]: 3 0 0 16 882 2 0 17 698 2 0 0 1 0 1 0 0 0 0 0Average delay: Cmax; Cmean:
source ip: 129.170.248.33dest ip: 208.253.154.210source port: 44806dest port: 23164
Protocol: TotalSize: #Delays[20]: 3 0 0 16 882 2 0 17 698 2 0 0 1 0 1 0 0 0 0 0Average delay: Cmax; Cmean:
Sensor
3 delays between 0sec and 1/40sec
882 delays between 4/40sec and 5/40sec
Key
Attributes
Nu
mb
er
of
De
lay
s
Delay – tenth of a sec
source ip: 129.170.248.33dest ip: 208.253.154.210source port: 56441dest port: 23041
01
max
N
N
Delay – tenth of a sec
Nu
mb
er
of
De
lay
s
source ip: 129.170.248.33dest ip: 208.253.154.210source port: 56441dest port: 23036
11
max
N
N
58
Capacity
max
1N
N
max
1N
N
Error rates and capacity
Error Probability
Confidence,
Bit/symbols
is a discrete random variable. A sample of is denoted by
Define a covert channel, which has the same sample space as namely
uses in the sense that whenever a covert message is communicated
is a sample of
Let and
is the probability of FALSE ALARM
Detection-Capacity Tradeoffs
S = {a, b, c, d, e, f, g, h}
D = {b, c, d} Symbols used for covert communication
Sample space
The probability of D according the natural distribution of symbols is the false alarm rate.
60False alarm
Covert Information
Let be the amount of information communicated by the covert channel per
sample from Define to be the entropy of the distribution
Noting that by assumption, then
The expected covert information communicated per sample is
Detection-Capacity Tradeoffs
Outline
1.Motivation and Terminology
2.PQS Approach
3. Implementation of a PQS detecting
a.Phishing
b.Data Exfiltration
c. Covert Channel
4.Flow Attribution and Aggregation
5.Conclusion and Acknowledgments
62
Current Analysis
Flow Attribution
Billions per hour
Hundreds of thousands per hour
PACKETS
BYTES
FLOWS
Thousands per hour
Hundreds per hour
Flow Aggregation
How data move
EVENTS Fewer events to be analyzed
Flow Analysis = Data Reduction
63
Flow Attribution and Aggregation
FLOW ATTRIBUTION FLOW AGGREGATION
Recognizing that different flows (components), apparently totally unrelated, nevertheless belong to the same broader action (event).
Views flows as components of broader activities.
The goal is to correlate flows based on certain criteria.
The final goal is to attribute flows to people. Intermediate steps are a required part of the attribution process.
Uses logs that can explain a flow as legitimate or malicious.
The goal is to explain flows.
64
Aggregation
Flow aggregation. Activity aggregation.Recognizing that similar activities occur regularly at the same time, or dissimilar activities occur regularly in the same sequence.
We correlate activities into activity groups, patterns.
Recognizing that different flows, apparently totally unrelated, nevertheless belong to the same broader event (activity).
Flows are aggregated from captured network packets.
We aggregate flows into activities.
Example:
User requests a webpage (all DNS and HTTP flows aggregated)
Examples:
• Nightly backups to all servers (each backup is an activity)
• User requests a sequence of web-pages every morning.
Packet = Aggregated BytesFlow = Correlated PacketsActivity = Correlated FlowsPattern = Correlated Activities
65
1. The browser communicates with a name server to translate the server name "www.dartmouth.edu" into an IP Address, which it uses to connect to the server machine.
2. The browser forms a connection to the web server at that IP address on port 80.
3. Following the HTTP protocol, the browser sends a GET request to the server, asking for the file "http://www.dartmouth.edu/index.html."
4. The web server sends the HTML text for the Web page to the browser.
5. The browser reads the HTML tags and formatted the page onto your screen.
6. Browser possibly initiates more DNS requests for media such as images and video.
7. Browser initiates more HTTP and/or FTP requests for media.
Web Surfing in Detail
A FLOW IS INITIATED
The browser breaks the URL into three parts: the protocol ("http"), the server name ("www.dartmouth.edu") and the file name (“index.html").
A FLOW IS INITIATED
MULTIPLE FLOWS ARE INITIATED…
68
CorrelatedNetworkFlowsWithina LAN
Complex Activities ....TCP portscan
UDP portscan
Regular browsing/ download behavior
Regular UDP broadcasts (NTP)
System upgrade
69
Scenario: several packets in a flow triggered IDS alerts
Flow + Snort Alerts
Snort rule 1560generates an alert when an attemptis made to exploit a known vulnerability in a web server or a web application.
Snort rule 1852 generates an alert when an attempt is made to access the 'robots.txt' file directly.
FLOW
SNORT ALERTS
The flow can be characterized as malicious and further investigation must be done.
70
Current focusTheoretical approach for clustering aggregated flows.
Flow = As definedActivity = Aggregated flows Pattern = Correlated Activities
Approach: Graph theory (flows are the nodes and the edges are between correlated nodes).
We are thinking about defining a metric that captures the closeness between two different activities to allow grouping into patterns.
x
s
t
y
x
s
t
yz
w
Activity 1. Activity 2.
Can they be grouped in one pattern?
Outline
1.Motivation and Terminology
2.PQS Approach
3. Implementation of a PQS detecting
a.Phishing
b.Data Exfiltration
c. Covert Channel
4.Flow Attribution and Aggregation
5.Conclusion and Acknowledgments
72
Contribution
• Identification of a new generation of threats.
• Identification and implementation of approaches based on a Process Query System to detect them.
• Introduction and implementation of flow attribution and aggregation.
73
Work in Progress
• Build a theory of flow attribution and aggregation.
• Develop a theory of tractability to characterize phenomenon in the sense of multi hypothesis tracking.
• Identification of new application domains
• Statistical theory of undetectable covert communication
74
Acknowledgements
Research Support: DARPA, DHS, ARDA/DTO, ISTS, I3P, AFOSR, Microsoft
George Cybenko
Alex BarsamianMarion BatesChad BehreVincent BerkValentino Crespi (Cal State LA)Ian deSouzaPaul ThompsonAnnarita Giani
Robert Gray (BAE Systems)G. Jiang (NEC LAB)Naomi Fox (UMass, Ph.D. student) Hrithik Govardhan, MS (Rocket Software)Yong Sheng Ph.D. (Dartmouth CS postdoc)Josh Peteet, MS (Greylock Partners)Alex Jordan, MS (BAE Systems)Chris Roblee, MS (Lawrence Livermore NL)George Bakos (Northrup Grumman)Doug Madory M.Sc. (BAE Systems)Wayne Chung Ph.D. (IDA/CCS)Glenn Nofsinger Ph.D. (BAE Systems)Yong Sheng Ph.D (CS Dartmouth College)
Active Members Alumni