ecs236 winter 2006: intrusion detection #4: anomaly detection for internet routing

02/06/2006 ecs236 winter 2006 1

ecs236 Winter 2006:

Intrusion DetectionIntrusion Detection#4: Anomaly Detection for Internet Routing

Dr. S. Felix Wu

Computer Science Department

University of California, Davishttp://www.cs.ucdavis.edu/~wu/

[email protected]

02/06/2006 ecs236 winter 2006 2

Intrusion DetectionIntrusion Detection

IntrusionDetection

Model

Input eventsequence Results

Pattern matching

02/06/2006 ecs236 winter 2006 3

Internet in 1969Internet in 1969

UTAH

UCLA

SRI

UCSBWhat was the link speed/bandwidth?

02/06/2006 ecs236 winter 2006 4

ARPANet in 1969 ARPANet in 1969 InternetInternet

UTAH

UCLA

SRI

UCSBWhat was the link speed/bandwidth?

56 kbps

02/06/2006 ecs236 winter 2006 5

The “Internet” The “Internet” as February 1, 2006 21319 Autonomous Systems 177300 IP Address Prefixes announced

http://bgp.potaroo.net/cidr/

02/06/2006 ecs236 winter 2006 6

AS and IP address prefixAS and IP address prefix

UCDavis:169.237/16

AS6192

Autonomous System:

AS6192 is the routers in UC DavisUC Davis owns 169.237/16

02/06/2006 ecs236 winter 2006 7

Address PrefixAddress Prefix

Prefix aggregation/de-aggregation

Notation of network address prefixes169.237.0.0/16 10101001111011010000000000000000

11111111111111110000000000000000Prefix Prefix length

169.237.0.0/16 (less specific)

169.237.128.0/17

169.237.192.0/18

169.237.204.0/19 (more specific)

169.237.0.0/17

BGP prefers more specific

02/06/2006 ecs236 winter 2006 8

Peering ASesPeering ASes

UCDavis:169.237/16

AS6192 AS11423 (UC)

AS11537 (CENIC)AS513

02/06/2006 ecs236 winter 2006 9

AS6192 AS11423

UCDavis:169.237/16

AS6192

AS11423 (UC)


an AS Path:169.237/16 11423 6192

02/06/2006 ecs236 winter 2006 10

AS11423 AS11423 AS11537 AS11537

UCDavis:169.237/16

AS6192

AS11423 (UC)


an AS Path:169.237/16 1153711423 6192

02/06/2006 ecs236 winter 2006 11

AS11537 AS11537 AS513 AS513

UCDavis:169.237/16

AS6192

AS11423 (UC)


an AS Path:169.237/16 5131153711423 6192

02/06/2006 ecs236 winter 2006 12

Packet ForwardingPacket Forwarding

UCDavis:169.237/16

AS6192 AS11423 (UC)


an AS Path:169.237/16 5131153711423 6192

02/06/2006 ecs236 winter 2006 13

The Dynamics of “Internet”The Dynamics of “Internet”

Link/node failuresSoftware malfunctionsImplementation relatedPolicy configurationTopology changesOther “interesting” dynamics (that we can not explain well yet…)

02/06/2006 ecs236 winter 2006 14

The Scale of the “Internet”The Scale of the “Internet” Every single prefix, and their “dynamics”, must be

propagated to every single AS (21319). Every single AS must maintain the routing table

such that it knows how to route the traffic toward any one of the 177300 prefixes to the right destination.

BGP is the protocol to support the exchange of routing information for ALL prefixes in ALL ASes.

02/06/2006 ecs236 winter 2006 15

DNS and BGPDNS and BGP

DNS BGP BGP DNS Without DNS, BGP and the Internet can

still function. But, without BGP, DNS won’t work very

much.DNS

BGP – Internet Service

02/06/2006 ecs236 winter 2006 16

Routing Dynamics in 2001Routing Dynamics in 2001

# of BGP updates over a fixed period of time (e.g., 2 hours)

a color dot = an AS Path being used

02/06/2006 ecs236 winter 2006 17

DNS Root-A ServerDNS Root-A Server

2001.4.16:8.29 3333 9057 3356 3561 62452001.4.16:8.29 3333 9057 3356 701 62452001.4.16:8.49 3333 9057 3356 3561 62452001.4.16:8.55 3333 9057 3356 1239 62452001.4.16:8.56 3333 1103 8297 6453 1239 62452001.4.16:8.56 3333 1103 8297 6453 701 62452001.4.16:9.05 3333 1103 8297 6453 1239 62452001.4.16:9.24 3333 9057 3356 4544 62452001.4.16:9.27 3333 9057 3356 701 62452001.4.16:9.32 3333 1103 8297 6453 1239 6245 2001.4.16:9.33 Withdraw2001.4.16:9.38 3333 9057 3356 4544 62452001.4.16:9.38 3333 286 209 4544 62452001.4.16:9.40 Withdraw2001.4.16:10:2 3333 1103 8297 6453 1239 62452001.4.16:10:8 3333 9057 3356 3561 6245

02/06/2006 ecs236 winter 2006 18

Global FailureGlobal Failure AS7007 falsely de-aggregates 65000+

network prefixes in 1997 and the east coast Internet was down for 12 hours.

02/06/2006 ecs236 winter 2006 19

Packet ForwardingPacket Forwarding

UCDavis:169.237/16

AS6192 AS11423 (UC)


an AS Path:169.237/16 5131153711423 6192

02/06/2006 ecs236 winter 2006 20

Global FailureGlobal Failure AS7007 falsely de-aggregates 65000+

network prefixes in 1997 and the east coast Internet was down for 12 hours.

AS6192 AS11423 (UC)


169.237/16142.7.6/24204.5.68/24….

Black Hole

02/06/2006 ecs236 winter 2006 21

UnderstandUnderstand

Lots of Anomalies– Anomaly detection

Understand and Explain the Anomalies– Network Management– Valuable Inputs for the future Design– Better and more practical Mathematical Models

02/06/2006 ecs236 winter 2006 22

the Modelmodel-based

event analysis

observed system events

SBL-basedAnomalyDetection

analysisreports

ExampleSelection

Explanation Based

Learning

modelupdate

02/06/2006 ecs236 winter 2006 23

BGP Observation PointsBGP Observation Points (e.g. RIPE AS12654) (e.g. RIPE AS12654)

Internet

RIPE

…

Each peer will tell us, at any moment of time, how to reach each of the 177300 prefixes!

“Get the real BGP data”

02/06/2006 ecs236 winter 2006 24

Multiple BGP Observation PointsMultiple BGP Observation Points

Oregon

Internet

RIPE UC Davis

02/06/2006 ecs236 winter 2006 25

Real BGP Data ReplayReal BGP Data Replay

02/06/2006 ecs236 winter 2006 26

Origin AS in an AS PathOrigin AS in an AS Path UCDavis (AS-6192) owns 169.237/16 and AS-6192 is the

origin AS AS Path: 5131153711423 6192

– 12654 13129 6461 3356 11423 6192– 12654 9177 3320 209 11423 6192– 12654 4608 1221 4637 11423 6192– 12654 777 2497 209 11423 6192– 12654 3549 3356 11423 6192– 12654 3257 3356 11423 6192– 12654 1103 11537 11423 6192– 12654 3333 3356 11423 6192– 12654 7018 209 11423 6192– 12654 2914 209 11423 6192– 12654 3549 209 11423 6192

12654

6192

11423

2091153733564637

2914701835493333

02/06/2006 ecs236 winter 2006 27

2152 6192286 174 2152 6192

2914 174 2152 61923130 2914 174 2152 6192

3292 174 2152 61923549 174 2152 6192

2493 3602 174 2152 61925462 174 2152 61925503 174 2152 61925511 174 2152 6192

6667 174 2152 6192 6762 174 2152 61926895 174 2152 6192

15444 174 2152 6192293 2153 6192

2497 2152 61924777 2497 2152 61927500 2497 2152 6192

3303 2152 61923356 2152 6192

2905 701 3356 2152 61921239 3356 2152 6192

3130 1239 3356 2152 6192

1668 3356 2152 61923257 3356 2152 6192

21202 30912 29518 3549 3356 2152 6192

3561 3356 2152 61925511 3356 2152 61926453 3356 2152 61927018 3356 2152 6192

3557 2152 61921221 4637 2152 6192

6539 2152 61926939 2152 6192

3257 6939 2152 619216150 8434 3257 6939 2152 6192

5390 6939 2152 61928121 6939 2152 61928426 6939 2152 619212956 6939 2152 619213237 6939 2152 619215444 6939 2152 6192

11608 2152 6192 10876 4600 11537 2153 6192

7660 11537 2153 6192

169.237/16169.237/16

AS2152 CSU-53 California State UniversityAS2153 CSU-53 California State University

02/06/2006 ecs236 winter 2006 28

Origin AS Changes (OASC)Origin AS Changes (OASC) Ownership: UCDavis (AS-6192) owns

169.237/16 and AS-6192 is the origin AS Current

– AS Path: 291420911423 6192– for prefix: 169.237/16

New– AS Path: 29143011273 81– even worse: 169.237.6/24

Which route path to use? Normal or Abnormal??

12654

6192

11423

2093011

273

2914

81

169.237/16169.237.6/24

02/06/2006 ecs236 winter 2006 29

year Median number increase rate #BGP table entries increase rate1998 683 520001999 810.5 18.7% 60000 15.40%2000 951 17.3% 80000 33.30%2001 1294 34.8% 109000 36%

Max: 10226(9177 from a single AS)

02/06/2006 ecs236 winter 2006 30

Origin AS Changes (OASC)Origin AS Changes (OASC) Normal or Abnormal??

– How to handle this problem?

12654

6192

11423

2093011

273

2914

81

169.237/16169.237.6/24

02/06/2006 ecs236 winter 2006 31

decay

update

clean

compute thedeviation

alarm generation

threshold control

timer control

raw events long term profile

0 5 10 15 20 25 300

02/06/2006 ecs236 winter 2006 32

decay

update

clean

cognitivelyidentify thedeviation

alarm identification

InformationVisualizationToolkit

raw events cognitive profile

02/06/2006 ecs236 winter 2006 33

Real-Time OASC DetectionReal-Time OASC Detection Low level events: BGP Route Updates High level events: OASC

– 1000+ per day and max 10226 per day– per 3-minutes window in real-time demo

IP address blocks Origin AS in BGP Update Messages Different Types of OASC Events

02/06/2006 ecs236 winter 2006 34

1101

1000

1001

110001110011111001111011

110000110010111000111010

00110110

AS#

Qua-Tree Representation ofIP Address Prefixes

169.237/1610101001.11101101/16

02/06/2006 ecs236 winter 2006 35

1101

1000

1001

110001110011111001111011

110000110010111000111010

00110110AS#

AS# Representation

AS-1

AS-7777

AS-15412

AS-6192

AS-81

02/06/2006 ecs236 winter 2006 36

AS81 punched a “hole” on 169.237/16

yesterday169.237/16

today169.237/16169.237.6/24

yesterdayAS-6192

todayAS-81

victim

offender

02/06/2006 ecs236 winter 2006 37

OASC Event TypesOASC Event Types Using different colors to represent types of

OASC events C type: CSS, CSM, CMS, CMM H type: H B type: B O type: OS, OM

02/06/2006 ecs236 winter 2006 38

August 14, 2000August 14, 2000

AS-7777punchedhundreds ofholes.

02/06/2006 ecs236 winter 2006 39

April 6, 2001April 6, 2001

AS15412 caused 40K+ MOAS/OASC events within 2 weeks…

02/06/2006 ecs236 winter 2006 40

April 7-10, 2001April 7-10, 2001

04/07/2001 all 04/07/2001 15412 04/08/2001 all 04/08/2001 15412

04/09/2001 all 04/09/2001 15412 04/10/2001 all 04/10/2001 15412

02/06/2006 ecs236 winter 2006 41

April 11-14, 2001April 11-14, 2001

04/11/2001 all 04/11/2001 15412 04/12/2001 all 04/12/2001 15412

04/14/2001 all 04/14/2001 1541204/13/2001 1541204/13/2001 all

02/06/2006 ecs236 winter 2006 42

April 18-19, 2001 – Again??April 18-19, 2001 – Again??

04/18/2001 all 04/18/2001 15412 04/19/2001 all 04/19/2001 15412

02/06/2006 ecs236 winter 2006 43

SPRINT (AS-1239)SPRINT (AS-1239)(on December 3, 2000, 3000+ (on December 3, 2000, 3000+ BB events) events)

02/06/2006 ecs236 winter 2006 44

Which types of “screens” are more interesting and why? Why was AS15412 picked for further special

examination? Under this context, why were we only focusing on April

6-12 and April 18-19?– Or, why is April 16 irrelevant?

Why are April 12 and 18 similar? What is the difference between these two instances in

April of 2001?

Gaining Knowledge about OASCGaining Knowledge about OASC

02/06/2006 ecs236 winter 2006 45

the Modelmodel-based

event analysis

observed system events

SBL-basedAnomalyDetection

analysisreports

ExampleSelection

Explanation Based

Learning

modelupdate

02/06/2006 ecs236 winter 2006 46

The KDD ProcessThe KDD Process

Knowledge about the application domain Data preparation Data mining Interpretation Using the discovered knowledge

02/06/2006 ecs236 winter 2006 47

OASC DataOASC Data

How do we define an OASC event?– 169.237/16– Origin AS Changes from AS-6192 to AS-81

– But, exactly how should we obtain the information?

02/06/2006 ecs236 winter 2006 48

BGP Observation PointsBGP Observation Points (e.g. RIPE AS12654) (e.g. RIPE AS12654)

Internet

RIPE

…


“Get the real BGP data”

02/06/2006 ecs236 winter 2006 49

RIPE

…


OneOne Routing table forall 177300177300 prefixes

AS-12654AS-12654

02/06/2006 ecs236 winter 2006 50

Per-Day AnalysisPer-Day Analysis

Today’s routing table against yesterday’s– on ALL prefixes

02/06/2006 ecs236 winter 2006 51

Per-Update AnalysisPer-Update Analysis

Finer granularity Observing “per-peer” OASC events Correlation with AS Topology information

02/06/2006 ecs236 winter 2006 52

Project Proposal AreasProject Proposal Areas Network-based IDS Host-based IDS Application-based IDS Routing infrastructure Security Anomaly Detection and Alert Correlation IDS evaluation and Honeypot

Or, anything else you are interested

ecs236 winter 2006: intrusion detection #4: anomaly detection for internet routing

Documents