Inter-domain Routing: Today and Tomorrow

Dr. Jia Wangjiawang@research.att.com

AT&T Labs ResearchFlorham Park, NJ 07932, USA

http://www.research.att.com/~jiawang/

Prof. Zhuoqing Morley Maozmao@umich.edu

Department of EECSUniversity of Michigan

Ann Arbor, MI 48109, USAhttp://www.eecs.umich.edu/~zmao/

IEEE INFOCOM 2004 TutorialMarch 8, 2004

March 8, 2004 2

Outline

1. Overview of Inter-domain routing2. Routing policies3. Measuring inter-domain paths4. Routing instability5. BGP Beacon - measurement

infrastructure6. Implication on network engineering7. Security issuesOur opinions should not be taken to represent AT&T policies

Part I: Overview of Inter-domain Routing

March 8, 2004 4

Internet

Loose cooperative effort of Internet Service Providers (ISPs) E.g., AT&T, Sprint, UUNet, AOL

Best effort service Connectedness

Anyone connected to the Internet can exchange traffic with anyone else connected to the Internet

March 8, 2004 5

Internet routing

: Routing session

routes

Internet

IP traffic

Data plane:forward traffic

Control plane:exchange routes

rusty.cs.berkeley.eduIP=169.229.62.116

Prefix=169.229.0.0/16

www.cnn.comIP=64.236.16.52

Prefix=64.236.16.0/20

March 8, 2004 6

Internet routing dictates application performance

www.cnn.comIP=64.236.16.52

Prefix=64.236.16.0/20

: Routing session

routes

Internet

IP traffic

Data plane:forward traffic

Control plane:exchange routes

Fail over to alternate routerusty.cs.berkeley.edu

IP=169.229.62.116Prefix=169.229.0.0/16

March 8, 2004 7

Internet routing domain

Network devices under same technical and administrative control

Common routing policy E.g., ISPs, enterprise networks

March 8, 2004 8

Autonomous System (AS)

Autonomous routing domain with an AS number (ASN)

AS numbers 16 bits integer Public AS number: 1 – 64511 Private AS number: 64512 – 65535 Examples

AT&T: 7018, 6431, … Sprint: 1239, 1240, … MIT: 3

March 8, 2004 9

More than 14,000 ASes today

Qwest Sprint UUnet

University company

AT&T

business

ISP ISP ISP

ISP ISP ISP ISP

ISP

Qwest Sprint UUnet

University company

AT&T

business

ISP ISP ISP

ISP ISP ISP ISP

ISP

Berkeley

Internet

CNN

Calren

Level3

GNN

IP traffic

AutonomousSystem

Berkeley

Calren

Level3

Berkeley

Calren

Level3 Qwest Sprint UUnet

University Company

AT&T

Business

ISP ISP ISP

ISP ISP ISP ISP

ISP

March 8, 2004 10

Internet Initiative Japan (IIJ)

March 8, 2004 11

IIJ, Tokyo

March 8, 2004 12

Telstra international

March 8, 2004 13

WorldCom (UUNet)

March 8, 2004 14

UUNet, Europe

March 8, 2004 15

Sprint, USA

March 8, 2004 16

Charlotte

Portland

Providence

Newark

Cedar Knolls

Syracuse

Buffalo

White Plains

Rochester

Columbia

New Orleans

Nashville

AustinHouston

TulsaOklaho

maCity

Albuquerque

Phoenix

AnaheimAnaheim

Las Vegas

Salt LakeCity

ColoradoSprings

Milwaukee

Detroit

Columbus

Cincinnati

SeattleSpokane

Portland

Louisville

Little Rock

Jacksonville

Ft. Lauderdale

Miami

RaleighRichmond

DenverIndianapolis

Pittsburgh

Baltimore

Plymouth

Atlanta

Minneapolis

Gardena

Tampa

SanBernardino

Arlington

Ft. Worth

Rochelle Pk

Honolulu

Orlando

Sherman Oaks

Ojus

Hamilton SquareSilver

Springs

Wayne

Chicago

RollingMeadows

Omaha

St Louis

San Diego

Anchorage, AK

N X OC48

Backbone Node

Gateway Node

N X DS3N X OC3

Remote Access Router

R Remote GSR Access Router

N X OC12

NX OC192

CambridgeFramingham

Stamford Bridgeport

Grand Rapids Providence

Glenview

Albany

Sacramento

Oakland

Redwood City

San Jose

San Francisco

ChicagoSan

Francisco

Florissant

Davenport

Worcester

Madison

Camden, NJ

Norcross

New Brunswick

Birmingham

San Antonio

Oak Brook

South Bend

Dayton Bohemia

Hartford

San Juan PR

W. Palm Beach

Harrisburg

Des Moines

Memphis

Greensboro

Norfolk

R

KansasCity

AkronRR

RLos Angeles

Dallas

Wash.DC

St. Paul

Freehold

R

Manchester

R

RR

Ft. Lauderdale

Dunwoody

Note: Connectivity and nodes shown are targeted for deployment; actual deployment

may vary. Maps should not be used to predict service availability.

R

R

R

Phil

NYC

Cleveland

R

RNYC-Bdwy

Birmingham

LA-Airport

Blvd

Year end 2001

Rev. 6-4-01

AT&T IP Backbone, USA

March 8, 2004 17

GARR-B

March 8, 2004 18

Gigabit research network

March 8, 2004 19

(Summer '03)

UW-Superior

UW-StoutUW-River Falls

Fox Valley TC

UW-Oshkosh

UW-Milwaukee

UW-ParksideUW-Whitewater

UW-Madison

UW-Platteville

UW-La Crosse

UW-Eau Claire

UW-Stevens Point

UW-Green Bay

Marshfield

Rhinelander

Rice Lake

Clintonville

StilesJct.

Portage

Dodgeville

La Crosse

Genuity

OC-3 (155Mbps)

DS-3 (45Mbps)

T1 (1.5Mbps)

OC-12 (622Mbps)

Qwestand OtherProvider(s)

Internet 2& Qwest

Peering - Public and Private Commodity Internet Transit Internet2 Merit and Other State Networks National Education Network Regional Research Peers

Wausau

Gigabit Ethernet

Chicago - 1

Chicago - 2(Winter '02)

Chicago

wiscnet.net

GO BUCKY!

March 8, 2004 20

MIT.edu

http://bgp.lcs.mit.edu/

March 8, 2004 21

Internet routing architecture

IP traffic

Berkeley CNN

Level3

Internet

Calren GNN

Inter-domain routing

Intra-domain routing

March 8, 2004 22

Intra-domain routing

Run within a certain network infrastructure Optimize routes taken between points

within a network Internal Gateway Protocols (IGPs)

Metrics based OSPF (Open Shortest Path First) RIP (Routing Information Protocol) IS-IS (Intermediate System to Intermediate

System)

March 8, 2004 23

Inter-domain routing

Run between networks Provide full connectivity of entire

Internet External Gateway Protocol (EBGP)

Policy based BGP (Border Gateway Protocol)

March 8, 2004 24

Inter-domain routing and BGP

Static routing Mainly for stub networks

Default routing Small stub networks

Dynamic routing Via BGP

No need to run BGP in static routing and default routing.

March 8, 2004 25

Link state

Examples: OSPF, IS-IS Based on Dijkstra’s shortest path

computation Each router periodically floods immediate

reachability information to other routers Fast convergence High communication and computation

overhead Not scalable for large networks

Requires periodic refreshes

March 8, 2004 26

Vectoring

Distance vs. Path Vector Distance: hop count (RIP) Path: entire path (BGP)

Helps identify loops Supports policy-based routing based on path

Minimal communication overhead Takes longer to converge, i.e., in

proportion to the maximum path length

March 8, 2004 27

Link state vs. vectoring

OSPFIS-IS

RIP

BGP

IGP

EGP

Link state Vectoring

BGP is a path vector protocol

March 8, 2004 28

Classful addressing

IPv4: 32 bits Five classes of networks

Class

Address

Mask # of networks

# of hosts

A 0* 255.0.0.0 128 ~1.6M

B 10* 255.255.0.0 16384 65535

C 110* 255.255.255.0

~2.1M 255

D Used for multicast

E Reserved and currently unusedImprove scaling factor of routing in the Internet => classless

March 8, 2004 29

RFC1519: Classless Inter-domain Routing (CIDR)

No implicit mask based on the class of the network

Explicit masks passed in the routing protocol

Allow aggregation and hierarchical routing

March 8, 2004 30

CIDR addressing

00001100 00100110 00000000 00000000

11111111 11111111 11000000 00000000

IP address: 12.70.0.0 Mask: 255.255.252.0

CIDR representation: 12.70.0.0/22

Address

Mask

Network prefix Host identifier

00001100 00100110 00000000 00000000

11111111 11111111 11000000 00000000

March 8, 2004 31

Address aggregation

Internet

12.70.1.0/24

12.70.2.0/24

12.70.3.0/2412.70.0.0/24

ISP AISP B

12.70.0.0/2212.71.0.0/16

12.71.0.0/16

March 8, 2004 32

Routing and forwarding

Routing The decision process of choosing

optimal path that is consistent with the administrative or technical policy

Forwarding The act of receiving a packet, doing a

lookup, and copying a packet to the next hop

March 8, 2004 33

Classless forwarding

Internet

135.120.0.1

12.70.0.20

IP traffic

Prefix Next hop12.70.0.0/24 10.20.0.112.70.0.0/16 10.20.1.112.0.0.0/8 10.20.128.10.0.0.0 10.20.128.10

10.20.0.1

10.20.1.1

10.20.128.1

10.20.128.10

March 8, 2004 34

Inter-domain routing with CIDR support

BGP-4 [RFC1771] De facto EGP Path vector protocol Run on top of TCP for reliability Carry routing information between

ASes Policy based routing

March 8, 2004 35

BGP basic operations

Set up BGP session Exchange all candidate routes Send incremental updates

March 8, 2004 36

Establish BGP session

12.10.0.1 12.10.0.2

Establish neighboring session between 12.10.0.1 and 12.10.0.2

Prefix Next hop12.70.0.0/24 10.20.0.112.9.0.0/16 10.20.1.1

Prefix Next hop135.120.0.0/24 10.128.0.168.35.0.0/16 10.192.1.1

TCP 179

March 8, 2004 37

Exchange all candidate routes

12.10.0.1 12.10.0.2

Prefix Next hop12.70.0.0/24 10.20.0.112.9.0.0/16 10.20.1.1135.120.0.0/24 10.128.0.168.35.0.0/16 10.192.1.1

Prefix Next hop135.120.0.0/24 10.128.0.168.35.0.0/16 10.192.1.112.70.0.0/24 10.20.0.112.9.0.0/16 10.20.1.1

12.70.0.0/24 10.20.0.112.9.0.0/16 10.20.1.1

135.120.0.0/24 10.128.0.168.35.0.0/16 10.192.1.1

March 8, 2004 38

Send incremental updates

12.10.0.1 12.10.0.2

Prefix Next hop12.70.0.0/24 10.20.0.112.9.0.0/16 10.20.1.1135.120.0.0/24 10.128.0.168.35.0.0/16 10.192.1.1

Prefix Next hop135.120.0.0/24 10.128.0.168.35.0.0/16 10.192.1.112.70.0.0/24 10.20.0.112.9.0.0/16 10.20.1.1

Withdraw 12.9.0.0/16

March 8, 2004 39

BGP messages

OPEN: set up a peering session UPDATE: announce new routes or

withdraw previously announced routes

NOTIFICATION: shut down a peering session

KEEPALIVE: confirm active connection at regular interval

March 8, 2004 40

Internal vs. external BGP

Internet I-BGP

E-BGP

AS A

AS B

AS C

March 8, 2004 41

I-BGP mesh

I-BGP update

E-BGP update I-B

GP update

I-BGP update

March 8, 2004 42

Make I-BGP scale for large AS

Route reflectors Confederations

March 8, 2004 43

Route reflector

E-BGP update

RR RR

Only best paths being sent by RR

March 8, 2004 44

Confederation

AS 1000

EBGP

EBGP

EBGPIBGP IBGP

AS 65010 AS 65020

March 8, 2004 45

BGP updates

Three blocks Prefix Path attributes Unreachable routes

March 8, 2004 46

BGP attributes

Value Code Reference1 ORIGIN [RFC1771] 2 AS_PATH [RFC1771] 3 NEXT_HOP [RFC1771] 4 MULTI_EXIT_DISC [RFC1771] 5 LOCAL_PREF [RFC1771] 6 ATOMIC_AGGREGATE

[RFC1771] 7 AGGREGATOR [RFC1771] 8 COMMUNITY [RFC1997] 9 ORIGINATOR_ID [RFC1998] 10 CLUSTER_LIST [RFC1998]

11 DPA [Chen] 12 ADVERTISER [RFC1863] 13 RCID_PATH / CLUSTER_ID

[RFC1863] 14 MP_REACH_NLRI [RFC2283] 15 MP_UNREACH_NLRI

[RFC2283] 16 EXTENDED COMMUNITIES

[Rosen] 17 NEW_AS_PATH [E.Chen] 18 NEW_AGGREGATOR [E.Chen] 19 SAFI Specific Attribute

(SSA) [Nalawade] 20-254 Unassigned 255 reserved for development

http://www.iana.org/assignments/bgp-parameters

March 8, 2004 47

Establish connectivity

135.120.0.0/16

12.10.0.1

12.10.0.2

Prefix Next hop AS path135.120.0.0/16 12.10.0.1 1

EBGP

IBGPIBGP

IBGPEBGP

12.10.0.5

12.10.0.6

AS 1 AS 2

AS 3Prefix Next hop AS path135.120.0.0/16 12.10.0.5 2 1

Prefix Next hop AS path135.120.0.0/16 12.10.0.1 1

March 8, 2004 48

IGP and BGP working together

135.120.0.0/1612.10.0.1

12.10.0.2

Prefix Next hop AS path135.120.0.0/16 12.10.0.1 1

EBGP

IBGPIBGP

IBGPEBGP

12.10.0.5

12.10.0.6

AS 1 AS 2

AS 3Prefix Next hop AS path135.120.0.0/16 12.10.0.1 1

10.10.0.1

Prefix Next hop12.10.0.0/30 10.10.0.1135.120.0.0/16 10.10.0.1

12.10.0.0/30