Hierarchical Routing:1. Leonard Kleinrock and Farouk Kamoun:

Hierarchical routing for large networks: Performance evaluation and optimization, Computer Networks, Vol. 1(3):155-174, , Jan. 1977.

2. PNNI Routing, Private Network-Network Interface Specification Version 1.0, ATM Forum, March 1996.

3. Paul Tsuychia, The Landmark Hierarchy: A new hierarchy for routing in very large networks, ACM Sigcomm, p. 35-42, 1988.

Leonard Kleinrock and Farouk Kamoun: Hierarchical routing for large networks

Performance evaluation and optimization

Computer Networks, Vol. 1(3), Jan 1977, pp. 155-174

(uses slides prepared by Boris Drazic)

Routing in the 1970s

• Computer networks are new and have a small number of nodes

• ARPANET is predecessor of Internet

• Uses concepts of:– routing table (RT) – Distributed routing

protocolsARPANET 1969

Routing in the 1970s

• Computer networks are new and have a small number of nodes

• ARPANET is predecessor of Internet

• Uses concepts of:– routing table (RT) – Distributed routing

protocolsARPANET 1971

Routing in the 1970s

• Computer networks are new and have a small number of nodes

• ARPANET is predecessor of Internet

• Uses concepts of:– routing table (RT) – Distributed routing

protocols

• One routing table for each “node” of the network

ARPANET 1980

Number of Internet Hosts

Jul-77

May-78

Mar-79Jan

-80

Nov-80

Sep-81

Jul-82

May-83

Mar-84Jan

-85

Nov-85

Sep-86

Jul-87

May-88

Mar-89Jan

-90

Nov-90

Sep-91

Jul-92

May-93

Mar-94Jan

-95

Nov-95

Sep-96

Jul-97

May-98

Mar-99Jan

-00

Nov-00

Sep-01

Jul-02

May-03

Mar-04Jan

-05

Nov-05

Sep-06

Jul-07

May-08

1

10

100

1,000

10,000

100,000

1,000,000

10,000,000

100,000,000

1,000,000,000

Axis Title

Source: Internet Systems Consortium (www.isc.org)

Active BGP Entries (FIB)

Source: http://www.cidr-report.org/

Example of Routing Table (1)

Destination Next-hopb b

c c

d b

e b

f b

g b

h b

i b

Routing table for node a

Hierarchical Routing Schemes

• Goal: Reduce the size of routing tables (RT)• Approach: Keep complete routing information

about nodes that are “close” and less information about nodes that are “far away” – One RT entry for every destination that is “close”– One RT entry for every set of destinations that is “far

away”• Forwarding has two parts:

1. Forward the message “close” to the destination2. Forward the message to the destination

Example of RT (2)

Destination Next-hopb bc c

2 b

3 b

Routing table for node a

“close” destinations

“far away” destinations`

m-level Hierarchical Clustering

• Create a hierarchy with m levels by:– Each node is a 0th level cluster– Group nodes into 1st level clusters– Group 1st level clusters into 2nd level clusters– …

`

3rd level cluster

2nd level cluster

1st level cluster

0th level cluster

Tree representation in 3-level hierarchy

`

3rd level cluster

2nd level cluster

1st level cluster

0th level cluster

RT in 3-level Hierarchical ClusteringDestination Next-hop

1.1.1 self

1.1.2 1.1.2

1.1.3 1.1.3

1.1.4 1.1.2

1.1 self

1.2 1.1.2

1.3 1.1.2

1 self

2 1.1.2

3 1.1.3

nodes in same cluster

`

clusters in same supercluster

superclusters

0th level cluster entries

1st level cluster entries

2nd level cluster entries

Example: Source: 1.1.1

Destination: 3.2.2

Forwarding Rules:

1. Forward to leftmost cluster in address;

2. When reached, forward to next cluster to the right in the address

What is the optimal clustering in a network with N nodes?

• Proposition 1: Given the number of levels m, if each cluster has the same number of lower level clusters, the minimum routing table lengths is .

• Proposition 2: The global optimal clustering is achieved for m=ln(N), when each cluster has e lower level clusters, and the minimum table length is l=e ln(N).

Minimum Relative Table Length

RT size decreases fast for m < ln(N)

Minima for:m = ln(N)

Hierarchical Routing increases path length

Shortest path: 4 hops With hierarchical routing: 5 hops

Example: Source: a

Destination: i

Increased Path Length

• Assumption 1: Every cluster has the same number of sub-clusters and for every pair of nodes in a cluster a path exists between them in that cluster

• Assumption 2: The diameter of any kth level cluster is less or equal to some quantity dk

• hc is average path length with clustering • h is average path length without clustering

Increased Path Length

• Proposition 10: Under Assumptions 1 and 2

Limiting Performance

• Assumption 3: Any cluster contains the shortest path between any pair of nodes that belong to that cluster

• Proposition 11: Under Assumptions 1-3, for an m-level optimal clustering hierarchy where the diameter of any cluster is bounded by a power law function of the number of nodes in that cluster:

Increased Path Length

• Define as the increase in path length produced by introducing clustering

• Observe E, which is a bound on D

• Presented results for a network where Assumptions 1-3 hold

Decrease in RT Length for a Given Maximum Increase in Path Length

Interpret “E” as the tolerated “stretch” factor

Conclusion

• In very large networks, hierarchical routing schemes achieve great reduction of routing table size with only a small increase in path lengths between nodes, when compared to non-hierarchical routing schemes

Hierarchical Routing Application: ATM Networks

In the 1990s, the ATM Forum adopted a routing scheme which is based on hierarchical routing

ATM Connection Acronyms

• VC - virtual channel, synonymous with “circuit” or “connection”

• VPC - virtual path connection• VCC - virtual channel connection• PVC - permanent virtual circuit• PVP - permanent virtual path• SVC - switched virtual circuit• SVP - switched virtual path• SPVC - switched/signaled/soft permanent virtual circuit• PMP - point-to-multipoint

Traditional Network Infrastructure

CompanyA

CompanyB

Telephone network

Data network

Residential user

x

Video network

The B-ISDN vision (from mid 1980s!)

CompanyA

CompanyB

Residential user

x

BroadbandIntegrated Services

Network(B-ISDN)

ATM’s Key Concepts

• ATM uses Virtual-Circuit Packet Switching– ATM can reserve capacity for a virtual circuit. This is

useful for voice and video, which require a minimum level of service

– Overhead for setting up a connection is expensive if data transmission is short (e.g., web browsing)

• ATM packets are small and have a fixed sized – Packets in ATM are called cells– Small packets are good for voice and video

transmissionsHeader(5 byte)

Data (48 byte)

Cell is 53 byte long

Virtual Paths and Virtual Circuits

Virtual Path Connections

Virtual Channel

Connection

VPI identifies virtual path (8 or 12 bits)

VCI identifies virtual channel in a virtual path (16 bits)

Link

3/2432/172

7/2433/242

1/4041/241

Routing Table of switch v

portVPI/VCI

toVPI/VCI

Port 1

Port 2

Port 3

Port 4

Switch

VPI/VCI assignment at ATM switches

1/24 7/24

3/24 1/40

3/24

2/17

PNNI

UNI NNIATM

Switch

UNI

UNI NNI

ATMNetwork

UNI

ATMNetwork

ATMSwitch

UNI: User-to-Network InterfaceNNI: Network-to-Network Interface

PNNI: Suite of protocols for topology discovery and routing of an ATM network

PNNI Routing

• Goal of PNNI Routing: Establish a switching path from sender to receiver

• PNNI characteristics:– Link State Routing– Source Routing– Hierarchical Routing– Crankback

• Routing protocol information is sent on VPI/VCI=0/18

Link State Routing Algorithm

B.2.2B.2.1

B.2.3

B.1.1 B.1.2

B.3.2

B.3.1

• PNNI uses link state routing:– Each node floods route messages on its links

to all nodes in the network– Each node has complete topology information

• Source routing: PNNI computes route at the first switch

• QoS: Route messages also contain QoS information on links

• Hierarchy: PNNI deals with complex networks by using a hierarchy

PNNI Routing Hierarchy

• Routing hierarchy is defined recursively:– Neighbor nodes build Peer Groups by comparing their address prefixes (nodes with longest

prefix match are in the same peer group)– Each group behaves like a Logical Group Node (LGN) in the next level peer group

A.1.2

A.1.1

A.2.2

A.2.1

A.2.3

B.2.2B.2.1

B.2.3

B.1.1 B.1.2

B.3.2

B.3.1

C.1.2

C.1.1

A.1 A.2

B.2

B.1

B.3 C.1

A CB

Building the PNNI Routing Hierarchy

A.1.2

A.1.1

A.2.2

A.2.1

A.2.3

B.2.2B.2.1

B.2.3

B.1.1 B.1.2

B.3.2

B.3.1

C.1.2

C.1.1

• Each switch is initialized with a 20-byte ATM address

PNNI Routing Hierarchy

• Nodes with common prefix A.1, A.2, B.2, B.2, B.3, C.1 each form a logical group node

A.1.2

A.1.1

A.2.2

A.2.1

A.2.3

B.2.2B.2.1

B.2.3

B.1.1 B.1.2

B.3.2

B.3.1

C.1.2

C.1.1

A.1 A.2

B.2

B.3 C.1

B.1

PNNI Routing Hierarchy

• Nodes with common prefix A, B, C each form a logical group node at the next level

A.1.2

A.1.1

A.2.2

A.2.1

A.2.3

B.2.2B.2.1

B.2.3

B.1.1 B.1.2

B.3.2

B.3.1

C.1.2

C.1.1

A.1 A.2

B.2

B.3 C.1

B.1A C

B

Resulting Hierarchy

• Within each peer group, nodes or LGNs elect a peer group leader (PGL), which represents the peer group at the next level

A.1.2

A.1.1

A.2.2

A.2.1

A.2.3

B.2.2B.2.1

B.2.3

B.1.1 B.1.2

B.3.2

B.3.1

C.1.2

C.1.1

A.1 A.2

B.2

B.1

B.3 C.1

A CB

Complete Hierarchy

A.1 A.2B.2

B.1

B.3 C.1

A CB

Routing Messages Hierarchy

A.1 A.2B.2

B.1

B.3 C.1

A CB

Routing messages are called PNNI Topology State Packets (PTSP)1. Nodes within each peer group exchange PTSP2. Group Leader relays routing information from higher level topology to lower layer

Resulting Hierarchy

• Node B.2.3’s view of the network topology• Node can now run source routing

C

B.1

B.2.2B.2.1

B.2.3

B.2

CB

B.3A

Source routing in the hierarchy

• At A.1.1 : Route = [ (A.1.1, A.1.2), (A.1, A.2), (A, B)]• At A.1.2 : Route = [ (A.1, A.2), (A, B)]• At A.2.1 : Route = [ (A.2.1, A.2.2, A.2.3), (A.1, A.2), (A, B)]• At A.2.2 : Route = [ (A.2.1, A.2.2, A.2.3), (A.1, A.2), (A, B)]• At A.2.3 : Route = [ (A, B)]• At B.1: Route = [ (B.1, B.3), (A, B) ]• At B.3: Route = [ ]

A.1.2

A.1.1 A.2.1

A.2.2

A.2.3B.2

B.1

B.3

A.1A.2A

B

desti-nation

source

Crankback and Alternate routing

• At A.1.1 : Route = [ (A.1.1, A.1.2), (A.1, A.2), (A, B)]• At A.1.2 : Route = [ (A.1, A.2), (A, B)]• At A.2.1 : Route = [ (A.2.1, A.2.2, A.2.3), (A.1, A.2), (A, B)]• At A.2.2 : Route = [ crankback: (A.2.1, A.2.2, A.2.3) ,

(A.1, A.2), (A, B)]• At A.2.1 : Route = [ (A.2.1, A.2.3), (A.1, A.2), (A, B)]• At A.2.3 : Route = [ (A, B)]• and so on

A.1.2

A.1.1 A.2.1

A.2.2

A.2.3B.2

B.1

B.3

A.1A.2A

B

desti-nation

sourceQoS cannot be satisfied

Paul Tsuychia: The Landmark Hierarchy:

A new hierarchy for routing in very large networks

ACM Sigcomm 1988, p. 35-42

Drawback of Hierarchical Routing

Route from 3.2.1 1.2.3

1. Shortest path: 3.2.12.2.3 3.2.1

2. Hierarchical Route: 3.2.1 Area 3.1Area 3.1 Area 1 (Area 1.1)Area 1.1 Area 1.2 (1.2.1)1.2.1 1.2.21.2.2 1.2.3

1.2.1

1.2.2

Enter “Landmarks”

• Landmarks can be seen from far away!

• Landmark router: Normal router can see a landmark router from several hops away Information about landmark routers is disseminated to normal routers

Landmark router

• Router 1 is landmark router with radius 2– Routers up to 2 hops away have

routing entries for Router 1 Router 1 can be seen from 2 hops away

1

2

3

4

5

6 7

89

1011

Hierarchy of Landmarks

• Every router is a level-0 landmark with a small radius • Some level-0 landmarks are level-1 landmarks

1. Radius of level-1 landmark is greater than that of level-0;2. Each level-0 landmark LM0 [id] can reach at least one level-1 landmark

in r0[id] hops

• Some level-1 landmarks are level-2 landmarks 1. Radius of level-2 landmark is greater than that of level-1;2. Each level-1 landmark LM1 [id] can reach at least one level-11 landmark

in r1 [id] hops

……• A small number of routers are level-H landmarks (“global landmarks”)

1. Every router “can see” all level-H landmarks

Notation: i : level number (0 is lowest level, H is highest level)LMi : landmark at level-iLMi [id]: specific landmark at level i with identifier idri [id]: Radius of LMi [id]

Forwarding in Landmark Hierarchy• Destination: LM0[a]

• Source forwards packet towards LM2[c]

• Once the packet is within reach of r1[b], packet issent towards LM1[c]

• Once the packet is withinreach of r0[a], packet issent towards LM0[a]

r2[c]

r1[b]

r0[a]

Source

LM0[a]

LM1[b]

LM2[c]

Routing Table• Each router keeps a table of the next hop on the shortest path

to each landmark for which it has routing entries: – Each router has entries

… for every LM0[id] within a radius of r0[id],

… every LM1[id] within a radius of r1[id], … and so on.

• This ensure that a router – has full knowledge of all routers within immediate vicinity (i.e., the

LM0 routers)– has knowledge of a portion of the network routers further away– Has full local information, and increasingly less information further

away in all directions

• Note the difference to pure hierarchical routing

Addressing• In a pure hierarchy:

The address of a router is a reflection of the area(s) at each hierarchical level in which the router resides

e.g., telephone number +1-416-946-3403

• In a Landmark hierarchy: The address of a router is a reflection of the Landmark(s) at each hierarchical level which the router is near.

• Landmark Address is a series of Landmark identifiers LMH[idH].LMH-1[IDH-1]. . .LM0[id0]

such that:1. The landmark represented by each address component must be

within radius of the landmark in the next lower address component2. Nodes may have multiple address (since more than one level-n

landmark may be within radius of a level-n-1 landmark)

Forwarding• Path: Source LM0[a]

• Address of LM0[a]:LM2[c].LM1[b].LM0[a]

1. At source:– has entry for LM2[c]

– forwards towards LM2[c]

2. At first router within r1[b]:– has entry for LM1[b]

– forwards towards LM1[b]

3. At first router within r0[a]:– has entry for LM0[a]

– forwards towards LM0[a]

r2[c]

r1[b]

r0[a]

Source

LM0[a]

LM1[b]

LM2[c]

Forwarding1. Routing path in landmark

routing are longer than shortest path

2. Path do not necessarily traverse the landmarks given in a landmark address

r2[c]

r1[b]

r0[a]

Source

LM0[a]

LM1[b]

LM2[c]

Example:

d.d.a

d.d.bd.d.c

d.n.h

d.d.l

d.d.k

d.n.x

d.n.n

d.n.p

d.n.o

d.n.q

d.n.r

d.d.j

d.n.s

d.n.t

d.i.v

d.i.w

d.d.d

d.i.u

d.i.g

d.i.k

d.i.i

d.d.e

d.d.f

d.i.k

All routers are LM0 (r0=2)Diamonds: LM1 (r1=4)Large circles: LM2 (r2=8)

Example:

d.d.a

d.d.bd.d.c

d.n.h

d.d.l

d.d.k

d.n.x

d.n.n

d.n.p

d.n.o

d.n.q

d.n.r

d.d.j

d.n.s

d.n.t

d.i.v

d.i.w

d.d.d

d.i.u

d.i.g

d.i.k

d.i.i

d.d.e

d.d.f

All routers are LM0 (r0=2)Diamonds: LM1 (r1=4)Large circles: LM2 (r2=8)

Landmark Level Next Hop

LM2[d] 2 f

LM1[i] 1 k

LM0[e] 0 f

LM0[k] 0 k

LM0[f] 0 f

Routing table of g:

d.d.a

d.d.bd.d.c

d.n.h

d.d.l

d.d.k

d.n.x

d.n.n

d.n.p

d.n.o

d.n.q

d.n.r

d.d.j

d.n.s

d.n.t

d.i.v

d.i.w

d.d.d

d.i.u

d.i.g

d.i.k

d.i.i

d.d.e

d.d.f

Example:Source: gDestination: tAddress of t: d.n.t

Landmark Level Next Hop

LM2[d] 2 f

LM1[i] 1 k

LM0[e] 0 f

LM0[k] 0 k

LM0[f] 0 f

Routing table of g:

Dynamic Algorithms

• Several algorithms are needed:– Assign landmarks and determine radii– Discover landmarks and establish path to

landmarks– Needs to use distance vector, and not link state (Why?)– Modification to distance vector: radius limits advertisements

– Bind identifiers (id’s) to addresses– Addresses of a router can change

Performance

• Evaluated by simulation• Performance metrics:

– Average routing table size: R– Stretch: Ratio of paths length of landmark or

hierarchical routing ( ) to shortest path ( )

Simulations• Compared simulation

of pure hierarchy with landmark hierarchy

• 200 routers• Different settings

• Comment: Method of comparison questionable

• Landmark routing

ri : radius of level-i landmarkdi: distance to nearest

level-i landmark

Summary

• Landmark hierarchy has similar performance as pure hierarchical routing

• Easier to configure (claimed):– Not very sensitive to placement of landmarks– More sensitive to failures