1 maintaining packet order in two-stage switches isaac keslassy, nick mckeown stanford university
TRANSCRIPT
1
High PerformanceSwitching and RoutingTelecom Center Workshop: Sept 4, 1997.
Maintaining Packet Order in Two-Stage Switches
Isaac Keslassy, Nick McKeownStanford University
2
Two Successive Scaling Problems
OQ routers: + work-conserving- memory bandwidth =
(N+1)RR
R
RR
IQ routers: + memory bandwidth = 2R- arbitration complexity
Bipartite Matching
R R
3
Today: 64 ports at 10Gbps, 64-byte cells.
• Arbitration Time = = 51.2ns
• Request/Grant Communication BW = 17.5Gbps
10Gbps 64bytes
IQ Arbitration Complexity
Two main alternatives for scaling:1. Increase cell size (Kar et al., 2000)2. Eliminate arbitration (Chang et al., 2001)
Scaling to 160Gbps:• Arbitration Time = 3.2ns• Request/Grant Communication BW = 280Gbps
4
Desirable Characteristics for Router Architecture
Ideal: OQ• 100% throughput• Minimum delay• Maintains packet order
Necessary: able to regularly connect any input to any output
What if the world was perfect? Assume Bernoulli iid uniform arrival traffic...
5
Cyclic Shift?
1
N
1
N
Uniform Bernoulli iid traffic: 100% throughput
6
Cyclic Shift?
1
N
1
N
b(t) (t)q(t)
Problem: real traffic is non-uniform
01
1)()(
1lim
e
N
T
tttb
TT
Long-term service opportunities exceed arrivals:
1
)]([ eN
tbE
7
Two-Stage Switch
External Outputs
Internal Inputs
1
N
ExternalInputs
Load-balancing cyclic shift
Switching cyclic shift
1
N
1
N
11
2
2
100% throughput for broad range of traffic types (C.S. Chang et al., 2001)
8
Two-Stage SwitchExternal Outputs
Internal Inputs
1
N
ExternalInputs
1
N
1
N
1(t) 2(t)
b(t)
q(t)
a(t)
• Traffic rate: ΛeN
taEtEtatEtbE1
)]([)]([)]()([)]([ 11 ππ
)()()( 1 tattb π• First cyclic shift:
011
1)()(
1lim 2
e
Ne
N
T
tttb
TT
• Long-term service opportunities exceed arrivals:
• (2 = 1 possible)
9
• Eliminates arbitration
• 100% throughput
• Conventional router packaging
Two-Stage Switch Characteristics
10
1
2
3
Phase 2
Phase 1
Using a Single Stage Twice
Lookup
Buffer
Lookup
Buffer
Lookup
Buffer
Linecards
1
N
1
N
1
N
11
Racks of linecards
Optical links
Optical SwitchFabric
Two-Stage Switch Characteristics
12
Problem: unbounded mis-sequencing
External Outputs
Internal Inputs
1
N
ExternalInputs
Cyclic Shift Cyclic Shift
1
N
1
N
11
2
2
Two-Stage Switch Characteristics
13
Full Frames First (FFF):Intuitive Idea
External Outputs
Internal Inputs
1
N
ExternalInputs
Cyclic Shift Cyclic Shift
1
N
1
N
11
2
2
Idea: 1. Spread cells evenly across all linecards
t1 2
14
Full Frames First (FFF):Intuitive Idea
Idea: 1. Spread cells evenly across all linecards2. Read them in order
External Outputs
Internal Inputs
1
N
ExternalInputs
Cyclic Shift Cyclic Shift
1
N
1
N
11
2
3
2
15
Full Frames First (FFF):Intuitive Idea
Idea: 1. Spread cells evenly across all linecards2. Read them in order
External Outputs
Internal Inputs
1
N
ExternalInputs
Cyclic Shift Cyclic Shift
1
N
1
N
1
2
2
3
1
16
First Problem
Problem: if two packets don’t arrive consecutively, there may be a hole in the reading sequence
External Outputs
Internal Inputs
1
N
ExternalInputs
Cyclic Shift Cyclic Shift
1
N
1
N
11
2
2
3
t1 2 3
17
Coordination Buffer
Solution: collect cells from a flow in a coordination buffer, and load-balance them among linecards
1
N
Cyclic Shift Cyclic Shift
1
N
1
N
12
3
2
Flow Load Balancing
Coordination Buffer (VOQ)
1
t1 2 3
18
Second Problem
Problem: No access to cell 2 because of head-of-line blocking
1
N
Cyclic Shift Cyclic Shift
1
N
1
N
12
2
Flow Load Balancing
Coordination Buffer (VOQ)
1
3
a
b
t1 32
ta b
Input 1:Input 2:
19
Expanding VOQ Structure
Solution: expand VOQ structure by distinguishing among switch inputs
2
1
3
a
b
20
FFF: Guarantees
Theorem 2: for any arrival process, Davg (FFF) Davg (OQ) + (4N2 - 2)
Theorem 1: for any arrival process for which OQ has 100% throughput, so does FFF
Theorem 3: FFF maintains packet order
21
Two-Stage Switch in Optics
R/N
Passive mesh
1
23
1
23
R/NR/N
Passive mesh
1
23
1
23
2R/N
123
123
Cyclic Shift Cyclic Shift
R R
22
Summary
• FFF: practical algorithm that solves mis-sequencing
• Same throughput as OQ, and average delay within a bound
• New approach to optical switching