ip ngn backbone routers for the next decade€¦ · ip/mpls backbone ip/mpls internet peering pdh...
TRANSCRIPT
Cisco Public 1© 2010 Cisco and/or its affiliates. All rights reserved.
IP NGN Backbone Routersfor the Next DecadeJosef Ungerman
Consulting SE, CCIE #6167
© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 2
• Motivations for IP NGN
• Trends is IP/MPLS Core Design
• Router Anatomy Trends
• Latest Product Updates
• Switching Fabric Technologies
• Network Processor Technologies
Cisco Public 3© 2010 Cisco and/or its affiliates. All rights reserved.
Exponential Growth and Evolving Traffic Mix
Global IP Traffic2009 2014
Video & Multimedia
Mobile Internet
IT Services via Cloud
300+%Market Growth
39X Increasein Traffic
90% ConsumerTraffic
IP traffic will increase 4X
(767 exabytes by 2014)
Very Different
Traffic Profile
• IPv6 and IPv4 Address Exhaustion
• LTE moving from circuits to packets
• new access technologies – WiFi, FTTX
Source: Cisco Visual Networking Index—Forecast, 2009-2014
More Issues:
© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 4
Traffic
Revenue
Animated slide
MonetizationNew revenue streams
OptimizationEfficient delivery
Profitability
Cisco Public 5© 2010 Cisco and/or its affiliates. All rights reserved.
Network Architecture Trends
© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 6
ACCESS AGGREGATION BACKBONE
SDH/ATM
Mobile
SDH
FR/ATM
Metro Ethernet
PDH MW SDH
FMC/
Packet
Core
ATM
IP/MPLS Internet
Peering
PDH
CPE
DSLAM
MSAN
CPE
IGW
BRAS
CPE
Regional
DWDM
Backbone
DWDM
SGSN
GGSN
IGW
INTERNET
100’s 10’s 1’s1000’s
2G
3G
Data
Centers
© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 7
ANYACCESS
AGGREGATION
CarrierE AGGREGATIONIP/MPLS
BACKBONEIP/MPLS
Internet
Peering
PDH MW
2G
3G
CPE
DSLAM
IGW
CGv6
BRAS
DWDM
INTERNET
CORE
Ethernet
MPLS-TP
MPLS
…
PON
LTE, WiFI…
EDGE
IP NGN
• single multiservice IP/MPLS network
• operational hierarchy
• Circuits (Pseudowires) and Clouds (IP)
100’s 10’s1000’s 1’s
Data
Centers
© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 8
ANYACCESS
AGGREGATION EDGE
IP/MPLS Core Internet
Peering
PDH MW
DPI
2G
3G
CPE
DSLAM
IGWBRAS
DN
VPN PE
IPTV PE
FMC PE
BB PE
P P BB PE
DWDM
INTERNET
CORE
Ethernet
MPLS-TP
MPLS
…
PON
LTE, WiFI…
AN
100’s 10’s1000’s 1’s
CarrierE AGGREGATIONIP/MPLS
© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 9
ANYACCESS
AGGREGATION EDGE
IP/MPLS Core Internet
Peering
PDH MW
2G
3G
CPE
DSLAM
IGWAN
P BB PE
DWDM
INTERNET
CORE
Ethernet
MPLS-TP
MPLS
…
PON
LTE, WiFI…
Backbone Optimization
• IP Offload links, 100GE
• Flexible Edge placement
• Consolidation techniques
Existing PE’s
Existing PE’s
DN
100’s 10’s1000’s 1’s
CarrierE AGGIP/MPLS
Cisco Public 10© 2010 Cisco and/or its affiliates. All rights reserved.
Statistical Multiplexing(VPN optimized)
- O-E-O regeneration avoided as much as possibleno need for OTN Switching cross-connects in CEE countries
- Static long lambdas are usedno need for dynamic G.MPLS in the static Internet backbone
- Importance of OTN interfaces in routers (IPoDWDM)STM-256 (OTU3) and 100GE (OTU4)
PE P P PEPE PE
P P
PE P P PEPE PE
P P
Direct Link(Broadband optimized)
Saves Money, and Keeps the Core Hierarchy!
l1 l1
Traffic
Generator
Mngmt 1GE
Optical Monitoring Points
OSA
CRS-1
Poznan
CRS-1
Warszawa
40GE40GE613 km
Real-Life Example:
Warszawa-Poznan, 613km
40G over Siemens 10G WDM
Cisco Public 11© 2010 Cisco and/or its affiliates. All rights reserved.
100GE•IEEE 802.3ba, ITU OTU4
• router interface
• router router or transponder
40GE
•IEEE 802.3ba, ITU OTU3e
•data center interface
• router to transponder
Link Bundling (LACP)•up to 64x TGE today (32x deployed)
• dynamic adaptable hash (also 3,5,7,11 links)
• 7-tuple hash for equal load-sharing
100GE•throughput (100GE is like a bundle 12-14 TGEs)
• no hashing inefficiencies, easy troubleshooting
• contribution HDTV is 1.24Gbps single stream!
© 2010 Cisco Systems, Inc. All rights reserved. Cisco Public 12
RSP
LC
LC
LC
LC
RSP
LC
LC
LC
LC
Cluster (ASR9000)
Key motivation is in the Access edge:
Simpler Access Dual-homing• scaling the L2/L3 control plane (not data plane)
Key motivation is in the Core:
Simpler Core PoP• scaling the non-blocking data plane
• back-to-back, 2+1, 8+2, etc.
Cluster(one L2 & IP/MPLS control plane)
Cluster + Satellites(remote linecards)
Multi-Chassis(one router)
RP
LC
LC
RP
LC
Multi-Chassis (CRS)
LC
LC
LC
LC
dRP
© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 13
1) Reduce the number of networks
IP NGN = single multiservice network
2) Reduce the number of layers
IP NGN = IP/MPLS + DWDM
3) Reduce the number of nodes
Direct Links = huge broadband traffic takes shortest path
4) Reduce the number of links
MPLS Technology = statistical multiplex and hierarchy
5) Innovate – make use of modern technologies
Moore’s Law = Lower TCO, Price/Gigabit, Watt/Gigabit
Optimization: How to move bits cheaper......reduce OPEX, CAPEX, and keep reasonable quality?
© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 14
7.757.49 7.65
2.77 2.69 2.74
0.00
1.00
2.00
3.00
4.00
5.00
6.00
7.00
8.00
9.00
0
2000
4000
6000
8000
10000
12000
14000
4/S 8/S 16/S
Watt
s
CRS Form Factor
Power @ 40G/slot
Power @ 140G/slot
Total Power
W/Gbps @ 40G/slot
W/Gbps @ 140G/slot
© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 15
• Silicon has fundamentally followed Moore’s law
• Optics is fundamentally an analog problem
$-
$5,000.00
$10,000.00
$15,000.00
$20,000.00
$25,000.00
$30,000.00
$35,000.00
$40,000.00
$45,000.00
$50,000.00
1997 1999 2001 2003 2005 2007 2010 2012
$/Gbps
Routers: 23% Cumulative Average $/Gbps Drop per year / fewer ASICsOptics: $/G stays flat (best case) or increases from one technology to the next
Cisco Core Router Example
0
5000
10000
15000
20000
25000
30000
35000
40000
45000
0.00
500.00
1000.00
1500.00
2000.00
2500.00
2007 2008 2009 2010 2011 2012 2013 2014
Cap
acit
y i
n G
b/s
Co
st/
Gb
($U
SD
)
DWDM Optic Cost and DWDM Capacity
10Gb 40Gb 100Gb Capacity
10G/40G/100G Networking Ports Biannual Worldwide
and Regional Market Size and Forecasts
May 2010
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
10G % ASP
40G % ASP
100G % ASP
Future Trend
Cisco Core Routing Example
Switch Engine
Client Optics
DWDM Optics
Infrastructure
Cisco Public 16© 2010 Cisco and/or its affiliates. All rights reserved.
Router Anatomy Trends
© 2010 Cisco Systems, Inc. All rights reserved.BRKIPM-2012 17
2004: Cisco CRS-1 – 40G (STM-256) per slotFocus on Quality (scale, modularity, resiliency)
IOS IOS IOS IOS
RP (active) RP (standby)
SPA
SPA
SIP-800
mid
plan
e 4
0G
Switch Fabric Cards
(all 8 active)
MSC-40 – 40G
NP
NP
Qbuff.
IOS
Q
Q
buff.
FP-40 – 40G
NP
NP
Qbuff.
IOS
Q
Q
buff.
56G rx
112G tx
49G rx
98G tx
SPP (Silicon Packet Processor)
- 40 Gbps, 80 Mpps [u-programmable]
- one for Rx, one for Tx processing
PLIM
mid
plan
e 4
0G
4, 8 or 16 Linecard slots + 2 RP slots
56G rx
112G tx
49G rx
98G tx
56G rx
112G tx
© 2010 Cisco Systems, Inc. All rights reserved.BRKIPM-2012 18
IOS IOS IOS IOS
RP (active) RP (standby)
FP-140 – 140G MSC-140– 140G
SPA
SPA
SIP-800
mid
plan
e 4
0G
MSC-40 – 40G
NP
NP
Qbuff.
IOS
Q
Q
buff.
FP-40 – 40G
NP
NP
Qbuff.
IOS
Q
Q
buff.
NP’
NP’
Qbuff.
IOS
Q
Q
buff.
NP’
NP’
Qbuff.
IOS
Q
Q
buff.
PLIM
mid
plan
e 4
0G
14x 10GE
Switch Fabric Cards
(all 8 active)
4, 8 or 16 Linecard slots + 2 RP slots
100GE
QFA (Quantum Flow Array)
- 140 Gbps, 125 Mpps [u-programmable]
- one for Rx, one for Tx processing
141G rx
226G tx
mid
plan
e 1
40G
mid
plan
e 1
40G
2010: Cisco CRS-3 – 140G per slotFocus on Quality (scale, modularity, resiliency)
© 2010 Cisco Systems, Inc. All rights reserved.BRKIPM-2012 19
2009: Cisco ASR9000 – 8x 10GE per slotCompact Router/Switch
IOS IOS
RSP (active)
IOS IOS
RSP (fab. active)
NP
NP
NP
NP
buff.
buff.
buff.
buff.
IOS
Transport LC – 40G
NP
NP
NP
NP
buff.
buff.
buff.
buff.
IOS
NP
NP
NP
NP
buff.
buff.
buff.
buff.
Transport LC – 80G
4 or 8 Linecard slots
Trident Network Processor
- 30 Gbps, 28 Mpps [u-programmable]
- shared for Rx and Tx processing
- one per 10GE (up to 8 per card)
NP
NP
buff.
buff.
Transport LC – 16x TGE OS
NP
NP
buff.
buff.
IOSNP
NP
buff.
buff.
NP
NP
buff.
buff.
NP
NP
NP
NP
buff.
buff.
buff.
buff.
IOS
Transport LC – 8x TGE OS
RSP (Route/Switch Processor)• CPU + Switch Fabric
• active/active fabric elements
92G
92G
46G
46G
92G
92G
X
© 2010 Cisco Systems, Inc. All rights reserved.BRKIPM-2012 20
NP
NP
NP
NP
buff.
buff.
buff.
buff.
IOS
Transport LC – 40G
4 or 8 Linecard slots
IOS IOS
RSP’ (active)
IOS IOS
RSP’ (fab. active)
NP
NP
buff.
buff.
Transport LC – 24x TGE
NP’
NP’
buff.
buff.
IOSNP
NP
buff.
buff.
NP’
NP’
buff.
buff.
NP’
NP’
NP’
NP’
buff.
buff.
buff.
buff.
Transport LC – 2x 100GE
IOS
NP
NP
NP
NP
buff.
buff.
buff.
buff.
IOS
NP
NP
NP
NP
buff.
buff.
buff.
buff.
Transport LC – 80G
92G
92G
2011: Cisco ASR9000 – 2x 100GE per slotCompact Router/Switch
>400G
© 2010 Cisco Systems, Inc. All rights reserved.BRKIPM-2012 21
IOS IOS
RSP (active)
F
buff.SP RP
IOS IOS
RSP (standby)
F
buff.
SP RP
2003: Cisco 7600 – 4x 10GE per slotThe Switch/Router
NP
NP
NP
NP
buff.
buff.
buff.
buff.
F
IOS
ES+
3, 4, 6, 9 or 13 Linecard/RSP slots
NP
NP
buff.
buff.F
IOS
ES+
NP
NP
NP
NP
buff.
buff.
buff.
buff.
F
IOS
ES+
NP
NP
NP
NP
buff.
buff.
buff.
buff.
F
IOS
ES+
20G
20G
Edge or Core LC – 40G Edge or Core LC – 20G
32G local
switching
Trident Network Processor
- 10Gbps full-duplex
- H-QoS, VPLS, u-programmable
EARL Switching ASIC
- 48 Mpps
- Catalyst 6500 compatibility
RSP (Route/Switch Processor)• CPU + Switch Fabric + Switch ASIC
• active/standby fabric elements
© 2010 Cisco Systems, Inc. All rights reserved.BRKIPM-2012 22
IOS IOS
RSP (active)
F
buff.SP RP
IOS IOS
RSP (standby)
F
buff.
SP RP
2003: Cisco 7600 – 4x 10GE per slotThe Switch/Router
NP
NP
NP
NP
buff.
buff.
buff.
buff.
F
IOS
ES+
3, 4, 6, 9 or 13 Linecard/RSP slots
NP
NP
buff.
buff.F
IOS
ES+
NP
NP
NP
NP
buff.
buff.
buff.
buff.
F
IOS
ES+
NP
NP
NP
NP
buff.
buff.
buff.
buff.
F
IOS
ES+
Edge or Core LC – 40G Edge or Core LC – 20G
SIP-400
SPA
SPANP
buff.
NP
buff.
Bus (headers only) IOSbuff.
IOS
buff.
SPA
SPA
SIP-200
buff.
F
SPA
SPAIOS
NP
SIP-600
buff.
IPSEC
FWSM
ACE
buff.
buff.
buff.
buff.
buff.
buff.
buff.
© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 23
...and keep a reasonable quality?
1) Compact Anatomy
RSP, Route/Switch Processor (instead of RP and FC)
Ethernet-oriented Linecard (non-modular, less memory)
2) Linecard Architecture
Multiple smaller NP’s (eg. 4x 10G instead of 1x 40G)
One NP is shared for Rx and Tx (not dedicated NP’s per Rx and Tx)
Multiple smaller Fabric Ports (eg. 2x 20G instead of 1x 40G)
3) Special Core-facing Linecards
8/16 queues per port (instead of thousands)
lower-scale NP (no need for thousands of interfaces)
licenses for features that not everybody uses (eg. VPN, OTN, scale)
4) Oversubscribed Cards
2:1 ingress overbooking (eg. PON OLT Aggregation)
Cisco Public 24© 2010 Cisco and/or its affiliates. All rights reserved.
IP NGN Routers Update
© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 25
# of Slots
Height
Linecard [Gbps]
System [Tbps]
Linecard [Gbps]
System [Tbps]
Per Slot [Gbps]
CRS-8/S
8 (+2 RP)
½ rack
40
.640
140/105
2.24
400
CRS-16/S
16 (+2 RP)
1 rack
40
1.28
140/122
4.48
400
Core
Maximum BW / Protected BW with a failed fabric element
CRS-MC (8+2)
128 (+2 RP)
10 racks
40
10.24
140/122
35.84
400
CRS-4/S
4 (+2 RP)
1/3 rack
40
.320
140/105
1.12
-
Chassis
2004
(CRS-1)
2010
(CRS-3)
Future
~7500 systems
~450 customers
(~80 are CRS-3)
© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 26
14x 10GBE-WL-XFP
Line-rate performance (140Gbps)
Configurable LAN/WAN PHY
Interface Modules (PLIMs)
Each PLIM requires FP140 or other forwarding card
20x 10GBE-WL-XFP
Oversubscribed (140Gbps)
Configurable LAN/WAN PHY
1x 100GBE
Line-rate performance (100Gbps)
CFP optics (LR4, 10km)
© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 27
MSC-40 – High-speed edge @ 40Gbps
H-QoS (8,000 queues), 800 interfaces, WAN
FP-40 – IP/MPLS Core &Peering @ 40Gbps
Per-port QoS, IP/MPLS, ACL, Netflow…
Forwarding Cards
MSC-140 – High-speed edge
H-QoS (64,000 queues), scale (12,000 vlans)
FP-140 – IP/MPLS Core & Peering
Per-port QoS, IP/MPLS, ACL, Netflow…
LSP-140 – MPLS Core P
Per-port QoS, MPLS, IP Multicast, limited IP
© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 28
IPv6 Transition: CGSE Carrier-Grade Services Engine PLIM
Introducing the new engine for massive Cisco CGv6 deployments (XR 3.9.1)
now: Cisco CRS
2011: XR12K, ASR9K
Cisco CGSE
20+ million active translations
100s of thousands of subscribers
1+ million connections per second
20Gb/s of throughput
XML API (eg. port-forwarding)
Netflow V9 translation logging
Security
IPv6 Transition solution feature set
Carrier-Grade NAT44 (3.9.1)
NAT64 stateless (3.9.3)
6rd BR (3.9.3)
NAT64 stateful (4.1.2)
DS-Lite, 4rd, dIVI – planned
© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 29
# of Slots
Height
Linecard [Gbps]
System [Tbps]
Linecard [Gbps]
System [Tbps]
Per Slot [Gbps]
ASR-9006
4 (+2 RSP)
¼ rack
120/80
.960
240
1.92
ASR-9010
8 (+2 RSP)
½ rack
120/80
1.92
240
3.84
Edge and Aggregation
~3500 systems
~500 customers
Maximum BW / Protected BW with a failed fabric element
Full Rack Box
20 (+2 RSP)
1 rack
240
9.6
360
Pizza Box
2 sub-slots
2 RU
-
.120
Chassis
2009
2012
Future
© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 30
Fixed Ethernet LCs:
Line Rate: 40xGE, 2x10GE+20xGE, 4x10GE, 8x10GE
Oversubscribed: 8x10GE (60G), 16x10GE (90G/120G)
Ingress/Egress H-QoS, Netflow, IPoDWDM (G.709, FEC, XFP),
Video monitoring, SyncE, E-OAM
L2 Scalability: 1MMACs, 8kBDs, 32kPWs
L3 Scalability:1M routes, 4kVRFs, 4kL3intfs
3 LC versions (16x10GE OS “Medium Queue” only):
+licenses: L3VPN (/LC), G.709 (/LC), vidmon (/chassis)
Modular LCs: SIP-700 + max 4x SPA
QFP based
20Gbps Full Duplex
Ph1: ChOC12
Line Card EFPsEgress
QueuesPolicers Buffering
Low Queue -L 4k 8/port 8k 50ms
Medium Queue -B 16k 64k 128k 50ms
High Queue -E 32k 256k 256k 150ms
ASR 9000 #10GE LR #10GE OS
6-slot 32 64
10-slot 64 128
© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 31
...and keep a reasonable quality?
1) Compact Anatomy
RSP, Route/Switch Processor (instead of RP and FC)
Ethernet-oriented Linecard (non-modular, less memory)
2) Linecard Architecture
Multiple smaller NP’s (eg. 4x 10G instead of 1x 40G)
One NP is shared for Rx and Tx (not dedicated NP’s per Rx and Tx)
Multiple smaller Fabric Ports (eg. 2x 20G instead of 1x 40G)
3) Special Core-facing Linecards
8/16 queues per port (instead of thousands)
lower-scale NP (no need for thousands of interfaces)
licenses for features that not everybody uses (eg. VPN, OTN, scale)
4) Oversubscribed Cards
2:1 ingress overbooking (eg. PON OLT Aggregation)
Cisco Public 32© 2010 Cisco and/or its affiliates. All rights reserved.
Architecture Matters!
IP/MPLS
NPU
IP/MPLS
NPU
IP/MPLS
NPUIP/MPLS
NPU
Forwarding Architecture 101
Ingress
Linecards
Egress
Linecards
TX
RX
TX
TX
TX
RX RX RX
UNICAST
to slot 3
MULTICAST
to slots 2,3,4
1
2
3
4
1 2 3 4
Fabric Port
• addressable entity
• singleduplex pipe
What’s the capacity?
4 fab.ports @10Gbps
non-blocking
ENGINEERING:
4 * 10 = 40Gbps fdx
MARKETING:
4 * 10 *2 = 80Gbps
IP/MPLS
NPU
IP/MPLS
NPU
IP/MPLS
NPU
IP/MPLS
NPU
SWICTHING FABRIC
IP/MPLS unaware part
speaks cells/frames
NETWORK PROCESSOR
IP/MPLS aware part
speaks packets
CISCO INTERNAL
© 2010 Cisco and/or its affiliates. All rights reserved.
© 2011 Cisco and/or its affiliates. All rights reserved.
SMP Network Processor Example2010: CRS QFA (Quantum Flow Array)
BQS ASIC
- 64K queues
- 3L shaping
DRAM
Processing Pool
256 Engines
Resources & Memory Interconnect
headers only
Distribute & Gather Logic
SRAMRLDRAMDRAM TCAM4
PLU8/TLU5 Stats/Police ACL 125 Mpps, 140 Gbps
65nm, u-programmable
2 per LC (Rx, Tx)
© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 35
Bad Network Processor Example (non-Cisco)ACL performance impact
IP Performance Range
L3
L4
Vendor is Saving on Memory – ACL memory is shared with Route memory•Effect #1: ACL drastically impacts forwarding performance
•Effect #2: FIB cannot be hierarchical slow BGP convergence
© 2010 Cisco Systems, Inc. All rights reserved.BRKIPM-2012 36
Switching Fabric and Multicast Good vs. Bad IPTV Experience
TX
TX
RX
RX
RX
RX
Ingress
LinecardsGood:
Egress Replication
• Cisco CRS, 12000
• Cisco ASR9K, 7600
10Gbps of multicast
eats 10Gbps fabric bw!
Bad: Binary Ingress Replication
• dumb switch fabric
• non-Cisco
10Gbps of multicast
eats 80Gbps fabric bw!
(10G multicast impossible)
TX RX
Ingress
Linecards
TX RXTX RXTX RX
TX RXTX RXTX RXTX RX
RXRXRXRX
RXRXRXRX
161
7181
9202
1222
3242
5262
7293
0313
2
333
46
364
08
09
10
11
12
13
14
15
04
05
06
07
02
01
03
363
8404
2444
648
© 2010 Cisco Systems, Inc. All rights reserved.BRKIPM-2012 37
TX
ASR9000
• 80G eth. non-blocking with failed RSP
• cell dip is not an issue (super-frame based fabric, not cell based)
Active RSP80G 80G
TX RX
RX
CRS-1
• 40G eth. non-blocking with 1 or 2 failed fabric cards
TX RX
Good Fabric Redundancy
Active RSP
40G100G
40G
100G
X
X
CRS-1: 56G 49G 42G
CRS-3: 141G 123G 106G
X
failed RSP: 184G 92G
CRS-3
• 100G eth. non-blocking with 1 or 2 failed fabric cards
CRS-1: 112G 98G 84G
CRS-3: 226G 197G 169G
© 2010 Cisco Systems, Inc. All rights reserved.BRKIPM-2012 38
Non-Cisco
• 40G system sold as 100G
•if one CMP fails, system turns blocking
• unreliable 100GE
Bad Fabric Redundancy
TX
Active SFM
Active SFM
100G 100GX
failed SFM/CMP: 105G 52G
- huge cell dip at >40G
- blocking with 100GE
RX
Non-Cisco
• 70G system sold as 80G
•minimum speedup
• huge cell dip
• multicast impacts unicast
TX
Active SCB
80G 80G
TX RX
RXActive SCB
RX
RX
TX
TX
Active SCB
X
failed SCB: 126G 84G
- huge cell dip at >75G
- blocking with m’cast
x8
x8
© 2010 Cisco Systems, Inc. All rights reserved.BRKIPM-2012 39
Cell dip and SpeedupP
erc
en
tage
of
Lin
era
te
80%
60%
40%
20%
0%
100%
1N0%
40
10
0
20
0
30
0
40
0
50
0
60
0
70
0
80
0
90
0
10
00
11
00
12
00
13
00
14
00
15
00
Cell dip – per vendor @ 41B, 46B, 53B, 103B,...
The lower speedup, the more multicast, the more cell dips!
cell payload
[48B]
cell
hdr
[5B]
cell format
example:
Fixed overhead [cell header, ~10%]
Relative overhead [fabric header]
Variable overhead [padding]
40B IP
Packet:
IP Packet
[40B]
cell
hdr
[5B]
buffer
hdr
[8B]
Good efficiency1Mpps = 1Mcps
1Gb/s 1.33Gb/s
41B IP
Packet:
IP Packet
[first 40B]
cell
hdr
[5B]
buffer
hdr
[8B]
empty
[47B padding]
cell
hdr
[5B]
IP Packet [last 1B]
Poor efficiency1Mpps = 2Mcps
1Gb/s 2.6Gb/s
L3 Packet Size [B]
Cell Dip mitigation:1. Enough Speedup
2. Packet Packing (CRS)
3. Super-framing (ASR9K)
Another IP PacketPoor efficiency1Mpps = 2Mcps
1Gb/s 2.6Gb/s
© 2010 Cisco Systems, Inc. All rights reserved.BRKIPM-2012 40
Quality Differences in 40G Solutions40 ≠ 40
Good:
40 Gbps per slot
non-blocking
Ingress Linecard Egress Linecard
56G 112G
Cisco CRS-1 (MSC-40)
16 slots
[16x16 matrix]
40G
NPU
40G
NPU
40G 40GTX RX
Good-enough?:
40 Gbps per slot
non-blocking!
10G 18.8G 18.8G
18.8G 18.8G
18.8G 18.8G
18.8G 18.8G
Non-Cisco
12 slots
[48x48 matrix]
10G
10G
10G
10G
10G
10G
10G
10G
10G
10G
10G
10G
10G
10G
10G
TX
TX
TX
TX
RX
RX
RX
RX
26G 26G
Good-enough:
40 Gbps per slot
non-blocking
Ingress Linecard Egress Linecard
26G 26G
Non-Cisco
8 slots
[16x16 matrix]20G
ASIC
20G
ASIC
20G
ASIC
20G
ASIC
20G 20G
20G 20G
TX
TX
RX
RX
© 2010 Cisco Systems, Inc. All rights reserved.BRKIPM-2012 41
Good 100GE100GE non-blocking even with failed fabric card
1.4x 1.2x speedup
Cisco CRS-3
Ingress Linecard Egress Linecard
141G 225G140G NPU
Switching Fabric
4.48 Tbps
7+1 planes
140G NPU
TX RX
Bad 100GE
Non-Cisco
54G 54G
54G 54G
sub-interface 1
sub-interface 2
Ingress Linecard Egress Linecard
50G NPU
50GNPU
Switching Fabric
1.6 Tbps
4+1 SIB’s
50GNPUECMP/Link-Bundling
No 100G processor
available, per-destination
IP load-sharing across two
old 50G processors with
two old 54G fabric ports
50GNPU
sub-interfce 1
sub-interface 2
TX
TX
RX
RX
Impact of too many fabric connections“How to do 100GE?”
More than 54G Fabric Port gets overloaded!
Head of Line Blocking?
Port-Channel across 2 vlans on the same physical port
(to keep 1 IP and 1 MAC address per port)
no multi-vendor interoperability
© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 42
• Motivation for IP NGN = traffic growth
• How to make the Network cheaper
• How to make the Router cheaper
• Quality differences
There are Good, Good-enough or Bad Solutions.
Thank you.