© 2010 Cisco Systems, Inc. All rights reserved. Cisco Confidential100G LACNOG 2010
Igor Giangrossi
Consulting Systems Engineer
100G Deployment Considerations
© 2010 Cisco Systems, Inc. All rights reserved. Cisco Confidential100G LACNOG 2010 2
Agenda
The need for 100GE
Standards Work Status
100G and Optical Transport Systems
100GE Router Implementations
© 2010 Cisco Systems, Inc. All rights reserved. Cisco Confidential100G LACNOG 2010 33
0
10
20
30
40
50
60
70
2009 2010 2011 2012 2013 2014
Ex
ab
yte
s/m
o
Business
Consumer
34% CAGR 2009–2014
87%
13%
Global IP Traffic Growth
IP Traffic will increase 4.3x from 2009 to 2014
Source: Cisco Visual Networking Index (VNI) Global Forecast, 2009–2014
© 2010 Cisco Systems, Inc. All rights reserved. Cisco Confidential100G LACNOG 2010 44
Why Even Higher Speed Interfaces?
Current Interfaces:
10Gbps: Ethernet
40Gbps: SONET/SDH
Ethernet Link Aggregation
Max 8 links (Std)
Unequal Load Balance
Limits flow BW
Harder to manage
Inefficient fiber use
© 2010 Cisco Systems, Inc. All rights reserved. Cisco Confidential100G LACNOG 2010 5
Different SDOs for 100Gb/s Interfaces
Defined Ethernet MAC and PHY
specifications
Defining 100Gb/s signals over 50GHz
optical channels on DWDM systems
(modulation, etc)
Defining how to map 100Gb/s signals
over OTN/G.709 encapsulation
© 2010 Cisco Systems, Inc. All rights reserved. Cisco Confidential100G LACNOG 2010 66
IEEE 802.3ba Standard
Ratified on IEEE July 2010 Meeting
40 and 100 Gb/s Ethernet MAC and PHY
Cooper and fiber connectivity
© 2010 Cisco Systems, Inc. All rights reserved. Cisco Confidential100G LACNOG 2010 7
Why 40G and 100G Ethernet?Addressing Needs of Core and Access Network
2000 2005 2010 2015 20201995
100
1000
10 000
100 000
1 000 000 Core Networking
Doubling ≈ 18 months
Server I/O
Doubling ≈ 24 months
Gigabit Ethernet
10G Ethernet
100G Ethernet
40G Ethernet
100G forecast to be needed in the core in 2010
Server I/O will not require 100G for several years
- Will require 40G much earlier
- Growth follows Moore’s Law
Bandw
idth
(M
b/s
)
© 2010 Cisco Systems, Inc. All rights reserved. Cisco Confidential100G LACNOG 2010 10
IEEE 803.2ba Summary
40 GbE 100GbE Comments
1 m Backplane 40GBASE-KR4
7 m Copper 40GBASE-CR4 100GBASE-CR4 4/10 cooper @ 10Gb/s
100 m MM Fiber 40GBASE-SR4 100GBASE-SR10 4/10 fibers @ 10Gb/s
10 km SM Fiber 40GBASE-LR4 100GBASE-LR4 CWDM (4λ)
40 km SM Fiber 100GBASE-ER4 CWDM (4λ)
Additional Info:
- 802.3 Frame format is maintained
- 64B/66B Encoding: 40GBASE-R (40Gb/s) and 100GBASE-R (100Gb/s)
- Cooper and backplane PHYs have an optional FEC layer
© 2010 Cisco Systems, Inc. All rights reserved. Cisco Confidential100G LACNOG 2010 11
ITU Work on 40/100G (SG15 – OTN)
Wide Area Network
OTU1
(2.7Gb/s)
OTU2
(10.7Gb/s)
OTU3
(43Gb/s)
G.709: OTN Network Node Interface (NNI)
• Client interfaces specified for SDH services (STM-16, STM-64 and STM-256)
• OTU3 supports STM-256; proposed for 40GbE
• OTU4 (>100Gb/s) proposed for 100GbE
OTU4
(100Gb/s)
OTU4
© 2010 Cisco Systems, Inc. All rights reserved. Cisco Confidential100G LACNOG 2010 1212
Optical Characteristics for 100G/40G
100G vs 10G 100G vs 40G
OSNR Requirement 10 dB higher 4 dB higher
CD Tolerance 100 X less 6.25 X less
DGD Tolerance 10 X less 2.5 X less
PMD Limited
Distance100 X less 6.25 X less
Optical Bandwidth 10 X 2.5 X
Implementing 100G and 40G is much harder than
10G in the Optical Domain
© 2010 Cisco Systems, Inc. All rights reserved. Cisco Confidential100G LACNOG 2010
100G Optical Systems
Transmitter: Increasing speed means
Complex Optics = Complex Electronics = $$$$$$
More Optical Impairments
Receiver: Address impairments
Optical compensation vs. Electrical compensation
Coherent vs. Direct detection
Forward Error Correction (FEC)
Hard Decision FEC
Standard FEC – 6dB Coding Gain
Enhanced FEC – 8+dB Coding Gain
Newer FECs – 9+dB Coding Gains
Soft Decision FEC
Each of the above FECs can be used for any of the options
© 2010 Cisco Systems, Inc. All rights reserved. Cisco Confidential100G LACNOG 2010 1616
100G Work at the OIF
Several Projects for 100G DWDM Systems
100G Long Distance DWDM Transmission Framework
100G Long Distance DWDM Integrated Photonics Receiver
100G Long Distance DWDM Integrated Photonics Transmitter
Forward Error Correction (FEC) for 100G DP-QPSK Long Distance Communication
100G Long-Haul DWDM Transmission Module -Electromechanical
100G Long-Haul DWDM Transmission Module – MDI
Agreed on PM-QPSK* as the modulation scheme, using coherent receivers
* Polarization Multiplexing – Quadrature Phase Shift Keying
© 2010 Cisco Systems, Inc. All rights reserved. Cisco Confidential100G LACNOG 2010
100G – The Transmitter
Need to go slower
Optical impairments are directly related to signaling rates
Need to increase modulation efficiency
Signaling speed decreases & Information Rate increases
NRZ to ODB to (D)PSK to (D)QPSK
Need to increase optical efficiency
Split signal over two polarizations (PM – Mod Scheme)
1 bit/symbol
NRZ
0 1
1 bit/symbol
PSK
1-1
2 bits/symbol
QPSK
00
1011
01
© 2010 Cisco Systems, Inc. All rights reserved. Cisco Confidential100G LACNOG 2010
6 9 12 15 18 21 24 27 300.5
1
2
4
8
16
32
OSNR over 0.1 nm [dB]
Spectr
al effic
iency [
bit/s
/Hz]
PM-QPSK
PM-8QAM
PM-16QAM
PM-64QAM
PM-256QAM
PM-1024QAMShannon Bound
for PM formats
Requires extra 4 dB to
double spectral efficiency
Requires extra 8 dB to
triple spectral efficiency
OSNR Required at 111 Gb/s
Higher level modula-tion formats require increasingly higher OSNR even at constant bit rate
PM-16QAM requires 4 dB more than PM-QPSK
PM-64QAM requires 8 dB more than PM-QPSK
PM-256QAM would require 13 dB more than PM-QPSK!
BER=1E-3
© 2010 Cisco Systems, Inc. All rights reserved. Cisco Confidential100G LACNOG 2010 21
100GE Interfaces on a Router
CFPM
A
C
FWD
Engine
Fabric
IF
CFPM
A
C
FWD
Engine
Fabric
IF
CFPM
A
C
FWD
Engine
Fabric
IF
Linecard 1 Linecard 2Fabric
Not every router will be able to implement
100GE…
MM
M M
MM
M M
MM
M M
© 2010 Cisco Systems, Inc. All rights reserved. Cisco Confidential100G LACNOG 2010 2222
Some Challenges for a 100GE Linecard
Substantially larger and power hungry pluggable
Many current MAC Chip on FPGAs: more space, power
Forwarding Engine: 100GE != 2x50GE
Higher PPS, faster memory lookups, faster interfaces
Packet Buffering @ 100Gbps
Circuit Board challenges:
Real Estate – difficult to fit everything
Circuit routing – several layers
Xtalk between lanes @ high speed SerDes interfaces
Power dissipation
© 2010 Cisco Systems, Inc. All rights reserved. Cisco Confidential100G LACNOG 2010 23
Pluggable Comparison: CFP vs XFP
XFP CFP
Length 78 cm 145 cm
Width 18 cm 77 cm
Height 8.5 cm 13.6 cm
Power
Consumption3.5W 32W
© 2010 Cisco Systems, Inc. All rights reserved. Cisco Confidential100G LACNOG 2010 2525
Key Takeaways
100Gb/s Ethernet is already a reality
Implementing 100Gb/s links poses many challenges to many currently deployed systems
Optical costs become much more relevant at 100Gb/s links
Careful planning is needed to take full advantage of the bandwidth gains
© 2010 Cisco Systems, Inc. All rights reserved. Cisco Confidential100G LACNOG 2010 2626