trends in data rate and link length in evolving optical standards

Your Imagination, Our Innovation

Trends In Data Rate And Link Length In Evolving Optical Standards

David Cunningham22nd September 2013


Outline

• Building and Data Centre link lengths

• Trends for standards-based electrical interfaces

• Data rate, transmission media and optical technology trends

• Multimode fibre link lengths

• Form factor evolution

• Summary


Transmission Technologies For Data Centre Networking

• Electrical Links:Used for on-board, in chassis, and rack to rack.

• Optical Links:Used where electrical can’t work!

• As signaling speeds increase and the size of data centresexpand, optics plays an increasing role.


Standards Based Building Wiring Link LengthsCampus Distributor

Building Distributor

Building Distributor

Floor Distributor

Floor Distributor

Floor Distributor

Floor Distributor

Horizontal Outlet

Horizontal Outlet

Horizontal Outlet

1500 m

500 m

90 m

300 m

• Campus to campus link length 10 km

Fixed installed cable lengths per ISO/IEC 11801


Data Centres Now Use Structured Cabling Too

(>300 m)

From This Rack

• SMF solutions for 500 m under active consideration for higher speeds

• ISO/IEC 24764 Generic Cabling Systems for Data Centres

• But data centres are getting huge and supported MMF link lengths are decreasing at higher data rates

0

100

200

300

400

500

600

700

800

900

1000

0 5 10 15 20 25 30

Link

Len

gth, m

Data rate, Gb/s

Multimode Fibre Link Lengths Versus Data Rate

OM4

OM3

MMF Link Lengths Per Fibre Channel Standards


Trends For Standards-Based Electrical InterfacesAs the electrical data rate seems to be expected to increase with an exponential trend it will quickly get harder and harder to maintain electrical signal integrity.

Already, in order to maintain signal integrity, an evolution of various mitigation methods is obvious:

• Number of lanes: 16, 4, 1……10, 4, 1…..16, 8, 4 ,1…..

• Clock Recovery

• Equalization

• Error Correction

• Advanced Modulation Formats

Unfortunately these mitigations increase the required electrical power dissipation.

They also increase transmission latency.

0

10

20

30

40

50

60

70

80

90

100

110

120

2000 2005 2010 2015 2020 2025

Lane

Rate, Gb/s

Year Specification Technically Stable

Electrical Lane Data Rate

Equalisation Required

Multilevel Modulation Required

Error Correction Required

XAUI-4

XFISFI(16GFC)

XLAUI-4, CAUI-10, nPPI

100GBASE-KP4100GBASE-KR4

SFI(128GFC)

CDAUI-4

CEI-28-VSR

CAUI-4CDAUI-16

XLAUI

CDAUI-8

SBI-16

Label Colour Code Black: 1 Lane Blue: 4 Lanes Purple: 8 Lanes Brown: 10 Lanes Red: 16 Lanes

CEI-56-VSRSFI(32GFC)

• Development of Electrical Interfaces For 50-56 Gb/s per lane via NRZ/PAM4 and FEC has already started.


Basic Optical Technologies For Data Centres850nm VCSELs (Vertical- Cavity

Surface-Emitting Lasers)

core Cladding

OM3 and OM4 50-micron core fibre is most common today. Optimized for transmission using 850-nm VCSELs. Easy to maintain, low-cost connectors, low cost transceivers

Multimode Fibre (MMF)

Low-cost laser source Low Current & Low Power Compatible with MMF Enable low-cost, low power transceivers

1310nm edge-emitting lasers Single mode Fibre (SMF)

SMF generally used for >300 links using 1310nm sources Long reach capable, low cost/meter More expensive to connectorize than MMF

Higher cost laser source High Current & Power Used predominantly with SMF Various wavelengths for WDM applications Typically more expensive to package Used to power Si and InP Photonics


Fibre Channel Data Rate & Encoding Roadmap

1995 2000 2005 2010 2015 2020

1 Gb/s

10 Gb/s

100 Gb/s

Year

Data Rate

2G

4G

8G

16G

32G

64G

128G

4x32G (Parallel Multimode Fibres)

8B10B

8B10B

8B10B

8B10B

64B66B

256B/257B (FEC Code)

• Laser modulation rate not keeping pace with desired data rate.

• Available optical link budget decreases as data rate increases.

• More efficient encoding and FEC enabled 16GFC, 32GFC and 4x32GFC.

SFP+ (1.5W)

Fibre Channel Motto: “Keep it serial, keep it simple”

QSFP+ /QSFP28(3.5W)

SMF & MMF

NRZ/PAM4 (FEC Code)?

PAM4/Advanced Modulation (FEC Code)?


Ethernet Data Rate & Transmission Media Evolution

1990 1995 2000 2005 2010 2015 2020

400G

100 G

10 G

1G

100 M

10M

Year

Fast Ethernet

Gigabit Ethernet

10Gigabit Ethernet

100Gigabit Ethernet

Copper Twisted Pair (Cat 5) Multimode Fibre (OM1,OM2)

Copper Twisted Pair (Cat 5) Multimode Fibre (OM1, OM2)

Singlemode Fibre & WDM (4 Lanes) Multimode Fibre (OM3) Copper Twisted Pair (Cat 6, 4 Lanes)

Singlemode Fibre & WDM (4 Lanes) Parallel Multimode Fibre(OM4, 4 &10 Lanes)

40Gigabit Ethernet

400Gigabit Ethernet

Parallel Singlemode Fibre (4) Parallel Multimode Fibre (OM4, 16 Lanes) New MMF & CWDM

Data Rate


Ethernet Data Rate & Optical Device Technology Evolution

Year

Fast Ethernet

Gigabit Ethernet

10Gigabit Ethernet

100Gigabit Ethernet

Light emitting diode (LED)

Lasers(VCSEL & FP)

Lasers(VCSEL, FP and DFB)

Lasers & Laser Arrays(VCSEL, DFB’s, Modulators, Optical Amplifiers, Si and InP Photonic Integration)

1990 1995 2000 2005 2010 2015 2020

400G

100 G

10 G

1G

100 M

10M

400Gigabit Ethernet

Data Rate

CFP-based 100GBASE-LR4

SFP+


Trend Of Multimode Fibre Link Lengths

0

100

200

300

400

500

600

700

800

900

1000

0 5 10 15 20 25 30

Link

Len

gth, m

Data rate, Gb/s

Multimode Fibre Link Lengths Versus Data Rate

OM4

OM3

MMF Link Lengths Per Fibre Channel Standards

• The historical trend for multimode fibre link lengths is shown in the graph.

• Methods are under discussion to ensure at least 100 m operation for higher data rates.

Methods under discussion include:

• VCSEL’s operating at longer wavelengths than 850 nm: increases VCSEL bandwidth and decreases mode partition noise

• Multilevel modulation, which is enabled by the emergence of PAM based electrical specifications and FEC: even at 850 nm PAM4 would enable 100 m links for 64GFC

• VCSEL-based CWDM, wavelengths TBD

The need to reduce the lane count for SMF optics and the lack of bandwidth of the SMF optical components means that multilevel modulation with FEC is also being considered for next generation SMF links.


100GBASE-SR4 Parallel MMF (4x 25.78125 Gb/s)

• At 25.8 Gb/s MMF-based optics is bandwidth and noise challenged

• Therefore, 64b66b will be transcoded into 256b257b to provide FEC

• The Reed Solomon Code (RS (528,514)) provides about 3 dBo of coding gain

• The FEC reduces the optical power budget and ensures signal integrity

• Can expect that 100GBASE-SR4 will use QSFP+

• Board mounted 4x25.8 Gb/s optics will also be available for high density applications

HOST ASIC 4 Lane VCSEL Driver

4 Lane TIA-Amp’s

• Parallel MMF

• 100 m OM4

• 70 m OM3

CAUI-4 Repeater Interface

CAUI-4 Interface CAUI-4

Coupling optics

Coupling optics

256b257b FEC

encoding4 PIN Array

VCSEL Array


Pluggable Form Factor Choices For 10G to 100G Applications

Which to choose?

It depends upon the application.

Industry Pluggable Form factor (with Max power)

SFP+ (1.5W)

QSFP+ /QSFP28(3.5W)

CXP (3.5W)

CFP / CFP2 / CFP4(32W / 12W / 6W)

Lane

Sig

nalin

g Ra

te

~10G8GFC,10GbE Duplex MMF &

SMF

4x10GbE / 1x40GbEQDR-IB

Parallel MMF & SMF

12x10GbE /100G SR10QDR-IB

Parallel MMF only

CFP:100G MMF/SMFCFP2: 100G SR10

~14G16GFC

Duplex MMF & SMF

16GFCFDR-IB

MMF & SMF

FDR-IBProprietary inter.

Parallel MMF onlyNA

~25G32GFC

Duplex MMF & SMF

100GbE / EDR-IB128GFC

MMF & SMF

EDR-IBProprietary inter.

MMF only

CFP2: 100G SR4/LR4 CFP4 :100G SR4/LR4

MMF & SMF

It is expected that a new form factor “CDCFP” will emerge for 400Gigabit Ethernet.


Pluggable Embedded Parallel Optics & CXP Modules For High Density 100GbE & Infiniband

For MMF applications board mounted pluggable optics and front panel pluggable CXP modules with electrical nPPI interfaces offer higher density than front panel pluggable CFP# modules.


Summary• Standards-based electrical and optical lanes operating at 20 – 28 Gb/s are

almost specified and are getting ready for products.• The next generation of standards-based electrical and optical lane rates will be

around 50-56 Gb/s.• To support 50-56 Gb/s and higher lane rates various methods will be reused or

developed:• More lanes (via parallel conductors, fibres or wavelengths), Multilevel modulation,

FEC and integrated photonics.• For multimode fibre applications new, higher bandwidth, longer wavelength,

VCSEL’s are likely to emerge along with a new MMF optimised for CWDM operation at longer wavelengths. At a minimum these developments will maintain the MMF worst case link distance of 100 m.

• Whilst 100GbE implementations continue to reduce their lane count, power, size and cost, early implementations of 400GbE may start with 16 lanes of 25.8 Gb/s per lane in large packages and the cycle will continue again.


Backup Slides


Trends For Standards-Based Electrical Interfaces

The future trend, in blue, is based on the author’s interpretation of publicly available roadmaps or predictions per IEEE 802.3, Fibre Channel and the OIF websites.

A graph might show the trend better than a table…… queue next slide.

Interface Name Year Per Lane Rate (Gb/s) Number of Lanes Modulation (Encoding) FEC Equalization RequiredSBI 2000 0.645 16 NRZ (64B66B) None NoXAUI 2002 3.125 4 NRZ (8B10B) None NoXFI 2005 10.3125 1 NRZ (64B66B) None NoSFI 2009 10.3125 1 NRZ (64B66B) None No

SFI 16GFC 2009 14.025 1 NRZ (64B66B) None No10GBASE‐KR 2010 10.313 1 NRZ (64B66B) Optional YesXLAUI‐4 2011 10.3125 4 NRZ (64B66B) None NoCAUI‐10 2011 10.3125 10 NRZ (64B66B) None No

CEI‐28‐VSR 2014 28.050 As required by application NRZ (per application) None YesSFI 32GFC 2013 28.050 1 NRZ (256B/257B) RS(528,514) Yes

100GBASE‐KR4 2013 25.7813 4 NRZ (256B/257B) RS(528,514) Yes100GBASE‐KP4 2013 26.5625 4 PAM4 RS(544,514) Yes

128GFC (4x32GFC) 2014 28.0500 4 NRZ (256B/257B) RS(528,514) YesCAUI‐4 2015 25.7813 4 NRZ (256B/257B) RS(528,514) Yes

CDAUI‐16 2017 25.7813 16 NRZ (256B/257B) RS(528,514) YesCEI‐56‐VSR 2016 56.1 As required by application PAM4 ? YesSFI 64GFC 2016 56.1 1 PAM4 ? Yes (TBD) Yes

256GFC (4x64) 2017 56.1 4 PAM4 ? Yes (TBD) YesCDAUI‐8 2018 51.5625 8 PAM4 ? Yes (TBD) YesXLAU 2018 41.25 1 PAM4 ? Yes (TBD) Yes

SFI 128GFC 2020 112.2 1 Complex Modulation ? Yes (TBD) YesCDAUI‐4 2022 103.125 4 Complex Modulation ? Yes (TBD) Yes


CFP2 100GBASE-SR10

• 100 m on OM3

• 150 m on OM4


Pluggable AOCsPluggable transceivers

Board mounted optics

Optical Form Factor Options

Board mounted optics offer density advantages.

Pluggable optics offer flexibility.


Slide taken from: http://www.ieee802.org/3/400GSG/public/13_07/maki_400_01_0713.pdf

trends in data rate and link length in evolving optical standards

Documents