Optical Trends in the Data Center

Doug ColemanManager, Technology & Standardsg , gy

Distinguished AssociateCorning Cable Systems

Data Center Environment

Data Center EnvironmentData Center Environment

• Higher speeds• Higher speeds• Higher density• Higher

reliability• Lower capex • Lower opexLower opex• Green

VCSELs Drives MMF Value Proposition

• 850 nm VCSEL– Highly-efficient

manufacturing process (>100,000/wafer)

– Ease of packaging into transceiver TOSA

• Lowest Transceiver Price• Lowest Transceiver Price– 10G Serial x Serial

• 3:1 ($SM/$MM)Vertical Cavity Surface Emitting Laser

– 40G Serial x Parallel• 5:1 ($SM/$MM)

– 100G Serial x Parallel100G Serial x Parallel• 20:1 ($SM/$MM)

Data Center Multimode Fiber TypesData Center Multimode Fiber TypesMultimode Fiber Type

Wavelength (nm)

OFL BW(MHz km)

EMBMHz km)Fiber Type (nm) (MHz-km) MHz-km)

62.5/125 um(OM1)

8501300

200500

NA(OM1) 1300 500

50/125 um(OM2)

8501300

500500

NA

50/125 um(OM3)

8501300

1500500

2000

50/125 um(OM4)

8501300

3500*500

4700

*OFL BW utilized by fiber manufacturers who rely on DMD masks to validate the 4700 MHz-km EMB

Bend-Optimized Multimode FiberStandards Compliant / Reliabilityp / y

OM4 OM3 OM2OM4 OM3 OM2

Standards Compliancep

IEC 60793-2 10

Type A1a.3 Fib

Type A1a.2 Fib

Type A1a.1 Fib2-10

TIA/EIAFiber

492AAADFiber

492AAAC-AFiber

492AAAB

No interoperability issues with standards-compliant fibers!!

Data Center Environment

80%

60%

70%

OM4

30%

40%

50% OM4

OM3

OM2

OM1

10%

20%

% OM1

SMF

0%2004 2005 2006 2007 2008 2009 2010 2011

Source: Corning Cable Systems

Data Center Multimode Cable Length Distribution

Trunk LengthgProduct Manufactured 2009-2011

50% 1 Trunk, 40% 2nd Trunk, 10% 3rd Trunk

1800

2000

90%

100%

100 88%

1000

1200

1400

1600

1800

quen

cy

50%

60%

70%

80%

90%100m: 88%

0

200

400

600

800Freq

0%

10%

20%

30%

40%

Average = 54.2 m

0

10 20 30 40 50 60 70 80 90 100

110

120

130

140

150

160

170

180

190

200

210

220

230

240

250

More

Length (m)

0%

Frequency Cumulative %

Source: Corning Cable Systems

Data Center Single-mode Cable Length Distribution

Trunk Length, Single-modeProduct Manufactured January 2009-to-January 2012y y

50% 1st Trunk, 40% 2nd Trunk, 10% 3rd Trunk

3000

350090%100%

70m: 72%

1500

2000

2500

3000

requ

ency

50%60%70%80%

Average = 52.3 m

100m: 88%

Average = 54.2 mStd Dev. = 49.2

0

500

1000

1500

Fr

0%10%20%30%40%

0

10 30 50 70 90 110

130

150

170

190

210

230

250

270

290

310

330

350

370

390

Mor

e

Length (meters)

0%

Frequency Cumulative %

Source: Corning Cable Systems

Frequency Cumulative %

Standard Specified Distances

850 nm Ethernet Distance (m)

1G 10G 40G 100GOM3 1100 300 100 100OM4 1100 400 / 550* 150 150OM4 1100 400 / 550* 150 150

*Engineered Length

850 nm Fibre Channel Distance (m)4G 8G 16G

OM3 380 150 100OM4 480 190 125

OM3/OM4 Connectivity plus 850 nm VCSEL TransceiversTransceivers

VCSEL manufacturers are planning 25G VCSELs to support: 32G Fibre Channel

– 100GbE– EDR InfiniBand

VCSELs are field-proven, reliable, low-cost, low-power solutions for data center interconnectsdata center interconnects

Millions of VCSEL-based channels are shipped per monthMillions of VCSEL based channels are shipped per month

Synergistic MMF / VCSEL relationship provides lowest-cost solution in the data center

Migration to OM4

Finisar 28G modeling presented at the T11.2 Fibre Channel meeting (06/2010)

TIA-942A recommends OM4

Expect Fibre Channel and Ethernet to utilizeand Ethernet to utilize OM4 to define distance objectives

Transmission and Cable Standards a s ss o a d Cab e Sta da dsrecommend OM4

Migration to OM4

Do we need OM5

Do We Need OM5?

Chromatic Dispersion Limited

Source: VI Systems

Legacy transceiver technology hasminimal value with increased EMB

The Need for Speed: 10/40/100G

Server Virtualization Drives Higher Data Rates

• Multiple applications running in parallel on one server (e g 20-100 apps/server)one server. (e.g.., 20 100 apps/server)

• 10 to 50 servers consolidated • Increases utilization efficiency to =< 90%

l ( )• Multi-core processors (4,8,16..50)• PCIe2: 8 lanes @ 5G• PCIe3: 16 lanes @ 8G • Increased memory• Less connectivity and reduced electronic

ports p– Drive utilization of high-bandwidth

optical connectivity to mitigate system bottlenecks and support required pp qincreased I/O speeds

Server Virtualization Drives Higher Data Rates

Source: Discerning Analytics. 01/2012

40G / 100G Ethernet40G / 100G Ethernet

Source: Zarlink

10G/40G /100G Ethernet Switch Line Card Density10G/40G /100G Ethernet Switch Line Card Density

96 fibers OM3/OM4528 fibers OM3/OM48 fibers SMF620 fibers OM3/OM4620 fibers OM3/OM4

Source: Zarlink

Legacy Data Center Collapsed Architecture: EDGE, Aggregation and Core Switch Consolidation into MDAAggregation and Core Switch Consolidation into MDA

Data Center Flat Architecture: Top of Rack EDGE SwitchSwitch

Source: FCI

• 4 QSFP+ 40G ports Q p• 48 SFP+ ports – 10G or 1G• 48 server connections

Source: Blade Networks Technologies

Data Center Trend: Flat Architecture

• Two Tier Hierarchy ArchitectureCore to Edge to Server– Core to Edge to Server

– Non-blocking Core to Edge– Over-subscription Edge to Server

• Facilitates “east-west” layer-two switch to server traffic flow – Low network latency– Ease VM movement

• Transparent Interconnection of lots of links (TRILL) management increases overall ( ) gnetwork bandwidth and availability – Spanning tree replaced

Source: Fulcrum Microsystems

Data Center Trend: Server Density Increasing/Rack

40/50U+ Server RackRacks going taller to fit more gear in available vertical spacegear in available vertical space

Server technology states the 50U-rack has become the most common for their new data center builds

Higher-density top-of-rack switch/patch panel

Source: Rack Solutions

Data Center Trend: Optical Tap

Network security andNetwork security and performance

Monitoringoperational (opex) savings throughimproved operationsimproved operations

Better tracking and dh t SLAadherence to SLAs

enables move to private cloud infrastructures

Data Center Trends: Electronics and Connectivity

Optical Electronics and Connectivity Focusp y

Low Cost Low Power High Density g y

Increase data rates, transceiver size reduction, server consolidation/virtualization, multi-core processors, line card density, embedded optics, cloud computing, increased i t t d itinterconnect density

10G Ethernet

10G Optical Connectivity Value Proposition: Lower the Total Cost of Network OwnershipLower the Total Cost of Network Ownership

• Fiber optic cabling is less expensive to operate

10 Gb/s Operating CostFiber vs. Copper

Fiber Copper

• Less power consumption

Power Consumption

Cooling Requirements

~1-4 W ~4-5W

• Decreased cooling requirements

g q

Transceiver Size

• Smaller transceiver sizeSome estimates blame up to

60 t f ll d t t

Data Center Area$ $$$$

• Higher electronic port densities

60 percent of all data center downtime on heat-related issues

-- IDC

Benefits of 10G Optical Connectivity vs. Copper 10GBASE-T Electronics Issues10GBASE T Electronics Issues

• Significant switch power irequirements

• 10G copper 4 to 5 wattsper port (40 nm)

• Major silicon chip (28 nm) development required to reduce power

– Yield and $$$ issues• 10G optical switches 1 to 4 watts per

port– Typical SFP+ 0.5 watts

• 10G copper Latency – 2us/PHY

10GBASE-T Card

Source: Fulcrum Microsystems2us/PHY

Optical Connectivity Green Value Proposition: Low Electronics and Cooling Energy ConsumptionLow Electronics and Cooling Energy Consumption

OPTICAL CONNECTIVITY Enables The Green Data Center!

Optical Connectivity Green Value Proposition: Low CO2 EmissionsLow CO2 Emissions

OPTICAL CONNECTIVITY Enables The Green Data Center!

Fibre Channel

Data Center Fabrics

• Ethernet – LAN (1/10/40/100G)

OM3/OM4 fib– OM3/OM4 fiber– Non-deterministic

• Fibre ChannelFibre Channel– SAN (4/8/16G)– OM3/OM4 fiber/– Deterministic

Source: Info-Advantage

Fibre Channel Speed Roadmapp p

T11 Spec Market

1GFC 200 1.0625 1996

G C

ProductNaming

Throughput(MBps)

Line Rate(GBaud)

TechnicallyCompleted

(Year)‡

MarketAvailability

(Year)‡

1997

2GFC 400 2.125 2000

4GFC 800 4.25 2003

8GFC 1600 8.5 2006

2001

2005

2008

16GFC 3200 14.025 2009 2011

32GFC 6400 28.05 2012

64GFC 12800 TBD 2015

2014

Market Demand

128GFC 25600 TBD 2018 Market Demand

256GFC 12800 TBD 2021

512GFC 25600 TBD 2024

Market Demand

Market Demand

Fibre Channel4/8/16G Variants: OM2, OM3, OM4

FC-0 400-M5-SN-I 800-M5-SN-S 1600-M5-SN-SMultimode Cable Plant for OM2 Limiting Variants

Data Rate (MB/s) 400 800 1600

Operating Range (m) 0.5-150 0.5-50 0.5-35

Loss Budget (dB) 2.06 1.68 1.63

FC-0 400-M5E-SN-I 800-M5E-SN-I 1600-M5E-SN-I

Data Rate (MB/s) 400 800 1600

Multimode Cable Plant for OM3 Limiting Variants

Data Rate (MB/s) 400 800 1600

Operating Range (m) 0.5-380 0.5-150 0.5-100

Loss Budget (dB) 2.88 2.04 1.86M lti d C bl Pl t f OM4 Li iti V i tFC-0 400-M5F-SN-I 800-M5F-SN-I 1600-M5F-SN-I

Data Rate (MB/s) 400 800 1600

O i ( ) 0 00 0 90 0 2

Multimode Cable Plant for OM4 Limiting Variants

Operating Range (m) 0.5-400 0.5-190 0.5-125

Loss Budget (dB) 2.95 2.19 1.95

Fibre Channel8G and 16G Limiting Variants: OM2 OM3 OM48G and 16G Limiting Variants: OM2, OM3, OM4

Fibre ChannelFC-PI6 32G

• Fibre Channel, March 2010– 32G activity starting– Expected completion:

20132013– Commercial products:

2014

Brocade SAN Director

• Approved Objectives– OM3/OM4 70 m-100 m– SMF 10 km

SFP+ f f t

SFP+

70 m OM3– SFP+ form factor 70 m OM3

100 m OM4

Ethernet 40/100G

• IEEE 802.3 approved motions • 40 and 100 Gbps • At least 100 m on OM3 multimode fiber• At least 150 m on OM4 multimode fiber• At least 10 km on single-mode fiber• At least 40 km on single-mode fiber (100G only)• At least 7 m on copper cable assembly• At least 2 km on single-mode fiber (40G only)g ( y)

• Key project dates• Study group formed in July 2006Study group formed in July 2006• Project authorization in December 2007• Task force formed in January 2008• 40/100G standard completed June 201040/100G standard completed June 2010

40G Ethernet Parallel Optics: OM3/OM440G Ethernet Parallel Optics: OM3/OM4

12F MPO Connector Interface

QSFP TransceiverSource: Avago

Ethernet 40G and 100G: OM3, OM4

OM3 and OM4 distances contingent upon 1.5 and 1.0 total

connector loss, respectively

40G eSR4 QSFP+40G eSR4 QSFP

• QSFP+ eSR4 modules meets link distance specifications for 40G Ethernet applicationsEthernet applications

• 40G eSR4 parallel optics extended reachp p• OM3 / OM4: 300 m / 400 m • Corning Cable Systems modeling shows 12% data center

lengths >100 mlengths >100 m

• Expected availability: Mid 2012

40G Optical Transceiver: OM3/OM4p

QSFP transceiver technologyQSFP transceiver technology• Standard 12F MPO connector• =< 1.0 watts per port• Now used for 40G InfiniBand• Now used for 40G InfiniBand

Source: Zarlink

100G Ethernet Parallel Optics: OM3/OM4100G Ethernet Parallel Optics: OM3/OM4

24F MPO Connector InterfaceConnector Interface

Source: USConec

Source: Molex

100G Optical Connectivity: OM3/OM4

• 8 cards per chassis• 8 cards per chassis• 16 ports per card• 384 fibers per card• 3072 fibers per

chassis

• CXP Transceiver• 24F MPO Pinless

Connector• Key-up interface• Key-up interface

40/100G Data Center Architecture: Top of Rack EDGE SwitchTop of Rack EDGE Switch

40G: 2 x 24-fiber OM3/OM4 UplinkOM3/OM4 Uplink

100G: 2 x 48-fiber OM3/OM4 Uplink

40/100G Optical Transceiver: SMF40/100G Optical Transceiver: SMF

CFP T i t h lCFP Transceiver technology• Standard duplex LC

connectors Source: CFP MSAconnectors• =< 20 watts per port• 3 to 4 ports per cardp p• Large footprint

• Equivalent to two 10G XENPAKs

• Development in processCFP MSA

Duplex LC Connector• CFP MSA

40/100G: Twinax Copper/ pp

• Traditionally, has been used for short-length InfiniBand connectivity and must beconnectivity and must be factory-connectorized

• No guidance is included in the standard for CAT UTP/STP copper cable QSFP Direct-Attached Twinax Cablecopper cable

Source: FCI

40GBASE-T NOT EXPECTEDCopper Cable Manufacturers Promoting Interest

IEEE Next Generation 40/100 Gb/s Optical EthernetIEEE Next Generation 40/100 Gb/s Optical Ethernet

• Study Group Approved July 2010– Multimode parallel optics (100G)p p ( )

• 4x25 Gb/s expected• OM3/OM4: 100/150 m• QSFP28 12F MPO

– Single-mode (100G)• WDM, Parallel, PAM

– Single-mode (40G)l• Serial, WDM

– Completion 4th quarter 2014

• Future Activity• Future Activity– 16x25 Gb/s: 400G

• Multimode parallel optics– USConec 32F MPO 2012USConec 32F MPO 2012– Completion 2015+

IEEE NG 40/100G March PlenaryIEEE NG 40/100G March Plenary

Approved objectives for multimode and single-mode fibersApproved objectives for multimode and single-mode fibers– Define a 100 Gb/s PHY for operation up to at least 100 m of MMF

• 4x25G parallel optics, QSFP+/– Define a 100 Gb/s PHY for operation up to at least 20 m of MMF

• 4x25G parallel optics, QSFP+– Define a 100Gb/s PHY for operation up to at least 500 m of SMF p p

(mega data centers)• Task Force to define

– Define a 40 Gb/s PHY for operation over at least 40 km of SMFDefine a 40 Gb/s PHY for operation over at least 40 km of SMF• Task Force to define

Multimode NG 100G Activityy

FROM:TO:TO:

Source: Brocade

Mega Data Center

>300 m to 500 m Distances

Future Multimode 100G: Relative Connectivity Cost/CircuitRelative Connectivity Cost/Circuit

PMD Max Distance (m)

Fiber Count

Relative Connectivity

Cost /Circuit at 100 m

100GBASE-SR10-OM3 100 20 1.53100GBASE-SR10-OM4 150 20 1.78100GBASE-SR4-OM3 100 8 1100GBASE SR4 OM4 150 8 1 13100GBASE-SR4-OM4 150 8 1.13100GBASE-NR4-SM no WDM

2000 8 0.81

100GBASE-NR4-SM with WDM

2000 2 0.24

100GBASE-LR4-SM CFP2 10000 2 0.24

100GBASE-LR4-SM CFP 10000 2 0.24

Future Multimode 100G: Relative Transceiver Cost/CircuitRelative Transceiver Cost/Circuit

PMD Max Distance (m)

Relative Module Cost

Range

100GBASE-SR10 150 1 1100GBASE-SR4 100 1 0 8 1 2 1 6100GBASE SR4 100 1 0.8, 1.2, 1.6100GBASE-NR4 no WDM 2000 5 1, 3 - 6

100GBASE NR4 with 2000 6 5 8100GBASE-NR4 with WDM

2000 6 5-8

100GBASE-LR4-SM CFP2 10000 25 5-25

100GBASE-LR4-SM CFP 10000 32 15-32

Future Multimode 100G: Relative Link Cost/Circuit

25

20100GBASESR10-OM3

100GBASESR10-OM4

15100GBASESR4-OM3

100GBASESR4-OM4

100GBASENR4 - No WDM

5

10100GBASENR4 No WDM

100GBASENR4 - with WDM

100GBASELR4 CFP2

0

5

0 200 400 600 800 1000

100GBASELR4 CFP

0 200 400 600 800 1000

Distance (m)

Contact InfoContact Info

Doug Coleman

E mail doug coleman@corning comE-mail: doug.coleman@corning.com

Phone: 828-901-5580Phone: 828 901 5580

Fax: 828-901-5488

Address: 800 17th Street NW Hickory, NC 28601