Power Saving with Fiber Optic Structured Cabling in Data Centers

Andreas Koll

Manager Data Center SolutionsCorning Cable SystemsPrivate Networks EMEA

Agenda• Standards / Protocols

– New Path for Fiber Chancel (FCoE)– Next Generation Ethernet– New Fiber Class OM4

• Modular Systems with MPO/ MTP– Structure of Modular System– Fiber Polarity– Migration of Speed beyond 10G

• “Green aspects” of Cabling– Power Consumption of 10Gbit– Environmental Impact

• Improved Cooling with Optimized Airflow– complex simulation

Fiber Channel over Ethernet (FCoE)• Activity initiated at T11 Fiber

Channel, April 2007

• Encapsulate Fiber Channel Packet into a Ethernet Frame– Lossless Packet Performance

• Supports Utilization of low cost Ethernet electronics up to the SAN switch– FCoE Server Adapter Card– FCoE Line Cards

• Designed to operate at 10G • Large Data Centers focus• Standard Completion Mid-2008• Commercial Products 2009

Standards - Ethernet: The Next Generation

IEEE 802.3ba – Ethernet Task Force

• Objectives for 40 and 100Gb/s– Full duplex- operation with 802.3 Ethernet Frame-Format– Min. 40km on SMF – OS1/2– Min. 10km on SMF – OS1/2– Min. 100m on MMF – OM3 – Min. 10m on Copper

OM1/2 will not be recognized by the new standard!

3 Approaches: • MMF OM3: parallel (SDM)• MMF OM3: 2 λ CWDM• SMF OS2: 10 λ CWDM

Assumptions:• projected 2010 prices for

– Cables– Connectivity components– Laser (TRX)

Next Generation Ethernet -

• Recent history suggests that standards (and initial cabling sales) will lead actual port sales by ~3 years

• Given port sale historical trends, we can project initial applications ~ 2012

• Most applications not expected until >2013

Prepare Now

Cabling systems installed today will still be in operation.

Fiber Specifications – LOMMF for 40/100G• Initial modelling and actual tests at CFT

show achievable link lengths of 200m with OM3 and <10GbE sources

– > 95% of link lengths in data centres

• Further modelling shows that OM4 MMF can provide link lengths up to 250m with <10GbE sources

– ~ 99% of link lengths in data centres

• Proposal for OM4:EMB (850nm) requirement should be 4700MHz.km– 100m at 4x 25Gb/s are addressed

OFL specification to be eliminated– LX-4 / LRM not relevant for laser-optimized MMF;

850nm EMB covers all 850nm OFL applications

• CORNING Standards Initiative for 40/100G Ethernet:– OM3 minimum link length of 150m– OM4 minimum link length of 250m

Electronics for Parallel Optics

Parallel-Optical Tx / Rx (SNAP12)• XFP, hot-pluggable or on-board• MPO Interface• 1, 2, 4 Gb/s, SDR, DDR data rates

QSFP MSA Transceiver• four optical circuits per port• MPO Interface• 1, 2, 4 Gb/s, SDR, DDR Data rates• Hot-pluggable, Rate-select

Fibre number 1 2 3 4 … 9 10 11 12Channel Tx 1 2 3 4 … 4 3 2 1 Rx

Agenda• Standards / Protocols

– New Path for Fiber Chancel (FCoE)– Next Generation Ethernet– New Fiber Class OM4

• Modular Systems with MPO/ MTP– Structure of Modular System– Fiber Polarity– Migration of Speed beyond 10G

• “Green aspects” of Cabling– Power consumption of 10Gbit– Environmental Impact

• Improved Cooling with Optimized Airflow– complex simulation

Multi-Push-On Connector: MPO/MTP®

• 12 Fibers in one connector

• Used to: speed up Installationminimize errorssave space

• enables high-density trunk cables

• Push-Pull connector

• Alignment with precision guide pins

• According to IEC/EN 61754-7 MPO- connector family

• Standard adapter: “Key-up” <> “Key-down”accommodates SM-APC

Modular System Approach

Duplex patch cord

MTP - duplex harness

Tx/Rx

Tx/Rx

MTP-Extender trunk

12-Fibre MPO/MTP

MTP-Trunk cableModule

Solutions based on factory-terminated components:

• Rugged MTP trunk cables for high fiber counts• Small Form Factor connector modules and adapter panels

• SFF break-out cables and patch cords …

• Embedded in a versatile housing concept

Data Centre Management - Fiber Polarity

Fibers are physically straight, Optical Connectors are logically crossed

- Reverse Fiber Polarity(Standard)

Fibers are physically crossed, Optical Connectors are logically straight

(Non-standard)

TxRxTx

Rx

TxRx TxRx

Reverse Fiber Polarity (standard)

A Polarity Management System should …

Guarantee correct Fiber Polarity throughout the System Eliminate the need for advanced polarity Planning or MaintenanceUse standard Patch Cords and require no special polarity componentsAccommodate Single-Mode APC MPO/MTP®

Provide a Migration Path to Future Technologies such as Parallel Optics

Fiber Polarity Management and Migration

Rx

Tx Rx

Tx

MC

IC IC

STRUCTURED CABLING

Rx

Tx Rx

Tx

MC

IC IC

STRUCTURED CABLING

> Parallel Optics

Seamless Migration to Parallel Optics

Rx

Tx

MC

IC Strukturierte Verkabelung IC Rx

Tx Rx

Tx

Parallel Optics Requirements

Speed of light in glass: ~ 200000km/s or ~ 20cm/ns

Individual fibers need to be controlled regarding differences in both length and fibre skew.

Single-fiber cables may not be suitable for parallel optics applications.

Skew-controlled ribbon fibers provide the highest performance levels.

Skew:difference in propagation time between lanes of a parallel transmission system.

∆ propagation speed or distance

Infiniband Skew Requirements:

Are You ready for 100G?Check list:

• Skew-Performance according to12X-QDR (120G) IB specificationof 0.75ns up to 300m link length

• MTP(MPO) Connectivity and Ribbon-flip in P‘n‘P Universal Trunks

• Low attenuation of 2dB (1,5dB)

• EMB for 10x10Gb/s beyond 100m

Guaranteed performance up to 100Gb/sas per definitions of IEEE 802.3ba

Agenda• Standards / Protocols

– New Path for Fiber Chancel (FCoE)– Next Generation Ethernet– New Fiber Class OM4

• Modular Systems with MPO/ MTP– Structure of Modular System– Fiber Polarity– Migration of Speed beyond 10G

• “Green aspects” of Cabling– Power consumption of 10Gbit– Environmental Impact

• Improved Cooling with Optimized Airflow– complex simulation

Energy Saving in Data Centre• Improve efficiency of general infrastructure:

– Cooling (water, CO2, free cooling, alternative recooling)– Using rejected heat for heating the building (e.g. offices)– Design of cool and warm aisles– UPS Systems with better efficiency– Low energy building

• Increasing computing power:– Using Dual/Quad Core Processor technology– Blade server instead of Pizza box server

Did you already think about the influences of cabling?

Power Consumption for 10 Gbit over Fiber

1) Incl. Not used cooling (40% => 10W)2) Assumption: 4100 kWh / year3) 1 MWh equal to 0,830 T CO2

=> min. saving 12 W / port• 10Gbps transceiver use:

with copper: ≈15 W / port with Fibre Optic: 1-3 W / port

• Reduce power consumption (34W / used 10Gbit link)1)

• The savings of 14 converted links are equal to the power consumption of one-family house / year2)

• 4 converted link saves ~1 ton CO2 / year3)

Saving Resources with Fiber OpticBiggest copper

mine world wide

Chile; Chuquicamata

depth: 850mwidth: 4,3x3,0 km

Planned deepening up to 1,3 km

Saving Resources with Fiber Optic• Saving resources thru reduced use of materials

(energy, Raw materials, mining)- 1km Copper cable ~ 33kg Copper=> used for production 13.500 kg environment

- 1km Fiber optic cable1) ~ 1,37 kg glass=> used only for production 68,5 kg environment 2)

• Example: 24 Links (Channels) , length 45m

⇒ Balance for copper cable:65,9 kg total weight / 35,6 kg copper=> 14.240 kg used Environment +

30,3 kg petrol chemical plastics

⇒ Balance for fiber optic cable: 5,9 kg total weight / 0,062 kg glass=> 3,1 kg used environment +

5,84 kg petrol chemical plastics

1) 48 Faser Ribbon Fiber optic cable2) „The Fossil Makers“ – Friedrich Schmidt-Bleek - 1994

Agenda• Standards / Protocols

– New Path for Fiber Chancel (FCoE)– Next Generation Ethernet– New Fiber Class OM4

• Modular Systems with MPO/ MTP– Structure of Modular System– Fiber Polarity– Migration of Speed beyond 10G

• “Green aspects” of Cabling– Power consumption of 10Gbit– Environmental Impact

• Improved Cooling with Optimized Airflow– complex simulation

Cooling in Data Center – a Cabling issue

Badcooling

In raised flooror

racks

Simulation inside Raised FloorAfter all references for saving energy in a DC would be realized(without changing cabling from copper to fiber) following an example for saving energy with fiber optic cabling.Simulated with Computational Fluid Dynamics (CFD; Ansy-Fluent 6.3.26)

DC compartment with:• 178 m²• 6 rows each of them with

9 cabinets• 3 cold aisle / 4 warm aisle

• 40 complete Blade chassis • 180 2U servers (pizzabox)• 660 x 10Gbit Links

All sources: Corning 2008

Installed Server Power

Air temperature °C

Air temperature °C

variation 1:- Cabling Copper and fibre optic, 10 Gbit, - Air guide over room long side (259 kW), - height of raised floor 40cm- two cable tray in the raised floor- 2 recirculation cooling machines

All sources: Corning 2008

Comparison of Heat Distribution

variation 2:- Cabling only via fibre optic, 10 Gbit, - Air guide over room long side (239 kW),- only one Cable tray in the raised floor- height of raised floor 40cm- 2 recirculation cooling machines

Static Pressure of Optimized Traces in Raised Floor

All sources: Corning 2008

Basic data:height of raised floor: 40 cmTotal supply air flow: 92.700m³/h

- required pressure at variation 1with two traces: 116 Pa

- required pressure at variation 2with one traces : 99 Pa

=> 15 % less drop in pressure,so 6.400 kWh power saving / year

air speed m/s

Summary Airflow Simulation 10 Gbit via Copper

10Gbit via Fiber optic

Different absolute

Unit

Computer power(incl. Networking power)

2,268,840 2,084,880 183,960 [kWhr(elec)/year]

Cooling power 567,210 521,220 45,990 [kWhr(elec)/year]

Fan power 43,607 37,221 6,386 [kWhr(elec)/year]

Total Sum 2,879,657 2,643,321 236,336 [kWhr(elec)/year]196 tons1)

less CO2emissions / yearsources: Corning 2008

8% saving of energy

1) 1 MWh equal to 0,830 T CO2

Summary• MPO/MTP connectivity is the basis for current

high-end parallel optics systems

• Modular cabling systems speed up deployment, ease maintenance and system management and enable easy migration to next generation technology

• Corning laser-optimized Multimode Fibres support future performance requirements of modern data

• Significant power saving at 10Gbit Ethernet via fibre,Cabling is a part of a Green Data Centre

• Saving resources with fibre optic cables

• Using high fibre count cable improve Airflow in raised floor

Thank You for Your Attention!

Our Team in Dubai:

Baris Kukukay Baris.kukukay@corning.comMazin Sahli Mazin.Sahli@corning.comMustafa Keskin Mustafa.Keskin@corning.comMahmoud Al Ghazali Mahmoud.ghazali@corning.com

Andreas KollEmail: Andreas.Koll@corning.com