michigan collaboration summit - washtenaw community college

Roy Hockett - Network Architect ITS Communications Systems and Data Centers University of Michigan

Dan Orzech – Systems Consultant ADVA Optical Networking

David Altstaetter – Director, Applications & Solutions ADVA Optical Networking

Michigan Collaboration Summit Washtenaw Community College


Is collaborative effort between Merit, MiLR, REACH and ADVA. Working together, Merit coordinate the upgrade effort. U of M, MSU, Wayne State and Merit provided escort personal, ADVA supplied the people necessary to install and turn up the equipment. The network above, that consisted of 7 Add/Drop sites and 7 Line Amp sites was cut-over in 3 days.


Considerations

• Rack Space • Is there sufficient space in the rack to accommodate a FSP3000 shelf or

shelves?

• Power • Is there sufficient power in the rack to accommodate a FSP3000 shelf or

shelves? • Upgrade may be required

• Time Constraint / Time of Year • What is the maintenance window available in order to cut over the

network? • How many NE will need to be cutover? • What are their locations?

• Logistics / Staffing / Access • How many people will be required? • Travel distance site to site? • Is access to the sites set up?

• Fiber Cutover • Done in segments • Leapfrog bodies


Lesson learned

• Provide better documentation for ADVA for a more accurate network design • Ensure there sufficient space in the rack to accommodate a FSP3000 shelf or

shelves? • NE ID’s – IP address – Visio Diagram – Installation Documents – Power

Requirements

• Conference call prior to cutover • Discuss upgrade as a team. Different points of view bring up different possible

problems and solutions • Conference call open during the installation very helpful

• Set up via NMS after installation complete • Provisioning using the NMS will allow quicker discovery of services

• Detail address spaces, two nodes one ID_ problem • If this will be a problem, address resolution should be identified prior to the install

• OSC in the FSP3000 is a flat layer 2 and does not communicate with the RE OSC layer 3


What went well

• ADVA staged the entire network prior to the deployment. • A very a good thing - proper addressing for smoother transition

• Entire network done in 3 days - 7 Add/ Drop sites, 7 Amp sites

• Ring done in 3 separate segments

• Well documented diagrams / wave assignments


Reach-3MC Map


ADVA Company Overview

Your solutions partner for Packet Optical transport solutions

“Our mission is to be the trusted partner for innovative Optical+Ethernet transport solutions that ADVANCE next-generation networks for data, storage, voice and video services.”

Founded 1994

Public company (FSE:ADV)

$429M in 2012

1,300 employees

300+ in North America HQ- Norcross, Georgia Over 400 degreed engineers

Diverse global customer base Enterprise, R&E, Carrier

Market leader in Enterprise Connectivity Solutions


ADVA Fiber Service Platform Solutions

Ethernet Access Metro

Optical Access

Enterprise

Core

Unified network and service management

Business Ethernet

Mobile Backhaul

Broadband Backhaul

Data Center Connectivity

Agile Core Transport

Application-Optimized Enterprise Networks


FSP3000 System Architecture 9HU High End Shelf

High power support by

optimized air flow

management and individual

swappable fan units

Network Controller Unit (NCU-

II) for NE Management

Shelf Controller Units (SCU)

for Connectivity within and

between shelves

1000W Power Supply Units

(PSU) with Redundancy

1000W Power Supply Units

(PSU) with Redundancy

Separate electrical

and optical cable

management

Separate electrical

and optical cable

management

Central Equipment Management Panel

3x RJ45 Ethernet

Telemetry interface (16xI/O) and LED’s

4x PSCU interfaces for inventory of passive units

User interface (Display) and shelf alarm status LED’s


FSP 3000 System Architecture 7HU Common Shelf and 1HU Slimline Shelf

• 7HU Shelf (20 slots)

• Redundant PSU (400/600/800W AC/DC options)

• SCU

• NCU, NCU-II

• OSCM + OSFM

• EDFA + DCM

• Filters

• Transponders

• 1HU Slimline Shelf (2 slots)

• Redundant AC/DC PSU on board

• SCU, SCU-S

• NCU / NCU-II

• Filters

• Transponders


FSP 3000 Key Features

Lowest total cost of ownership R7 increases density - 16 10G transponders per shelf

R7 w/ 4WCE can have 64 x 10G on a single 7RU shelf.

Flexibility, Simplicity

Full Connectivity Portfolio for SAN and HPC-Clustering Long distance FC transport (4G-FC) on 10G Core Muxponder

Dual muxponder for highest density & lowest cost FC transport

8G-FC and Infiniband support with 10G Access Transponder

16G FC with 4WCE –IBM Certified

Lowest latency transport in the industry

Power efficiency and rack space

Technology Leadership - Proven track record of commitment to investment protection

The scalable optical transport solution

Support of all carrier and enterprise protocols and I/Fs: SDH/SONET, ODU/OTU, ESCON, FICON,1/2/4/8/10G FC, GbE, 10GE, IB (SDR, DDR, QDR), 40G, 100G

Qualifications for all major enterprise apps (e.g. IBM GDPS(PSIFB))

Encryption

Sophisticated optical layer: ROADMs, EDFAs, RAMAN, DWDM 120l


METRO Convergence

Fiber Pair

Fibre Channel

FICON Channel

ESCON Channel

Infiniband

System z

FabricPath

Brocade One

QFabric

Sonet/SDH

Video

Ethernet

ADVA FSP3000

DWDM‘s

FULLY ENCRYPTED PAYLOAD AT THE PHYSICAL LAYER

WAN

WDM-transport

Site B

WDM-transport

Site A Router

FC switch

Router

FC switch WDM encryption

Fibre Channel

FICON Channel

ESCON Channel

Infiniband

System z

FabricPath

Brocade One

QFabric

Sonet/SDH

Video

Ethernet


Typical Enterprise implementations Point to Point Network

• Optical networks for business continuity and disaster recovery

• Guaranteeing the survival of essential data even in case of a catastrophic failure

• Static high-bandwidth connections for instant data replication by synchronous disk mirroring

Tape Tape

FSPManagement Suite

Disk array Disk array

Data center A Data center B

Server ServerSwitches/Directors

Switches/Directors

Tape Tape

FSPManagement Suite

Disk array Disk array

Data center A Data center B

Server ServerSwitches/Directors

Switches/Directors


Typical Enterprise implementations Ring Network • Optical networks for data center inter-connection

• Network infrastructure to enable storage and server virtualization across data center sites

• Static high-bandwidth connections for data transmission between processing and storage sites

• Essential to exploit gains from statistical resource allocation and therefore increase efficiency

Multi-site Network

Data Center South-West

Data Center South-East

Data Center North

Storage Farm West

Server Farm East


Multiservice Application Overview - Native Service Offerings

Gbit/s

SDH

155,5

2

STM-1

622,0

8

STM-4

2.4

88

9.9

53

39.8

13

LAN

OTN

2.6

66

10.7

09

43.0

18

LAN/ WAN

12,5

125

Fast Ethernet

Gigabit Ethernet

1.2

50

10.3

12

SAN

200

ESCON

4.2

50

4G FC/ FICON

8.5

00

8G FC

10.5

18

10G FC

1.0

62,5

2.1

25

HPC-Cluster

8…

16

Sysplex Timer ETR,CLO

ISC-3 Infini Band SDR

2.5

00

Infini Band

1xDDR

5.0

00

Infini Band

1xQDR

10.0

00

Coupling Link

ISC-2

Mbit/s

103.2

50

OTU4

111.8

25

100GbE 40GbE

41.2

50

OTU3

16G FC

14.0

25

STM-256

OTU2 OTU1

10GbE WAN-PHY LAN-PHY

STM-64

FC/ FICON

2G FC/ FICON

STM-16

10Gbit/s 100Gbit/s

Ethernet


Line speed: 40G 10G 100G 4G/2G7

Core cards

Performance optimized

Full G.709 support

Enterprise cards

Fully qualified*

Access cards

Cost optimised

Protocol aware

WCC-TN

4TCC-PCTN WCC-PCTN LN/V

4TCC-PCTN LN/V

10TCC-PCTN LN/V

2TWCC-PCN

4TCC-PCN

4TCA-PCN-4G*

WCA-PCN

Pluggable IF Tunable IF

2WCA-PCN*

10PCA-PCN

Transponders and Muxponders Channel Cards – Full Service Support

2PCA-PCN

5TCE-PCTN LN/V*

5TCE-PCTN-AES*

2WCC-PCN

10TCC-PCN

WCC-PCTN

10TCE-PCN*

WCE-PCN*

4WCE-PCN-16G* R11.2

Application Optimized 100G Solutions


Motivation for 100G - Why?

• Data rates growth

• Network consolidation & optimization

• Improve Efficency

• Cost

• Spectral occupancy

• Power consumption

• Space

In 2015 total traffic will be 4 times larger than in 2010


IEEE 802.3ba 40/100G Interfaces Client options for 100G

Nomenclature for the 3-part suffix

• 40 = 40 Gb/s, 100 = 100 Gb/s

• Medium type

• Copper • K = Backplane

• C = Cable assembly

• Optical • S = Short Reach (100 m)

• L = Long Reach (10 km)

• E = Extended Long Reach (40 km)

• R = 64B/66B block coding

• Number of lanes or wavelengths

• Copper: n = 4 or 10

• Optical: n = number of lanes or wavelengths

IEEE P

802.3

ba T

ask F

orc

e m

eeting,

May 2

008,

Munic

h,

Germ

any

PHY Description Port Type

40G Backplane PHY 40GBASE-KR4

40G Cable Assembly PHY

100G Cable Assembly PHY

40GBASE-CR4

100GBASE-CR10

40G MMF 100m PHY (Ribbon)

100G MMF 100m PHY (Ribbon)

40GBASE-SR4

100GBASE-SR10

40G SMF 10km PHY

100G SMF 10km PHY

40GBASE-LR4

100GBASE-LR4

100GBASE-LR10*

100G SMF 40km PHY 100GBASE-ER4

* Not standardized, but MSA available.

Customer Provider

Optical Transport Network

100GbE

Client I/F

IEEE 802.3ba TF

100G-BASE-xx

DWDM Line I/F

ITU-T SG15

OTU4

N x 100GbE

OIF Electronic I/F specs,

100G DWDM optical specs,

LH modules (DP-QPSK)


100G Solutions Optimum bandwidth delivery of every need

• Performance, Capacity & Reach optimized

• Longer Distances (up to >2500km) with many ROADMs

• 10GbE, SDH, OTN

• Cost, Power & Space optimized

• Shorter distance (10-500km) with few or none ROADMs

• Huge amount of different channels (10GbE, 8-16G FC...)

Coherent DP-DQPSK

• DSP technology

• Integrated compensation

Direct Detect 4x28G

• LR4 concept for DWDM

• Based on 10G technology

Decision

Decision

Decision

Decision

FEC

Core & Metro Core

Metro & Enterprise


Optimized 100G Solutions

Metro & Enterprise Core & Metro Core


100G Core Interface

• Most efficient DSP implementation

• Lowest power in market

• High performance 60 GS/s sampling rate ADC

• SD-FEC based on Turbo Product Code

• Coding Gain: 11.1 dB @ 15% overhead

• No external FEC needed (error floor<<10-18)

• Burst error robustness: >2000

• Dispersion setting and channel selectivity within ms


100G Core Interface …delivers optimum performance

• 10G-like performance

• OSNR @ FEC typ. 12.3 dB BOL (13.8dB EOL)

• Multiple ROADM nodes (25GHz BW @ <0.5dB penalty)

• High robustness

• CD +/- 60,000 ps/nm

• PMD 30 ps (100ps of DGD) with fast tracking (90ps in < 1 ms, SOP < 10 rad/ms)

• PDL 3dB @ <0.5dB penalty

• Simple operation

• Quick cold start time of interface of <500 ms

• Quick adaption to dispersion and PMD (~ms)

• Integrated link performance monitoring (DGD, CD and pre-FEC statistics)


Transmission Results – TW Example

• 1520km (20-span) TrueWave(RS) link

• 96x 100G with hybrid Raman/EDFAs and flex-grid ROADM

• All channels modulated at 100G

• Average power per channel ~-3dBm (per Eng. Rules and not optimized)

• Min. Q margin 2.6dB (BER 2.8 x10-3 / 6.0dB OSNR margin)

• Q margins on other channels > 3.6dB


Published ULH Performance

• 3760km (47x80km) published trial with a US carrier, recently published

• SD-FEC, 60,000ps/nm CD compensation, fast PMD tracking

• 2.5dB margin


Directionless Colorless Add Drop a cost advantage of coherent detection

Lin

e s

wit

ch

A

dd

dro

p

1:9

9:1

WSS

9ROADM

1:9

9:1 WSS

9CCM 1:8

8PSM

8:1

72 colorless directionless A/D channels per branch

Channel selection by coherent receiver

1:9

9:1

WSS

9ROADM

1:9

9:1

WSS

9ROADM

1:9

9:1

WSS

9ROADM

1:9

9:1

WSS

9ROADM

OSC

OSC

AMP

OSC

OSC

AMP

OSC

OSC

AMP

OSC

OSC

AMP

• 9CCM-96 and 8PSM planned for R13


• Cost breaking 100G solution for up to 500km

• Multi-service: Eth, FC, SDH

• Multi-rate: 4G, 8G, 10G, 16G*, 40G, 100G clients

• Ultra high density

• Low power: less than 1 Watt per Gigabit

• Low latency: configurable down to 3.5 microseconds

Metro & Access 100G Solution

Direct Detect 100G “Metro” Scheme

Decision

Decision

Decision

Decision

FEC

Re

Im ODB

802.3ba

0 10 100 1000 10000

Cabling Access

Distance (km)

LH

100G coherent

Metro

100G direct detect


4x 2

8G

b/

s

DW

DM

1x 40GE Switch/Router

with 40G Module (SR4)

MTP connector with up to 12x parallel lanes

Fan-out cable from MTP into 4x parallel 10G lanes using LC

1x 40GE

Similar approach for 100GE (10 parallel lanes of 10G)

100G Metro/Enterprise Muxponder 40GE & 100GE support

MTP LC Duplex 1

2

3

4

12 fiber


100G for Metro and Core: One size does not fit all!

• 100G for Core Networks

• Longer Distances (>300km)

• 10GbE, SDH, OTN

• Performance, Capacity & Reach optimized

• 100G for Metro and Enterprise Networks

• Shorter distance (10-200km)

• Huge amount of different channels (10GbE, 8-16G FC...)

• Cost, Power & Space optimized

100G looks different in the Metro/Enterprise than in the Core!

[email protected]

Thank you

IMPORTANT NOTICE

The content of this presentation is strictly confidential. ADVA Optical Networking is the exclusive owner or licensee of the content, material, and information in this presentation. Any reproduction, publication or reprint, in whole or in part, is strictly prohibited.

The information in this presentation may not be accurate, complete or up to date, and is provided without warranties or representations of any kind, either express or implied. ADVA Optical Networking shall not be responsible for and disclaims any liability for any loss or damages, including without limitation, direct, indirect, incidental, consequential and special damages, alleged to have been caused by or in connection with using and/or relying on the information contained in this presentation.

Copyright © for the entire content of this presentation: ADVA Optical Networking.

http://www.linkedin.com/groups?about=&gid=1194227&trk=anet_ug_grppro

http://twitter.com/ADVAOpticalNews

http://www.facebook.com/pages/ADVA-Optical-Networking/37630238931?ref=ts

http://blog.advaoptical.com/

michigan collaboration summit - washtenaw community college

Technology

optical accessbroadband

fsp3000 shelf

ru shelf

g transponders

sufficient power

hu common shelf

shelf r7 w

hu slimline shelf7hu