OptIPuter Backplane: Architecture, Research Plan, Implementation Plan

Joe Mambretti, Director, (j-mambretti@northwestern.edu)International Center for Advanced Internet Research (www.icair.org)

Director, Metropolitan Research and Education Network (www.mren.org)Partner, StarLight/STAR TAP, PI-OMNINet (www.icair.org/omninet)

OptIPuter Backplane Workshop

OptIPuter AHMCalIT2

January 17, 2006

Invisible Nodes, Elements,

Hierarchical,Centrally Controlled,

Fairly Static

Traditional Provider Services:Invisible, Static Resources,

Centralized Management

Distributed Device, Dynamic Services, Visible & Accessible Resources, Integrated As Required By Apps

Limited Functionality,Flexibility

Unlimited Functionality,Flexibility

LambdaGrid Control Plane Paradigm Shift

Ref: OptIPuter Backplane Project, UCLP

OptIPuter Architecture, Joint Project w/UCSD, EVL, UIC

Source: Andrew Chien, UCSDOptIPuter Software Architect

ODINSignalling, Control,

Management Techniques

Controller

Client Device

Client Controlle

r

Controller

ControllerControlle

r

Controller

Optical Control Plane

Client Layer Control Plane Optical Layer Control Plane

Client Layer Traffic Plane

Optical Layer – Switched Traffic Plane

UNI

I-UNI

CICICI

Network Side Interface

• WS BPEL• APIs• GMPLS as a uniform control plane• SNMP, with extensions as the basis of a management

plane• Extended MIB capabilities

– L3– IEEE L2 MIB Developments– MIB Integration with higher layer functionality

• Monitoring, Analysis, Reporting

L2 10 GE

• 10 GE Node Compute Clusters • APIs• 10 GE NICs• 10 Gbps Switch on a Chip• Currently, Low Cost Devices, Lost Per Port Cost• 240 GE• SNMP• Standard Services

– Spanning Tree

– vLANs

– Priority Queuing

• IEEE Enhancing Scalability

IEEE L2 Scaling Enhancements

• Current Lack of Hierarchy• IEEE Developing Hierarchical Architecture• Network Partitioning (802.1q, vLAN tagging)• Multiple Spanning Trees (802.1s)• Segmentation (802.1ad, “Provider Bridges”)• Enables Subnets To be Characterized Differently Than Core• IETF – Architecture for Closer Integration With Ethernet

– GMPLS As Uniform Control Plane

– Generalized UNI for Subnets

– Link State Routing In Control Plane

– TTL Capability to Data Plane

– Pseudo – Wire Capabilities

L2 Services Enhancements

• Metro Ethernet Forum • Three Primary Technical Specifications/Standards

– Ethernet Services Model (ESM)• Ethernet Service Attributes (Core “Building Blocks”)• Architectural Framework For Creating an Ethernet Service• No Specific Service – Any Potential Service

– Ethernet Services Definitions (ESD)• Guidelines for Using ESM Components for Service Development• Provides Example Service Types and Variations of Types

– Ethernet Line (E-Line)– Ethernet LAN (E-LAN)

– Ethernet Traffic Management (ETM)• Implications for Operations, Traffic Management, Performance, eg,

Managing Services vs Pipes• Quality of Service Agreements, Guarantees

L1 10 Gbps

• 10 GE Node Compute Clsuters

• APIs

• Automated Switch Panels

• GMPLS

• IETF GMPLS UNI (vs ONI UNI, Implications for Restoration Reliability)

• 10 G Ports

• MEMs Based– Services

• Lightpaths with Attributes, Uni-directional, Bi-directional

• Highly Secure Paths

• OVPN

• Optical Multicast

• Protected Through Associated Groups

• ITU-T SG Generic VPN Architecture (Y.1311), Service Requirements (Y.1312), L1 VPN Architecture (Y.1313)

Resource ResourceResource Resource

Physical Processing Monitoring and Adjustment

HP-PPFS HP-APP2 HP-APP3 HP-APP4

VS VS VS VS

ODIN ServerCreates/Deletes

LPs, Status Inquiry

tcptcp

AccessPolicy (AAA)

Process Registration

Discovery/ResourceManager, Incl Link Groups

Addresses

Previously OGSA/OGSI, Soon OGSA/OASIS WSRF

ProcessInstantiationMonitoring

ConfDB

Lambda Routing:

Topology discovery, DB of physical links

Create new path, optimize path selection

Traffic engineering

Constraint-based routing

O-UNI interworking and control integration

Path selection, protection/restoration tool - GMPLS

DataPlane

System ManagerDiscoveryConfigCommunicateInterlinkStop/Start ModuleResource BalanceInterface Adjustments

GMPLS ToolsLP Signaling for I-NNIAttribute Designation, egUni, Bi directionalLP LabelingLink Group designations

Control Channel monitoring, physical fault detection, isolation, adjustment, connection validation etc

OSM

UNI-N

PNWGPSeattle

Pacific Wave

NLR

NLR

NASA Goddard2Level 3

Level 3

NASA JPL

NASA Ames

4

CalREN-XD

4

CENICSan DiegoGigaPOP

2

UCI

ISI CENICLos Angeles

GigaPOP

U Amsterdam

CERN

NetherLightAmsterdam

2

The OptIPuter LambdaGrid

UCSD

StarLightChicago

UIC EVL

NU

AmsterdamChicagoSeattle

San Diego

StarLight

Northwestern

UIC UoA

CERN

NASA Goddard

CENIC San Diego GigaPOP

UCSD

CENIC LA GigaPOP

NASA Ames

NASA JPL

ISI

UCI

OMNInet Network Configuration 2006

10 GE

10 GE

To Ca*Net 4

710 Lake Shore

Photonic Node

600 S. Federal

Photonic Node

W Taylor 750 North Lake ShorePhotonic

Node10/100/GIGE

10/100/GIGE

10/100/GIGE

10/100/GIGE

10 GE

Optera5200

10Gb/sTSPR

Photonic Node

PP

8600

10 GEPP

8600

PP

8600

Optera5200

10Gb/sTSPR

10 GE

10 GE

Optera5200

10Gb/sTSPR

Optera5200

10Gb/sTSPR

1310 nm 10 GbEWAN PHY interfaces

10 GE

10 GE

PP

8600

Fiber

KM MI1* 35.3 22.02 10.3 6.43* 12.4 7.74 7.2 4.55 24.1 15.06 24.1 15.07* 24.9 15.58 6.7 4.29 5.3 3.3

NWUEN Link

Span Length

…CAMPUS

FIBER (16)

EVL/UICOM5200

LAC/UICOM5200

CAMPUSFIBER (4)

INITIALCONFIG:10 LAMBDA(all GIGE)

StarLightInterconnect

with otherresearchnetworks

10GE LAN PHY (Dec 03)

• 8x8x8 Scalable photonic switch• Trunk side – 10 G WDM

• OFA on all trunks

TECH/NU-EOM5200

CAMPUSFIBER (4)

INITIALCONFIG:10 LAMBDAS(ALL GIGE)

Optera Metro 5200 OFA

NWUEN-1

NWUEN-5

NWUEN-6NWUEN-2

NWUEN-3

NWUEN-4

NWUEN-8 NWUEN-9

NWUEN-7

Fiber in use

Fiber not in use

5200 OFA

5200 OFA

Optera 5200 OFA

5200 OFA

DOTClusters

Optical Switch

Optical Switch

Optical Switch

Optical Switch

High Performance L2 Switch

High Performance L2 Switch

High Performance L2 Switch

High Performance L2 Switch

1 x 10G Wan

1x 10G Wan1 x 10G Wan

1 x 10G Wan

Trib ContentOC-192 – with TFEC 16OC-192 – without TFEC 12Ge 8OC-48 0

TFEC Link

Non - TFEC Link

Ge (x2)

Only TFEC link can support OC-192c (10G Wan) operation

Non -TFEC link used to transport Ge traffic

Ge (x2) Ge (x2)

Ge (x2)

Default configuration:

Tribs can be moved as needed

Could have 2 facing L2 SW

Default configuration:

Tribs can be moved as needed

Could have 2 facing L2 SW

TFEC = Out of band error correctionOMNInet 2005

750 North Lake Shore Drive710 North Lake Shore Drive

1890 W. Taylor

600 N. Federal

Extensions to Other Sites Via Illinois’ I-WIRE

Research Areas•Displays/VR•Collaboration•Rendering•Applications•Data Mining

NCSA

Argonne UIC/EVL

Research Areas• Latency-Tolerant

Algorithms• Interaction of

SAN/LAN/WAN technologies

• Clusters

UIUC

CS

StarLight

Source: Charlie Catlett

Summary Optical Services: Baseline + 5 Years

2005 2006 2007 2008 2009 2010Dedicated Lightpaths

Enhanced Direct

Addressing

Additional LPs National, Global

Additional LPs National, Global

Site Expansion:

Multiple Labs

Site Expansion:

Multiple Labs

Dynamic Lightpath Allocation

Increased Number of Nodes on LPs

Increased Allocation Capacity US

Increased Allocation Capacity Global

Increased Allocation Capacity: Sites

Increased Allocation Capacity: Sites

Highly Distributed

Control Plane

Persistent Inter Domain Signaling

National Global

Multi-Domain

Distributed

Control

Extension to

Additional Net

Elements

Persistence: Common Facilities

Additional

Facility

Implementations

Deterministic

Paths

Close Integration

w/ App Signaling

Increased Attribute Parameters

Increased Adjustment Parameters

Performance Metrics and Methods

Enhanced Recovery

Restoration

Autonomous Dyn.

Lightpath Peer.

Multi-Domain

ADLP

Integration with

Management Sys

Extensions of ADLP Peering

E2E ADLP Recovery

Restoration

Multi-Service

Layer Integration

5-10 MSI Facilities

10-20 MSI

Facilities

20-40 MSI

Facilities

Additional US, Global Facilities

Additional US, Global Facilities

Optical Multicast Enhanced Control of OM

OM/App Integration

Expansion to

Addtn’l Objects

Expansion to

Addtn’l Apps

Expansion to

Addtn’l Apps

App/Optical

Integration

App API-Op Ser

Validation

Integration with

Optical Services

Monitoring

Techniques

Analysis Techniques

Recovery,

Restoration

Wavelength Routing

Persistent

Wavelength Routing

Multi-Domain

Wavelength Routing

Multi-layer

Integration

Multi-Services

Integration

Enhanced Recovery

Restoration

Summary Optical Technologies: Baseline + 5 Years 2005 2006 2007 2008 2009 2010O-APIs Additional

Experiments w/

Architecture

App Specific

APIs

Variable APIs Integrated with Common Services

Enhancement of

Architecture

Additional Deployment

Distributed Control Systems,

Multi-Domain

Additional Experiments w/

Architecture

Integration with

ROADMs

Expansion to Edge Sites

Enhancement of

Architecture

Additional Deployment

OOO

Switches

At Selected

Core Sites

At Selected

Core, Edge Sites

+ Experimental

Solid State OSWs

Solid State OSW

Deployment

Solid State

At Core, Edge

O-UNIs At Selected

Core Sites

At Selected

Core, Edge Sites

Deployment At All Key Sites

Additional

Deployments

Wide Deployment

Service Abstraction –GMPLS Integr.

Additional Signaling Integration

Increased Transparency &

LayerElimination

Increased Integration with ID/Obj.Discovery

Prototype Arch for App Specific

Serv Abstractns.

Formalization of

Enhanced Architecture

Policy Based

Access Control

Additional Experiments w/

Architecture

Formalization of

Architecture, eg

Via WS

Expansion to Additional

Resources

L1 Security Enhancements

Formalization of

Enhanced Architecture

New Id, Object and Discovery Mechanisms

Integration of New Id, Obj, Dis w/ New Arch.

Integration

With Multiple

Integrated Serv.

Integration w/New Management Sys

Extensions to various TE Functions

Persistent at

Core, Edge Facilities

DWDM DWDM

CWDM

Integration with Edge Optics

Integration with BP Optics

Additional

MUX/DMUX

Increased Stream

Granularity

2D MEMs 3D LP Switches Experimental Opt Packet SWs

Prototype Deployed OPSW

At Core Sites At Edge and Core Sites

Summary Optical Interoperability Issues: Baseline + 5 Years

2005 2006 2007 2008 2009 2010Common Open Services

Definitions

Common Services R&D

Common Services

Experimentation

Initial Standards

Formalization

Establish CSD Enhancement Process

On-Going

COS

Architecture

Initial Implementations

Expansion of

Functionality

Initial Standards

Formalization

Enhancement

Process

On-Going

Open Protocols

And Standards

Initial Implementations

Expansion of

Functionality

Initial Standards

Formalization

Enhancement

Process

On-Going

Distributed

Control

V2 with WS

Integration

Multi-Service

Integration

New Services

Integration

Extensions, Horizontal, Vert

Integration with New Opt Core

Multi-Domain

Interoperability

Enhancement of

Signaling Functionality

Access Policy

Services

Expansion to Additional Domains

Increasing US, Global Extensions

Increasing US, Global Extensions

Implementation

At GLIF Open

Exchanges (4)

5-10 OE Sites 10-20 OE Sites 20-30 OE Sites 30-40 OE Sites 40-50 OE Sites

Basics Services at

Key US, Global Research Sites

15-30 Sites 30-60 Sites 60-90 Sites 90-120 Sites 120-150 Sites

Basic Services at

Key US Science Sites

7-15 Sites 15-30 Sites 30-45 Sites 45-60 Sites 60-75 Sites

Service Est.

At Selected Labs

30-60 Labs 60-120 Labs 120-180 Labs 180-240 Labs 240-300 Labs

Overall Networking Plan

Seattle

Chicago

San Diego (iGRID,UCSD)

Dedicated LightpathsNLRPacific WaveCENIC

PW/CENIC

University of AmsterdamStarLight

NetherLight 4 Dedicated Paths

Route B

NetherLightDedicated Lightpaths

San Diego (iGRID, UCSD)Seattle

4*1GpbsPaths +

One Control Channel

AMROEBA Network Topology

L2SW

L3 (GbE)L2SW

iGRID Conference

OME

UvA VanGogh Grid ClustersiCAIR DOT Grid ClustersiGRID Demonstartion

Control

L2SW

L2SW

L2SW

SURFNet/University of AmsterdamStarLight

L2SW

Visualization

Visualization

www.startap.net/starlight