Control Plane Issues for NRENs

Gigi Karmous-Edwardsgigi@mcnc.org

May 22, 2006

EARNEST ForesightWorkshop

• E-Science

• Research Challenges

• Examples of Infrastructure for Network Research Experimentation• NLR• GENI• GLIF

• International Collaboration Experiences w/EnLIGHTened Computing

• Conclusions

Outline

Motivation

• E-science: global, large scale scientific collaborations enabled through distributed computational and communication infrastructure

• Combines scientific instruments and sensors, distributed data archives, computing resources and visualization to solve complex scientific problems

• In physics, molecular biology, environmental, Health, Entertainment, etc.

• E-Science Definitions Oxford E-Science Center• The Department of Trade and Industry defines e-Science as: "Science increasingly

performed through distributed global collaborations enabled by the Internet, using very large data collections, terascale computing resources and high performance visualizations"

• This essentially means that many areas of science currently using computing resources as part of their research, will soon have the ability to utilize more powerful computing resources across a new infrastructure commonly described as the 'grid'. Scientists will have access to very large data sets and perform real time experiments on this data. This will ultimately lead to scientists tackling the 'big scientific questions' hitherto unexplorable.

E-science

• Grid computing: main enabler of E-science• Grid is concerned with "coordinated resource sharing and problem solving in dynamic, multi-institutional virtual organizations." (Ian Foster)

• Migration of the E-science community towards Grid Computing emerged from three converging trends;

i) Advances in optical networking technologies. Widespread deployment of the fiber infrastructure has led to low-cost, high-capacity optical connections.

ii) Affordability of the required computational resources through sharing. The increasing demand of computational power and bandwidth by the new e-science applications is proving to be a financially difficult and nearly impossible task unless resources are shared across research institutions.

iii) Need for interdisciplinary research. The growing complexity of scientific problems is driving the need for increasing numbers of scientists from diverse disciplines and locations to work together in order to achieve breakthrough results.

E-Science and Grid computing

• Korea’s HVEM • One of a kind in the world -

• Provide global access to

unique instruments for the purpose of advancing science for humanity • WEB service interface• High capacity optical network for output

Developing a Global E-science Laboratory (GEL)

The tasks that the HVEM users can perform:• Requesting the general operations of the goniometer, TEM, and CCD camera• Viewing the real-time video from the CCD camera• Accessing or manipulating the 2-D or 3-D images• Generating the workflow specification and requesting the

workflow to be executed• Searching the images or video files, papers, and

experiments in the databases or storages

Hyuck Han, Hyungsoo Jung, Heon Y. Yeom, Hee S. Kweon, and Jysoo Lee ”HVEM Grid: Experiences in Constructing an Electron Microscopy Grid”

• Governments world-wide promoting global E-science research programs• New Zealand, $43 million to establish the Advanced Network• United States: from Optiputer over Cyber-infrastructure to DDDAS , and DoE ($60M)

• Canada: CA*net4 and i-Infrastructure• Australia: e-Research Initiative (Started Oct 2004)• Europe: 7th framework (€1B/year), GEANT2 (€93M), Netherlight, Geodise, UK E-science, etc.

• Private sector setting a strong pace • Google building their own L1/L2 network• Multi $B investments from large businesses: IBM, HP, Intel,…

• Smaller businesses: Grid-based enterprise applications, data centers solution, commercialization of Globus toolkit,…

E-Science and Grid Opportunity

Network Requirements

• High bandwidth connectivity between supercomputers (teraflops+)

• Large file transfers, over long distances - rethink TCP (FAST)

• Applications/end-users/sensors/instruments requesting optical networking resources host-to-host connections

• Determinism (QoS), jitter and latency requirements (difficult to do with today’s Internet, and may be unfair to transfer terabyte of data to other apps)

• Coordination of network with computational and non-computational resources (CPU, databases, sensors, instruments )

New Demands on NetworksEmerging High-End applications

• Exchange data with sensors via potentially other physical resources. wireless

• Destination may not be known initially rather only a service is requested from source and the destination is derived from the request information

• Mechanisms for retrieving near-real-time information about network resources and network states

• Mechanism for both advance and fast on-the-fly reservation and set-up

• Low latency on-demand connection requests

• Policy and security enforcement in open scientific environments

New Demands on NetworksEmerging High-End applications (cont’d)

• 1000 channels per fiber….. Experimentation with 160G per channel

• Dark Fiber every where …. Paradigm shift on ownership– Fiber is much cheaper…US Headlines: Google buys Fiber

• All-optical switches are getting faster and smaller (ns switch reconfiguration)

• Control Plane protocols, SOA, continue to mature - but should be revisited

• Layer one Optical switches relatively cheaper than other technologies

• Electronic Dispersion Compensation • Fiber, optical impairments control, and

transceiver technology continue to advance while reducing prices

Advances in Optical Technologies(How do we take advantage?)

Research Challenges

• Coordination of resources per request for both on-the-fly and advanced reservations - Network resources is an integral part of the application’s request for shared resources

• Advanced reservation in distributed form - Borrow from ATM research

• Optimization of Resource Allocation

• Interdomain across Global Grid networks - network interdomain protocols, policies (management plane and control plane, Grid … WEB services )

• Dynamic and Adaptive on-demand use of end-to-end networking resources (requires near real-time feedback loop)- Identification of functions and interactions between the control plane, management plane, and Grid middleware

Research Challenges

• Monitoring information of resources - i) identification of information, ii) abstraction of information, and iii) frequency of updates

• Software algorithms to support multiple classes of software including highly-dynamic, workflow engines, data-driven and event-driven applications

• Rethinking the Behavioral Control of Networks • Control/management planes interacting

with middleware• Centralized vs. distributed functionality

Research Challenges (cont’d)

Resource Allocation

Resource ManagerCo-Scheduler

Resource Monitoring

ApplicationsEdge

RoutersWorkflowEngines

Application Abstraction Layer (API)

Policy

Translate app request to policy

•Discovery

•Performance

•Policy

For SLA

MonitoringPolicy

Feedback LoopAbstraction

Distributed Control: Control Plane• Infrastructure and distributed intelligence

that controls the establishment and maintenance of connections in the network, including protocols and mechanisms to disseminate this information; and algorithms for engineering an optimal path between end points.

Control Plane vs. Management Plane

Centralized Control: Management Plane •Management plane mechanisms rely on client/ server model, usually involving one or more management applications (structured hierarchically) communicating to each network element in its domain via a management protocol, (i.e., SNMP, Tl1, XML, etc).

Centralized vs. Distributed…

Key areas for Today’s Control Plane are:1) Provisioning 2) Recovery

Network Management

(Hierarchical )

NENENE

Migration

Centralized (vertical)

Network Management

NENENE

Distributed (Horizontal)

Protocols Protocols

NetworkBehavioral

Control

• Routing - Intra-domain and Inter-domain1) automatic topology and resource discovery 2) path computation (How do we use the infrastructure)

• Signaling - standard communications protocols between network elements for the establishment and maintenance of connections

• Neighbor discovery - NE sharing of details of connectivity to all its neighbors (very powerful tool)

• Local resource management - accounting of local available resources

Control Plane Functions

• Re-thinking control functionality in terms of (centralized or Distributed):

• Information exchanged

• Algorithms for path computation and recovery (CPU power vs. fast reaction time)

• Discovery and advertising of resources

• Scalability (frequency and amount of data exchange)

• Timing (reaction time of events)

• Interdomain interactions (Is BGP the solution or should it be centralized?)

• Policy enabled (where is it residing vs. executing)

Centralized vs. Distributed Behavioral Control of Networks

Examples of Infrastructure for Network Research Experimentation

•NLR•GENI•GLIF

NLRwww.nlr.net

courtesy of Tom West, CEO NLR

To advance the research, clinical and educational goals of members and other institutions by establishing and maintaining a nationwide advanced network infrastructure.

courtesy of Tom West, CEO NLR

National LambdaRail Mission

• Support experimental and production networks

• Foster networking research

• Promote next-generation applications

• Facilitate interconnectivity among high-performance research and education networks

courtesy of Tom West, CEO NLR

National LambdaRail Goals

GENIwww.geni.net

Facility Design: Key ConceptsGENI (Global Environment Network Innovations)

Slicing, Virtualization, Programmability

Mobile Wireless Network

Sensor Network

Edge Site

FederatedInternational Facility

Slide from: Guru Parulkar, CISE, NSF

GENI Facility GoalsEnable exploration of new network architectures, mechanisms, and distributed system capabilities

A shared facility that allows • Concurrent exploration of a broad range of

experimental networks and distributed services

• Interconnection among experimental networks & the commodity Internet

• Users and applications to “opt-in”

• Observation, measurement, and recording of outcomes enabled

Develop stronger scientific base

Slide from: Guru Parulkar, CISE, NSF

GLIFwww.glif.is

GLIF is a collaboration of institutions, organizations, consortia and country National Research and Education Networks (NRENs) who voluntarily share optical networking resources and expertise for the advancement of scientific collaboration and discovery.

GLIF's mission : to create and sustain a Global Facility that supports leading-edge capabilities based on new and emerging technologies and paradigms related to advanced optical networking to enable high-performance applications and services.

What is GLIF?

Global Lambda Integrated Facility

www.glif.is

Visualization courtesy of Bob Patterson, NCSA.

GLIF Automation?

NRENControl Plane

NRENControl Plane

NRENControl Plane

Network Management

Network Management

Network Management

Client A Client B

Grid middleware

Grid middleware

WEBServices ?

Mission: To agree on the interfaces and protocols to automate and use the control planes of the contributed Lambda resources to help users on a global scale access optical resources on-demand or pre scheduled.

GLIF Control Plane and Grid Middleware Integration wg

1. Work with GLIF Tech group top establish what are GLIF resources (GOLEs)

2. Defined Network Elements in RDF3. Software that reads RDF description4. Need to write to Google MAP APIs to draw resources

on a global bases5. Provide algorithm to compute path from broker

information6. Establish WEB services for connection services

GLIF Control Plane and Grid Middleware Integration wg

Thanks to Jereon Van Der Ham

EnLIGHTened ComputingInternational Collaboration

Experiences

EnLIGHTened Computing connectivity diagram with partners

Cisco/UltraLight wave

EnLIGHTened wave (Cisco/NLR)

LONI wave

Members:- MCNC GCNS- LSU CCT-NCSU-(Subcontract) RENCI

Official Partners:- AT&T Research- SURA- NRL- Cisco Systems- Calient Networks- IBM

NSF Project Partners- OptIPuter- UltraLight- WAN-in-LAB- DRAGON- Cheetah

International Partners •LUCIFER - EC•G-Lambda - Japan-GLIF

CHI

HOU

DAL

TUL

KAN

PIT

WDC

OGD

BOI

CLE

POR

DEN

SVL

SEA

Baton Rouge

Raleigh

To Asia To Canada To Europe

L.A.

San Diego

CAVE wave

Chicago

3 EU NRENs are partners + 3 national test-beds + 3 research networks in US and Canada + 5 expressed interest through LoIs

These community representatives are willing to monitor project progress, collaborate and exploit its results

Enlightened/LUCIFER

Sister Projects - Similar Goals

Testbeds

Japan’s G-Lambda research collaboration

Slide: Courtesy of Michiaki HayashiKDDI R&D Laboratories Inc.

Japan’s G-Lambda research collaboration

Slide: Courtesy of Michiaki HayashiKDDI R&D Laboratories Inc.

computeresource

GL Grid EL Grid

GLCRM

computeresource

HARCCRM

networkresource

GLNRM network

resource

HARCNRM

GLGRS

HARCAcceptor

GNS-WSI

GL-CRMI/F

GNS-WSIWrapper

GNS-WSI

HARC-CIFWrapper

GNS-WSI

GL-CRMI/F

HARC NIF

HARC NIF HARC CIF

HARC CIF

HARC NIF

GL-CRMI/F

HARC CIF

GL Request

ELRequest

Slide: Courtesy of Lina Battestilli

Two interfaces emerging1) For Network resources2) For Compute resources

Conclusions

Further Reference

IEEE Communications Magazine

Feature Topic

Optical Control Plane for Grid Networks: Opportunities,

Challenges and the

Vision

Guest Editors: Admela Jukan and Gigi Karmous-Edwards

March, 2006

Vol.44 No.3March 2006

An Optical Control Plane for The Grid Community

A Book written by the Community

Coming soon

Conclusions• Control Plane research is vital to meeting future generation NRENs - with

a strong focus on algorithms to meet the needs of driving applications

• Dynamic reconfigurability of L1/2 is essential to bring down cost and meet application requirements.

• Paradigm Shifts: i) ownership and control of network infrastructure, ii) network resources are treated as an integral Grid resource. - affect Interdomain policies

• The Research networks are taking these bold steps on GLIF, testbed infrastructures… apply lessons learned to production quickly.

• International Collaboration is a very Key ingredient for the future of Scientific discovery - The Optical network plays the most critical role in achieving this!

Yufeng Xin, Steve Thorpe, Bonnie Hurst, Lina Battestilli, Mark Johnson , John Moore, Ed Seidel, Gabriele Allen, Seung Jong (Jay) Park , Jon Maclaren, Andrei Hutanu, Lonnie Leger, Lavanya Ramakrishnan, Joel Dunn, Savera Tanwir, Harry Perros, Javad Boroumand, Russ Gyurek, Wane Clark, Kevin McGrattan, Rick Schlichting, John Strand, Matti Hiltunen, Gary Crane, Hank Dardy, Olivier Jerphagnon, Ron Mackey, John Bowers,Carla Hunt, Andrew Mabe, Gigi Karmous-Edwards

Acknowledgments

The Enlightened Team

Thank You!

Gigi Karmous-Edwardsgigi@mcnc.org

May 22, 2006

EARNEST Foresight WorkshopBerlin