National LightRail: Towards a National Optical Research Network

Marla Meehl

UMC

2/20/03

2

National Light Rail

Dark Fiber National footprint– Building from California via Denver to Atlanta initially and

completing national footprint in a ring back to California as funds and partners available

Serves very high-end experimental and research applications

4 - 10Gb Wavelengths initially

Capable of 40 10Gbps wavelengths at build-out

Partnership model

NLR Footprint and Layer 1 Topology

15808 Terminal, Regen or OADM site (OpAmp sites not shown)Fiber route

PITPIT

PORPOR

FREFRE

RALRAL

WALWAL

NASNASPHOPHO

OLGOLGATLATL

CHICHI

CLECLE

KANKAN

OGDOGD

SACSAC BOSBOSNYCNYC

WDCWDC

STRSTR

DALDAL

DENDEN

LAXLAX

SVLSVL

SEASEA

SDGSDG

JACJAC

4

Goal of NLR

“The fundamental and overriding goal of NLR is to provide an enabling experimental infrastructure for new forms and methods of science and engineering.” – Peter O’Neil

5

Elements of NLR

Enable science that cannot be done today• Distributed Terascale Facility/Extended Terascaled

Facility (DTF/ETF) access• Grid/Distributed computing• Distributed Storage Area Networks (SANs)• Real-time, high-speed data access and service• Distributed coupled modeling

6

Elements of NLR

There are three broad areas of research activity that the proposed NLR infrastructure would facilitate:– Experimental Computational Science and Engineering

• Application Research grid computing, load migration, latency tolerant algorithms, resource aware algorithm selection and scheduling, distributed and multidisciplinary efforts involving multiple length and time scales in the simulations and multiple science and engineering disciplines, very tightly-coupled simulation codes and algorithms, data sensing and collection elements, data repositories, visualization environments, etc.

7

Elements of NLR

High Performance Networking Research• conduct network layer or link-layer research on some

wavelengths, while maintaining a stable link layer and network layer on other wavelengths to conduct research at the transport layer or higher layers

• build the next generation of network protocols using a protocol development environment (PDE)

• conduct research activities in the optronics space itself

Research andDevelopment

Commercialization

Partnerships

Privatization

Today’s Internet

‘NGI’ efforts

Source: Ivan Moura Campos

Research Nets

Evolution of Networking ‘Spiral’

And, Long Term

Common Denominator, Advanced Services Research & Education Network

Bleeding-edge e2e services & facilities for key experiments, programs, projects, researchers & for next gen. tech., architectures, Grids, content, apparatus, etc. ‘Things that haven’t been done before; with, to, on or over networks’.

Leading-Edge ip Services for demanding apps . & middleware

Advanced Services for R&E users

Business,.Gov.& General PublicCommodity Internet

I2-Abilene, GigaPoPs; Fed. ‘NGI’ Nets. Internet-2,GigaPoPs I2-K20

ISP’s, hosting.com’s etc.

Operational High Performance Research Support Networks

CS, Comp. Science & ‘Grid’ researchers,+ major experiments (eg DTF) & net. experimenters + leading edge labs and centers, optics sys.& net. Industry R&D groups + I-WIREPACIFIC LIGHTRAIL +Internet-2 wave projects;

Research

’s,P2p fiber,

& >=10g ip …

Experimental &Developmental

Networks

LEADERS NETWORK TYPE CAPABILITIES/USERS

R&E Network Tiers NETWORK DISCONTINUITIES

12/05/01

Computer Science & Net. Researchers Academic Research, papers, lab. Experiments, spools of fiber In labs & sparse testbeds

10

Drivers and Motivations

“NSF must be prepared to assume a great S&E infrastructure role for the benefit of the Nation.”

An increasing number of researcher and educators, working as individuals and in groups, need to be connected to sophisticated array of facilities, instruments, and data bases.”

Science and Engineering Infrastructure for the 21st Century Report

11

Drivers and Motivations

“Develop and deploy an advanced cyberinfrastructure to enable new Science & Engineering in the 21st century”

“Expand education and training opportunities at new and existing research facilities.”

“Develop interagency plans and strategies to provide high-end high performance computing and networking infrastructure.”

Science and Engineering Infrastructure for the 21st Century Report

12

Drivers and Motivations

Cost

Control

Flexibility

Responsiveness

Innovation

End-to-End networking

Rationale for NLR:

Current market demonstrates willingness of private sector setting stage for strong partnership of academia, corporations and government

Research community in USA and worldwide working on large scale research and development applications calling for increasing bandwidth as evidenced by the Grid and Terascale projects

Unified approach best investment for biggest payoff for the most users

Federal program funding support focused on research and experimental applications needing significant network capacity

Leadership worldwide in research and education increasingly dependent upon large-scale collaborative research activities

Meet the future needs of the research and community most economically

14

Why Fiber?

Capacity needed is not otherwise affordable

Capabilities needed are not available

Cheaper in the long range

Leverage with “carriers”

Insurance against monopoly behavior

Stable and predictable anchor points

15

NLR Service Offerings

Waves!!!!Dark Fiber‘MetaPoP’ capabilitiesDedicated OC-192c or 10GE waves– No wave can interfere with others

Shared IP service– Must be able to tolerate outages– Supports experimental code

Shared Layer 2 serverAccess to 2 unused wavesSubdivided waves for dedicated uses– Delivered as gigabit Ethernet

16

NLR Governance

LLC - Equity Partners

Joint Research Council

Joint Technology Council

Distributed Network Engineering/ Operations

Waves governed/managed by various groups

17

Potential Participants

Carnegie Mellon/PSC

Virginia Tech

Duke

Georgia Tech

Florida Group (FSU lead)

Texas Group (UT Austin lead)

CENIC

NOTE: There are many sites behind each of these lead sites

PNWGP

UCAR

UCAID

18

Technology Partners

Cisco Systems

Level 3

19

Total NLR Costs

$80.2 Million Over 5 Years

20

Westnet LightRail (WLR) Costs

$4.6 Million over 5 yearsWestnet LightRail is a consortium of the FRGP Abilene members and the InterMountain GigaPop (IMG)– University of Utah, Utah State, Utah Education

Network

Share geography and research goalsIMG would become members of the FRGP to help off-set total costs

21

Costs to UCAR/UCAR

Share commodity Internet, engineer, equipment, Abilene, and NLR costs

$100-$200K new dollars per year for UCAR for five years depending on WLR financial commitments

Without WLR consortium the cost to UCAR would be $920K per year for five years

22

UCAR Benefits/RisksBenefits

Enables advanced scientific researchLeast expensive way to obtain DTF/ETF waves to LA or ChicagoPositions participation in NSF solicitations e.g. Experimental Infrastructure Network and Networking Research TestbedsProvides experimental lambda to PSC for jointly funded workFurther aggregation of Abilene and commodity links to reduce future costsFounding partner least cost/ potential profit position

RisksAs founding partner, UCAR provider of services for WLR members - bears more financial & liability risksFRGP and IMG may take different time frames to gather fundsWLR members can’t join corporations of any kindPuts 5 year liability w/o offset asset on our booksSome risk, but still most prudent and cost effective way for UCAR to proceedLiability and financial risks are being further evaluated by legal and finance

Questions?