1

Supporting Research on Today’s Optical Network Infrastructure

Presented by

Jerry Keith

Director of Communications

University of California, Riverside

UCCSAug. 2, 2004

2

The Story of the TrailerThe Story of the Trailer

• The Call

• The Visits

• Another Call

• New Information

• Visit Again

• What We Did

3

The CallThe Call

– One hot customer – had talked to a number of people, not satisfied

– Well punctuated conversation

– By the end of the conversation I knew exactly how important this project was

4

The VisitsThe Visits

• Visit from desktop staff

• Visit from network staff

• Actual hubs, put together locally– They asserted it had been working fine

• Theory of virus attack– There really was one, we cured it

• We relaxed – problem solved

5

Another CallAnother Call

• Different PI

• More facts, specific scenarios

• The person made sense

• The virus had not been the problem

• It was time for another visit

6

Visit AgainVisit Again

• Asked staff to return and look again

• We sent a multidiscipline team

• Looked farther afield

• Found additional facts

7

New InformationNew Information

– Local copper T1 on less than Cat 3– Shared a small router with all other T1’s– Interference from large, multi-day transfer

into off-campus site– Researchers had moved their two projects

voluntarily to facilitate function in their home department

8

What We Did & Are DoingWhat We Did & Are Doing

– Installed a Wireless hop and new switch gear to improve the local situation

– Moving the off campus facility with heavy traffic to dark fiber we are leasing from the City of Riverside

– Move this project’s servers to an on-campus site with good connectivity

9

The Story of the Trailer - FindingsThe Story of the Trailer - Findings

• Wrong place– Temporary facilities—no fiber planned– Nearly impossible to upgrade the copper path

• Wrong technology – T1 not Ethernet– Antiquated router, OK for low use T1s

• Wrong network assumptions– PIs assumed they could move anywhere on campus and have service

• Wrong support structure– Sent one discipline at a time– No formal mechanism to get wider set of expertise on the problem quickly

• Wrong planning– Could say, they should have asked us– Our business is research, their processing must work– We believe that we must be proactive in this arena

10

The Story of the TrailerThe Story of the Trailer

• All was well until the Research units moved in: – E-mail worked– Web surfing worked– Administrative applications also

11

Research & SystemsResearch & Systems

• The relationship between research and systems is different than a few years ago:

– more emphasis on computing and communications, joint or matrixed computing efforts are common

– This essential because it would take lifetimes to analyze some data sets without improved technology

12

Research & SystemsResearch & Systems

• Faster• More powerful

– Grid computing at many levels– E.g. Teragrid

ComputingComputing

13

Research & SystemsResearch & Systems

• Faster• More Capacity• Relatively Cheaper• Transmission alternatives – WDM• Can be structured for specific purposes

NetworksNetworks

14

The CalREN NetworkThe CalREN Network

CENIC CENIC ExperimentalExperimental

NetworkNetwork

CENIC High CENIC High PerformancePerformance

Research Research NetworkNetwork

CENIC DigitalCENIC DigitalCaliforniaCalifornia

NetworkNetwork

CalREN XPCalREN XP

CalREN HPRCalREN HPR

CalREN DCCalREN DC

The CalREN network has three distinct service structures to support day to day requirements as well as research needs.

15

CalREN StructureCalREN Structure

• The CalREN backbone is based on dark fiber optic lines throughout the state

• Each network runs on its own wavelength or color of light

• The technology is called DWDM or Dense Wave Division Multiplexing

• It is very cost effective to expand this type of network either for more speed or to add additional networks.

• The next slide depicts this schematically ----

16

CalREN DC/HPR/NLR PoP CalREN DC/HPR/NLR PoP ArchitectureArchitecture

Long HaulOC48/OC192/10GigE

DWDM

DWDM

Gig E10 Gig E or OC192

CalREN DCHPR

CalREN DC & HPR

NLR

Existing7500

Campus or Metro InterconnectOne of above options used for

Each campus

15500

CalREN/DC HPR NLR

Experimental10Gig E

Gig E

17

CalREN BackboneCalREN Backbone TopologyTopology

Nat. Lambda Rail 10G

Tier 2 10G

Tier 3 2.5G

Metro ring dark fiber (Tiers 2 and 3)

Chico

San Diego

Sacramento

Stockton/Modesto

PAIX-Palo Alto

Bakersfield

Fresno

San Luis Obispo

Los Angeles

Santa Barbara

Salinas

Anaheim

SF

Emeryville

SDSC

Portland

Seattle

WestEd Santa Ana

Sunnyvale

8 Nov 2001

UCSC

UCSB

CPSLO

UCM

A Statewide View of the CalREN Backbone

18

Campus ConnectionsCampus Connections

• The next slide shows the campus connections

• Each Campus has two separate connections to CalREN HPR and to CalREN DC

• Each campus can reflect this structure onto their campus net as they see fit.

-these slides courtesy of Gregg Scott at CENIC

19

Default Last Mile Design SpecificationDefault Last Mile Design Specification

Hub Site“B”

Tier 2

Tier 2

Tier 3

Tier 3

CENICHub Site

“A”

CENIC

CENIC

CENIC

ONI Demarc

Campus Network

Gig Enet

Campus/Institution Last Miles

Gig Enet

20

CENIC CENIC ExperimentalExperimental

NetworkNetworkCENIC High CENIC High PerformancePerformance

Research Research NetworkNetwork

CENIC DigitalCENIC DigitalCaliforniaCalifornia

NetworkNetwork

CalREN XPCalREN XP

CalREN CalREN

HPRHPR

CalREN DCCalREN DC

CalREN DC provides internet connectivity for typical networking needs, CalREN HPR is focused on providing high performance for research while the XP network opens the door to experimenting with networks or very unique connectivity for projects.

CalREN NetworksCalREN Networks

21

ResearchResearch

• Impact of Research on Networks and Computing– Moving data– Processing Data– Testing protocols and transmission

• Not every researcher understands the recent improvements in networks or how to use them.

22

Interesting sitesInteresting sites

• TeraGrid – http://www.teragrid.org/

• National LambdaRail [NLR]– http://www.nlr.net/

23

A Quick Sidebar on UCR FiberA Quick Sidebar on UCR Fiber

• UCR uses air-blown fiber [ABF] from Sumitomo• Install tubing, then blow fiber strands with

compressed air• Possible to pull out and replace with ease• Easy to add new runs by blowing new strands• Used by Getty Museum, Department of Defense,

and others

24

Air Blown FiberAir Blown Fiber

25

Air Blown FiberAir Blown Fiber

26

Another Researcher CallsAnother Researcher Calls

• Slow data transfers

• We thought we knew the solution– New fiber to building (ABF)– New Switch for building entry– New gigabit connection to campus core

• This appears to be the solution

• But, you don’t always know

27

Who knows what’s out there?Who knows what’s out there?

WorkstationWorkstation

FirewallServerServer

Firewall

• Firewalls, slow networks, broken protocols, old servers, etc.

• To support research you need to know

28

End to End Performance TestingEnd to End Performance Testing

• Interenet2 has End to End Performance Initiative

• Look on www.internet2.edu– Click on link “E2EPI”

• They initially thought this would be a short stop-gap

• There is lots to read about here

29

E2EPI GoalsE2EPI Goals

• Enable end-users and support personnel to:– Determine E2E performance capabilities– Locate E2E problems– Contact the right person to get an E2E problem

resolved

• Enable remote initiation of partial path performance tests

• Make partial path performance data publicly available

30

Basic End to End PerformanceBasic End to End Performance

31

Support of local users and researchersSupport of local users and researchers

• Good documentation

• Team approach – multi discipline

• Communicate effectively with end users

• Develop relationship with VC for Research

• Standards for networking assumptions made in grant requests

• Maintain E2E perspective

32

Research Opportunities Facilitated by CalREN Research Opportunities Facilitated by CalREN HPR and XP ArchitectureHPR and XP Architecture

An Opportunity to think about Research and Extramurally Funded Opportunities in a New Way

• Creating Private, Ad Hoc Networks

• Data Collection Devices, Servers, Data, and Databases can be linked across campus, the UC system, or even throughout the United States via ultra-high speed networks.

UC’s ability to create ad hoc, private networks (on-campus and within the UC system) allows researchers to engage in unique inter- and intra-campus collaborations, to share equipment and systems, and to address problems and propose solutions utilizing innovative methodologies. Ultimately, UC’s network infrastructure provides researchers with an important tool to help secure research (and

other extramurally funded) opportunities.

33

One Research Opportunity Facilitated by UCR’s One Research Opportunity Facilitated by UCR’s Network InfrastructureNetwork Infrastructure

UCR’s Infomine Project, A Campus Example

• This extramurally funded project will build “virtual library collections” for campus faculty through focused web “crawling” utilizing very high amounts of bandwidth.

How does UCR’s Network Facilitate this Effort?

• Air Blown Fiber.

• Electronics Supporting Direct Connection to Internet 2.

• Network Traffic Shaping to Hold Down Costs.

• This project will consume double the existing amount of campus network bandwidth WITHOUT IMPACTING THE “PRODUCTION NETWORK” AND WITHOUT DRAMATICALLY INCREASED INTERNET COSTS.

34

Logical Overview of “Private Network” ProvidingLogical Overview of “Private Network” ProvidingBandwidth to Focused Web Crawling InitiativeBandwidth to Focused Web Crawling Initiative

35

Research Opportunities Facilitated via UCR’s Research Opportunities Facilitated via UCR’s Network InfrastructureNetwork Infrastructure

• Why Are Private Research Networks Important?

• Dedicated Bandwidth• Not Subject to “Normal”

Campus Rules• Can be Constructed to

Individual Project or Researcher’s Needs

PrivatePrivateExperimental Experimental NetworkNetwork

Private andPrivate andHigh AvailabilityHigh AvailabilityResearch NetworkResearch Network

High AvailabilityHigh AvailabilityAcademic andAcademic andAdministrativeAdministrativeNetworkNetwork

CENIC CENIC ExperimentalExperimental

NetworkNetwork

CENIC High CENIC High PerformancePerformance

Research Research NetworkNetwork

CENIC DigitalCENIC DigitalCaliforniaCalifornia

NetworkNetwork

CaliforniaCalifornia UCRUCR

36

Next Steps for UCR…Next Steps for UCR…

Cross-functional team to support this effort.

• Network, Systems, and Security.

• Grant Writing Expertise.

• Collaboratory Support for Research Design and Methodology Support.

• Announcement in the Fall 2004.

37

For further consideration …For further consideration …

• Interesting Projectshttp://oceanstore.cs.berkeley.edu/info/overview.html

The Archipelago Project investigates efficient ways to form a secure extended ad-hoc network of laptops, handhelds, and other wireless capable devices, and bridge it to the Internet.

The Oceanstore Project

Using Networks to promote Collaborative Research

The Archipelago Project

http://www-itg.lbl.gov/~johnston/Virtual.Labs.html

http://www.cnds.jhu.edu/research/networks/archipelago/

* Comments and Questions are welcome:

jerry.keith @ucr.edu