the optiputer project: from the grid to the lambdagrid invited talk ieee orange county computer...
TRANSCRIPT
The OptIPuter Project: From the Grid to the LambdaGrid
Invited Talk
IEEE Orange County Computer Society
Irvine, CA
October 24, 2005
Dr. Larry Smarr
Director, California Institute for Telecommunications and Information Technologies
Harry E. Gruber Professor,
Dept. of Computer Science and Engineering
Jacobs School of Engineering, UCSD
Abstract
While the Internet and the World Wide Web have become ubiquitous, their shared nature severely limits the bandwidth available to an individual user. However, during the last few years, a radical restructuring of optical networks supporting e-Science projects is beginning to occur around the world. Amazingly, scientists are now able to acquire the technological capability for private, high bandwidth light pipes (termed "lambdas") which create deterministic network connections coming right into their laboratories. These dedicated connections have a number of significant advantages over shared internet connections, including high bandwidth (10Gbps+), controlled performance (no jitter), lower cost per unit bandwidth, and security. By connecting scalable Linux clusters with these lambdas, one essentially creates supercomputers on the scale of a nation or even the planet Earth.
One of the largest research projects on LambdaGrids is the NSF-funded OptIPuter (www.optiputer.net), which uses large medical and earth sciences imaging as application drivers. The OptIPuter has two regional cores, one in Southern California and one in Chicago, which has now been extended to Amsterdam. One aim of the OptIPuter project is to make interactive visualization of remote gigabyte data objects as easy as the Web makes manipulating megabyte-size data objects today. Providing access to individual user laboratories on our university campuses will require new planning for dedicated optical networks as part of the campus fiber build out.
The Grid Links People with Distributed Resources
http://science.nas.nasa.gov/Groups/Tools/IPG
Industry is AdoptingGrid Technology
Major Challenge for Grid Enabled Science: Bandwidth Barriers Between User and Remote Resources
National Partnership for Advanced Computational Infrastructure
Part of the UCSD CRBS Center for Research on Biological Structure
Average File Transfer ~10-50 Mbps Over Internet2 Backbone
NIH’s Biomedical Informatics Research Network
fc *
Solution: Individual 1 or 10Gbps Lightpaths -- “Lambdas on Demand”
(WDM)
Source: Steve Wallach, Chiaro Networks
“Lambdas”
San Francisco Pittsburgh
Cleveland
National Lambda Rail (NLR) and TeraGrid Provides Cyberinfrastructure Backbone for U.S. Researchers
San Diego
Los Angeles
Portland
Seattle
Pensacola
Baton Rouge
HoustonSan Antonio
Las Cruces /El Paso
Phoenix
New York City
Washington, DC
Raleigh
Jacksonville
Dallas
Tulsa
Atlanta
Kansas City
Denver
Ogden/Salt Lake City
Boise
Albuquerque
UC-TeraGridUIC/NW-Starlight
Chicago
International Collaborators
NLR 4 x 10Gb Lambdas Initially Capable of 40 x 10Gb wavelengths at Buildout
NSF’s TeraGrid Has 4 x 10Gb Lambda Backbone
Links Two Dozen State and Regional Optical
Networks
DOE, NSF, & NASA
Using NLR
The Global Lambda Integrated Facility (GLIF) Creates MetaComputers on the Scale of Planet Earth
Many Countries are Interconnecting Optical Research Networks
to form a Global SuperNetwork
www.glif.is
Created in Reykjavik, Iceland 2003
www.glif.is
Created in Reykjavik, Iceland 2003
September 26-30, 2005Calit2 @ University of California, San Diego
California Institute for Telecommunications and Information Technology
The Networking Double Header of the Century Is Driven by LambdaGrid Applications
iGrid
2oo5T H E G L O B A L L A M B D A I N T E G R A T E D F A C I L I T Y
Maxine Brown, Tom DeFanti, Co-Organizers
www.startap.net/igrid2005/
http://sc05.supercomp.org
Lambdas Enable First Remote Interactive High Definition Video Exploration of Deep Sea Vents
Source John Delaney & Deborah Kelley, UWash
Canadian-U.S. Collaboration
The OptIPuter Project:Sun’s Slogan Realized…
Really
“When the Network is as fast as the computer’s internal links,
the machine disintegrates across the Net into a set of special purpose appliances”
-Gilder Technology Report June 2000
The OptIPuter -- From the Grid to the LambdaGrid:High Resolution Portals to Global Science Data
Green: Purkinje CellsRed: Glial CellsLight Blue: Nuclear DNA
Source: Mark
Ellisman, David Lee,
Jason Leigh
300 MPixel Image!
Calit2 (UCSD, UCI) and UIC Lead Campuses—Larry Smarr PIPartners: SDSC, USC, SDSU, NW, TA&M, UvA, SARA, KISTI, AIST
Scalable Displays Allow Both Global Content and Fine Detail
Source: Mark
Ellisman, David Lee,
Jason Leigh
30 MPixel SunScreen Display Driven by a 20-node Sun Opteron Visualization Cluster
Allows for Interactive Zooming from Cerebellum to Individual Neurons
Source: Mark Ellisman, David Lee, Jason Leigh
Calit2 Is Applying OptIPuter Technologiesto Post-Hurricane Recovery
Working with NASA, USGS, NOAA, NIEHS, EPA, SDSU, SDSC, Duke, …
Calit2 @ UCI Has the Largest Tiled Display Wall
Calit2@UCI Apple Tiled Display WallDriven by 25 Dual-Processor G5s
50 Apple 30” Cinema Displays200 Million Pixels of Viewing Real Estate!
Source: Falko Kuester, Calit2@UCINSF Infrastructure Grant
Data—One Foot Resolution USGS Images of La Jolla, CA
HDTV
Digital Cameras Digital Cinema
The Great Wall of TV…www.ocregister.com/ocregister/news/article_726920.php
OptIPuter Research Enables Massive Data Mining
• Developing New Data Mining Algorithms for Massive Scientific Data Sets, Using Optiputer for High-Speed Remote Data Streaming, & Multi-Tile Display Walls for Visualization
Source: Padhraic Smyth, UCI
Visualization of Brain Image Data
• Data Streaming in Over OptIPuter Links from Remote Sites
• Research Challenge: – How to Effectively Combine:
– Computational Power,
– Pixel Real-estate,
– Human Visual Capabilities
– To Develop New Paradigms for Exploratory Data Analysis
Brain Imaging (Schizophrenia) Kuester in Collaboration with the UCI Brain Imaging Center (BIC) and BIRN
Source: Padhraic Smyth, Falko Kuester, UCI
Variations of the Earth Surface TemperatureOver One Thousand Years—THE Challenge of the 21st Century
Source: Charlie Zender, UCI
Applying OptIPuter Technologies to Support Global Change Research
• UCI Earth System Science Modeling Facility (ESMF)– NSF’s CISE Science and Engineering Informatics Program Funded
ESMF and Calit2 to Improve Distributed Data Reduction & Analysis – Calit2 and UCI is Adding ESMF to the OptIPuter Testbed– Link to Calt2@UCI HiPerWall
• The Resulting Scientific Data LambdaGrid Toolkit will Support the Next Intergovernmental Panel on Climate Change (IPCC) Assessment Report
Source: Charlie Zender, UCI
HIPerWall & The GridAllows High Performance Linkages to National Digital Assets
UCI is Adding Real Time Control to the Calit2 OptIPuter Testbed
• Application Development Experiments Requires Institutional Collaboration– An Experiment for Remote Access and Control within the UCI Campus– A Step Toward Preparation of an Experiment for Remote Access and Control
of Electron Microscopes at UCSD-NCMIR
CalREN-HPR
CalREN-HPR
ChiaroEnstara
UCSD
Microscope(NCMIR)
10 Gb1 Gb
x2CalREN-XD
UC Irvine
Cam
pus
Bac
kbon
e
SPDSCluster
HIPerWall
Storage &Rendering
Cluster
Source: Steve Jenks, Kane Kim, Falko Kuester UCI
UCI DREAM Lab
First Trans-Pacific Super High Definition Telepresence Meeting in New Calit2 Digital Cinema Auditorium
Keio University President Anzai
UCSD Chancellor Fox
Lays Technical Basis for
Global Digital
Cinema
Sony NTT SGI
OptIPuter Scalable Adaptive Graphics Environment (SAGE) Allows Integration of HD Streams
LambdaCam Used to Capture the Tiled Display on a Web Browser
• HD Video from BIRN Trailer
• Macro View of Montage Data
• Micro View of Montage Data
• Live Streaming Video of the RTS-2000 Microscope
• HD Video from the RTS Microscope Room
Source: David Lee, NCMIR, UCSD
SAGE Developed Under
Jason Leigh, EVL
Extending Telepresence with Remote Interactive Analysis of Data Over NLR
HDTV Over Lambda
OptIPuter Visualized
Data
SIO/UCSD
NASA Goddard
www.calit2.net/articles/article.php?id=660
August 8, 2005
25 Miles
Venter Institute
Two New Calit2 Buildings Will Provide a Persistent Collaboration “Living Laboratory”
• Over 1000 Researchers in Two Buildings– Linked via Dedicated Optical Networks– International Conferences and Testbeds
• New Laboratory Facilities– Virtual Reality, Digital Cinema, HDTV– Nanotech, BioMEMS, Chips, Radio, Photonics,
Grid, Data, Applications
Bioengineering
UC San Diego
UC Irvine
Learning to Live on Lambdas
The OptIPuter Enabled Collaboratory:Remote Researchers Jointly Exploring Complex Data
OptIPuter will ConnectFalko Kuester’s
Calit2@UCI 200M-Pixel Wall and the 30M-Pixel Displayat UCSD Ellisman’sBIRN Laboratories
With Shared Fast Deep Storage
“SunScreen” Run by Sun Opteron Cluster
UCI
UCSD