"toward a global interactive earth observing cyberinfrastructure"
DESCRIPTION
"Toward a Global Interactive Earth Observing Cyberinfrastructure". Invited Talk to the 21st International Conference on Interactive Information Processing Systems (IIPS) for Meteorology, Oceanography, and Hydrology Held at the 85th AMS Annual Meeting San Diego, CA January 12, 2005. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: "Toward a Global Interactive Earth Observing Cyberinfrastructure"](https://reader035.vdocuments.mx/reader035/viewer/2022081516/56814575550346895db2469a/html5/thumbnails/1.jpg)
"Toward a Global Interactive Earth Observing Cyberinfrastructure"
Invited Talk to the21st International Conference on Interactive Information Processing
Systems (IIPS) for Meteorology, Oceanography, and HydrologyHeld at the 85th AMS Annual Meeting
San Diego, CAJanuary 12, 2005
Dr. Larry Smarr
Director, California Institute for Telecommunications and Information Technology
Harry E. Gruber Professor,
Dept. of Computer Science and Engineering
Jacobs School of Engineering, UCSD
![Page 2: "Toward a Global Interactive Earth Observing Cyberinfrastructure"](https://reader035.vdocuments.mx/reader035/viewer/2022081516/56814575550346895db2469a/html5/thumbnails/2.jpg)
Abstract
As the earth sciences move toward an interactive global observation capability, a new generation of cyberinfrastructure is required. Realtime control of remote instruments, remote visualization or large data objects, metadata searching of federated data repositories, and collaborative analysis of complex simulations and observations must be possible using software agents interacting with web and Grid services. Several prototyping projects are underway, funded by NSF, NASA, and NIH, which are building national to global scale examples of such systems. These are driven by remote observation and simulation of the solid earth, oceans, and atmosphere with a specific focus on the coastal zone and environmental hydrology. I will review several of these projects and describe the cyber-architecture which is emerging.
![Page 3: "Toward a Global Interactive Earth Observing Cyberinfrastructure"](https://reader035.vdocuments.mx/reader035/viewer/2022081516/56814575550346895db2469a/html5/thumbnails/3.jpg)
Evolutionary Stages of an InteractiveEarth Sciences Architecture
• Library– Asynchronous Access to Instrumental Data
• Web – Synchronous Access to Instrumental Data
• Telescience– Synchronous Access to Instruments and Data
![Page 4: "Toward a Global Interactive Earth Observing Cyberinfrastructure"](https://reader035.vdocuments.mx/reader035/viewer/2022081516/56814575550346895db2469a/html5/thumbnails/4.jpg)
Earth System Enterprise-Data Lives in Distributed Active Archive Centers (DAAC)
SEDAC (0.1 TB)Human Interactions in
Global Change
GES DAAC-GSFC (1334 TB)
Upper AtmosphereAtmospheric Dynamics, Ocean
Color, Global Biosphere, Hydrology, Radiance Data
ASDC-LaRC (340 TB)Radiation Budget,CloudsAerosols, Tropospheric
Chemistry
ORNL (1 TB)Biogeochemical
DynamicsEOS Land Validation
NSIDC (67 TB)Cryosphere
Polar Processes
LPDAAC-EDC (1143 TB)Land Processes
& Features
PODAAC-JPL (6 TB)Ocean Circulation
Air-Sea Interactions
ASF (256 TB)SAR Products
Sea IcePolar Processes
GHRC (4TB)Global
Hydrology
EOS Aura Satellite Has Been LaunchedChallenge is How to Evolve to New Technologies
![Page 5: "Toward a Global Interactive Earth Observing Cyberinfrastructure"](https://reader035.vdocuments.mx/reader035/viewer/2022081516/56814575550346895db2469a/html5/thumbnails/5.jpg)
Challenge: Average Throughput of NASA Data Products to End User is Only < 50 Megabits/s
Tested from GSFC-ICESATJanuary 2005
http://ensight.eos.nasa.gov/Missions/icesat/index.shtml
![Page 6: "Toward a Global Interactive Earth Observing Cyberinfrastructure"](https://reader035.vdocuments.mx/reader035/viewer/2022081516/56814575550346895db2469a/html5/thumbnails/6.jpg)
0
5
10
15
20
25
DO
E
NS
F
DO
E
DO
E
DO
E
DO
E
DO
E
DO
E
NS
F
NO
AA
DO
D
NS
F
DO
D
DO
E
NO
AA
NO
AA
NS
F
DO
E
DO
E
DO
E
DO
E
DO
E
DO
E
NA
SA
NS
F
NA
SA
NA
SA
DO
D
DO
D
NS
F
NS
F
DO
E
NO
AA
DO
E
NS
F
DO
D
DO
E
DO
E
DO
D
DO
E
DO
E
DO
D
DO
E
DO
E
DO
E
NS
F
NA
SA
DO
D
DO
E
NS
F
DO
E
DO
E
DO
D
DO
D
DO
D
NS
F
DO
E
NS
F
Pea
k S
peed
(GF)
Federal Agency Supercomputers Faster Than 1TeraFLOP Nov 2003
DOE
NSF
DOD
NOAANASA
Aggregate Peak Speed
Conclusion: NASA is Underpowered in High-End Computing
For Its Mission
Goddard
AmesJPL
Data From Top500 List (November 2003) Excluding No-name Agencies
From Smarr March 2004 NAC Talk
![Page 7: "Toward a Global Interactive Earth Observing Cyberinfrastructure"](https://reader035.vdocuments.mx/reader035/viewer/2022081516/56814575550346895db2469a/html5/thumbnails/7.jpg)
NASA Ames Brings Leadership to High-End Computing
0
5
10
15
20
25
DO
E
NS
F
DO
E
DO
E
DO
E
DO
E
DO
E
DO
E
NS
F
NO
AA
DO
D
NS
F
DO
D
DO
E
NO
AA
NO
AA
NS
F
DO
E
DO
E
DO
E
DO
E
DO
E
DO
E
NA
SA
NS
F
NA
SA
NA
SA
DO
D
DO
D
NS
F
NS
F
DO
E
NO
AA
DO
E
NS
F
DO
D
DO
E
DO
E
DO
D
DO
E
DO
E
DO
D
DO
E
DO
E
DO
E
NS
F
NA
SA
DO
D
DO
E
NS
F
DO
E
DO
E
DO
D
DO
D
DO
D
NS
F
DO
E
NS
F
Pe
ak
Sp
ee
d (
GF
)
Project Columbia! 60TF
20 x 512-ProcessorSGI Altix Single-System Image
Supercomputers= 10,240 Intel IA-64 Processors
Estimated #1 or 2Top500 (Nov. 2004)
![Page 8: "Toward a Global Interactive Earth Observing Cyberinfrastructure"](https://reader035.vdocuments.mx/reader035/viewer/2022081516/56814575550346895db2469a/html5/thumbnails/8.jpg)
Increasing Accuracy in Hurricane Forecasts Ensemble Runs With Increased Resolution
Operational ForecastResolution of National Weather Service
Higher Resolution Research ForecastNASA Goddard Using Ames Altix
5.75 Day Forecast of Hurricane Isidore
Resolved Eye Wall
Intense Rain-
Bands
4x Resolution
Improvement
Source: Bill Putman, Bob Atlas, GFSC
InterCenter Networking
is Bottleneck
![Page 9: "Toward a Global Interactive Earth Observing Cyberinfrastructure"](https://reader035.vdocuments.mx/reader035/viewer/2022081516/56814575550346895db2469a/html5/thumbnails/9.jpg)
Optical WAN Research Bandwidth Has Grown Much Faster than Supercomputer Speed!
1.E+00
1.E+01
1.E+02
1.E+03
1.E+04
1.E+05
1.E+06
1985 1990 1995 2000 2005
Ba
nd
wid
th (
Mb
ps
)
Megabit/s
Gigabit/s
Terabit/s
Source: Timothy Lance, President, NYSERNet
Full NLR
1 GFLOP Cray2
60 TFLOP Altix
Bandwidth of NYSERNet Research Network Backbones
T1
3210Gb
“Lambdas”
![Page 10: "Toward a Global Interactive Earth Observing Cyberinfrastructure"](https://reader035.vdocuments.mx/reader035/viewer/2022081516/56814575550346895db2469a/html5/thumbnails/10.jpg)
NLR Will Provide an Experimental Network Infrastructure for U.S. Scientists & Researchers
First LightSeptember 2004
“National LambdaRail” PartnershipServes Very High-End Experimental and Research Applications
4 x 10Gb Wavelengths Initially Capable of 40 x 10Gb wavelengths at Buildout
Links Two Dozen
State and Regional Optical
Networks
![Page 11: "Toward a Global Interactive Earth Observing Cyberinfrastructure"](https://reader035.vdocuments.mx/reader035/viewer/2022081516/56814575550346895db2469a/html5/thumbnails/11.jpg)
Global Lambda Integrated Facility:Coupled 1-10 Gb/s Research Lambdas
Predicted Bandwidth, to be Made Available for Scheduled Application and Middleware Research Experiments by December 2004
Visualization courtesy of Bob Patterson, NCSA
www.glif.isCal-(IT)2 Sept 2005
![Page 12: "Toward a Global Interactive Earth Observing Cyberinfrastructure"](https://reader035.vdocuments.mx/reader035/viewer/2022081516/56814575550346895db2469a/html5/thumbnails/12.jpg)
The OptIPuter Project – Creating a LambdaGrid “Web” for Gigabyte Data Objects
• NSF Large Information Technology Research Proposal– Cal-(IT)2 and UIC Lead Campuses—Larry Smarr PI– USC, SDSU, NW, Texas A&M, Univ. Amsterdam Partnering Campuses
• Industrial Partners– IBM, Sun, Telcordia, Chiaro, Calient, Glimmerglass, Lucent
• $13.5 Million Over Five Years• Optical IP Streams From Lab Clusters to Large Data Objects NIH Biomedical Informatics NSF EarthScope
and ORION
http://ncmir.ucsd.edu/gallery.html
siovizcenter.ucsd.edu/library/gallery/shoot1/index.shtml
Research Network
![Page 13: "Toward a Global Interactive Earth Observing Cyberinfrastructure"](https://reader035.vdocuments.mx/reader035/viewer/2022081516/56814575550346895db2469a/html5/thumbnails/13.jpg)
What is the OptIPuter?
• Optical networking, Internet Protocol, Computer Storage, Processing and Visualization Technologies– Dedicated Light-pipe (One or More 1-10 Gbps WAN Lambdas)– Links Linux Cluster End Points With 1-10 Gbps per Node– Clusters Optimized for Storage, Visualization, and Computing– Does NOT Require TCP Transport Layer Protocol – Exploring Both Intelligent Routers and Passive Switches
• Applications Drivers: – Interactive Collaborative Visualization of Large Remote Data Objects
– Earth and Ocean Sciences– Biomedical Imaging
• The OptIPuter Exploits a New World in Which the Central Architectural Element is Optical Networking, NOT Computers - Creating "SuperNetworks"
![Page 14: "Toward a Global Interactive Earth Observing Cyberinfrastructure"](https://reader035.vdocuments.mx/reader035/viewer/2022081516/56814575550346895db2469a/html5/thumbnails/14.jpg)
Currently Developing OptIPuter Software to Coherently Drive 100 MegaPixel Displays
• 55-Panel Display – 100 Megapixel
• Driven by 30 Dual-Opterons (64-bit)
• 60 TB Disk
• 30 10GE interfaces– 1/3 Tera bit/sec!
• Linked to OptIPuter
• We are Working with NASA ARC Hyperwall Team to Unify Software
Source: Jason Leigh, Tom DeFanti, EVL@UICOptIPuter Co-PIs
![Page 15: "Toward a Global Interactive Earth Observing Cyberinfrastructure"](https://reader035.vdocuments.mx/reader035/viewer/2022081516/56814575550346895db2469a/html5/thumbnails/15.jpg)
EVL
10GE OptIPuter CAVEWAVEHelped Launch the National LambdaRail
Next Step: Coupling NASA Centers
to NSF OptIPuter Source: Tom DeFanti, OptIPuter co-PI
![Page 16: "Toward a Global Interactive Earth Observing Cyberinfrastructure"](https://reader035.vdocuments.mx/reader035/viewer/2022081516/56814575550346895db2469a/html5/thumbnails/16.jpg)
Interactive Retrieval and Hyperwall Display of Earth Sciences Images on a National Scale
Earth science data sets created by GSFC's Scientific Visualization Studio were retrieved across the NLR in real time from OptIPuter servers in Chicago and San Diego and from GSFC servers in McLean, VA, and displayed
at the SC2004 in Pittsburgh
Enables Scientists To Perform Coordinated Studies Of
Multiple Remote-Sensing Or Simulation Datasets
http://esdcd.gsfc.nasa.gov/LNetphoto3.html
Source: Milt Halem & Randall Jones, NASA GSFC& Maxine Brown, UIC EVL
Eric Sokolowsky
![Page 17: "Toward a Global Interactive Earth Observing Cyberinfrastructure"](https://reader035.vdocuments.mx/reader035/viewer/2022081516/56814575550346895db2469a/html5/thumbnails/17.jpg)
OptIPuter and NLR will Enable Daily Land Information System Assimilations
• The Challenge:– More Than Dozen Parameters, Produced Six Times A Day,
Need to be Analyzed
• The LambdaGrid Solution:– Sending this Amount of Data to NASA Goddard from
Project Columbia at NASA Ames for Human Analysis Would Require < 15 Minutes/Day Over NLR
• The Science Result:– Making Feasible Running This Land Assimilation System
Remotely in Real Time
Source: Milt Halem, NASA GSFC
![Page 18: "Toward a Global Interactive Earth Observing Cyberinfrastructure"](https://reader035.vdocuments.mx/reader035/viewer/2022081516/56814575550346895db2469a/html5/thumbnails/18.jpg)
U.S. Surface Evaporation Mexico Surface Temperature
Global 1 km x 1 km Assimilated Surface Observations AnalysisRemotely Viewing ~ 50 GB per Parameter
Randall Jones
![Page 19: "Toward a Global Interactive Earth Observing Cyberinfrastructure"](https://reader035.vdocuments.mx/reader035/viewer/2022081516/56814575550346895db2469a/html5/thumbnails/19.jpg)
Next Step: OptIPuter, NLR, and Starlight EnablingCoordinated Earth Observing Program (CEOP)
Note Current Throughput 15-45 Mbps:OptIPuter 2005 Goal is ~1-10 Gbps!
http://ensight.eos.nasa.gov/Organizations/ceop/index.shtml
Accessing 300TB’s of Observational Data in Tokyo and 100TB’s of Model Assimilation Data in MPI in Hamburg -- Analyzing Remote Data Using GRaD-DODS at These Sites Using OptIPuter Technology Over the NLR and Starlight
Source: Milt Halem, NASA GSFC
SIO
![Page 20: "Toward a Global Interactive Earth Observing Cyberinfrastructure"](https://reader035.vdocuments.mx/reader035/viewer/2022081516/56814575550346895db2469a/html5/thumbnails/20.jpg)
Variations of the Earth Surface TemperatureOver One Thousand Years
Source: Charlie Zender, UCI
![Page 21: "Toward a Global Interactive Earth Observing Cyberinfrastructure"](https://reader035.vdocuments.mx/reader035/viewer/2022081516/56814575550346895db2469a/html5/thumbnails/21.jpg)
Prototyping OptIPuter Technologies in Support of the IPCC
• UCI Earth System Science Modeling Facility – Calit2 is Adding ESMF to the OptIPuter Testbed
• ESMF Challenge:– Improve Distributed Data Reduction and Analysis– Extending the NCO netCDF Operators
– Exploit MPI-Grid and OPeNDAP
– Link IBM Computing Facility at UCI over OptIPuter to:– Remote Storage
– at UCSD– Earth System Grid (LBNL, NCAR, ONRL) over NLR
• Support Next IPCC Assessment Report
Source: Charlie Zender, UCI
![Page 22: "Toward a Global Interactive Earth Observing Cyberinfrastructure"](https://reader035.vdocuments.mx/reader035/viewer/2022081516/56814575550346895db2469a/html5/thumbnails/22.jpg)
Components of a Future Global System for Earth Observation(Sensor Web)
Creating an Integrated InteractiveInformation System for Earth Exploration
Focus on Sub-Surface
Networks
![Page 23: "Toward a Global Interactive Earth Observing Cyberinfrastructure"](https://reader035.vdocuments.mx/reader035/viewer/2022081516/56814575550346895db2469a/html5/thumbnails/23.jpg)
LOOKING (Laboratory for the Ocean Observatory Knowledge
Integration Grid) –Integrates Sensors From
Canada and Mexico
New OptIPuter Driver: Gigabit Fibers on the Ocean FloorAdding Web Services to LambdaGrids
www.neptune.washington.edu
(Funded by NSF ITR-John Delaney, UWash, PI)
![Page 24: "Toward a Global Interactive Earth Observing Cyberinfrastructure"](https://reader035.vdocuments.mx/reader035/viewer/2022081516/56814575550346895db2469a/html5/thumbnails/24.jpg)
LOOKING -- Cyberinfrastructure for Interactive Ocean Observatories
• Laboratory for the Ocean Observatory Knowledge INtegration Grid• NSF Information Technology Research (ITR) Grant 2004-2008
– Cooperative Agreements with UW and Scripps/UCSD– Largest ITR Awarded by NSF in 2004
• Principal Investigators– John Orcutt & Larry Smarr - UCSD– John Delaney & Ed Lazowska --UW, Mark Abbott – OSU– Collaborators at MBARI, WHOI, NCSA, UIC, CalPoly, CANARIE, Microsoft,
UVic, NEPTUNE-Canada
• Develop A Working Prototype Cyberinfrastructure for NSF’s ORION – Fully Autonomous Robotic Sensor Network of Interactive Platforms – Capable of Evolving and Adapting to Changes in:
– User Requirements,
– Available Technology
– Environmental Stresses
– During The Life Cycle Of The Ocean Observatory
![Page 25: "Toward a Global Interactive Earth Observing Cyberinfrastructure"](https://reader035.vdocuments.mx/reader035/viewer/2022081516/56814575550346895db2469a/html5/thumbnails/25.jpg)
www.sccoos.org/
LOOKING will Partner with the Southern California Coastal Ocean Observing System
• Cal Poly, San Luis Obispo
• Cal State Los Angeles
• CICESE
• NASA JPL
• Scripps Institution of Oceanography, University of California, San Diego
• Southern California Coastal Water Research Project Authority
• UABC
• University of California, Santa Barbara
• University of California, Irvine
• University of California, Los Angeles
• University of Southern California
![Page 26: "Toward a Global Interactive Earth Observing Cyberinfrastructure"](https://reader035.vdocuments.mx/reader035/viewer/2022081516/56814575550346895db2469a/html5/thumbnails/26.jpg)
Pilot Project ComponentsPilot Project Components
SCCOOS Pilot Project Components
• Moorings• Ships• Autonomous Vehicles • Satellite Remote Sensing• Drifters• Long Range HF Radar • Near-Shore Waves/Currents (CDIP)• COAMPS Wind Model• Nested ROMS Models• Data Assimilation and Modeling• Data Systems
www.sccoos.org/
![Page 27: "Toward a Global Interactive Earth Observing Cyberinfrastructure"](https://reader035.vdocuments.mx/reader035/viewer/2022081516/56814575550346895db2469a/html5/thumbnails/27.jpg)
ROADNet Sensor Types
• Seismometers• Accelerometers• Displacement• Barometric pressure• Temperature• Wind Speed• Wind Direction• Infrasound• Hydroacoustic
• Differential Pressure Gauges• Strain• Solar Insolation• pH• Electric Current• Electric Potential• Dilution of oxygen• Still Camera Images• Codar
![Page 28: "Toward a Global Interactive Earth Observing Cyberinfrastructure"](https://reader035.vdocuments.mx/reader035/viewer/2022081516/56814575550346895db2469a/html5/thumbnails/28.jpg)
ROADNet Architecture
KeplerWeb ServicesSRBAntelope
Frank Vernon, SIO; Tony Fountain, Ilkay Altintas, SDSC
![Page 29: "Toward a Global Interactive Earth Observing Cyberinfrastructure"](https://reader035.vdocuments.mx/reader035/viewer/2022081516/56814575550346895db2469a/html5/thumbnails/29.jpg)
Applying Web Services to the Interactive Earth Observing Vision
Federated System of Ocean Observatory Networks Extending from the Wet Side to a Shore-Based Observatory Control Facilities onto the Internet Connecting to Scientists
and Their Virtual Ocean Observatories
![Page 30: "Toward a Global Interactive Earth Observing Cyberinfrastructure"](https://reader035.vdocuments.mx/reader035/viewer/2022081516/56814575550346895db2469a/html5/thumbnails/30.jpg)
MARS New Gen Cable Observatory Testbed - Capturing Real-Time Basic Environmental Data
Tele-Operated Crawlers
Central Lander
MARS Installation Oct 2005 -Jan 2006
Source: Jim
Bellingham, MBARI