high data volume transfer issues at noaa christopher d. elvidge earth observation group national...
TRANSCRIPT
High Data Volume Transfer Issues at
NOAA
Christopher D. ElvidgeEarth Observation Group
National Oceanic and Atmospheric Administration
National Geophysical Data CenterBoulder, ColoradoAugust 26, 2005
2
NOAA’s Vision and Mission
• VISION – An informed society that uses a comprehensive understanding of the role of the oceans, coasts, and atmosphere in the global ecosystem to make the best social and economic decisions.
• MISSION – To understand and predict changes in the Earth’s environment and conserve and manage coastal and marine resources to meet our Nation’s economic, social, and environmental needs.
3
Benefits of Satellite Earth Observations
Energy Resources
Natural & Human Induced Disasters
Human Health & Well-Being
Climate Variability & Change
Water Resources
Weather Information, Forecasting &
Warning
Terrestrial, Coastal & Marine
Ecosystems
Sustainable Agriculture &
Desertification
Biodiversity
4
NOAA’s Current Satellite Observing Systems
• Polar-orbiting Operational Environmental Satellites (POES) ~10 GB per day.
• Geostationary Operational Environmental Satellites (GOES) ~ 10 GB per day.
• Defense Meteorological Satellite Program (DMSP) ~ 8 GB per day.
5
NOAA’s Future Satellite Observing Systems
• NPOESS Preparatory Project (NPP) ~4 TB per day 2008- 2012.
• National Polar-orbiting Operational Environmental Satellite System (NPOESS) C1 ~4 TB per day 2010-2015.
• National Polar-orbiting Operational Environmental Satellite System (NPOESS) C2 ~4 TB per day 2012-2017.
• National Polar-orbiting Operational Environmental Satellite System (NPOESS) C3 ~4 TB per day 2014-2019.
• Geostationary Operational Environmental Satellites-R (GOES-R) ~ 2 TB per day 2012-2017.
6
Projected NOAA Data Holdings By Year
The NOAA Comprehensive Large Array-data Stewardship System(CLASS) will be the long term
archive for the data from these systems, providing data access for
the scientific community.
8
Hardware and Network Design
• CLASS will have three operational sites (NGDC, NCDC, and NASA IV&V)
• CLASS will have a centralized development environment and a centralized integration and test environment at NSOF
• CLASS will use a Multi-protocol Label Switching (MPLS)-based peerless IP network used for site to site communication
• New hardware and network architecture– Implemented by January 2006– Modular, scalable, and redundant– Addresses security requirements
9
CLASS Multi-Site ConfigurationCLASS Multi-Site Configuration
CISCOSYSTEMS
CISCOSYSTEMS
CISCOSYSTEMS
CISCOSYSTEMS
CISCOSYSTEMS
Fairmont
DMZ
AIX Nodes
Web ServersBoulder
AshevilleData Provider(e.g., IDPS)
Suitland
DMZWeb Servers
AIX Nodes
AIX Nodes
DMZWeb Servers
DMZ
Web Servers
AIX Nodes
CISCOSYSTEMS
Cisco 7606 Router
Cisco 6509 w/ FWSM
OC-12
SX-MM Fiber
Copper GE
MPLS
10
CLASS Distribution Process
11
NGDC will have direct access to the NLR via the Front Range GigaPop
for use in delivering data from CLASS
12
The Data Hog?
The Visible-Infrared Imaging Radiometer Suite (VIIRS) will fly on NPP and each of the NPOESS satellites.
Raw data (RDRs) from a VIIRS will run 116 GB per day.
Radiance calibrated geolocated data (SDRs) will tilt to 815 GB per day.
Environmental data records (EDRs) will bulge towards 2 TB per day.
13
The Challenges
Moving the data within NOAA - from the Suitland to the CLASS nodes.
Providing simple, useable tools to establish and manage large volume data subscriptions.
Transferring large volumes of data to the users, including those in the APAN region.
MAFFIN, Chiba U., U. Tokyo, Tokoku U. in Japan have expressed interest in receiving global VIIRS data!
14
Thanks for Listening
QUESTIONS?