peter cornillon university of rhode island presented at the 10 september 2003 nvods workshop
DESCRIPTION
The National Virtual Ocean Data System (NVODS): A Status Report. Peter Cornillon University of Rhode Island Presented at the 10 September 2003 NVODS Workshop Washington DC. Workshop Objectives. Inform the community – NVODS status. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Peter Cornillon University of Rhode Island Presented at the 10 September 2003 NVODS Workshop](https://reader036.vdocuments.mx/reader036/viewer/2022062301/56814558550346895db22790/html5/thumbnails/1.jpg)
Peter CornillonPeter CornillonUniversity of Rhode IslandUniversity of Rhode Island
Presented at the 10 September 2003 Presented at the 10 September 2003 NVODS WorkshopNVODS Workshop
Washington DCWashington DC
The National Virtual Ocean The National Virtual Ocean Data System (NVODS): Data System (NVODS):
A Status ReportA Status Report
![Page 2: Peter Cornillon University of Rhode Island Presented at the 10 September 2003 NVODS Workshop](https://reader036.vdocuments.mx/reader036/viewer/2022062301/56814558550346895db22790/html5/thumbnails/2.jpg)
Workshop ObjectivesWorkshop Objectives
Inform the community – NVODS status
Solicit input with regard to the future direction of NVODS.
Encourage participation in the NVODS effort.
![Page 3: Peter Cornillon University of Rhode Island Presented at the 10 September 2003 NVODS Workshop](https://reader036.vdocuments.mx/reader036/viewer/2022062301/56814558550346895db22790/html5/thumbnails/3.jpg)
Workshop ObjectivesWorkshop Objectives1. Develop a Comprehensive IOOS Data Model
2. Deliver time-critical (real-time) data to data assembly and operational modeling sites
3. Develop DMAC Middleware
4. Make data available using IOOS middleware solution
5. Data Manipulation Services
6. Develop Metrics and Implement Performance Monitoring
7. Implement Middleware Security (Cross-discipline effort with all DMAC)
8. Provide guaranteed geo-temporal-referenced browse for all IOOS data
![Page 4: Peter Cornillon University of Rhode Island Presented at the 10 September 2003 NVODS Workshop](https://reader036.vdocuments.mx/reader036/viewer/2022062301/56814558550346895db22790/html5/thumbnails/4.jpg)
To Inform - OutlineTo Inform - Outline
Overview of NVODS
OPeNDAP – the NVODS core infrastructure
NVODS/OPeNDAP Status
The future
![Page 5: Peter Cornillon University of Rhode Island Presented at the 10 September 2003 NVODS Workshop](https://reader036.vdocuments.mx/reader036/viewer/2022062301/56814558550346895db22790/html5/thumbnails/5.jpg)
What is NVODS?What is NVODS?
NVODS is operated from URI, but involves significant participation from a number of oceanographic institutions in the US and abroad.
NVODS involves for oceanographic data all data system aspects from data discovery through delivery to the user’s application.
Data transport within NVODS is based on the OPeNDAP data access protocol.
![Page 6: Peter Cornillon University of Rhode Island Presented at the 10 September 2003 NVODS Workshop](https://reader036.vdocuments.mx/reader036/viewer/2022062301/56814558550346895db22790/html5/thumbnails/6.jpg)
Specifically, NVODSSpecifically, NVODS Facilitates access to oceanographic data and data
products via the Internet regardless of: data type, where the data are stored, the format in which they are stored, the user's visualization package, or the user's level of expertise,
Is envisioned as a component of IOOS, the Integrated Ocean Observing System,
![Page 7: Peter Cornillon University of Rhode Island Presented at the 10 September 2003 NVODS Workshop](https://reader036.vdocuments.mx/reader036/viewer/2022062301/56814558550346895db22790/html5/thumbnails/7.jpg)
NVODSNVODS
Has been desgined to evolve with
the Internet, computer hardware, computer software, and the interests of its users,
Involves a major population effort, and
Makes use of the GCMD, the Global Change Master Directory, for data discovery.
![Page 8: Peter Cornillon University of Rhode Island Presented at the 10 September 2003 NVODS Workshop](https://reader036.vdocuments.mx/reader036/viewer/2022062301/56814558550346895db22790/html5/thumbnails/8.jpg)
NVODS does NVODS does NOTNOT deal with deal with
The acquisition of data at the sensor,
Issues related to the long term storage of these data,
A system processing capability,
The actual network (hardware) infrastructure, nor
The very low level data transport protocols, such as tcp/ip.
![Page 9: Peter Cornillon University of Rhode Island Presented at the 10 September 2003 NVODS Workshop](https://reader036.vdocuments.mx/reader036/viewer/2022062301/56814558550346895db22790/html5/thumbnails/9.jpg)
NVODS and the End-to-End Data SystemNVODS and the End-to-End Data System
![Page 10: Peter Cornillon University of Rhode Island Presented at the 10 September 2003 NVODS Workshop](https://reader036.vdocuments.mx/reader036/viewer/2022062301/56814558550346895db22790/html5/thumbnails/10.jpg)
NVODS Evolved from DODSNVODS Evolved from DODS
DODS, the Distributed Oceanographic Data System begun in the early 1990s, consisted of two fundamental parts:
a discipline independent core infrastructure for moving data on the net,
a discipline specific portion related to data –population, location, specialized clients, etc.
![Page 11: Peter Cornillon University of Rhode Island Presented at the 10 September 2003 NVODS Workshop](https://reader036.vdocuments.mx/reader036/viewer/2022062301/56814558550346895db22790/html5/thumbnails/11.jpg)
DODS DODS NVODS & OPeNDAP NVODS & OPeNDAP
To isolate the discipline independent part of the system from the discipline specific part, two entities have been formed:
National Virtual Ocean Data System (NVODS)
Open Source Project for a Network Data Access Protocol (OPeNDAP)
![Page 12: Peter Cornillon University of Rhode Island Presented at the 10 September 2003 NVODS Workshop](https://reader036.vdocuments.mx/reader036/viewer/2022062301/56814558550346895db22790/html5/thumbnails/12.jpg)
OPeNDAPOPeNDAP
OPeNDAP was formed to maintain and evolve the DODS core infrastructure,
OPeNDAP is a non-profit corporation, and
OPeNDAP is a discipline neutral data access protocol.
![Page 13: Peter Cornillon University of Rhode Island Presented at the 10 September 2003 NVODS Workshop](https://reader036.vdocuments.mx/reader036/viewer/2022062301/56814558550346895db22790/html5/thumbnails/13.jpg)
OPeNDAP in the End-to-End Data SystemOPeNDAP in the End-to-End Data System
![Page 14: Peter Cornillon University of Rhode Island Presented at the 10 September 2003 NVODS Workshop](https://reader036.vdocuments.mx/reader036/viewer/2022062301/56814558550346895db22790/html5/thumbnails/14.jpg)
NVODS/OPeNDAPNVODS/OPeNDAPStatusStatus
![Page 15: Peter Cornillon University of Rhode Island Presented at the 10 September 2003 NVODS Workshop](https://reader036.vdocuments.mx/reader036/viewer/2022062301/56814558550346895db22790/html5/thumbnails/15.jpg)
OPeNDAP/NVODS Server Sites
![Page 16: Peter Cornillon University of Rhode Island Presented at the 10 September 2003 NVODS Workshop](https://reader036.vdocuments.mx/reader036/viewer/2022062301/56814558550346895db22790/html5/thumbnails/16.jpg)
Institutions with OPeNDAP ServersInstitutions with OPeNDAP Servers
Carolinas Coastal Ocean Observing and Prediction System (Caro-COOPS) Center for Ocean Land Atmosphere Studies (COLA) Columbia University/LDEO - International Research Institute (IRI/LDEO) Florida State University - Center for Ocean-Atmospheric Prediction Studies (COAPS) George Mason University - Seasonal to Interannual Earth Science Information Partner Gulf of Maine Ocean Observing System (GoMOOS) Maine - Department of Marine Resources (Maine DMR) Monterey Bay Aquarium Research Institute (MBARI) Naval Oceanographic Office (NAVOCEANO) NASA/GSFC - Goddard Distributed Active Archive Center (GDAAC) NASA/JPL - Physical Oceanography Distributed Active Archive Center (PODAAC) NASA/JPL - Ocean Earth Science Information Partner (OceanESIP) NCAR - Vegetation/Ecosystem Modeling and Analysis Project (VEMAP) New York University - Center of Atmosphere Ocean Science NOAA - Climate Diagnostic Center (CDC) NOAA - Geophysical Fluid Dynamics Laboratory (GFDL) NOAA - National Center for Environmental Prediction (NCEP)
Universities/NGO Research Labs State Organizations
NOAA NASA
USGS Navy
![Page 17: Peter Cornillon University of Rhode Island Presented at the 10 September 2003 NVODS Workshop](https://reader036.vdocuments.mx/reader036/viewer/2022062301/56814558550346895db22790/html5/thumbnails/17.jpg)
Institutions with OPeNDAP ServersInstitutions with OPeNDAP Servers
NOAA - National Climatic Data Center (NCDC) NOAA - National Data Buoy Center (NDBC) NOAA - Northeast Fisheries Science Center (NMFS/NEFSC) NOAA - Pacific Marine Environmental Laboratory (PMEL) Oregon State University - Remote Sensing Ocean Optics (ORSOO) Science Applications International Corporation (SAIC) Texas A&M University (TAMU) UCAR - Unidata Internet Data Distribution (IDD) United States Geological Survey - Woods Hole Field Center (USGS) University of Alabama/Huntsville - Global Hydrology Resource Center (GHRC) University of Hawaii/SOEST - International Pacific Research Center University of Hawaii/SOEST - Sea Level Center University of Kansas - Kansas Geological Survey University of Massachusetts - School of Marine Science and Technology Universite Pierre et Marie Curie - Laboratoire de Meteorlogie Dynamique du C.N.R.S University of Rhode Island - Graduate School of Oceanography Woods Hole Oceanographic Institution - US/GLOBEC Georges Bank Program (GLOBEC) Woods Hole Oceanographic Institution - Upper Ocean Mooring Data Archive (UOP)
Universities/NGO Research Labs State Organizations
NOAA NASA
USGS Navy
![Page 18: Peter Cornillon University of Rhode Island Presented at the 10 September 2003 NVODS Workshop](https://reader036.vdocuments.mx/reader036/viewer/2022062301/56814558550346895db22790/html5/thumbnails/18.jpg)
![Page 19: Peter Cornillon University of Rhode Island Presented at the 10 September 2003 NVODS Workshop](https://reader036.vdocuments.mx/reader036/viewer/2022062301/56814558550346895db22790/html5/thumbnails/19.jpg)
OPeNDAP Client and Server StatusOPeNDAP Client and Server Status
![Page 20: Peter Cornillon University of Rhode Island Presented at the 10 September 2003 NVODS Workshop](https://reader036.vdocuments.mx/reader036/viewer/2022062301/56814558550346895db22790/html5/thumbnails/20.jpg)
Special ServersSpecial Servers
![Page 21: Peter Cornillon University of Rhode Island Presented at the 10 September 2003 NVODS Workshop](https://reader036.vdocuments.mx/reader036/viewer/2022062301/56814558550346895db22790/html5/thumbnails/21.jpg)
0
200
400
600
800
1000
1200
1400
GDS Unique Users per Month
![Page 22: Peter Cornillon University of Rhode Island Presented at the 10 September 2003 NVODS Workshop](https://reader036.vdocuments.mx/reader036/viewer/2022062301/56814558550346895db22790/html5/thumbnails/22.jpg)
0
200,000
400,000
600,000
800,000
1,000,000
1,200,000
1,400,000
1,600,000
1,800,000
Sep-01
Oct-01
Nov-01
Dec-01
Jan-02
Feb-02
Mar-02
Apr-02
May-02
Jun-02
Jul-02
Aug-02
Sep-02
Oct-02
Nov-02
Dec-02
Jan-03
Feb-03
Mar-03
Apr-03
May-03
Jun-03
GDS Data Access per Month
![Page 23: Peter Cornillon University of Rhode Island Presented at the 10 September 2003 NVODS Workshop](https://reader036.vdocuments.mx/reader036/viewer/2022062301/56814558550346895db22790/html5/thumbnails/23.jpg)
0
10
20
30
40
50
60
70
80
90
Dat
a V
olu
me
(GB
)
Sep-01
Oct-01
Nov-01
Dec-01
Jan-02
Feb-02
Mar-02
Apr-02
May-02
Jun-02
Jul-02
Aug-02
Sep-02
Oct-02
Nov-02
Dec-02
Jan-03
Feb-03
Mar-03
Apr-03
May-03
Jun-03
GDS Data Volume Delivered per Month
![Page 24: Peter Cornillon University of Rhode Island Presented at the 10 September 2003 NVODS Workshop](https://reader036.vdocuments.mx/reader036/viewer/2022062301/56814558550346895db22790/html5/thumbnails/24.jpg)
00,000
50,000
100,000
150,000
200,000
250,000
Sep-01
Oct-01
Nov-01
Dec-01
Jan-02
Feb-02
Mar-02
Apr-02
May-02
Jun-02
Jul-02
Aug-02
Sep-02
Oct-02
Nov-02
Dec-02
Jan-03
Feb-03
Mar-03
Apr-03
May-03
Jun-03
GDS Analysis Expressions per Month
![Page 25: Peter Cornillon University of Rhode Island Presented at the 10 September 2003 NVODS Workshop](https://reader036.vdocuments.mx/reader036/viewer/2022062301/56814558550346895db22790/html5/thumbnails/25.jpg)
Interesting OPeNDAP Access StatisticsInteresting OPeNDAP Access Statistics
LDEO data accesses for 1st quarter of 2002
Type Requests % Volume (gb) %
OPeNDAP
191,611 8.5 375.2 69.4
Other 2,062,681 91.5 165.2 30.6
Total 2,254,292 100.0 540.4 100.0
Type Requests % Volume (gb) %
OPeNDAP
643,892 16.5 880.2 79.6
Other 3,264,729 83.5 225.9 20.4
Total 3,908,621 100.0 1,106.1 100.0
LDEO data accesses for 4th quarter of 2002
![Page 26: Peter Cornillon University of Rhode Island Presented at the 10 September 2003 NVODS Workshop](https://reader036.vdocuments.mx/reader036/viewer/2022062301/56814558550346895db22790/html5/thumbnails/26.jpg)
The FutureThe Future
Near term plans – existing funding (sort of)
New funding
Challenges
![Page 27: Peter Cornillon University of Rhode Island Presented at the 10 September 2003 NVODS Workshop](https://reader036.vdocuments.mx/reader036/viewer/2022062301/56814558550346895db22790/html5/thumbnails/27.jpg)
Near Term PlansNear Term Plans OPeNDAP
DAP 4 Provide web service access to the DAP
XML version of the DAP
SOAP interface to the DAP
Add bit-image data type
Add cataloging capabilities to OPeNDAP-servers using THREDDS technology
Extend Ancillary Information Services (AIS) capabilites
Complete the OPeNDAP data Connector
![Page 28: Peter Cornillon University of Rhode Island Presented at the 10 September 2003 NVODS Workshop](https://reader036.vdocuments.mx/reader036/viewer/2022062301/56814558550346895db22790/html5/thumbnails/28.jpg)
Near Term PlansNear Term Plans
NVODSPopulate Ancillary Information Services (AIS)
site for oceanographic data
Work with groups implementing pilots
![Page 29: Peter Cornillon University of Rhode Island Presented at the 10 September 2003 NVODS Workshop](https://reader036.vdocuments.mx/reader036/viewer/2022062301/56814558550346895db22790/html5/thumbnails/29.jpg)
New FundingNew Funding
NASA REASoN CAN – Thematic Portals for Satellite-Derived Surface Ocean Parameters2003-2008 - Between $500 and $750/year to NVODSAbout ½ of funding for OPeNDAPWill provide OPeNDAP/LAS access to satellite-derived
SST, Surface Winds, Ocean Color, Precipitation and SSH data sets.
NOPP – GODAE2003-2005 - $70k/year to OPeNDAP Will work with the data assimilation community to
make their data accessible via OPeNDAP/LAS
![Page 30: Peter Cornillon University of Rhode Island Presented at the 10 September 2003 NVODS Workshop](https://reader036.vdocuments.mx/reader036/viewer/2022062301/56814558550346895db22790/html5/thumbnails/30.jpg)
ChallengesChallenges
Data discovery – Directory population
GIS access
Consistent semantic data model(s)
![Page 31: Peter Cornillon University of Rhode Island Presented at the 10 September 2003 NVODS Workshop](https://reader036.vdocuments.mx/reader036/viewer/2022062301/56814558550346895db22790/html5/thumbnails/31.jpg)
ChallengesChallenges
Data discovery – Directory population Finding new servers and new data sets at existing servers
OPeNDAP-enabled servers are rapidly being installed
New data sets are being added at existing server sites
Obtaining complete and consistent data set descriptions Inadequate data set descriptions
No link between the data set description and OPeNDAP access to the data.
Data sets consisting of collections of holdings pose special problems
Each of the individual items may be well defined, but not the collection as a whole, or
There may not be a single entry point for the collection
![Page 32: Peter Cornillon University of Rhode Island Presented at the 10 September 2003 NVODS Workshop](https://reader036.vdocuments.mx/reader036/viewer/2022062301/56814558550346895db22790/html5/thumbnails/32.jpg)
ChallengesChallenges
Data discovery – Directory population (cont.)Handling virtual data sets and server services
New data sets obtained by combining data from several sites
• The collection of mooring data obtained from several sites New data objects obtained via operations on the data at a site
• Wind stress curl derived from the wind stress field
Handling replicated data sets Some of the more popular data sets are replicated at several sites
![Page 33: Peter Cornillon University of Rhode Island Presented at the 10 September 2003 NVODS Workshop](https://reader036.vdocuments.mx/reader036/viewer/2022062301/56814558550346895db22790/html5/thumbnails/33.jpg)
ChallengesChallenges
GIS accessAccess from GIS-based servers
Grid access
Feature access
Access from GIS packages Grid access
Feature access
![Page 34: Peter Cornillon University of Rhode Island Presented at the 10 September 2003 NVODS Workshop](https://reader036.vdocuments.mx/reader036/viewer/2022062301/56814558550346895db22790/html5/thumbnails/34.jpg)
ChallengesChallenges
Consistent semantic data model(s)
![Page 35: Peter Cornillon University of Rhode Island Presented at the 10 September 2003 NVODS Workshop](https://reader036.vdocuments.mx/reader036/viewer/2022062301/56814558550346895db22790/html5/thumbnails/35.jpg)
NVODS:NVODS: http://nvods.org
OPeNDAP:OPeNDAP: http://opendap.org
DODS:DODS: http://unidata.ucar.edu/packages/dods
Data System URLSData System URLS