odip: integrating regional marine data infrastructures for global ocean science
DESCRIPTION
Presentation for eResearch Australasia conference outlining the Ocean Data Interoperability Platform (ODIP) projectTRANSCRIPT
Helen Glaves Senior Data Scientist
British Geological Survey
Today’s presentation
Where it all started
Ocean and marine data:
Management on the regional scale
Moving to a global framework: the ODIP approach
Re-use of marine – why it’s matters
First ocean explorers
Phoenicians and Greeks:
First known ocean explorers (2000 – 400BC)
Extensive knowledge of marine science
Early example of the importance of data/ information preservation!
The Vikings: early marine scientists
Explored North Atlantic:• Iceland – 700AD• Greenland - 995AD• North America -
1000AD
Developed detailed knowledge of:
• Currents• Tides• Winds
Age of Discovery: 1400s- 1900s:A time of exploration and adventurous men!
• Christopher Columbus• Ferdinand Magellan• Vasco De Gama• Captain Cook
Captain Cook: first ocean scientist?• 3 voyages (1768 – 1780)
• Produced maps, charts &scientific samples
• Charted Australia & New Zealand
• Explored Hawaii
• First to include a full-time naturalist (Joseph Banks)
Oceanographic data
Wide range of measurements and variables
Derived from broad spectrum of multidisciplinary projects/programmes
Collected by multitude of research institutes, governmental organisations and private companies
Using various sensors to measure physical, chemical, biological, geological and geophysical parameters
Data acquistion Sensors installed on various platforms:
Research vessels
Satellites
Buoys/floats/gliders
Aircraft
Submersibles
Fixed moorings
Fauna
Barriers to re-using data
Use of different
Formats
Standards
Best practice
Co-ordinate systems
Technologies
National and organisational data access policies
9
Regional data infrastructures A number of regional initiatives have
developed marine data management
infrastructures
Promoted and supported by international
organizations - UNESCO‘s
Intergovernmental Oceanographic
Commission (IOC), GEO etc.
BUTImplemented according to regional
requirements and priorities
USA
Europe
Australia
Ocean Data Interoperability Platform
EU-US-Australia collaborative project
Grant Number: 312492
Call: FP7-INFRASTRUCTURES-2012-1-INFSOActivity: INFRA-2012-3.2: International co-operation with the USA
on common e-infrastructure for scientific data
Start date: 1 October 2012
Duration: 36 months
Funded in parallel by European Commission, National Science Foundation (NSF) and Australian Government
How ODIP is achieving its objectives? Developing a collaborative approach and promoting organised
dialogue between partners
Establishing a European - USA - Australia co-ordination platform to support development of interoperability between existing marine data management infrastructures
Creating and publishing inventories of existing standards and policies
Regular joint workshops to develop interoperability solutions and/or agree on common standards
Development of prototype for testing and evaluating potential interoperability solutions
Europe: 10 EU funded partners: 6 countriesNERC-BGS/BODC, MARIS, OGS, IFREMER, HCMR, ENEA, ULG,
CNR, RBINS-MUMM, TNO
ODIP partners
USA: NSF funded partners (supplement to existing R2R project)
San Diego Supercomputer Center (SDSC)
Scripps Institution of Oceanography (SIO)
Woods Hole Oceanographic Institute (WHOI)
Lamont-Doherty Earth Observatory (LDEO)
Florida State University: Center for Ocean-Atmospheric Prediction Studies (FSU)
Australia
Integrated Marine Observing System (IMOS)
International
UNESCO IOC-IODE
Associate partners
Europe
Alfred Wegener Institute for Polar Research (AWI)
MARUM
USA
NOAA US-IOOS, NOAA US-NODC, NOAA NGDC
UNIDATA
Australia
Australian National Data Service (ANDS)
Geoscience Australia (GA)
CSIRO
How ODIP is achieving its objectives? Developing a collaborative approach and promoting organised
dialogue between partners
Establishing a European - USA - Australia co-ordination platform to support development of interoperability between existing marine data management infrastructures
Creating and publishing inventories of existing standards and policies
Regular joint workshops to develop interoperability solutions and/or agree on common standards
Development of prototypes for testing and evaluating potential interoperability solutions
How ODIP is achieving its objectives? Developing a collaborative approach and promoting organised
dialogue between partners
Establishing a European - USA - Australia co-ordination platform to support development of interoperability between existing marine data management infrastructures
Creating and publishing inventories of existing standards and policies
Regular joint workshops to develop interoperability solutions and/or agree on common standards
Development of prototypes for testing and evaluating potential interoperability solutions
NERC Vocabulary Server (NVS)
• lists of standard terms for populating fields in oceanographic metadata
• Used by SeaDataNetfor population of CDI metadata records
• Accessed via RESTful URIs or SOAP
• SPARQL endpoint available http://vocab.nerc.ac.uk/sparql
How ODIP is achieving its objectives?
Developing collaboration platform for organised dialogue between partners
Creating and publishing inventories of existing standards and policies
Regular joint workshops to develop interoperability solutions and/or agree on common standards
Development of prototypes for testing and evaluating potential interoperability solutions
3rd ODIP workshopTownsville, Australia
5 – 8 August 2014
How ODIP is achieving its objectives?
Developing collaboration platform for organised dialogue between partners
Creating and publishing inventories of existing standards and policies
Regular joint workshops to develop interoperability solutions and/or agree on common standards
Development of prototypes for testing and evaluating potential interoperability solutions
ODIP 1: objective
Establishing interoperability between the SeaDataNet, IMOS and NODC data discovery and access services using brokering services
Lead by European partners via SeaDataNet
Initially addressing use of brokers at the metadata level
Progress to data access services (possibly including authentication, authorisation and accounting (AAA) systems)
ODIP 1: the plan make use of the (Euro)GEOSS (GEO-DAB) broker service to
harmonise 3 regional services to a common level
SeaDataNet (Europe)
IMOS (Australia)
NODC (USA)
start at metadata level, but progress to data access, including providing solutions for possible AAA systems
use the broker to facilitate access to data from the regional services by the GEOSS portal and Ocean Data Portal (ODP)
Interoperability established between SeaDataNet metadata discovery services and IODE-ODP and GEOSS portals
Creation of SeaDataNet web service for collections of metadata records
Using GEO-Discovery and Access Broker (DAB)developed for GEOSS
SeaDataNet collections now exposed on GEOSS and ODP portals
ODIP 1: progress
ODIP 1: SeaDataNet web service
Established web service for collections of metadata entries (ISO19115 – 19139 schema)
Collections made by aggregation on Discipline (SDN vocab P08), Data centre (SDN EDMO-code), and geometric type (point / track / surface)
Around 1.5 million CDI granules resulted in approx. in 400CDI collections (includes URL to CDI service for details)
REST web service deliver collections in XML format:
http://seadatanet.maris2.nl/gi-cat-seadatanet/sdn-cdi-aggr-seadatanet_v3.xml
25
Interoperability established between SeaDataNetmetadata discovery services and IODE-ODP and GEOSS portals
Creation of SeaDataNet web service for collections of metadata records
Using GEO-Discovery and Access Broker (DAB)developed for GEOSS
SeaDataNet collections now exposed on GEOSS and ODP portals
ODIP 1: progress
GEOSS Brokerage service Developed and maintained by CNR (Italy)
Middleware for connecting heterogeneous/distributed resources contributing to the GEOSS portal
3 main functionalities:
Discovery of brokered resources
Semantics-enriched discovery
Access of resources
Used to harvest the SeaDataNet collections and convert to Generic Brokerage Reference Schema, adopting SeaDataNet vocabs
27
ODIP 1 – SeaDataNet webservices
SeaDataNet collections available as 2 public web services provided by CNR via the GEO-DAB broker:
OGC Catalogue Service for the Web (CSW) Version 2.0.2Service – HTTP POST method:http://seadatanet.essi-lab.eu/gi-cat/services/cswiso
OAI-PMH interface, at:http://seadatanet.essi-lab.eu/gi-cat/services/oaipmh
Update of SeaDataNet metadata catalogue triggers GEO-DAB harvesting of XML for new records
28
Interoperability established between SeaDataNetmetadata discovery services and IODE-ODP and GEOSS portals
Creation of SeaDataNet web service for collections of metadata records
Using GEO-Discovery and Access Broker (DAB) developed for GEOSS
SeaDataNet collections now exposed on GEOSS and ODP portals
ODIP 1: progress
ODIP 1 – SeaDataNet in GEOSS GEOSS portal harvests SeaDataNet metadata from the
OGC-CSW
Test Client at CNR (ESSI lab) :
http://seadatanet.essi-lab.eu/gi-cat/gi-portal/
30
ODIP 1: SeaDataNet in IODE-ODP
IODE ODP portal harvests SeaDataNet collections from OIA-PMH service
31
ODIP 2: objective ODIP 2: Establishing interoperability between cruise
summary reporting systems in Europe, the USA and
Australia
Lead by Rolling Deck to Repository (R2R) partners
(USA)
Improvement of delivery and exchange of cruise
summary information through the use of common
formats and vocabularies
Use GeoNetWorks for routine harvesting of cruise data
for delivery via the Partnership for Observation of
Global Oceans (POGO) portal
ODIP 2: the plan Publish ISO Cruise Summary Reports at regional
nodes:
Marine National Facility (Australia)
SeaDataNet (Europe)
R2R (USA)
Deploy GeoNetwork catalogues at regional nodes
providing both a GUI (web portal) and API (CSW
service)
Harvest GeoNetwork nodes into POGO global
catalogue
SeaDataNet CSR (Cruise Summary Report) schema adopted by:
R2R consortium partners (USA)
Marine National Facility (Australia)
ISO Cruise Summary Reports published at regional nodes
Deployment of GeoNetwork catalogues at regional nodes underway
ODIP 2: progress
European Directory of Marine Organisations
(EDMO)
SeaDataNet CSR (Cruise Summary Report) schema adopted by:
R2R consortium partners (USA)
Marine National Facility (Australia)
ISO Cruise Summary Reports published at regional nodes
Deployment of GeoNetwork catalogues at regional nodes underway
ODIP 2: progress
SeaDataNet CSR (Cruise Summary Report) schema adopted by:
R2R consortium partners (USA)
Marine National Facility (Australia)
ISO Cruise Summary Reports published at regional nodes
Deployment of GeoNetwork catalogues at regional nodes underway
ODIP 2: progress
EU GeoNetwork deploymentEurope: http://www.ifremer.fr/geonetwork-sdn/
R2R GeoNetwork CSR instance
http://catalog.rvdata.us/geonetwork
GeoNetwork deployed:
http://www.cmar.csiro.au/geonetwork
Published a test set of CSR for R/V Southern Surveyor
Marine National Facility GeoNetworkinstance
ODIP 3: objective
Establishment of a prototype for a Sensor Observation Service (SOS) for selected sensors installed on vessels and in real-time monitoring systems using sensor web enablement (SWE)
Lead by AODN (Australia)
regional initiatives progress towards the adoption of SWE allowing direct standardised access to the data from operational sensor systems
ODIP 3: the plan
establish a collaboration tool (Github)
compile inventory of SOS services and their endpoints
compile inventory of instrument SensorML records & O&M structures
compile inventory of vocabulary and registry services
Working groups to:
assess SOS performance
propose templates for SensorML/StarFL and O&M profiles
examine vocabulary services and potential mappings
Set-up a test bed
ODIP 3: progress GitHub collaborative tool set-up
Test bed established
2 public SOS servers running V4.0 and V3.6 of 52oNorth SOS server
Fully open allowing full set of SOS requests including registration of sensors and adding data
Working groups established:
• assess SOS performance
• propose templates for SensorML/StarFl and O&M profiles
• examine vocab services and potential mappings
ODIP 3: progress GitHub collaborative tool set-up
Test bed established
2 public SOS servers running V4.0 and V3.6 of 52oNorth SOS server
Fully open allowing full set of SOS requests including registration of sensors and adding data
Working groups established:
• assess SOS performance
• propose templates for SensorML/StarFl and O&M profiles
• examine vocab services and potential mappings
ODIP3: SOS services: 52o North v4 SOS V4.0
Base Address: http://115.146.93.169:8080/IMOS-SOS/
XML Address: http://115.146.93.169:8080/IMOS-SOS/sos/pox/
Two Features of Interest
IMOS/DAVIES/SF1 = Sensor Float 1 at Davies Reef
IMOS/HERON/RP8 = Relay Pole 8 at Heron Island
Two parameters
Water temperature (Deg. C.)
Depth (m)
SOS v3.6
Base Address: http://130.220.209.177:8080/IMOS-SOS-36/
XML Address: http://130.220.209.177:8080/IMOS-SOS-36/sos/
To be set up with same Features of Interest and parameters
Supports SOS v1 and v2
ODIP3: SOS services: 52o North v3.6
ODIP 3: progress GitHub collaborative tool set-up
Test bed established
2 public SOS servers running V4.0 and V3.6 of 52oNorth SOS server
Fully open allowing full set of SOS requests including registration of sensors and adding data
Working groups established:
• assess SOS performance
• propose templates for SensorML/StarFl and O&M profiles
• examine vocab services and potential mappings
Underpinning development of a robust common global framework for marine data management by:
Establishing interoperability between existing regional data management infrastructures
Creating an approach that can be adopted by agencies and organisations in other geographical area
Supporting and promoting international collaboration across the marine data management community
Facilitating re-use of marine data
ODIP: what is it achieving?
Data is fundamental to a range of activities in the marine domain:
Research
Monitoring
Forecasting
Management
Many of these activities are now highly multidisciplinary/ ecosystem level requiring easy access to large volumes of data
Re-use: why is it important?
Marine data is precious
expensive to collect
Inherently unique
Sparse spatial and temporal coverage
Needed to provide answers to local and global issues
Capture once – use it many times
Find out more
Project website
Join the ODIP community
Contact us
Social media
International conferences
Other related initiatives
BCube (NSF)
Ocean Data Portal (ODP)
Research Data Alliance
Belmont Forum
www.odip.eu
Thank you to: ODIP partners
Dick Schaap (SeaDataNet)
Roger Proctor/ Scott Bainbridge (IMOS)
Bob Arko (R2R)
Sergey Belov (IODE)
Chris Oosthuizen (IMOS)
Genny Anderson (
Thank you!