data management updates
DESCRIPTION
Data Management Updates. Kristen Gunthardt , EPA OW Nate Booth, USGS CIDA NWQMC February 1, 2010. Topics. Water Quality Data Exchange Portal Update Towards a Multidisciplinary Data Sharing FRamework. What is WQX?. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Data Management Updates](https://reader036.vdocuments.mx/reader036/viewer/2022062500/56815b2c550346895dc8f086/html5/thumbnails/1.jpg)
Data Management Updates
Kristen Gunthardt, EPA OWNate Booth, USGS CIDA
NWQMC February 1, 2010
![Page 2: Data Management Updates](https://reader036.vdocuments.mx/reader036/viewer/2022062500/56815b2c550346895dc8f086/html5/thumbnails/2.jpg)
Topics• Water Quality Data Exchange• Portal Update• Towards a Multidisciplinary Data Sharing FRamework
![Page 3: Data Management Updates](https://reader036.vdocuments.mx/reader036/viewer/2022062500/56815b2c550346895dc8f086/html5/thumbnails/3.jpg)
What is WQX?• WQX defines the framework by which EPA compiles water
quality monitoring data in the STORET Data Warehouse
• WQX is governed by a standardized format, so all data must comply with this format
• The WQX format allows anybody to share data regardless of what the original source of the data was
• WQX provides a common suite of data elements that we can use to share data across sources – NWIS Water Quality and STORET Warehouse data
![Page 4: Data Management Updates](https://reader036.vdocuments.mx/reader036/viewer/2022062500/56815b2c550346895dc8f086/html5/thumbnails/4.jpg)
Today’s Status• 37 State agencies have successfully flowed data via WQX or WQX
Web since 2007 (LINK)
• Over 80 Tribal organizations have successfully flowed data via WQX or WQX Web since 2007
• Other states and tribes continue to come on-line, and/or have been funded through EPA Exchange Network grant dollars to transition to WQX
• STORET Helpdesk assistance, grant funding, as well as individual consultation and training facilitate the transition to WQX
• EPA Office of Water and Office of Environmental Information continue to partner to provide tools for all data providers
![Page 5: Data Management Updates](https://reader036.vdocuments.mx/reader036/viewer/2022062500/56815b2c550346895dc8f086/html5/thumbnails/5.jpg)
1960’s 1970’s 1980’s 1990’s 2000’s
Legacy STORET
Data Integration Timeline
WATSTORE
Modern STORET
NWIS
NWISWeb
QW WebServices
STORET warehouse
Data copied
![Page 6: Data Management Updates](https://reader036.vdocuments.mx/reader036/viewer/2022062500/56815b2c550346895dc8f086/html5/thumbnails/6.jpg)
U.S. Water Data Portal Project: Integrating Water Information
• The United States Geological Survey (USGS) and the United States Environmental Protection Agency (USEPA) have enhanced water quality monitoring data access
• Common data standards and web services improve on the historical approach
• Water managers and the public will access integrated water quality monitoring data from multiple agencies through a singular data portal
What is a web service?•Computer-to-computer data sharing•Uses Input parameters and outputs XML•Can be used in multiple ways by many applications
USGS
EPA
Internet (XML)
•For more information, please visit: http://qwwebservices.usgs.gov/ and http://www.epa.gov/storet/web_services.html
![Page 7: Data Management Updates](https://reader036.vdocuments.mx/reader036/viewer/2022062500/56815b2c550346895dc8f086/html5/thumbnails/7.jpg)
![Page 8: Data Management Updates](https://reader036.vdocuments.mx/reader036/viewer/2022062500/56815b2c550346895dc8f086/html5/thumbnails/8.jpg)
![Page 9: Data Management Updates](https://reader036.vdocuments.mx/reader036/viewer/2022062500/56815b2c550346895dc8f086/html5/thumbnails/9.jpg)
9
Components of Longer Term Data Integration
Other Data Partners
Integrated water quality monitoring data is made possible through:• Web services technology• Standardized metadata• Compatible search parameters• Common vocabularies
![Page 10: Data Management Updates](https://reader036.vdocuments.mx/reader036/viewer/2022062500/56815b2c550346895dc8f086/html5/thumbnails/10.jpg)
![Page 11: Data Management Updates](https://reader036.vdocuments.mx/reader036/viewer/2022062500/56815b2c550346895dc8f086/html5/thumbnails/11.jpg)
Gulf of Maine+Sharing a Variety of Data for the
Northeast Coastal and Ocean Data Partnership
Learn more at www.necodp.org
![Page 12: Data Management Updates](https://reader036.vdocuments.mx/reader036/viewer/2022062500/56815b2c550346895dc8f086/html5/thumbnails/12.jpg)
• Further Integration– Common spatial frameworks (NHD)– Common analytical method metadata (NEMI)
• Improve the federated dataset • A single web portal for water-quality data
– Unified downloads
Future Work
![Page 13: Data Management Updates](https://reader036.vdocuments.mx/reader036/viewer/2022062500/56815b2c550346895dc8f086/html5/thumbnails/13.jpg)
USGS / USEPA Water-Quality Data ExchangeLower PotomacHydrologic Unit 02070011161 USGS, 169 EPA stream sites
USGS EPA
150M water quality observations over last 100 yrs
![Page 14: Data Management Updates](https://reader036.vdocuments.mx/reader036/viewer/2022062500/56815b2c550346895dc8f086/html5/thumbnails/14.jpg)
Towards an International Multidisciplinary Water Data Sharing Framework
• A common information model for the entire hydrologic cycle
• Data Integration Framework • Open Geospatial Consortium Hydrology
Domain Working Group• National and International partners
![Page 15: Data Management Updates](https://reader036.vdocuments.mx/reader036/viewer/2022062500/56815b2c550346895dc8f086/html5/thumbnails/15.jpg)
Credit: David J. Schwab
![Page 16: Data Management Updates](https://reader036.vdocuments.mx/reader036/viewer/2022062500/56815b2c550346895dc8f086/html5/thumbnails/16.jpg)
Helping the World to CommunicateGeographically
Hydrology Domain Working Group
• A joint working group of the OGC and WMO constituted as an OGC Domain Working Group.
• Brings together interested parties to develop and promote the technology for greatly improving the way in which water information is described and shared.
• Co-chaired by representatives nominated by the OGC TC and the World Meteorological Organisation’s (WMO) Commission for Hydrology (CHy).
• Current Co-Chairs: Ilya Zaslavsky (SDSC), Ulrich Looser (GRDC) and David Lemon (CSIRO)
• > 50 Participants, > 30 Organisations
Courtesy: David Lemon, CSIRO
![Page 17: Data Management Updates](https://reader036.vdocuments.mx/reader036/viewer/2022062500/56815b2c550346895dc8f086/html5/thumbnails/17.jpg)
Meets every 3 months
Teleconferences most weeks
WaterML Version 2 standard being proposed
Vote for adoption 3-6 months later
Jointly with World Meteorological Organization
Evolving WaterML into an International Standard
November 2009
![Page 18: Data Management Updates](https://reader036.vdocuments.mx/reader036/viewer/2022062500/56815b2c550346895dc8f086/html5/thumbnails/18.jpg)
• CUAHSI is a consortium representing 125 US universities
• Supported by the National Science Foundation Earth Science Division
• Advances hydrologic science in nation’s universities
• Includes a Hydrologic Information System project
http://www.cuahsi.org
Courtesy: David Maidment
![Page 19: Data Management Updates](https://reader036.vdocuments.mx/reader036/viewer/2022062500/56815b2c550346895dc8f086/html5/thumbnails/19.jpg)
19
What have we learned?
• We have custom-built a very large scale services-oriented architecture and a sophisticated user interface to it– A much simpler and more general pattern has emerged
based on existing Open Geospatial Consortium (OGC) information exchange standards and extensions to them
• We have exposed a very large volume of information– It needs to be carefully organized to be most useful
Courtesy: David Maidment
![Page 20: Data Management Updates](https://reader036.vdocuments.mx/reader036/viewer/2022062500/56815b2c550346895dc8f086/html5/thumbnails/20.jpg)
20
Open Geospatial Consortium Web Service Standards
• Map Services
• Web Map Service (WMS)• Web Feature Service (WFS)• Web Coverage Service
(WCS)• Catalog Services for the Web
(CS/W)
• Observation Services
• Observations and Measurements Model
• Sensor Web Enablement (SWE)
• Sensor Observation Service (SOS)
These standards have been developed over the past 10 years ….…. by 400 companies and agencies working within the OGC
![Page 21: Data Management Updates](https://reader036.vdocuments.mx/reader036/viewer/2022062500/56815b2c550346895dc8f086/html5/thumbnails/21.jpg)
Towards an International Multidisciplinary Water Data Sharing Framework
• Open Geospatial Consortium Hydrology Domain Working Group Interoperability Experiments – Academic, Government, Industry– Groundwater– Surface Water– Flood Forecasting– Met/Oceans
![Page 22: Data Management Updates](https://reader036.vdocuments.mx/reader036/viewer/2022062500/56815b2c550346895dc8f086/html5/thumbnails/22.jpg)
Groundwater Data Exchange Experiment
Test and enhance OGC standards for water observations
Exchange groundwater well characteristics and water levels with Canada
Start with Lake Superior Basin
![Page 23: Data Management Updates](https://reader036.vdocuments.mx/reader036/viewer/2022062500/56815b2c550346895dc8f086/html5/thumbnails/23.jpg)
![Page 24: Data Management Updates](https://reader036.vdocuments.mx/reader036/viewer/2022062500/56815b2c550346895dc8f086/html5/thumbnails/24.jpg)
Groundwater Data Portal
![Page 25: Data Management Updates](https://reader036.vdocuments.mx/reader036/viewer/2022062500/56815b2c550346895dc8f086/html5/thumbnails/25.jpg)
Groundwater Data Portal
![Page 26: Data Management Updates](https://reader036.vdocuments.mx/reader036/viewer/2022062500/56815b2c550346895dc8f086/html5/thumbnails/26.jpg)
Groundwater Data Portal
![Page 27: Data Management Updates](https://reader036.vdocuments.mx/reader036/viewer/2022062500/56815b2c550346895dc8f086/html5/thumbnails/27.jpg)
Surface Water Interoperability Experiment (3 use cases)
Cross border – share streamflow data across regional boundaries (Europe)
Forecasting (Deltares, USGS, NOAA)– share streamflow and rating curve for flood forecasting
Global Runoff (Kisters) – Real-time calculations of streamflow volume to oceans
![Page 28: Data Management Updates](https://reader036.vdocuments.mx/reader036/viewer/2022062500/56815b2c550346895dc8f086/html5/thumbnails/28.jpg)
Surface Water Interoperability Experiment – USGS gages
ACWI SOH, IWRSS
![Page 29: Data Management Updates](https://reader036.vdocuments.mx/reader036/viewer/2022062500/56815b2c550346895dc8f086/html5/thumbnails/29.jpg)
Courtesy: Kelli Page, GLOS
Met / Oceans
![Page 30: Data Management Updates](https://reader036.vdocuments.mx/reader036/viewer/2022062500/56815b2c550346895dc8f086/html5/thumbnails/30.jpg)
Credit: David J. Schwab
![Page 31: Data Management Updates](https://reader036.vdocuments.mx/reader036/viewer/2022062500/56815b2c550346895dc8f086/html5/thumbnails/31.jpg)
![Page 32: Data Management Updates](https://reader036.vdocuments.mx/reader036/viewer/2022062500/56815b2c550346895dc8f086/html5/thumbnails/32.jpg)
![Page 33: Data Management Updates](https://reader036.vdocuments.mx/reader036/viewer/2022062500/56815b2c550346895dc8f086/html5/thumbnails/33.jpg)
Great Lakes Restoration Initiative –GLOS backbone· Support cross-
agency and cross-discipline data analysis and modeling through a data standards-based virtual observatory
![Page 34: Data Management Updates](https://reader036.vdocuments.mx/reader036/viewer/2022062500/56815b2c550346895dc8f086/html5/thumbnails/34.jpg)
Nested watershed models and sensors
· Hierarchy of watershed water-quality models:· SPARROW (spatial variability, annual time-step)· HSPF (daily time step) – 2 ag watersheds, 1 on LM· Sensor derived surrogates (~real-time) – 30 sites / 8
on LM
· Common basin-wide hydrologic model that can test climate change scenarios
· Evolve data standards to couple to in-lake models and observations
![Page 35: Data Management Updates](https://reader036.vdocuments.mx/reader036/viewer/2022062500/56815b2c550346895dc8f086/html5/thumbnails/35.jpg)
U.S. Department of the InteriorU.S. Geological Survey
Kristen Gunthardt
USEPA Office of Water
Email: [email protected]
Thank you
Nate Booth, USGS
USGS Center for Integrated Data Analytics (CIDA)
Ph: 608-821-3822
Email: [email protected]