CUAHSI Hydrologic Information Systems and Web Services

By David R. Maidment

With support from many collaborators: Ilya Zaslavsky, David Valentine, Reza Wahadj, Chaitan Baru, Praveen Kumar, Michael Piasecki, Rick Hooper, Jon Duncan, David Tarboton, Jeff Horsburgh, Venkat Lakshmi, Chunmaio Zheng, Xu Liang, Yao Liang, Ken Reckhow, Upmanu Lall, LeRoy Poff, Dennis Lettenmaier, Barbara Minsker, ……

And many graduate students and post-docs: Venkatesh Merwade, Tim Whiteaker, Jon Goodall, Gil Strassberg, Ben Ruddell, Luis Bermudez, Bora Beran, ……

Thanks to everyone for all their help!

HIS Goals

• Hydrologic Data Access System – better access to a large volume of high quality hydrologic data

• Support for Observatories – synthesizing hydrologic data for a region

• Advancement of Hydrologic Science – data modeling and advanced analysis

• Hydrologic Education – better data in the classroom, basin-focused teaching

HIS User Assessment (Chapter 4 in Status Report)

Data Access

Science Observatorysupport

Education

Which of the four HIS goals is most important to you?

HIS Goals

• Hydrologic Data Access System – better access to a large volume of high quality hydrologic data

• Support for Observatories – synthesizing hydrologic data for a region

• Advancement of Hydrologic Science – data modeling and advanced analysis

• Hydrologic Education – better data in the classroom, basin-focused teaching

Rainfall & SnowWater quantity

and quality

Remote sensing

Water Data

Modeling Meteorology

Soil water

Water Data Web Sites

NWISWeb site output# agency_cd Agency Code# site_no USGS station number# dv_dt date of daily mean streamflow# dv_va daily mean streamflow value, in cubic-feet per-second# dv_cd daily mean streamflow value qualification code## Sites in this file include:# USGS 02087500 NEUSE RIVER NEAR CLAYTON, NC#agency_cd site_no dv_dt dv_va dv_cdUSGS 02087500 2003-09-01 1190USGS 02087500 2003-09-02 649USGS 02087500 2003-09-03 525USGS 02087500 2003-09-04 486USGS 02087500 2003-09-05 733USGS 02087500 2003-09-06 585USGS 02087500 2003-09-07 485USGS 02087500 2003-09-08 463USGS 02087500 2003-09-09 673USGS 02087500 2003-09-10 517USGS 02087500 2003-09-11 454

Time series of streamflow at a gaging station

CUAHSI Hydrologic Data Access System

A common data window for accessing, viewing and downloading hydrologic information

USGSUSGS

NASANASANCDCNCDCEPAEPA NWSNWS

Observatory DataObservatory Data

http://river.sdsc.edu/HDAS

Observation Stations

Ameriflux Towers (NASA & DOE) NOAA Automated Surface Observing System

USGS National Water Information System NOAA Climate Reference Network

Map for the US

NWIS Station Observation Metadata

Describe what has been measured at this station

Web Page Scraping

ProgrammaticallyProgrammatically construct a construct a URL string as produced by URL string as produced by manual usemanual use of the web page of the web page

http://nwis.waterdata.usgs.gov/nwis/discharge?site_no=02087500&agency_cd=USGS&....http://nwis.waterdata.usgs.gov/nwis/discharge?site_no=02087500&agency_cd=USGS&....

ParseParse the resulting ASCII file the resulting ASCII file

CUAHSI Web Services

Web ServicesLibrary

Web Application: Data Portal

Your application• Excel, ArcGIS, Matlab• Fortran, C/C++, Visual Basic• Hydrologic model• …………….

Your operating system• Windows, Unix, Linux, Mac

Internet Simple Object Access Protocol

Series and FieldsFeatures

Point, line, area, volumeDiscrete space representation

Series – ordered sequence of numbersTime series – indexed by time

Frequency series – indexed by frequency

Surfaces Fields – multidimensional arrays

Scalar fields – single value at each locationVector fields – magnitude and direction Random fields – probability distribution

Continuous space representation

mm / 3 hours

Precipitation Evaporation

North American Regional Reanalysis of Climate

Variation during the day, July 2003

NetCDF format

Arc Hydro Time Series

Feature Class (HydroID)

Attribute Series Table (FeatureID)

HydroID 2906

Geospatial features associate with time series

TSDateTime

FeatureID

TSType

TSValue

Arc Hydro Time Series Object

TSType Table

Feature Class(point, line, area)

Time

Space (x,y,z)

Variable

Value

NetCDF Data Model

Attributes

Dimensions andCoordinates

NWISNWIS

ArcGISArcGIS

ExcelExcel

NCARNCAR

UnidataUnidata

NASANASAStoretStoret

NCDCNCDC

AmerifluxAmeriflux

MatlabMatlab

AccessAccess SASSAS

FortranFortran

Visual BasicVisual Basic

C/C++C/C++

Some operational services

CUAHSI Web ServicesCUAHSI Web Services

Data SourcesData Sources

ApplicationsApplications

Extract

Transform

Load

http://www.cuahsi.org/his/

Core Web Services

Service Input Output

GetSites Obs Network All station codes in network

GetSiteInfo Station Code Lat/long, station name

GetVariables Obs Network or data source

All variable codes

GetVariableInfo Variable code Description of variable

GetValues Station code or lat/long point, variable code, begin date, end date

A time series of values

GetChart As for GetValue A chart plotting the values

Operational Services

Service Ameriflux Daymet MODIS NWIS NAM

GetSites Yes

GetSiteInfo Yes Yes

GetVariables Yes Yes

GetVariableInfo Yes Yes

GetValues Yes Yes Yes Yes Yes

GetChart Yes Yes

CUAHSI Web Serviceshttp://www.cuahsi.org/his/webservices.html

NCEP North American Forecast Model 12 Km grid for continental US

Water OneFlow• Like Geospatial OneStop, we need a “Water

OneFlow” – a common window for water data and models

• Advancement of water science is critically dependent on integration of water information

Federal

AcademicLocal

State

Conclusions

• It would be good to define a collaboration between CUAHSI and Unidata for web services that has a consistent vocabulary

• We in CUAHSI would defer to Unidata for definition of how to ingest real-time weather information as fields (netCDF with CF conventions)

• Try to define services that represent “time histories” of variables, past, present and future e.g. precipitation, evaporation, surface temp