introduction to cuahsi hydrologic information...
TRANSCRIPT
Introduction to CUAHSI Hydrologic Information Systems
Presented by Tim Whiteaker
The University of Texas at Austin
4 June, 2009
Objectives
• Understand what the CUAHSI Hydrologic Information System (HIS) is and how it is used
• Learn how to access data in HIS
• Learn how to publish your data with HIS
• Know where to find more info -http://his.cuahsi.org
Outline
• Day 1
– Morning: Intro to HIS Presentation, Demos
– Afternoon: Loading data into HIS (hands-on)
• Day 2
– Morning: Publishing data with HIS (hands-on)
What is HIS
• The CUAHSI* Hydrologic Information System (HIS) provides web services, tools, standards and procedures that enhance access to more and better data for hydrologic analysis.
• http://his.cuahsi.org
*Consortium of Universities for the Advancement of Hydrologic Science, Inc.http://www.cuahsi.org
Why HIS
• Academics– Recognition of work– Data publication is mandated by the funding agency– To support science in the US and promote collaboration
• Agencies– Standardize data access (both internally and externally)– Time savings in developing a publication system– Public benefit with publication– Return on investment – people can get the data themselves without
requiring a “middle-man”– Get all the state data “together”
• Who is using HIS?– USGS, NCDC, several universities– More at http://his.cuahsi.org/partners.html
HydroSeekDemo:
Intro to Data Access
MetadataCatalog
HIS Central HydroSeek
HIS Central
HIS System Overview
Users
USGS
EPA
Data
HIS ServersData Discovery
And Access
Data Access
Data Registration
DataWeb
Service
too much water
too little water
dirty water
water environment
Rainfall & SnowWater quantity
and quality
Remote sensing
Water Data
ModelingMeteorology
Soil water
Point Water Observations Time Series
A point location in space A series of values in time
Sources of Observations Data
Getting Water Data (the old way)
Different Query Pages Different Query Responses
Web Pages and Web Serviceshttp://www.safl.umn.edu/ http://his.safl.umn.edu/SAFLMC/cuahsi_1_0.asmx
Uses Hypertext Markup Language (HTML)
Uses WaterML(a Markup Language for water data)
HTML as a Web Language
Text and Picturesin Web Browser
<title>Texas Water Development Board</title><!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"><html><head><meta name = "Robots" content = "index,follow"><meta name = "Priority" content = "home,twdb,homepage"><meta name = "Author" content = "Texas Water Development Board, Agency
Number 580"><meta name = "Title" content = "Texas Water Development Board"><meta name = "Description" content = "Texas Water Development Board Home
Page"><meta name = "Keywords" content =
"water,drought,rain,conservation,groundwater,surfacewater,lake,reservoir,hydrology,geology,desalination,TWDB,loans,grants,wastewater,sewage,Clean Water,Drinking Water,State Revolving Fund,planning,State Water Plan,GIS,Geographic Information Systems,Mapping,data">
HyperTextMarkup Language
WaterML as a Web LanguageDischarge of the San Marcos River at Luling, June 28 - July 18, 2002
Streamflow data in WaterML language
Point Observations Information Model
Data Source
Network
Sites
Variables
Values
{Value, Time, Metadata}
Utah State Univ
Little Bear River
Little Bear River at Mendon Rd
Dissolved Oxygen
9.78 mg/L, 1 October 2007, 5PM
• A data source operates an observation network• A network is a set of observation sites• A site is a point location where one or more variables are measured• A variable is a property describing the flow or quality of water• A value is an observation of a variable at a particular time• Metadata provide additional information about the value
GetSites
GetSiteInfo
GetVariableInfo
GetValues
WaterOneFlow Service
WaterOneFlow• Set of query functions • Returns data in WaterML
Site Codes
Variable Codes
Date Ranges
WaterML and WaterOneFlow
GetSitesGetSiteInfoGetVariableInfoGetValues
WaterOneFlowWeb ServiceClient
DEC
UVMUSGS
DataRepositories
Data
DataData
EXTRACTTRANSFORMLOAD
WaterML
WaterML is an XML language for communicating water dataWaterOneFlow is a set of web services based on WaterML
HydroExcel Demo: Introducing
WaterOneFlow Methods
HIS System – HIS Server
Users
TCEQ
Data
HIS ServersData Discovery
And Access
Data Access
Data Registration
WebService
TWDB
Data
MetadataCatalog
HIS Central HydroSeek
HIS Central
• Built for data
– Storage
– Loading
– Analysis
– Publication
• HIS Software free of charge
HIS Server
ODM and Streaming Data Loaders
Real-time Sensors
WaterOneFlowWeb Service
Data Archives
Outside UsersLocal Users
ODM Tools
HIS
Ser
ver
SQL Server
Observations DataModel Database
Name Latitude Longitude
Cane Creek 41.1 -103.2
Cane Creek 41.1 -103.2
Town Lake 40.3 -103.3
Town Lake 40.3 -103.3
Data Storage – Relational Database
Values
ValueDateSiteVariable
Sites
SiteNameLatitudeLongitude
Value Date Site Variable
4.5 3/3/2007 1 Streamflow
4.2 3/4/2007 1 Streamflow
33 3/3/2007 2 Temperature
34 3/4/2007 2 Temperature
Site Name Latitude Longitude
1 Cane Creek 41.1 -103.2
2 Town Lake 40.3 -103.3
Simple Intro to “What Is a Relational Database”
The Data Cube – “what-where-when”
Space, S
Time, T
Variables, V
s
t
Vi
D
“Where”
“What”
“When”
A data value
4.2
CaneCreek
3/4/2007
Streamflow
Data Values Table
Space, S
Time, T
Variables, V
s
t
Vi
vi (s,t)
Observations Data Model
Horsburgh, J. S., D. G. Tarboton, D. R. Maidment and I. Zaslavsky, (2008), "A Relational Model for Environmental and Water Resources Data," Water Resour. Res., 44: W05406, doi:10.1029/2007WR006392.
Values and Time
Variables
Sites
Units Table
UnitsID UnitsName UnitsAbbreviation
12 parts per million ppm
23 cubic feet per second cfs
Spatial References Table
SpatialReferenceID SRSID SRSName
0 Unknown
1 4267 NAD27
2 4269 NAD83
Sites Table
SiteID SiteCode SiteName Latitude Longitude LatLongID
1 AcmeP1 Backyard Pond 34.565 -93.232 1
2 AcmePR2 Mill River gage Station 34.2 -93.4 1
Simplified ODM Structure
Loading data into ODM
• Interactive ODM Data Loader– Loads data from spreadsheets and
comma separated tables in simple format
• Streaming Data Loader (SDL)– Loads data from datalogger files on
a prescribed schedule
– Interactive configuration
• SQL Server Integration Services (SSIS)– Microsoft application accompanying
SQL Server useful for programming complex loading or data management functions
ODM Data Loader
SDL
SSIS
• Operated by Utah State University
• Sensors monitor variables important in watershed flux calculations
• Data published with HIS Software and also their own tools
HIS Server in Action: Little Bear River (LBR) Test Bed
(more examples later)
LBR’s Own Map Application
LBR Sites in HydroSeek
LBR Data in HydroExcel
• Free software for working with and publishing data
• Publishing with the WaterOneFlow standard lets your data plug right in to applications like HydroExcel – it just works!
• Participate in the Water Data Federationhttp://his.cuahsi.org/wdfederation.html
Why HIS Server
How to Get an HIS Server• Options:
–Do it yourselfhttp://his.cuahsi.org/hisserver.html
– SDSC staff sets up your server for you
–Virtual server at SDSC
• You may need software license for SQL Server Standard
What have we covered
• HIS Overview
• HIS Server– ODM
– WaterOneFlow
Users
TCEQ
Data
HIS ServersData Discovery
And Access
Data Access
Data Registration
WebService
TWDB
Data
MetadataCatalog
HIS Central HydroSeek
HIS Central
What’s next• HIS Central
• Data publication overview
• Advanced publication options
Questions?More on ODM?More on XML?
Discharge, Stage, Concentration and Daily Average Example
Water Chemistry from a profile in a lake
Stage and Streamflow Example
Daily Average Discharge ExampleDaily Average Discharge Derived from 15 Minute Discharge Data
XML – A Primer
• eXtensible Markup Language
• XML represents data
• XML is both human and machine readable
<site>
<name>Colorado Rv at Austin, TX</name>
</site>
<site></site>
XML Structure
element
start tag end tag
• Example – Let’s describe a stream gage on the Colorado River
element name = “site”
Can also write as
<site>
</site>
stream gage
Elements can have Children
element
child element
element text
<site>
<name>Colorado Rv at Austin, TX</name>
</site>
Elements can have Attributes
attribute name attribute value
<site>
<name>Colorado Rv at Austin, TX</name>
<siteCode network=“NWIS”>08158000</siteCode>
</site>
XML Nesting
<site>
<name>Colorado Rv at Austin, TX</name>
<siteCode network=“NWIS”>08158000</siteCode>
<location>
<latitude>30.24</latitude>
<longitude>-97.69</longitude>
</location>
</site>
WaterML
• XML for describing water data
• WaterOneFlow returns data in WaterML format
WaterML defines elements and nesting
specific element names
specific element nesting
WaterML includes location, variables, and time series
location
variable
time series