CUAHSI Hydrologic Information SystemStatus Review, July 28, 2004

Agenda

• Review the work of the five project partners– CUAHSI (Rick Hooper, Jon Duncan)– San Diego Supercomputer Center (John Helly, ….)– University of Texas (David Maidment, …)– University of Illinois (Praveen Kumar ….)– Drexel University (Michael Piasecki…)

• Involving the collaborators: V. Lakshmi, X. Liang, Y. Liang, U. Lall, L. Poff, K. Reckhow, D. Tarboton, I. Zaslavsky, C. Zheng

• HIS review meetings – SDSC (August 12-13) – technical detail– Logan (August 23) – user needs assessment

Agenda

• Review the work of the five project partners– CUAHSI (Rick Hooper, Jon Duncan) –

meeting with NSF today– San Diego Supercomputer Center (John

Helly, ….)– University of Texas (David Maidment, …)– University of Illinois (Praveen Kumar,...)– Drexel University (Michael Piasecki,…)

Agenda

• Review the work of the five project partners– CUAHSI (Rick Hooper, Jon Duncan) – Neuse

HO report status– San Diego Supercomputer Center (John

Helly, ….)– University of Texas (David Maidment, …)– University of Illinois (Praveen Kumar ….)– Drexel University (Michael Piasecki…)

Agenda

• Review the work of the five project partners– CUAHSI (Rick Hooper, Jon Duncan) – Neuse

HO report status– San Diego Supercomputer Center (John

Helly, ….)– University of Texas (David Maidment, …)– University of Illinois (Praveen Kumar ….)– Drexel University (Michael Piasecki…)

UT Update

• General issues

• Landscape characterization for HO Design

• Flux algebra for surface water systems

• XML for interchange of groundwater objects

Science Tools Corporation

• http://sciencetools.com/• work with NASA and other

institutions integrating databases for scientific purposes

• Chief Scientist is Richard Troy – he wants to explore potential of working with CUAHSI

• Commercial system that operates over Oracle, SQL/Server,…

• Company is based in Oakland, CA

GenScn

• A tool for generation and analysis of model simulation scenarios for watersheds

• Incorporated in EPA Basins system

• Produced by AquaTerra in Decatur, GA

• Handles lots of different time series types

Suwannee River Watershed Data

• Contact from Wendy Graham (former Vice-Chair of HIS Committee)

• Offering data for consideration in HIS data model

• How to discuss this in Logan?

UT Update

• General issues

• Landscape characterization for HO Design

• Flux algebra for surface water systems

• XML for interchange of groundwater objects

Landscape Characterization for HO Design

• Idea suggested by Larry Band at the end of our call on July 14– have a set of rules for defining subdivisions of

the landscape using orders of magnitude of catchment size, type of land use, etc

– need to work with LIDAR as well as regular DEM

– define points at outlets of these catchments as potential gage sites

EDNA-Elevation Derivatives for National Application

Pfaffstetter Basins

9 basins divided into 99 basins divided into 999 basins

Email from Larry BandIn this case the emphasisis on first retrieving all catchments of a certain size (or range of> sizes) developed by specifying threshold areas or perhaps other > criteria for identifying first order catchments. Likely this would > actually be area as we are less interested in knowing precisely where > channelized flow begins as identifying characteristics of catchments > of specified drainage areas. In response to a set of scientific > questions or hypotheses dealing with scaling issues, we may specify we > need to gauge X streams for each of 5 orders of magnitude of drainage > area that satisfy a set of selection criteria. You're correct that > the procedure would select the set of candidate sites, from which a > final set would need to be chosen.>> The rules can be quite simple, such as 1. all catchments with > 20% > impervious cover (assuming we have an impervious surface layer), or > more complex such as 2. all catchments with > 80% forest in a > specified riparian buffer. It could also include topographic > characteristics including extent and development of floodplain using > some of the indices John Gallant has recently introduced. You're also > correct that this can be a complex problem specifically with lidar > data due to the size of the dataset and also the lack of elevation > data in open water, and the potential apparent drainage disruption due > to infrastructure. My impression is that most or at least many > hydrologists who can carry out this type of activity with standard USGS DEM would have difficulty> handling the lidar data. Most software packages cannot handle the> data volumes. I raised this as a question regarding whether this > would be an efficient use of the HIS groups time and abilities, or > whether this is too specific an application and should be left to the > individual HO. Their ability to handle these and similar problems may > be a good attribute to consider in the proposals.

UT Update

• General issues

• Landscape characterization for HO Design

• Flux algebra for surface water systems

• XML for interchange of groundwater objects

Mass Balancing

ETRain

Flow In

Flow OutFlow Out

A South Florida Basin

What volume of water is stored within this basin?

Process

Hydrologic DataModel

Select Time SeriesRelated to Basin

Horizontal Inflow

Horizontal Outflow

#1

Horizontal Outflow

#2

Vertical Inflow

Vertical Outflow

Add to Calculate aNet Flow

NetHorizontal

Inflow

NetVertical Inflow

NetTotal InflowIntegrate

To CalculateA Storage

CumulativeHorizontal

Storage

CumulativeVertical Storage

CumulativeTotal

Storage

-0.50

0.00

0.50

1.00

1.50

2.00

2.50

3.00

3.50

4.0011

/1/0

2

12/1

/02

1/1/

03

2/1/

03

3/1/

03

4/1/

03

5/1/

03

6/1/

03

7/1/

03

8/1/

03

9/1/

03

10/1

/03

Date

Ver

tical

Flu

x [in

/day

]

Evapotranspiration Rainfall

Daily Averaged Vertical Fluxes

Daily Averaged Horizontal Flow Rates

-4.00E+08

-3.00E+08

-2.00E+08

-1.00E+08

0.00E+00

1.00E+08

2.00E+08

3.00E+08

4.00E+0811

/1/0

2

12/1

/02

1/1/

03

2/1/

03

3/1/

03

4/1/

03

5/1/

03

6/1/

03

7/1/

03

8/1/

03

9/1/

03

10/1

/03

Date

Flo

wra

te [

ft3/

day]

Daily Averaged Net Flow Rates

-1.00E+03

0.00E+00

1.00E+03

2.00E+03

3.00E+03

4.00E+03

5.00E+03

6.00E+03

11/1

/200

2

12/1

/200

2

1/1/

2003

2/1/

2003

3/1/

2003

4/1/

2003

5/1/

2003

6/1/

2003

7/1/

2003

8/1/

2003

9/1/

2003

10/1

/200

3

Date

Dai

ly A

vera

ged

Net

Flo

wra

te [

cfs]

Net Horizontal Flow (cfs) Net Vertical Flow (cfs)

Cumulative Storage Since Nov. 1, 2001

-1.50E+09

-1.00E+09

-5.00E+08

0.00E+00

5.00E+08

1.00E+09

1.50E+09

2.00E+09

2.50E+09

1-Sep-02

21-Oct-02

10-Dec-02

29-Jan-03

20-Mar-03

9-May-03

28-Jun-03

17-Aug-03

6-Oct-03

25-Nov-03

Date

Vol

ume

(ft3

)

Horizontal Net Inflow Vertical Net Inflow Total Net Inflow

Complications to Process

Units Conversions

Discrete-Continuous Time

Dimensions Conversions

Need spatial and temporal integration

Need unit conversions

Need a spatiotemporal referencing system (TGIS)

Extracting time seriesNeed ability to query a large database to extract relevant time series for one or more discrete watersheds

UT Update

• General issues

• Landscape characterization for HO Design

• Flux algebra for surface water systems

• XML for interchange of groundwater objects

Creating a 3D model of the subsurface

Stratigraphy from the North Carolina database (tabular), imported into ArcGIS

Importing borehole data to GMS

Data is imported from GIS into GMS (Groundwater Modeling System)

Solid model

Solids are generated in GMS using the Horizons methodContacts are assigned horizons (from bottom to top) and then solids

are created by interpolating a surface for each horizon extruding downward.

Solid model in GIS

The solid model is read back into ArcGIS through an XML file

Transfer of the solid model via XML

• Solids can be represented as a set of vertices and triangles

• Each vertex has a x, y, and z coordinates

• Each triangle is constructed of three vertices

Storing solids in an XML file

Solids represented as a set of vertices and triangles

Vertices

Triangles

Full process

Stratigraphy information in a spatial database

Interpolation in external software(for example GMS)

Store solids in XMLBack to spatial database

Agenda

• Review the work of the five project partners– CUAHSI (Rick Hooper, Jon Duncan) – Neuse

HO report status– San Diego Supercomputer Center (John

Helly, ….)– University of Texas (David Maidment, …)– University of Illinois (Praveen Kumar ….) –

Praveen is overseas…– Drexel University (Michael Piasecki…)

The Modelshed Framework

Update July 28, 04

What is a Modelshed?• A volumetric spatial (GeoVolume?) model unit, registered in three

dimensions by a GIS, with which time-varying data, model fluxes, spatial relationships and descriptive metadata are associated

What can the Modelshed Framework do?

• Store data for diverse spatio-temporal applications & phenomena• A generalized 4D data model for environmental science• Addresses issues of scale, heterogeneity, and resolution• Build on top of existing data models (e.g. ArcHydro) to leverage

existing data structures and tools• Establish new relationships• Models environmental fluxes• Connects raster data and numerical models with object-relational

data models

Modelshed UML

-HydroID : esriFieldTypeInteger-HydroCode : esriFieldTypeString

Hydrography::HydroFeature

-ModelshedTypeID : esriFieldTypeInteger

Modelshed

Modelshed::ModelPoint

Modelshed::ModelLine

Modelshed::ModelArea

-ModelShedTypeID : esriFieldTypeInteger-FeatureID : esriFieldTypeInteger-TSTypeID : esriFieldTypeInteger-ZLayerID : esriFieldTypeInteger-TSDateTime : esriFieldTypeDate-TS_MEAN : esriFieldTypeDouble-TS_MEDIAN : esriFieldTypeDouble-TS_MAJORITY : esriFieldTypeDouble-TS_COUNT : esriFieldTypeDouble-TS_MIN : esriFieldTypeDouble-TS_MAX : esriFieldTypeDouble-TS_STD : esriFieldTypeDouble-TS_SUM : esriFieldTypeDouble-TS_SKEWNESS : esriFieldTypeDouble-TS_KURTOSIS : esriFieldTypeDouble-TS_ERROR : esriFieldTypeDouble = 0.0

StatisticalTS

1

1

*

1

-ModelshedTypeID : esriFieldTypeInteger-ModelshedClass : ModelshedClass-Description : esriFieldTypeString

ModelshedType

1

*

Timeseries UML

-FeatureID : esriFieldTypeInteger-TSTypeID : esriFieldTypeInteger-TSDateTime : esriFieldTypeDate-TSValue : esriFieldTypeDouble

TimeSeries -TSTypeID : esriFieldTypeInteger-Variable : esriFieldTypeString-Units : esriFieldTypeString-IsRegular : AHBoolean-TSInterval : TSIntervalType-DataType : TSDataType-Origin : TSOrigins

TSType

1* TSTypeHasTimeSeries

-ModelShedTypeID : esriFieldTypeInteger-FeatureID : esriFieldTypeInteger-TSTypeID : esriFieldTypeInteger-ZLayerID : esriFieldTypeInteger-TSDateTime : esriFieldTypeDate-TS_MEAN : esriFieldTypeDouble-TS_MEDIAN : esriFieldTypeDouble-TS_MAJORITY : esriFieldTypeDouble-TS_COUNT : esriFieldTypeDouble-TS_MIN : esriFieldTypeDouble-TS_MAX : esriFieldTypeDouble-TS_STD : esriFieldTypeDouble-TS_SUM : esriFieldTypeDouble-TS_SKEWNESS : esriFieldTypeDouble-TS_KURTOSIS : esriFieldTypeDouble-TS_ERROR : esriFieldTypeDouble = 0.0

StatisticalTS

1

* TSTypeHasStatisticalTS

-ModelshedTypeID : esriFieldTypeInteger-ModelshedClass : ModelshedClass-Description : esriFieldTypeString

ModelshedType1*

-ZLayerID : esriFieldTypeInteger-AltitudeUnits : esriFieldTypeString-AltitudeDatum : esriFieldTypeString-Description : esriFieldTypeString-LayerBottomAltitude : esriFieldTypeDouble-LayerTopAltitude : esriFieldTypeDouble-ZLayerAboveID : esriFieldTypeInteger-ZLayerBelowID : esriFieldTypeInteger

ZLayer1

*

Flux UML

Modelshed::ModelArea Modelshed::ModelLine Modelshed::ModelPoint

-FluxTypeID : esriFieldTypeInteger-FluxLinkID : esriFieldTypeInteger-FromFeatureID : esriFieldTypeInteger-ToFeatureID : esriFieldTypeInteger-FromZLayerID : esriFieldTypeInteger-ToZLayerID : esriFieldTypeInteger-FromModelShedTypeID : esriFieldTypeInteger-ToModelShedTypeID : esriFieldTypeInteger-TSTypeID : esriFieldTypeInteger

FluxLink

-FluxLinkID : esriFieldTypeInteger-DateTime : esriFieldTypeDate-Value : esriFieldTypeDouble

FluxRecord

-FluxTypeID : esriFieldTypeInteger-Description : esriFieldTypeString

FluxType

1 11

1

1

*

1

*

-ModelshedTypeID : esriFieldTypeInteger-ModelshedClass : ModelshedClass-Description : esriFieldTypeString

ModelshedType

-ZLayerID : esriFieldTypeInteger-AltitudeUnits : esriFieldTypeString-AltitudeDatum : esriFieldTypeString-Description : esriFieldTypeString-LayerBottomAltitude : esriFieldTypeDouble-LayerTopAltitude : esriFieldTypeDouble-ZLayerAboveID : esriFieldTypeInteger-ZLayerBelowID : esriFieldTypeInteger

ZLayer

ArcHydro NativeImplementation Classes1 *

1

*

1 *

1

*

AreaLink UML

Modelshed::ModelArea1

*

1

*

-Area1FeatureID : esriFieldTypeInteger-Area2FeatureID : esriFieldTypeInteger-ModelshedClass1 : ModelshedClass-ModelshedClass2 : ModelshedClass-FractionOf1In2 : esriFieldTypeDouble

AreaLink

-ModelshedTypeID : esriFieldTypeInteger-ModelshedClass : ModelshedClass-Description : esriFieldTypeString

ModelshedType

1

*

1

*

ArcHydro NativeImplementation Classes

OrthogonalLink UML

-ModelShedTypeID : esriFieldTypeInteger-FeatureID : esriFieldTypeInteger-posYFeatureID : esriFieldTypeInteger-posXFeatureID : esriFieldTypeInteger-negYFeatureID : esriFieldTypeInteger-negXFeatureID : esriFieldTypeInteger-posXposYFeatureID : esriFieldTypeInteger-posXnegYFeatureID : esriFieldTypeInteger-negXnegYFeatureID : esriFieldTypeInteger-negXposYFeatureID : esriFieldTypeInteger

OrthogonalLink

Modelshed::ModelArea

-ModelshedTypeID : esriFieldTypeInteger-ModelshedClass : ModelshedClass-Description : esriFieldTypeString

ModelshedType

1

*

1

1

Applications: Helping Raster & Vector Talk

• How can continuous data in rasters be related to database objects?

– Summarize the data using statistics, aggregated by overlapping Modelshed areas

– Statistics are stored as indexed data records

– Modelsheds can be physically meaningful, like watersheds

– This process can be automated for a large number of rasters

Applications: Helping Raster & Vector Talk

Applications: Automating data management with the Modelshed Tools

• The ModelShed Tools automate some database tasks:– Adding new descriptive indexes– Building the index of raster datasets– Automatically processing a timeseries of raster datasets based

on areas in the database, and ingesting the statistical data into the database

– Building AreaLink tables

• ModelShed Tools are an extension to ArcGIS 8, and use ArcGIS Spatial Analyst geoprocessing routines

Dynamic Features

• Supports database features that move and change in time

• The full range of Modelshed features are still supported, including vertical indexing, flux links, and area links.

• A parallel UML structure for static and dynamic features

Dynamic Features in Time

t1

t2

t3t4

11

1

*

1

-DynamicFeatureID-ZLayerID-TSDateTime

DynamicFeature

DynamicPoint{GeometryType = esriGeometryPoint}

DynamicArea{GeometryType = esriGeometryPolygon}

DynamicLine{GeometryType = esriGeometryPolyline}

Modelshed::Modelshed-HydroID-HydroCode

Hydrography::HydroFeature

-DynamicFeatureID-Description-ModelshedTypeID

Objects::DynamicFeatureIndex

*

1

*

*

-ModelshedTypeID-FeatureID-TSTypeID-ZLayerID-TSDateTime-TS_MEAN-TS_MEDIAN-TS_MAJORITY-TS_COUNT-TS_MIN-TS_MAX-TS_STD-TS_SUM-TS_SKEWNESS-TS_KURTOSIS-TS_ERROR = 0.0

Objects::StatisticalTS

1

*1

1

-FluxTypeID-FluxLinkID-FromFeatureID-ToFeatureID-FromZLayerID-ToZLayerID-FromModelshedTypeID-ToModelshedTypeID

Objects::FluxLink

11

-Area1FeatureID-Area2FeatureID-ModelshedTypeID1-ModelshedTypeID2-FractionOf1In2

Objects::AreaLink

1

*

1

*

Applications 2: ILRDB

• A prototype geodatabase of the Illinois River Basin using the Modelshed geodata model

• Combining base hydrography from the NHD / ArcHydroUSA database with supercomputer-generated regional climate data, remote sensing data, land use data, and multi-layer soils data

• A proof of concept for study using a much more extensive multi-disciplinary integrated database

SALT

SPOON

KANKAKEE

IROQUOIS

UPPER FOX

LOWER ILLINOIS

LA MOINE

DES PLAINES

VERMILION

MACKINAW

MACOUPIN

LOWER FOX

UPPER SANGAMON

CHICAGO

UPPER ILLINOISLOWER ILLINOIS-SENACHWINE LAKE

LOWER SANGAMON

SOUTH FORK SANGAMON

LOWER ILLINOIS-LAKE CHAUTAUQUA

Illinois River Basin Database (ILRDB)

Studying the relationships between large-scale phenomena and hydrology using the ILRDB

• Climate simulation precipitation and humidity data is modeled along with NDVI vegetation and surface hydrology

• Query-based analysis is used to analyze the relationships between these datasets

0

0.5

1

1.5

2

2.5

3

3.5

1 2 3 4 5 6 7 8 9 10 11 12

month

rati

o t

o s

eri

es m

ean

monthly average streamflow in the Illinois River at Valley City, IL

climate simulation precipitation

normalized difference vegetation index

moisture flux

Agenda

• Review the work of the five project partners– CUAHSI (Rick Hooper, Jon Duncan) – Neuse

HO report status– San Diego Supercomputer Center (John

Helly, ….)– University of Texas (David Maidment, …)– University of Illinois (Praveen Kumar ….) –

Praveen is overseas…– Drexel University (Michael Piasecki…)

Drexel Progress

CUAHSI

July 28 2004

Controlled Vocabulary for the Neuse River Basin

ONTOLOGIES

Stream Gauges Datums Site TypesCountiesAgencies

Soil Types *Municipal Wells *Units *

* In progresshttp://loki.cae.drexel.edu/~how/cuahsi/2004/07/neuse-station.owl

MTF and MIF files based on ISO-19115

Example for municipal wells… end of this week. Controlled vocabularies will be used to annotate the values

MTF Available in the Web : Version 01 based on ISO:19115http://loki.cae.drexel.edu/~how/cuahsi/2004/07/cuahsi_v01.mtf

Agenda

• Review the work of the five project partners– CUAHSI (Rick Hooper, Jon Duncan)– San Diego Supercomputer Center (John Helly, ….)– University of Texas (David Maidment, …)– University of Illinois (Praveen Kumar ….)– Drexel University (Michael Piasecki…)

• Involving the collaborators: V. Lakshmi, X. Liang, Y. Liang, U. Lall, L. Poff, K. Reckhow, D. Tarboton, I. Zaslavsky, C. Zheng

• HIS review meetings – SDSC (August 12-13) – technical detail– Logan (August 24) – user needs assessment

Agenda

Involving the collaborators: V. Lakshmi, X. Liang, Y. Liang, U. Lall, L. Poff, K. Reckhow, D. Tarboton, I. Zaslavsky, C. Zheng

• Development of concept papers for particular areas of HIS– LeRoy Poff: an assessment of needs for an HIS to

support aquatic ecology……– Manu Lall: a survey of methodology for statistical space-

time interpretation of data……– We have a draft of a written scopes for Manu’s paper

and it has been reviewed by the group

Agenda

• HIS review meetings – SDSC (Thursday, Friday, August 12-13) – technical

detail on HIS development especially on metadata definition http://cuahsi.sdsc.edu/

– Logan (Monday, August 23) – Status report on HIS project and assessment of user needs for HIS in the hydrologic observatories http://www.usu.edu/water/cuahsi

• CUAHSI will provide travel funds for HIS project PI’s and collaborators to travel to one of these meetings