australian bureau of meteorology water information program - building a national water database tony...

Download Australian Bureau of Meteorology Water Information Program - Building a national water database Tony Boston Assistant Director Water Data Services

Post on 11-Jan-2016

212 views

Category:

Documents

1 download

Embed Size (px)

TRANSCRIPT

  • Australian Bureau of Meteorology Water Information Program - Building a national water databaseTony BostonAssistant DirectorWater Data Services

  • OutlineBureau of Meteorology - Water Information ProgramWater Act 2007 and RegulationsWater data transfer standardsIngestion of water observations dataAustralian Water Resources Information System (AWRIS)Australian Hydrological Geospatial Fabric (Geofabric)

  • Bureau of Meteorology functions

  • Rainfall Deciles 1 November 2001 to 31 October 2008Distribution based on Gridded Data Product of the National Climate Centre

  • The water information value ladder

  • Bureau of Meteorology - Water Information Program10-year Commonwealth program, started July 07$450m funding110 new staff + systems + R&Dsupport for data providersLegislative backingWater Act 2007Water Regulations 2008Relies on collaboration with data providersThe Bureau will become Australias leading water information provider

  • Water Regulations 2008Specify a complex matrix of categories of data, types of organisation and timeframes for delivery to the BureauInitial Regulations ask for any data in electronic format => lots of CSV with no schema!Bureau can define National Water Information StandardsData transfer standards will be mandated in future versions of the Water Regulations

  • Categories of water informationsurface water resourcesgroundwater resources information on major and minor storagesmeteorological informationwater userights, allocations and tradesurban water managementwater restrictionswater qualitymetadata for the above

  • Types of OrganisationLead State/Commonwealth water agenciesOther State/Commonwealth agenciesHydroelectricity generatorsOwners or operators of major storagesRural water utilitiesUrban water utilitiesCMAs and like organisationsFlood level information providers

  • Data Transfer to Bureau of MeteorologyCanberraSource: David Maidment 2009

  • Water Regulations data delivery

    ten primary categories of data (~65 variables)eight categories of person (246 persons in total)104-105 observation points with time seriesentire historical archive provided at firstupdated thereafter real time, daily, weekly, monthly or yearly

  • Water data transfer standardsUnder WIRADA research alliance, CSIRO and the Bureau have developed the Water Data Transfer Format (WDTF)Version 0.2 September 2008Version 0.3 February 2009Version 1.0 October 2009Developed through the Bureaus Water Information Research and Development Alliance (WIRADA) with CSIRO$50M over 5 years

  • Water Data Transfer Format (WDTF)XML format Based on standards stack from Open Geospatial Consortium (OGC):Geography Markup Language (GML)Observations and Measurements (O&M)Simple features profile=> compatible with WFS and SOS=> compatible with other environmental data

  • Development of WaterML 2.0Aim: Harmonize Australian WDTF and CUAHSI WaterML 1.X Link to more hydrologic feature-types . not just point monitoring sitesImproved vocabulary/ontology management, mapping, publication

    international standard for water observations data, interoperable with other domains in environmental sciencesCSIROCUAHSIOGC Hydrology Domain Working GroupWMO Commission for Hydrology BoM

  • WaterML 2.0 evolution

  • AWRIS Core Solution Overview

    Data

    Database

    IMAP Virtual Server

    $

    Bank

    Building 2

    Warehouse

    Warehouse

    Manufacturing

    Packaging

    Copy Center

    Personnel

    Data Ingest

    Data Maintenance

    Data Storageand Versioning

    File Package Acquisition(FTP, CD, Email)

    File Storeand Catalogue

    Format Standardisation

    Normalisation and Validation

    Holdings Catalogue

    Data Provider

    Data Ingest

    Data Maintenance

    Data Storageand Versioning

    Data Return

    Format Standardisation

    Normalisation and Validation

    Holdings Catalogue

    Data Export

    Workbenche.g. Aquarius

    Data Operator

    Data Ingest

    Data Maintenance

    Data Storageand Versioning

    File Package Acquisition(FTP, CD, Email)

    File Storeand Catalogue

    Holdings Catalogue

    Manual DataEntry / Edit

    Data Operator

    Data Provider

    Systems Management

    Data Ingest

    Data Maintenance

    File Storeand Catalogue

    Holdings Catalogue

    Format Standardisation

    Normalisation and Validation

    Data Export

    Data Storageand Versioning

    File Package Acquisition(FTP, CD, Email)

    Provider Feedback

    Data Return

    External Data Exchange(Flood, ADAM, AWDIP)

    Data Aggregation

    Data Derivation

    Manual DataEntry / Edit

    Reference Data(Spatial Data Sets,Identifier Management)

    Product Publishing

    DBA and Archiving

    Audit, Logging and Notification

    Messaging

    Access Control(Identity Mgmt, Security)

    Process Configuration(Rules and Dependencies)

    Operations Support

    Content Management

    ContentGeneration

    Dashboards

    Publishing

    Mapping

    Visualisation

    Search

    Download

    Data Ingest

    Data Maintenance

    Data Storageand Versioning

    External Data Exchange(Flood, ADAM, AWDIP)

    Format Standardisation

    Normalisation and Validation

    Holdings Catalogue

    Data Export

    External Data Repository(Flood, ADAM, AWDIP)

    Data Maintenance

    Data Storageand Versioning

    Holdings Catalogue

    Data Export

    Product Publishing

    Content Management

    Workbench, GIS,Modelling Tools

    Publishing

    PDF

    Excel, Word etc

    Hydrologist

    Systems Management

    Audit, Logging and Notification

    Messaging

    Access Control(Identity Mgmt, Security)

    Process Configuration(Rules and Dependencies)

    Operations Support

    DBA and Archiving

    Data Maintenance

    Holdings Catalogue

    Data Export

    Product Publishing

    Data Aggregation

    Data Derivation

    Reference Data(Spatial Data Sets,Identifier Management)

    Content Management

    ContentGeneration

    Data Storageand Versioning

    Dashboards

    Mapping

    Visualisation

    Search

    Download

    Publishing

    Web User

    Data Maintenance

    Reference Data(Spatial Data Sets,Identifier Management)

    Enterprise GIS

    Spatial DataExpert

    SystemManager

    Data Storageand Versioning

    Data Export

  • AWRIS: Receive Regulations Data

    Data

    Database

    IMAP Virtual Server

    $

    Bank

    Building 2

    Warehouse

    Warehouse

    Manufacturing

    Packaging

    Copy Center

    Personnel

    Data Ingest

    Data Maintenance

    Data Storageand Versioning

    File Package Acquisition(FTP, CD, Email)

    File Storeand Catalogue

    Format Standardisation

    Normalisation and Validation

    Holdings Catalogue

    Data Provider

    Data Ingest

    Data Maintenance

    Data Storageand Versioning

    Data Return

    Format Standardisation

    Normalisation and Validation

    Holdings Catalogue

    Data Export

    Workbenche.g. Aquarius

    Data Operator

    Data Ingest

    Data Maintenance

    Data Storageand Versioning

    File Package Acquisition(FTP, CD, Email)

    File Storeand Catalogue

    Holdings Catalogue

    Manual DataEntry / Edit

    Data Operator

    Data Provider

    Systems Management

    Data Ingest

    Data Maintenance

    File Storeand Catalogue

    Holdings Catalogue

    Format Standardisation

    Normalisation and Validation

    Data Export

    Data Storageand Versioning

    File Package Acquisition(FTP, CD, Email)

    Provider Feedback

    Data Return

    External Data Exchange(Flood, ADAM, AWDIP)

    Data Aggregation

    Data Derivation

    Manual DataEntry / Edit

    Reference Data(Spatial Data Sets,Identifier Management)

    Product Publishing

    DBA and Archiving

    Audit, Logging and Notification

    Messaging

    Access Control(Identity Mgmt, Security)

    Process Configuration(Rules and Dependencies)

    Operations Support

    Content Management

    ContentGeneration

    Dashboards

    Publishing

    Mapping

    Visualisation

    Search

    Download

    Data Ingest

    Data Maintenance

    Data Storageand Versioning

    External Data Exchange(Flood, ADAM, AWDIP)

    Format Standardisation

    Normalisation and Validation

    Holdings Catalogue

    Data Export

    External Data Repository(Flood, ADAM, AWDIP)

    Data Maintenance

    Data Storageand Versioning

    Holdings Catalogue

    Data Export

    Product Publishing

    Content Management

    Workbench, GIS,Modelling Tools

    Publishing

    PDF

    Excel, Word etc

    Hydrologist

    Systems Management

    Audit, Logging and Notification

    Messaging

    Access Control(Identity Mgmt, Security)

    Process Configuration(Rules and Dependencies)

    Operations Support

    DBA and Archiving

    Data Maintenance

    Holdings Catalogue

    Data Export

    Product Publishing

    Data Aggregation

    Data Derivation

    Reference Data(Spatial Data Sets,Identifier Management)

    Content Management

    ContentGeneration

    Data Storageand Versioning

    Dashboards

    Mapping

    Visualisation

    Search

    Download

    Publishing

    Web User

    Data Maintenance

    Reference Data(Spatial Data Sets,Identifier Management)

    Enterprise GIS

    Spatial DataExpert

    SystemManager

    Data Storageand Versioning

    Data Export

  • File statistics

  • The Australian Hydrological Geospatial Fabric (the Geofabric)The G