GeoViQua: GeoViQua: the quality challenges for the quality challenges for

GEOSSGEOSS

YANG Xiaoyu, BLOWER Jon, CORNFORD Dan, LUSH Victoria, MASO Joan, ZABALA Alaitz, Nüst Daniel

Center of Research in Ecology and Forestry Applications (CREAF)contact@geoviqua.org

www.geoviqua.org

QUAlity aware

VIsualisation for the

Global Earth Observation system of systems

www.geoviqua.org

The problem

• Is there quality information in the GCI?– There is some in the form of ISO19115 DQ elements and lineage– Not enough

• The GEOSS Common Infrastructure does not follow a global model for quality

• The GEOPortal search and results – are not ranged by quality– quality indicators are not shown

• Common data viewers do not generally include quality information in parallel with the data

www.geoviqua.org

The aim

GeoViQua will provide a set of scientifically developed software components and services that facilitate the creation, search and visualization of quality information on EO data integrated and validated in the GEOSS Common Infrastructure.

Pilot case studies

CC RR OO SS SS

SS BB AA

Communitybuilding

GEO S&T Label

www.geoviqua.org

Time table

Start PrototypesValidation

Mobile Solutions

Search & Visualization

Data ready

Quality recommendations

Testing

solutionsPilot cases

User & technical requirements to CoP

User & technical solutions to CoP

Workshops

Proposals evaluation Final documentGeoLabel

Metadata extraction

Best practices quality encoding

Direct extraction from continuous variables

Quality elicitation User feedbackExtraction from categorical variables

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36

Febr

uary

201

1

Janu

ary

2012

Janu

ary

2013

Dec

embe

r 201

3

2014

Requirements and Data Model phase finished,

www.geoviqua.org

Community Views on Data Quality

• Many researchers refer to the ‘famous five’ as the common criteria for evaluating spatial data quality– lineage; completeness; consistency; positional accuracy; and

attribute accuracy.

• Broad scientific acceptance of the common spatial quality elements does not apply to all cases for “fitness-for-use” evaluation– user requirements can go far beyond the widely accepted ‘famous

five’.

• We used semi-structured telephone and face-to-face interviews with a variety of geospatial data users and experts from a number of countries and application domains.

www.geoviqua.org

What users want?

• Users are exceedingly interested in good quality metadata records – And information that can help to assess fitness-for-use of the data

• Users find metadata records typically incomplete with essential data omitted– The process of dataset discovery and selection is more difficult

• Users are also interested in ‘soft’ knowledge about data quality– Data providers’ comments on the overall quality of a dataset, known data errors, potential

data usage– Peers’ reviews and recommendations (they contact their peers to obtain suggestions)– Dataset provenance, citation and licensing information

• Citation is incomplete (lack of valid producer contact details), and licensing often missing• Citation: users rely on data from good reputation producers

• Currently, some of these cannot be recorded in standard metadata

• Need for easily and systematically compare metadata records– Side-by-side visualisation of all metadata elements would allow geospatial datasets to be

compared more effectively, • especially when datasets are very similar and differences are hard to distinguish

www.geoviqua.org

Producer’s-consumer’s quality

• Producer’s quality metadata– In the producers metadata records– Encoded in the classical ISO 19115/19139– Some extensions required– Stored in the current catalogues (GEOSS Clearinghouse, etc)

• Consumer’s quality metadata– In independent metadata repositories– Linked to producer’s metadata by id– Future component of the GCI?– Contains comments, “like it”, star rates, etc

www.geoviqua.org

The ISO classical view

Quality indicators Provenance/Lineage

Usage

www.geoviqua.org

Add ‘soft’ knowledge to producer’s metadata

Data Quality

Quality Element Non-quantitative Quality Information

Positional Accuracy

Temporal Accuracy

Omission Commission

Missing Items

Number of Missing Items

••

Thematic Accuracy

Quality Parameter (ISO 19157)

Completeness

QualityScope

Metadata

Logical Consistency Usability

Quantitativeattribute accuracy

Non-quantitativeattribute correctness

Classification correctness

Misclassification rate

Misclassificationmatrix

••

Quality Indicator (ISO 19157)

Quality Measure (ISO19157, UncertML)

Dataset series

Dataset

Subset of data

Metadata Packages

0..*

Metaquality

User Feedback

Publication

••

Lineage

Discovered Issues Universe of

Discourse

Feature Type

www.geoviqua.org

Quality model is much more that positional accuracy

• There are many quantifiable aspects that can be recorded– Consistency, completeness, positional,

thematic and temporal accuracy…• There are many qualitative aspects that are

needed– Lineage (traceability), scientific papers, user

feedback, data usage…

www.geoviqua.org

GeoViQua Data model: statistical uncertainties

<gmd:DQ_QuantitativeAttributeAccuracy><gmd:result>

<gmd:DQ_QuantitativeResult><gmd:valueUnit>m</gmd:valueUnit>

<gmd:value> <gco:Record>3.6</gco:Record>

</gmd:value></gmd:DQ_QuantitativeResult>

</gmd:result></gmd:DQ_QuantitativeAttributeAccuracy>

<gmd:DQ_QuantitativeAttributeAccuracy><gmd:result>

<gmd:DQ_QuantitativeResult><gmd:valueType>

<gco:RecordType xlink:href=“http://www.uncertml.org/distributions/normal”>Value of the vertical DEM accuracy

</gco:RecordType></gmd:valueType><gmd:valueUnit>m</gmd:valueUnit>

<gmd:value> <gco:Record>

<un:NormalDistribution><un:mean>1.2</un:mean><un:variance>3.6</un:variance>

</un:NormalDistribution></gco:Record>

</gmd:value></gmd:DQ_QuantitativeResult>

</gmd:result></gmd:DQ_QuantitativeAttributeAccuracy>

Explicit recognition that errors acceptably fit a Normal distribution with mean 1.2 • An overall positive bias was observed • A difficult feature to convey by traditional means)

www.geoviqua.org

The need for a measure dictionary

Absolute external positional accuracy 2Anweisung Straßeninformationsbank (Bundes… 1Codelist omission 2completeness 198Feature represented as a single object 2horizontal 3146Horizontal Positional Accuracy 3265Lagegenauigkeit 3Latitude Resolution 3437Longitude Resolution 3350Mean value of positional uncertainties (2D) 3Overlapping polygon 2Quantitative Attribute Accuracy Assessment 255Rate of missing items 87Sach- und Geodatenüberprüfung 7Temporal Resolution 2870Überprüfung der Toplogie 2Valid code Test 2Vertical Positional Accuracy 1826Vertical Resolution 812vertikal 348Vollständigkeit 4

• Current quality measure names in the GCI– Nothing to do with

ISO19138 list of possible measures

– Not well defined

www.geoviqua.org

Data Quality Measure Dictionary

• Some quality indicators are used, but the name and description of the measure used to derive the indicator are rarely well described.

• Problems can occur due to the lack of semantic definitions of quality measures.

– “uncertainty at 90% significance level” ??. • A Quality Measure Dictionary is proposed that

includes:– vocabularies for quality measures– associated semantic annotations – integrate UncertML concepts and vocabularies.

• Composed on quality measures provided by – ISO138 ISO19157 – UncertML.

• Measure has a unique ID– quality element, value type, quality basic measure,

description, example use, etc. • “uncertainty at 90% significance level” can be

annotated using UncertML vocabulary “ConfidenceInterval”(URI: http://www.uncertml.org/statistics/confidence-interval)

Description

Quality element

Basic measureValue type

Definition

Value structureParameter

Example use

UncertMLrepresentation

(URI=“”)

Source reference

Quality Measure ID(ID=“” Name=“”, Alias=“”)

UncertMLDictionary

URI

<un:ConfidenceInterval xmlns:un="http://www.uncertml.org/2.0">   

<un:lower level="0.05">      <un:values>3.14</un:values>

</un:lower><un:upper level="0.95">

<un:values>6.28</un:values></un:upper>

</un:ConfidenceInterval> 

www.geoviqua.org

Quality Metadata Levels

777 333 --- 333 000

Level: theme=contour line Overwrite positional accuracy:

1.5 m

Level: sheet=73-30 Overwrite content date:

October 2009

Level: dataset (theme=contour line, sheet=73-30)

Positional accuracy: 1.5 m Content date: October 2009

Level: Multiseries Positional accuracy: 2.5 m

Content date: 2009-2010

Multiseries

Series Sheet or Scene

Dataset (raster or feature instance)

www.geoviqua.org

Registered Community Resources

Community Portals

Client Applications

Client Tier

Business Process Tier

CommunityCatalogues

AlertServers

WorkflowManagement

ProcessingServers

Access Tier

GEONETCastProduct Access

ServersSensor Web

ServersModel Access

Servers

GEOSSClearinghouse

GEO Web Portals

GEOSS Common Infrastructure

Components & Services

Standards andInteroperability

Best PracticesWiki

User Requirements

Registries

Main GEOWeb Site

GEOSS common infrastructure

www.geoviqua.org

Before GEOSS

Access Tier

GEONETCast

Product AccessServers

Sensor WebServers

Model AccessServers

Business Process Tier

CapacityCatalogues

Capacity Resource

User

SBA

Disasters

Health

Energy

Climate

Water

Weather

Ecosystems

Agriculture

Biodiversity

www.geoviqua.org

GEOSS Common Infrastructure

How GEOSS worked yesterday

Access Tier

GEONETCast

Product AccessServers

Sensor WebServers

Model AccessServers

Business Process Tier

CapacityCatalogues

Capacity Resource

User

SBA

Disasters

Health

Energy

Climate

Water

Weather

Ecosystems

Agriculture

Biodiversity

Components & Services

Registry

GEO Web Portal

GEOSSClearinghouse

Catalogue

DB

www.geoviqua.org

GEOSS Common Infrastructure

How GEOSS is going to work

Access Tier

GEONETCast

Product AccessServers

Sensor WebServers

Model AccessServers

Business Process Tier

CommunityCatalogues

Capacity Resource

User

SBA

Disasters

Health

Energy

Climate

Water

Weather

Ecosystems

Agriculture

Biodiversity

Components & Services

Registry

GEO Web Portal

GEOSSClearinghouse

Catalogue

DB

CommunityCatalogueCommunity

CatalogueCommunityCatalogueCapacity

Catalogue

EuroGEOSSBroker

www.geoviqua.org

GEOSS Common Infrastructure

How GEOSS is going to work

Access Tier

GEONETCast

Product AccessServers

Sensor WebServers

Model AccessServers

Business Process Tier

CommunityCatalogues

Capacity Resource

User

SBA

Disasters

Health

Energy

Climate

Water

Weather

Ecosystems

Agriculture

Biodiversity

Components & Services

Registry

GEO Web Portal

GEOSSClearinghouse

Catalogue

DB

CommunityCatalogueCommunity

CatalogueCommunityCatalogueCapacity

Catalogue

EuroGEOSSBroker

EuroGEOSSBroker

www.geoviqua.org

Data Quality

Quality Element Non-quantitative Quality Information

Positional Accuracy

Temporal Accuracy

Omission Commission

Missing Items

Number of Missing Items

••

Thematic Accuracy

Quality Parameter (ISO 19113)

Completeness

QualityScope

Metadata

Product Specification

Logical Consistency Usability

Quantitativeattribute accuracy

Non-quantitativeattribute correctness

Classification correctness

Misclassification rate

Misclassificationmatrix

••

Quality Indicator (ISO 19113)

Quality measure (ISO19114/ISO19138, UncertML)

Dataset series

Dataset

Subset of data

Feature types

Rules Quality requirements

Metadata Packages

0..*

Universe of Discourse (i.e. Reality)

Metaquality

Comments/ Peer Review

Publication

••

Lineage

Discovered Issues

GeoViQua quality model

EuroGEOSSBroker model

GeoViQua Model

www.geoviqua.org

GEOSS Common Infrastructure

Quality in GEOSS

Access Tier

GEONETCast

Product AccessServers

Sensor WebServers

Model AccessServers

Business Process Tier

CapacityCatalogues

Capacity Resource

User

SBA

Disasters

Health

Energy

Climate

Water

Weather

Ecosystems

Agriculture

Biodiversity

Components & Services

Registry

GEO Web Portal

GEOSSClearinghouse

Catalogue

DB

CommunityCatalogueCommunity

CatalogueCommunityCatalogueCapacity

Catalogue

EuroGEOSSBroker

Enhanced geo-search

tools

www.geoviqua.org

Including data quality in search

• SELECT WHERE positional_accuracy < 20 and classification_correctness > 90%FROM GEOSS_GCI

Devillers R, Bédard Y, R Jeansoulin (2005) Multidimensional Management of Geospatial Data Quality Information for its Dynamic Use Within GIS

Enhanced geo-search

tools

www.geoviqua.org

Consumer’s data quality

• More informal• Based on social network patterns

– Comments– Linked data– Like it– Star ratings

• More dinàmic• Need for an encoding• Need for an independent repository

www.geoviqua.org

GEOSSBack

http://www.ogc.uab.cat/GEOSSBack

• Just a prototype to play with and demonstrate a concept.

www.geoviqua.org

Producer’s+consumer’s GeoViQua Broker

cmp GeoViQua Components Agreed So Far

EuroGEOSS Discov er broker Q

CSW Clearinghoure

Capacity Catalogues

SOS-Q + SensorML

WMS

Metadata Import tool

+ HDF+ netCDF+ others...

WAF

Sensor Registry Q

SOS-Q + SensorML

FeedBack Serv er

GeoViQua Broker

CSW-Q

CSW

unknown

CSW

www.geoviqua.org

Quality Metadata comparison

www.geoviqua.org

Conclusions

• After user interviews• Producer’s quality model

– GeoViQua quality model is based in ISO– With extensions for ‘soft’ knowledge– Inclusions of uncertML– Quality measure dictionary

• Consumer’s quality model– Based on social network patterns– Encoded independently (from producers)

• Linked by the GeoViQua broker (extension/complement of the EuroGEOSS broker)

www.geoviqua.org

• What is it?– The GEO Label is intended to “assist the user to assess the scientific relevance,

quality, acceptance and societal needs of the components” (ST-09-02 Task Team, 2010).

• Purposes?– be a quality indicator for GEOSS geospatial data and datasets

• Problem: Usability depends on data application; there is no defined threshold.– improve user recognition and trust in validated datasets.

• Problem: who is going to certify this?– assist in searching by providing users with visual clues of dataset quality and

relevance.– provide accreditation, provenance, monitoring– increase visibility of EO data– Emphasize in open access and easy availability

• Possible shape?– Certification label– A formal way to present

• quality indicators• provenance• attribution

GEOLabel

Task performed in collaboration with EGIDA FP7 project and the GEO task ST-09-02

www.geoviqua.org

• Until the end of this week• Publicly available in the web• We encourage you to participate!

GEOLabel

Please participate in the questionnaire:

http://geolabel.questionpro.com just a couple of days left!!

Thanks

Joan.Maso@uab.cat(CREAF)

Please participate in the questionnaire:

http://geolabel.questionpro.com just a couple of days left!!

Thanks

Joan.Maso@uab.cat(CREAF)

www.geoviqua.org

GEOSS Common Infrastructure

How GEOSS is going to work

Access Tier

GEONETCast

Product AccessServers

Sensor WebServers

Model AccessServers

Business Process Tier

CapacityCatalogues

Capacity Resource

User

SBA

Disasters

Health

Energy

Climate

Water

Weather

Ecosystems

Agriculture

Biodiversity

Components & Services

Registry

GEO Web Portal

GEOSSClearinghouse

Catalogue

DB

CommunityCatalogueCommunity

CatalogueCommunityCatalogueCopacity

Catalogue

EuroGEOSSBroker

Quality AccessBroker

Quality aware visualisation

tools

www.geoviqua.org

Quality map visualization

• Dark color represents poor quality and light color good quality

Blackmond Laskey K, EJ. Wright PCG da Costa (2009) Envisioning uncertainty in geospatial information

Quality aware visualisation

toolsExpress data quality using maps

Devillers R, Bédard Y, R Jeansoulin (2005) Multidimensional Management of Geospatial Data Quality Information for its Dynamic Use Within GIS

www.geoviqua.org

• 3D representations– representation of

estimated water balance surplus/deficit and their uncertainty (using bars above and below the surface).

• Map representations have some problems– Makes visualization more complicated

and difficult to understand

– Attracting the attention to the more uncertain objects!!

MacEachren AM, A Robinson, S Hopper, S Gardner, R Murray, M Gahegan, E Hetzler (2005) Visualizing Geospatial Information Uncertainty; What We Know and What We Need to Know

Pang A (2001) Visualizing Uncertainty in Geo-spatial Data

Quality aware visualisation

tools

Quality map visualization

www.geoviqua.org

Pilot Case scenarios

Agriculture

Based on many user stories among GEOSS SBA

Global Carbon

Air Quality

Please participate in the questionnaire:

http://geolabel.questionpro.com just a couple of days left!!

Thanks

Joan.Maso@uab.cat(CREAF)