cobweb - infrastructure and platform for environmental crowd sensing and big data

COBWEB

EnviroInfo & ICT4S 2015,

Workshop - Infrastructures and Platforms

for Environmental Crowd Sensing and

Big Data,

Copenhagen,

9th Sept, 2015

Chris Higginschris.higgins@ed.ac.uk

Sta

Introduction to COBWEB

• Research Project: Funded under the European Commission’s Framework Programme 7

• Started Nov 2012 for 4 years (Month 35 of 48)

• First demonstrator completed and tested during the 2015 field season

• Why?

– GPS enabled, internet connected mobile devices now ubiquitous

– Lots of potential, eg, can citizen sourced environmental data be useful for decision making?

Citizen Observatory Web

• Generic crowdsourcing infrastructure

– A toolkit which can be downloaded and used

in multiple scenarios

• Data which supports policy

• Address data quality issues

• Open standards

Project Partners

UNESCO World Network of Biosphere Reserves

Sites of excellence to foster

harmonious integration of people and

nature for sustainable development

through participation, knowledge

sharing, poverty reduction and human

well-being improvements, cultural

values and society's ability to cope

with change, thus contributing to the

Millennium Development Goals

COBWEB Biosphere Reserves

Biosffer Dyfi Biosphere

Mount Olympus

Gorge of Samaria

Wadden See & Hallig islands

Dyfi Biosphere Reserve #1

www.biosfferdyfi.org.uk

Dyfi Biosphere Reserve #2

Contains Ordnance Survey data © Crown copyright and database right (2013)

©C

ou

ntr

ysid

e C

ou

ncil

fo

r W

ale

s. A

ll r

igh

ts r

eserved

.

Co-design

Co-design – Snowdonia National Park

• Japanese Knotweed - Fallopia japonica

• Giant Knotweed – Fallopia sachalinensis

• Hybrid knotweed – Fallopia x bohemica

Co-design – RSPB

• Domenlas Saltmarsh

– Survey vegetation

– Use quadrats

– Soil salinity levels

– Weather conditions

– Look at reversion process

• Covert Coch Peatbog

– Survey vegetation

– Use quadrats

– Soil moisture

– Grazing history

Janetbaxter.co.uk

Co-Design - Penparcau Community Forum

Co-Design - Penparcau Community Forum

• Temperature

• Humidity

• Wind speed

• Wind direction

• Pressure

• Precipitation

In situ sensors

Usability Testing

Not just apps…

A number of demonstrator mobile phone

applications

– Exactly what, deliberately left open and

subject to discussion with community

3 pilot case study areas:

1. Validating earth

observation products

2. Biological monitoring

3. Flooding

COBWEB Framework

Key components at different TRL’s

• Conflation

• QA workflow editor

• QA WPS/services

• Sensor networks

• GeoNetwork/Portal

• Middleware

• Authoring tool/Survey designer

• Apps

• User management and privacy

• Access control

• Authentication

The app itself

Key features

• Capture information

– Images

– Audio

– Text

– Location

• High quality background maps

• Saved maps for use “offline”

• Custom data collection forms

• Manual location correction

View COBWEB Portal

Join COBWEB Portal

Customise your own app for your survey

Customise your own app for your survey

Customise your own app for your survey

Citizen captures data on their phone

Approach to QA

• Quality Assurance: complex problem and often use case specific:

– Internal quality (metrics)

– External quality (fit for purpose)

• A generic system that can be easily customised to fit new use cases, based on standards

• Understanding data quality is likely to require a combination of approaches and tests, the system is designed to enable this

• As quality requirements are use case specific, the quality control tests are configurable to utilise a wide variety of datasets and parameters

Classifying quality: Seven pillarsPillar Example Test Notes

Pillar 1 – Location Based services

Assessment of spatial accuracy – estimate from a mobile device and number of satellites

Tests often carried out on the mobile device

Pillar 2 – Cleaning Removal of junk data via an attribute text check

Very lightweight, can flag or remove malicious entries

Pillar 3 – Automatic validation

Analysis whether an image is blurry

Higher level testing, often used to assess ranges

Pillar 4 – Comparison with authoritative data

Use of a set of boundary polygons to check whether an observation is in or out

Wide variety of tests that involve comparison with what it known

Pillar 5 – Model based validation

Running a flood model Can be complex, and may also include question based modeling

Pillar 6 – Big/Linked data Querying Twitter via a hashtag for similar phenomena

Tapping into large databases such as sensor records and social media

Pillar 7 – Semanticharmonisation

Rationalisation of entries via an ontology

Attempts to recognisemultiple entries of the same observation

Web Processing Service

• OGC Standard

• Web facing

• Public

• Holds spatial and non-spatial processes

• Processes are suited to chaining

• BPEL is the traditional method of chaining

– However, removed from WPS 2.0

BPMN Workflow Engine

• JBOSS JBPM – A Java based BPM workflow engine

• Orchestrates processes in a given order with defined inputs

• Chains results from one process to the next

• Can be executed remotely via REST

• Graphical interface utilises BPMN2.0 – a recently ratified OMG standard

• Unlike BPEL this interface is standardised

Current questions

• What is the Post Quality database. How to manage different schemas and different data sources?• RDF to handle the fuzzy view of crowdsourced

data?• How is client side QA being managed?

• GeoPackage + watching service?• Client side vs server side

• Security• Can we protect certain datasets/processes and

make them available to those who have the correct credentials?

• Many others!!

For more information on QA – see:

• Relevant sessions at the OGC Technical Committee in Nottingham next week

• Architecture Implementation Pilot – 8 deep dive videos:

http://www.ogcnetwork.net/node/1969

Service Provider (SP) Identity Provider (IdP)Discovery Service (DS)

“GEOSS user” Single-Sign-On

Trust Gateway (TG) to OpenID

Google OpenId

COBWEB/GEOSS AIP-6 Federation

NASA Ames

Secure Dimensions

CUAHSI*Catapult

University of Edinburgh

Kst. GDI.DE

*: Consortium of Universities for the Advancement of Hydrologic Science

EarthServer (FP7) project

MEEO

Remainder of project…emphasis shifting

• Rolling out software

• Getting greater buy-in

• Sustainability

• More focus on research agenda

• Greater focus on Greece, Germany

• Additional Biosphere Reserves

Thank you

• Website: http://cobwebproject.eu/

• Mailing list: http://eepurl.com/Ita05 or use the QR code

• Follow us: @cobwebfp7

• And look out for members of the COBWEB team at events.

chris.higgins@ed.ac.uk