practical issues of sensor web implementation and gridification kussul n., korbakov m., kravchenko o

Practical issues of Sensor Web Practical issues of Sensor Web implementation and implementation and gridificationgridification

Kussul N., Korbakov M., Kravchenko O.

OutlookOutlook

• Sensor Web: overview

• Test case: floodings

• SensorML: experience

• Sensor Observation Service: experience

• Sensor Web: gridification

• Our plans

Sensor Web: the purposeSensor Web: the purpose

• Integration of heterogeneous sensors into the information infrastructure

• Sensors discovery and data access

• Composition of dataflows between system components

• Events triggering by sensors conditions

OGC O&M Observations & Measurements Approved

SensorML Sensor Model Language Approved

TransducerML Transducer Model Language Approved

OGC SOS Sensor Observations Service Approved

OGC SPS Sensor Planning Service Approved

OGC SAS Sensor Alert Service In progress

OGC WNS Web Notification Services In progress

OpenGIS StandardsOpenGIS Standards

• SW Enablement working group at OGC have developed a number of standards governing different aspects of Sensor Web

Test CaseTest Case

• The task under study is flooding in different regions of world (requested by Red Cross )

• Particular test case is floodings in Mozambique

Test case: data flowTest case: data flow

Internet

RSGSESA LAADS

Data Storage

Envisat, ERS-2 Radarsat MODIS

USGS EarthExplorer

Landsat

RSGS Grid

Envisat/ASAR WSM&GM

Processing subsystem

Optical-based floodextent extraction

SAR-based flood extentextraction

Water bodiescartography

Visualization subsystem

UMN MapServerWeb-server/OpenLayers

Internet

ESA G-POD

UA Space Grid

InterGrid

Users

EUMETCastMSG, MetOp

Data integration

Test Case: data sourcesTest Case: data sources

• ASAR

• MODIS

• MERIS

• LandSat

• DEM

Weather model

SOS

SOS

SPS

Hydrologicalmodel

SOS

Sim

ulat

ion

data

Sim

ulat

ion

data

Order

SOS

Weatherstation

SOS

Hydrologicalstation

Mea

sure

men

ts

Mea

sure

men

ts

SAS

Test Case: SW perspectiveTest Case: SW perspective

Test Case: MozambiqueTest Case: Mozambique

http://floods.ikd.kiev.ua

Test Case: MozambiqueTest Case: Mozambique

SensorMLSensorML

• Sensor modeling language is the cornerstone of all SW services

• It provides comprehensive description of sensor parameters and capabilities

• It can be used for describing different kind of sensors:– Stationary or dynamic– Remote or in-situ– Physical measurements or simulations

SensorML: exampleSensorML: example

..............<inputs> <InputList> <input name="ambiantTemperature"> <swe:Quantity definition= "urn:ogc:def:phenomenon:temperature"/> </input> <input name="atmosphericPressure"> <swe:Quantity definition= "urn:ogc:def:phenomenon:pressure"/> </input> <input name="windSpeed"> <swe:Quantity definition= "urn:ogc:def:phenomenon:windSpeed"/> </input></InputList></inputs>..............

.............<outputs> <OutputList> <output name="weatherMeasurements"> <swe:DataGroup> <swe:component name="time"> <swe:Time definition="urn:ogc:def:phenomenon:time“ uom="urn:ogc:def:unit:iso8601"/> </swe:component> <swe:component name="temperature"> <swe:Quantitydefinition="urn:ogc:def:phenomenon:temperature uom="urn:ogc:def:unit:celsius"/> </swe:component> <swe:component name="barometricPressure"> <swe:Quantity definition="urn:ogc:def:phenomenon:pressure“ uom="urn:ogc:def:unit:bar" scale="1e-3"/> </swe:component> <swe:component name="windSpeed"> <swe:Quantity definition="urn:ogc:def:phenomenon:windSpeed“ uom="urn:ogc:def:unit:meterPerSecond"/> </swe:component> </swe:DataGroup> </output> </OutputList></outputs>.............

SensorML: WRF modelSensorML: WRF model

• Modeling and simulation are very important parts of environmental monitoring

• Sensor Web infrastructure should be able to integrate modeling data in convenient way

• We have tried to describe weather modeling process using WRF numerical model in terms of SensorML

An example of single model input in SensorML:

<sml:input name="QVAPOR"> <swe:DataArray definition="urn:ogc:def:phenomenon:time"> <swe:elementCount> <swe:Count definition="urn:ogc:def:property:OGC:numberOfPixels"><swe:value>1</swe:value></swe:Count> </swe:elementCount> <swe:elementType name=""> <swe:DataArray definition="urn:ogc:def:phenomenon:altitude"> <swe:elementCount> <swe:Count definition="urn:ogc:def:property:OGC:numberOfPixels"><swe:value>30</swe:value></swe:Count> </swe:elementCount> <swe:elementType name=""> <swe:DataArray definition="urn:ogc:def:phenomenon:latitude"> <swe:elementCount> <swe:Count definition="urn:ogc:def:property:OGC:numberOfPixels"><swe:value>202</swe:value></swe:Count> </swe:elementCount> <swe:elementType name=""> <swe:DataArray definition="urn:ogc:def:phenomenon:longtitude"> <swe:elementCount> <swe:Count definition="urn:ogc:def:property:OGC:numberOfPixels"><swe:value>219</swe:value></swe:Count> </swe:elementCount> <swe:elementType name=""> <swe:Quantity definition="urn:ogc:def:phenomenon:QVAPOR"><swe:uom code="kg_kg-1"/></swe:Quantity> </swe:elementType> </swe:DataArray> </swe:elementType> </swe:DataArray> </swe:elementType> </swe:DataArray> </swe:elementType> </swe:DataArray></sml:input>

SensorML: WRF modelSensorML: WRF model

Axis descriptionsAxis descriptions

Value description

SensorML: WRF modelSensorML: WRF model

• There are nearly 50 inputs and 20 outputs for basic WRF configuration

• Each of them requires quite significant amount of XML code to be properly described

• It would be great if next revision of SensorML will include some elements for simpler description of multidimensional data

• Another negative issue is inconsistency between SML specification, published XML schemas and educational materials

Sensor Observation ServiceSensor Observation Service

• We have studied two possible implementations of Sensor Observation Service (SOS) for serving temperature sensors data

• Implementations under study were:– UMN Mapserver v5 (http://mapserver.gis.umn.edu/)– 52North SOS (http://52north.org/)

• The conclusion of study follows: there isn’t (yet) really good and reliable solution for serving data through SOS protocol

• However for some cases 52North’s implementation provides good experience

Sensor Observation ServiceSensor Observation Service

• UMN Mapserver (as SOS server)– Pros:

• Very good and reliable abstraction for different data sources (raster files, spatial databases, WFS, etc)

• Simple application model (CGI executable)• Wide set of features beside SOS• Open software

– Cons:• SOS support is declared but far from being working

implementation• Poor documentation on SOS topic• Strange plans for future development (automatic

SensorML generation)

Sensor Observation ServiceSensor Observation Service

• 52North SOS– Pros:

• SOS implementation is stable and complete• Platform-independent (Java-based)• A part of wider SW implementations stack (SPS, SAS)• Open software• Source code is clean and easily reusable

– Cons:• No data abstraction: the only data source is relational

database of specific structure• Database structure is far from optimal (strings as primary

keys, missed indexes, etc)• Complex application model (Java web application)

Sensor Observation ServiceSensor Observation Service

• We have used 52North implementation for building a testbed SOS server:– http://web.ikd.kiev.ua:8080/52nsos/sos

• Server is providing data of temperature sensors over Ukraine and South Africa region

• Data comes from PostGIS database with some tweaks to make is compatible with 52North database structure (VIEWS, index tables, etc)

• Performance is quite good for our DB. Yet, for other DBs such adaptations could lead to unacceptable drops in performance

Sensor Observation ServiceSensor Observation Service

• The database accessed through SOS contains more than 2 millions records

• Served by PostgreSQL with PostGIS extensions• We have used a number of VIEWS and stored

procedures to adapt it to 52North• Queries are complex:

SELECT observations."time" AS time_stamp, "procedure".procedure_id, feature_of_interest.feature_of_interest_id, phenomenon.phenomenon_id, offering.offering_id, '' AS text_value, observations.t AS numeric_value, '' AS mime_type, observations.oid AS observation_id FROM observations, "procedure", proc_foi, feature_of_interest, proc_off, offering_strings offering, foi_off, phenomenon, proc_phen, phen_off WHERE "procedure".procedure_id::text = proc_foi.procedure_id::text AND proc_foi.feature_of_interest_id::text = feature_of_interest.feature_of_interest_id AND "procedure".procedure_id::text = proc_off.procedure_id::text AND proc_off.offering_id::text = offering.offering_id::text AND foi_off.offering_id::text = offering.offering_id::text AND foi_off.feature_of_interest_id::text = feature_of_interest.feature_of_interest_id AND proc_phen.procedure_id::text = "procedure".procedure_id::text AND proc_phen.phenomenon_id::text = phenomenon.phenomenon_id::text AND phen_off.phenomenon_id::text = phenomenon.phenomenon_id::text AND phen_off.offering_id::text = offering.offering_id::text AND observations.wmoid::text = feature_of_interest.feature_of_interest_id;

Sensor Observation ServiceSensor Observation Service

• Despite of complex queries and large datasets SOS service shows linear scalability with respect to query depth:

0

200

400

600

800

1000

1200

1400

1600

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24

Months

ms

Sensor Observation ServiceSensor Observation Service

Sensor Observation ServiceSensor Observation Service

• Example of single SOS measurement...

<om:Measurement gml:id="o255136"> <om:samplingTime> <gml:TimeInstant xsi:type="gml:TimeInstantType"> <gml:timePosition>2005-04-14T04:00:00+04</gml:timePosition> </gml:TimeInstant> </om:samplingTime> <om:procedure xlink:href="urn:ogc:object:feature:Sensor:WMO:33506"/> <om:observedProperty xlink:href="urn:ogc:def:phenomenon:OGC:1.0.30:temperature"/> <om:featureOfInterest> <sa:Station gml:id="33506"> <gml:name>WMO33506</gml:name> <sa:sampledFeature xlink:href=""/> <sa:position> <gml:Point> <gml:pos srsName="urn:ogc:crs:epsg:4326">34.55 49.6</gml:pos> </gml:Point> </sa:position> </sa:Station> </om:featureOfInterest> <om:result uom="celsius">10.9</om:result> </om:Measurement>

Sensor Observation ServiceSensor Observation Service

• ... and the whole time serie of observations

<om:result>2005-03-14T21:00:00+03,33506,-5@@2005-03-15T00:00:00+03,33506,-5.2@@2005-03-15T03:00:00+03,33506,-5.5@@2005-03-15T06:00:00+03,33506,-4.6@@2005-03-15T09:00:00+03,33506,-2.2@@2005-03-15T12:00:00+03,33506,1.7@@2005-03-15T15:00:00+03,33506,1.7@@2005-03-15T18:00:00+03,33506,2.4@@2005-03-15T21:00:00+03,33506,-0.7@@2005-03-16T00:00:00+03,33506,-1.4@@2005-03-16T03:00:00+03,33506,-1.1@@2005-03-16T06:00:00+03,33506,-1.1@@2005-03-16T09:00:00+03,33506,-1.3@@2005-03-16T12:00:00+03,33506,0.5@@2005-03-16T15:00:00+03,33506,1.7@@2005-03-16T18:00:00+03,33506,1.5@@</om:result>

Gridification: rationaleGridification: rationale

• Sensor Web services like SOS, SPS and SAS can benefit from integration with Grid platform like Globus Toolkit

• Advantages includes:– Convenient way for implementation of notifications

and event triggering– Sensors discovery through Index Service– High-level access to XML description– Reliable data transfer for large datasets– Enforcement of data and services access policies

Gridification: implementationGridification: implementation

• We have developed a testbed SOS service using the Globus Toolkit platform

• For now, service works as proxy translating and redirecting user request to usual SOS-server

Intranet

SOS Server DatabaseGrid Server/SOS Service

Gridification: implementationGridification: implementation

• We have developed a testbed SOS service using the Globus Toolkit platform

• For now, service works as proxy translating and redirecting user request to usual SOS-server

• Next version should have in-service implementation of SOS-server functionality

Intranet

SOS Server DatabaseGrid Server/SOS Service

Gridification: problemsGridification: problems

• The main problem of implementation of OGC Grid service lies in complexity of XML schema used

• According to OGC SOAP Interoperability Experiment, none of available SOAP binding tools were able to parse OGC schemas completely (year 2003)

• Situation haven’t improved significantly till now

• The main problem of complexity is GML data types

Gridification: problemsGridification: problems

• This problems could be solved by using custom serializers for services XML data or by discarding high-level access to XML data in service implementation

• Custom serialization is complex in implementation and debugging

• Discarding high-level OO access to XML is applicable for services acting like proxies to other XML-generating facilities

• Let’s hope that situation will improve from both sides

Gridification: eventsGridification: events

• GT4 middleware provides Trigger Service that can be used for triggering events basing specific conditions

• We have implemented additional request to SOS Grid service that registers specific observation for periodic update and exports it

• After registering observation user can pass it to the Trigger Service together with specific condition (in form of XPath expression) and an action to be fired

Our plansOur plans

Our future works include:

• Implementation of Mozambique test case in terms of Sensor Web

• To participate in IC "Space and Major Disasters“ with architectural proposals

• To provide stable Grid-based implementation of Sensor Web services

• To collaborate with International Red Cross organization within it’s tasks

Our plans: Red Cross tasksOur plans: Red Cross tasks

ThemeTheme--Based Flood Product Generation for I FRCBased Flood Product Generation for I FRC1

From portal select desired theme(s) and area of interest

Wizard picks appropriate workflow for desired result

Wizard

Mozambique

Disaster Management Information System (DMIS)

Workflows

Estimated rainfall accumulation and flood prediction model

Flood Model

Selected workflow automatically activates needed assets and models

Baseline water level, flood waters and predicted flooding

Thank you!