sensor metadata management and its application in collaborative environmental research
DESCRIPTION
Sensor Metadata Management and its Application in Collaborative Environmental Research. Nicholas Dawes, Ashwin Kumar, Sebastian Michel , Karl Aberer, Michael Lehning. Ecole Polytechnique Fédérale de Lausanne, Switzerland - PowerPoint PPT PresentationTRANSCRIPT
TRAMM HYDROSYS SENSORSCOPE GSN SENSORMAP BIGLINK RECORD APUNCH EXTREMES MOUNTLAND COGEAR HYDROMON
PERMASENSE
Swiss ExperimentInterdisciplinary Environmental Research
Nicholas Dawes, Ashwin Kumar, Sebastian Michel, Karl Aberer, Michael LehningEcole Polytechnique Fédérale de Lausanne, SwitzerlandSwiss Federal Institute for Snow and Avalanche Research (SLF), Davos, Switzerland
TRAMM HYDROSYS SENSORSCOPE GSN SENSORMAP BIGLINK RECORD APUNCH EXTREMES MOUNTLAND COGEAR HYDROMON
PERMASENSE
Swiss ExperimentInterdisciplinary Environmental Research
Outline
• Introduction to Swiss Experiment (SwissEx)• Problem Statement and Motivation• The SwissEx Wiki• SwissEx Infrastructure (Data)• Semantic Forms• Querying Metadata• Conclusion and Future Work
TRAMM HYDROSYS SENSORSCOPE GSN SENSORMAP BIGLINK RECORD APUNCH EXTREMES MOUNTLAND COGEAR HYDROMON
PERMASENSE
Swiss ExperimentInterdisciplinary Environmental Research
Swiss ExperimentInterdisciplinary Environmental Research
Swiss Experiment (SwissEx Aims):
• Provision of a generic infrastructure of:
• web based technologies
• wireless communications
• low cost high density sensors
to serve the environmental science community, encourage collaboration and provide a portal for public information on environmental research
• Climate change forecasts show the greatest impact in high alpine scenarios. These changes are already being observed and answers to environmental questions need to be found quickly. The Swiss Experiment infrastructure aims to speed up that process
www.swiss-experiment.ch
TRAMM HYDROSYS SENSORSCOPE GSN SENSORMAP BIGLINK RECORD APUNCH EXTREMES MOUNTLAND COGEAR HYDROMON
PERMASENSE
Swiss ExperimentInterdisciplinary Environmental Research
SwissEx Infrastructure
Black boxes indicate widespread research areas with multiple experiments within
• SwissEx infrastucture is built to serve many environmental research projects
• Where experimental areas overlap, projects can work more efficiently by sharing data
• Projects can benefit from external data sources
TRAMM HYDROSYS SENSORSCOPE GSN SENSORMAP BIGLINK RECORD APUNCH EXTREMES MOUNTLAND COGEAR HYDROMON
PERMASENSE
Swiss ExperimentInterdisciplinary Environmental Research
Requirement for Metadata
• Temperatures from Wannengrat, Davos 22/01/07 – 28/01/07
Seem legitimate for 2400m altitude in winter?
TRAMM HYDROSYS SENSORSCOPE GSN SENSORMAP BIGLINK RECORD APUNCH EXTREMES MOUNTLAND COGEAR HYDROMON
PERMASENSE
Swiss ExperimentInterdisciplinary Environmental Research
Requirement for Metadata
• Temperature/Humidity sensor 25/01/07
Sensor opening covered in reimThe data from this sensor during the preceding period is unusable, but indistinguishable from the data after this photo was taken.
…….never trust somebody elses data without the associated metadata
TRAMM HYDROSYS SENSORSCOPE GSN SENSORMAP BIGLINK RECORD APUNCH EXTREMES MOUNTLAND COGEAR HYDROMON
PERMASENSE
Swiss ExperimentInterdisciplinary Environmental Research
The SwissEx WikiInformation, Data Portal and Metadata Database Interface
• Central to the functioning of the project
• Incorporates SensorMap and GSN access:
• If you can access the data, this is the place to have the metadata access
• MediaWiki uses RDF format to store semantics
• RDF enables structure to be added to a mass of web based data
TRAMM HYDROSYS SENSORSCOPE GSN SENSORMAP BIGLINK RECORD APUNCH EXTREMES MOUNTLAND COGEAR HYDROMON
PERMASENSE
Swiss ExperimentInterdisciplinary Environmental Research
SwissEx Infrastructure
TRAMM HYDROSYS SENSORSCOPE GSN SENSORMAP BIGLINK RECORD APUNCH EXTREMES MOUNTLAND COGEAR HYDROMON
PERMASENSE
Swiss ExperimentInterdisciplinary Environmental Research
SwissEx Metadata
TRAMM HYDROSYS SENSORSCOPE GSN SENSORMAP BIGLINK RECORD APUNCH EXTREMES MOUNTLAND COGEAR HYDROMON
PERMASENSE
Swiss ExperimentInterdisciplinary Environmental Research
Observation with station no.
The Metadata Model• The metadata model:
Invalid data registration Observation
Invalid data registration
Observation
Observation with station no.
• Can also be viewed as:
Parameter
TRAMM HYDROSYS SENSORSCOPE GSN SENSORMAP BIGLINK RECORD APUNCH EXTREMES MOUNTLAND COGEAR HYDROMON
PERMASENSE
Swiss ExperimentInterdisciplinary Environmental Research
LabelModel
entities Semantic Annotations
A Station Station name, General information, Geographical coordinate
B ActionStation name, Parameter, Observation/Invalid data, Start data, End date
C Sensor Sensor Serial no, Model no
D Model Model no, Manufacturer, Measuredquantities, Units
E ParameterParameter name, Station name, Sensor serial no, Measured quantity, Units, Sampling frequency, Measurement accuracy
Semantic Annotations
TRAMM HYDROSYS SENSORSCOPE GSN SENSORMAP BIGLINK RECORD APUNCH EXTREMES MOUNTLAND COGEAR HYDROMON
PERMASENSE
Swiss ExperimentInterdisciplinary Environmental Research
Semantic FormsMetadata Database Interface
• We can use semantic wiki language to store data in the background as RDF
• Semantic wiki language is complicated, not what scientists need to deal with when they come back from the field
• Semantic Forms take away this complexity at the expense of flexibility - a well designed system can simplify data entry
TRAMM HYDROSYS SENSORSCOPE GSN SENSORMAP BIGLINK RECORD APUNCH EXTREMES MOUNTLAND COGEAR HYDROMON
PERMASENSE
Swiss ExperimentInterdisciplinary Environmental Research
Use of forms in entering semantics
TRAMM HYDROSYS SENSORSCOPE GSN SENSORMAP BIGLINK RECORD APUNCH EXTREMES MOUNTLAND COGEAR HYDROMON
PERMASENSE
Swiss ExperimentInterdisciplinary Environmental Research
Station (Model Entity A)
General info on the station/vs
Addition of observations to the data or station/vs and notation of invalid data periods
Parameters
All possible sensor models
List of assigned sensors
Observations pertianing to the overall station
TRAMM HYDROSYS SENSORSCOPE GSN SENSORMAP BIGLINK RECORD APUNCH EXTREMES MOUNTLAND COGEAR HYDROMON
PERMASENSE
Swiss ExperimentInterdisciplinary Environmental Research
Action (Model Entity B)
Observation or invalid data between 2 dates
TRAMM HYDROSYS SENSORSCOPE GSN SENSORMAP BIGLINK RECORD APUNCH EXTREMES MOUNTLAND COGEAR HYDROMON
PERMASENSE
Swiss ExperimentInterdisciplinary Environmental Research
Associated parameter names
Sensor (Model Entity C)
Association between parameter name and model
TRAMM HYDROSYS SENSORSCOPE GSN SENSORMAP BIGLINK RECORD APUNCH EXTREMES MOUNTLAND COGEAR HYDROMON
PERMASENSE
Swiss ExperimentInterdisciplinary Environmental Research
Model information
Sensor Model (Model Entity D)
Association sensor instances (serial numbers)
TRAMM HYDROSYS SENSORSCOPE GSN SENSORMAP BIGLINK RECORD APUNCH EXTREMES MOUNTLAND COGEAR HYDROMON
PERMASENSE
Swiss ExperimentInterdisciplinary Environmental Research
Parameter (Virtual Sensor) (Model Entity E)
Parameter information
Registered invalid data periods and observations
Dependencies
TRAMM HYDROSYS SENSORSCOPE GSN SENSORMAP BIGLINK RECORD APUNCH EXTREMES MOUNTLAND COGEAR HYDROMON
PERMASENSE
Swiss ExperimentInterdisciplinary Environmental Research
How do we make the tables?Semantic Queries
{{#ask:[[station name::{{{stationname}}}]] [[Sensor serialno::+]]|?Sensor serialno|?Action|?Action at|format=table}}
• Standard Wiki query
• SPARQL queryPREFIX a:<http://www.swiss-experiment.ch/index.php/Special:URIResolver/>SELECT DISTINCT ?smodelWHERE {?page a:Property:Sensor_model?smodel}
Query languages for RDF databases:
TRAMM HYDROSYS SENSORSCOPE GSN SENSORMAP BIGLINK RECORD APUNCH EXTREMES MOUNTLAND COGEAR HYDROMON
PERMASENSE
Swiss ExperimentInterdisciplinary Environmental Research
Semantic Queries
• BUT…. SPARQL can query multiple sources
PREFIX a:<http://www.experiment1.ch/index.php/Special:URIResolver/>PREFIX b:<http://www.experiment2.ch/index.php/Special:URIResolver/>SELECT ?smodelFROM NAMED <wiki1.rdf>FROM NAMED <wiki2.rdf>WHERE {GRAPH <wiki1.rdf> {?x a:Property:Sensor_model ?smodel .} .GRAPH <wiki2.rdf> {?y b:Property:Sensor_model ?smodel .} .}
• So metadata from A,B,C,D and E can be queried simultaneously
• SPARQL also gives us the possibility to query across metadata from different experiments……so we have the possibility of querying the data based on its metadata
TRAMM HYDROSYS SENSORSCOPE GSN SENSORMAP BIGLINK RECORD APUNCH EXTREMES MOUNTLAND COGEAR HYDROMON
PERMASENSE
Swiss ExperimentInterdisciplinary Environmental Research
Metadata Export using SPARQL
SPARQL is already used to query the wiki and import metadata to SensorMap
TRAMM HYDROSYS SENSORSCOPE GSN SENSORMAP BIGLINK RECORD APUNCH EXTREMES MOUNTLAND COGEAR HYDROMON
PERMASENSE
Swiss ExperimentInterdisciplinary Environmental Research
Can I use the wiki as a front end for any database?
• Importing to the Wiki from a relational database:• Using R2D2 (http://aksw.org) SPARQL queries can be translated to SQL for
querying a relational database. The result is translated back to RDF format using the same tool.
• The RDF files can be imported to a wiki page.
•Why is this interesting?
• If there are external data sources with metadata, we can query it and give the results in the same place as the rest of the data
• Exporting to a relational database from the wiki:• A script should be written to perform the relevant SPARQL query and wrap
the result into a SQL query.
TRAMM HYDROSYS SENSORSCOPE GSN SENSORMAP BIGLINK RECORD APUNCH EXTREMES MOUNTLAND COGEAR HYDROMON
PERMASENSE
Swiss ExperimentInterdisciplinary Environmental Research
Conclusion
• Framework for Metadata capturing• Integrated in a Wiki• Intuitive to use• Provide query interface to applications• Support for import/export to relational DBs
TRAMM HYDROSYS SENSORSCOPE GSN SENSORMAP BIGLINK RECORD APUNCH EXTREMES MOUNTLAND COGEAR HYDROMON
PERMASENSE
Swiss ExperimentInterdisciplinary Environmental Research
Questions