Semantics and Sensors

Wednesday Nights in the Tetherless World (TWed)November 28th, 2012

Joshua Shinavier

• intro to sensors

• semantic sensor networks

• components of an application

• a demo

Overview

2

What is a sensor?

3

• any device that measures a physical quantity

• e.g. temperature, pressure, electrical potential

• more abstract quantities are sometimes measured by “sensors”

• e.g. the number of bicycles available at a bicycle sharing station

• however, for our purposes...

Sensors look like this

4

How do you talk to a sensor?

5

• basic sensors produce simple analog or digital signals,

• or use simple protocols such as I²C

• they don’t understand HTTP

• nor even TCP/IP

• they need to be connected with a more general-purpose device

• e.g. a microcontroller

OK, how do you talk to a microcontroller?

6

• concerns of low-power, low-resource devices

• constrained CPU and memory

• battery life, if battery-powered

• range, if wireless

• there are many solutions

• e.g. Constrained Application Protocol (CoaP)

• interoperates with HTTP

• here, we will use Open Sound Control (OSC)

• simple messaging format for multimedia devices

Some interrelated concepts

7

• ubiquitous computing

• ambient intelligence

• perceptual intelligence

• Humanistic Intelligence

• wearable computers

• Internet of Things

• Web of Things

Some quantities we can measure with sensors

8

Simple physics

9

• force / pressure

• strain / deformation

• flex / bend

• stretch

• linear displacement

• angular displacement

• electrical potential, current

• magnetic field strength

• flow of liquids and gases

• etc. etc. etc.

The environment

10

• air temperature

• barometric pressure

• humidity

• dust and smoke level

• gas concentrations

• ambient light level(s)

• ambient noise

• ambient vibration

Where you are, what’s around you

11

• GPS position

• compass heading

• attitude, acceleration, angular momentum

• altitude

• identified objects

• (RFID, visual object recognition, etc.)

• distance and motion

• (doppler, rangefinders)

Sensing yourself

12

• respiratory rate

• heart rate / pulse

• blood pressure

• skin temperature

• skin resistance and capacitance

• perspiration

• muscle activity (EMG)

• eye movement (EOG)

• brain activity (EEG)

• body position and movement

• facial expression...

• voice...

Common themes in semantic sensor networks

13

Sources of sensor data and metadata

14

• gathering sensor data from source x, e.g.

• sensor data from wireless networks

• sensor data from mobile phones

• citizen sensing

• gathering sensor metadata

• through human input

• derived from raw measurements

Modeling and publication of sensor data

15

• ontological representation of sensor data

• sensor metadata as Linked Data

• integration of streaming sensor data with static metadata, web service APIs

• annotation of sensor data with quality measures

• publishing historical time series data

Applications of sensor data

16

• complex event processing over sensor data streams

• composition of sensor web APIs

• ubiquitous computing applications

• search over sensor data

• stream reasoning over sensor data

Components of a semantic sensor network application

17

• time-series applications

• sensors make observations at regular intervals

• e.g. air temperature sampled every minute, smoke level sampled every ten seconds

• event-driven applications

• sensors make observations in response to events

• e.g. movement detected via infrared or RFID, flipping of a switch, events of voice or gesture

Two kinds of sensor-based applications

18

Sensor network ontologies

19

• Observations and Measurements model in OWL

• OGC Sensor Web Enablement Initiative

• Semantic Sensor Network ontology

• W3C Semantic Sensor Network Incubator Group

• OntoSensor ontology

• builds on SUMO and ISO 19115 (geo metadata)

• there are many others

Modeling an observation

20

Observation

Sensor Place

locationFeature

featureOfInterest

ResultData

result

Interval

samplingTime

Instant Instant

hasBeginning hasEnd

PropertyobservedProperty

UnitOfMeasurement

uom

(numeric)

value

Some options for RDF streams

21

• SPARQL 1.1 Update

• SparqlPuSH

• SPARQL + pubsubhubbub

• RDFAgents

• provenance chains via Named Graphs

• roll your own

Continuous SPARQL

22

• typically, SPARQL is used to query for historical data

• continuous SPARQL queries for future data

• first submit the query, then wait for matching data

• as matching data arrives, answers are produced

• continuous SPARQL languages and frameworks:

• C-SPARQL, SPARQLstream, EP-SPARQL, StreamingSPARQL, Groppe et al.’s, INSTANS, ...

Demo: the Monitron

23

OmnisensoryMonitron

RGB LED

speaker

humidity and temperature

sensor

piezo vibration sensor

electret microphone color light

sensor

optical dust detector

photoresistor

barometric pressure sensor

hostcomputer

PIR motion sensor

Monitron architecture

24

• Arduino Nano + multiple sensors

• sensors are sampled continuously in a cycle

• samples are aggregated, observations made at regular intervals

• observations and error messages are sent to a host computer as OSC messages

• host computer may send commands to the Monitron

• a process on the host computer rdfizes received observations,

• adding them to an RDF stream using RDFAgents

• each observation is contained in an RDF graph

Step #1: build the device

25

• order parts

• snip wires

• solder here

• heat-shrink there

• done!

Step #2: write the firmware

26

• the output of the device looks like:

• for each observation:

• sensor id, time interval, #samples, min val, max val, mean, variance

/om/system/time 1753A23/om/sensor/7bb206l0/vibr 1753A23 1754CEE 10000 0.003 0.022 0.021 0.000000/om/sensor/md9745apzf/sound 1753A23 1754CEE 10000 0.254 0.792 0.507 0.001740/om/sensor/photo/light 1753A23 1754CEE 10000 0.561 0.567 0.565 0.000020/om/sensor/rht03/humid 1754CFC 1754D03 1 0.490 0.490 0.490 0.000000/om/sensor/rht03/temp 1754CFC 1754D03 1 1.000 22.600 22.600 0.000000/om/sensor/bmp085/press 1754D0B 1754D17 1 1.000 101209.000 101209.000 0.000000/om/sensor/bmp085/temp 1754D0B 1754D17 1 1.000 23.500 23.500 0.000000/om/sensor/gp2y1010au0f/dust 1754D1F 1754D29 1 0.160 0.160 0.160 0.000000/om/sensor/se10/motion 1753A23 1754D2E 1 0

Step #3: extend the sensor ontology

27

• you need classes for the specific sensors of your device

• each one produces a different kind of observation

• with different units of measurement

Step #4: describe your sensors

28

• each sensor is an instance of a Sensor subclass

• a device contains more than one sensor if it measures more than one quantity (e.g. temperature + pressure)

:avago-adjd-s311-cr999_1 a m:ColorLightLevelSensor ; rdfs:label "color light level sensor of Monitron #1" ; rdfs:comment "an Avago Technologies ADJD-S311-CR999 miniature surface mount RGB digital color sensor which has been calibrated to output red, green, and blue color values from 0 to 1" .

:bosch-bmp085_1_thermometer a m:Thermometer ; rdfs:label "thermometer #1 of Monitron #1" ; rdfs:comment "a Bosch BMP085 digital (barometric) pressure sensor which has been calibrated to output temperature values in degrees Celsius" .

:bosch-bmp085_1_barometer a m:Barometer ; rdfs:label "barometer of Monitron #1" ; rdfs:comment "a Bosch BMP085 digital (barometric) pressure sensor which has been calibrated to output pressure values in pascals" .

Step #5: add programming logic

29

• what should the host computer do when it receives a “coffeepot half full” observation?

• each type of observation must be translated in a slightly different way

• a class hierarchy in Java works nicely

Step #6: start streaming

30

@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .@prefix monitron: <http://fortytwo.net/2012/08/monitron#> .@prefix om: <http://schemas.opengis.net/om/2.0/> .@prefix time: <http://www.w3.org/2006/time#> .@prefix xsd: <http://www.w3.org/2001/XMLSchema#> .@prefix : <http://fortytwo.net/2012/08/universe#> .

:event-UXhu5B5 om:Procedure :bosch-bmp085_1_thermometer ; om:featureOfInterest :room_1 ; om:observationLocation :room_1 ; om:observedProperty monitron:airTemperature ; om:result [ om:uom monitron:degreesCelsius ; om:value 23.3 ; a om:ResultData] ; om:samplingTime [ a time:Interval ; time:hasBeginning [ a time:Instant ; time:inXSDDateTime "2012-11-28T07:50:06.132-05:00"^^xsd:dateTime] ; time:hasEnd [ a time:Instant ; time:inXSDDateTime "2012-11-28T07:50:06.145-05:00"^^xsd:dateTime]] ; a monitron:AirTemperatureObservation .

Step #7: submit a continuous query

31

PREFIX : <http://fortytwo.net/2012/08/universe#>PREFIX m: <http://fortytwo.net/2012/08/monitron#>PREFIX om: <http://schemas.opengis.net/om/2.0/>PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>

SELECT ?result ?value WHERE {

?obs rdf:type m:VibrationLevelObservation .

?obs om:result ?result . ?result om:value ?value . ?obs om:observationLocation ?loc . ?loc m:containedIn :winslowBuilding .}

Step #8: receive results

32

sensors

Monitron

I²C,etc.

host computer

OSC

subscribers

RDFAgents

• Tetherless World Constellation:

• http://tw.rpi.edu

• Code:

• https://github.com/joshsh/laboratory

• Contact:

• josh@fortytwo.net, @joshsh

Thanks!

33