1

Semantic technologies for the Internet of Things

Payam BarnaghiInstitute for Communication Systems (ICS)University of SurreyGuildford, United Kingdom

International “IoT 360″ Summer SchoolOctober 2015– Rome, Italy

Real world data

2image credits: Smarter Data - I.03_C by Gwen Vanhee

Data in the IoT

− Data is collected by sensory devices and also crowd sensing sources.

− It is time and location dependent.− It can be noisy and the quality can vary. − It is often continuous - streaming data.

− There are other important issues such as:− Device/network management− Actuation and feedback (command and control)− Service and entity descriptions

Device/Data interoperability

4The slide adapted from the IoT talk given by Jan Holler of Ericsson at IoT Week 2015 in Lisbon.

Heterogeneity, multi-modality and volume are among the key issues.

We need interoperable and machine-interpretable solutions…

5

Semantics and Data

− Data with semantic annotations− Provenance, quality of information− Interpretable formats− Links and interconnections− Background knowledge, domain information− Hypotheses, expert knowledge − Adaptable and context-aware solutions

6

Interoperable and semantically described data is the starting point to create an efficient set of actions.

The goal is often to create actionable information.

Wireless Sensor (and Actuator) Networks

Sinknode Gateway

Core networke.g. Internet

Gateway

End-user

Computer services

- The networks typically run Low Power Devices- Consist of one or more sensors, could be different type of sensors (or actuators)

Operating Systems?

Services?

Protocols?Protocols?

In-node Data

Processing

Data Aggregation/

Fusion

Inference/Processing of IoT data

Interoperable/Machine-

interpretablerepresentations

Interoperable/Machine-

interpretableRepresentations?

“Web of Things”

Interoperable/Machine-

interpretablerepresentations

What we are going to study − The sensors (and in general “Things”) are increasingly being

connected with Web infrastructure.− This can be supported by embedded devices that directly

support IP and web-based connection (e.g. 6LowPAN and CoAp) or devices that are connected via gateway components. − Broadening the IoT to the concept of “Web of Things”

− There are already standards such as Sensor Web Enablement (SWE) set developed by the Open Geospatial Consortium (OGC) that are widely being adopted in industry, government and academia.

− While such frameworks provide some interoperability, semantic technologies are increasingly seen as key enabler for integration of IoT data and broader Web information systems.

9

Observation and measurement data- annotation

10

Tags

Data formats

Location

Source: Cosm.com

Observation and measurement data

11

value

Unit of measurement

Time

Longitude

Latitude

How to make the data representations more machine-readable and machine-interpretable;

15, C, 08:15, 51.243057, -0.589444

Observation and measurement data

12

<value>

<unit>

<Time>

<Longitude>

<Latitude>

And this?

<value>15</value><unit>C</unit><time>08:15</time><longitude>51.243057</longitude><latitude>-0.58944</latitude>

15, C, 08:15, 51.243057, -0.589444

XML Representation

<?xml version="1.0“ encoding="ISO-8859-1"?><measurement>

<value type=“Decimal”>15</value><unit>C</unit><time>08:15</time><longitude>51.243057</longitude><latitude>-0.58944</latitude>

</measurement>

13

Well Formed XML Documents

− A "Well Formed" XML document has correct XML syntax.

− XML documents must have a root element− XML elements must have a closing tag− XML tags are case sensitive− XML elements must be properly nested− XML attribute values must be quoted

14Source: W3C Schools, http://www.w3schools.com/

XML Documents– revisiting the example

<?xml version="1.0"?><measurement>

<value>15</value><unit>C</unit><time>08:15</time><longitude>51.243057</longitude><latitude>-0.58944</latitude>

</measurement>

15

<?xml version="1.0"?> <sensor_data>

<reading>15</reading><u>C</u><timestamp>08:15</timestamp><long>51.243057</long><lat>-0.58944</lat>

</sensor_data>

XML

− Meaning of XML-Documents is intuitively clear− due to "semantic" Mark-Up− tags are domain-terms

− But, computers do not have intuition− tag-names do not provide semantics for machines.

− DTDs or XML Schema specify the structure of documents, not the meaning of the document contents

− XML lacks a semantic model− has only a "surface model”, i.e. tree

16Source: Semantic Web, John Davies, BT, 2003.

Semantic Web technologies

− XML provide a metadata format.− It defines the elements but does not provide

any modelling primitive nor describes the meaningful relations between different elements.

− Using semantic technologies to solve these issues.

17

A bit of history

− “The Semantic Web is an extension of the current web in which information is given well-defined meaning, better enabling computers and people to work in co-operation.“ (Tim Berners-Lee et al, 2001)

18Image source: Miller 2004

Semantics & the IoT

−The Semantic Sensor (&Actuator) Web is an extension of the current Web/Internet in which information is given well-defined meaning, better enabling objects, devices and people to work in co-operation and to also enable autonomous interactions between devices and/or objects.

19

Resource Description Framework (RDF)

− A W3C standard− Relationships between documents− Consisting of triples or sentences:

− <subject, property, object>− <“Sensor”, hasType, “Temperature”>− <“Node01”, hasLocation, “Room_BA_01” >

− RDFS extends RDF with standard “ontology vocabulary”:− Class, Property− Type, subClassOf− domain, range

20

RDF for semantic annotation

− RDF provides metadata about resources− Object -> Attribute-> Value triples or− Object -> Property-> Subject− It can be represented in XML − The RDF triples form a graph

21

RDF Graph

22

xsd:decimal

Measurement

hasValuehasTime

xsd:double

xsd:time

xsd:double

xsd:string

hasLongitude hasLatitude

hasUnit

RDF Graph- an instance

23

15

Measurement#0001

hasValuehasTime

-0.589444

08:15

51.243057

C

hasLongitude hasLatitude

hasUnit

RDF/XML (simplified)

<rdf:RDF><rdf:Description rdf:about=“Measurment#0001"><hasValue>15</hasValue><hasUnit>C</hasUnit><hasTime>08:15</hasTime><hasLongitude>51.243057</hasLongitude><hasLatitude>-0.589444</hasLatitude></rdf:Description>

</rdf:RDF>

24

Let’s add a bit more structure (complexity?)

25

xsd:decimal

Location

hasValue

hasTime

xsd:double

xsd:time

xsd:double

xsd:string

hasLongitude

hasLatitude

hasUnitMeasurement

hasLocation

An instance of our model

26

15

Location#0126

hasValue

hasTime

51.243057

08:15

-0.589444

C

hasLongitude

hasLatitude

hasUnit

Measurement#0001

hasLocation

RDF: Basic Ideas

−Resources−Every resource has a URI (Universal Resource

Identifier)−A URI can be a URL (a web address) or a some

other kind of identifier;−An identifier does not necessarily enable

access to a resources−We can think of a resources as an object that

we want to describe it.−Car−Person−Places, etc.

27

RDF: Basic Ideas

− Properties− Properties are special kind of resources;− Properties describe relations between resources.− For example: “hasLocation”, “hasType”, “hasID”, “sratTim

e”, “deviceID”,.− Properties in RDF are also identified by URIs.− This provides a global, unique naming scheme.− For example:

− “hasLocation” can be defined as:− URI: http://www.loanr.it/ontologies/DUL.owl#hasLocation

− SPARQL is a query language for the RDF data. − SPARQL provide capabilities to query RDF graph patterns

along with their conjunctions and disjunctions.

28

Ontologies

−The term ontology is originated from philosophy. In that context it is used as the name of a subfield of philosophy, namely, the study of the nature of existence.

−In the Semantic Web:−An ontology is a formal specification of a

domain; concepts in a domain and relationships between the concepts (and some logical restrictions).

29

Ontologies and Semantic Web

− In general, an ontology describes a set of concepts in a domain.

− An ontology consists of a finite list of terms and the relationships between the terms.

− The terms denote important concepts (classes of objects) of the domain.

− For example, in a university setting, staff members, students, courses, modules, lecture theatres, and schools are some important concepts.

30

Web Ontology Language (OWL)− RDF(S) is useful to describe the concepts and

their relationships, but does not solve all possible requirements

− Complex applications may want more possibilities:− similarity and/or differences of terms (properties or

classes)− construct classes, not just name them− can a program reason about some terms? e.g.:

− each «Sensor» resource «A» has at least one «hasLocation»− each «Sensor» resource «A» has maximum one ID

− This lead to the development of Web Ontology Language or OWL.

31

Ontology engineering

− An ontology: classes and properties (also referred to as schema ontology)

− Knowledge base: a set of individual instances of classes and their relationships

− Steps for developing an ontology:− defining classes in the ontology and arranging the

classes in a taxonomic (subclass–superclass) hierarchy− defining properties and describing allowed values and

restriction for these properties− Adding instances and individuals

Basic rules for designing ontologies

− There is no one correct way to model a domain; there are always possible alternatives. − The best solution almost always depends on the

application that you have in mind and the required scope and details.

− Ontology development is an iterative process. − The ontologies provide a sharable and extensible form to

represent a domain model.− Concepts that you choose in an ontology should

be close to physical or logical objects and relationships in your domain of interest (using meaningful nouns and verbs).

A simple methodology

1. Determine the domain and scope of the model that you want to design your ontology.2. Consider reusing existing concepts/ontologies; this will help to increase the interoperability of your ontology. 3. Enumerate important terms in the ontology; this will determine what are the key concepts that need to be defined in an ontology. 4. Define the classes and the class hierarchy; decide on the classes and the parent/child relationships5. Define the properties of classes; define the properties that relate the classes; 6. Define features of the properties; if you are going to add restriction or other OWL type restrictions/logical expressions. 7. Define/add instances8. Document your ontology

34

Semantic technologies in the IoT

−Applying semantic technologies to IoT can support: −Interoperability−effective data access and integration−resource discovery −reasoning and processing of data−knowledge extraction (for automated decision

making and management)

35

Sensor Markup Language (SensorML)

36

Source: http://www.mitre.org/

The Sensor Model Language Encoding (SensorML) defines models and XML encoding to represent the geometric, dynamic, and observational characteristics of sensors and sensor systems.

Semantic modelling

− Lightweight: experiences show that a lightweight ontology model that well balances expressiveness and inference complexity is more likely to be widely adopted and reused; also large number of IoT resources and huge amount of data need efficient processing

− Compatibility: an ontology needs to be consistent with those well designed, existing ontologies to ensure compatibility wherever possible.

− Modularity: modular approach to facilitate ontology evolution, extension and integration with external ontologies.

37

Existing models- SSN Ontology

− W3C Semantic Sensor Network Incubator Group’s SSN ontology (mainly for sensors and sensor networks, platforms and systems).

http://www.w3.org/2005/Incubator/ssn/

SSN Ontology Modules

39

SSN Ontology

40

Ontology Link: http://www.w3.org/2005/Incubator/ssn/ssnx/ssn

M. Compton et al, "The SSN Ontology of the W3C Semantic Sensor Network Incubator Group", Journal of Web Semantics, 2012.

41

W3C SSN Ontology

41

makes observations of this type

Where it is

What it measures

units

SSN-XG ontologies

SSN-XG annotationsSSN-XG Ontology Scope

What SSN does not model

− Sensor types and models− Networks: communication, topology− Representation of data and units of measurement

− Location, mobility or other dynamic behaviours− Control and actuation− ….

42

43

IoT and Semantics: Challenges and issues

Several ontologies and description models

44

45

We have good models and description frameworks;

The problem is that having good models and developing ontologies is not enough.

46

Semantic descriptions are intermediary solutions, not the end product.

They should be transparent to the end-user and probably to the data producer as well.

47

A WoT/IoT Framework

WSN

WSN

WSN

WSNWSN

Network-enabled Devices

Semantically annotate data

GatewayCoAP

HTTP

CoAP

CoAP

HTTP

6LowPAN

Semantically annotate data

http://mynet1/snodeA23/readTemp?

WSNMQTT

MQTT

Gateway

And several other protocols and solutions…

Publishing Semantic annotations

− We need a model (ontology) – this is often the easy part for a single application.

− Interoperability between the models is a big issue.

− Express-ability vs Complexity is a challenge

− How and where to add the semantics− Where to publish and store them− Semantic descriptions for data, streams, devices

(resources) and entities that are represented by the devices, and description of the services.

48

49

Simplicity can be very useful…

Hyper/CAT

50Source: Toby Jaffey, HyperCat Consortium, http://www.hypercat.io/standard.html

- Servers provide catalogues of resources toclients.

- A catalogue is an array of URIs.- Each resource in the catalogue is annotatedwith metadata (RDF-like triples).

Hyper/CAT model

51Source: Toby Jaffey, HyperCat Consortium, http://www.hypercat.io/standard.html

52

Complex models are (sometimes) good for publishing research papers….

But they are often difficult to implement and use in real world products.

What happens afterwards is more important

− How to index and query the annotated data− How to make the publication suitable for

constrained environments and/or allow them to scale

− How to query them (considering the fact that here we are dealing with live data and often reducing the processing time and latency is crucial)

− Linking to other sources

53

IoT is a dynamic, online and rapidly changing world

54

isPartOf

Annotation for the (Semantic) Web

Annotation for the IoT

Image sources: ABC Australia and 2dolphins.com

Make your model fairly simple and modular

55

SSNO model

56

Creating common vocabularies and taxonomies are also equally important e.g. event taxonomies.

57

We should accept the fact that sometimes we do not need (full) semantic descriptions.

Think of the applications and use-cases before starting to annotate the data.

58

Semantic descriptions can be fairly static on the Web;

In the IoT, the meaning of data and the annotations can change over time/space…

Dynamic annotations for data in the process chain

59S. Kolozali et al, A Knowledge-based Approach for Real-Time IoT Data Stream Annotation and Processing", iThings 2014, 2014.

Dynamic annotations for provenance data

60S. Kolozali et al, A Knowledge-based Approach for Real-Time IoT Data Stream Annotation and Processing", iThings 2014, 2014.

61

Semantic descriptions can also be learned and created automatically.

Overall, we need semantic technologies in the IoT and these play a key role in providing interoperability.

However, we should design and use the semantics carefully and consider the constraints and dynamicity of the IoT environments.

#1: Design for large-scale and provide tools and APIs.

#2: Think of who will use the semantics and how when you design your models.

#3: Provide means to update and change the semantic annotations.

64

#4: Create tools for validation and interoperability testing.

#5: Create taxonomies and vocabularies.

#6: Of course you can always create a better model, but try to re-use existing ones as much as you can.

65

#7: Link your data and descriptions to other existing resources.

#8: Define rules and/or best practices for providing the values for each attribute.

#9: Remember the widely used semantic descriptions on the Web are simple ones like FOAF.

66

#10: Semantics are only one part of the solution and often not the end-product so the focus of the design should be on creating effective methods, tools and APIs to handle and process the semantics.

Query methods, machine learning, reasoning and data analysis techniques and methods should be able to effectively use these semantics.

67

Some examples

IoTLite

69

http://iot.ee.surrey.ac.uk/fiware/ontologies/iot-lite

IoTLite ontology

70

Stream Annotation Ontology (SAO)

71

http://iot.ee.surrey.ac.uk/citypulse/ontologies/sao/sao

W3C/OGC Working Group on Spatial Data on the Web

72

http://www.w3.org/2015/spatial/wiki/Main_Page

- Use cases

- Best Practices

Q&A

− Thank you.

http://personal.ee.surrey.ac.uk/Personal/P.Barnaghi/

@pbarnaghi

p.barnaghi@surrey.ac.uk