information engineering in the age of the internet of things

Information Engineering in the Age of the Internet of Things

1

Payam BarnaghiInstitute for Communication Systems (ICS)/5G Innovation Centre University of SurreyGuildford, United Kingdom

Digital Catapult, December 2015

“A hundred years hence people will be so avid of every moment of life, life will be so full of busy delight, that time-saving inventions will be at a huge premium…”

“…It is not because we shall be hurried in nerve-shattering anxiety, but because we shall value at its true worth the refining and restful influence of leisure, that we shall be impatient of the minor tasks of every day….”

The March 26, 1906, New Zealand Star :

Source: http://paleofuture.com

3IBM Mainframe 360, source Wikipedia

Apollo 11 Command Module (1965) had 64 kilobytes of memory operated at 0.043MHz.

An iPhone 5s has a CPU running at speeds of up to 1.3GHzand has 512MB to 1GB of memoryCray-1 (1975) produced 80 million Floating point operations per second (FLOPS)10 years later, Cray-2 produced 1.9G FLOPS

An iPhone 5s produces 76.8 GFLOPS – nearly a thousand times more

Cray-2 used 200-kilowatt power

Source: Nick T., PhoneArena.com, 2014image source: http://blog.opower.com/

Computing Power

5

−Smaller size−More Powerful−More memory and more storage

−"Moore's law" over the history of computing, the number of transistors in a dense integrated circuit has doubled approximately every two years.

Smaller in size but larger in scale

6

The old Internet timeline

7Source: Internet Society

Connectivity and information exchange was (and is ) the main motivation behind the Internet; but Content and Services are now the key elements;

and all started growing rapidly by the introduction of the World Wide Web (and linked information and search and discovery services).

8

Early days of the web

9

Search on the Internet/Web in the early days

10

11

AnyPlace AnyTime

AnyThing

Data Volume

Security, Reliability, Trust and Privacy

Societal Impacts, Economic Values and Viability

Services and Applications

Networking andCommunication

12

Sensor devices are becoming widely available

- Programmable devices- Off-the-shelf gadgets/tools

Internet of Things: The story so far

RFID based solutions

Wireless Sensor andActuator networks

, solutions for communication

technologies, energy efficiency, routing, …

Smart Devices/Web-enabled

Apps/Services, initial products,

vertical applications, early concepts and demos, …

Motion sensor

Motion sensor

ECG sensor

Physical-Cyber-Social Systems, Linked-data,

semantics, M2M, More products, more

heterogeneity, solutions for control and

monitoring, …

Future: Cloud, Big (IoT) Data Analytics, Interoperability,

Enhanced Cellular/Wireless Com. for IoT, Real-world operational

use-cases and Industry and B2B services/applications,

more Standards…

Real world data

14

Data in the IoT

− Data is collected by sensory devices and also crowd sensing resources.

− It is time and location dependent.− It can be noisy and the quality can vary. − It is often continuous - streaming data.

− There are several important issues such as:− Device/network management− Actuation and feedback (command and control)− Service and entity descriptions.

IoT data- challenges

− Multi-modal, distributed and heterogeneous− Noisy and incomplete− Time and location dependent − Dynamic and varies in quality − Crowdsourced data can be unreliable − Requires (near-) real-time analysis− Privacy and security are important issues− Data can be biased- we need to know our data!

16P. Barnaghi, A. Sheth, C. Henson, "From data to actionable knowledge: Big Data Challenges in the Web of Things," IEEE Intelligent Systems, vol.28 , issue.6, Dec 2013.

Making IoT data widely available and (re-)usable− Machine-readable and/or human interpretable

meta-data− Open IoT data portals (static and streaming)− Open APIs and discoverable interfaces − Discoverable data and patterns (and resources)− (easily) Sharable and connectable data− Opportunistic and ad-hoc discovery, matching

and mash-up solutions − Quality and fit-for-purpose data − Trust, privacy and security aware data collection,

sharing and access solutions

17

Device/Data interoperability

18The slide adapted from the IoT talk given by Jan Holler of Ericsson at IoT Week 2015 in Lisbon.

Heterogeneity, multi-modality and volume are among the key issues.

We need interoperable and machine-interpretable solutions…

19

A bit of history

− “The Semantic Web is an extension of the current web in which information is given well-defined meaning, better enabling computers and people to work in co-operation.“ (Tim Berners-Lee et al, 2001)

20Image source: Miller 2004

Semantics & the IoT

−The Semantic Sensor (&Actuator) Web is an extension of the current Web/Internet in which information is given well-defined meanings, better enabling objects, devices and people to work in co-operation and to also enable autonomous interactions between devices and/or objects.

21

Semantic Descriptions in Semantic (Web) World

22

23

The world of IoT and Semantics

Example: IoT-A information model

24

25

Some good existing models: SSN Ontology

Ontology Link: http://www.w3.org/2005/Incubator/ssn/ssnx/ssnM. Compton, P. Barnaghi, L. Bermudez, et al, "The SSN Ontology of the W3C Semantic Sensor Network Incubator Group", Journal of Web Semantics, 2012.

26

But why do we still not have fully integrated semantic solutions in the IoT?

Semantic Web these days…

27

Several existing models

28

A simple Google search for “IoT data model” returned around 3,720,000 results. 

29

We have good models and description frameworks;

The problem is that having good models and developing ontologies are not enough.

30

Semantic descriptions are intermediary solutions, not the end product.

They should be transparent to the end-user and probably to the data producer as well.

A WoT/IoT Framework

WSN

WSN

WSN

WSNWSN

Network-enabled Devices

Semantically annotate data

31

GatewayCoAP

HTTP

CoAP

CoAP

HTTP

6LowPAN

Semantically annotate data

http://mynet1/snodeA23/readTemp?

WSNMQTT

MQTT

Gateway

And several other protocols and solutions…

Publishing Semantic annotations

− We need a model (ontology) – this is often the easy part for a single application.

− Interoperability between the models is a big issue.

− Express-ability vs Complexity is a challenge.

− How and where to add the semantics− Where to publish and store them− Semantic descriptions for data, streams, devices

(resources) and entities that are represented by the devices, and description of the services.

32

33

Simplicity can be very useful…

Hyper/CAT

34Source: Toby Jaffey, HyperCat Consortium, http://www.hypercat.io/standard.html

- Servers provide catalogues of resources toclients.

- A catalogue is an array of URIs.- Each resource in the catalogue is annotatedwith metadata (RDF-like triples).

Hyper/CAT model

35Source: Toby Jaffey, HyperCat Consortium, http://www.hypercat.io/standard.html

36

Perhaps complex models are (sometimes) good for publishing research papers….

But they are often difficult to implement and use in real world products.

What happens afterwards is more important

− How to use the data (documentation, tools, interfaces)

− How to index and query the annotated data− How to make the publication suitable for

constrained environments and/or allow them to scale

− How to query them (considering the fact that here we are dealing with live data and often reducing the processing time and latency is crucial)

− Linking to other sources

37

The IoT is a dynamic, online and rapidly changing world

38

isPartOf

Annotation for the (Semantic) Web

Annotation for the IoT

Image sources: ABC Australia and 2dolphins.com

Tools and APIs- e.g. Sense2Web

39P. Barnaghi, M. Presser, K. Moessner, "Publishing Linked Sensor Data", in Proc. of the 3rd Int. Workshop on Semantic Sensor Networks (SSN), ISWC2010, 2010.

Tools and API – e.g. FIWARE IoT Discovery Generic Enabler

40http://catalogue.fiware.org/enablers/iot-discovery/documentation

Model documentation- e.g. SAO ontology

41http://iot.ee.surrey.ac.uk/citypulse/ontologies/sao/sao

Providing flexible APIs- e.g. SAOPY

42http://iot.ee.surrey.ac.uk/citypulse/ontologies/sao/saopy.html

43

Creating common vocabularies and taxonomies are also equally important

e.g. event and unit taxonomies, common formats for representing location, vocabularies to describe proximity and relations between Things and their data.

44

We should accept the fact that sometimes we do not need (full) semantic descriptions.

Think of the applications and use-cases before starting to annotate the data.

An example: a discovery method in the IoT

time

location

type

Query formulating

[#location | #type | time]

Discovery ID

Discovery/DHT Server

Data repository(archived data)

#location#type

#location#type

#location#type

Data hypercube

Gateway

Core network

Network ConnectionLogical Connection

Data

An example: a discovery method in the IoT

46

S. A. Hoseinitabatabaei, P. Barnaghi, C. Wang, R. Tafazolli, L. Dong, "Method and Apparatus for Scalable Data Discovery in IoT Systems", US Patents, 2015.

101 Smart City Use-case Scenarios

47http://www.ict-citypulse.eu/page/content/smart-city-use-cases-and-requirements

48

Semantic descriptions can be fairly static on the Web;

In the IoT, the meaning of data and the annotations can change over time/space…

Dynamic Semantics

<iot:measurement><iot:type> temp</iot:type><iot:unit>Celsius</iot:unit><time>12:30:23UTC</time><iot:accuracy>80%</iot:accuracy><loc:long>51.2365<loc:lat><loc:lat>0.5703</loc:lat></iot:measurment>

49

But this could be a function of time and

location;

What would be the accuracy 5 seconds

after the measurement?

Dynamic annotations for data in the process chain

50S. Kolozali et al, A Knowledge-based Approach for Real-Time IoT Data Stream Annotation and Processing", iThings 2014, 2014.

Dynamic annotations for provenance data

51S. Kolozali et al, A Knowledge-based Approach for Real-Time IoT Data Stream Annotation and Processing", iThings 2014, 2014.

52

Metadata (Semantic) or higher-level event descriptions can also be learned and created automatically.

Extraction of events and semantics from social media

53

City Infrastructure

Tweets from a city

https://osf.io/b4q2t/

Pramod Anantharam, Payam Barnaghi, Krishnaprasad Thirunarayan, Amit P Sheth, "Extracting City Traffic Events from Social Streams", ACM Transactions on Intelligent Systems and Technology, 2015

Ontology learning from real world data

54Frieder Ganz, Payam Barnaghi, Francois Carrez, "Automated Semantic Knowledge Acquisition from Sensor Data", IEEE Systems Journal, 2014.

Overall we need semantics and metadata in the IoT and these play a key role in providing interoperability.

However, we should design and use the data publication, access, sharing and their associated metadata models carefully and consider the constraints and dynamicity of the IoT environments.

Data Lifecycle

57Source: The IET Technical Report, Digital Technology Adoption in the Smart Built Environment: Challenges and opportunities of data driven systems for building, community and city-scale applications, http://www.theiet.org/sectors/built-environment/resources/digital-technology.cfm

Information Engineering for the IoT:Design Commandments

58

#1: Design for large-scale and provide tools and APIs.

#2: Think of who will use the data and how when you design your models.

#3: Provide means to update and change the semantic annotations.

59

Smart data collection

− Smart data collection

− Sooner or later we need to think whether we need to collect that data, how often we need to collect it and what volume.

− Intelligent data Processing (selective attention and information-extraction)

60

(image source: KRISTEN NICOLE, siliconangle.com)

Image sources : The dailymail, http://helenography.net/, http://edwud.com/

Designing for real world problems

#4: Create tools, open APIs and datasets for validation, evaluation, and interoperability testing.

#5: Create common vocabularies and provide documentation.

#6: Of course you can always create a better model, but try to re-use existing ones as much as you can.

63

Reference Datasets

64http://iot.ee.surrey.ac.uk:8080/datasets.html

Importance of Complementary Data

65

Open Data/Open APIs

−Open data is often misinterpreted as free data and publically available data.

−You can have open data, but with the right access controls, with trust and privacy.

66

#7: Link your data and descriptions to other existing resources.

#8: Define rules and/or best practices for providing the values for each attribute.

#9: Remember the widely used (semantic) models on the Web are simple ones like FOAF.

67

Best Practices: an example (early draft)

68

Spatial Data on the Web- Best Practices (early draft)

69

#10: Design for different audience (data consumers, developers, providers) and think about real impact and sustainability.

#11: Specify (and encourage others to do the same) data governance and privacy procedures, explain the ownership and re-use rules, and give control to the owners of data

70

71Source LAT Times, http://documents.latimes.com/la-2013/

Future cities: A view from 1998

72

Source: http://robertluisrabello.com/denial/traffic-in-la/#gallery[default]/0/

Source: wikipedia

Back to the Future: 2013

73

Users in control or losing control?

74

Image source: Julian Walker, Flicker

#12: Semantics and information engineering are only one part of the solution and often not the end-product so the focus of the design should be on creating effective methods, tools and APIs to handle and process the semantics.

75

Technical challenges and research directions

76

Technical (and non-technical) Challenges

− Creating common models to represent, publish, and (re-)use and share IoT data.− creating an IoT data market and data-driven innovation− Developing standards

− Providing best practices, demonstrators and open data portals for streaming and dynamic IoT data.

− Provide governance, dependability, reliability, trust and security models.

77

Research challenges

−Transforming raw data to actionable-information.−Machine learning and data analytics for large-

scale, multi-modal and dynamic (streaming data).

− Making data more accessible and discoverable.

−Energy and computationally efficient data collection, aggregation and abstraction (for both edge and Cloud processing).

78

Research challenges (continued)

−Integration and combination of Physical-Cyber-Social data.

−Use of data for automated interactions and autonomous services in different domains.

−Resource-aware and context-aware security, privacy and trust solutions.

79

Desire for innovation

80Driverless Car of the Future (1957)

Image: Courtesy of http://paleofuture.com

In conclusion

− IoT information engineering is different from common models of web data and/or other types of big data.

− Data collection in the IoT comes at the cost of bandwidth, network, energy and other resources.

− Data collection, delivery and processing is also depended on multiple layers of the network.

− We need more resource-aware data analytics methods and cross-layer optimisations (Deep IoT).

− The solutions should work across different systems and multiple platforms (Ecosystem of systems).

− Data sources are more than physical (sensory) observation.− The IoT requires integration and processing of physical-cyber-

social data.− The extracted insights and information should be converted

to a feedback and/or actionable information. 81

Let’s hope

−The Internet of the Future will be −For everyone, everywhere, available at

anytime,−People will have control on their data−Data will be used for helping people−Smart applications will contribute to a

better life and to a better use of of our resources in the world!

82

Other challenges and topics that I didn't talk about

Resilience and reliability

Noise and incomplete data

Cloud and distributed computing

Networks, test-beds and mobility

Mobile computingServices

83

IET sector briefing report

84

Available at: http://www.theiet.org/sectors/built-environment/resources/digital-technology.cfm

85

Useful information:

http://www.raeng.org.uk/publications/reports/connecting-data-driving-productivity

Q&A

− Thank you.

http://personal.ee.surrey.ac.uk/Personal/P.Barnaghi/

@pbarnaghi

p.barnaghi@surrey.ac.uk