internet of things and data analytics for smart cities and ehealth

Internet of Things and Data Analytics for Smart Cities and eHealth

1

Payam BarnaghiInstitute for Communication Systems (ICS)/5G Innovation Centre University of SurreyGuildford, United Kingdom

University of York, November 2016

“A hundred years hence people will be so avid of every moment of life, life will be so full of busy delight, that time-saving inventions will be at a huge premium…”

“…It is not because we shall be hurried in nerve-shattering anxiety, but because we shall value at its true worth the refining and restful influence of leisure, that we shall be impatient of the minor tasks of every day….”

The March 26, 1906, New Zealand Star :

Source: http://paleofuture.com

3IBM Mainframe 360, source Wikipedia

Apollo 11 Command Module (1965) had 64 kilobytes of memory operated at 0.043MHz.

An iPhone 5s has a CPU running at speeds of up to 1.3GHzand has 512MB to 1GB of memoryCray-1 (1975) produced 80 million Floating point operations per second (FLOPS)10 years later, Cray-2 produced 1.9G FLOPS

An iPhone 5s produces 76.8 GFLOPS – nearly a thousand times more

Cray-2 used 200-kilowatt power

Source: Nick T., PhoneArena.com, 2014image source: http://blog.opower.com/

Computing Power

5

−Smaller size−More Powerful−More memory and more storage

−"Moore's law" over the history of computing, the number of transistors in a dense integrated circuit has doubled approximately every two years.

Smaller in size but larger in scale

6

The old Internet timeline

7Source: Internet Society

Connectivity and information exchange was (and is) the main motivation behind the Internet; but Content and Services are now the key elements;

and all started growing rapidly by the introduction of the World Wide Web (and linked information and search and discovery services).

8

Early days of the web

9

Search on the Internet/Web in the early days

10

Source: Intel, 2012

Source: http://www.techspartan.co.uk

13P. Barnaghi et al., "Digital Technology Adoption in the Smart Built Environment", IET Sector Technical Briefing, The Institution of Engineering and Technology (IET), I. Borthwick (editor), March 2015.

14

Sensor devices are becoming widely available

- Programmable devices- Off-the-shelf gadgets/tools

Internet of Things: The story so far

RFID based solutions

Wireless Sensor andActuator networks

, solutions for communication

technologies, energy efficiency, routing, …

Smart Devices/Web-enabled

Apps/Services, initial products,

vertical applications, early concepts and demos, …

Motion sensor

Motion sensor

ECG sensor

Physical-Cyber-Social Systems, Linked-data,

semantics, M2M, More products, more

heterogeneity, solutions for control and

monitoring, …

Future: Cloud, Big (IoT) Data Analytics, Interoperability,

Enhanced Cellular/Wireless Com. for IoT, Real-world operational

use-cases and Industry and B2B services/applications,

more Standards…

Data- Challenges

− Multi-modal and heterogeneous− Noisy and incomplete− Time and location dependent − Dynamic and varies in quality − Crowed sourced data can be unreliable − Requires (near-) real-time analysis− Privacy and security are important issues− Data can be biased- we need to know our data!

16

Speed of light?

17Image source: The Brain with David Eagleman, BBC

Device/Data interoperability

18The slide adapted from the IoT talk given by Jan Holler of Ericsson at IoT Week 2015 in Lisbon.

WoT/IoT

WSN

WSN

WSN

WSNWSN

Network-enabled Devices

Semantically annotate data

19

GatewayCoAP

HTTP

CoAP

CoAP

HTTP

6LowPAN

Semantically annotate data

http://mynet1/snodeA23/readTemp?

WSNMQTT

MQTT

Gateway

Gateway

20

Some good existing models: W3C SSN Ontology

Ontology Link: http://www.w3.org/2005/Incubator/ssn/ssnx/ssnM. Compton, P. Barnaghi, L. Bermudez, et al, "The SSN Ontology of the W3C Semantic Sensor Network Incubator Group", Journal of Web Semantics, 2012.

IoT-lite ontology

21

Spatial Data on the Web WG

https://www.w3.org/2015/spatial/charter

23

Hyper/CAT

24Source: Toby Jaffey, HyperCat Consortium, http://www.hypercat.io/standard.html

- Servers provide catalogues of resources toclients.

- A catalogue is an array of URIs.- Each resource in the catalogue is annotatedwith metadata (RDF-like triples).

FIWARE IoT Discovery Generic Enabler

25http://catalogue.fiware.org/enablers/iot-discovery/documentation

New Generation of Search Engines

26P. Barnaghi, A. Sheth, “On Searching the Internet of Things: Requirements and Challenges”, to appear, IEEE Intelligent Systems, 2016.

On Searching the Internet of Things

27P. Barnaghi, A. Sheth, “On Searching the Internet of Things: Requirements and Challenges”, to appear, IEEE Intelligent Systems, 2016.

A discovery engine for the IoT

28A. HosseiniTabatabaie, P. Barnaghi, C. Wang, L. Dong, R. Tafazolli, "Method and Apparatus for Scalable Data Discovery in IoT Systems”, US Patents, CNV12174, May 2014.

Let’s assume that attribute x has an alphabet Ax ={ax1,…,axs}. Query for a data item (q) that is described with attributes x, y and z, is then represented as q={x=axk & y=ayl & z=azm}

The average ratio of matching processes that are required to resolve this query at n:

A GMM model for indexing

29

Average Success ratesFirst attempt: 92.3% (min) At first DS: 92.5 % (min) At first DSL2 : 98.5 % (min)

Number of attempts

Perc

enta

ge o

f the

tota

l que

ries

0 10 20 30 40 50 6010-4

10-3

10-2

10-1

100

DSL2 capacity 1DSL2 capacity 2DSL2 capacity 3DSL2 capacity 4

A. HosseiniTabatabaie, P. Barnaghi, C. Wang, L. Dong, R. Tafazolli, "Method and Apparatus for Scalable Data Discovery in IoT Systems”, US Patents, CNV12174, May 2014.

Indexing spatial data with multiple attributes

30Fathy Y., Barnaghi P., Tafazolli R., “Distributed in-network indexing mechanism for the Internet of Things (IoT)”, submitted to IEEE ICC 2017.

Fathy Y., Barnaghi P., Enshaeifar S., Tafazolli R., "A Distributed In-network Indexing Mechanism for the Internet of Things", IEEE World Forum on IoT, 2016.

Adaptive Clustering

31D. Puschmann, P. Barnaghi, R.Tafazolli, "Adaptive Clustering for Dynamic IoT Data Stream", IEEE Internet of Things Journal, 2016.

Adaptive clustering

32D. Puschmann, P. Barnaghi, R. Tafazolli, "Marginal Distribution Clustering of Multi-variate Streaming IoT Data", IEEE World Forum on IoT, Dec. 2016.

Dynamic clusters

33D. Puschmann, P. Barnaghi, R. Tafazolli, "Marginal Distribution Clustering of Multi-variate Streaming IoT Data", IEEE World Forum on IoT, Dec. 2016.

Dynamic clusters - multivariate data

34D. Puschmann, P. Barnaghi, R. Tafazolli, "Marginal Distribution Clustering of Multi-variate Streaming IoT Data", IEEE World Forum on IoT, Dec. 2016.

Creating Patterns- Adaptive sensor SAX

35F. Ganz, P. Barnaghi, F. Carrez, "Information Abstraction for Heterogeneous Real World Internet Data”, IEEE Sensors Journal, 2013.

From SAX patterns to events/occurrences

36F. Ganz, P. Barnaghi, F. Carrez, "Automated Semantic Knowledge Acquisition from Sensor Data", IEEE Systems Journal, 2014.

Learning ontology from sensory data

37

Patterns and Segmentation of Time-series data

38A. Gonzalez-Vidal, P. Barnaghi, A. F. Skarmeta, BEATS: Blocks of Eigenvalues Algorithm for Time series Segmentation, Submitted to IEEE TKDE, 2016.

KAT- Knowledge Acquisition Toolkit

F. Ganz, D. Puschmann, P. Barnaghi, F. Carrez, "A Practical Evaluation of Information Processing and Abstraction Techniques for the Internet of Things", IEEE Internet of Things Journal, 2015. 39

https://github.com/CityPulse/Knowledge-Acquisition-Toolkit-2.0

http://kat.ee.surrey.ac.uk

KAT V.2.0

40

IoT data

41

Analysing social streams

42Collaboration with Wright State University:

City event extraction from social streams

43

Tweets from a city POS Tagging

Hybrid NER+ Event term extraction

Geohashing

Temporal Estimation

Impact Assessment

Event Aggregatio

nOSM

LocationsSCRIBE

ontology

511.org hierarchy

City Event ExtractionCity Event Annotation

P. Anantharam, P. Barnaghi, K. Thirunarayan, A.P. Sheth, "Extracting City Traffic Events from Social Streams", ACM Trans. on Intelligent Systems and Technology, 2015.

CRF formalisation – for annotation

44

A General CRF Model

Extracted events and the ground truth

45Open source software: https://osf.io/b4q2t/

Extracting city events

46

City Infrastructure

Yes it is police @hasselager … there directing traffic

CRF-based NER

TaggingMulti-view Event

Extraction

Loc. Est. = “hasselager, aarhus”

Temp. Est. = “2015-2-19 21:07:17”

Level = 2

Event = Traffic

OSM Loc. CrimeTrans

p.

City Event Extraction

CNN POS+NE

R Event term

extraction

Cultural Social Enviro

. Sport Health

Data

Transp.

Yes <O> it <O> is <O> police <B-CRIME> @hasselager <B-LOCATION>… <O> there <O>

directing <O> traffic <B-TRAFFIC>

Yes <S-NP/O> it <S-NP/O> is <S-VP/O> police <S-NP/O> @hasselager <S-LOC> ... <O/O> there

<S-NP/O> directing <S-VP/O> traffic <S-NP/O>

Nazli FarajiDavar, Payam Barnaghi, "A Deep Multi-View Learning Framework for City Event Extraction from Twitter Data Streams", submitted to ACM Transactions on Intelligent Systems and Technology (TIST), Nov. 2015.

Extracting city events

47

http://iot.ee.surrey.ac.uk/citypulse-social/

Nazli FarajiDavar, Payam Barnaghi, "A Deep Multi-View Learning Framework for City Event Extraction from Twitter Data Streams", submitted to ACM Transactions on Intelligent Systems and Technology (TIST), Nov. 2015.

Cities of the future

48http://www.globalnerdy.com/2007/08/28/home-electronics-of-the-future-as-predicted-28-years-ago/

49Source: BBC News

Source: The dailymail, http://helenography.net/, http://edwud.com/

What are smart cities?

51

“An ecosystem of systems enabled by the Internet of Things and information communication

technologies.”

“People, resources, and information coming together, operating in an ad-

hoc and/or coordinated way to improve city operations and

everyday activities.”

What does makes smart cities “smart”?

Smart Citizens (more informed and more in control)

Smart Governance (better services and informed decisions)

Smart Environment

Providing more equality and wider reach

Context-aware and situation-aware services

Cost efficacy and supporting innovation

What does makes smart cities “smart”?

How do cities get smarter?

How do cities get smarter?

55

Continuous (near-) real-time sensing/monitoringand data collection

Linked/integrated data and linked/integrated services

Real-time intelligence and actionable-informationfor different situations/services

Smart interaction and actuation

Creating awareness and effective participation

How can technology help to make cities smarter?

The role of data

57Source: The IET Technical Report, Digital Technology Adoption in the Smart Built Environment:

Challenges and opportunities of data driven systems for building, community and city-scale applications, http://www.theiet.org/sectors/built-environment/resources/digital-technology.cfm

58

“Each single data item can be important.”

“Relying merely on data from sources that are unevenly

distributed, without considering background information or social context, can lead to imbalanced interpretations and decisions.”?

59

“The ultimate goal is transforming the raw data to insights and

actionable information and/or creating effective representation

forms for machines and also human users, and providing automated

services.”

This usually requires data from multiple sources, (near-) real time analytics and visualisation and/or

semantic representations.

IoT environments are usually dynamic and (near-) real-time

60

Off-line Data analytics

Data analytics in dynamic environments

Image sources: ABC Australia and 2dolphins.com

What type of problems we expect to solve using the IoT and data analytics solutions?

62Source LAT Times, http://documents.latimes.com/la-2013/

A smart City exampleFuture cities: A view from 1998

63Source:

http://robertluisrabello.com/denial/traffic-in-la/#gallery[default]/0/

Source: wikipedia

Back to the Future: 2013

Common problems

64

Guildford, Surrey

65

101 Smart City scenarios

66http://www.ict-citypulse.eu/scenarios/

Dr Mirko PresserAlexandra Institute

Denmark

Live data

67

68

Event Visualisation

CityPulse demo

69

Users in control or losing control?

70

Image source: Julian Walker, Flicker

71

http://www.ict-citypulse.eu/

https://github.com/CityPulse

eHealth

72Ramesh Jain, Micro reports and Situation Recognition at social machines workshop, 2016.

73Ramesh Jain, Micro reports and Situation Recognition at social machines workshop, 2016.

Medical/Health Data

− The average person is likely to generate more than one million gigabytes of health-related data in their lifetime. This is equivalent to 300 million books.

− Medical data is expected to double every 73 days by 2020.

− 80% of health data is invisible to current systems because it’s unstructured.

− Less than 50% of medical decisions meet evidence-based standards. (source: The rand corporation)

74Source: IBM Research

Unstructured data!

Heterogeneity, multi-modality and volume are among the key issues.

Often natural language!

We need interoperable and machine-interpretable solutions…

75

Medical/Health decision making

− One in five diagnoses are incorrect or incomplete and nearly 1.5 million medication errors are made in the US every year.

− Medical journals publish new treatments and discoveries every day.

− The amount of medical information available is doubling every five years and much of this data is unstructured - often in natural language.

− 81 percent of physicians report that they spend five hours per month or less reading journals.

76Source: IBM Research

Medical/Health data in decision making

− Patient histories can give clues. − Electronic medical record data provide lots of

information.− Current observation and measurement data and

fast analysis of the data can help (combined with other data/medical records).

− This needs fast/accurate/secure data: − Collection/retrieval− Communication− Sharing/Integration− Processing/Analysis − Visualisation/presentation

77

IBM Watson

78

Watson can process the patient data to find relevant facts about family history, current medications and other existing conditions.

It can combines this information with current findings from tests and instruments and then examines all available data sources to form hypotheses and test them.

Watson can also incorporate treatment guidelines, electronic medical record data, doctor's and nurse's notes, research, clinical studies, journal articles, and patient information into the data available for analysis.

Source: IBM

Watson can read 40 million documents in 15 seconds.

Sensely

79Source: http://sense.ly/

Healthcare data analytics- Symptom management

80N. Papachristou, C. Miaskowski, P. Barnaghi, R. Maguire, N. Farajidavar, B. Cooper and X. Hu, "Comparing Machine Learning Clustering with Latent Class Analysis on Cancer Symptoms’ Data", IEEE-NIH 2016, Nov. 2016.

Technology Integrated Health Management (TIHM)− An Internet of Things testbed to support dementia

patients and their carers/doctors. − For patients with early to mild dementia− Remote and technology assisted care, monitoring

and alert.

81

Innovation Partners Nine companies with 25+ devices and services, including monitors, sensors, apps, hubs, virtual assistants, location devices and wearables

The Health Challenge: Dementia 16,801 people with dementia in Surrey – set to rise to

19,000 by 2020 (estimated) - nationally 850,000 - estimated 1m by 2025 (Alzheimer’s Society)

Estimated to cost £26bn p/a in the UK (Alzheimer’s Society): health and social care (NHS and private) + unpaid care

Devices in the IoT will provide actionable data on agitation, mood, sleep, appetite, weight loss, anxiety and wandering – all have a big impact on quality of life and wellbeing

The Health Challenge: Falls Surrey spends £10m a year on fracture care –

with 95% of hip fractures caused by falls

People with dementia suffer significantly higher fall rates that cause injury – with falls the most common cause of injury-related deaths in the over-75s

Devices in the IoT will monitor location, activity and incident, supporting health/care staff and carers, enabling early intervention

The Health Challenge: Carers 5.4m carers supporting ill, older or disabled family

members, friends and partners in England - expected to rise by 40% over the next 20 years.

Value of such informal care estimated at £120bn a year – but carer ‘burnout’ a key reason why loved ones require admission to a care/nursing home.

Devices in the IoT will support carers in their caring asks – and support their own health and wellbeing.

Infrastructure

Interoperability, integration

Security

Data governance

Scalability

Technical Challenge

Device/Data interoperability

87

FIHR4TIHM

88

Health and Safety Monitoring and Alert

Privacy

Security

Tru

st

De

pe

nd

ab

ilit

y

Gateway

Gateway

Data Analytics Engine

IoT Test Bed Cloud

External NHS, GP IT systems

Possible links toOther Test Beds

HyperCat

Gateway

HyperCat

HyperCat

HyperCat

Data-driven and patient centered

Healthcare Applications

Extend into homes – year 1 via two CCG areas, rolling out across four more CCGs in year 2

Reach 350 homes – with a control group of 350 – via dementia register

Focus on most effective product combinations – with potential for more via an open call

Roll Out

NE Hants & FarnhamLiving Lab

Guildford & Waverley

Rest of Surrey

And beyond…

In Conclusion

− Lots of opportunities and in various application domains;

− Enhanced and (near-) real-time insights;− Supporting more automated decision making and

in-depth analysis of events and occurrences by combining various sources of data;

− Providing more and better information to citizens;− Citizens in control; − Transparency and data management issues

(privacy, security, trust, …);− Reliability and dependability of the systems.

92

Accumulated and connected knowledge?

93Image courtesy: IEEE Spectrum

Other challenges and topics that I didn't talk about

Security

Privacy

Trust, resilience and reliability

Noise and incomplete data

Cloud and distributed computing

Networks, test-beds and mobility

Mobile computing

Applications and use-case scenarios

94

Q&A

− Thank you.

http://personal.ee.surrey.ac.uk/Personal/P.Barnaghi/

@pbarnaghi

p.barnaghi@surrey.ac.uk