internet of things and large-scale data analytics

Internet of Things and Large-scale Data Analytics

1

Payam BarnaghiInstitute for Communication Systems (ICS)/5G Innovation Centre University of SurreyGuildford, United Kingdom

The IET Surrey Network, September 2015

2IBM Mainframe 360, source Wikipedia

Apollo 11 Command Module (1965) had 64 kilobytes of memory operated at 0.043MHz.

An iPhone 5s has a CPU running at speeds of up to 1.3GHzand has 512MB to 1GB of memory

Cray-1 (1975) produced 80 million Floating point operations per second (FLOPS)10 years later, Cray-2 produced 1.9G FLOPS

An iPhone 5s produces 76.8 GFLOPS – nearly a thousand times more

Cray-2 used 200-kilowatt power

Source: Nick T., PhoneArena.com, 2014

Computing Power

4

−Smaller size−More Powerful−More memory and more storage

−"Moore's law" over the history of computing, the number of transistors in a dense integrated circuit has doubled approximately every two years.

5

Sensor devices are becoming widely available

- Programmable devices- Off-the-shelf gadgets/tools

6

More “Things” are being connected

Home/daily-life devicesBusiness and Public infrastructureHealth-care…

7

People Connecting to Things

Motion sensorMotion sensor

Motion sensor

ECG sensor

Internet

8

Things Connecting to Things

- Complex and heterogeneous resources and networks

Example: Radiation Sensor Board (Libelium)

Source: Wireless Sensor Networks to Control Radiation Levels, David Gascón, Marcos Yarza, Libelium, April 2011.

Waspmote

Connected world

10Image courtesy: Wilgengebroed

11

Internet of Things (IoT)

− Extending the current Internet and providing connection, communication, and inter-networking between devices and physical objects, or "Things," is a growing trend that is often referred to as the Internet of Things.

− “The technologies and solutions that enable integration of real world data and services into the current information networking technologies are often described under the umbrella term of the Internet of Things (IoT)”

Mobile Technologies

12Image courtesy: Economist

1G

AMPS, NMT, TACS

2G

GSM. GPRS, TDMA IS-136,

CDMA IS-95, PDC

3G

UMTS, CDMA2000,

4G5G

LTE, LTE-A

PeopleThings

Voice

Text

Data

5G technologiesand standards

Connection + Control M2M/IoT

Change in the communication technologies

Mobile Services and Applications

14

Image courtesy: Economist

15

Things, Devices, Data, and lots of it

image courtesy: Smarter Data - I.03_C by Gwen Vanhee

Cyber-Physical-Social Data

16P. Barnaghi et al., "Digital Technology Adoption in the Smart Built Environment", IET Sector Technical Briefing, The Institution of Engineering and Technology (IET), I. Borthwick (editor), March 2015.

Internet of Things: The story so far

RFID based solutions Wireless Sensor and

Actuator networks, solutions for

communication technologies,

energy efficiency, routing, …

Smart Devices/Web-enabled

Apps/Services, initial products,

vertical applications, early concepts and

demos, …

Motion sensor

Motion sensor

ECG sensor

Physical-Cyber-Social Systems, Linked-data,

semantics,More products, more

heterogeneity, solutions for control and

monitoring, …

Future: Cloud, Big (IoT) Data Analytics, Interoperability, Enhanced Cellular/Wireless

Com. for IoT, Real-world operational use-cases and

Industry and B2B services/applications,

more Standards… P. Barnaghi, A. Sheth, "Internet of Things: the story so far", IEEE IoT Newsletter, September

2014.

17

18

“Each single data item is important.”

“Relying merely on data from sources that are unevenly distributed, without considering background information or social context, can lead to imbalanced interpretations and decisions.”?

Data- Challenges

− Multi-modal and heterogeneous− Noisy and incomplete− Time and location dependent − Dynamic and varies in quality − Crowed sourced data can be unreliable − Requires (near-) real-time analysis− Privacy and security are important issues− Data can be biased- we need to know our data!

19

Data Lifecycle

20Source: The IET Technical Report, Digital Technology Adoption in the Smart Built Environment: Challenges and opportunities of data driven systems for building, community and city-scale applications, http://www.theiet.org/sectors/built-environment/resources/digital-technology.cfm

21

“The ultimate goal is transforming the raw data to insights and actionable knowledge and/or creating effective representation forms for machines and also human users and creating automation.”

This usually requires data from multiple sources, (near-) real time analytics and visualisation and/or semantic representations.

22

“Data will come from various source and from different platforms and various systems.”

This requires an ecosystem of IoT systems with several backend support components (e.g. pub/sub, storage, discovery, and access services). Semantic interoperability is also a key requirement.

Device/Data interoperability

23The slide adapted from the IoT talk given by Jan Holler of Ericsson at IoT Week 2015 in Lisbon.

Search on the Internet/Web in the early days

2424

Accessing IoT data

25

“ The internet/web norm (for now) is often to use an interface to search for the data; the search engines are usually information locators – return the link to the information; IoT data access is more opportunistic and context aware”.

The IoT requires context-aware and opportunistic push mechanism, dynamic device/resource associations and (software-defined) data routing networks.

IoT environments are usually dynamic and (near-) real-time

26

Off-line Data analytics

Data analytics in dynamic environments

Image sources: ABC Australia and 2dolphins.com

What type of problems we expect to solve using the IoT and data analytics solutions?

28Source LAT Times, http://documents.latimes.com/la-2013/

A smart City exampleFuture cities: A view from 1998

29Source: http://robertluisrabello.com/denial/traffic-in-la/#gallery[default]/0/

Source: wikipedia

Back to the Future: 2013

Common problems

30Source: thestar.com.my & skyscrappercity.com

Guildford, Surrey

Applications and potentials

− Analysis of thousands of traffic, pollution, weather, congestion, public transport, waste and event sensory data to provide better transport and city management.

− Converting smart meter readings to information that can help prediction and balance of power consumption in a city.

− Monitoring elderly homes, personal and public healthcare applications.

− Event and incident analysis and prediction using (near) real-time data collected by citizen and device sensors.

− Turning social media data (e.g. Tweets) related to city issues into event and sentiment analysis.

− Any many more…

31

32

EU FP7 CityPulse Project

33

34

CityPulse Consortium

Industrial SIE (Austria,

Romania),ERIC

SME AI,

HigherEducation

UNIS, NUIG,UASO, WSU

City BR, AA

Partners:

Duration: 36 months (2014-2017)

35

AnalyticsToolbox

Context-awareDecision Support,

Visualisation

Knowledge-based

Stream Processing

Real-TimeMonitoring &

Testing

Accuracy & Trust

Modelling

SemanticIntegration

On Demand Data

Federation

OpenReferenceData Sets

Real-TimeIoT InformationExtraction

IoT StreamProcessing

Federation ofHeterogenousData Streams

Design-Time Run-Time Testing

Exposure APIs

Designing for real world problems

101 Smart City scenarios

37http://www.ict-citypulse.eu/scenarios/

Dr Mirko PresserAlexandra Institute Denmark

38

Data Visualisation

39

Event Visualisation

CityPulse demo

40

Data abstraction

41F. Ganz, P. Barnaghi, F. Carrez, "Information Abstraction for Heterogeneous Real World Internet Data", IEEE Sensors Journal, 2013.

Adaptable and dynamic learning methods

http://kat.ee.surrey.ac.uk/

Correlation analysis

43

Analysing social streams

44With

City event extraction from social streams

45

Tweets from a city POS Tagging

Hybrid NER+ Event term extraction

Geohashing

Temporal Estimation

Impact Assessment

Event Aggregatio

nOSM

LocationsSCRIBE

ontology

511.org hierarchy

City Event ExtractionCity Event Annotation

P. Anantharam, P. Barnaghi, K. Thirunarayan, A.P. Sheth, "Extracting City Traffic Events from Social Streams", ACM Trans. on Intelligent Systems and Technology, 2015.

Collaboration with Kno.e.sis, Wright State University

Geohashing

46

0.6 miles

Max-lat

Min-lat

Min-long

Max-long

0.38 miles

37.7545166015625, -122.40966796875

37.7490234375, -122.40966796875

37.7545166015625, -122.420654296875

37.7490234375, -122.420654296875

437.74933, -122.4106711

Hierarchical spatial structure of geohash for representing locations with variable precision.

Here the location string is 5H34

0 1 2 3 4 5 67 8 9 B C D EF G H I J K L

0 172 3 4

5 6 8 9

0 1 2 3 4

5 6 7

0 1 23 4 5

6 7 8

Social media analysis

47

City Infrastructure

Tweets from a city

P. Anantharam, P. Barnaghi, K. Thirunarayan, A. Sheth, "Extracting city events from social streams,“, ACM Transactions on TICS, 2014.

Social media analysis (deep learning – under construction)

48

http://iot.ee.surrey.ac.uk/citypulse-social/

Accumulated and connected knowledge?

49Image courtesy: IEEE Spectrum

Users in control or losing control?

50

Image source: Julian Walker, Flicker

Data Analytics solutions for IoT data

− Great opportunities and many applications;− Enhanced and (near-) real-time insights;− Supporting more automated decision making and

in-depth analysis of events and occurrences by combining various sources of data;

− Providing more and better information to citizens;− …

51

However…

− We need to know our data and its context (density, quality, reliability, …)

− Open Data (there needs to be more real-time data)

− Complementary data − Citizens in control − Transparency and data management issues

(privacy, security, trust, …)− Reliability and dependability of the systems

52

In conclusion

− IoT data analytics is different from common big data analytics.

− Data collection in the IoT comes at the cost of bandwidth, network, energy and other resources.

− Data collection, delivery and processing is also depended on multiple layers of the network.

− We need more resource-aware data analytics methods and cross-layer optimisations.

− The solutions should work across different systems and multiple platforms (Ecosystem of systems).

− Data sources are more than physical (sensory) observation.− The IoT requires integration and processing of physical-cyber-

social data.− The extracted insights and information should be converted

to a feedback and/or actionable information. 53

IET sector briefing report

54

Available at: http://www.theiet.org/sectors/built-environment/resources/digital-technology.cfm

CityPulse stakeholder report

55http://www.ict-citypulse.eu/page/sites/default/files/citypulse_annual_report.pdf

Other challenges and topics that I didn't talk about

Security

Privacy

Trust, resilience and reliability

Noise and incomplete data

Cloud and distributed computing

Networks, test-beds and mobility

Mobile computing

Applications and use-case scenarios

56

Q&A

− Thank you.

http://personal.ee.surrey.ac.uk/Personal/P.Barnaghi/

@pbarnaghi

p.barnaghi@surrey.ac.uk