is1825 multimedia development for internet applications lecture 06: big data and the internet of...

IS1825 Multimedia Development for Internet ApplicationsLecture 06: Big data and the Internet of ThingsRob Gleasure

R.Gleasure@ucc.iehttp://corvus2.ucc.ie/phd/rgleasure/index.html

IS1825

Today’s session Introduction to big data The 3 V’s of big data The Internet of Things

Big data

The idea is that the vast amounts of interaction data allow for systems that are nuanced and responsive in ways that were previously not possible

Also a realisation that, if it can be analysed, this data is a huge commodity, meaning new business models are possible

So when is data ‘big data’

3 Vs of Big data

Volume Facebook generates 10TB of new data daily, Twitter 7TB A Boeing 737 generates 240 terabytes of flight data during a

flight from one side of the US to the other

We can use all of this data to tell us something, if we know the right questions to ask

3 Vs of Big data

From http://www.slideshare.net/ibmcanada/big-dataturning-data-into-insights?qid=0b4c69bc-3db2-4e12-ae47-a362a25752eb&v=qf1&b=&from_search=3

Traditional Approach Big Data Approach

Analyze small subsets of data

Analyze all data

Analyzedinformation

All available information

All available informationanalyzed

3 Vs of Big data

Velocity Clickstreams and asynchronous data transfer can capture what

millions of users are doing right now

Think back to AirBnB – make a change, then watch the response.

No guesswork required up front as to what to gather, we can induce the interesting stuff as we see it

3 Vs of Big data

From http://www.slideshare.net/ibmcanada/big-dataturning-data-into-insights?qid=0b4c69bc-3db2-4e12-ae47-a362a25752eb&v=qf1&b=&from_search=3

Start with hypothesis and test against selected data

Explore all data andidentify correlations

Hypothesis Question

DataAnswer

Exploration

CorrelationInsight

Traditional Approach Big Data Approach

Data

3 Vs of Big data

Variety Move from structured data to unstructured data, including image

recognition, text mining, etc. Gathered from users, applications, systems, sensors

Increasingly comprehensive data view of our ecosystem The Internet of Things

The Internet of Things

From http://www.pcworld.com/article/2039413/new-intel-ceo-creates-mysterious-new-devices-division.html

The Internet of Things

RFID sensors, bluetooth, microprocessors, wifi all becoming easier to embed in ‘dumb’ devices

Move to mobile also means more data streaming from us at all times, e.g. location, call activity, net use

The Internet of Things

Smart homes/smart cities Temperature, lighting, food stocks, energy, security

Smart cars Diagnostics, traffic suggestions, sensors, self-driving

Smart healthcare Worn and intravenous computing detects issues early and

monitors care outcomes remotely

Smart factories, farms Machines coordinated efficiently, linked dynamically to

consumption models

Big data

Success stories Books

Barnes and Noble: Discovered that readers often quit nonfiction books less than halfway through. Introduced highly successful new series of short books on topical themes

Amazon: originally used a panel of expert reviewers for books. Data surplus allowed them to create increasingly predictive recommendations. Panel has since been disbanded and 1/3 of sales are now driven by the recommender system

Big data and the Internet of Things Success stories (continued)

Transport Flyontime.us: used historical weather and flight delay

information to predict likelihood of flights get delayed Farecast: looked at ticket prices for specific flights based on

historical data, then advised users to buy or wait according to predicted fare costing trajectory

UPS: Uses a range of traffic data to calculate most efficient time/fuel efficient routes according to complex algorithm

Big data and the Internet of Things Famous success stories (continued)

Healthcare Modernizing Medicine EMA dermatology system

https://www.youtube.com/watch?v=jMGaGtK9nzU

Big data and the Internet of Things Famous success stories (continued)

Social media Google (data for information relevance) Twitter (c.f. #RescuePH) Facebook (social data)

Issues with big data

Google Flu Trends Life imitating data, imitating life?

No one is really average height

Your Xbox knows you like that Katy Perry song

Also, Target called to say your teenage daughter is pregnant.

Icecream sales and shark attacks…

From http://xkcd.com/552/

Icecream sales and shark attacks continued (correlation, not causation)

Target’s family monitoring continued

Readings

Mayer-Schönberger, V. and Cukier, K. (2014). Big Data: A Revolution That Will Transform How We Live, Work, and Think, John Murray Publishers, UK.

http://nextcity.org/daily/entry/rescuers-use-social-media-twitter-to-find-disaster-victims