IS3321 Information Systems Solutions for the Digital Enterprise Lecture 2: Big data and the Internet of Things Rob Gleasure robgleasure.com IS3321 Todays session Introduction to big data The 3 Vs of big data The Internet of Things Big data Last session The cloud Bandwidth capacity increasing year on year Move to pay-as-you-go web-hosted services for software (Saas) platforms (PaaS) infrastructures (Iaas) All of this interaction with one linked information system means vast quantities of data can be captured throughout user interaction, often in real-time big data Big data The idea is that the vast amounts of interaction data allow for systems that are nuanced and responsive in ways that were previously not possible Also a realisation that, if it can be analysed, this data is a huge commodity, meaning new business models are possible So when is data big data 3 Vs of Big data Volume Facebook generates 10TB of new data daily, Twitter 7TB A Boeing 737 generates 240 terabytes of flight data during a flight from one side of the US to the other We can use all of this data to tell us something, if we know the right questions to ask 3 Vs of Big data FromTraditional ApproachBig Data Approach Analyze small subsets of data Analyze all data Analyzed information All available information All available information analyzed 3 Vs of Big data Velocity Clickstreams and asynchronous data transfer can capture what millions of users are doing right now Think back to AirBnB make a change, then watch the response. No guesswork required up front as to what to gather, we can induce the interesting stuff as we see it 3 Vs of Big data FromStart with hypothesis and test against selected data Explore all data and identify correlations HypothesisQuestion Data Answer Exploration Correlation Insight Traditional ApproachBig Data Approach Data 3 Vs of Big data Variety Move from structured data to unstructured data, including image recognition, text mining, etc. Gathered from users, applications, systems, sensors Increasingly comprehensive data view of our ecosystem The Internet of Things The Internet of Things From The Internet of Things RFID sensors, bluetooth, microprocessors, wifi all becoming easier to embed in dumb devices Move to mobile also means more data streaming from us at all times, e.g. location, call activity, net use The Internet of Things Smart homes/smart cities Temperature, lighting, food stocks, energy, security Smart cars Diagnostics, traffic suggestions, sensors, self-driving Smart healthcare Worn and intravenous computing detects issues early and monitors care outcomes remotely Smart factories, farms Machines coordinated efficiently, linked dynamically to consumption models Big data Success stories Books Barnes and Noble: Discovered that readers often quit nonfiction books less than halfway through. Introduced highly successful new series of short books on topical themes Amazon: originally used a panel of expert reviewers for books. Data surplus allowed them to create increasingly predictive recommendations. Panel has since been disbanded and 1/3 of sales are now driven by the recommender system Big data and the Internet of Things Success stories (continued) Transport Flyontime.us: used historical weather and flight delay information to predict likelihood of flights get delayed Farecast: looked at ticket prices for specific flights based on historical data, then advised users to buy or wait according to predicted fare costing trajectory UPS: Uses a range of traffic data to calculate most efficient time/fuel efficient routes according to complex algorithm Big data and the Internet of Things Famous success stories (continued) Healthcare Modernizing Medicine EMA dermatology system https://www.youtube.com/watch?v=jMGaGtK9nzU Big data and the Internet of Things Famous success stories (continued) Social media Google (data for information relevance) Twitter (c.f. #RescuePH) Facebook (social data) Big data and privacy Morey et al. argue people create roughly three types of data of increasing sensitivity Self-reported data Digital exhaust Profiling data Due to the growth in biotechnologies and sensors, theres an argument that profiling data could be further broken down to differentiate between digital profile data and biometric data Data beneficiaries Companies may then use the data in three different ways Improve product or service Facilitate targeted marketing Sell data to third parties Google search is an example of a digital business that combines all of these Your search behaviour becomes customised Ads are placed in front of you according to your history and location Click-through behaviour and user overviews are provided to third parties Issues with big data Google Flu Trends Life imitating data, imitating life? No one is really average height Your Xbox knows you like that Katy Perry song Also, Target called to say your teenage daughter is pregnant. Icecream sales and shark attacks FromIcecream sales and shark attacks continued (correlation, not causation) Targets family monitoring continued Readings Mayer-Schnberger, V. and Cukier, K. (2014). Big Data: A Revolution That Will Transform How We Live, Work, and Think, John Murray Publishers, UK. Morey, T., Forbath, T. T., & Schoop, A. (2015). Customer Data: Designing for Transparency and Trust. Harvard Business Review, 93(5), find-disaster-victims