guest lecture on big data in business,
Post on 13-Apr-2017
150 Views
Preview:
TRANSCRIPT
1© Copyright 2013 Pivotal. All rights reserved. 1.
@krishdpi
The Foundation for ChangeBig Data in Business
10/10/2016SJSU
2© Copyright 2013 Pivotal. All rights reserved. 2.
@krishdpi
kriss@mba.berkeley.edu@krishdpihttp://www.linkedin.com/in/kriss
SK(Saravana Krishnamurthy)
3© Copyright 2013 Pivotal. All rights reserved. 3
@krishdpi
What?
Why?
How?(Use Cases)
Next?
4© Copyright 2013 Pivotal. All rights reserved. 4
@krishdpi
What?
Why?
How?(Use Cases)
Next?
5© Copyright 2013 Pivotal. All rights reserved. 5.
@krishdpi
What is “Big Data”
“Big Data” refers to datasets whose size is beyond the ability of typical database software tools to capture, store, manage, and analyze. This definition is intentionally subjective and incorporates a moving definition of how big a dataset needs to be in order to be considered big data—i.e., we don’t define big data in terms of being larger than a certain number of terabytes (thousands of gigabytes).
-McKinsey Global Institute, May 2011
6© Copyright 2013 Pivotal. All rights reserved. 6.
@krishdpi
!!!
!!!
!!!
!!!
!!!“Big Data Is Less About Size, And More About Freedom”
―Techcrunch
!!!
!!!“Findings: ‘Big Data’ Is More Extreme Than Volume”
― Gartner “Big Data! It’s Real, It’s Real-time, and It’s Already Changing Your World” ―IDC
“Total data: ‘bigger’ than big data” ― 451 Group
THE ERA OF
BIG DATA
IS HERE
7© Copyright 2013 Pivotal. All rights reserved. 7.
@krishdpi
Data VolumeGrowing 44x
2020: 35.2 Zettabytes
2010:1.2
Zettabytes
The Digital Universe 2010 - 2020
Source: IDC Digital Universe Study, sponsored by EMC, May 2010
Terabyte < Petabyte < Exabyte < Zettabyte < Yottabyte < Xenottabyte < Shilentnobyte < Domegemegrottebyte
8© Copyright 2013 Pivotal. All rights reserved. 8.
@krishdpi
Growth of Data• 2015: Youtube users were uploading 400 hours worth of video every minute works out to be 1PB raw capacity every day
– 2 TB = 500HDD– 500 * 365 = 182,000HDD/Year– $18m /Year in HDD alone.
• Number of Internet-connected devices will reach 33bn by 2020 – Strategy Analytics
• By 2015, Mobile data traffic is predicted to be 75 Exabytes annually – Cisco• Healthcare (as of 2011) is calculated at 150 Exabytes – SAS
9© Copyright 2013 Pivotal. All rights reserved. 9
@krishdpi
What?
Why?
How?(Use Cases)
Next?
10© Copyright 2013 Pivotal. All rights reserved. 10
@krishdpi
11© Copyright 2013 Pivotal. All rights reserved. 11
@krishdpi
12© Copyright 2013 Pivotal. All rights reserved. 12.
@krishdpi
Economics Have Changed the Game
13© Copyright 2013 Pivotal. All rights reserved. 13.
@krishdpi
Big Data Analytics: Big Data Analytics: The Path to The Path to
Business ValueBusiness Value
IN THE BIG DATA ERA: ANALYTICS ARE THE KEY TO SUCCESS
14© Copyright 2013 Pivotal. All rights reserved. 14
@krishdpi
15 Exa Bytes(15,000,000,000,000,000,000 bytes)
15© Copyright 2013 Pivotal. All rights reserved. 15.
@krishdpi
John Reese: “I never understood why people put all their information on those sites. Used to make our job a lot easier at the CIA.”
Harold Finch: “Of course. That's why I created them.”
John Reese: “You're telling me you invented online social networking, Finch?”
Harold Finch: “The Machine needed more information. People's social graph, their associations. The government had been trying to figure it out for years. Turns out most people were happy to volunteer it. Business wound up being quite profitable, too.”
16© Copyright 2013 Pivotal. All rights reserved. 16.
@krishdpi
Analytics TermsAnalytics
The practice of applying aggregations, statistics and models to large datasets to solve problems in business and industry
Business intelligenceAnother term for analytics, but often used to refer specifically to reporting, OLAP and other descriptive statistics
Machine learningAlgorithms that allow computers to learn behaviors from data(Ex. Facebook PYMK, Spam Filters, Speech/Face recognition, self-driving cars)
17© Copyright 2013 Pivotal. All rights reserved. 17
@krishdpi
Transactions(OLTP)
Analytics(OLAP)
Learning(AI/DL)
Order an under cabinet kitchen water filtering system(Jan 1, 2016)
Dear Jon Doe, The water carbon filter needs to be replacedevery 6 months. Please click here if you would like us to send one tomorrow.(July 1, 2016)
A drone maps out optimal route using geospatial data, weather data, other traffic in the air, location data to deliver the filter at your door steps. (July 2, 2016)
18© Copyright 2013 Pivotal. All rights reserved. 18.
@krishdpi
Analytics Evolution Desired by CustomerHIGH
FutureLOW Past Time
BUSINESS VALUE
ThenBusiness Intelligence(Descriptive)
NowPredictive Analytics and Data Mining
Artificial Intelligence
19© Copyright 2013 Pivotal. All rights reserved. 19
@krishdpi
What?
Why?
How?(Use Cases)
Next?
20© Copyright 2013 Pivotal. All rights reserved. 20.
@krishdpi
Private/Hybrid Cloud Infrastructure or Appliance
Data Access & Query Layer
Tools & Services
Analytic Productivity Layer
Hadoop
Data Scientist
Data Engineer
Data Analyst
Bl Analyst
LOB User
DatabaseData Platform Admin
DA
TA S
CIE
NC
E T
EA
M
Visualization Layer
CxO/Decision Maker
21© Copyright 2013 Pivotal. All rights reserved. 21.
@krishdpi
Industries Are Broadly Embracing Big Data
Big Data Users
22© Copyright 2013 Pivotal. All rights reserved. 22.
@krishdpi
Big Data Ecosystem Enablers
23© Copyright 2013 Pivotal. All rights reserved. 23
@krishdpi
24© Copyright 2013 Pivotal. All rights reserved. 24.
@krishdpi
ORACLESQL ServerSAP HANATerradataGreenplum
MS ExcelSASBusiness ObjectsPivotal
Platform SupportRedHatWindowsServerPivotalVMware
Modern Big Data Architecture
25© Copyright 2013 Pivotal. All rights reserved. 25.
@krishdpi
Use Cases
26© Copyright 2013 Pivotal. All rights reserved. 26.
@krishdpi
OLTP(Oracle)
OLAP(Teradata, Hadoop)Extract
(Nightly)
Machine Learning
@Verizon Service #Sucks! @verizon too many call drops
Special Offers
27© Copyright 2013 Pivotal. All rights reserved. 27.
@krishdpi
Flight Test
ObjectiveOptimize flight time
ProblemManual diagnostics4 hours test flight is 2 TB400 000 parameters, only widely 4000 used
SolutionRealtime big data analyticsMachine learning
28© Copyright 2013 Pivotal. All rights reserved. 28.
@krishdpi
ObjectiveImprove patient care
ProblemScattered member dataFrequent hospital visit
SolutionCombine behavioral, contextual dataUtilize member history and data scienceProvide accurate diagnostics
29© Copyright 2013 Pivotal. All rights reserved. 29.
@krishdpi
Physical Data Strategy
Extreme OLTP(Cassandra)
Streaming Data
Interactive Data
Operational(DB2, Oracle,
Informix)
Landing(Hadoop)
Repository(Teradata)
OLTP(DB2, Oracle,
Informix)
Repository(lower SLA)(Greenplum)
Batch
BI(Teradata,
Oracle)
General BI
Perf Analytics(Greenplum)
Lab Analytics(Hadoop)
RL 2.0
Analytics Lab
30© Copyright 2013 Pivotal. All rights reserved. 30
@krishdpi
31© Copyright 2013 Pivotal. All rights reserved. 31
@krishdpi
32© Copyright 2013 Pivotal. All rights reserved. 32
@krishdpi
33© Copyright 2013 Pivotal. All rights reserved. 33
@krishdpi
Autonomous Cars
34© Copyright 2013 Pivotal. All rights reserved. 34
@krishdpi
35© Copyright 2013 Pivotal. All rights reserved. 35
@krishdpi
Machine LearningDeep Learning
(Beat Lee Sedol in AlphaGo. 2 followed by 56 zeros possible board positions)
36© Copyright 2013 Pivotal. All rights reserved. 36.
@krishdpi
Q & A
top related