guest lecture on big data in business,

1© Copyright 2013 Pivotal. All rights reserved. 1.

@krishdpi

The Foundation for ChangeBig Data in Business

10/10/2016SJSU


@krishdpi

[email protected]@krishdpihttp://www.linkedin.com/in/kriss

SK(Saravana Krishnamurthy)

mailto:[email protected]

http://www.linkedin.com/in/kriss

3© Copyright 2013 Pivotal. All rights reserved. 3

@krishdpi

What?

Why?

How?(Use Cases)

Next?


@krishdpi

What?

Why?

How?(Use Cases)

Next?


@krishdpi

What is “Big Data”

“Big Data” refers to datasets whose size is beyond the ability of typical database software tools to capture, store, manage, and analyze. This definition is intentionally subjective and incorporates a moving definition of how big a dataset needs to be in order to be considered big data—i.e., we don’t define big data in terms of being larger than a certain number of terabytes (thousands of gigabytes).

-McKinsey Global Institute, May 2011


@krishdpi

!!!

!!!

!!!

!!!

!!!“Big Data Is Less About Size, And More About Freedom”

―Techcrunch

!!!

!!!“Findings: ‘Big Data’ Is More Extreme Than Volume”

― Gartner “Big Data! It’s Real, It’s Real-time, and It’s Already Changing Your World” ―IDC

“Total data: ‘bigger’ than big data” ― 451 Group

THE ERA OF

BIG DATA

IS HERE


@krishdpi

Data VolumeGrowing 44x

2020: 35.2 Zettabytes

2010:1.2

Zettabytes

The Digital Universe 2010 - 2020

Source: IDC Digital Universe Study, sponsored by EMC, May 2010

Terabyte < Petabyte < Exabyte < Zettabyte < Yottabyte < Xenottabyte < Shilentnobyte < Domegemegrottebyte


@krishdpi

Growth of Data• 2015: Youtube users were uploading 400 hours worth of video every minute works out to be 1PB raw capacity every day

– 2 TB = 500HDD– 500 * 365 = 182,000HDD/Year– $18m /Year in HDD alone.

• Number of Internet-connected devices will reach 33bn by 2020 – Strategy Analytics

• By 2015, Mobile data traffic is predicted to be 75 Exabytes annually – Cisco• Healthcare (as of 2011) is calculated at 150 Exabytes – SAS


@krishdpi

What?

Why?

How?(Use Cases)

Next?


@krishdpi


@krishdpi

Economics Have Changed the Game


@krishdpi

Big Data Analytics: Big Data Analytics: The Path to The Path to

Business ValueBusiness Value

IN THE BIG DATA ERA: ANALYTICS ARE THE KEY TO SUCCESS


@krishdpi

15 Exa Bytes(15,000,000,000,000,000,000 bytes)


@krishdpi

John Reese: “I never understood why people put all their information on those sites. Used to make our job a lot easier at the CIA.”

Harold Finch: “Of course. That's why I created them.”

John Reese: “You're telling me you invented online social networking, Finch?”

Harold Finch: “The Machine needed more information. People's social graph, their associations. The government had been trying to figure it out for years. Turns out most people were happy to volunteer it. Business wound up being quite profitable, too.”


@krishdpi

Analytics TermsAnalytics

The practice of applying aggregations, statistics and models to large datasets to solve problems in business and industry

Business intelligenceAnother term for analytics, but often used to refer specifically to reporting, OLAP and other descriptive statistics

Machine learningAlgorithms that allow computers to learn behaviors from data(Ex. Facebook PYMK, Spam Filters, Speech/Face recognition, self-driving cars)


@krishdpi

Transactions(OLTP)

Analytics(OLAP)

Learning(AI/DL)

Order an under cabinet kitchen water filtering system(Jan 1, 2016)

Dear Jon Doe, The water carbon filter needs to be replacedevery 6 months. Please click here if you would like us to send one tomorrow.(July 1, 2016)

A drone maps out optimal route using geospatial data, weather data, other traffic in the air, location data to deliver the filter at your door steps. (July 2, 2016)


@krishdpi

Analytics Evolution Desired by CustomerHIGH

FutureLOW Past Time

BUSINESS VALUE

ThenBusiness Intelligence(Descriptive)

NowPredictive Analytics and Data Mining

Artificial Intelligence


@krishdpi

What?

Why?

How?(Use Cases)

Next?


@krishdpi

Private/Hybrid Cloud Infrastructure or Appliance

Data Access & Query Layer

Tools & Services

Analytic Productivity Layer

Hadoop

Data Scientist

Data Engineer

Data Analyst

Bl Analyst

LOB User

DatabaseData Platform Admin

DA

TA S

CIE

NC

E T

EA

M

Visualization Layer

CxO/Decision Maker


@krishdpi

Industries Are Broadly Embracing Big Data

Big Data Users


@krishdpi

Big Data Ecosystem Enablers


@krishdpi


@krishdpi

ORACLESQL ServerSAP HANATerradataGreenplum

MS ExcelSASBusiness ObjectsPivotal

Platform SupportRedHatWindowsServerPivotalVMware

Modern Big Data Architecture


@krishdpi

Use Cases


@krishdpi

OLTP(Oracle)

OLAP(Teradata, Hadoop)Extract

(Nightly)

Machine Learning

@Verizon Service #Sucks! @verizon too many call drops

Special Offers


@krishdpi

Flight Test

ObjectiveOptimize flight time

ProblemManual diagnostics4 hours test flight is 2 TB400 000 parameters, only widely 4000 used

SolutionRealtime big data analyticsMachine learning


@krishdpi

ObjectiveImprove patient care

ProblemScattered member dataFrequent hospital visit

SolutionCombine behavioral, contextual dataUtilize member history and data scienceProvide accurate diagnostics


@krishdpi

Physical Data Strategy

Extreme OLTP(Cassandra)

Streaming Data

Interactive Data

Operational(DB2, Oracle,

Informix)

Landing(Hadoop)

Repository(Teradata)

OLTP(DB2, Oracle,

Informix)

Repository(lower SLA)(Greenplum)

Batch

BI(Teradata,

Oracle)

General BI

Perf Analytics(Greenplum)

Lab Analytics(Hadoop)

RL 2.0

Analytics Lab


@krishdpi


@krishdpi

Autonomous Cars


@krishdpi


@krishdpi

Machine LearningDeep Learning

(Beat Lee Sedol in AlphaGo. 2 followed by 56 zeros possible board positions)


@krishdpi

Q & A

guest lecture on big data in business,

Technology