what is big data?media.ogn.s3.amazonaws.com/ogn33/keynote-inessombra.pdf · relational nosql “the...

26
BIG DATA? What is

Upload: others

Post on 28-May-2020

11 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: What is BIG DATA?media.ogn.s3.amazonaws.com/ogn33/keynote-InesSombra.pdf · Relational NoSQL “The rest ... Big Data/NoSQL brings data closer to the format it’s generated. SOA,

BIG DATA?What is

Page 2: What is BIG DATA?media.ogn.s3.amazonaws.com/ogn33/keynote-InesSombra.pdf · Relational NoSQL “The rest ... Big Data/NoSQL brings data closer to the format it’s generated. SOA,

INES SOMBRA Data @

Engine Yard

Page 3: What is BIG DATA?media.ogn.s3.amazonaws.com/ogn33/keynote-InesSombra.pdf · Relational NoSQL “The rest ... Big Data/NoSQL brings data closer to the format it’s generated. SOA,

ENGINE YARD

Page 4: What is BIG DATA?media.ogn.s3.amazonaws.com/ogn33/keynote-InesSombra.pdf · Relational NoSQL “The rest ... Big Data/NoSQL brings data closer to the format it’s generated. SOA,
Page 5: What is BIG DATA?media.ogn.s3.amazonaws.com/ogn33/keynote-InesSombra.pdf · Relational NoSQL “The rest ... Big Data/NoSQL brings data closer to the format it’s generated. SOA,
Page 6: What is BIG DATA?media.ogn.s3.amazonaws.com/ogn33/keynote-InesSombra.pdf · Relational NoSQL “The rest ... Big Data/NoSQL brings data closer to the format it’s generated. SOA,

VS VS

THE DATA UNIVERSERelational NoSQL “The rest”

Page 7: What is BIG DATA?media.ogn.s3.amazonaws.com/ogn33/keynote-InesSombra.pdf · Relational NoSQL “The rest ... Big Data/NoSQL brings data closer to the format it’s generated. SOA,

YOUR DATA JOURNEY

Page 8: What is BIG DATA?media.ogn.s3.amazonaws.com/ogn33/keynote-InesSombra.pdf · Relational NoSQL “The rest ... Big Data/NoSQL brings data closer to the format it’s generated. SOA,

VERTICALLY SCALE

Page 9: What is BIG DATA?media.ogn.s3.amazonaws.com/ogn33/keynote-InesSombra.pdf · Relational NoSQL “The rest ... Big Data/NoSQL brings data closer to the format it’s generated. SOA,

READ REPLICAS

Page 10: What is BIG DATA?media.ogn.s3.amazonaws.com/ogn33/keynote-InesSombra.pdf · Relational NoSQL “The rest ... Big Data/NoSQL brings data closer to the format it’s generated. SOA,

OH NO, SHARDING

Page 11: What is BIG DATA?media.ogn.s3.amazonaws.com/ogn33/keynote-InesSombra.pdf · Relational NoSQL “The rest ... Big Data/NoSQL brings data closer to the format it’s generated. SOA,

OH NO, SHARDING

Page 12: What is BIG DATA?media.ogn.s3.amazonaws.com/ogn33/keynote-InesSombra.pdf · Relational NoSQL “The rest ... Big Data/NoSQL brings data closer to the format it’s generated. SOA,

POLYGLOT PERSISTENCE

Page 13: What is BIG DATA?media.ogn.s3.amazonaws.com/ogn33/keynote-InesSombra.pdf · Relational NoSQL “The rest ... Big Data/NoSQL brings data closer to the format it’s generated. SOA,

POLYGLOT PERSISTENCE

Page 14: What is BIG DATA?media.ogn.s3.amazonaws.com/ogn33/keynote-InesSombra.pdf · Relational NoSQL “The rest ... Big Data/NoSQL brings data closer to the format it’s generated. SOA,

POLYGLOT PERSISTENCE

Page 15: What is BIG DATA?media.ogn.s3.amazonaws.com/ogn33/keynote-InesSombra.pdf · Relational NoSQL “The rest ... Big Data/NoSQL brings data closer to the format it’s generated. SOA,

POLYGLOT PERSISTENCE

Page 16: What is BIG DATA?media.ogn.s3.amazonaws.com/ogn33/keynote-InesSombra.pdf · Relational NoSQL “The rest ... Big Data/NoSQL brings data closer to the format it’s generated. SOA,
Page 17: What is BIG DATA?media.ogn.s3.amazonaws.com/ogn33/keynote-InesSombra.pdf · Relational NoSQL “The rest ... Big Data/NoSQL brings data closer to the format it’s generated. SOA,

Big Data/NoSQL brings data closer to the format it’s

generated

Page 18: What is BIG DATA?media.ogn.s3.amazonaws.com/ogn33/keynote-InesSombra.pdf · Relational NoSQL “The rest ... Big Data/NoSQL brings data closer to the format it’s generated. SOA,

SOA, THE HOLIEST OF GRAILSMy awesome e-commerce site

Shopping cart & session data

Completed orders

Inventory and item pricing

Recommendations engine

Session Storage serviceK/V Store

Order Persistence

serviceDocument

Store

Inventory & Price service

RDBMS

Nodes and relations service

Graph Store

NoSQL Distilled: Fowler & Sadalage

Page 19: What is BIG DATA?media.ogn.s3.amazonaws.com/ogn33/keynote-InesSombra.pdf · Relational NoSQL “The rest ... Big Data/NoSQL brings data closer to the format it’s generated. SOA,

MOTIVATING FACTORSAvailability needs

Data & querying models

Consistency needs

Scalability needs

Page 20: What is BIG DATA?media.ogn.s3.amazonaws.com/ogn33/keynote-InesSombra.pdf · Relational NoSQL “The rest ... Big Data/NoSQL brings data closer to the format it’s generated. SOA,

A few Big Data

PLAYERS

Page 21: What is BIG DATA?media.ogn.s3.amazonaws.com/ogn33/keynote-InesSombra.pdf · Relational NoSQL “The rest ... Big Data/NoSQL brings data closer to the format it’s generated. SOA,

Master

Replica

DATATYPE SERVERFAST

EASY SETUPBEST IF DATA FITS

MEMORY

Page 22: What is BIG DATA?media.ogn.s3.amazonaws.com/ogn33/keynote-InesSombra.pdf · Relational NoSQL “The rest ... Big Data/NoSQL brings data closer to the format it’s generated. SOA,

DOCUMENT ORIENTEDFAST(ISH)

BEST IF WORKING SET FITS MEMORY

Primary

Secondary Secondary

Page 23: What is BIG DATA?media.ogn.s3.amazonaws.com/ogn33/keynote-InesSombra.pdf · Relational NoSQL “The rest ... Big Data/NoSQL brings data closer to the format it’s generated. SOA,

node1.4.1

node1.4.1

node1.4.1

node1.4.1

node1.4.1

node1.4.2

KEY/VALUEEXTREMELY FAULT

TOLERANTGREAT OPS STORY

Page 24: What is BIG DATA?media.ogn.s3.amazonaws.com/ogn33/keynote-InesSombra.pdf · Relational NoSQL “The rest ... Big Data/NoSQL brings data closer to the format it’s generated. SOA,

FULL ECOSYSTEM: M/R, HDFS, TASK

TRACKER, ETCEXTREMELY LARGE

DATA VOLUMES THINK BATCH

Page 25: What is BIG DATA?media.ogn.s3.amazonaws.com/ogn33/keynote-InesSombra.pdf · Relational NoSQL “The rest ... Big Data/NoSQL brings data closer to the format it’s generated. SOA,

wipro.com

Page 26: What is BIG DATA?media.ogn.s3.amazonaws.com/ogn33/keynote-InesSombra.pdf · Relational NoSQL “The rest ... Big Data/NoSQL brings data closer to the format it’s generated. SOA,

QUESTIONS?

wipro.com