big data | small problems
DESCRIPTION
Given at OSCON 2011. Version with notes available here: http://dl.dropbox.com/u/348446/big-data-small-problems_with-notes.pdfTRANSCRIPT
A!PRESENTATION!BY @timangladEFROM!YOUR!FRIENDS!AT @cloudant
BigData
SmallProblems
NOSQL TApesNOSQL SUmmer.org
.com
Big EFFING DEALBig Data
Do we need to goover this again?Look out! HE’s got GRAPHS!
1
10
100
1,000
10,000
100,000
1,000,000
1981 1985 1990 1994 1998 2003 2007
Consumer Cost per GB
Sources: Ivan Smith+ IDC, 2008
0
12,500
25,000
37,500
50,000
1986 1994 2002 2010
WorldWIDE STORAGE CAPACITY / CAPITA
Source: The World’s Technological Capacity to Store, Communicate, and Compute Information, Hilbert, et al. Science 1 April 2011: 60-65.
Just Data SIZE?
if I’m asking, you probably know the answer!
The Big DataTrifecta
1. Gigabytes 2. IOPS 3. DOLLARS
Why NOW?
Just technology?
What Ails US…
AILMENT #1Digital Disconnect
AILMENT #2DimensionalOVERLOAD
AILMENT #3Infobesity
Text
AILMENT #4DeterminismDISORDER
AILMENT #5YEARNINGFOR MEANING
AILMENT #6REVOLUTIONDELUSION
AILMENT #7BUBBLYBUZZWORDBUG
AILMENT #8BLACK BOX
AILMENT #9CARGO CULTS
AILMENT #10SCALE-OIL SALESMEN
How do IBig DATA?
RULE #1PAPER / Practice
RULE #2Only UseNOSQL (or Newsql)if you have to.
RULE #3Distribution modelDATA MODELDISK STRUCTURE
RULE #4CATCHY PHRASESARE RED HERRINGS
RULE #5WHO DO YOU TRUST?
A Word aboutRELATIONAL SYSTEMS
A Word aboutSWITCHING
NOSQL / NewSQL
Did we reallyNEED A SEQUEL?
Did we reallyNEED A REMAKE?
NEWSQL
VoltdbClustrixGenieDBXEROUNDDrizzleAzureAMAZON RDS
I CAN HAZPARTITIONTOLERANCE?
IT-FriendlyCompliance-Friendly
NewSQLis about BEING…
NOSQL
Mongodb Couchdb RIAK Cassandra NEO4J(BigCOUCH) Redis? Hadoop?
3 NOSQL TRENDS
TREND #180% vs. 20%
80% Databases
20% Databases
ALSO Hadoop REDIS
TREND #2Storage Separation
You GOTYOUR CASSANDRAIN MY HADOOP
You GOTYOUR ACUNUIN MY CASSANDRA
You GOTYOUR LEVELDBIN MY RIAK
TREND #3Ease of ACCESS
Riak SeARCH PIPE SECONDARY IND.CLOUDANT SEARCHCASSANDRA CQL
Dev-FriendlyOPS-Friendly
NOSQLis about…
.com
Thank You