the nosql ecosystem, relaxed consistency, and …the list, so you don't yell at me cassandra...
TRANSCRIPT
![Page 1: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.vdocuments.mx/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/1.jpg)
The NoSQL Ecosystem,Relaxed Consistency, and
Snoop Dogg
Adam MarcusMIT CSAIL
[email protected] / @marcua
![Page 2: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.vdocuments.mx/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/2.jpg)
About Me
● Social Computing + Database Systems● Easily Distracted: Wrote The NoSQL Ecosystem in
The Architecture of Open Source Applications 1
1 http://www.aosabook.org/en/nosql.html
![Page 3: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.vdocuments.mx/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/3.jpg)
Preliminaries
● Almost always, use PostgreSQL or MySQL● Most problems can be solved by a single (large)
machine● Consider paying a DB vendor to solve problem
![Page 4: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.vdocuments.mx/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/4.jpg)
But sometimes, need >1 machines
● Analytics on MapReduce or Column Stores● Facebook famously stores 1B+ users on ~10K
machines
![Page 5: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.vdocuments.mx/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/5.jpg)
Scaling transactional workloads
● Google spends thousands of person-hours on Megastore/Spanner
● Everyone else jumps through hoops to stick to “relational model” and “SQL”● Partition data to avoid multinode transactions (2PC)● Avoid multirow transactions to prevent locking
![Page 6: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.vdocuments.mx/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/6.jpg)
History, Compressed
Late 1990s Today
![Page 7: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.vdocuments.mx/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/7.jpg)
History, Compressed
Late 1990s Today
Buy RAM
![Page 8: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.vdocuments.mx/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/8.jpg)
History, Compressed
Late 1990s
Wrap RDBMSs
Today
Buy RAM
![Page 9: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.vdocuments.mx/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/9.jpg)
History, Compressed
NoSQL
Late 1990s
Wrap RDBMSs
Today
Buy RAM
![Page 10: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.vdocuments.mx/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/10.jpg)
History, Compressed
NoSQL
Late 1990s
Wrap RDBMSs
Today
Buy RAM
Not Only SQL
![Page 11: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.vdocuments.mx/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/11.jpg)
History, Compressed
NoSQL
Late 1990s
Wrap RDBMSs
Today
Buy RAM Commercialize
![Page 12: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.vdocuments.mx/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/12.jpg)
History, Compressed
NoSQL
Late 1990s
Wrap RDBMSs
Today
Buy RAM Commercialize
![Page 13: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.vdocuments.mx/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/13.jpg)
The List, So You Don't Yell at Me
Cassandra
HBase
Voldemort
Riak
RedisMongoDB
HyperTable
Neo4j
HyperGraphDBDEX
InfoGrid
VertexDB
Sones
CouchDB
AllegroGraph
BerkeleyDB
Oracle NoSQLMemcacheDBTokyo Cabinet
FlockDB
![Page 14: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.vdocuments.mx/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/14.jpg)
The List, So You Don't Yell at Me
Cassandra
HBase
Voldemort
Riak
RedisMongoDB
HyperTable
Neo4j
HyperGraphDBDEX
InfoGrid
VertexDB
Sones
CouchDB
AllegroGraph
BerkeleyDB
Oracle NoSQLMemcacheDBTokyo Cabinet
FlockDB
Marcus' Law of Databases:
The number of persistence options doubles every 1.5 years
![Page 15: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.vdocuments.mx/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/15.jpg)
history
the parents of nosql
tradeoffs + demos
scaling with snoop dogg
real-world usage
discussion
![Page 16: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.vdocuments.mx/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/16.jpg)
BigTable
● Google, 2006● Column store with sloppy schemas● Strong consistency● Open source: HBase
![Page 17: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.vdocuments.mx/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/17.jpg)
Dynamo
● Amazon, 2007● Key-value store● Allows eventual consistency● Open source: Voldemort, Riak
![Page 18: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.vdocuments.mx/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/18.jpg)
history
the parents of nosql
tradeoffs + demos
scaling with snoop dogg
real-world usage
discussion
![Page 19: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.vdocuments.mx/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/19.jpg)
understanding NoSQL=
understanding a long series of tradeoffs
![Page 20: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.vdocuments.mx/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/20.jpg)
Standards-compliant SQL Systems
● Relational model● Powerful query language● Transactional semantics● Predefined schemas● Strong consistency between replicas*
![Page 21: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.vdocuments.mx/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/21.jpg)
Standards-compliant SQL Systems
● Relational model● Powerful query language● Transactional semantics● Predefined schemas● Strong consistency between replicas*
NoSQL: Maybe you don't need all of these?
![Page 22: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.vdocuments.mx/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/22.jpg)
NoSQL Systems are a Buffet
![Page 23: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.vdocuments.mx/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/23.jpg)
NoSQL Systems are a Buffet(of progressively larger grenades)
![Page 24: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.vdocuments.mx/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/24.jpg)
NoSQL Systems are a Buffet(of progressively larger grenades)
● Data model● Query model● Durability● Transactional consistency● Partitioning● Replica consistency
![Page 25: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.vdocuments.mx/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/25.jpg)
Prof. Madden will not quiz you on specific features of NoSQL systems
(i.e., don't memorize that MongoDB is a document store)
![Page 26: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.vdocuments.mx/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/26.jpg)
Data Model
● Usually key-based...● Binary blob: Voldemort● Documents: MongoDB, CouchDB, Riak● Data structures: Redis● Column-families: HBase, Cassandra
![Page 27: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.vdocuments.mx/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/27.jpg)
Data Model
● Usually key-based...● Binary blob: Voldemort● Documents: MongoDB, CouchDB, Riak● Data structures: Redis● Column-families: HBase, Cassandra
● ...but not always● Graph stores
![Page 28: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.vdocuments.mx/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/28.jpg)
Query Model
● Redis: data structure-specific operations● CouchDB, Riak: MapReduce● Cassandra, MongoDB: SQL-like languages, no
joins or transactions
![Page 29: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.vdocuments.mx/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/29.jpg)
Query Model
● Redis: data structure-specific operations● CouchDB, Riak: MapReduce● Cassandra, MongoDB: SQL-like languages, no
joins or transactions● Third-party
● High-level: PigLatin, HiveQL● Library: Cascading, Crunch● Streaming: Flume, Kafka, S4, Scribe
![Page 30: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.vdocuments.mx/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/30.jpg)
Transactions
● Full ACID for single key● Redis: multi-key single-node transactions
![Page 31: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.vdocuments.mx/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/31.jpg)
demo!
![Page 32: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.vdocuments.mx/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/32.jpg)
history
the parents of nosql
tradeoffs + demos
scaling with snoop dogg
real-world usage
discussion
![Page 33: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.vdocuments.mx/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/33.jpg)
Single-server durability
● Memory only: memcached● Single-server durability: the rest
● fsync every N seconds: most● Write-ahead logging: Cassandra, HBase, Redis, Riak● Group commit: Cassandra, HBase/HDFS
![Page 34: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.vdocuments.mx/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/34.jpg)
When one server is not enough
● Replicate
● Partition
![Page 35: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.vdocuments.mx/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/35.jpg)
When one server is not enough
● Replicate● Performance● K-safety
● Partition
![Page 36: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.vdocuments.mx/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/36.jpg)
When one server is not enough
● Replicate● Performance● K-safety
● Partition● Vertical● Horizontal
![Page 37: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.vdocuments.mx/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/37.jpg)
When one server is not enough
● Replicate● Performance● K-safety
● Partition● Vertical● Horizontal
Consistency/Availability
![Page 38: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.vdocuments.mx/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/38.jpg)
When one server is not enough
● Replicate● Performance● K-safety
● Partition● Vertical● Horizontal Partitioning Scheme
Consistency/Availability
![Page 39: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.vdocuments.mx/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/39.jpg)
Eventual vs. Strong Consistency: FIGHT!
![Page 40: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.vdocuments.mx/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/40.jpg)
Eventual vs. Strong Consistency: FIGHT!
N = # replicasW = # write acknowledgementsR = # read acknowledgements
![Page 41: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.vdocuments.mx/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/41.jpg)
N = # replicasW = # write acknowledgementsR = # read acknowledgements
N = 3
Sally:100K
Sally:100K
Sally:100K
![Page 42: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.vdocuments.mx/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/42.jpg)
N = # replicasW = # write acknowledgementsR = # read acknowledgements
N = 3
Give Sally a raise!
Sally:100K
Sally:100K
Sally:100K
120K
![Page 43: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.vdocuments.mx/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/43.jpg)
N = # replicasW = # write acknowledgementsR = # read acknowledgements
N = 3
Give Sally a raise!
Sally:100K
Sally:100K
Sally:100K
120K
![Page 44: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.vdocuments.mx/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/44.jpg)
N = # replicasW = # write acknowledgementsR = # read acknowledgements
N = 3, W = 1, R = 1
Give Sally a raise!
Sally:100K
Sally:100K
Sally:100K
120K
![Page 45: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.vdocuments.mx/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/45.jpg)
N = # replicasW = # write acknowledgementsR = # read acknowledgements
N = 3, W = 1, R = 1
Sally:120K
Sally:120K
Sally:100K
![Page 46: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.vdocuments.mx/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/46.jpg)
N = # replicasW = # write acknowledgementsR = # read acknowledgements
N = 3, W = 1, R = 1
Sally:120K
Sally:120K
Sally:100K
![Page 47: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.vdocuments.mx/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/47.jpg)
N = # replicasW = # write acknowledgementsR = # read acknowledgements
N = 3, W = 1, R = 1
Sally:120K
Sally:120K
Sally:100K
Sally's Salary?
?
![Page 48: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.vdocuments.mx/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/48.jpg)
N = # replicasW = # write acknowledgementsR = # read acknowledgements
N = 3, W = 1, R = 1
Sally:120K
Sally:120K
Sally:100K
Sally's Salary?
100K
![Page 49: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.vdocuments.mx/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/49.jpg)
N = # replicasW = # write acknowledgementsR = # read acknowledgements
N = 3
Give Sally a raise!
Sally:100K
Sally:100K
Sally:100K
120K
![Page 50: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.vdocuments.mx/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/50.jpg)
N = # replicasW = # write acknowledgementsR = # read acknowledgements
N = 3, W = 3, R = 1
Give Sally a raise!
Sally:100K
Sally:100K
Sally:100K
120K
![Page 51: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.vdocuments.mx/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/51.jpg)
N = # replicasW = # write acknowledgementsR = # read acknowledgements
N = 3, W = 3, R = 1
Sally:100K
Sally:100K
Sally:100K
120K
Waste of time,Waste of time,Waste of time
![Page 52: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.vdocuments.mx/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/52.jpg)
N = # replicasW = # write acknowledgementsR = # read acknowledgements
N = 3, W = 3, R = 1
Sally:100K
Sally:100K
Sally:100K
120K
![Page 53: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.vdocuments.mx/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/53.jpg)
N = # replicasW = # write acknowledgementsR = # read acknowledgements
N = 3, W = 3, R = 1
Sally:120K
Sally:120K
Sally:120K
![Page 54: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.vdocuments.mx/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/54.jpg)
N = # replicasW = # write acknowledgementsR = # read acknowledgements
N = 3, W = 3, R = 1
Sally:120K
Sally:120K
Sally:120K
Sally's Salary?
?
![Page 55: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.vdocuments.mx/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/55.jpg)
N = # replicasW = # write acknowledgementsR = # read acknowledgements
N = 3, W = 3, R = 1
Sally:120K
Sally:120K
Sally:120K
Sally's Salary?
120K
![Page 56: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.vdocuments.mx/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/56.jpg)
N = # replicasW = # write acknowledgementsR = # read acknowledgements
N = 3
Give Sally a raise!
Sally:100K
Sally:100K
Sally:100K
120K
![Page 57: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.vdocuments.mx/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/57.jpg)
N = # replicasW = # write acknowledgementsR = # read acknowledgements
N = 3, W = 2, R = 2
Give Sally a raise!
Sally:100K
Sally:100K
Sally:100K
120K
![Page 58: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.vdocuments.mx/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/58.jpg)
N = # replicasW = # write acknowledgementsR = # read acknowledgements
N = 3, W = 2, R = 2
Sally:120K
Sally:120K
Sally:100K
![Page 59: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.vdocuments.mx/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/59.jpg)
N = # replicasW = # write acknowledgementsR = # read acknowledgements
N = 3, W = 2, R = 2
Sally:120K
Sally:120K
Sally:100K
![Page 60: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.vdocuments.mx/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/60.jpg)
N = # replicasW = # write acknowledgementsR = # read acknowledgements
N = 3, W = 2, R = 2
Sally:120K
Sally:120K
Sally:100K
Sally's Salary?
?
![Page 61: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.vdocuments.mx/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/61.jpg)
N = # replicasW = # write acknowledgementsR = # read acknowledgements
N = 3, W = 2, R = 2
Sally:120K
Sally:120K
Sally:100K
100K120K120K
QUORUM!
![Page 62: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.vdocuments.mx/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/62.jpg)
N = # replicasW = # write acknowledgementsR = # read acknowledgements
N = 3, W = 2, R = 2
Sally:120K
Sally:120K
Sally:100K
100K120K120K
![Page 63: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.vdocuments.mx/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/63.jpg)
N = 3, W = 1, R = 1
Consistency wild west
Available, inconsistent on failure
![Page 64: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.vdocuments.mx/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/64.jpg)
N = 3, W = 3, R = 1
N = 3, W = 1, R = 1
Consistency wild west
Available, inconsistent on failure
Consistent, unavailable writes on failure
![Page 65: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.vdocuments.mx/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/65.jpg)
N = 3, W = 2, R = 2
N = 3, W = 3, R = 1
N = 3, W = 1, R = 1
Consistency wild west
Available, inconsistent on failure
Consistent, unavailable writes on failure
Available, consistent with 1 failure
![Page 66: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.vdocuments.mx/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/66.jpg)
N = 3, W = 2, R = 2
N = 3, W = 3, R = 1
N = 3, W = 1, R = 1
Consistency wild west
Available, inconsistent on failure
Consistent, unavailable writes on failure
Available, consistent with 1 failure
![Page 67: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.vdocuments.mx/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/67.jpg)
Consistency
![Page 68: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.vdocuments.mx/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/68.jpg)
Consistency (not aCid)
![Page 69: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.vdocuments.mx/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/69.jpg)
Consistency (not aCid)
![Page 70: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.vdocuments.mx/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/70.jpg)
Consistency
● Strong: Appears that all replicas see all writes● Eventual: If left alone, replicas eventually converge● Weak: Replicas have different, divergent versions
![Page 71: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.vdocuments.mx/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/71.jpg)
Consistency
● Strong: Appears that all replicas see all writes● Eventual: If left alone, replicas eventually converge● Weak: Replicas have different, divergent versions
The magicalCAP theorem
line
![Page 72: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.vdocuments.mx/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/72.jpg)
Consistency math
![Page 73: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.vdocuments.mx/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/73.jpg)
Consistency math
Strong: N < R + W
![Page 74: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.vdocuments.mx/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/74.jpg)
Consistency math
Strong: N < R + W Easy: W = N Tricky: W < N
![Page 75: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.vdocuments.mx/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/75.jpg)
Consistency math
Strong: N < R + W Easy: W = N Tricky: W < NEventual/Weak: N >= R + W
![Page 76: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.vdocuments.mx/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/76.jpg)
Consistency Options
● BigTable (HBase): Strong● Dynamo (Voldemort/Riak/Cassandra):
tunable strong or eventual
![Page 77: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.vdocuments.mx/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/77.jpg)
Consistency Options
● BigTable (HBase): Strong● Dynamo (Voldemort/Riak/Cassandra):
tunable strong or eventual
● ...and many others (Yahoo! PNUTs has timeline consistency)
![Page 78: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.vdocuments.mx/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/78.jpg)
How do we getconsistent (eventually)?
Sally:120K
Sally:120K
Sally:100K
Sally:120K
Sally:120K
Sally:120K
![Page 79: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.vdocuments.mx/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/79.jpg)
How does Dynamo go from weak to eventual consistency?
● Version w/ vector clocks, read repair● Hinted handoff on failure● Anti-entropy● Gossip-based membership
![Page 80: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.vdocuments.mx/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/80.jpg)
Version w/ vector clocks
Sally:120K
Sally:100K
?
![Page 81: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.vdocuments.mx/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/81.jpg)
How replicas get out of sync
![Page 82: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.vdocuments.mx/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/82.jpg)
Give Sally a raise!
Sally:100K
Sally:100K
Sally:100K
120K
N = 3, W = 1, R = 1
How replicas get out of sync
![Page 83: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.vdocuments.mx/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/83.jpg)
Sally:120K
Sally:120K
Sally:100K
N = 3, W = 1, R = 1
How replicas get out of sync
![Page 84: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.vdocuments.mx/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/84.jpg)
Sally:120K
Sally:120K
Sally:100K
N = 3, W = 1, R = 1
How replicas get out of sync
![Page 85: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.vdocuments.mx/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/85.jpg)
Sally:120K
Sally:120K
Sally:100K
130K
N = 3, W = 1, R = 1
How replicas get out of sync
MOAR RAISE!
![Page 86: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.vdocuments.mx/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/86.jpg)
Sally:120K
Sally:120K
Sally:130K
N = 3, W = 1, R = 1
How replicas get out of sync
![Page 87: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.vdocuments.mx/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/87.jpg)
Sally:120K
Sally:120K
Sally:130K
N = 3, W = 1, R = 1
How replicas get out of sync
Sally Salary?
?
![Page 88: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.vdocuments.mx/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/88.jpg)
Simple versioning doesn't work!
Sally:100K
Sally:100K
Sally:100K
Version: 20 Version: 20 Version: 20
![Page 89: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.vdocuments.mx/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/89.jpg)
Sally:120K
Sally:120K
Sally:100K
Version: 21 Version: 21 Version: 20
Simple versioning doesn't work!
![Page 90: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.vdocuments.mx/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/90.jpg)
Sally:120K
Sally:120K
Sally:130K
Version: 21 Version: 21 Version: 21
Simple versioning doesn't work!
![Page 91: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.vdocuments.mx/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/91.jpg)
Vector clocks to the rescue
● Maintain vector with version per replica● Each client request sent to one replica, which
forwards to others● Routing replica updates its vector entry by 1
![Page 92: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.vdocuments.mx/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/92.jpg)
Vector clocks to the rescue
● Maintain vector with version per replica● Each client request sent to one replica, which
forwards to others● Routing replica updates its vector entry by 1
Sally:100K
Sally:100K
Sally:100K
Version:<1,1,1>
Version:<1,1,1>
Version:<1,1,1>
![Page 93: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.vdocuments.mx/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/93.jpg)
Vector clocks to the rescue
● Maintain vector with version per replica● Each client request sent to one replica, which
forwards to others● Routing replica updates its vector entry by 1
Sally:100K
Sally:100K
Sally:100K
Version:<1,1,1>
Version:<1,1,1>
Version:<1,1,1>
120K
![Page 94: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.vdocuments.mx/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/94.jpg)
Vector clocks to the rescue
● Maintain vector with version per replica● Each client request sent to one replica, which
forwards to others● Routing replica updates its vector entry by 1
Sally:120K
Sally:100K
Sally:100K
Version:<2,1,1>
Version:<1,1,1>
Version:<1,1,1>
120K
![Page 95: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.vdocuments.mx/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/95.jpg)
Vector clocks to the rescue
● Maintain vector with version per replica● Each client request sent to one replica, which
forwards to others● Routing replica updates its vector entry by 1
Sally:120K
Sally:120K
Sally:100K
Version:<2,1,1>
Version:<2,1,1>
Version:<1,1,1>
120K
120K
![Page 96: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.vdocuments.mx/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/96.jpg)
Vector clocks to the rescue
● Maintain vector with version per replica● Each client request sent to one replica, which
forwards to others● Routing replica updates its vector entry by 1
Sally:120K
Sally:120K
Sally:100K
Version:<2,1,1>
Version:<2,1,1>
Version:<1,1,1>
120K
120K
![Page 97: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.vdocuments.mx/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/97.jpg)
Read repair
● Client reads a value from multiple replicas● Compares vector clocks
● <2,1,1> = <2,1,1>
(in sync)● <2,1,1> > <1,1,1>
(update <1,1,1> before conflict)● <2,1,1> !! <1,1,2>
(conflict)
![Page 98: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.vdocuments.mx/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/98.jpg)
● Version w/ vector clocks, read repair● Hinted handoff on failure● Anti-entropy● Gossip-based membership
How does Dynamo go from weak to eventual consistency?
![Page 99: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.vdocuments.mx/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/99.jpg)
● Version w/ vector clocks, read repair● Hinted handoff on failure● Anti-entropy● Gossip-based membership
Proactivelystay in sync
How does Dynamo go from weak to eventual consistency?
![Page 100: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.vdocuments.mx/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/100.jpg)
● Version w/ vector clocks, read repair● Hinted handoff on failure● Anti-entropy● Gossip-based membership
Proactivelystay in sync
Track replicassymmetrically
How does Dynamo go from weak to eventual consistency?
![Page 101: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.vdocuments.mx/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/101.jpg)
Replication
● Consistency● Availability● Playing catchup
![Page 102: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.vdocuments.mx/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/102.jpg)
Partitioning
● Consistent hashing: Voldemort, Riak● Range partitioning: HBase, MongoDB● Both: Cassandra
![Page 103: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.vdocuments.mx/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/103.jpg)
Your partitioning scheme matters
● Partition so all requests go to one machine● Ideally, all joins happen on one machine● Example: map all GMail for a user to one machine
![Page 104: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.vdocuments.mx/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/104.jpg)
Your partitioning scheme matters
● Partition so all requests go to one machine● Ideally, all joins happen on one machine● Example: map all GMail for a user to one machine
● Highly networked data (Facebook News) hard to partition: roughly two options● Each request floods many partitions● Denormalize data (replicate), partition by user
![Page 105: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.vdocuments.mx/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/105.jpg)
history
the parents of nosql
tradeoffs + demos
scaling with snoop dogg
real-world usage
discussion
![Page 106: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.vdocuments.mx/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/106.jpg)
NoSQL Use-cases
● Cassandra at Netflix● HBase at Facebook● MongoDB at Craigslist
![Page 107: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.vdocuments.mx/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/107.jpg)
Cassandra
● BigTable data model: key column family● Dynamo sharding model: consistent hashing● Eventual or strong consistency
![Page 108: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.vdocuments.mx/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/108.jpg)
Cassandra at Netflix
● Transitioned from Oracle● Store customer profiles, customer:movie watch
log, and detailed usage logging
Note: no multi-record locking (e.g., bank transfer)
In-datacenter: 3 replicas, per-app consistency
read/write quorum = 1, ~1ms latency
read/write quorum = 2, ~3ms latency
“Replicating Datacenter Oracle with Global Apache Cassandra on AWS” by Adrian Cockcroft http://www.slideshare.net/adrianco/migrating-netflix-from-oracle-to-global-cassandra
![Page 109: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.vdocuments.mx/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/109.jpg)
Cassandra at Netflix
● Transitioned from Oracle● Store customer profiles, customer:movie watch
log, and detailed usage logging ● In-datacenter: 3 replicas, per-app consistency
“Replicating Datacenter Oracle with Global Apache Cassandra on AWS” by Adrian Cockcroft http://www.slideshare.net/adrianco/migrating-netflix-from-oracle-to-global-cassandra
![Page 110: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.vdocuments.mx/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/110.jpg)
Cassandra at Netflix (cont'd)
● Benefit: async inter-datacenter replication● Benefit: no downtime for schema changes● Benefit: hooks for live backups
![Page 111: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.vdocuments.mx/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/111.jpg)
HBase
● Data model: key column family● Sharding model: range partitioning● Strong consistency
Applications
Logging events/crawls, storing analytics
Twitter: replicate data from MySQL, Hadoop analytics
Facebook Messages
![Page 112: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.vdocuments.mx/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/112.jpg)
HBase
● Data model: key column family● Sharding model: range partitioning● Strong consistency
● Applications● Logging events/crawls, storing analytics● Twitter: replicate data from MySQL, Hadoop analytics● Facebook Messages
![Page 113: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.vdocuments.mx/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/113.jpg)
HBase for Facebook Messages
● Cassandra/Dynamo eventual consistency was difficult to program against
Benefit: simple consistency model
Benefit: flexible data model
Benefit: simple sharding, load balancing, replication
![Page 114: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.vdocuments.mx/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/114.jpg)
HBase for Facebook Messages
● Cassandra/Dynamo eventual consistency was difficult to program against
● Benefit: simple consistency model● Benefit: flexible data model● Benefit: simple partitioning, load balancing,
replication
![Page 115: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.vdocuments.mx/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/115.jpg)
MongoDB
● Document-based data model● Range-based partitioning● Consistency depends on how you use it
![Page 116: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.vdocuments.mx/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/116.jpg)
MongoDB: Two use-cases
● Archiving at Craigslist● 2.2B historical posts, semi-structured● Relatively large blobs: avg 2KB, max > 4 MB
![Page 117: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.vdocuments.mx/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/117.jpg)
MongoDB: Two use-cases
● Archiving at Craigslist● 2.2B historical posts, semi-structured● Relatively large blobs: avg 2KB, max > 4 MB
● Checkins at Foursquare● Geospatial indexing● Small location-based updates, partitioned on user
![Page 118: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.vdocuments.mx/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/118.jpg)
history
the parents of nosql
tradeoffs + demos
scaling with snoop dogg
real-world usage
discussion
![Page 119: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.vdocuments.mx/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/119.jpg)
Polyglot Persistence
● Account data in MySQL● Cache in memcached● Counters + queues in redis● Statistics in MongoDB● Event logging in HDFS/HBase/Cassandra
![Page 120: The NoSQL Ecosystem, Relaxed Consistency, and …The List, So You Don't Yell at Me Cassandra HBase Voldemort Riak Redis MongoDB HyperTable Neo4j HyperGraphDB DEX InfoGrid VertexDB](https://reader030.vdocuments.mx/reader030/viewer/2022020302/5a9b91a37f8b9a30358e3471/html5/thumbnails/120.jpg)
When you see “NoSQL,” think “tradeoffs”
● Data model● Query Model● Transactions● Consistency vs. Availability● Partitioning schemes
Adam [email protected]
@marcua