nosql: death to relational databases(?)

bensco'ield – viget labsrubyconf20 november 2009

NoSQLDeath to Relational Databases(?)

Friday, November 20, 2009

Motivations


Performance


Scalability


Flexibility


Locus of Work


Charlie Chaplin

Jet Li

Marian Collier

Hank Mann


Domain Complexity


Taxonomy


Key-‐Value Stores


distributed hash tables


DynamoGT.MPStoreRedis


Column-‐Oriented Stores


semi-‐structured


BigTableCassandraHBase


Document-‐Oriented Stores


also semi-‐structured


CouchDBMongoDBRDDBRiak


Graph Databases


graph theory


ActiveRDF AllegroGraphNeo4J


Comparisons


Performance


key-‐value storescolumn-‐oriented storesdocument-‐oriented storesrelational databasesgraph databases


Scalability


Flexibility


key-‐value storesdocument-‐oriented storesgraph databases column-‐oriented storesrelational databases


Locus of Work


Domain Complexity


Examples


Redis


Data Typesstringslistssets


In-‐Memoryperiodic snapshotsmaster-‐slave replicationmemory-‐bound


require 'redis'

gl = Redis.new

# A stringgl['name'] = 'Kyle Rayner'gl['name']gl.delete('name')

# A listgl.push_tail 'to-dos', 'Lose Ion power'gl.push_tail 'to-dos', 'Mourn dead loved ones'gl.push_tail 'to-dos', 'Blow up zombie lanterns'

gl.list_range('to-dos', 0, -1)


Tokyo Cabinet


Data Typesstringstables!


Related Projectstyrantdystopiapromenade


require 'rufus/tokyo'

# Key-valueleagues = Rufus::Tokyo::Cabinet.new('jl.tch')leagues['JLI'] = [ 'Batman', 'Black Canary', 'Blue Beetle', 'Captain Marvel', 'Doctor Light', 'Doctor Fate', 'Guy Gardner','Martian Manhunter', 'Mister Miracle'].to_yaml

YAML.load(leagues['JLI'])


require 'rufus/tokyo'

# Tablebig7 = Rufus::Tokyo::Table.new('big7.tct')

big7['s'] = {'name' => 'Superman', 'role' => 'deus ex machina'}big7['b'] = {'name' => 'Batman', 'role' => 'mastermind'}big7['gl'] = {'name' => 'Green Lantern', 'role' => 'space cop'}big7['f'] = {'name' => 'Flash', 'role' => 'speedster'}big7['mm'] = {'name' => 'Martian Manhunter', 'role' => '?'}big7['ww'] = {'name' => 'Wonder Woman', 'role' => 'hitter'}big7['a'] = {'name' => 'Aquaman', 'role' => 'fish-talking'}

big7.query {|q| q.add_condition 'role', :streq, 'fish-talking'}


Cassandra


GenealogyDynamoBigTable


Column-‐Orientedcolumnssupercolumnscolumn families


Distributedautomatic replicationeventual consistencyeasy expansion


Availabilityweak readsquorum reads


require 'cassandra'

op = Cassandra.new('OnePiece')

op.insert(:People, '1', {'name' => 'Luffy'})op.insert(:People, '2', {'name' => 'Crocodile'})op.insert(:People, '3', {'name' => 'Mr. 3'})

op.insert(:Fights, '1', {'opponents' => {UUID.new => '2'}})op.insert(:Fights, '1', {'opponents' => {UUID.new => '3'}})

luffy_fights = op.get(:Fights, '1', 'opponents')luffy_fights.map {|t, opp| op.get(:People, opp, 'name')}


CouchDB


Web-‐InspiredJSON storageHTTP / RESTful interface


ViewspredeVined, updated incrementallyjavascript for map/reduce


Updatesfull, including embedded documents


require 'couchrest'

konoha = CouchRest.database!('http://127.0.0.1:5984/konoha')naruto = konoha.save_doc { 'name' => 'Naruto Uzumaki', 'chakra' => 'wind'}shikamaru = konoha.save_doc { 'name' => 'Shikamaru Nara', 'chunin' => true}

konoha.save_doc { '_id' => '_design/first', :views => { :chunin => { :map => 'function(doc){if(doc.chunin){emit(null, doc);}}' } }}

puts konoha.views('first/chunin')['rows'].inspect


MongoDB


Storagebinary JSON documents


Accessnative clients


Queriesdynamicindex-‐based


Updatesallows partial updates


require 'mongo'

avengers = Mongo::Connection.new.db('avengers')members = avengers.collection('members')

members.insert {'name' => 'Ant-Man'}members.insert {'name' => 'Hulk'}members.insert {'name' => 'Iron Man'}members.insert {'name' => 'Thor'}members.insert {'name' => 'Wasp'}

members.create_index('name')

pym = members.find {'name' => 'Ant-Man'}pym['name'] = 'Giant-Man'pym.save

members.remove {'name' => 'Hulk'}

members.insert {'name' => 'Captain America'}


Neo4J


Structurenodes and edgeskey-‐value pairs


Querieslucene


require 'neo4j'

class Person include Neo4j::NodeMixin property :name, :mutant index :name, :mutant has_n :interests has_n :dates has_n :marriages def initialize(name, mutant = true) name = name mutant = mutant endend


Neo4j::Transaction.run do magneto = Person.new('Magneto') esme = Person.new('Esme') rogue = Person.new('Rogue') magda = Person.new('Magda', false) wasp = Person.new('Wasp', false) magneto.interests << wasp magneto.marriages << magda magneto.dates << rogue esme.interests << magneto magda.marriages << magneto rogue.dates << magnetoend


magneto = Person.find(:name => 'Magneto')

# Who likes Magneto?magneto.relationships.incoming(:interests).nodes

# Which non-mutants has Magneto dated?magneto.dates{ !mutant? }.to_a


Simulations


Structure


people{ ‘name’:‘Jimmy Olsen’ ‘title’:‘Superman’s Pal’ ‘company_id’:12441}

companies{ _id:12441 ‘name’:‘Daily Planet’}


Lack of Structure


mysql> SELECT * FROM people LIMIT 1 \G*************************** 1. row *************************** id: 1content: --- company: Daily Planetname: Jimmy Olsentitle: Superman’s Pal


mysql> desc people;+-------+-------------+------+-----+---------+-------+| Field | Type | Null | Key | Default | Extra |+-------+-------------+------+-----+---------+-------+| id | int(11) | YES | | NULL | || name | varchar(50) | YES | | NULL | |+-------+-------------+------+-----+---------+-------+

mysql> desc attributes;+-----------+--------------+------+-----+---------+-------+| Field | Type | Null | Key | Default | Extra |+-----------+--------------+------+-----+---------+-------+| id | int(11) | YES | | NULL | || person_id | int(11) | YES | | NULL | || attribute | varchar(50) | YES | | NULL | || value | varchar(100) | YES | | NULL | |+-----------+--------------+------+-----+---------+-------+


Not Only SQL


Caching


Already in Usememcached


Logging


Rails Log Replacementhttp://github.com/peburrows/mongo_db_logger


http://github.com/peburrows/mongo_db_logger

http://github.com/peburrows/mongo_db_logger

Hybrid Domains


different domains


Publishinge-‐commercedocuments


Datinge-‐commercesocial graph


different scales


Photo Sharinguser accountsuploaded photos


Next Steps


Explore


Database Listhttp://internetmindmap.com/database_software

NoSQL Google Grouphttp://groups.google.com/group/nosql-‐discussion

NoSQL Ecosystemhttp://www.rackspacecloud.com/blog/2009/11/09/nosql-‐ecosystem/

Wave!


http://internetmindmap.com/database_software

http://internetmindmap.com/database_software

http://groups.google.com/group/nosql-discussion




Ignore the Database


Logical Modeling Firstbe mindful


Change the Default


Application Templatesstart with something new


@bsco'ieldben.sco'[email protected]://spkr8.com/bscoVieldhttp://viget.com/extendhttp: //benscoVield.com

bensco'ield


mailto:[email protected]

mailto:[email protected]

http://spkr8.com/bscofield

http://spkr8.com/bscofield

http://viget.com/extend

http://viget.com/extend

nosql: death to relational databases(?)

Technology

condition role

konoha naruto

tail todos

list gl

keyvalue leagues

ion power gl

kyle rayner glname gl

martian manhunter