bensco'ield  –  viget  labsdeveloper  day  durham27  february  2010

The  State  of  NoSQL

Motivations

PerformanceScalability

Meh

FlexibilityComplexityFunctionality

“Comics”  Is  Hard

Charlie  Chaplin

Jet  Li

Marian  Collier

Hank  Mann

Taxonomy

Key-­‐Value  Stores

distributed  hash  tables

PerformanceScalabilityFlexibilityComplexityFunctionality

highhighhighnonevariable  (none)

DynamoGT.MPStoreRedis

Column-­‐Oriented  Stores

semi-­‐structured

PerformanceScalabilityFlexibilityComplexityFunctionality

highhighmoderatelowminimal

BigTableCassandraHBase

Document-­‐Oriented  Stores

also  semi-­‐structured

PerformanceScalabilityFlexibilityComplexityFunctionality

highvariable  (high)highlowvariable  (low)

CouchDBMongoDBRDDBRiak

Graph  Databases

graph  theory

PerformanceScalabilityFlexibilityComplexityFunctionality

variablevariablehighhighgraph  theory

ActiveRDF  AllegroGraphNeo4J

Relational  Databases

PerformanceScalabilityFlexibilityComplexityFunctionality

variablevariablelowmoderaterelational  algebra

Examples

Redis

Data  Typesstringslistssetssorted  sets

In-­‐Memorysemi-­‐persistent  /  fully  persistentmaster-­‐slave  replicationmemory-­‐bound

require 'redis'

gl = Redis.new

# A stringgl['name'] = 'Kyle Rayner'gl['name']gl.delete('name')

# A listgl.push_tail 'to-dos', 'Lose Ion power'gl.push_tail 'to-dos', 'Mourn dead loved ones'gl.push_tail 'to-dos', 'Blow up zombie lanterns'

gl.list_range('to-dos', 0, -1)

Tokyo  Cabinet

Data  Typesbinary  datastrings

Tables!?

Related  Projectstyrantdystopiapromenade

require 'rufus/tokyo'

# Key-valuejli = Rufus::Tokyo::Cabinet.new('jl.tch')jli['members'] = [ 'Batman', 'Black Canary', 'Blue Beetle', 'Captain Marvel', 'Doctor Light', 'Doctor Fate', 'Guy Gardner', 'Martian Manhunter', 'Mister Miracle'].to_yaml

YAML.load(jli['members'])

require 'rufus/tokyo'

# Tablebig7 = Rufus::Tokyo::Table.new('big7.tct')

big7['s'] = {'name' => 'Superman', 'role' => 'deus ex machina'}big7['b'] = {'name' => 'Batman', 'role' => 'mastermind'}big7['gl'] = {'name' => 'Green Lantern', 'role' => 'space cop'}big7['f'] = {'name' => 'Flash', 'role' => 'speedster'}big7['mm'] = {'name' => 'Martian Manhunter', 'role' => '?'}big7['ww'] = {'name' => 'Wonder Woman', 'role' => 'hitter'}big7['a'] = {'name' => 'Aquaman', 'role' => 'fish-talking'}

big7.query {|q| q.add_condition 'role', :streq, 'fish-talking'}

Cassandra

GenealogyDynamoBigTable

Column-­‐Orientedcolumnssupercolumnscolumn  families

Distributedautomatic  replicationeventual  consistencyeasy  expansion

Availabilityweak  reads/writesquorum  reads/writes

require 'cassandra'

op = Cassandra.new('OnePiece')

op.insert(:People, '1', {'name' => 'Luffy'})op.insert(:People, '2', {'name' => 'Crocodile'})op.insert(:People, '3', {'name' => 'Mr. 3'})

op.insert(:Fights, '1', {'opponents' => {UUID.new => '2'}})op.insert(:Fights, '1', {'opponents' => {UUID.new => '3'}})

luffy_fights = op.get(:Fights, '1', 'opponents')luffy_fights.map {|t, opp| op.get(:People, opp, 'name')}

CouchDB

Web-­‐InspiredJSON  storageHTTP  /  RESTful  interface

ViewspredeWined,  updated  incrementallyjavascript  for  map/reduce

Updatesfull,  including  embedded  documents

require 'couchrest'

konoha = CouchRest.database!('http://127.0.0.1:5984/konoha')naruto = konoha.save_doc { 'name' => 'Naruto Uzumaki', 'chakra' => 'wind'}shikamaru = konoha.save_doc { 'name' => 'Shikamaru Nara', 'chunin' => true}

konoha.save_doc { '_id' => '_design/first', :views => { :chunin => { :map => 'function(doc){if(doc.chunin){emit(null, doc);}}' } }}

puts konoha.views('first/chunin')['rows'].inspect

MongoDB

Storagebinary  JSON  documentshttp://bsonspec.org

Accessnative  clients

Queriesdynamicindex-­‐based

Updatesallows  partial  updates

require 'mongo'

avengers = Mongo::Connection.new.db('avengers')members = avengers.collection('members')

members.insert {'name' => 'Ant-Man'}members.insert {'name' => 'Hulk'}members.insert {'name' => 'Iron Man'}members.insert {'name' => 'Thor'}members.insert {'name' => 'Wasp'}

members.create_index('name')

pym = members.find {'name' => 'Ant-Man'}pym['name'] = 'Giant-Man'pym.save

members.remove {'name' => 'Hulk'}

members.insert {'name' => 'Captain America'}

Riak

Ask  Sean

also  Web-­‐InspiredJSON  storageHTTP  /  RESTful  interfacelinks  for  relationships

Decentralizedno  privileged  nodes

ConWigurablestore  /  read  /  write

require 'jiak'

jc = JiakClient.new('127.0.0.1', 8098)jc.set_bucket_schema('supervillains', { 'allowed_fields' => ['name', 'alias', 'power']})

jc.store({ 'bucket' => 'supervillains', 'key' => 'Normie', 'object' => { 'name' => 'Norman Osborn', 'alias' => 'Green Goblin', 'power' => 'Supreme jerkfacedness' }, 'links' => []})

kth = jc.fetch('supervillains', 'Normie')

Neo4J

Structurenodes  and  edgeskey-­‐value  pairs

Querieslucenegremlin

require 'neo4j'

def initialize(name, mutant = true) name = name mutant = mutant endend

class Person include Neo4j::NodeMixin property :name, :mutant index :name, :mutant has_n :crushes has_n :hookups has_n :marriages

Neo4j::Transaction.run do magneto = Person.new('Magneto') esme = Person.new('Esme') rogue = Person.new('Rogue') magda = Person.new('Magda', false) wasp = Person.new('Wasp', false) magneto.crushes << wasp magneto.hookups << rogue magneto.marriages << magda esme.crushes << magneto rogue.hookups << magneto magda.marriages << magnetoend

magneto = Person.find(:name => 'Magneto')

# Who likes Magneto?magneto.relationships.incoming(:crushes).nodes

# Which non-mutants has Magneto dated?magneto.hookups{ !mutant? }.to_a

Simulations

Structure

people{ ‘name’:‘Jimmy Olsen’ ‘title’:‘Superman’s Pal’ ‘company_id’:12441}

companies{ _id:12441 ‘name’:‘Daily Planet’}

Lack  of  Structure

mysql> SELECT * FROM people LIMIT 1 \G*************************** 1. row *************************** id: 1content: --- company: Daily Planetname: Jimmy Olsentitle: Superman’s Pal

But  wait!friendfeedfriendly

mysql> desc people;+-------+-------------+------+-----+---------+-------+| Field | Type | Null | Key | Default | Extra |+-------+-------------+------+-----+---------+-------+| id | int(11) | YES | | NULL | || name | varchar(50) | YES | | NULL | |+-------+-------------+------+-----+---------+-------+

mysql> desc attributes;+-----------+--------------+------+-----+---------+-------+| Field | Type | Null | Key | Default | Extra |+-----------+--------------+------+-----+---------+-------+| id | int(11) | YES | | NULL | || person_id | int(11) | YES | | NULL | || attribute | varchar(50) | YES | | NULL | || value | varchar(100) | YES | | NULL | |+-----------+--------------+------+-----+---------+-------+

Not  Only  SQL

Polyglot  Persistence

Caching

Already  in  Usememcached

Queues

Long-­‐running  processesresque

Logging

Rails  Log  Replacementhttp://github.com/peburrows/mongo_db_logger

Hybrid  Domains

different  domains

Publishinge-­‐commercedocuments

Datinge-­‐commercesocial  graph

different  scales

Photo  Sharinguser  accountsuploaded  photos

Next  Steps

Explore

Database  Listhttp://internetmindmap.com/database_software

NoSQL  Google  Grouphttp://groups.google.com/group/nosql-­‐discussion

Ignore  the  Database

Logical  Modeling  Firstbe  mindful

Change  the  Default

Application  Templatesstart  with  something  new

@bscoWieldben.scoWield@viget.comhttp://spkr8.com/bscoWieldhttp://viget.com/extendhttp:  //benscoWield.com

benscoWield