RIAK 2.0Engin Yöyen

October, 2014

What is Riak

• Key/Value Store

• hashTable[‘D018319’] = ‘Influenza’

• diseases = hashTable[‘D018319’]

Buckets

• There is no global namespace.

• All keys must belong to a bucket.

• diseases[‘D018319’] = ‘Influenza’

• symptoms[‘D018319’] = ‘cervical pains’

Why Bucket Types• diseases[‘infectious’][‘D018319’] = ‘Influenza’

• diseases[‘blood-borne’][‘F023831’] = ‘Hepatitis B’

• Groups of buckets with a similar set of properties.

• diseases.props = {"search_index":"anyplace"}

The API

The API

via HTTP methods PUT, GET, POST, DELETE

Example

[hostAddress]/types/<type>/buckets/<bucket>/keys/<key>

PUTcurl -XPUT "http://localhost:8098/types/diseases/buckets/infectious/keys/D018319" \

-H "Content-Type:text/plain" \

-d "Influenza, commonly known as, the flu, ...."

riak-admin bucket-type create diseases '{"props":{"datatype":"set"}}'

riak-admin bucket-type activate diseases

PUT (2)

curl -v -XPUT "http://localhost:8098/types/diseases/buckets/infectious/keys/D018319" \

-H "Content-Type: application/json" \

-d '{"desc": "Influenza, commonly known as, the flu, …."}'

Hint : http://localhost:8098/types/diseases/buckets/infectious/keys/D018319?returnbody=true

GEThttp://localhost:8098/types/diseases/buckets/infectious/keys/D018319

Posthttp://localhost:8098/types/diseases/buckets/infectious/keys

Delete

I am sure you all figured out by now….

Hints

Hints - Buckets

http://localhost:8098/types/diseases/buckets?buckets=true

{

"buckets": [

"blood-borne",

"infectious"

]

}

Hints - Keyshttp://localhost:8098/types/diseases/buckets/infectious/keys?keys=true

{

"keys": [

"Uhz73mNfxkEkcmFsLcgu3Kan5Dw",

"D018319",

"WapIL5eeRbnXkocmcK3YK5EDjFV",

"9CkpJtAFziWuYSTfadW2rsf4bpF"

]

}

or you can stream it

http://localhost:8098/types/diseases/buckets/infectious/keys?keys=stream

Hints - Propertieshttp://localhost:8098/types/diseases/buckets/infectious/props

{

"props": {

"name": "infectious",

"w": "quorum",

"rw": "quorum",

"r": "quorum",

"pw": 0,

"precommit": [],

"pr": 0,

"postcommit": [],

"notfound_ok": true,

"n_val": 3,

"last_write_wins": false,

"dw": "quorum",

"dvv_enabled": true,

"chash_keyfun": {

"basic_quorum": false,

"allow_mult": true,

"datatype": "set",

"active": true,

"claimant": “riak@127.0.0.1”,

…

}

}

Backends

• Bitcask, eLevelDB, Memory or Multi

The Cluster

The Cluster

A cluster in Riak is a managed collection of nodes that share a common Ring.

The Ring

Figure : Riak Ring, (Redmond & Daily, 2014, P. 11)

Replication + Partitions

But how?

Figure : Replication Partitions, (Redmond & Daily, 2014, P. 10)

vnodes

ring_creation_size = 64

favorite = 75017a36ec07fd4c377a0d2a011400ab193e61db

n_val = 3

Vnodes, are responsible for claiming a partition in the Riak Ring, and they coordinate requests for these partitions.

Figure : Riak Ring, (Redmond & Daily, 2014, P. 11)

nodes & vnodes

Figure : Riak Ring, (Redmond & Daily, 2014, P. 12)

The CAP

The CAP• Consistency (all nodes see the same data at the same

time)

• Availability (a guarantee that every request receives a response about whether it was successful or failed)

• Partition tolerance (the system continues to operate despite arbitrary message loss or failure of part of the system)

CAP loosely states that you can have a C (consistent), A (available), or P (partition-tolerant) system, but you can only choose 2, meaning CP or AP.

Replication & Write & Read

• n_val=3

• w=all (consistent, high latency)

• r=all (consistent read, high latency)

• w=3,r=1(likely consistent, low latency)

per bucket basis

Figure : NRW, (Redmond & Daily, 2014, P. 14)

The Quorum

(floor(N/2) + 1)• nodes A,B,C,D,E,F,G,L,K,J (Don’t count it, it is 10)

“n_val” : "5" “w" : "quorum"

• w = floor(5/2)+1

• w=3

• PUT may respond successfully after writing to (A,B,C) and data will be eventually replicated to (D,E)

why?

What about Delete?

What about Delete?• Object are marked as deleted with a

marker(tombstone) and another process(reaper) will finish deleting the marked objects

• A delete is actually a read and a write

• Checking for the existence of a key(e.g. MapReduce,List) is not enough to know if an object exists.

• Check for tombstone metadata(X-Riak-Deleted=true)

A Delete Operation• Client makes the request

• Vnode : update vclock, store with tombstone

• delete_mode = keep, 3000 ms, immediate

• backend flags it as a tombstone (Bitcask or LevelDB) or wipes it immediately (in-memory)

all r/w/n_val parameters must be met

Data Types

Data TypesAllows RIAK to make sense of data(a data-aware system), so it can solve the conflicts.

• Bucket Level: counters, sets, maps

• In Maps : flags(enable, disable) and registers

With and Without DataTypes

[hostAddress]/types/<type>/buckets/<bucket>/keys/<key>

[hostAddress]/types/<type>/buckets/<bucket>/datatypes/<key>

Data Type : SetAdding symptoms to the Influenza

http://localhost:8098/types/diseases/buckets/infectious/datatypes/D018319

{"add_all":["Headaches", "Fatigue"]}

{"add_all":["Cough", "Sore throat"]}

{"remove": "Cough"}

Result : {["Headaches","Fatigue","Sore throat"]}

Data Type : MapAdding more than symptoms to the Influenza

http://localhost:8098/types/diseases/buckets/infectious/datatypes/D018319 { "update": { "symptoms_set": { "add_all": [ "Headaches", "Fatigue", "Sore throat" ] } }}

{ "update": { "name_register": "Influenza", "icd-9_register": "487" }}

Data Type : MapAdding and removing items from set { "update": { "symptoms_set": { "remove": "Headaches", "add": "Fever" } }}Maps within maps{ "update": { "virus_classification_map": { "update": { "influenzavirus_A_register": "Some Text", "influenzavirus_B_register": "Some Text", "influenzavirus_C_register": "Some Text" } } } }

Data Type : MapAnd result is { "type": "map", "value": { "virus_classification_map": { "influenzavirus_C_register": "Some Text", "influenzavirus_B_register": "Some Text", "influenzavirus_A_register": "Some Text" }, "symptoms_set": [ "Fatigue", "Fever", "Sore throat" ], "name_register": "Influenza", "icd-9_register": "487" }, "context": "g2wAAAABaAJtAAAACCMJ/vlUQtGHYQRq"}

Last but not least

• Search

• Security

• Hooks

• A Little Riak Book : http://littleriakbook.com/

Questions!

Reference

• Redmond,  E.,  &  Daily,  J.  (2014).  A  Little  Riak  Book.    • Riak  Docs.    http://docs.basho.com/riak/latest/