chord presentation at papers we love sf, august 2016
TRANSCRIPT
Chord: A scalable peer-to-peer lookup service for internet applications
Papers We Love SFAugust 2016
Chord is a completely peer-to-peer distributed key management system that works under dynamic membership churn.
Context
Idea 1: Consistent Hashing
Consistent Hashing
• Map keys to a hash m-bits long, e.g. SHA-1.
• Construct a ring with operations performedmod 2
m
Consistent Hashing
• Map keys to a hash m-bits long, e.g. SHA-1.
• Construct a ring with operations performed
• For example, take
• Gives us separate addresses.
m = 3
23 = 8
mod 2
m
Nodes and Keys
• Each node in the network has an address, typically
• We define , the successor of , defined as
• Key is stored at node
• Each node knows
• gives lookup performance, where is the number of nodes
addr = hash(ip)
succ(k)
min(n) | n � k mod 2
m
succ(k) O(N)N
k succ(k)
k
nn0
= succ(n+ 1 mod 2
m)
Idea 2: Finger Tables
Finger tables
• To move from to Chord uses a “finger table” to track nodes around the ring.
• Fundamental insight- dense information nearby, - sparse information far away
• Table defined by:
• Also track
O(n) O(log n)
finger[k].start = (n+ 2
k�1) mod 2
m, 1 k m
.interval = [finger[k].start, finger[k + 1].start)
.node = succ(finger[k].start)
successor = finger[1].node
predecessor
Example Layoutm = 62m = 64
Node Locationα 7β 16γ 42δ 44ε 50ζ 52η 3θ 4
This table does not exist!
The View from α
k start end n1 8 8 β2 9 10 β3 11 14 β4 15 22 β5 23 38 γ6 39 7 γ
Starting from α, retrieve key 51
First step, ask γ
The View from γ
k start end n1 43 43 δ2 44 45 δ3 46 49 ε4 50 57 ε5 58 9 η6 10 42 β
Second step, ask ε
The View from ε
k start end n1 51 51 ζ2 52 53 ζ3 54 57 η4 58 1 η5 2 17 η6 18 50 γ
Third step, ask ζ
At this point, so ζ will have the key
succ(51) = ⇣
Idea 3: Handling Churn
Joining the network
Once a node has assigned itself an id, , it does 3 things:
1. Builds its finger table and predecessor
n0k start n123
4
5
… … …
n0 + 1 succ(n0 + 1)
n0 + 2
n0 + 4
n0 + 8
n0 + 16 succ(n0 + 16)
succ(n0 + 2)
succ(n0 + 4)
succ(n0 + 8)
Joining the network
Once a node has assigned itself an id, , it does 3 things:
1. Builds its finger table and predecessor
2. Updates other nodes that should have their finger tables point to
3. Notify upper layers of software that they need to move keys.
n0
n0
Joining the network
Once a node has assigned itself an id, , it does 3 things:
1. Builds its finger table and predecessor
2. Updates other nodes that should have their finger tables point to
3. Notify upper layers of software that they need to move keys.
n0
n0
Joins take messages
keys will be moved
O(log2n)
O(1
N)
Concurrency & Failure
Two basic mechanisms:
1. Every node periodically performs stabilization
2. Each node maintains a successor list rather than a single successor
When a node fails, it’s keys are lost. Other mechanisms are used by higher levels to build resiliency, e.g. republishing or replication.
Related Work
Related Work
• Pastry
• CAN
• Kademlia
• Tapestry
Impact
Impact
• Research applications in domains such as distributed file systems, pub-sub, document sharing, search algorithms.
• Basis for sharing data to nodes in systems like Cassandra without requiring a global index.
The End!