Distributed Hash Tables

David Tam

Patrick Pang

Presentation Outline• What is DHT (Distributed Hash Table)?

• Why DHTs?

• Applications

• How lookup works?

• Alternatives to DHTs

• Performance – Routing

• Performance – Load Balancing

• Security – Routing Attack

• Security – Inconsistent Behaviour

• Comparison to Other Facilities

• Current Research Projects

• Conclusion

What is DHT?

Distributed hash table

Distributed application

get (key) data

node node node….

put(key, data)

(Figure adopted from Frans Kaashoek)

DHT provides the information look up service for P2P applications.• Nodes uniformly distributed across key space• Nodes form an overlay network• Nodes maintain list of neighbours in routing table• Decoupled from physical network topology

Why DHTs?

Why Do We Need DHTs?• Simplifies the development for large-scale distributed Apps • Better security and robustness• Simple API• Exploits P2P resources

Why Middleware?• Simplifies the development for large-scale distributed Apps • Better security and robustness• Simple API

Applications

• Anything that requires a hash table• Databases, FSes, storage, archival• Web serving, caching• Content distribution• Query & indexing• Naming systems• Communication primitives• Chat services• Application-layer multi-casting• Event notification services• Publish/subscribe systems ?

How lookup works?

214

12

107

5

0

3

4

6

89

11

13start interval succ.

3 [3,4) 5

4 [4,6) 5

6 [6,10) 7

10 [10,2) 10

Finger Table for Node 215 1

Example: Chord [Stoica et. al.]

How lookup works?

214

12

107

5

0

3

4

6

89

11

13

15

start interval succ.

11 [11,12) 12

12 [12,14) 12

14 [14,2) 14

2 [2,10) 2

Finger Table for Node 101

Example: Chord

How lookup works?

214

12

107

5

0

3

4

6

89

11

13

15

start interval succ.

11 [11,12) 12

12 [12,14) 12

14 [14,2) 14

2 [2,10) 2

Finger Table for Node 101

Example: Chord

How lookup works?

1

214

12

107

5

0

3

4

6

89

11

13

15

start interval succ.

15 [15,0) 15

0 [0,2) 1

2 [2,6) 2

6 [6,13) 7

Finger Table for Node 14

Example: Chord

How lookup works?

1

214

12

107

5

0

3

4

6

89

11

13

15

start interval succ.

15 [15,0) 15

0 [0,2) 1

2 [2,6) 2

6 [6,13) 7

Finger Table for Node 14

Example: Chord

How lookup works?

214

12

107

5

0

3

4

6

89

11

15

Now Node 2 can retrive information for key 0 from Node 1.

1Example: Chord

Alternatives to DHTs• Distributed file system• Centralized lookup• P2P flooding queries

Server

Client

ClientClient

ClientInternet

Server (Figures adopted from Frans Kaashoek)

N4Target

Start

N6N9

N7N8

N3

N2

N1

N1

0

N4Target

Start

N6N9

N7

N8

N3

N2

N1

N10DB

Performance -- Lookup

Purpose -- to locate a target node•Each step, try to get closer to locating target node

• Ask a closer neighbour• Performance & scalability tied directly to lookup algorithm

2 Aspects to Scalability• size of routing table – O(log N)• lookup path length – O(log N)

2 Aspects to Performance• Path latency• Lookup path length (# hops)

3 Techniques• proximity lookup• proximity neighbour selection• geographic layout

Performance -- Load Balancing

Issues• Hot-spots

• Content• Lookup

• Heterogeneous nodes & paths• System flux

Solution• Replication is the key

• Also good for fault-tolerance• Cache lookup answers backwards along path

Security – Incorrect Lookup (1)• When asked for the “next hop”, give a wrong answer

start interval succ.

3 [3,4) 5

4 [4,6) 5

6 [6,10) 7

10 [10,2) 10

Finger Table for Node 2

Node 2 to Node 10: Please tell me how to reach key 0 ….

214

12

107

5

3

4

6

89

11

13

15 10

Security – Incorrect Lookup (2)• When asked for the “next hop”, give a wrong answer

start interval succ.

11 [11,12) 12

12 [12,14) 12

14 [14,2) 14

2 [2,10) 2

Finger Table for Node 10

214

12

107

5

0

3

4

6

89

11

13

15 1

Node 2 to Node 10: Please tell me how to reach key 0 ….

Node 10 answers: ask Node 14

• When asked for the “next hop”, give a wrong answer

start interval succ.

15 [15,0) 15

0 [0,2) 1

2 [2,6) 2

6 [6,13) 7

Finger Table for Node 14

214

12

107

5

3

4

6

89

11

13

15 1

Node 2 to Node 14: Please tell me how to reach key 0 ….

Node 14 answers: ask Node 10

Security – Incorrect Lookup (3)

0

Security – Incorrect Lookup (4)

Solution [Sit and Morris]:• “Define verifiable system invariant”• “Allow the querier to observe lookup progress”

Our idea how this can be implemented:• Concretely, using an integral monotonically decreasing quantity to implement the idea of “progress”.• The concept of “monotonically decreasing quantity” has been used in program construction guaranteeing total correctness. [Parnas]

Security – Inconsistent Behaviour• Inconsistent Behaviour, i.e., lie intelligibly

• Sybil attack [Kaashoek]

Solution 1: public key solution

Security – Inconsistent Behaviour• Inconsistent Behaviour, i.e., lie intelligibly

• Sybil attack [Kaashoek]

Solution 1: public key solution

Solution 2: Byzantine Protocol

Byzantine Generals Problem:

How to find out the traitors among the Generals? [Lamport]

• Inconsistent Behaviour, i.e., lie intelligibly

• Sybil attack [Kaashoek]

Security – Inconsistent Behaviour

Solution 1: public key solution

Solution 2: Byzantine Protocol

Byzantine Generals Problem:

How to find out the traitors among the Generals? [Lamport]

Commander

Lieutenant 1 Lieutenant 2“he said ‘retreat’”

attackattack

Security – Inconsistent Behaviour• Inconsistent Behaviour, i.e., lie intelligibly

• Sybil attack [Kaashoek]

Solution 1: public key solution

Solution 2: Byzantine Protocol

Byzantine Generals Problem:

How to find out the traitors among the Generals? [Lamport]

Commander

Lieutenant 1 Lieutenant 2

retreatattack

“he said ‘retreat’”

Comparison to Other Facilities

Facility Abstraction Easy Use/Prg Scalability Load-Balance

DHT high high high yes

Centralized Lookup medium medium low no

P2P flooding queries medium high low no

Distributed FS low medium medium no

Facility Fault-Tolerance Self-Org Admin

DHT high yes low

Centralized Lookup low no medium

P2P flooding queries depends yes low

Distributed FS medium no high

Research Projects

Iris – security & fault-tolerance – US Gov’tChord – circular key spacePastry – circular key spaceTapestry – hypercube spaceCAN – n-dimensional key spaceKelips – n-dimensional key spaceDDS -- middleware platform for internet service construction

-- cluster-based-- incremental scalability

Summary

• Good middleware platform

• Exploits P2P networks

• An exciting new research area

References

• Lamport, Leslie et. al. The Byzantine Generals Problem

• Sit, Emil, Morris, Robert. Security Considerations for Peer-to-Peer Distributed Hash Tables

• Kaashoek, Frans. Distributed Hash Tables – Building large-sacle, robust distributed applications

• Stoica, Ion et. al. Chord: A scalable peer-to-peer lookup service for Internet applications

• Parnas, D. L. Connecting Theory to Practice: Software Engineering Programme