1 the design of a robust peer-to-peer system rodrigo rodrigues, barbara liskov, liuba shrira...
TRANSCRIPT
1
The Design of a RobustPeer-to-Peer System
Rodrigo Rodrigues, Barbara Liskov, Liuba Shrira
Presented by Yi Chen
Some slides are borrowed from the authors’
2
Talk Outline
Existing P2P systems Motivation for a robust P2P system The new P2P architecture Conclusion and future work Discussions
3
Existing P2P Systems
Started with explosion of file sharing apps Existing P2P systems:
• All nodes have identical responsibilities • All communication is symmetric• Volunteer nodes join and leave system at any time
Advantages:• Harness idle storage and network resources• Automatic load balancing • Self-organization• Desirable properties scale as we add nodes
4
Motivation: Support More Applications
Current applications that use P2P storage systems• Assume states are in secure place• Use untrusted P2P storage system to publish data
E.g. content sharing
What if we want to store the state of the application in a P2P system (e.g. support writes)?
E.g. digital library archiving, mail service
5
Motivation: Better Fault-Tolerance
Current P2P systems tolerate failstop failures• Faulty nodes stop
Current P2P systems does not handle Byzantine failures• faulty node may behave arbitrary and malicious
e.g. an attacker can join the system with multiple Ids, provide fake information
6
Motivation: Efficient Lookups
Current approach: Chord Every node maintains
a routing table about O(logN) entries
Routing algorithm locates data with latency O(logN),
N8
N51
N42
N1
N38
N32N21
N14
LookupK36
K36 N8+1: N14N8+2: N14
N8+8: N21N8+4: N14
N8+16: N32
N8+32: N42
7
Talk Outline
Existing P2P systems Motivation for a robust P2P system The new P2P architecture
• Configuration Service• P2P nodes
Conclusion and future work Discussions
8
Architecture
Configuration Service (CS): • can run on subset of P2P nodes or be separate from P2P nodes• determines current configuration and propagate the information
P2P nodes (server machines)
ConfigurationService P2P Nodes
9
Configuration Service
Four main functions:• Admission control• Node monitoring• Deciding on a new configuration• Propagating configuration information
10
Admission Control
Prevents a single user controlling large fraction of the ID space
Hard to do in volunteer-based system Approach:
• Maintain a list of authorities who are permitted to add nodes to the system
• Each new nodes inform CS its ID, IP and public key
11
Node Monitoring
How to determine a node should be evicted? Failstop: ping nodes periodically Byzantine: hard!
• Probing: must be indistinguishable from client requests
• Still, faulty nodes may lie in wait• Rely on proactive recovery
12
Proactive Recovery
All nodes in the system are recovered proactively and frequently.
Approach:• Each node has a watchdog timer to trigger recovery• During a recovery, a node reboots from the saved
correct state. Then, in order to bring the states up-to-date, it sends status message to replicas, who detect missing messages (if any) and retransmits them.
13
Proactive Recovery (cont.)
To ensure correctness, choose recovery frequency, such that:
There are (3f+1) the number of replicas for a data item. For any moment, there are at most f faulty replicas (see algorithm in [2])
Byzantine faulty nodes may only present temporarily in the system
14
Assumptions of the model
To apply proactive recovery, P2P nodes are server machines, such that• less failure-prone• dedicated to handle distributed applications. not
constantly joining and leaving the system, and routing states changes infrequently
• still can run symmetric protocols, and distributed across a wide area
15
Deciding on a New Configuration
Include newly joined nodes Evict potentially faulty nodes
• Byzantine faulty nodes only present temporarily, therefore do need to detect and remove them.
• Remove potential failstop nodes that are not reachable for certain time period
Produce new configuration periodicallye.g. once every hour
16
Propagating Configuration
What to propagate? Disseminate entire config• Assumption: few nodes joining/leaving• Advantage: enable efficient lookup• Overhead:
• Local storage of configuration information
100,000 nodes -> 14.7 MB• Transmit using diffs and compressions
How to propagate? Will be discussed later
17
Talk Outline
Existing P2P systems Motivation for a robust P2P system The new P2P architecture
• Configuration Service• P2P nodes
Conclusion and future work Discussions
18
Two Layers on P2P Nodes
Lookup layer:
Receive/maintain configuration info
Storage layer:
Store/retrieve/transfer data items• Upon configuration changes, need state transfer• During lookup, ask lookup layer for the replicas that
hold the data of interest, then contact the storage layer at the replicas directly
19
Two Types of Data
Immutable, self-verifying data.Use the hash of its content as ID
Mutable data• Extend the data with a monotonically increasing
version number
• Retrieval needs to make sure to be the latest version
20
Data Management of Storage Layer
Storage/Retrieval with static configuration• Immutable data• Mutable data
Storage/Retrieval during re-configuration• State transfer• Configuration propagation
21
Storage/Retrieval with Static Configuration
Need to ensure that 2f+1 replicas claim to have stored the data
Assume no concurrent writes to the same data items
22
Immutable Data
Adversary cannot forge them Not updateable – never out-of-date Write to and receive acknowledgement from at
least 2f + 1 replicas Read from only one replica, if the content hash
matches the ID
23
Mutable data
Replay attack Must include version number Write to 2f+1 replicas
• One round-trip if version number is known• Otherwise learn current version number by an extra
round-trip
Read from 2f+1 replicas and choose the one with highest version number to guarantee correctness
24
Store / Retrieve Algorithms
Write2f+1 replicas
Read2f+1 replicas
In total n=3f+1 replicas At any moment, at most f faulty replicas Therefore read at least 1 correct replica
25
Storage/Retrieval during Re-configuration
When configuration changes, the set of replicas of a data item may change
Need to • State transfer
Bring the up-to-date version of data to P2P nodes that are responsible for them
• Configuration propagation
Propagate new configuration to all nodes over some time
26
State Transfer
If a node receives a new configuration, and realizes it is responsible for new data items
Then it pulls the up-to-date data from replicas of last configuration• Immutable data: read one replica if the content hash
matches id• Mutable data: read 2f+1 replicas to ensure
correctness
27
Configuration Propagation
Nodes include id of the latest configuration they know in the messages they exchange
If a node detects a new configuration, it requests for a copy of that configuration
28
Talk Outline
Existing P2P systems Motivation for a robust P2P system The new P2P architecture Conclusion and future work Discussions
29
Conclusions
A hybrid system consists of a set of servers as P2P nodes, and a CS.
P2P nodes are sever machines that dedicated for the storage application
CS are responsible for computing and propagating the current configuration
A Byzantine fault-tolerant system by applying proactive recovery
30
Future Work
Implementation Improve algorithms – delete, quotas, … Eliminate rebooting requirement
31
Discussions
Compare its underlying assumptions with existing P2P systems
Proposed improvements over existing P2P systems• Support more applications • Better fault-tolerance• Efficient lookups
32
Discussion: Assumptions
Compare its underlying assumptions with existing P2P systems• All nodes have identical responsibilities? Yes • All communication is symmetric? Yes• Volunteer nodes join / leave system at any time? No
• Assume participating nodes are dedicated to handle distributed applications, and do not frequently join/leave
• Node join under admission control
Similar as a distributed system?
33
Discussion: Support More Applications
Current P2P systems use P2P storage to publish data, only useful for content sharing
This system supports state changes of the data (writes), useful for archiving
Read/write 2f+1 replicas, big overhead?
34
Discussion: Better fault-tolerance
Current P2P systems: failstop fault-tolerant
This system: Byzantine fault-tolerant• By applying proactive recovery • Only suitable for server machines
35
Discussion: Efficient Lookups
Existing P2P systems:A node is responsible for the routing states of a subset
of the nodes. Routing requires several lookups
This system: Every node stores the whole configuration, enables
direct lookups• Pros: efficient lookups• Cons: big storage; big transfer; many routing state
changes when nodes join/leave
36
Questions?
Thank you!