Linearizing Peer-to-Peer Systems with Oracles

by

Rizal Mohd NorMikhail NesterenkoSébastien Tixeuil

SSS 2013Nov 13-16, 2013

2

SSS 2013

Introduction

Fundamental limits on construction of structured peer-to-peer systems in asynchronous [1]

§ inability to connect a disconnected network§ discard peer identifiers that are not present

in the system

Studying these limits§ identify peer-to-peer system specific

oracles and isolate the source of impossibility

§ show minimality of oracles and proving their necessity for solution existence

§ provide an oracle-based algorithm

Why Linearization (topological sort)§ foundational for most popular peer-to-peer

systems as construction starts with topologically sorting the peers in the system

§ similar to consensus as one can observe how it pertain to all peer-to-peer systems

Rizal Mohd Nor, Mikhail Nesterenko, and Christian Scheideler. Corona: A stabilizing deterministic message-passing skip list. In 13th. International Symposium on Stabilization, Safety and security of Distributed Systems (SSS) pages 356-370, October 2011

[1]

3

SSS 2013

Outline

linearization problem

solution oracles

necessary conditions

linearization solutions

execution example

oracle implementation

4

SSS 2013

Linearization Problem

Strict (SL) and eventual (EL) linearization variants SL requires each process to output its neighbors exactly once and allows only

correct output EL’s computation contains a suffix where the output of each process is correct

Existing (EID) and non-existing (NID) identifiers within the linearization problem EID prohibits the existence of non-existing identifiers NID allows non-existing identifiers

5

SSS 2013

Outline

linearization problem

solution oracles

necessary conditions

linearization solutions

execution example

oracle implementation

6

SSS 2013

Solution Oracles

weak connectivity oracle (WC) has a single action that: selects a pair of processes p and q such that they are disconnected in the channel

connectivity graph CC, then connecting graph by adding q to the incoming channel of p creating a link (p,q)

participant detector (PD) oracle removes a non-existent identifier stored in p.

neighbor output oracle (NO) just output identifiers stored in left and right variables of p

consequent process detector (CD) outputs the stored identifier only if it is consequent with p the guard of CD mentions all the identifiers of the system

7

SSS 2013

Outline

linearization problem

solution oracles

necessary conditions

linearization solutions

execution example

oracle implementation

8

SSS 2013

Necessary Conditions: Weak Connectivity Oracle

Theorem 1. Every solution to the linearization problem requires a weak connectivity oracle.

For a linearization algorithm A, p1 has to eventually output p2

p1 p2 p3 p4 p5

p1 will eventually add p2, if the computation contains the actions of a weak connectivity oracle. Without WC, it will remain disconnected.

9

SSS 2013

Subset Splittable

subset splittable prevents a subset of processes from learning the state of the rest of the system denoted as SS (subset spilttable) and NSS (non-subset spilttable)

neighborhood output oracle (NO) is subset spilttable (SS) subset splittable since its guards only mention local variables

participant detector oracle (PD) is subset spilttable (SS)

a b nid d e

Consider 2 sets of non-intersecting processes, S1 and S2

S1 S2 If PD is enabled at b, it would also be enabled at state

10

SSS 2013

Subset Splittable

weak connectivity oracle (WC) is subset spilttable (SS)

a b c d e

Consider 2 sets of non-intersecting processes, S1 and S2, where c and d are disconnected

S1 S2 If WC is enabled at c, connecting c to d, it would also be enabled at state c

11

SSS 2013

Non Subset Splittable

consequent process detector (CD) is not subset spilttable (NSS)

Consider 2 sets of non-intersecting processes, S1 and S2 are totally ordered

a c b d e

S1 S2

If a process has a consequent process, it is enabled. If b is added to set S1, c would be enabled. This disables CD enabled in the previous state.

12

SSS 2013

Necessary Conditions: Non-subset Splittable Oracle

Theorem 2. A solution to the strict linearization problem requires a non-subset splittable oracle.

Consider 2 sets of non-intersecting processes, S1 and S2

a b c d e

S1 S2

13

SSS 2013

Necessary Conditions: Non-subset Splittable Oracle

Theorem 2. A solution to the strict linearization problem requires a non-subset splittable oracle.

a b c d e

Since strict linearization requires each process to output only once, in this case, b or c cannot output without knowing the state of the rest of the system

14

SSS 2013

Necessary Conditions: NID requires PD oracles

Theorem 3. A proper solution to the linearization problem that allows non-existing identifiers requires a participant detector oracle.

p1 p2 p3 p4 p5

Without PD, a process p1 may be connected to a non-existing identifier and result in a graph to be disconnected from the higher-id processes forever

PD is required to remove the right id of process p1 and left id of process p2.

15

SSS 2013

Outline

oracles

linearization problem

solution oracles

necessary conditions

linearization solutions

execution example

oracle implementation

16

SSS 2013

Necessary Oracles

The necessary oracles to solve the 4 variants of the linearization problem.

Eventual Linearization (EL)

Strict Linearization (SL)

Existing IDs (EID) WC WC+NSS

Non-existing IDs (NID) WC+PD WC+PD+NSS

Eventual Linearization (EL)

Strict Linearization

(SL)Existing IDs

(EID) L+WC+NO L+WC+CD

Non-existing IDs (NID) L+WC+NO+PD L+WC+CD+PD

Linearization algorithm L with a combination of oracles to solve the linearization problem

17

SSS 2013

Outline

oracles

linearization problem

solution oracles

necessary conditions

linearization solutions

execution example

oracle implementation

18

SSS 2013

EL+EID Execution Example using L+WC

The linearization algorithm L, linearizes a list of peers actions

receive ID from right → set closer right neighbor or forward to right neighbor receive ID from left → set closer left neighbor or forward to left neighbor true (timeout) → send ID to right and left neighbor

WC Oracle actions graph contains disconnected

components of and such that send to

Starting from an arbitrary state where processes are weakly connected except process a

a b c d e

19

SSS 2013

EL+EID Execution Example using L+WC

The linearization algorithm L, linearizes a list of peers actions

receive ID from right → set closer right neighbor or forward to right neighbor receive ID from left → set closer left neighbor or forward to left neighbor true (timeout) → send ID to right and left neighbor

WC Oracle actions graph contains disconnected

components of and such that send to 𝑒∈𝐶 2

a b c d e

𝑎∈𝐶 1

20

SSS 2013

EL+EID Execution Example using L+WC

Since and disconnected, WC is enabled at process a

WC Oracle actions graph contains disconnected

components of and such that send to

a b c d e

a

𝑒∈𝐶 2

𝑎∈𝐶 1

Node a WC oracle is enabled and node a sends its ID to node e

21

SSS 2013

EL+EID Execution Example using L+WC

§ starting from this state, processes are weakly connected and WC is never enabled

§ the only actions enabled are the actions of the linearization algorithm L

§ actions§ receive ID from right → set closer right neighbor or

forward to right neighbor§ receive ID from left → set closer left neighbor or

forward to left neighbor§ true (timeout) → send ID to right and left neighbor

a b c d ea

Node e receive a's ID, forward to c

22

SSS 2013

Node c receive a's ID, forward to b

EL+EID Execution Example using L+WC

§ the only actions enabled are the actions of the linearization algorithm L

§ actions§ receive ID from right → set closer right neighbor or

forward to right neighbor§ receive ID from left → set closer left neighbor or

forward to left neighbor§ true (timeout) → send ID to right and left neighbor

a b c d e

a

23

SSS 2013

Node b receive a's ID, set its new left neighbor

EL+EID Execution Example using L+WC

§ the only actions enabled are the actions of the linearization algorithm L

§ actions§ receive ID from right → set closer right neighbor or

forward to right neighbor§ receive ID from left → set closer left neighbor or

forward to left neighbor§ true (timeout) → send ID to right and left neighbor

a b c d e

24

SSS 2013

Node b sends it's ID, to Node a

EL+EID Execution Example using L+WC

§ the only actions enabled are the actions of the linearization algorithm L

§ actions§ receive ID from right → set closer right neighbor or

forward to right neighbor§ receive ID from left → set closer left neighbor or

forward to left neighbor§ true (timeout) → send ID to right and left neighbor

a b c d e

b

25

SSS 2013

Node a receive b's ID, set its new right neighbor

EL+EID Execution Example using L+WC

§ the only actions enabled are the actions of the linearization algorithm L

§ actions§ receive ID from right → set closer right neighbor or

forward to right neighbor§ receive ID from left → set closer left neighbor or

forward to left neighbor§ true (timeout) → send ID to right and left neighbor

a b c d e

26

SSS 2013

EL+EID Execution Example using L+WC

processes take independent actionsc1.

a b c d e

27

SSS 2013

EL+EID Execution Example using L+WC

processes take independent actions1.

2.

a b c d e

c

a b c d e

d

28

SSS 2013

EL+EID Execution Example using L+WC

processes take independent actions1.

2.

3.

a b c d e

c

a b c d e

d

a b c d e

d

29

SSS 2013

EL+EID Execution Example using L+WC

processes take independent actions1.

2.

3.

4.

a b c d e

c

a b c d e

d

a b c d e

d

a b c d e

30

SSS 2013

EL+EID Execution Example using L+WC

processes take independent actionse

1.

2.

3.

4.

5.

a b c d e

c

a b c d e

d

a b c d e

d

a b c d e

a b c d e

31

SSS 2013

EL+EID Execution Example using L+WC

processes take independent actionse

1.

2.

3.

4.

5.

a b c d e

c

a b c d e

d

a b c d e

d

a b c d e

a b c d e

6.e

a b c d e

32

SSS 2013

EL+EID Execution Example using L+WC

processes take independent actionse

1.

2.

3.

4.

5.

a b c d e

c

a b c d e

d

a b c d e

d

a b c d e

a b c d e

6.e

a b c d e

7. e

a b c d e

33

SSS 2013

Outline

oracles

linearization problem

solution oracles

necessary conditions

linearization solutions

execution example

oracle implementation

34

SSS 2013

Oracle Implementation Example: WC

WC repairs the network disconnections, an encapsulation of bootstrap service commonly found in peer-to-peer systems

Example: One bootstrap process b is always present in the system the responsibility of this process is to maintain the greatest and smallest identifier

of the system if process p is disconnected, it does not have a left or right neighbor, it assumes

that its own identifier is the greatest or, respectively, smallest process p then sends its identifier to b process b then either confirms this assumption or sends p, its current smallest or

greatest identifier weak connectivity is restored

b

qp

A disconnected peer will contact the bootstrap service to get smallest or greatest identifier

q

35

SSS 2013

Oracle Implementation Example: PD

PD encapsulates the limits between relative process speeds and maximum message propagation delay and can be implemented using a heartbeat protocol

Example: if process p contains an identifier q, p sends q a heartbeat message requesting a

reply if p does not receive this reply after the time above the maximum network delay, p

considers q non-existent and discards it

p q

heartbeatNo reply from q

Discard left link

36

SSS 2013

Future Research

Narrowing the gap between necessary and sufficient

Strict linearization solution relies on CD, which is a specific kind of the necessary non-subset splittable detector

Narrowing the gap between necessary and sufficient conditions for the solution to the strict linearizability problem remains to be addressed in future research.

Thank you. Any Questions.