milan vojnović microsoft research cambridge collaborators: e. perron and d. vasudevan 1 consensus...
TRANSCRIPT
![Page 1: Milan Vojnović Microsoft Research Cambridge Collaborators: E. Perron and D. Vasudevan 1 Consensus – with Limited Processing and Signalling](https://reader033.vdocuments.mx/reader033/viewer/2022051613/55147592550346284e8b6282/html5/thumbnails/1.jpg)
Milan VojnovićMicrosoft Research Cambridge
Collaborators: E. Perron and D. Vasudevan
1 Consensus – with Limited Processing and Signalling
![Page 2: Milan Vojnović Microsoft Research Cambridge Collaborators: E. Perron and D. Vasudevan 1 Consensus – with Limited Processing and Signalling](https://reader033.vdocuments.mx/reader033/viewer/2022051613/55147592550346284e8b6282/html5/thumbnails/2.jpg)
This Talk Based on
MSR Technical Report – MSR-TR-2008-114 – Aug 2008
2
![Page 3: Milan Vojnović Microsoft Research Cambridge Collaborators: E. Perron and D. Vasudevan 1 Consensus – with Limited Processing and Signalling](https://reader033.vdocuments.mx/reader033/viewer/2022051613/55147592550346284e8b6282/html5/thumbnails/3.jpg)
Binary Consensus Problem
0
1
0
11
1
10
0
Goal: each node wants to correctly decide whether 0 or 1 was initially held by majority of nodes
3
![Page 4: Milan Vojnović Microsoft Research Cambridge Collaborators: E. Perron and D. Vasudevan 1 Consensus – with Limited Processing and Signalling](https://reader033.vdocuments.mx/reader033/viewer/2022051613/55147592550346284e8b6282/html5/thumbnails/4.jpg)
Consensus Problem (Cont’d)
1
1
1
11
1
11
1
Correct decision
4
![Page 5: Milan Vojnović Microsoft Research Cambridge Collaborators: E. Perron and D. Vasudevan 1 Consensus – with Limited Processing and Signalling](https://reader033.vdocuments.mx/reader033/viewer/2022051613/55147592550346284e8b6282/html5/thumbnails/5.jpg)
Consensus Problem (Cont’d)
0
0
0
00
0
00
0
Incorrect decision
5
![Page 6: Milan Vojnović Microsoft Research Cambridge Collaborators: E. Perron and D. Vasudevan 1 Consensus – with Limited Processing and Signalling](https://reader033.vdocuments.mx/reader033/viewer/2022051613/55147592550346284e8b6282/html5/thumbnails/6.jpg)
Applications
0
0
0
1
1
1
10
0
Ex. Opinion formation in social networks
6
![Page 7: Milan Vojnović Microsoft Research Cambridge Collaborators: E. Perron and D. Vasudevan 1 Consensus – with Limited Processing and Signalling](https://reader033.vdocuments.mx/reader033/viewer/2022051613/55147592550346284e8b6282/html5/thumbnails/7.jpg)
Applications (Cont’d)
01101
Ex. Distributed databases Top-k query processing
Query: Is object X most preferred by majority of nodes?
7
![Page 8: Milan Vojnović Microsoft Research Cambridge Collaborators: E. Perron and D. Vasudevan 1 Consensus – with Limited Processing and Signalling](https://reader033.vdocuments.mx/reader033/viewer/2022051613/55147592550346284e8b6282/html5/thumbnails/8.jpg)
Notation
0
0
0
11
1
10
0
8
![Page 9: Milan Vojnović Microsoft Research Cambridge Collaborators: E. Perron and D. Vasudevan 1 Consensus – with Limited Processing and Signalling](https://reader033.vdocuments.mx/reader033/viewer/2022051613/55147592550346284e8b6282/html5/thumbnails/9.jpg)
Notation (Cont’d)
1
01
0
0
0
0
1
1
9
![Page 10: Milan Vojnović Microsoft Research Cambridge Collaborators: E. Perron and D. Vasudevan 1 Consensus – with Limited Processing and Signalling](https://reader033.vdocuments.mx/reader033/viewer/2022051613/55147592550346284e8b6282/html5/thumbnails/10.jpg)
System’s Desiderata
Reach correct consensus – initial majority
Fast convergence
Small communication overhead
Small processing per node
Decentralized
10
![Page 11: Milan Vojnović Microsoft Research Cambridge Collaborators: E. Perron and D. Vasudevan 1 Consensus – with Limited Processing and Signalling](https://reader033.vdocuments.mx/reader033/viewer/2022051613/55147592550346284e8b6282/html5/thumbnails/11.jpg)
Related Work – Classical Voter Model
Node takes over the state of the contacted node
Binary state per node & binary signaling
0 initially held by V nodes,1 initially held by U nodes
Complete graph node interactionsProbability of incorrect consensus
UVVU
Uf VU
for ,,
1
0
0
0
1
0
1
1
11
![Page 12: Milan Vojnović Microsoft Research Cambridge Collaborators: E. Perron and D. Vasudevan 1 Consensus – with Limited Processing and Signalling](https://reader033.vdocuments.mx/reader033/viewer/2022051613/55147592550346284e8b6282/html5/thumbnails/12.jpg)
Related Work – m-ary Hypothesis Testing
Q: How much state does S need to decide correct hypothesis with probability going to 1 with the number of observations ?
1,,0 ),,[ : 1 miaaH iii
12
000110111110100011
Hi
i. i. d. mean S
00 a 1ma1a
A: m+1 necessary and sufficient (Koplowitz, IEEE Trans IT ’75)
![Page 13: Milan Vojnović Microsoft Research Cambridge Collaborators: E. Perron and D. Vasudevan 1 Consensus – with Limited Processing and Signalling](https://reader033.vdocuments.mx/reader033/viewer/2022051613/55147592550346284e8b6282/html5/thumbnails/13.jpg)
Ternary Protocol
Both processing and signaling take one of three states 0 or 1 or e e = “indecisive” state
1
0
e
0
0
0
e
0
e
1
1
1
13
![Page 14: Milan Vojnović Microsoft Research Cambridge Collaborators: E. Perron and D. Vasudevan 1 Consensus – with Limited Processing and Signalling](https://reader033.vdocuments.mx/reader033/viewer/2022051613/55147592550346284e8b6282/html5/thumbnails/14.jpg)
Binary Protocol
Processing same as for ternary protocol Binary signaling – takes one of two states 0 or 1
e e
signals 0 or 1 with equal probability
14
![Page 15: Milan Vojnović Microsoft Research Cambridge Collaborators: E. Perron and D. Vasudevan 1 Consensus – with Limited Processing and Signalling](https://reader033.vdocuments.mx/reader033/viewer/2022051613/55147592550346284e8b6282/html5/thumbnails/15.jpg)
Binary Signaling – A Motivation
Nodes may not be able to signal indifference – by the very nature of the application
Ex. two news pieces may be equally most read but only one can be recommended to the user
15
US navy ship stems into port where Russian...
US navy ship stems into port where Russian...
Soldier forced to sleep in car after hotel...
![Page 16: Milan Vojnović Microsoft Research Cambridge Collaborators: E. Perron and D. Vasudevan 1 Consensus – with Limited Processing and Signalling](https://reader033.vdocuments.mx/reader033/viewer/2022051613/55147592550346284e8b6282/html5/thumbnails/16.jpg)
Questions of Interest
Probability of convergence to incorrect consensus ?
Time to reach consensus ?
Dependence on the number of nodes N and initial fraction of nodes holding the majority state ?
16
![Page 17: Milan Vojnović Microsoft Research Cambridge Collaborators: E. Perron and D. Vasudevan 1 Consensus – with Limited Processing and Signalling](https://reader033.vdocuments.mx/reader033/viewer/2022051613/55147592550346284e8b6282/html5/thumbnails/17.jpg)
This Talk Assumptions
Complete graph node interactions Each node samples a node uniformly at random
across all nodes at instances of a Poisson process with intensity 1
Arbitrary graph interactions of interest – for future work
17
![Page 18: Milan Vojnović Microsoft Research Cambridge Collaborators: E. Perron and D. Vasudevan 1 Consensus – with Limited Processing and Signalling](https://reader033.vdocuments.mx/reader033/viewer/2022051613/55147592550346284e8b6282/html5/thumbnails/18.jpg)
Summary of Results – Talk Outline
Ternary protocol Prob of error decays exponentially with the
number of nodes N – found exact exponent log(N) convergence time
Binary protocol Prob of error worse than for ternary protocol
for a factor exponentially increasing with N, but not worse than for classical voter
Convergence time C log(N) with 2 C 3
18
![Page 19: Milan Vojnović Microsoft Research Cambridge Collaborators: E. Perron and D. Vasudevan 1 Consensus – with Limited Processing and Signalling](https://reader033.vdocuments.mx/reader033/viewer/2022051613/55147592550346284e8b6282/html5/thumbnails/19.jpg)
Ternary Protocol - Dynamics
U = number of nodes in state 0 V = number of nodes in state 1 N = total number of nodes
19
N
UVVU
N
VVUNVU
N
VUVU
N
UVUNVU
VU
: )1,(
)(: )1,(
: ),1(
)(: ),1(
),(
(U,V) Markov process:
![Page 20: Milan Vojnović Microsoft Research Cambridge Collaborators: E. Perron and D. Vasudevan 1 Consensus – with Limited Processing and Signalling](https://reader033.vdocuments.mx/reader033/viewer/2022051613/55147592550346284e8b6282/html5/thumbnails/20.jpg)
Ternary Protocol - Probability of Error
Theorem – probability of error:
U
jjVjU
VUVU
jaf
1)()(
,, 2
)(
2
1
jU
jVjU
jVjU
UVja VU
)()(
)()()(,
(U, V) = initial point, V > U
20
![Page 21: Milan Vojnović Microsoft Research Cambridge Collaborators: E. Perron and D. Vasudevan 1 Consensus – with Limited Processing and Signalling](https://reader033.vdocuments.mx/reader033/viewer/2022051613/55147592550346284e8b6282/html5/thumbnails/21.jpg)
Proof Outline
First-step analysis:
with
Boundary conditions:
1,1,,1,1,)2( VUVUVUVUVU UVfaVfUVfaUffUVaVaU
VUNa
0 for 10 for ,0 0,,0 U, fVf UV
21
![Page 22: Milan Vojnović Microsoft Research Cambridge Collaborators: E. Perron and D. Vasudevan 1 Consensus – with Limited Processing and Signalling](https://reader033.vdocuments.mx/reader033/viewer/2022051613/55147592550346284e8b6282/html5/thumbnails/22.jpg)
Proof Outline (Cont’d)
Lemma – solution of
Boundary conditions:
VUVUVU fff ,11,, 2
1
2
1
VUf ,
0 for 10 for ,0 0,,0 U, fVf UV
22
VUf ,
}0{
}0{
12
1:),1(
12
1:)1,(
),(
U
V
VU
VUVU
i.e. is error probability of
![Page 23: Milan Vojnović Microsoft Research Cambridge Collaborators: E. Perron and D. Vasudevan 1 Consensus – with Limited Processing and Signalling](https://reader033.vdocuments.mx/reader033/viewer/2022051613/55147592550346284e8b6282/html5/thumbnails/23.jpg)
Proof Outline (Cont’d)23
U
VfU,U = 1/2
(U, V)
(j, j)
U
jjjVjUVU nf
1)()(, 2
1
Number of pathsfrom (U, V) to (j, j) that do not intersect the line U = V-- Ballot theorem
![Page 24: Milan Vojnović Microsoft Research Cambridge Collaborators: E. Perron and D. Vasudevan 1 Consensus – with Limited Processing and Signalling](https://reader033.vdocuments.mx/reader033/viewer/2022051613/55147592550346284e8b6282/html5/thumbnails/24.jpg)
Probability of Error (Cont’d)
Corollary – For
H() = entropy of a Bernoulli random variable with mean
Ob. Exponential decay for large N.
NHfN VU large )],(1[~)log(1
,
1 1/2 ),,1(/))0(),0(( NVU
24
![Page 25: Milan Vojnović Microsoft Research Cambridge Collaborators: E. Perron and D. Vasudevan 1 Consensus – with Limited Processing and Signalling](https://reader033.vdocuments.mx/reader033/viewer/2022051613/55147592550346284e8b6282/html5/thumbnails/25.jpg)
Convergence Time
Initial state:
Limit ODE:
Time:
)0()0(
))0()0((log
)()(
))()((log
33
vu
uv
tvtu
tutvt
))(2)(1)(()(
))(2)(1)(()(
tutvtvtvdt
d
tvtututudt
d
))0(),0((/))0(),0(( vuNVU
25
![Page 26: Milan Vojnović Microsoft Research Cambridge Collaborators: E. Perron and D. Vasudevan 1 Consensus – with Limited Processing and Signalling](https://reader033.vdocuments.mx/reader033/viewer/2022051613/55147592550346284e8b6282/html5/thumbnails/26.jpg)
Convergence Time (Cont’d)
NNNt large ),log(~)(
26
Time it takes for (u(t), v(t)) to go from (u(0), v(0)) to (u(t), v(t)) such that 1-v(t) is of order 1/N
![Page 27: Milan Vojnović Microsoft Research Cambridge Collaborators: E. Perron and D. Vasudevan 1 Consensus – with Limited Processing and Signalling](https://reader033.vdocuments.mx/reader033/viewer/2022051613/55147592550346284e8b6282/html5/thumbnails/27.jpg)
Binary Protocol – Reminder
Processing same as for ternary protocol Binary signaling – takes one of two states 0 or 1
e e
signals 0 or 1 with equal probability
27
![Page 28: Milan Vojnović Microsoft Research Cambridge Collaborators: E. Perron and D. Vasudevan 1 Consensus – with Limited Processing and Signalling](https://reader033.vdocuments.mx/reader033/viewer/2022051613/55147592550346284e8b6282/html5/thumbnails/28.jpg)
Binary Protocol – Dynamics
(U,V) Markov process:
N
VUVVU
N
UVVUNVU
N
UVUVU
N
VUVUNVU
VU
12
1: )1,(
1)(2
1: )1,(
12
1: ),1(
1)(2
1: ),1(
),(
28
![Page 29: Milan Vojnović Microsoft Research Cambridge Collaborators: E. Perron and D. Vasudevan 1 Consensus – with Limited Processing and Signalling](https://reader033.vdocuments.mx/reader033/viewer/2022051613/55147592550346284e8b6282/html5/thumbnails/29.jpg)
Probability of Error – Binary Signaling
Theorem –
where
UVVU pf ,
12
12
!
!2
1N
Ni
i
N
UVNi
i
UV
iN
iN
p
))]2log(1(21[~)log(1 UVpN
29
Corollary – for large N
![Page 30: Milan Vojnović Microsoft Research Cambridge Collaborators: E. Perron and D. Vasudevan 1 Consensus – with Limited Processing and Signalling](https://reader033.vdocuments.mx/reader033/viewer/2022051613/55147592550346284e8b6282/html5/thumbnails/30.jpg)
Probability of Error (Cont’d)
Ob. Worse than under ternary protocol for a factor exponentially increasing with N
UVN
Uf VU for ,,
30
But …
Theorem –
– Not worse than classical voter model
![Page 31: Milan Vojnović Microsoft Research Cambridge Collaborators: E. Perron and D. Vasudevan 1 Consensus – with Limited Processing and Signalling](https://reader033.vdocuments.mx/reader033/viewer/2022051613/55147592550346284e8b6282/html5/thumbnails/31.jpg)
Probability of Error – Exponentially Bounded ?
Suggested by numerical results
31
![Page 32: Milan Vojnović Microsoft Research Cambridge Collaborators: E. Perron and D. Vasudevan 1 Consensus – with Limited Processing and Signalling](https://reader033.vdocuments.mx/reader033/viewer/2022051613/55147592550346284e8b6282/html5/thumbnails/32.jpg)
Binary Protocol – Many-Nodes Limit
The limit ODE:
For z = u + v and w = v – u, we have
)]())(1())(1[()(
)]())(1())(1[()(
2
2
tvtututvdt
d
tutvtvtudt
d
)())(1(2
1)(
)(2
1)(
2
31)( 2
twtztwdt
d
twtztzdt
d
32
![Page 33: Milan Vojnović Microsoft Research Cambridge Collaborators: E. Perron and D. Vasudevan 1 Consensus – with Limited Processing and Signalling](https://reader033.vdocuments.mx/reader033/viewer/2022051613/55147592550346284e8b6282/html5/thumbnails/33.jpg)
Convergence Time
Theorem – Convergence time:
A, B = constants independent on N
- Slower than ternary signaling by at least factor 2
- Not slower than factor 3
NBNNtAN large for ,)log(3)()log(2
33
![Page 34: Milan Vojnović Microsoft Research Cambridge Collaborators: E. Perron and D. Vasudevan 1 Consensus – with Limited Processing and Signalling](https://reader033.vdocuments.mx/reader033/viewer/2022051613/55147592550346284e8b6282/html5/thumbnails/34.jpg)
Proof Basic Steps
(u(t),v(t)) in this set in a finite time independent of N
Asserted bounds follow by ODE comparisons
34
![Page 35: Milan Vojnović Microsoft Research Cambridge Collaborators: E. Perron and D. Vasudevan 1 Consensus – with Limited Processing and Signalling](https://reader033.vdocuments.mx/reader033/viewer/2022051613/55147592550346284e8b6282/html5/thumbnails/35.jpg)
Convergence Time (Cont’d)
(u(0), v(0)) = (0.3, 0.7)
35
![Page 36: Milan Vojnović Microsoft Research Cambridge Collaborators: E. Perron and D. Vasudevan 1 Consensus – with Limited Processing and Signalling](https://reader033.vdocuments.mx/reader033/viewer/2022051613/55147592550346284e8b6282/html5/thumbnails/36.jpg)
Conclusion
“Good news” results for binary consensus on complete graphs
Ternary signaling Probability of error decays exponentially with
the number of nodes N log(N) convergence time
Binary signaling Probability of error worse than for the ternary
signaling for a factor exponentially increasing with N, but not worse than for classical voter
Convergence time C log(N) with 2 C 3
36
![Page 37: Milan Vojnović Microsoft Research Cambridge Collaborators: E. Perron and D. Vasudevan 1 Consensus – with Limited Processing and Signalling](https://reader033.vdocuments.mx/reader033/viewer/2022051613/55147592550346284e8b6282/html5/thumbnails/37.jpg)
Future work
Arbitrary graphs ?
Top k ?
37
![Page 38: Milan Vojnović Microsoft Research Cambridge Collaborators: E. Perron and D. Vasudevan 1 Consensus – with Limited Processing and Signalling](https://reader033.vdocuments.mx/reader033/viewer/2022051613/55147592550346284e8b6282/html5/thumbnails/38.jpg)
Arbitrary graphs
There exist graphs for which ternary protocol provides no benefits over classical voter
Ex. path with initial state:
38
1 01 1 1 0 0 0. . . . . .
U V
![Page 39: Milan Vojnović Microsoft Research Cambridge Collaborators: E. Perron and D. Vasudevan 1 Consensus – with Limited Processing and Signalling](https://reader033.vdocuments.mx/reader033/viewer/2022051613/55147592550346284e8b6282/html5/thumbnails/39.jpg)
Path
Path graph evolves essentially as under voter model
39
01 1 1 0 0 0
01 1 0 0 0e
01 1 0 0 00
1/2
1/2
1/2
![Page 40: Milan Vojnović Microsoft Research Cambridge Collaborators: E. Perron and D. Vasudevan 1 Consensus – with Limited Processing and Signalling](https://reader033.vdocuments.mx/reader033/viewer/2022051613/55147592550346284e8b6282/html5/thumbnails/40.jpg)
Heterogeneous Rates of Interactions
40
0
1
1
0
0
0
e
0 1
e
10
e
0
1
0
Still complete graph interactions
Two node types:
Light – small interaction rate
Heavy– large interaction rate
Q: Can initial minority prevail ?
![Page 41: Milan Vojnović Microsoft Research Cambridge Collaborators: E. Perron and D. Vasudevan 1 Consensus – with Limited Processing and Signalling](https://reader033.vdocuments.mx/reader033/viewer/2022051613/55147592550346284e8b6282/html5/thumbnails/41.jpg)
Can Initial Minority Prevail ? – Yes.
41
Example: Node types
0.2 light 0.8 heavy
Interaction rates0.1 light2 heavy
U V
Light 0.1 0.05
Heavy 0.35 0.45
0.45 0.5
V state nodes(initial majority)