supporting and analyzing probabilistic consistency in distributed … · 2016-12-02 · randomized...
TRANSCRIPT
![Page 1: SUPPORTING AND ANALYZING PROBABILISTIC CONSISTENCY IN DISTRIBUTED … · 2016-12-02 · RANDOMIZED REGISTERS • Lee and Welch (2004) • random register satisfies three conditions:](https://reader034.vdocuments.mx/reader034/viewer/2022042104/5e817a7a267ae15e4918bc82/html5/thumbnails/1.jpg)
SUPPORTING AND ANALYZING
PROBABILISTIC CONSISTENCY
IN DISTRIBUTED STORAGE
SYSTEMS
Wojciech [email protected]
![Page 2: SUPPORTING AND ANALYZING PROBABILISTIC CONSISTENCY IN DISTRIBUTED … · 2016-12-02 · RANDOMIZED REGISTERS • Lee and Welch (2004) • random register satisfies three conditions:](https://reader034.vdocuments.mx/reader034/viewer/2022042104/5e817a7a267ae15e4918bc82/html5/thumbnails/2.jpg)
OUTLINE
Background and Motivation
• quorum replication
• weak consistency using partial quorums
Overview of Prior Work
• probabilistic quorums and random registers
• probabilistically bounded staleness
• consistency benchmarking
• consistency-latency tuning
Ongoing Work at Waterloo
• mathematical model of eventual consistency
• improved consistency-latency tuning
2
![Page 3: SUPPORTING AND ANALYZING PROBABILISTIC CONSISTENCY IN DISTRIBUTED … · 2016-12-02 · RANDOMIZED REGISTERS • Lee and Welch (2004) • random register satisfies three conditions:](https://reader034.vdocuments.mx/reader034/viewer/2022042104/5e817a7a267ae15e4918bc82/html5/thumbnails/3.jpg)
Background and Motivation
3
![Page 4: SUPPORTING AND ANALYZING PROBABILISTIC CONSISTENCY IN DISTRIBUTED … · 2016-12-02 · RANDOMIZED REGISTERS • Lee and Welch (2004) • random register satisfies three conditions:](https://reader034.vdocuments.mx/reader034/viewer/2022042104/5e817a7a267ae15e4918bc82/html5/thumbnails/4.jpg)
ROLES OF RANDOMIZATION
4
computability
complexity
![Page 5: SUPPORTING AND ANALYZING PROBABILISTIC CONSISTENCY IN DISTRIBUTED … · 2016-12-02 · RANDOMIZED REGISTERS • Lee and Welch (2004) • random register satisfies three conditions:](https://reader034.vdocuments.mx/reader034/viewer/2022042104/5e817a7a267ae15e4918bc82/html5/thumbnails/5.jpg)
RANDOMIZED MUTUAL
EXCLUSION
5
input algorithm output
adversary(controls schedule of process steps)
(max. numberof processes: N)
(order of entryinto critical section)
complexity measure:number of remote memory references (RMRs) required to enter and leave the critical section once
![Page 6: SUPPORTING AND ANALYZING PROBABILISTIC CONSISTENCY IN DISTRIBUTED … · 2016-12-02 · RANDOMIZED REGISTERS • Lee and Welch (2004) • random register satisfies three conditions:](https://reader034.vdocuments.mx/reader034/viewer/2022042104/5e817a7a267ae15e4918bc82/html5/thumbnails/6.jpg)
RANDOMIZED MUTUAL
EXCLUSION
6
Time complexity of one passage through a mutual exclusion algorithm in the asynchronous shared memory model with Read and Write operations:
Worst-case: (log N)Attiya, Hendler, and Woelfel (2008)
Expected: O(log N / log log N)
Hendler and Woelfel (2009/2011)
![Page 7: SUPPORTING AND ANALYZING PROBABILISTIC CONSISTENCY IN DISTRIBUTED … · 2016-12-02 · RANDOMIZED REGISTERS • Lee and Welch (2004) • random register satisfies three conditions:](https://reader034.vdocuments.mx/reader034/viewer/2022042104/5e817a7a267ae15e4918bc82/html5/thumbnails/7.jpg)
CONCURRENT DATA STRUCTURE
(SHARED MEMORY)
7
input algorithm output
adversary(controls schedule of process steps and the operations invoked)
(max. numberof processes: N)
(operation responses)
complexity measure:number of steps requiredto complete one operation
![Page 8: SUPPORTING AND ANALYZING PROBABILISTIC CONSISTENCY IN DISTRIBUTED … · 2016-12-02 · RANDOMIZED REGISTERS • Lee and Welch (2004) • random register satisfies three conditions:](https://reader034.vdocuments.mx/reader034/viewer/2022042104/5e817a7a267ae15e4918bc82/html5/thumbnails/8.jpg)
DISTRIBUTED STORAGE SYSTEM(S.M. ON TOP OF MESSAGE PASSING)
8
input algorithm output
adversary(controls schedule of process steps, operations invoked, message delays)
(tuning knobs) (operation responses)
performance metrics:latency, consistency(real numbers!)
![Page 9: SUPPORTING AND ANALYZING PROBABILISTIC CONSISTENCY IN DISTRIBUTED … · 2016-12-02 · RANDOMIZED REGISTERS • Lee and Welch (2004) • random register satisfies three conditions:](https://reader034.vdocuments.mx/reader034/viewer/2022042104/5e817a7a267ae15e4918bc82/html5/thumbnails/9.jpg)
WEAK CONSISTENCY IN ACTION
Write(x, 1)writer
x = 0
x = 0
x = 0
9
![Page 10: SUPPORTING AND ANALYZING PROBABILISTIC CONSISTENCY IN DISTRIBUTED … · 2016-12-02 · RANDOMIZED REGISTERS • Lee and Welch (2004) • random register satisfies three conditions:](https://reader034.vdocuments.mx/reader034/viewer/2022042104/5e817a7a267ae15e4918bc82/html5/thumbnails/10.jpg)
WEAK CONSISTENCY IN ACTION
Write(x, 1)writer
x = 1
x = 0
x = 0
(waiting for onereplica to respond)
latest value determinedusing a timestamp
(not shown)
10
![Page 11: SUPPORTING AND ANALYZING PROBABILISTIC CONSISTENCY IN DISTRIBUTED … · 2016-12-02 · RANDOMIZED REGISTERS • Lee and Welch (2004) • random register satisfies three conditions:](https://reader034.vdocuments.mx/reader034/viewer/2022042104/5e817a7a267ae15e4918bc82/html5/thumbnails/11.jpg)
WEAK CONSISTENCY IN ACTION
Read(x)x = 1
x = 0
x = 0
reader
11
messages
in flight
![Page 12: SUPPORTING AND ANALYZING PROBABILISTIC CONSISTENCY IN DISTRIBUTED … · 2016-12-02 · RANDOMIZED REGISTERS • Lee and Welch (2004) • random register satisfies three conditions:](https://reader034.vdocuments.mx/reader034/viewer/2022042104/5e817a7a267ae15e4918bc82/html5/thumbnails/12.jpg)
WEAK CONSISTENCY IN ACTION
reader
returns 0
x = 1
x = 0
x = 0
reader
stale value
messages
in flight
12
![Page 13: SUPPORTING AND ANALYZING PROBABILISTIC CONSISTENCY IN DISTRIBUTED … · 2016-12-02 · RANDOMIZED REGISTERS • Lee and Welch (2004) • random register satisfies three conditions:](https://reader034.vdocuments.mx/reader034/viewer/2022042104/5e817a7a267ae15e4918bc82/html5/thumbnails/13.jpg)
WEAK CONSISTENCY IN ACTION
x = 1
x = 1
x = 1
messages
arrive
13
![Page 14: SUPPORTING AND ANALYZING PROBABILISTIC CONSISTENCY IN DISTRIBUTED … · 2016-12-02 · RANDOMIZED REGISTERS • Lee and Welch (2004) • random register satisfies three conditions:](https://reader034.vdocuments.mx/reader034/viewer/2022042104/5e817a7a267ae15e4918bc82/html5/thumbnails/14.jpg)
STALE READS CONSIDERED
DANGEROUS!
14 "Photo privacy violation" example from Lloyd, Freedman, Kaminsky, and Andersen (2014)
How often?How long?
![Page 15: SUPPORTING AND ANALYZING PROBABILISTIC CONSISTENCY IN DISTRIBUTED … · 2016-12-02 · RANDOMIZED REGISTERS • Lee and Welch (2004) • random register satisfies three conditions:](https://reader034.vdocuments.mx/reader034/viewer/2022042104/5e817a7a267ae15e4918bc82/html5/thumbnails/15.jpg)
GOAL
What is the expected proportion of stale reads in the following workload?
• 6 servers
• replication factor 3, partial quorum size 1
• 1000 ops/s/server, Poisson arrivals
• 25% Write, 75% Read operations
• mean network delay 100ms, exponentially distributed
• processing delay 0ms
15
![Page 16: SUPPORTING AND ANALYZING PROBABILISTIC CONSISTENCY IN DISTRIBUTED … · 2016-12-02 · RANDOMIZED REGISTERS • Lee and Welch (2004) • random register satisfies three conditions:](https://reader034.vdocuments.mx/reader034/viewer/2022042104/5e817a7a267ae15e4918bc82/html5/thumbnails/16.jpg)
Overview of Prior Work
16
![Page 17: SUPPORTING AND ANALYZING PROBABILISTIC CONSISTENCY IN DISTRIBUTED … · 2016-12-02 · RANDOMIZED REGISTERS • Lee and Welch (2004) • random register satisfies three conditions:](https://reader034.vdocuments.mx/reader034/viewer/2022042104/5e817a7a267ae15e4918bc82/html5/thumbnails/17.jpg)
ASSUMPTIONS
17
• Read and Write operations
• asynchronous model
• processes may fail by crashing
• network is reliable but delays not bounded
• exceptions: link failures and bounded
network delays in some papers
![Page 18: SUPPORTING AND ANALYZING PROBABILISTIC CONSISTENCY IN DISTRIBUTED … · 2016-12-02 · RANDOMIZED REGISTERS • Lee and Welch (2004) • random register satisfies three conditions:](https://reader034.vdocuments.mx/reader034/viewer/2022042104/5e817a7a267ae15e4918bc82/html5/thumbnails/18.jpg)
P1.W(x)a
P3.W(x)b
P4.R(x)b
P2.R(x)a
?W(x)? ?W(x)? ?R(x)? ?R(x)?
P1.W(x)a
P2.W(x)b
P4.R(x)b
P3.R(x)a
P1.W(x)a P3.R(x)a P2.W(x)b P4.R(x)b
linearizable not linearizabletime
Herlihy and Wing (1990)18
LINEARIZABILITY
processID
operationobject value
writtenor read
![Page 19: SUPPORTING AND ANALYZING PROBABILISTIC CONSISTENCY IN DISTRIBUTED … · 2016-12-02 · RANDOMIZED REGISTERS • Lee and Welch (2004) • random register satisfies three conditions:](https://reader034.vdocuments.mx/reader034/viewer/2022042104/5e817a7a267ae15e4918bc82/html5/thumbnails/19.jpg)
P1.W(x)a
P3.W(x)b
P4.R(x)b
P2.R(x)a
?W(x)? ?W(x)? ?R(x)? ?R(x)?
P1.W(x)a
P2.W(x)b
P4.R(x)b
P3.R(x)a
P1.W(x)a P2.W(x)b P4.R(x)b
regular not regulartime
based on Lamport (1986)19
REGULARITY (GENERALIZED)
processID
operationobject value
writtenor read
![Page 20: SUPPORTING AND ANALYZING PROBABILISTIC CONSISTENCY IN DISTRIBUTED … · 2016-12-02 · RANDOMIZED REGISTERS • Lee and Welch (2004) • random register satisfies three conditions:](https://reader034.vdocuments.mx/reader034/viewer/2022042104/5e817a7a267ae15e4918bc82/html5/thumbnails/20.jpg)
P1.W(x)a P1.W(x)b
P2.R(x)b P3.R(x)a
20
EXAMPLE OF HISTORY THAT IS
REGULAR BUT NOT LINEARIZABLE
new/old inversion
time
![Page 21: SUPPORTING AND ANALYZING PROBABILISTIC CONSISTENCY IN DISTRIBUTED … · 2016-12-02 · RANDOMIZED REGISTERS • Lee and Welch (2004) • random register satisfies three conditions:](https://reader034.vdocuments.mx/reader034/viewer/2022042104/5e817a7a267ae15e4918bc82/html5/thumbnails/21.jpg)
ABD SIMULATION
• Attiya, Bar-Noy, Dolev (1990)
• single-writer multi-reader register simulation
on top of message passing
• asynchronous model with process crash
failures and dynamic link failures
• majority of processes must be correct
• ensures linearizability
• 1 roundtrip for writer, 2 roundtrips for reader
21
![Page 22: SUPPORTING AND ANALYZING PROBABILISTIC CONSISTENCY IN DISTRIBUTED … · 2016-12-02 · RANDOMIZED REGISTERS • Lee and Welch (2004) • random register satisfies three conditions:](https://reader034.vdocuments.mx/reader034/viewer/2022042104/5e817a7a267ae15e4918bc82/html5/thumbnails/22.jpg)
ABD SIMULATION
Write(x, 1)writer
x = 0
x = 0
x = 0
22
![Page 23: SUPPORTING AND ANALYZING PROBABILISTIC CONSISTENCY IN DISTRIBUTED … · 2016-12-02 · RANDOMIZED REGISTERS • Lee and Welch (2004) • random register satisfies three conditions:](https://reader034.vdocuments.mx/reader034/viewer/2022042104/5e817a7a267ae15e4918bc82/html5/thumbnails/23.jpg)
ABD SIMULATION
Write(x, 1)writer
x = 1
x = 0
x = 0
(waiting for majorityto respond)
23
![Page 24: SUPPORTING AND ANALYZING PROBABILISTIC CONSISTENCY IN DISTRIBUTED … · 2016-12-02 · RANDOMIZED REGISTERS • Lee and Welch (2004) • random register satisfies three conditions:](https://reader034.vdocuments.mx/reader034/viewer/2022042104/5e817a7a267ae15e4918bc82/html5/thumbnails/24.jpg)
ABD SIMULATION
Read(x)
x = 1
x = 0
x = 0
readerwriterWrite(x, 1)
(waiting for majorityto respond)
24
![Page 25: SUPPORTING AND ANALYZING PROBABILISTIC CONSISTENCY IN DISTRIBUTED … · 2016-12-02 · RANDOMIZED REGISTERS • Lee and Welch (2004) • random register satisfies three conditions:](https://reader034.vdocuments.mx/reader034/viewer/2022042104/5e817a7a267ae15e4918bc82/html5/thumbnails/25.jpg)
ABD SIMULATION
x = 1
x = 0
x = 0
reader
reader
discovers 1writer
Write(x, 1)
(waiting for majorityto respond)
(waiting for majorityto respond)
25
![Page 26: SUPPORTING AND ANALYZING PROBABILISTIC CONSISTENCY IN DISTRIBUTED … · 2016-12-02 · RANDOMIZED REGISTERS • Lee and Welch (2004) • random register satisfies three conditions:](https://reader034.vdocuments.mx/reader034/viewer/2022042104/5e817a7a267ae15e4918bc82/html5/thumbnails/26.jpg)
ABD SIMULATION
x = 1
x = 1
x = 1
reader
reader
broadcasts 1writer
Write(x, 1)
(waiting for majorityto respond)
26
![Page 27: SUPPORTING AND ANALYZING PROBABILISTIC CONSISTENCY IN DISTRIBUTED … · 2016-12-02 · RANDOMIZED REGISTERS • Lee and Welch (2004) • random register satisfies three conditions:](https://reader034.vdocuments.mx/reader034/viewer/2022042104/5e817a7a267ae15e4918bc82/html5/thumbnails/27.jpg)
ABD SIMULATION
x = 1
x = 1
x = 1
reader
Read(x)
returns 1writer
Write(x, 1)
(waiting for majorityto respond) (two round trips)
27
![Page 28: SUPPORTING AND ANALYZING PROBABILISTIC CONSISTENCY IN DISTRIBUTED … · 2016-12-02 · RANDOMIZED REGISTERS • Lee and Welch (2004) • random register satisfies three conditions:](https://reader034.vdocuments.mx/reader034/viewer/2022042104/5e817a7a267ae15e4918bc82/html5/thumbnails/28.jpg)
DECIDING LINEARIZABILITY
• Gibbons and Korach (1997)
• algorithm works for histories over Read and
Write operations
• assumes the "reads-from" mapping is known,
for example because all Write operations on
a given object assign distinct values
• O(N log N) steps for a history of N
operations
28
![Page 29: SUPPORTING AND ANALYZING PROBABILISTIC CONSISTENCY IN DISTRIBUTED … · 2016-12-02 · RANDOMIZED REGISTERS • Lee and Welch (2004) • random register satisfies three conditions:](https://reader034.vdocuments.mx/reader034/viewer/2022042104/5e817a7a267ae15e4918bc82/html5/thumbnails/29.jpg)
DECIDING LINEARIZABILITY
W(x)0
W(x)1 R(x)1
R(x)0
min(finish times)max(start times)
forward zone
FZ-1
backward zone
BZ-0
min(finish times) max(start times)
cluster-0
cluster-1
29
![Page 30: SUPPORTING AND ANALYZING PROBABILISTIC CONSISTENCY IN DISTRIBUTED … · 2016-12-02 · RANDOMIZED REGISTERS • Lee and Welch (2004) • random register satisfies three conditions:](https://reader034.vdocuments.mx/reader034/viewer/2022042104/5e817a7a267ae15e4918bc82/html5/thumbnails/30.jpg)
DECIDING LINEARIZABILITY
A history of Read and Write operations is linearizable if
every Read returns the value of some Write, and no two
zones conflict.
Two forward zones conflict if they overlap:
A forward zone conflicts with a backward if the former is
a superset of the latter:
30
![Page 31: SUPPORTING AND ANALYZING PROBABILISTIC CONSISTENCY IN DISTRIBUTED … · 2016-12-02 · RANDOMIZED REGISTERS • Lee and Welch (2004) • random register satisfies three conditions:](https://reader034.vdocuments.mx/reader034/viewer/2022042104/5e817a7a267ae15e4918bc82/html5/thumbnails/31.jpg)
DECIDING LINEARIZABILITY
31
W(x)0
W(x)1R(x)1
R(x)0
W(x)0
W(x)1
R(x)1
R(x)0
FZ-0
FZ-1
FZ-0
BZ-1
![Page 32: SUPPORTING AND ANALYZING PROBABILISTIC CONSISTENCY IN DISTRIBUTED … · 2016-12-02 · RANDOMIZED REGISTERS • Lee and Welch (2004) • random register satisfies three conditions:](https://reader034.vdocuments.mx/reader034/viewer/2022042104/5e817a7a267ae15e4918bc82/html5/thumbnails/32.jpg)
PROBABILISTIC QUORUM
SYSTEMS
• Malkhi, Reiter, Wool, and Wright (2001)
• -intersecting quorum system: any two
"quorums" must overlap with probability at
least 1 – with respect to an access strategy
• example:
N = 2 processes
Read and Write operations access one server
chosen uniformly at random
= 1/2
32
![Page 33: SUPPORTING AND ANALYZING PROBABILISTIC CONSISTENCY IN DISTRIBUTED … · 2016-12-02 · RANDOMIZED REGISTERS • Lee and Welch (2004) • random register satisfies three conditions:](https://reader034.vdocuments.mx/reader034/viewer/2022042104/5e817a7a267ae15e4918bc82/html5/thumbnails/33.jpg)
RANDOMIZED REGISTERS
• Lee and Welch (2004)
• random register satisfies three conditions:1. every operation terminates
2. every read operation reads from some write
3. for any given write, the probability that this write is read from infinitely often is 0 if there are infinitely many writes
• relaxation of Lamport's regularity property for single-writer multi-reader registers
• implementable using probabilistic quorums
• alternative definitions: P-bounded and monotone random registers
33
![Page 34: SUPPORTING AND ANALYZING PROBABILISTIC CONSISTENCY IN DISTRIBUTED … · 2016-12-02 · RANDOMIZED REGISTERS • Lee and Welch (2004) • random register satisfies three conditions:](https://reader034.vdocuments.mx/reader034/viewer/2022042104/5e817a7a267ae15e4918bc82/html5/thumbnails/34.jpg)
P1.W(x)a P1.W(x)b P3.R(x)aP2.R(x)b
RANDOMIZED REGISTERS
Possible behavior:
34
reads from
. . .
repeatedinfinitely often
time
![Page 35: SUPPORTING AND ANALYZING PROBABILISTIC CONSISTENCY IN DISTRIBUTED … · 2016-12-02 · RANDOMIZED REGISTERS • Lee and Welch (2004) • random register satisfies three conditions:](https://reader034.vdocuments.mx/reader034/viewer/2022042104/5e817a7a267ae15e4918bc82/html5/thumbnails/35.jpg)
RANDOMIZED REGISTERS
35 Lee and Welch (2004)
k = quorum size(uniform accessstrategy)
l-outdated read:returned valueis not allowablebut is the valueof the l-th writepreceding thebeginning of the read
![Page 36: SUPPORTING AND ANALYZING PROBABILISTIC CONSISTENCY IN DISTRIBUTED … · 2016-12-02 · RANDOMIZED REGISTERS • Lee and Welch (2004) • random register satisfies three conditions:](https://reader034.vdocuments.mx/reader034/viewer/2022042104/5e817a7a267ae15e4918bc82/html5/thumbnails/36.jpg)
QUANTIFYING STALENESS
• Golab, Li, and Shah (2011)
• techniques for quantifying both the severity
and frequency of linearizability anomalies
• builds on Gibbons and Korach (1997)
• anomalies counted at the granularity of
"clusters" (subsets of operations applied to
one object that access the same value)
36
![Page 37: SUPPORTING AND ANALYZING PROBABILISTIC CONSISTENCY IN DISTRIBUTED … · 2016-12-02 · RANDOMIZED REGISTERS • Lee and Welch (2004) • random register satisfies three conditions:](https://reader034.vdocuments.mx/reader034/viewer/2022042104/5e817a7a267ae15e4918bc82/html5/thumbnails/37.jpg)
QUANTIFYING STALENESS:
SEVERITY
The linearizability anomalies in a history have
maximum severity at most time units if
decreasing the start time of every Read
operation by makes the history linearizable.
37
W(x)0
W(x)1R(x)1
R(x)0
W(x)0
W(x)1R(x)1
R(x)0
FZ-0
FZ-1
FZ-0
FZ-1
![Page 38: SUPPORTING AND ANALYZING PROBABILISTIC CONSISTENCY IN DISTRIBUTED … · 2016-12-02 · RANDOMIZED REGISTERS • Lee and Welch (2004) • random register satisfies three conditions:](https://reader034.vdocuments.mx/reader034/viewer/2022042104/5e817a7a267ae15e4918bc82/html5/thumbnails/38.jpg)
QUANTIFYING STALENESS:
SEVERITY
Severity is quantified by a score function
Fx(v, w) that defines how far the start times of
reads on object x must be shifted to resolve
any conflict between the zone for v and the
zone for w.
38
Rahman, Golab, AuYoung, Keeton, Wylie (2012)
![Page 39: SUPPORTING AND ANALYZING PROBABILISTIC CONSISTENCY IN DISTRIBUTED … · 2016-12-02 · RANDOMIZED REGISTERS • Lee and Welch (2004) • random register satisfies three conditions:](https://reader034.vdocuments.mx/reader034/viewer/2022042104/5e817a7a267ae15e4918bc82/html5/thumbnails/39.jpg)
QUANTIFYING STALENESS:
FREQUENCY
Frequency is quantified as the proportion of
values that participate in linearizability
violations for object x:
number of values v for which Fx(v, ∙ ) > 0
total number of distinct values accessed
39
![Page 40: SUPPORTING AND ANALYZING PROBABILISTIC CONSISTENCY IN DISTRIBUTED … · 2016-12-02 · RANDOMIZED REGISTERS • Lee and Welch (2004) • random register satisfies three conditions:](https://reader034.vdocuments.mx/reader034/viewer/2022042104/5e817a7a267ae15e4918bc82/html5/thumbnails/40.jpg)
PROVIDING A MEASURE OF
INSTANTANEOUS CONSISTENCY
• Golab and Wylie (2012)
• builds on Golab, Li, and Shah (2011)
• instantaneous staleness at time t with
respect to object x: maximum of the score
function Fx(v, w) for any pair of values v and
w whose zones overlap at time t.
40
FZ-0
FZ-1
![Page 41: SUPPORTING AND ANALYZING PROBABILISTIC CONSISTENCY IN DISTRIBUTED … · 2016-12-02 · RANDOMIZED REGISTERS • Lee and Welch (2004) • random register satisfies three conditions:](https://reader034.vdocuments.mx/reader034/viewer/2022042104/5e817a7a267ae15e4918bc82/html5/thumbnails/41.jpg)
PROVIDING A MEASURE OF
INSTANTANEOUS CONSISTENCY
41
US Patent 9292566artificial delay
![Page 42: SUPPORTING AND ANALYZING PROBABILISTIC CONSISTENCY IN DISTRIBUTED … · 2016-12-02 · RANDOMIZED REGISTERS • Lee and Welch (2004) • random register satisfies three conditions:](https://reader034.vdocuments.mx/reader034/viewer/2022042104/5e817a7a267ae15e4918bc82/html5/thumbnails/42.jpg)
PROVIDING A MEASURE OF
INSTANTANEOUS CONSISTENCY
Example of service level agreement (SLA):
X% of the time
the instantaneous staleness is Y ms
(+ bound on latency, for example 95%-ile)
42
![Page 43: SUPPORTING AND ANALYZING PROBABILISTIC CONSISTENCY IN DISTRIBUTED … · 2016-12-02 · RANDOMIZED REGISTERS • Lee and Welch (2004) • random register satisfies three conditions:](https://reader034.vdocuments.mx/reader034/viewer/2022042104/5e817a7a267ae15e4918bc82/html5/thumbnails/43.jpg)
PROVIDING A MEASURE OF
INSTANTANEOUS CONSISTENCY
Tuning technique: artificial delay
43
W(x)0
W(x)1R(x)1
R(x)0
W(x)0
W(x)1R(x)1
R(x)1
![Page 44: SUPPORTING AND ANALYZING PROBABILISTIC CONSISTENCY IN DISTRIBUTED … · 2016-12-02 · RANDOMIZED REGISTERS • Lee and Welch (2004) • random register satisfies three conditions:](https://reader034.vdocuments.mx/reader034/viewer/2022042104/5e817a7a267ae15e4918bc82/html5/thumbnails/44.jpg)
PROBABILISTICALLY BOUNDED
STALENESS (PBS)
• Bailis, Venkataraman, Franklin, Hellerstein,
and Stoica (2012)
• mathematical model of weak consistency
based on probabilistic quorums
• t-visibility: probability that a Read invoked t
time units after the completion of a Write
returns the value assigned by that Write
• concurrent reading and writing outside the
scope of the model
44
![Page 45: SUPPORTING AND ANALYZING PROBABILISTIC CONSISTENCY IN DISTRIBUTED … · 2016-12-02 · RANDOMIZED REGISTERS • Lee and Welch (2004) • random register satisfies three conditions:](https://reader034.vdocuments.mx/reader034/viewer/2022042104/5e817a7a267ae15e4918bc82/html5/thumbnails/45.jpg)
PROBABILISTICALLY BOUNDED
STALENESS (PBS)
Write-Ack-Read-Response (WARS) model:
45 Bailis et al. (2012)
![Page 46: SUPPORTING AND ANALYZING PROBABILISTIC CONSISTENCY IN DISTRIBUTED … · 2016-12-02 · RANDOMIZED REGISTERS • Lee and Welch (2004) • random register satisfies three conditions:](https://reader034.vdocuments.mx/reader034/viewer/2022042104/5e817a7a267ae15e4918bc82/html5/thumbnails/46.jpg)
PROBABILISTICALLY BOUNDED
STALENESS (PBS)
46 Bailis et al. (2012)
![Page 47: SUPPORTING AND ANALYZING PROBABILISTIC CONSISTENCY IN DISTRIBUTED … · 2016-12-02 · RANDOMIZED REGISTERS • Lee and Welch (2004) • random register satisfies three conditions:](https://reader034.vdocuments.mx/reader034/viewer/2022042104/5e817a7a267ae15e4918bc82/html5/thumbnails/47.jpg)
PROBABILISTICALLY BOUNDED
STALENESS (PBS)
<live demo>
http://pbs.cs.berkeley.edu/
47
![Page 48: SUPPORTING AND ANALYZING PROBABILISTIC CONSISTENCY IN DISTRIBUTED … · 2016-12-02 · RANDOMIZED REGISTERS • Lee and Welch (2004) • random register satisfies three conditions:](https://reader034.vdocuments.mx/reader034/viewer/2022042104/5e817a7a267ae15e4918bc82/html5/thumbnails/48.jpg)
BENCHMARKING EVENTUAL
CONSISTENCY
• Golab, Rahman, AuYoung, Keeton, Gupta
(2014)
• evaluated effect of system and workload
parameters on staleness measurements
• staleness quantified using a score function
(gamma) similar to the one introduced by
Golab, Li, and Shah (2011)
48
![Page 49: SUPPORTING AND ANALYZING PROBABILISTIC CONSISTENCY IN DISTRIBUTED … · 2016-12-02 · RANDOMIZED REGISTERS • Lee and Welch (2004) • random register satisfies three conditions:](https://reader034.vdocuments.mx/reader034/viewer/2022042104/5e817a7a267ae15e4918bc82/html5/thumbnails/49.jpg)
BENCHMARKING EVENTUAL
CONSISTENCY
49
Golab et al. (2014)
theoreticalworst-case
= 1.00
(write at oneserver, readimmediatelyat another)
![Page 50: SUPPORTING AND ANALYZING PROBABILISTIC CONSISTENCY IN DISTRIBUTED … · 2016-12-02 · RANDOMIZED REGISTERS • Lee and Welch (2004) • random register satisfies three conditions:](https://reader034.vdocuments.mx/reader034/viewer/2022042104/5e817a7a267ae15e4918bc82/html5/thumbnails/50.jpg)
FINE-TUNING THE CONSISTENCY-
LATENCY TRADE-OFF
• McKenzie, Fan, and Golab (2015)
• technique #1: artificial delay (AD)
• technique #2: continuous partial quorums
(CPQ)
• observation: AD works best when network
delay is constant, CPQ better when
distribution of network delays has long tail
50
![Page 51: SUPPORTING AND ANALYZING PROBABILISTIC CONSISTENCY IN DISTRIBUTED … · 2016-12-02 · RANDOMIZED REGISTERS • Lee and Welch (2004) • random register satisfies three conditions:](https://reader034.vdocuments.mx/reader034/viewer/2022042104/5e817a7a267ae15e4918bc82/html5/thumbnails/51.jpg)
FINE-TUNING THE CONSISTENCY-
LATENCY TRADE-OFF
51
(in)consistency plots – CPQ (left) and AD (right)
McKenzie, Fan, Golab (2015)
![Page 52: SUPPORTING AND ANALYZING PROBABILISTIC CONSISTENCY IN DISTRIBUTED … · 2016-12-02 · RANDOMIZED REGISTERS • Lee and Welch (2004) • random register satisfies three conditions:](https://reader034.vdocuments.mx/reader034/viewer/2022042104/5e817a7a267ae15e4918bc82/html5/thumbnails/52.jpg)
FINE-TUNING THE CONSISTENCY-
LATENCY TRADE-OFF
52
latency plots – CPQ (left) and AD (right)
McKenzie, Fan, Golab (2015)
![Page 53: SUPPORTING AND ANALYZING PROBABILISTIC CONSISTENCY IN DISTRIBUTED … · 2016-12-02 · RANDOMIZED REGISTERS • Lee and Welch (2004) • random register satisfies three conditions:](https://reader034.vdocuments.mx/reader034/viewer/2022042104/5e817a7a267ae15e4918bc82/html5/thumbnails/53.jpg)
WATCA: THE WATERLOO
CONSISTENCY ANALYZER
• Fan, Chatterjee, Golab (2016)
• real-time consistency metric computation
and visualization
• built-in support for CPQ and AD
• open-source software:
https://github.com/wgolab/WatCA
53
![Page 54: SUPPORTING AND ANALYZING PROBABILISTIC CONSISTENCY IN DISTRIBUTED … · 2016-12-02 · RANDOMIZED REGISTERS • Lee and Welch (2004) • random register satisfies three conditions:](https://reader034.vdocuments.mx/reader034/viewer/2022042104/5e817a7a267ae15e4918bc82/html5/thumbnails/54.jpg)
PROBABILISTIC CAP (PCAP)
• Rahman, Tseng, Nguyen, Gupta, Vaidya (2016)
• mathematical model of consistency-latency
trade-off + adaptive tuning framework
• staleness quantified similarly to Golab, Li, and
Shah (2011) under the assumption that a Write
takes effect at its invocation (model ignores
write latency)
• (tc, pic)-consistency: fraction of Reads returning
values >tc time units stale is at most pic
54
![Page 55: SUPPORTING AND ANALYZING PROBABILISTIC CONSISTENCY IN DISTRIBUTED … · 2016-12-02 · RANDOMIZED REGISTERS • Lee and Welch (2004) • random register satisfies three conditions:](https://reader034.vdocuments.mx/reader034/viewer/2022042104/5e817a7a267ae15e4918bc82/html5/thumbnails/55.jpg)
PROBABILISTIC CAP (PCAP)
Impossibility result for consistency-latency
trade-off:
• tc: upper bound on staleness
• ta: upper bound on operation latency
• tp: upper bound on message delay
Theorem 1: tc + ta tp
55
![Page 56: SUPPORTING AND ANALYZING PROBABILISTIC CONSISTENCY IN DISTRIBUTED … · 2016-12-02 · RANDOMIZED REGISTERS • Lee and Welch (2004) • random register satisfies three conditions:](https://reader034.vdocuments.mx/reader034/viewer/2022042104/5e817a7a267ae15e4918bc82/html5/thumbnails/56.jpg)
PROBABILISTIC CAP (PCAP)
If tc = 0 then Theorem 1 resembles the lower
bound of Lipton and Sandberg (1988):
Any implementation of a sequentially
consistent read-write register must satisfy
|r|+|w| ≥ d, where |r| is the latency of a Read,
|w| is the latency of a Write, and d is the
network delay.
56
![Page 57: SUPPORTING AND ANALYZING PROBABILISTIC CONSISTENCY IN DISTRIBUTED … · 2016-12-02 · RANDOMIZED REGISTERS • Lee and Welch (2004) • random register satisfies three conditions:](https://reader034.vdocuments.mx/reader034/viewer/2022042104/5e817a7a267ae15e4918bc82/html5/thumbnails/57.jpg)
PROBABILISTIC CAP (PCAP)
Probabilistic variation:
• pic: proportion of reads with staleness > tc
• pua: proportion of operations with latency > ta
• p: proportion of messages with delay > tp
Theorem 2: if tc + ta < tp then pic + pua p.
57
![Page 58: SUPPORTING AND ANALYZING PROBABILISTIC CONSISTENCY IN DISTRIBUTED … · 2016-12-02 · RANDOMIZED REGISTERS • Lee and Welch (2004) • random register satisfies three conditions:](https://reader034.vdocuments.mx/reader034/viewer/2022042104/5e817a7a267ae15e4918bc82/html5/thumbnails/58.jpg)
PROBABILISTIC CAP (PCAP)
58
Rahman, Tseng, Nguyen, Gupta, Vaidya (2016)58
![Page 59: SUPPORTING AND ANALYZING PROBABILISTIC CONSISTENCY IN DISTRIBUTED … · 2016-12-02 · RANDOMIZED REGISTERS • Lee and Welch (2004) • random register satisfies three conditions:](https://reader034.vdocuments.mx/reader034/viewer/2022042104/5e817a7a267ae15e4918bc82/html5/thumbnails/59.jpg)
Ongoing Work at Waterloo
59
![Page 60: SUPPORTING AND ANALYZING PROBABILISTIC CONSISTENCY IN DISTRIBUTED … · 2016-12-02 · RANDOMIZED REGISTERS • Lee and Welch (2004) • random register satisfies three conditions:](https://reader034.vdocuments.mx/reader034/viewer/2022042104/5e817a7a267ae15e4918bc82/html5/thumbnails/60.jpg)
MATHEMATICAL MODEL OF
EVENTUAL CONSISTENCY
Prior work does not answer the question posed
earlier:
• analysis of probabilistic quorums does not
account for eventual consistency
• PBS focuses on a single Write/Read pair
• PCAP describes worst-case behavior
60
![Page 61: SUPPORTING AND ANALYZING PROBABILISTIC CONSISTENCY IN DISTRIBUTED … · 2016-12-02 · RANDOMIZED REGISTERS • Lee and Welch (2004) • random register satisfies three conditions:](https://reader034.vdocuments.mx/reader034/viewer/2022042104/5e817a7a267ae15e4918bc82/html5/thumbnails/61.jpg)
IMPROVED ADAPTIVE
CONSISTENCY-LATENCY TUNING
Experimental analysis by Shankha Chatterjee (MASc candidate)61
targetdelay
![Page 62: SUPPORTING AND ANALYZING PROBABILISTIC CONSISTENCY IN DISTRIBUTED … · 2016-12-02 · RANDOMIZED REGISTERS • Lee and Welch (2004) • random register satisfies three conditions:](https://reader034.vdocuments.mx/reader034/viewer/2022042104/5e817a7a267ae15e4918bc82/html5/thumbnails/62.jpg)
Preguntas y Respuestas
(Q&A)
62