distributed shared memory: ivy & memcached costin raiciu advanced topics in distributed systems...
TRANSCRIPT
![Page 1: Distributed Shared Memory: Ivy & memcached Costin Raiciu Advanced Topics in Distributed Systems 2012 Slides courtesy of Brad Karp (UCL) and Robert Morris](https://reader033.vdocuments.mx/reader033/viewer/2022051215/56649ebc5503460f94bc4c22/html5/thumbnails/1.jpg)
Distributed Shared Memory:Ivy & memcached
Costin RaiciuAdvanced Topics in Distributed
Systems2012
Slides courtesy of Brad Karp (UCL)and Robert Morris (MIT)
![Page 2: Distributed Shared Memory: Ivy & memcached Costin Raiciu Advanced Topics in Distributed Systems 2012 Slides courtesy of Brad Karp (UCL) and Robert Morris](https://reader033.vdocuments.mx/reader033/viewer/2022051215/56649ebc5503460f94bc4c22/html5/thumbnails/2.jpg)
2
Distributed Shared Memory: Problem
• An application has a shared address space; all memory locations accessible to all instructions
• Divide code for application into pieces, assign one piece to each of several computers on a LAN
• Each computer has own separate memory
• Each piece of code may want to read or write any part of data
• Where do you put the data?
![Page 3: Distributed Shared Memory: Ivy & memcached Costin Raiciu Advanced Topics in Distributed Systems 2012 Slides courtesy of Brad Karp (UCL) and Robert Morris](https://reader033.vdocuments.mx/reader033/viewer/2022051215/56649ebc5503460f94bc4c22/html5/thumbnails/3.jpg)
3
Ivy: Distributed Shared Memory
• Supercomputer: super-expensive 100-CPU machine, custom-built hardware
• Ivy: 100 cheap PCs and a LAN (all off-the-shelf hardware!)
• Both offer same easy view for programmer:– single, shared memory, visible to all
CPUs
![Page 4: Distributed Shared Memory: Ivy & memcached Costin Raiciu Advanced Topics in Distributed Systems 2012 Slides courtesy of Brad Karp (UCL) and Robert Morris](https://reader033.vdocuments.mx/reader033/viewer/2022051215/56649ebc5503460f94bc4c22/html5/thumbnails/4.jpg)
4
Distributed Shared Memory: Solution
• Goal: create illusion that all boxes share single memory, accessible by all
• Shared memory contents divided across nodes– Programmer needn’t explicitly communicate
among nodes– Pool memory of all nodes into one shared
memory• Performance? Correctness?
– Far slower to read/write across LAN than read/write from/to local (same host’s) memory
– Remember NFS: caching should help– Remember NFS: caching complicates
consistency!
![Page 5: Distributed Shared Memory: Ivy & memcached Costin Raiciu Advanced Topics in Distributed Systems 2012 Slides courtesy of Brad Karp (UCL) and Robert Morris](https://reader033.vdocuments.mx/reader033/viewer/2022051215/56649ebc5503460f94bc4c22/html5/thumbnails/5.jpg)
5
Context: Parallel Computation
• Still need to divide program code across multiple CPUs
• Potential benefit: more total CPU power, so faster execution
• Potential risk: how will we know if distributed program executes correctly?
• To understand distributed shared memory, must understand what “correct” execution means…
![Page 6: Distributed Shared Memory: Ivy & memcached Costin Raiciu Advanced Topics in Distributed Systems 2012 Slides courtesy of Brad Karp (UCL) and Robert Morris](https://reader033.vdocuments.mx/reader033/viewer/2022051215/56649ebc5503460f94bc4c22/html5/thumbnails/6.jpg)
6
Simple Case: Uniprocessor Correctness
• When you only have one processor, what does “correct” mean?
• Define “correct” separately for each instruction
• Each instruction takes machine from one state to another (e.g., ADD, LD, ST)– LD should return value of most recent
ST to same memory address
“Correct” means:Execution gives same result as if you ran one instruction at a time, waiting for each to complete
![Page 7: Distributed Shared Memory: Ivy & memcached Costin Raiciu Advanced Topics in Distributed Systems 2012 Slides courtesy of Brad Karp (UCL) and Robert Morris](https://reader033.vdocuments.mx/reader033/viewer/2022051215/56649ebc5503460f94bc4c22/html5/thumbnails/7.jpg)
7
Why Define Correctness?
• Programmers want to be able to predict how CPU executes program!– …to write correct program
• Note that modern CPUs don’t execute instructions one-at-a-time in program order– Multiple instruction issue– Out-of-order instruction issue
• Nevertheless, CPUs must behave such that they obey uniprocessor correctness!
![Page 8: Distributed Shared Memory: Ivy & memcached Costin Raiciu Advanced Topics in Distributed Systems 2012 Slides courtesy of Brad Karp (UCL) and Robert Morris](https://reader033.vdocuments.mx/reader033/viewer/2022051215/56649ebc5503460f94bc4c22/html5/thumbnails/8.jpg)
8
Distributed Correctness:Naïve Shared Memory
• Suppose we have multiple hosts with (for now) naïve shared memory– 3 hosts, each with one CPU, connected
by Internet– Each host has local copy of all memory– Reads local, so very fast– Writes sent to other hosts (and
execution continues immediately)• Is naïve shared memory correct?
![Page 9: Distributed Shared Memory: Ivy & memcached Costin Raiciu Advanced Topics in Distributed Systems 2012 Slides courtesy of Brad Karp (UCL) and Robert Morris](https://reader033.vdocuments.mx/reader033/viewer/2022051215/56649ebc5503460f94bc4c22/html5/thumbnails/9.jpg)
9
Example 1: Mutual Exclusion
CPU0:x = 1;if (y == 0)
critical section;
CPU1:y = 1;if (x == 0)
critical section;• Why is code correct?
– If CPU0 sees y == 0, CPU1 can’t have executed “y = 1”
– So CPU1 will see x == 1, and can’t enter critical section
Initialization: x = y = 0 on both CPUs
So CPU0 and CPU1 cannot simultaneously enter critical section
![Page 10: Distributed Shared Memory: Ivy & memcached Costin Raiciu Advanced Topics in Distributed Systems 2012 Slides courtesy of Brad Karp (UCL) and Robert Morris](https://reader033.vdocuments.mx/reader033/viewer/2022051215/56649ebc5503460f94bc4c22/html5/thumbnails/10.jpg)
10
Naïve Distributed Memory:Incorrect for Example 1
• Problem A:– CPU0 sends “write x=1”, reads local “y
== 0”– CPU1 reads local “x == 0” before write
arrives• Local memory and slow writes cause
disagreement about read/write order!– CPU0 thinks its “x = 1” was before
CPU1’s read of x– CPU1 thinks its read of x was before
arrival of “write x = 1”• Both CPU0 and CPU1 enter critical
section!
![Page 11: Distributed Shared Memory: Ivy & memcached Costin Raiciu Advanced Topics in Distributed Systems 2012 Slides courtesy of Brad Karp (UCL) and Robert Morris](https://reader033.vdocuments.mx/reader033/viewer/2022051215/56649ebc5503460f94bc4c22/html5/thumbnails/11.jpg)
11
Example 2:Data Dependencies
CPU0:v0 = f0();done0 = true;
CPU1:while (done0 == false)
;v1 = f1(v0);done1 = true;
CPU2:while (done1 == false)
;v2 = f2(v0, v1);
Intent:CPU2 should run f2() with results from CPU0 and CPU1Waiting for CPU1 implies waiting for CPU0
![Page 12: Distributed Shared Memory: Ivy & memcached Costin Raiciu Advanced Topics in Distributed Systems 2012 Slides courtesy of Brad Karp (UCL) and Robert Morris](https://reader033.vdocuments.mx/reader033/viewer/2022051215/56649ebc5503460f94bc4c22/html5/thumbnails/12.jpg)
12
Naïve Distributed Memory:Incorrect for Example 2
• Problem B:– CPU0’s writes of v0 and done0 may be
reordered by network, leaving v0 unset, but done0 true
• But even if each CPU sees each other CPU’s writes in issue order…
• Problem C:– CPU2 sees CPU1’s writes before CPU0’s
writes– i.e., CPU2 and CPU1 disagree on order of
CPU0’s and CPU1’s writes
Naïve distributed memory isn’t correct(Or we shouldn’t expect code like these examples to work…)
![Page 13: Distributed Shared Memory: Ivy & memcached Costin Raiciu Advanced Topics in Distributed Systems 2012 Slides courtesy of Brad Karp (UCL) and Robert Morris](https://reader033.vdocuments.mx/reader033/viewer/2022051215/56649ebc5503460f94bc4c22/html5/thumbnails/13.jpg)
13
Distributed Correctness:Consistency Models
• How can we write correct distributed programs with shared storage?
• Need to define rules that memory system will follow
• Need to write programs with these rules in mind
• Rules are a consistency model• Build memory system to obey model;
programs that assume model then correct
![Page 14: Distributed Shared Memory: Ivy & memcached Costin Raiciu Advanced Topics in Distributed Systems 2012 Slides courtesy of Brad Karp (UCL) and Robert Morris](https://reader033.vdocuments.mx/reader033/viewer/2022051215/56649ebc5503460f94bc4c22/html5/thumbnails/14.jpg)
14
How Do We Choose aConsistency Model?
• No such thing as “right” or “wrong” model– All models are artificial definitions
• Different models may be harder or easier to program for– Some models produce behavior that is
more intuitive than others• Different models may be harder or
easier to implement efficiently– Performance vs. semantics trade-off, as
with NFS/RPC
![Page 15: Distributed Shared Memory: Ivy & memcached Costin Raiciu Advanced Topics in Distributed Systems 2012 Slides courtesy of Brad Karp (UCL) and Robert Morris](https://reader033.vdocuments.mx/reader033/viewer/2022051215/56649ebc5503460f94bc4c22/html5/thumbnails/15.jpg)
15
Back to Ivy:What’s It Good For?
• Suppose you’ve got 100 PCs on a LAN and shared memory across all of them
• Fast, parallel sorting program:Load entire array into shared memoryEach PC processes one section of arrayOn PC i:
sort own piece of arraydone[i] = true;wait for all done[] to be truemerge my piece of array with my neighbors’…
![Page 16: Distributed Shared Memory: Ivy & memcached Costin Raiciu Advanced Topics in Distributed Systems 2012 Slides courtesy of Brad Karp (UCL) and Robert Morris](https://reader033.vdocuments.mx/reader033/viewer/2022051215/56649ebc5503460f94bc4c22/html5/thumbnails/16.jpg)
16
Partitioning Address Space:Fixed Approach
• Fixed approach:– First MB on host 0, 2nd on host 1, &c.– Send all reads and writes to “owner” of
address– Each CPU read- and write-protects pages
in address ranges held by other CPUs• Detect reads and writes to remote pages
with VM hardware• What if we placed pages on hosts
poorly?• Can’t always predict which hosts will
use which pages
![Page 17: Distributed Shared Memory: Ivy & memcached Costin Raiciu Advanced Topics in Distributed Systems 2012 Slides courtesy of Brad Karp (UCL) and Robert Morris](https://reader033.vdocuments.mx/reader033/viewer/2022051215/56649ebc5503460f94bc4c22/html5/thumbnails/17.jpg)
17
Partitioning Address Space:Dynamic, Single-Copy Approach
• Move the page to the reading/writing CPU each time it is used
• CPU trying to read or write must find current owner, then take page from it
• Requires mechanism to find current location of page
• What if many CPUs read same page?
![Page 18: Distributed Shared Memory: Ivy & memcached Costin Raiciu Advanced Topics in Distributed Systems 2012 Slides courtesy of Brad Karp (UCL) and Robert Morris](https://reader033.vdocuments.mx/reader033/viewer/2022051215/56649ebc5503460f94bc4c22/html5/thumbnails/18.jpg)
18
Partitioning Address Space:Dynamic, Multi-Copy Approach
• Move page for writes, but allow read-only copies
• When CPU reads page it doesn’t have in its own local memory, find other CPU that most recently wrote to page
• Works if pages are read-only and shared or read-write by one host
• Bad case: write sharing– When does write sharing occur?– False sharing, too…
![Page 19: Distributed Shared Memory: Ivy & memcached Costin Raiciu Advanced Topics in Distributed Systems 2012 Slides courtesy of Brad Karp (UCL) and Robert Morris](https://reader033.vdocuments.mx/reader033/viewer/2022051215/56649ebc5503460f94bc4c22/html5/thumbnails/19.jpg)
19
Simple Ivy:Centralized Manager (Section 3.1)
CPU0
CPU1
CPU2 / MGR
lock access
owner?
lock access
owner?
lock access
owner?
lock copy_set
owner
ptable info
• ptable (all CPUs)access: R, W, or
nilowner: T or F
• info (MGR only)copy_set: list of
CPUs with read-only copies
owner: CPU that can write page
![Page 20: Distributed Shared Memory: Ivy & memcached Costin Raiciu Advanced Topics in Distributed Systems 2012 Slides courtesy of Brad Karp (UCL) and Robert Morris](https://reader033.vdocuments.mx/reader033/viewer/2022051215/56649ebc5503460f94bc4c22/html5/thumbnails/20.jpg)
20
Centralized Manager (2):Message Types Between CPUs
• RQ (read query, reader to MGR)• RF (read forward, MGR to owner)• RD (read data, owner to reader)• RC (read confirm, reader to MGR)• WQ (write query, writer to MGR)• IV (invalidate, MGR to copy_set)• IC (invalidate confirm, copy_set to MGR)• WF (write forward, MGR to owner)• WD (write data, owner to writer)• WC (write confirm, writer to MGR)
![Page 21: Distributed Shared Memory: Ivy & memcached Costin Raiciu Advanced Topics in Distributed Systems 2012 Slides courtesy of Brad Karp (UCL) and Robert Morris](https://reader033.vdocuments.mx/reader033/viewer/2022051215/56649ebc5503460f94bc4c22/html5/thumbnails/21.jpg)
21
Centralized Manager Example 1:Owned by CPU0, CPU1 wants to
read
CPU0
CPU1
CPU2 / MGR
lock access
owner?
F nil F
…
lock access
owner?
F nil F
…
lock access
owner?
F W T
…
lock copy_set
owner
F {} CPU0
…ptable info
read
![Page 22: Distributed Shared Memory: Ivy & memcached Costin Raiciu Advanced Topics in Distributed Systems 2012 Slides courtesy of Brad Karp (UCL) and Robert Morris](https://reader033.vdocuments.mx/reader033/viewer/2022051215/56649ebc5503460f94bc4c22/html5/thumbnails/22.jpg)
22
Centralized Manager Example 1:Owned by CPU0, CPU1 wants to
read
CPU0
CPU1
CPU2 / MGR
lock access
owner?
T nil F
…
lock access
owner?
F nil F
…
lock access
owner?
F W T
…
lock copy_set
owner
F {} CPU0
…ptable info
read
RQ
![Page 23: Distributed Shared Memory: Ivy & memcached Costin Raiciu Advanced Topics in Distributed Systems 2012 Slides courtesy of Brad Karp (UCL) and Robert Morris](https://reader033.vdocuments.mx/reader033/viewer/2022051215/56649ebc5503460f94bc4c22/html5/thumbnails/23.jpg)
23
Centralized Manager Example 1:Owned by CPU0, CPU1 wants to
read
CPU0
CPU1
CPU2 / MGR
lock access
owner?
T nil F
…
lock access
owner?
F nil F
…
lock access
owner?
F W T
…
lock copy_set
owner
T {} CPU0
…ptable info
read
RQ
![Page 24: Distributed Shared Memory: Ivy & memcached Costin Raiciu Advanced Topics in Distributed Systems 2012 Slides courtesy of Brad Karp (UCL) and Robert Morris](https://reader033.vdocuments.mx/reader033/viewer/2022051215/56649ebc5503460f94bc4c22/html5/thumbnails/24.jpg)
24
Centralized Manager Example 1:Owned by CPU0, CPU1 wants to
read
CPU0
CPU1
CPU2 / MGR
lock access
owner?
T nil F
…
lock access
owner?
F nil F
…
lock access
owner?
F W T
…
lock copy_set
owner
T {CPU1}
CPU0
…ptable info
read
RQ
RF
![Page 25: Distributed Shared Memory: Ivy & memcached Costin Raiciu Advanced Topics in Distributed Systems 2012 Slides courtesy of Brad Karp (UCL) and Robert Morris](https://reader033.vdocuments.mx/reader033/viewer/2022051215/56649ebc5503460f94bc4c22/html5/thumbnails/25.jpg)
25
Centralized Manager Example 1:Owned by CPU0, CPU1 wants to
read
CPU0
CPU1
CPU2 / MGR
lock access
owner?
T nil F
…
lock access
owner?
F nil F
…
lock access
owner?
T W T
…
lock copy_set
owner
T {CPU1}
CPU0
…ptable info
read
RQ
RF
![Page 26: Distributed Shared Memory: Ivy & memcached Costin Raiciu Advanced Topics in Distributed Systems 2012 Slides courtesy of Brad Karp (UCL) and Robert Morris](https://reader033.vdocuments.mx/reader033/viewer/2022051215/56649ebc5503460f94bc4c22/html5/thumbnails/26.jpg)
26
Centralized Manager Example 1:Owned by CPU0, CPU1 wants to
read
CPU0
CPU1
CPU2 / MGR
lock access
owner?
T nil F
…
lock access
owner?
F nil F
…
lock access
owner?
T R T
…
lock copy_set
owner
T {CPU1}
CPU0
…ptable info
read
RQ
RF
RD
![Page 27: Distributed Shared Memory: Ivy & memcached Costin Raiciu Advanced Topics in Distributed Systems 2012 Slides courtesy of Brad Karp (UCL) and Robert Morris](https://reader033.vdocuments.mx/reader033/viewer/2022051215/56649ebc5503460f94bc4c22/html5/thumbnails/27.jpg)
27
Centralized Manager Example 1:Owned by CPU0, CPU1 wants to
read
CPU0
CPU1
CPU2 / MGR
lock access
owner?
T nil F
…
lock access
owner?
F nil F
…
lock access
owner?
F R T
…
lock copy_set
owner
T {CPU1}
CPU0
…ptable info
read
RQ
RF
RD
RC
![Page 28: Distributed Shared Memory: Ivy & memcached Costin Raiciu Advanced Topics in Distributed Systems 2012 Slides courtesy of Brad Karp (UCL) and Robert Morris](https://reader033.vdocuments.mx/reader033/viewer/2022051215/56649ebc5503460f94bc4c22/html5/thumbnails/28.jpg)
28
Centralized Manager Example 1:Owned by CPU0, CPU1 wants to
read
CPU0
CPU1
CPU2 / MGR
lock access
owner?
T R F
…
lock access
owner?
F nil F
…
lock access
owner?
F R T
…
lock copy_set
owner
T {CPU1}
CPU0
…ptable info
read
RQ
RF
RD
RC
![Page 29: Distributed Shared Memory: Ivy & memcached Costin Raiciu Advanced Topics in Distributed Systems 2012 Slides courtesy of Brad Karp (UCL) and Robert Morris](https://reader033.vdocuments.mx/reader033/viewer/2022051215/56649ebc5503460f94bc4c22/html5/thumbnails/29.jpg)
29
Centralized Manager Example 1:Owned by CPU0, CPU1 wants to
read
CPU0
CPU1
CPU2 / MGR
lock access
owner?
F R F
…
lock access
owner?
F nil F
…
lock access
owner?
F R T
…
lock copy_set
owner
T {CPU1}
CPU0
…ptable info
read
RQ
RF
RD
RC
![Page 30: Distributed Shared Memory: Ivy & memcached Costin Raiciu Advanced Topics in Distributed Systems 2012 Slides courtesy of Brad Karp (UCL) and Robert Morris](https://reader033.vdocuments.mx/reader033/viewer/2022051215/56649ebc5503460f94bc4c22/html5/thumbnails/30.jpg)
30
Centralized Manager Example 1:Owned by CPU0, CPU1 wants to
read
CPU0
CPU1
CPU2 / MGR
lock access
owner?
F R F
…
lock access
owner?
F nil F
…
lock access
owner?
F R T
…
lock copy_set
owner
F {CPU1}
CPU0
…ptable info
read
RQ
RF
RD
RC
![Page 31: Distributed Shared Memory: Ivy & memcached Costin Raiciu Advanced Topics in Distributed Systems 2012 Slides courtesy of Brad Karp (UCL) and Robert Morris](https://reader033.vdocuments.mx/reader033/viewer/2022051215/56649ebc5503460f94bc4c22/html5/thumbnails/31.jpg)
31
Centralized Manager Example 2:Owned by CPU0, CPU2 wants to
write
CPU0
CPU1
CPU2 / MGR
lock access
owner?
F R F
…
lock access
owner?
F nil F
…
lock access
owner?
F R T
…
lock copy_set
owner
F {CPU1}
CPU0
…ptable info
write
![Page 32: Distributed Shared Memory: Ivy & memcached Costin Raiciu Advanced Topics in Distributed Systems 2012 Slides courtesy of Brad Karp (UCL) and Robert Morris](https://reader033.vdocuments.mx/reader033/viewer/2022051215/56649ebc5503460f94bc4c22/html5/thumbnails/32.jpg)
32
Centralized Manager Example 2:Owned by CPU0, CPU2 wants to
write
CPU0
CPU1
CPU2 / MGR
lock access
owner?
F R F
…
lock access
owner?
T nil F
…
lock access
owner?
F R T
…
lock copy_set
owner
F {CPU1}
CPU0
…ptable info
write
WQ
![Page 33: Distributed Shared Memory: Ivy & memcached Costin Raiciu Advanced Topics in Distributed Systems 2012 Slides courtesy of Brad Karp (UCL) and Robert Morris](https://reader033.vdocuments.mx/reader033/viewer/2022051215/56649ebc5503460f94bc4c22/html5/thumbnails/33.jpg)
33
Centralized Manager Example 2:Owned by CPU0, CPU2 wants to
write
CPU0
CPU1
CPU2 / MGR
lock access
owner?
F R F
…
lock access
owner?
T nil F
…
lock access
owner?
F R T
…
lock copy_set
owner
T {CPU1}
CPU0
…ptable info
write
WQ
IV
![Page 34: Distributed Shared Memory: Ivy & memcached Costin Raiciu Advanced Topics in Distributed Systems 2012 Slides courtesy of Brad Karp (UCL) and Robert Morris](https://reader033.vdocuments.mx/reader033/viewer/2022051215/56649ebc5503460f94bc4c22/html5/thumbnails/34.jpg)
34
Centralized Manager Example 2:Owned by CPU0, CPU2 wants to
write
CPU0
CPU1
CPU2 / MGR
lock access
owner?
T nil F
…
lock access
owner?
T nil F
…
lock access
owner?
F R T
…
lock copy_set
owner
F {CPU1}
CPU0
…ptable info
write
WQ
IVIC
![Page 35: Distributed Shared Memory: Ivy & memcached Costin Raiciu Advanced Topics in Distributed Systems 2012 Slides courtesy of Brad Karp (UCL) and Robert Morris](https://reader033.vdocuments.mx/reader033/viewer/2022051215/56649ebc5503460f94bc4c22/html5/thumbnails/35.jpg)
35
Centralized Manager Example 2:Owned by CPU0, CPU2 wants to
write
CPU0
CPU1
CPU2 / MGR
lock access
owner?
F nil F
…
lock access
owner?
T nil F
…
lock access
owner?
F R T
…
lock copy_set
owner
T {} CPU0
…ptable info
write
WQ
IVIC
WF
![Page 36: Distributed Shared Memory: Ivy & memcached Costin Raiciu Advanced Topics in Distributed Systems 2012 Slides courtesy of Brad Karp (UCL) and Robert Morris](https://reader033.vdocuments.mx/reader033/viewer/2022051215/56649ebc5503460f94bc4c22/html5/thumbnails/36.jpg)
36
Centralized Manager Example 2:Owned by CPU0, CPU2 wants to
write
CPU0
CPU1
CPU2 / MGR
lock access
owner?
F nil F
…
lock access
owner?
T nil F
…
lock access
owner?
T nil F
…
lock copy_set
owner
T {} CPU0
…ptable info
write
WQ
IVIC
WF
WD
![Page 37: Distributed Shared Memory: Ivy & memcached Costin Raiciu Advanced Topics in Distributed Systems 2012 Slides courtesy of Brad Karp (UCL) and Robert Morris](https://reader033.vdocuments.mx/reader033/viewer/2022051215/56649ebc5503460f94bc4c22/html5/thumbnails/37.jpg)
37
Centralized Manager Example 2:Owned by CPU0, CPU2 wants to
write
CPU0
CPU1
CPU2 / MGR
lock access
owner?
F nil F
…
lock access
owner?
T W T
…
lock access
owner?
F nil F
…
lock copy_set
owner
T {} CPU0
…ptable info
write
WQ
IVIC
WF
WD
WC
![Page 38: Distributed Shared Memory: Ivy & memcached Costin Raiciu Advanced Topics in Distributed Systems 2012 Slides courtesy of Brad Karp (UCL) and Robert Morris](https://reader033.vdocuments.mx/reader033/viewer/2022051215/56649ebc5503460f94bc4c22/html5/thumbnails/38.jpg)
38
Centralized Manager Example 2:Owned by CPU0, CPU2 wants to
write
CPU0
CPU1
CPU2 / MGR
lock access
owner?
F nil F
…
lock access
owner?
F W T
…
lock access
owner?
F nil F
…
lock copy_set
owner
T {} CPU2
…ptable info
write
WQ
IVIC
WF
WD
WC
![Page 39: Distributed Shared Memory: Ivy & memcached Costin Raiciu Advanced Topics in Distributed Systems 2012 Slides courtesy of Brad Karp (UCL) and Robert Morris](https://reader033.vdocuments.mx/reader033/viewer/2022051215/56649ebc5503460f94bc4c22/html5/thumbnails/39.jpg)
39
What if Two CPUs Want to Write to Same Page at Same Time?
• Write has several steps, modifies multiple tables
• Invariants for tables:– MGR must agree with CPUs about single
owner– MGR must agree with CPUs about
copy_set– copy_set != {} must agree with read-
only for owner• Write operation should thus be
atomic!• What enforces atomicity?
![Page 40: Distributed Shared Memory: Ivy & memcached Costin Raiciu Advanced Topics in Distributed Systems 2012 Slides courtesy of Brad Karp (UCL) and Robert Morris](https://reader033.vdocuments.mx/reader033/viewer/2022051215/56649ebc5503460f94bc4c22/html5/thumbnails/40.jpg)
40
Sequential Consistency: Definition
• Must exist total order of operations such that:– All CPUs see results consistent with that total
order (i.e., LDs see most recent ST in total order)
– Each CPU’s instructions appear in order in total order
• Two rules sufficient to implement sequential consistency [Lamport, 1979]:– Each CPU must execute reads and writes in
program order, one at a time– Each memory location must execute reads and
writes in arrival order, one at a time
![Page 41: Distributed Shared Memory: Ivy & memcached Costin Raiciu Advanced Topics in Distributed Systems 2012 Slides courtesy of Brad Karp (UCL) and Robert Morris](https://reader033.vdocuments.mx/reader033/viewer/2022051215/56649ebc5503460f94bc4c22/html5/thumbnails/41.jpg)
41
Ivy and Consistency Models
• Consider done{0,1,2} example:– v0 = fn0(); done0 = true– In Ivy, can other CPU see done ==
true, but still see old v0?• Does Ivy obey sequential
consistency?– Yes!– Each CPU does R/W in program
order– Each memory location does R/W in
arrival order
![Page 42: Distributed Shared Memory: Ivy & memcached Costin Raiciu Advanced Topics in Distributed Systems 2012 Slides courtesy of Brad Karp (UCL) and Robert Morris](https://reader033.vdocuments.mx/reader033/viewer/2022051215/56649ebc5503460f94bc4c22/html5/thumbnails/42.jpg)
Ivy: Optimizations
• Centralized manager– Read/write access without going through
manager (removes the need for confirm messages)
• Distributed Manager– Static: hash-based allocation of
ownership– Dynamic 1: broadcast allocation– Dynamic 2: probable owner chains– Dynamic 3: mix of 1 and 2
42
![Page 43: Distributed Shared Memory: Ivy & memcached Costin Raiciu Advanced Topics in Distributed Systems 2012 Slides courtesy of Brad Karp (UCL) and Robert Morris](https://reader033.vdocuments.mx/reader033/viewer/2022051215/56649ebc5503460f94bc4c22/html5/thumbnails/43.jpg)
43
Ivy: Evaluation
• Experiments include performance of matrix multiplication, and “block odd-even based merge-split algorithm”
• How to measure performance?– Speedup: x-axis is number of CPUs used,
y-axis is how many times faster the program ran with that many CPUs
• What’s the best speedup you should ever expect?– Linear
When do you expect speedup to be linear?
![Page 44: Distributed Shared Memory: Ivy & memcached Costin Raiciu Advanced Topics in Distributed Systems 2012 Slides courtesy of Brad Karp (UCL) and Robert Morris](https://reader033.vdocuments.mx/reader033/viewer/2022051215/56649ebc5503460f94bc4c22/html5/thumbnails/44.jpg)
44
What’s “Block Odd-Even Based Merge-Split Algorithm?”
• Partition data to be sorted over N CPUs, held in one shared array
• Sort data in each CPU locally• View CPUs as in a line, number 0 to N-1• Repeat N times:
– Even CPUs send to (higher) odd CPUs– Odd CPUs merge, send lower half back to even
CPUs– Odd CPUs send to (higher) even CPUs– Even CPUs merge, send lower half back to odd
CPUs• “Send” just means “receiver reads from
right place in shared memory”
![Page 45: Distributed Shared Memory: Ivy & memcached Costin Raiciu Advanced Topics in Distributed Systems 2012 Slides courtesy of Brad Karp (UCL) and Robert Morris](https://reader033.vdocuments.mx/reader033/viewer/2022051215/56649ebc5503460f94bc4c22/html5/thumbnails/45.jpg)
45
Ivy’s Speedup
• Matrix multiplication: linear• Sorting: worse than linear,
flattens significantly beyond 2 CPUs
![Page 46: Distributed Shared Memory: Ivy & memcached Costin Raiciu Advanced Topics in Distributed Systems 2012 Slides courtesy of Brad Karp (UCL) and Robert Morris](https://reader033.vdocuments.mx/reader033/viewer/2022051215/56649ebc5503460f94bc4c22/html5/thumbnails/46.jpg)
46
Ivy vs. RPC
• When would you prefer DSM to RPC?– More transparent– Easier to program for
• When would you prefer RPC to DSM?– Isolation– Control over communication– Latency-tolerance– Portability
• Could Ivy benefit from RPC?– Possibly for efficient blocking/unblocking
![Page 47: Distributed Shared Memory: Ivy & memcached Costin Raiciu Advanced Topics in Distributed Systems 2012 Slides courtesy of Brad Karp (UCL) and Robert Morris](https://reader033.vdocuments.mx/reader033/viewer/2022051215/56649ebc5503460f94bc4c22/html5/thumbnails/47.jpg)
47
DSM: Successful Idea?
• Spreading a computation across workstations?– Yes! Google, Inktomi, Beowulf, …
• Coherent access to shared memory?– Yes! Multi-CPU PCs use Ivy-like protocols
for cache coherence between CPUs• DSM as model for programming
workstation cluster?– Little evidence of broad adoption– Too little control over communication,
and communication dictates performance
![Page 48: Distributed Shared Memory: Ivy & memcached Costin Raiciu Advanced Topics in Distributed Systems 2012 Slides courtesy of Brad Karp (UCL) and Robert Morris](https://reader033.vdocuments.mx/reader033/viewer/2022051215/56649ebc5503460f94bc4c22/html5/thumbnails/48.jpg)
Memcached
48
![Page 49: Distributed Shared Memory: Ivy & memcached Costin Raiciu Advanced Topics in Distributed Systems 2012 Slides courtesy of Brad Karp (UCL) and Robert Morris](https://reader033.vdocuments.mx/reader033/viewer/2022051215/56649ebc5503460f94bc4c22/html5/thumbnails/49.jpg)
Memcached: a different Distributed Shared Memory
• Distributed caching system
• Open source, high-performance
• Widely used in practice– To speed up web applications– Client API available for most
programming languages
49
![Page 50: Distributed Shared Memory: Ivy & memcached Costin Raiciu Advanced Topics in Distributed Systems 2012 Slides courtesy of Brad Karp (UCL) and Robert Morris](https://reader033.vdocuments.mx/reader033/viewer/2022051215/56649ebc5503460f94bc4c22/html5/thumbnails/50.jpg)
Memcached Overview
• Pools the memory of datacenter machines
• Servers provide a key-value store API• Clients have to choose from different
servers– Send put/get commands
50
![Page 51: Distributed Shared Memory: Ivy & memcached Costin Raiciu Advanced Topics in Distributed Systems 2012 Slides courtesy of Brad Karp (UCL) and Robert Morris](https://reader033.vdocuments.mx/reader033/viewer/2022051215/56649ebc5503460f94bc4c22/html5/thumbnails/51.jpg)
Memcached Servers
• No coordination between servers– Ensures scalability: more servers, more
memory, better performance!• Servers are agnostic to the format of
the data– They just receive blobs for keys and
values
51
![Page 52: Distributed Shared Memory: Ivy & memcached Costin Raiciu Advanced Topics in Distributed Systems 2012 Slides courtesy of Brad Karp (UCL) and Robert Morris](https://reader033.vdocuments.mx/reader033/viewer/2022051215/56649ebc5503460f94bc4c22/html5/thumbnails/52.jpg)
Memcached API
• Storage commands – set/add/replace (key, value)– append/prepend (key, value)– cas (check and set)– delete (key)
• Retrieval commands– get (key(s))– gets– incr / decr
52
![Page 53: Distributed Shared Memory: Ivy & memcached Costin Raiciu Advanced Topics in Distributed Systems 2012 Slides courtesy of Brad Karp (UCL) and Robert Morris](https://reader033.vdocuments.mx/reader033/viewer/2022051215/56649ebc5503460f94bc4c22/html5/thumbnails/53.jpg)
Cache replacement policy
• Do we care about coherence in memcached?
• Least Recently Used replacement policy
• Used can also specify how long the object should be cached
53
![Page 54: Distributed Shared Memory: Ivy & memcached Costin Raiciu Advanced Topics in Distributed Systems 2012 Slides courtesy of Brad Karp (UCL) and Robert Morris](https://reader033.vdocuments.mx/reader033/viewer/2022051215/56649ebc5503460f94bc4c22/html5/thumbnails/54.jpg)
Memcached Protocol
• Request – response, runs over UDP or TCP
• Key size – limited to 250B• Data – under 1MB
54
![Page 55: Distributed Shared Memory: Ivy & memcached Costin Raiciu Advanced Topics in Distributed Systems 2012 Slides courtesy of Brad Karp (UCL) and Robert Morris](https://reader033.vdocuments.mx/reader033/viewer/2022051215/56649ebc5503460f94bc4c22/html5/thumbnails/55.jpg)
How is data load-balanced across servers?
• Clients decide how to split data across servers
• They can use– Plain hashing– Consistent hashing
55
![Page 56: Distributed Shared Memory: Ivy & memcached Costin Raiciu Advanced Topics in Distributed Systems 2012 Slides courtesy of Brad Karp (UCL) and Robert Morris](https://reader033.vdocuments.mx/reader033/viewer/2022051215/56649ebc5503460f94bc4c22/html5/thumbnails/56.jpg)
Code example (php)
$data_for_front_page = $db->getDataForFrontPage();$query = “Select * from pages where …”;$key = sha1($query); if($key FOUND IN memcached){
$value = get($key); $resultset = unserialize($value); return $resultset;
}else {
$resultset = mysql_query($query); set($key, serialize($resultset), $TTL); return $resultset;
}
56
![Page 57: Distributed Shared Memory: Ivy & memcached Costin Raiciu Advanced Topics in Distributed Systems 2012 Slides courtesy of Brad Karp (UCL) and Robert Morris](https://reader033.vdocuments.mx/reader033/viewer/2022051215/56649ebc5503460f94bc4c22/html5/thumbnails/57.jpg)
Possible problems
• What happens on failure?
• What happens on restart?
57
![Page 58: Distributed Shared Memory: Ivy & memcached Costin Raiciu Advanced Topics in Distributed Systems 2012 Slides courtesy of Brad Karp (UCL) and Robert Morris](https://reader033.vdocuments.mx/reader033/viewer/2022051215/56649ebc5503460f94bc4c22/html5/thumbnails/58.jpg)
Conclusions
• DSM is a powerful concept, but it can be instantiated in more than one way
• Ivy-like algorithms are used for cache coherence today
• Memcached is used to allow “pooling” of server memory in a cluster
58