the push/pull model of transactions - yale university · the push/pull model of transactions eric...
TRANSCRIPT
The Push/Pull Model of Transactions
Eric KoskinenYale University, New Haven
United States
Matthew ParkinsonMicrosoft Research, Cambridge
United Kingdom
Appeared in PLDI 2015
CertiKOS Weekly Lunch 16 October 2015
Concurrent HashMap
Concurrent Queue
Concurrent List
Thread 1
Thread 3
Thread 2
Thread 4
put get enq deq add rm
Shared Memory
rd wr
Concurrent HashMap
Concurrent Queue
Concurrent List
Thread 1
Thread 3
Thread 2
Thread 4
put get enq deq add rm
Shared Memory
rd wr
Adding atomic to a language . . .
Version 4.7
Serializable. Opaque. …
Concurrent HashMap
Concurrent Queue
Concurrent List
Thread 1
Thread 3
Thread 2
Thread 4
put get enq deq add rm
Shared Memory
rd wr
Adding atomic to a language . . .
Concurrent HashMap
Concurrent Queue
Concurrent List
Thread 1
Thread 3
Thread 2
Thread 4
put get enq deq add rm
Shared Memory
rd wr
many issues arise in the implementation.
Serializable. Opaque. …? ? ?
Adding atomic to a language . . .
Pessimism!Thread
atomic { x := 3; y := 9; }
x changed?
y changed?
Thread
atomic { x := 3; y := 9; }all ok?
Optimism!
Serializability!
Interleaved execution equivalent to some
serial execution.
Opacity!
Interleaved execution equivalent to some
serial execution and
cannot observe intermediate state.
Pessimism!
Optimism!
Dependent Txns
ADT Ops (Boosting)
Thread
atomic { x := 3; y := 9; }
Thread
atomic { stk.push(4); ht.get('a'); }
Serializability!
Opacity!Pessimism!
Optimism!Memory Ops
ADT Ops (Boosting)
Serializability!
Opacity!Pessimism!
Optimism!Memory Ops
Hardware TMSoftware TM
Hybrid TM
Dependent Txns
ADT Ops (Boosting)
Serializability!
Opacity!Pessimism!
Optimism!Memory Ops
Hardware TMSoftware TM
Hybrid TM Open Nested
Dependent Txns
Irrev
ocab
ility
Consistency
Eager Writes
Lazy Reads
ADT Ops (Boosting)
Serializability!
Opacity!Pessimism!
Optimism!Memory Ops
Hardware TMSoftware TM
Hybrid TM Open Nested
Dependent Txns
Irrev
ocab
ility
Consistency
Eager Writes
Lazy Reads
Serializable. Opaque. …? ? ?
The PUSH/PULL Model
Push⟨ht.get(5),_⟩
Pull⟨ht.map(3,x),_⟩⟨ht.map(3,x),_⟩
⟨q.enq(‘a’),_⟩ ⟨q.enq(‘a’),_,gUC⟩⟨ht.map(3,x),_,gUC⟩⟨ht.map(7,2),_,gUC⟩
The PUSH/PULL Model
Push⟨ht.get(5),_⟩
Pull⟨ht.map(3,x),_⟩⟨ht.map(3,x),_⟩
⟨q.enq(‘a’),_⟩ ⟨q.enq(‘a’),_,gUC⟩⟨ht.map(3,x),_,gUC⟩⟨ht.map(7,2),_,gUC⟩
Serializable. …Opaque.
Serializable. …Opaque.
Linearizable Commute
The PUSH/PULL Model
Push⟨ht.get(5),_⟩
Pull⟨ht.map(3,x),_⟩⟨ht.map(3,x),_⟩
⟨q.enq(‘a’),_⟩ ⟨q.enq(‘a’),_,gUC⟩⟨ht.map(3,x),_,gUC⟩⟨ht.map(7,2),_,gUC⟩
Threadatomic { if(b) { opA(…); } else … }
Threadatomic { while(…){ if(x>MAX) opB(…); … }Linearizable
OperationLinearizable Operation
1. Abstract away the language semantics
Thread Thread
2. Abstract away the state (with logs)
{tx c1, σ1, L1} {tx c2, σ2, L2}G
⟨op1,_⟩⟨op2,_⟩
1. Abstract away the language semantics
Thread Thread
2. Abstract away the state (with logs)
{tx c1, σ1, L1} {tx c2, σ2, L2}G
Closed under log prefix
Parameterized by predicate allowed
⟨op1,_⟩⟨op2,_⟩
Thread Thread
{tx c1, σ1, L1} {tx c2, σ2, L2}G
Closed under log prefix
Parameterized by predicate allowed
Log Equality
⟨op1,_⟩⟨op2,_⟩
Thread Thread
{tx c1, σ1, L1} {tx c2, σ2, L2}G
Log Equality
⟨op1,_⟩⟨op2,_⟩
7 Simple Rules:
Apply Push Pull
Unpull Unpush Unapply
Commit
Apply Push Pull Unpull Unpush Unapply Commit
Thread Thread
{tx c1, σ1, L1} {tx c2, σ2, L2}Two Logs
atomic { op1; op2; … }
atomic { op3; op4; … }
G
⟨ht.map(3,x),σ,σ1,c⟩⟨ht.map(7,2),_,gUC⟩
Apply Push Pull Unpull Unpush Unapply Commit
Thread Thread
{tx c1, σ1, L1} {tx c2, σ2, L2}
atomic { op1; op2; … }
atomic { op3; op4; … }A
pp
ly
G
1
⟨ht.map(3,x),σ,σ1,c⟩⟨q.enq(‘a’),σ1,σ'1,c1⟩
⟨ht.map(7,2),_,gUC⟩
1
Apply Push Pull Unpull Unpush Unapply Commit
Thread Thread
{tx c1, σ1, L1} {tx c2, σ2, L2}
atomic { op1; op2; … }
atomic { op3; op4; … }A
pp
ly
G
Criterion (i):Criterion (ii):Criterion (iii):
L1 allows ⟨m,σ1,σ'1,id⟩
fresh(id)
APPLY
1
⟨ht.map(3,x),σ,σ1,c⟩⟨q.enq(‘a’),σ1,σ'1,c1⟩
⟨ht.map(7,2),_,gUC⟩
1
{tx c1, σ1, L1}, G ➝
{tx c'1, σ'1, L1·[op,σ1,σ'1,c1]}, G
append
Possible next op
Valid semantics of log
Apply Push Pull Unpull Unpush Unapply Commit
Thread Thread
{tx c1, σ1, L1} {tx c2, σ2, L2}
atomic { op1; op2; … }
atomic { op3; op4; … }U
na
pp
ly
G
UNAPPLY
⟨ht.map(7,2),_,gUC⟩⟨ht.map(3,x),σ,σ1,c⟩⟨q.enq(‘a’),σ1,σ'1,c1⟩
{tx c'1, σ'1, L1·[op,σ1,σ'1,c1]}, G ➝
{tx c1, σ1, L1}, G
Apply Push Pull Unpull Unpush Unapply Commit
Thread Thread
{tx c1, σ1, L1} {tx c2, σ2, L2}Two Logs
atomic { op1; op2; … }
atomic { op3; op4; … }
⟨ht.map(3,x),_⟩⟨q.enq(‘a’),_⟩
G
⟨ht.map(7,2),_,gUC⟩
Apply Push Pull Unpull Unpush Unapply Commit
Thread Thread
{tx c1, σ1, L1} {tx c2, σ2, L2}Two Logs
atomic { op1; op2; … }
atomic { op3; op4; … }
⟨ht.map(3,x),_⟩Push
⟨q.enq(‘a’),_⟩⟨ht.map(3,x),_,gUC⟩
G
⟨ht.map(7,2),_,gUC⟩
Apply Push Pull Unpull Unpush Unapply Commit
Thread Thread
{tx c1, σ1, L1} {tx c2, σ2, L2}Two Logs
atomic { op1; op2; … }
atomic { op3; op4; … }
Push
G
PUSH
Criterion (i):Criterion (ii):Criterion (iii):
op ◀ ⎣`L⎦unpushed
G allows op
Act as if op happens next
No conflict w/ other uncmted
Left-mover over logs⟨ht.map(3,x),_⟩⟨q.enq(‘a’),_⟩
⟨ht.map(3,x),_,gUC⟩⟨ht.map(7,2),_,gUC⟩
{tx c, σ, `L·[op]·L'}, G ➝
{tx c, σ, `L·[op]·L'}, G·[op,gUC]
⎣G⎦gUC ∖ ⎣L1⎦pushed ◀ op
Uncommitted
Application: Out-of-order PUSHing with redo-logsApplication: Optimism vs. Pessimism
Apply Push Pull Unpull Unpush Unapply Commit
Thread Thread
{tx c1, σ1, L1} {tx c2, σ2, L2}
atomic { op1; op2; … }
atomic { op3; op4; … }
⟨ht.map(3,x),_⟩Pull
G
PULL
L2 allows opCriterion (i):Criterion (ii):
op ◀ ⎣L2⎦pushed ∪ ⎣L2⎦unpushedCriterion (iii):
op ∉ L2Didn’t pull already
Local log allows op
Can act as if op happened earlier
⟨ht.map(3,x),_⟩⟨q.enq(‘a’),_⟩
⟨ht.map(3,x),_,gUC⟩⟨ht.map(7,2),_,gUC⟩
{tx c2, σ2, L2}, G1·[op,g]·G2 ➝
{tx c2, σ2, L2·[op]}, G1·[op,g]·G2
Application: Opacity [GK’08] and dependent transactions [RRHW’09]
⟨q.enq(‘a’),_,gUC⟩Push
Apply Push Pull Unpull Unpush Unapply Commit
Thread Thread
{tx c1, σ1, L1} {tx c2, σ2, L2}
atomic { op1; op2; … }
atomic { op3; op4; … }A
pp
ly ⟨ht.get(5),_⟩
G
⟨ht.map(3,x),_⟩⟨ht.map(3,x),_⟩⟨q.enq(‘a’),_⟩ ⟨q.enq(‘a’),_,gUC⟩
⟨ht.map(3,x),_,gUC⟩⟨ht.map(7,2),_,gUC⟩
Apply Push Pull Unpull Unpush Unapply Commit
Thread Thread
{tx c1, σ1, L1} {tx c2, σ2, L2}
atomic { op1; op2; … }
atomic { op3; op4; … }
Push⟨ht.get(5),_,gUC⟩⟨ht.get(5),_⟩
Pull
G
⟨ht.get(5),_⟩⟨ht.map(3,x),_⟩⟨ht.map(3,x),_⟩
⟨q.enq(‘a’),_⟩ ⟨q.enq(‘a’),_,gUC⟩⟨ht.map(3,x),_,gUC⟩⟨ht.map(7,2),_,gUC⟩
Apply Push Pull Unpull Unpush Unapply Commit
Thread Thread
{tx c1, σ1, L1} {tx c2, σ2, L2}
atomic { op1; op2; … }
atomic { op3; op4; … }
⟨ht.get(5),_,gUC⟩⟨ht.get(5),_⟩
G
⟨ht.get(5),_⟩⟨ht.map(3,x),_⟩⟨ht.map(3,x),_⟩
⟨q.enq(‘a’),_⟩ ⟨q.enq(‘a’),_,gUC⟩⟨ht.map(3,x),_,gUC⟩⟨ht.map(7,2),_,gUC⟩
Pull⟨q.enq(‘a’),_⟩
Apply Push Pull Unpull Unpush Unapply Commit
Thread Thread
{tx c1, σ1, L1} {tx c2, σ2, L2}
atomic { op1; op2; … }
atomic { op3; op4; … }
⟨ht.get(5),_,gUC⟩⟨ht.get(5),_⟩
G
⟨ht.get(5),_⟩⟨ht.map(3,x),_⟩⟨ht.map(3,x),_⟩
⟨q.enq(‘a’),_⟩ ⟨q.enq(‘a’),_,gUC⟩⟨ht.map(3,x),_,gUC⟩⟨ht.map(7,2),_,gUC⟩
⟨q.enq(‘a’),_⟩⟨ht.get(9),_⟩
Co
mm
it
Apply Push Pull Unpull Unpush Unapply Commit
Thread Thread
{tx c1, σ1, L1} {tx c2, σ2, L2}
atomic { op1; op2; … }
atomic { op3; op4; … }
⟨ht.get(5),_,gUC⟩⟨ht.get(5),_⟩
G
⟨ht.get(5),_⟩⟨ht.map(3,x),_⟩⟨ht.map(3,x),_⟩
⟨q.enq(‘a’),_⟩ ⟨q.enq(‘a’),_,gUC⟩⟨ht.map(3,x),_,gUC⟩⟨ht.map(7,2),_,gUC⟩
⟨q.enq(‘a’),_⟩⟨ht.get(9),_⟩
Co
mm
it
COMMIT
cmt(G, L1, G')⎣L1⎦pulled ⊆ ⎣G⎦gC
Criterion (ii):Criterion (iii):Criterion (iv):
L1 ⊆ GCriterion (i): fin(c1)
Pushed all my stuff
Pulled ops are committed
Swap my flags from gUC to gC
{tx c1, σ1, L1}, G ➝
{tx c1, σ1, L1}, G’
Apply Push Pull Unpull Unpush Unapply Commit
Thread Thread
{tx c1, σ1, L1} {tx c2, σ2, L2}
atomic { op1; op2; … }
atomic { op3; op4; … }
⟨ht.get(5),_,gUC⟩⟨ht.get(5),_⟩⟨ht.get(5),_⟩
⟨ht.map(3,x),_⟩⟨ht.map(3,x),_⟩⟨q.enq(‘a’),_⟩ ⟨q.enq(‘a’),_,gUC⟩
⟨ht.map(3,x),_,gUC⟩⟨ht.map(7,2),_,gUC⟩
⟨q.enq(‘a’),_⟩⟨
Un
ap
ply
G
Apply Push Pull Unpull Unpush Unapply Commit
Thread Thread
{tx c1, σ1, L1} {tx c2, σ2, L2}
atomic { op1; op2; … }
atomic { op3; op4; … }
⟨ht.get(5),_,gUC⟩⟨ht.get(5),_⟩⟨ht.get(5),_⟩
⟨ht.map(3,x),_⟩⟨ht.map(3,x),_⟩⟨q.enq(‘a’),_⟩ ⟨q.enq(‘a’),_,gUC⟩
⟨ht.map(3,x),_,gUC⟩⟨ht.map(7,2),_,gUC⟩
⟨q.enq(‘a’),_⟩U
np
ull
G
UNPULL
(Local log is still allowed, even with element removed.)
Apply Push Pull Unpull Unpush Unapply Commit
Thread Thread
{tx c1, σ1, L1} {tx c2, σ2, L2}
atomic { op1; op2; … }
atomic { op3; op4; … }
⟨ht.get(5),_,gUC⟩⟨ht.get(5),_⟩
⟨ht.map(3,x),_⟩⟨ht.map(3,x),_⟩⟨q.enq(‘a’),_⟩ ⟨q.enq(‘a’),_,gUC⟩
⟨ht.map(3,x),_,gUC⟩⟨ht.map(7,2),_,gUC⟩
⟨q.enq(‘a’),_⟩
G
Unpush
UNPUSH
(Everything I subsequently pushed could have been pushed earlier.)
Application: Inverses or “compensating actions”
Apply Push Pull Unpull Unpush Unapply Commit
Thread Thread
{tx c1, σ1, L1} {tx c2, σ2, L2}
atomic { op1; op2; … }
atomic { op3; op4; … }
⟨ht.get(5),_,gUC⟩⟨ht.get(5),_⟩
⟨ht.map(3,x),_⟩⟨ht.map(3,x),_⟩⟨q.enq(‘a’),_⟩
⟨ht.map(3,x),_,gUC⟩⟨ht.map(7,2),_,gUC⟩
⟨q.enq(‘a’),_⟩
G
Un
ap
ply
q.deq()
Apply Push Pull Unpull Unpush Unapply Commit
Thread Thread
{tx c1, σ1, L1} {tx c2, σ2, L2}
atomic { op1; op2; … }
atomic { op3; op4; … }
⟨ht.get(5),_,gUC⟩⟨ht.get(5),_⟩
⟨ht.map(3,x),_⟩⟨ht.map(3,x),_⟩⟨q.enq(‘a’),_⟩
⟨ht.map(3,x),_,gUC⟩⟨ht.map(7,2),_,gUC⟩
⟨q.enq(‘a’),_⟩
G
Un
ap
ply
q.deq()
Theorem. The Push/Pull model is serializable.Simulation with uninterleaved machine
Closure under log rewind
Preservation invariant, universal quantification over dropping uncommitted global ops universal quantifying over local log rewind.
The PUSH/PULL Model
Push⟨ht.get(5),_⟩
Pull⟨ht.map(3,x),_⟩⟨ht.map(3,x),_⟩
⟨q.enq(‘a’),_⟩ ⟨q.enq(‘a’),_,gUC⟩⟨ht.map(3,x),_,gUC⟩⟨ht.map(7,2),_,gUC⟩
Theorem. Fragments of The Push/Pull model are opaque.
The PUSH/PULL Model
Push⟨ht.get(5),_⟩
Pull⟨ht.map(3,x),_⟩⟨ht.map(3,x),_⟩
⟨q.enq(‘a’),_⟩ ⟨q.enq(‘a’),_,gUC⟩⟨ht.map(3,x),_,gUC⟩⟨ht.map(7,2),_,gUC⟩
example: Don’t PULL uncommitted effects.
Other Applications.
The PUSH/PULL Model
Push⟨ht.get(5),_⟩
Pull⟨ht.map(3,x),_⟩⟨ht.map(3,x),_⟩
⟨q.enq(‘a’),_⟩ ⟨q.enq(‘a’),_,gUC⟩⟨ht.map(3,x),_,gUC⟩⟨ht.map(7,2),_,gUC⟩
• Pessimistic models of STM. • Pessimistic models of STM/Boosting. • Optimistic models of STM. • Optimistic models of STM/Boosting (TRANSACT’15) • Closed-nested transactions: don’t share until commit. • Dependent transactions: view uncommitted effects but
delay commit (or cascading aborts)
The PUSH/PULL Model
Push⟨ht.get(5),_⟩
Pull⟨ht.map(3,x),_⟩⟨ht.map(3,x),_⟩
⟨q.enq(‘a’),_⟩ ⟨q.enq(‘a’),_,gUC⟩⟨ht.map(3,x),_,gUC⟩⟨ht.map(7,2),_,gUC⟩
Mixing hardware TM with transactional boosting
Models that are yet-to-come.
Serializable. …Opaque.
Linearizable Commute
The PUSH/PULL Model
Push⟨ht.get(5),_⟩
Pull⟨ht.map(3,x),_⟩⟨ht.map(3,x),_⟩
⟨q.enq(‘a’),_⟩ ⟨q.enq(‘a’),_,gUC⟩⟨ht.map(3,x),_,gUC⟩⟨ht.map(7,2),_,gUC⟩