workshop in verification of distributed protocolsodedp/workshop18/ivy-workshop18.pdf · state of...
TRANSCRIPT
![Page 1: Workshop in Verification of Distributed Protocolsodedp/workshop18/ivy-workshop18.pdf · State of the art in formal verification ess Automation Proof Assistants 22 Ultimately limited](https://reader033.vdocuments.mx/reader033/viewer/2022060207/5f03da427e708231d40b1558/html5/thumbnails/1.jpg)
Workshop in Verification of Distributed Protocols
Mooly Sagiv, Oded Padon
08-March-2018
http://www.cs.tau.ac.il/~odedp/workshop18/
http://microsoft.github.io/ivy/
![Page 2: Workshop in Verification of Distributed Protocolsodedp/workshop18/ivy-workshop18.pdf · State of the art in formal verification ess Automation Proof Assistants 22 Ultimately limited](https://reader033.vdocuments.mx/reader033/viewer/2022060207/5f03da427e708231d40b1558/html5/thumbnails/2.jpg)
Administration
• Start-off meeting (today)
• Project teams:
• 2-3 students
• Each team will take different a project, and work independently during the semester
• Meet with Oded / Mooly as needed
• If needed, we’ll have more workshop meeting during the semester
• 14/6 – project presentation meeting
• Each team will present project
• Project must be finished and approved by Oded / Mooly before
2
![Page 3: Workshop in Verification of Distributed Protocolsodedp/workshop18/ivy-workshop18.pdf · State of the art in formal verification ess Automation Proof Assistants 22 Ultimately limited](https://reader033.vdocuments.mx/reader033/viewer/2022060207/5f03da427e708231d40b1558/html5/thumbnails/3.jpg)
Possible Projects
• Use Ivy to verify any distributed / shared memory algorithm
• Paxos variants
• Disk Paxos, Generalised Paxos, EPaxos (see http://paxos.systems/variants.html for ideas)
• Prove reconfiguration / failure recovery / log truncation / liveness
• Mutual Exclusion Algorithms
• Knuth’s Algorithm, Lamport’s Bakery, Patterson, …
• Prove safety and liveness
• Blockchain algorithms
• Algorand, HoneyBadgerBFT, Bitcoin-NG, …
• Improve Ivy
• Experiment with other SMT solvers (e.g. iProver, CVC4, Vampire, SPASS)
3
![Page 4: Workshop in Verification of Distributed Protocolsodedp/workshop18/ivy-workshop18.pdf · State of the art in formal verification ess Automation Proof Assistants 22 Ultimately limited](https://reader033.vdocuments.mx/reader033/viewer/2022060207/5f03da427e708231d40b1558/html5/thumbnails/4.jpg)
Why verify distributed protocols?
• Distributed systems are everywhere
• Safety-critical systems
• Cloud infrastructure
• Blockchain
• Distributed systems are notoriously hard to get right
• Even small protocols can be tricky
• Bugs occur on rare scenarios
• Testing is costly and not sufficient
![Page 5: Workshop in Verification of Distributed Protocolsodedp/workshop18/ivy-workshop18.pdf · State of the art in formal verification ess Automation Proof Assistants 22 Ultimately limited](https://reader033.vdocuments.mx/reader033/viewer/2022060207/5f03da427e708231d40b1558/html5/thumbnails/5.jpg)
Why verify distributed protocols?
• Distributed systems are everywhere
• Safety-critical systems
• Cloud infrastructure
• Blockchain
• Distributed systems are notoriously hard to get right
• Even small protocols can be tricky
• Bugs occur on rare scenarios
• Testing is costly and not sufficient
![Page 6: Workshop in Verification of Distributed Protocolsodedp/workshop18/ivy-workshop18.pdf · State of the art in formal verification ess Automation Proof Assistants 22 Ultimately limited](https://reader033.vdocuments.mx/reader033/viewer/2022060207/5f03da427e708231d40b1558/html5/thumbnails/6.jpg)
Why verify distributed protocols?
• Distributed systems are everywhere
• Safety-critical systems
• Cloud infrastructure
• Blockchain
• Distributed systems are notoriously hard to get right
• Even small protocols can be tricky
• Bugs occur on rare scenarios
• Testing is costly and not sufficient
![Page 7: Workshop in Verification of Distributed Protocolsodedp/workshop18/ivy-workshop18.pdf · State of the art in formal verification ess Automation Proof Assistants 22 Ultimately limited](https://reader033.vdocuments.mx/reader033/viewer/2022060207/5f03da427e708231d40b1558/html5/thumbnails/7.jpg)
![Page 8: Workshop in Verification of Distributed Protocolsodedp/workshop18/ivy-workshop18.pdf · State of the art in formal verification ess Automation Proof Assistants 22 Ultimately limited](https://reader033.vdocuments.mx/reader033/viewer/2022060207/5f03da427e708231d40b1558/html5/thumbnails/8.jpg)
Proving distributed systems is hard
• Amazon [CACM’15] uses TLA+ for testing protocols, but no proofs
• IronFleet [SOSP’15] – verification of Multi-Paxos in Dafny (3.7 person-years)
• Verdi [PLDI’15] – verification of Raft in Coq (50,000 lines of proofs)
Our goal: reduce human effort while maintaining flexibility
Our approach: decompose verification into decidable problems
[CACM’15] Newcombe et al. How Amazon Web Services Uses Formal Methods
[SOSP’15] Hawblitzel et al. IronFleet: proving practical distributed systems correct
[PLDI’15] Wilcox et al. Verdi: a framework for implementing and formally verifying distributed systems
![Page 9: Workshop in Verification of Distributed Protocolsodedp/workshop18/ivy-workshop18.pdf · State of the art in formal verification ess Automation Proof Assistants 22 Ultimately limited](https://reader033.vdocuments.mx/reader033/viewer/2022060207/5f03da427e708231d40b1558/html5/thumbnails/9.jpg)
Automatic verification of infinite-state systems
Property
VerificationIs there a behavior
of S that violates ?
Counterexample Proof
System S
![Page 10: Workshop in Verification of Distributed Protocolsodedp/workshop18/ivy-workshop18.pdf · State of the art in formal verification ess Automation Proof Assistants 22 Ultimately limited](https://reader033.vdocuments.mx/reader033/viewer/2022060207/5f03da427e708231d40b1558/html5/thumbnails/10.jpg)
Property
VerificationIs there a behavior
of S that violates ?
Counterexample Proof
System S
Rice’s Theorem
I can’t decide!
Unknown
Automatic verification of infinite-state systems
![Page 11: Workshop in Verification of Distributed Protocolsodedp/workshop18/ivy-workshop18.pdf · State of the art in formal verification ess Automation Proof Assistants 22 Ultimately limited](https://reader033.vdocuments.mx/reader033/viewer/2022060207/5f03da427e708231d40b1558/html5/thumbnails/11.jpg)
Property
VerificationIs there a behavior
of S that violates ?
Counterexample Proof
System S
Rice’s Theorem
I can’t decide!
Unknown
Automatic verification of infinite-state systems
![Page 12: Workshop in Verification of Distributed Protocolsodedp/workshop18/ivy-workshop18.pdf · State of the art in formal verification ess Automation Proof Assistants 22 Ultimately limited](https://reader033.vdocuments.mx/reader033/viewer/2022060207/5f03da427e708231d40b1558/html5/thumbnails/12.jpg)
Inductive invariants
System S is safe if all the reachable states satisfy the property P = ¬Bad
Reach
System State Space Safety Property
Bad
Initial
![Page 13: Workshop in Verification of Distributed Protocolsodedp/workshop18/ivy-workshop18.pdf · State of the art in formal verification ess Automation Proof Assistants 22 Ultimately limited](https://reader033.vdocuments.mx/reader033/viewer/2022060207/5f03da427e708231d40b1558/html5/thumbnails/13.jpg)
Inductive invariants
System State Space Safety Property
Bad Inv
Initial
System S is safe iff there exists an inductive invariant Inv :
System S is safe if all the reachable states satisfy the property P = ¬Bad
TR
TR
TR
Inv Bad = (Safety)
Init Inv (Initiation)
if Inv and ’ then ’ Inv (Consecution)
![Page 14: Workshop in Verification of Distributed Protocolsodedp/workshop18/ivy-workshop18.pdf · State of the art in formal verification ess Automation Proof Assistants 22 Ultimately limited](https://reader033.vdocuments.mx/reader033/viewer/2022060207/5f03da427e708231d40b1558/html5/thumbnails/14.jpg)
Counterexample To Induction (CTI)
• States σ,σ’ are a CTI of Inv if:
• σ ∈ Inv
• σ’ ∉ Inv
• σ σ’
• A CTI may indicate:• A bug in the system• A bug in the safety property• A bug in the inductive invariant
• Too weak• Too strong
14
Inv
σ ∈ Inv
σ’ ∉ Inv
![Page 15: Workshop in Verification of Distributed Protocolsodedp/workshop18/ivy-workshop18.pdf · State of the art in formal verification ess Automation Proof Assistants 22 Ultimately limited](https://reader033.vdocuments.mx/reader033/viewer/2022060207/5f03da427e708231d40b1558/html5/thumbnails/15.jpg)
Strengthening & weakening from CTI
15
Inv
σ∈Inv
σ’∉Inv
Inv’Inv’ σ’
σ σ
σ'
Strengthening Weakening
![Page 16: Workshop in Verification of Distributed Protocolsodedp/workshop18/ivy-workshop18.pdf · State of the art in formal verification ess Automation Proof Assistants 22 Ultimately limited](https://reader033.vdocuments.mx/reader033/viewer/2022060207/5f03da427e708231d40b1558/html5/thumbnails/16.jpg)
Simple example: loop invariants
x=7, y =6x=3, y =0
x=3, y =2
x=5, y =4x := 1;y := 2;while * do {assert ¬even[x];x := x + y;y := y + 2;}
x=4, y =5
x=2, y =5
x=2, y =3
x=2, y =4
x=3, y =4
x=1, y =2
x=1, y =0
x=1, y =3
x=1, y =1
even[x]
x=1, y =0
TR
![Page 17: Workshop in Verification of Distributed Protocolsodedp/workshop18/ivy-workshop18.pdf · State of the art in formal verification ess Automation Proof Assistants 22 Ultimately limited](https://reader033.vdocuments.mx/reader033/viewer/2022060207/5f03da427e708231d40b1558/html5/thumbnails/17.jpg)
x=7, y =6x=3, y =0
x=3, y =2
x=5, y =4
x=4, y =5
x=2, y =5
x=2, y =3
x=2, y =4
Counterexample to induction (CTI)x=3, y =4
x=1, y =2
x=1, y =0
x=1, y =3
x=1, y =1
even[x]
x=1, y =0
x := 1;y := 2;while * do {assert ¬even[x];x := x + y;y := y + 2;}
Simple example: loop invariants
¬even[x]
TR
![Page 18: Workshop in Verification of Distributed Protocolsodedp/workshop18/ivy-workshop18.pdf · State of the art in formal verification ess Automation Proof Assistants 22 Ultimately limited](https://reader033.vdocuments.mx/reader033/viewer/2022060207/5f03da427e708231d40b1558/html5/thumbnails/18.jpg)
x=7, y =6x=3, y =0
x=3, y =2
x=5, y =4
x=4, y =5
x=2, y =5
x=2, y =3
x=2, y =4
x=3, y =4
x=1, y =2
x=1, y =0
x=1, y =3
x=1, y =1
even[x]
x=1, y =0
Inv = ¬even[x] ∧ even[y]
x := 1;y := 2;while * do {assert ¬even[x];x := x + y;y := y + 2;}
Simple example: loop invariants
TR
![Page 19: Workshop in Verification of Distributed Protocolsodedp/workshop18/ivy-workshop18.pdf · State of the art in formal verification ess Automation Proof Assistants 22 Ultimately limited](https://reader033.vdocuments.mx/reader033/viewer/2022060207/5f03da427e708231d40b1558/html5/thumbnails/19.jpg)
x=7, y =6x=3, y =0
x=3, y =2
x=5, y =4
x=4, y =5
x=2, y =5
x=2, y =3
x=2, y =4
x=3, y =4
x=1, y =2
x=1, y =0
x=1, y =3
x=1, y =1
even[x]
x=1, y =0
Inv = ¬even[x] ∧ even[y]
Simple example: loop invariants
x := 1;y := 2;while * do {assert ¬even[x];x:=(x*x–y*y)/(x-y);y := y + 2;}
TR
![Page 20: Workshop in Verification of Distributed Protocolsodedp/workshop18/ivy-workshop18.pdf · State of the art in formal verification ess Automation Proof Assistants 22 Ultimately limited](https://reader033.vdocuments.mx/reader033/viewer/2022060207/5f03da427e708231d40b1558/html5/thumbnails/20.jpg)
Challenges in Deductive Verification
1. Formal specification: formalizing infinite-state systems
• Modeling the system and property (TR, Init, Bad)
2. Deduction: checking inductiveness
• Undecidability of implication checking
• Unbounded state (threads, messages), arithmetic, quantifier alternation
3. Inference: inferring inductive invariants (Inv)
• Hard to specify
• Hard to infer
• Undecidable even when deduction is decidable
20
![Page 21: Workshop in Verification of Distributed Protocolsodedp/workshop18/ivy-workshop18.pdf · State of the art in formal verification ess Automation Proof Assistants 22 Ultimately limited](https://reader033.vdocuments.mx/reader033/viewer/2022060207/5f03da427e708231d40b1558/html5/thumbnails/21.jpg)
State of the art in formal verification
Exp
ress
iven
ess
Automation
Proof Assistants
21
Ultimately limited by human
“the proofs consisted of about 5000 lines and assumed several nontrivial invariants of the Raft protocol. This paper discusses the verification of Raft as a whole, including all the invariants from the original Raft paper [32]. These new proofs consist of about 45000 additional lines” [Verdi, CPP’16]
proof/code:
Verdi: ~10
IronFleet: ~4
![Page 22: Workshop in Verification of Distributed Protocolsodedp/workshop18/ivy-workshop18.pdf · State of the art in formal verification ess Automation Proof Assistants 22 Ultimately limited](https://reader033.vdocuments.mx/reader033/viewer/2022060207/5f03da427e708231d40b1558/html5/thumbnails/22.jpg)
State of the art in formal verification
Exp
ress
iven
ess
Automation
Proof Assistants
22
Ultimately limited by human
proof/code:
Verdi: ~10
IronFleet: ~4
“but our input language cannot compete in generality with mechanized proof methods that rely heavily on human expertise, e.g., IVY [55], Verdi [68], IronFleet [38], TLAPS [16]” [Konnov et al, POPL’17]
Model CheckingStatic AnalysisType Checking
Ultimately limited by undecidability
IVyDecidable Reasoning
Finite Counterexamples
proof/code: ~0.2
![Page 23: Workshop in Verification of Distributed Protocolsodedp/workshop18/ivy-workshop18.pdf · State of the art in formal verification ess Automation Proof Assistants 22 Ultimately limited](https://reader033.vdocuments.mx/reader033/viewer/2022060207/5f03da427e708231d40b1558/html5/thumbnails/23.jpg)
IVy’s Principles
• Specify systems and properties in decidable fragment of first-order logic (EPR)
• Allows quantifiers to reason about unbounded sets
• Decidable to check inductiveness
• Finite counterexamples to induction, display graphically
• Logic is mostly hidden
• Interact with the user to find inductive invariants
• Challenge: use restricted logic to verify interesting systems
• Paxos, Reconfiguration, Byzantine Fault Tolerance
• Liveness and Temporal Properties
23
![Page 24: Workshop in Verification of Distributed Protocolsodedp/workshop18/ivy-workshop18.pdf · State of the art in formal verification ess Automation Proof Assistants 22 Ultimately limited](https://reader033.vdocuments.mx/reader033/viewer/2022060207/5f03da427e708231d40b1558/html5/thumbnails/24.jpg)
Example: Leader Election in a Ring
• Nodes are organized in a ring
• Each node has a unique numeric id
• Protocol:
• Each node sends its id to the next
• A node that receives a message passes it (to the next) if the id in the message is higher than the node’s own id
• A node that receives its own id becomes the leader
• Theorem:
• The protocol selects at most one leader
[CACM’79] E. Chang and R. Roberts. An improved algorithm for decentralized extrema-finding in circular configurations of processes
3 5
2
4
1
6next
next next
next
next
next
![Page 25: Workshop in Verification of Distributed Protocolsodedp/workshop18/ivy-workshop18.pdf · State of the art in formal verification ess Automation Proof Assistants 22 Ultimately limited](https://reader033.vdocuments.mx/reader033/viewer/2022060207/5f03da427e708231d40b1558/html5/thumbnails/25.jpg)
Example: Leader Election in a Ring
• Nodes are organized in a ring
• Each node has a unique numeric id
• Protocol:
• Each node sends its id to the next
• A node that receives a message passes it (to the next) if the id in the message is higher than the node’s own id
• A node that receives its own id becomes the leader
• Theorem:
• The protocol selects at most one leader
[CACM’79] E. Chang and R. Roberts. An improved algorithm for decentralized extrema-finding in circular configurations of processes
3 5
2
4
1
6next
next next
next
next
next
![Page 26: Workshop in Verification of Distributed Protocolsodedp/workshop18/ivy-workshop18.pdf · State of the art in formal verification ess Automation Proof Assistants 22 Ultimately limited](https://reader033.vdocuments.mx/reader033/viewer/2022060207/5f03da427e708231d40b1558/html5/thumbnails/26.jpg)
Leader Election Protocol (IVy)• (ID, ID) – total order on node id’s
• btw (Node, Node, Node) – the ring topology
• id: Node ID – relate a node to its unique id
• pending(ID, Node) – pending messages
• leader(Node) – leader(n) means n is the leader
|
Axiomatized in first-order logic
structureprotocol state
≤
n1L
id1
n2L
id2
n3L
≤ id3
n4L
n5L
id5 id6≤ ≤
<n5, n1, n3> ∈ 𝐼(btw)
id4
n6L
≤
n1
3 5
2
4
1
6next
next next
next
next
next 25
pndid
id id idpnd
n5
![Page 27: Workshop in Verification of Distributed Protocolsodedp/workshop18/ivy-workshop18.pdf · State of the art in formal verification ess Automation Proof Assistants 22 Ultimately limited](https://reader033.vdocuments.mx/reader033/viewer/2022060207/5f03da427e708231d40b1558/html5/thumbnails/27.jpg)
Leader Election Protocol (IVy)• (ID, ID) – total order on node id’s
• btw (Node, Node, Node) – the ring topology
• id: Node ID – relate a node to its unique id
• pending(ID, Node) – pending messages
• leader(Node) – leader(n) means n is the leader
|
protocol = (send | receive)*
assert I0 = ∀ x,y: Node. leader(x)leader(y) → x = y
action receive(n: Node, m: ID) = {requires pending(m, n);if id(n) = m then// found leaderleader(n) := true
else if id(n) m then// pass message“s := next(n)”;pending(m, s) := true
}
action send(n: Node) = {“s := next(n)”;pending(id(n),s) := true
}
∃n,s: Node. “s := next(n)” ∧ ∀x:ID,y:Node. pending'(x,y)↔ (pending(x,y)∨(x=id(n)∧y=s))
![Page 28: Workshop in Verification of Distributed Protocolsodedp/workshop18/ivy-workshop18.pdf · State of the art in formal verification ess Automation Proof Assistants 22 Ultimately limited](https://reader033.vdocuments.mx/reader033/viewer/2022060207/5f03da427e708231d40b1558/html5/thumbnails/28.jpg)
1 L
next
2L
nextid id
3L
idnext
1 L
next
2L
nextid id
3L
idnext
1 L
next
2L
nextid id
3L
idnext
pnd
1 L
next
2L
nextid id
3L
idnext
pnd
1 L
next
2L
nextid id
3L
idnext
pnd
![Page 29: Workshop in Verification of Distributed Protocolsodedp/workshop18/ivy-workshop18.pdf · State of the art in formal verification ess Automation Proof Assistants 22 Ultimately limited](https://reader033.vdocuments.mx/reader033/viewer/2022060207/5f03da427e708231d40b1558/html5/thumbnails/29.jpg)
1 L
next
2L
nextid id
3L
idnext
1 L
next
2L
nextid id
3L
idnext
1 L
next
2L
nextid id
3L
idnext
pnd
1 L
next
2L
nextid id
3L
idnext
pnd
1 L
next
2L
nextid id
3L
idnext
pnd
≤
1 L
next
2L
nextid id
3L
≤
idnext
≤
1 L
next
2L
nextid id
3L
≤
idnext
pnd
≤
1 L
next
2L
nextid id
3L
≤
idnext
≤
1 L
next
2L
nextid id
3L
≤
idnext
pnd
1 L
next
2L
nextid id
3L
idnext
1 L
next
2L
nextid id
3L
idnext
pnd
…
Specify and verify the protocol for any number of nodes in the ring
![Page 30: Workshop in Verification of Distributed Protocolsodedp/workshop18/ivy-workshop18.pdf · State of the art in formal verification ess Automation Proof Assistants 22 Ultimately limited](https://reader033.vdocuments.mx/reader033/viewer/2022060207/5f03da427e708231d40b1558/html5/thumbnails/30.jpg)
Inductive Invariant for Leader Election• (ID, ID) – total order on node id’s
• btw (Node, Node, Node) – the ring topology
• id: Node ID – relate a node to its id
• pending(ID, Node) – pending messages
• leader(Node) – leader(n) means n is the leader
Safety property: I0I0 = x, y: Node. leader(x) ∧ leader(y) x = y
Inductive invariant: Inv = I0 I1 I2 I3
I1 = n1,n2: Node. leader(n2) id[n1] id[n2]
I2 = n1,n2: Node. pending(id[n2], n2) id[n1] id[n2]
I3 =n1,n2,n3: Node. btw(n1, n2, n3) pending(id[n2], n1)id[n3]id[n2]
h
How can we find an inductive invariant without knowing it?
The leader has the highest ID
Only the leader can be self-pending
Cannot bypass higher nodes
![Page 31: Workshop in Verification of Distributed Protocolsodedp/workshop18/ivy-workshop18.pdf · State of the art in formal verification ess Automation Proof Assistants 22 Ultimately limited](https://reader033.vdocuments.mx/reader033/viewer/2022060207/5f03da427e708231d40b1558/html5/thumbnails/31.jpg)
Invariant Inference in IVyModel Candidate Inductive Invariant
Inductive?Yes
No
Find “minimal” CTI
Modify candidate invariant
Generalize from CTI
User Automation
Inductive Invariant Found
![Page 32: Workshop in Verification of Distributed Protocolsodedp/workshop18/ivy-workshop18.pdf · State of the art in formal verification ess Automation Proof Assistants 22 Ultimately limited](https://reader033.vdocuments.mx/reader033/viewer/2022060207/5f03da427e708231d40b1558/html5/thumbnails/32.jpg)
IVy: Check Inductiveness
Leader Protocol Inv = I0 I1 I2
rcv(1, id(2))
I0I1 I2 I2
1 L
next
2L
nextid id
pnd
3L
idnext
1 L
next
2L
nextid id
pnd
3L
idnext
Check Inductiveness
CTI
![Page 33: Workshop in Verification of Distributed Protocolsodedp/workshop18/ivy-workshop18.pdf · State of the art in formal verification ess Automation Proof Assistants 22 Ultimately limited](https://reader033.vdocuments.mx/reader033/viewer/2022060207/5f03da427e708231d40b1558/html5/thumbnails/33.jpg)
IVy: Generalize from CTI
Cannot bypass nodeswith higher ids
n1
Lnext
id1
n2
L
nextid id
id2pnd
n3
L
id3
idnext
Project to {pnd,, id,btw}
n1
id1
n2
id id
id2pnd
n3
id3
id
btw
pnd
n1 n2
id
id2
n3
id3
id
btw
Interp(3)
C3 = n1, n2, n3 : Node. (n1,n2,n3) (id[n1],id[n2],id[n3]) id[n1] id[n2] id[n3] pnd(id[n2], n1) btw(n1, n2, n3)
C’3 = n1, n2, n3 : Node. btw(n1, n2, n3) pnd(id[n2], n1) id[n2] id[n3]
This looks good, add to the invariant as I3
I3
![Page 34: Workshop in Verification of Distributed Protocolsodedp/workshop18/ivy-workshop18.pdf · State of the art in formal verification ess Automation Proof Assistants 22 Ultimately limited](https://reader033.vdocuments.mx/reader033/viewer/2022060207/5f03da427e708231d40b1558/html5/thumbnails/34.jpg)
IVy: Check Inductiveness
Bad = I0
VC Generator
Leader Protocol Inv = I0 I1 I2 I3
EPR Solver
Init InvInv(V) TR(V,V’) Inv(V’)
Inv(V) Bad(V)
Proof
I0 I1 I2 I3 is an inductive invariant for the leader protocol,which proves the protocol is safe
![Page 35: Workshop in Verification of Distributed Protocolsodedp/workshop18/ivy-workshop18.pdf · State of the art in formal verification ess Automation Proof Assistants 22 Ultimately limited](https://reader033.vdocuments.mx/reader033/viewer/2022060207/5f03da427e708231d40b1558/html5/thumbnails/35.jpg)
L L≤
id idpnd
pndid≤
id
btw
Init Inv (Initiation)if σ Inv and σ σ’ then σ’ Inv (Consecution)Inv Bad = (Safety)
≤
L
id id
![Page 36: Workshop in Verification of Distributed Protocolsodedp/workshop18/ivy-workshop18.pdf · State of the art in formal verification ess Automation Proof Assistants 22 Ultimately limited](https://reader033.vdocuments.mx/reader033/viewer/2022060207/5f03da427e708231d40b1558/html5/thumbnails/36.jpg)
Leader Election Protocol (axioms)• (ID, ID) – total order on node id’s
• btw (a: Node, b: Node, c: Node) – the ring topology
• id: Node ID – relate a node to its unique id
• pending(ID, Node) – pending messages
• leader(Node) – leader(n) means n is the leader
|
Natural Interpretation EPR Modeling
Node ID’s Integers
i:ID. i i Reflexivei, j, k: ID. i jj ki k Transitivei, j: ID. i jj ii=j Anti-Symmetrici, j: ID. i j j i Totalx, y: Node. id(x) = id(y) x=y Injective
Ring TopologyNext edges + Transitive closure
x, y, z: Node. btw(x, y, z) btw(y, z, x) Circular shiftsx, y, z, w: Node. btw(w, x, y) btw(w, y, z) btw(w, x, z) Transitivex, y, w: Node. btw(w, x, y) btw(w, y, x) A-Symmetricx, y, z, w: Node. distinct(x, y, z) btw(w, x, y) btw(w, y, x)
“next(a)=b” x: Node.X=aX=bbtw(a,b,x)
![Page 37: Workshop in Verification of Distributed Protocolsodedp/workshop18/ivy-workshop18.pdf · State of the art in formal verification ess Automation Proof Assistants 22 Ultimately limited](https://reader033.vdocuments.mx/reader033/viewer/2022060207/5f03da427e708231d40b1558/html5/thumbnails/37.jpg)
Challenge: How to use restricted first-order logic to verify interesting systems?
37
• Expressing transitive closure
• Linked lists
• Ring protocols
• Expressing Consensus
• Paxos, Multi-Paxos
• Reconfiguration
• Byzantine Fault Tolerance
• Liveness and temporal Properties
![Page 38: Workshop in Verification of Distributed Protocolsodedp/workshop18/ivy-workshop18.pdf · State of the art in formal verification ess Automation Proof Assistants 22 Ultimately limited](https://reader033.vdocuments.mx/reader033/viewer/2022060207/5f03da427e708231d40b1558/html5/thumbnails/38.jpg)
Key idea: representing deterministic paths[Itzhaky SIGPLAN Dissertation Award 2016]
Alternative 1: maintain n• n* defined by transitive closure of n• not definable in first-order logic
nn
n*
h t
nnh t
Alternative 2: maintain n*
• n defined by transitive reduction of n*
• Unique due to outdegree 1• Definable in first order logic (for roots)
• n+(a,b) n*(a, b)ab• n(a, b) n+(a,b) z: n+(a, z)n*(b, z)
n*
h tNot first order expressible
First order expressible
n* btw
![Page 39: Workshop in Verification of Distributed Protocolsodedp/workshop18/ivy-workshop18.pdf · State of the art in formal verification ess Automation Proof Assistants 22 Ultimately limited](https://reader033.vdocuments.mx/reader033/viewer/2022060207/5f03da427e708231d40b1558/html5/thumbnails/39.jpg)
Paxos made EPR
Methodology for decidable verification of infinite-state systems
Formal specification in first-order logic
Formal specification with decidable VC
Protocol
AbstractionDomain knowledge
1 2
Modeling Transforming
![Page 40: Workshop in Verification of Distributed Protocolsodedp/workshop18/ivy-workshop18.pdf · State of the art in formal verification ess Automation Proof Assistants 22 Ultimately limited](https://reader033.vdocuments.mx/reader033/viewer/2022060207/5f03da427e708231d40b1558/html5/thumbnails/40.jpg)
Paxos
• Single decree Paxos – consensuslets nodes make a common decision despite node crashes and packet loss
• Paxos family of protocols – state machine replicationvariants for different tradeoffs, e.g., Fast Paxos is optimized for low contention, Vertical Paxos is reconfigurable, etc.
• Pervasive approach to fault-tolerant distributed computing
• Google Chubby
• VMware NSX
• AWS
• Many more…
![Page 41: Workshop in Verification of Distributed Protocolsodedp/workshop18/ivy-workshop18.pdf · State of the art in formal verification ess Automation Proof Assistants 22 Ultimately limited](https://reader033.vdocuments.mx/reader033/viewer/2022060207/5f03da427e708231d40b1558/html5/thumbnails/41.jpg)
Challenge: reasoning about Paxos in FOL
• Consensus algorithms use set cardinalities
• Wait for messages from more than N / 2 nodes
• Insight: set cardinalities are used to get a simple effect
Can be modeled in first-order logic!
• Solution: axiomatize quorums in first-order logic
sort quorumrelation member (node, quorum)– set membership (2nd-order logic in first-order)
axiom ∀q1,q2: quorum. ∃n: node. member(n, q1) ∧ member(n, q2)
action propose(r:round) {requires “>N/2 join_msg’s”…
}
action propose(r:round) {requires ∃q.∀n.member(n,q) →∃r’,v’.join_msg(n,r,r’,v’)
…}
![Page 42: Workshop in Verification of Distributed Protocolsodedp/workshop18/ivy-workshop18.pdf · State of the art in formal verification ess Automation Proof Assistants 22 Ultimately limited](https://reader033.vdocuments.mx/reader033/viewer/2022060207/5f03da427e708231d40b1558/html5/thumbnails/42.jpg)
Principle: domain knowledge
Concept Intention First-order abstraction
Quorums Majority setsrelation member (node, quorum)axiom ∀q1,q2:quorum∃n:node. member(n, q1) ∧ member(n, q2)
RoundsNaturalnumbers
relation ≤(round,round) axiom ∀x:round. x ≤ x reflexiveaxiom ∀x,y,z:round. x≤y ∧ y≤z → x ≤ z transitiveaxiom ∀x,y:round. x≤y ∧ y≤ x → x=y anti-symmetricaxiom ∀x,y:round. x≤y ∨ y ≤ x total
Messages
Network with:droppingduplicationreordering
relation start_msg(round)relation join_msg(node,round,round,value)relation propose_msg(round,value)relation vote_msg(node,round,value)
Formal specificationin first-order logic
Protocol 1
![Page 43: Workshop in Verification of Distributed Protocolsodedp/workshop18/ivy-workshop18.pdf · State of the art in formal verification ess Automation Proof Assistants 22 Ultimately limited](https://reader033.vdocuments.mx/reader033/viewer/2022060207/5f03da427e708231d40b1558/html5/thumbnails/43.jpg)
Paxos in first-order logic
VC’s in first-order logic
![Page 44: Workshop in Verification of Distributed Protocolsodedp/workshop18/ivy-workshop18.pdf · State of the art in formal verification ess Automation Proof Assistants 22 Ultimately limited](https://reader033.vdocuments.mx/reader033/viewer/2022060207/5f03da427e708231d40b1558/html5/thumbnails/44.jpg)
Step 2: Obtaining decidable VC’sChallenge : quantifier alternation cycles
• Axiom
∀q1,q2: quorum. ∃n: node. member(n, q1) ∧ member(n, q2)
• Propose action precondition
∃q:quorum. ∀n:node. member(n,q) → ∃r’:round,v’:value. join_msg(n,r,r’,v’)
• Inductive invariant
∀r:round,v:value. decision(r,v) → ∃q:quorum. ∀n:node. member(n,q) → vote_msg(n,r,v)
round
value
nodequorum
Quantifier Alternation Cycle
![Page 45: Workshop in Verification of Distributed Protocolsodedp/workshop18/ivy-workshop18.pdf · State of the art in formal verification ess Automation Proof Assistants 22 Ultimately limited](https://reader033.vdocuments.mx/reader033/viewer/2022060207/5f03da427e708231d40b1558/html5/thumbnails/45.jpg)
Solution: derived relations and rewrites
∃q:quorum. ∀n:node. member(n,q) → ∃r’:round,v’:value. join_msg(n,r,r’,v’)
![Page 46: Workshop in Verification of Distributed Protocolsodedp/workshop18/ivy-workshop18.pdf · State of the art in formal verification ess Automation Proof Assistants 22 Ultimately limited](https://reader033.vdocuments.mx/reader033/viewer/2022060207/5f03da427e708231d40b1558/html5/thumbnails/46.jpg)
Solution: derived relations and rewrites
∃q:quorum. ∀n:node. member(n,q) → ∃r’:round,v’:value. join_msg(n,r,r’,v’)
new relation: joined(n:node,r:round) ≡ ∃r’:round,v’:value. join_msg(n,r,r’,v’)
update code:
∃q:quorum. ∀n:node. member(n,q) → joined(n,r)
rewrite
action join(n:node, r:round) {requires start_round_msg(r)let maxr,v := …join_msg(n,r,maxr,v) := truejoined(n,r) := true
}
![Page 47: Workshop in Verification of Distributed Protocolsodedp/workshop18/ivy-workshop18.pdf · State of the art in formal verification ess Automation Proof Assistants 22 Ultimately limited](https://reader033.vdocuments.mx/reader033/viewer/2022060207/5f03da427e708231d40b1558/html5/thumbnails/47.jpg)
joined(n:node,r:round) ≡ ∃r’:round,v’:value. join_msg(n,r,r’,v’)
left(n:node,r:round) ≡ ∃r’,r’’:round,v’:value. join_msg(n,r’,r’’,v’) ∧ r’>r
VC’s are decidable!
Solution: derived relations and rewrites
![Page 48: Workshop in Verification of Distributed Protocolsodedp/workshop18/ivy-workshop18.pdf · State of the art in formal verification ess Automation Proof Assistants 22 Ultimately limited](https://reader033.vdocuments.mx/reader033/viewer/2022060207/5f03da427e708231d40b1558/html5/thumbnails/48.jpg)
Principle: decomposing into decidable checks
• User defines:
• Derived relations
• Rewrites
• Inductive invariants
• Decidable checks:
⊨Invaux
Spec in FOL
Invaux ⊨ ↔ ⊨Inv
Modified Spec
Formal specification in first-order logic
Formal specification with decidable VC
2
![Page 49: Workshop in Verification of Distributed Protocolsodedp/workshop18/ivy-workshop18.pdf · State of the art in formal verification ess Automation Proof Assistants 22 Ultimately limited](https://reader033.vdocuments.mx/reader033/viewer/2022060207/5f03da427e708231d40b1558/html5/thumbnails/49.jpg)
Inductive Invariant of Paxos# safety property
conjecture decision(N1,R1,V1) & decision(N2,R2,V2) -> V1 = V2
# proposals are unique per round
conjecture proposal(R,V1) & proposal(R,V2) -> V1 = V2
# only vote for proposed values
conjecture vote(N,R,V) -> proposal(R,V)
# decisions come from quorums of votes:
conjecture forall R, V. (exists N. decision(N,R,V)) -> exists Q. forall N. member(N, Q) -> vote(N,R,V)
# properties of one_b_max_vote
conjecture one_b_max_vote(N,R2,none,V1) & ~le(R2,R1) -> ~vote(N,R1,V2)
conjecture one_b_max_vote(N,R,RM,V) & RM ~= none -> ~le(R,RM) & vote(N,RM,V)
conjecture one_b_max_vote(N,R,RM,V) & RM ~= none & ~le(R,RO) & ~le(RO,RM) -> ~vote(N,RO,VO)
# property of choosable and proposal
conjecture ~le(R2,R1) & proposal(R2,V2) & V1 ~= V2 -> exists N. member(N,Q) & left_rnd(N,R1) & ~vote(N,R1,V1)
# property of one_b, left_rnd
conjecture one_b(N,R2) & ~le(R2,R1) -> left_rnd(N,R1)
![Page 50: Workshop in Verification of Distributed Protocolsodedp/workshop18/ivy-workshop18.pdf · State of the art in formal verification ess Automation Proof Assistants 22 Ultimately limited](https://reader033.vdocuments.mx/reader033/viewer/2022060207/5f03da427e708231d40b1558/html5/thumbnails/50.jpg)
ProtocolModel[LOC]
Invariant[Conjectures]
EPR [sec]𝝁 𝝈
RW[sec]
Paxos 85 11 1.0 0.1 1.2
Multi-Paxos 98 12 1.2 0.1 1.4
Vertical Paxos* 123 18 2.2 0.2 -
Fast Paxos* 117 17 4.7 1.6 1.5
Flexible Paxos 88 11 1.0 0 1.2
Stoppable Paxos* 132 16 3.8 0.9 1.6
Experimental Evaluation
*first mechanized verificationTransformation to EPR reusable across all variants!
![Page 51: Workshop in Verification of Distributed Protocolsodedp/workshop18/ivy-workshop18.pdf · State of the art in formal verification ess Automation Proof Assistants 22 Ultimately limited](https://reader033.vdocuments.mx/reader033/viewer/2022060207/5f03da427e708231d40b1558/html5/thumbnails/51.jpg)
51
![Page 52: Workshop in Verification of Distributed Protocolsodedp/workshop18/ivy-workshop18.pdf · State of the art in formal verification ess Automation Proof Assistants 22 Ultimately limited](https://reader033.vdocuments.mx/reader033/viewer/2022060207/5f03da427e708231d40b1558/html5/thumbnails/52.jpg)
52
![Page 53: Workshop in Verification of Distributed Protocolsodedp/workshop18/ivy-workshop18.pdf · State of the art in formal verification ess Automation Proof Assistants 22 Ultimately limited](https://reader033.vdocuments.mx/reader033/viewer/2022060207/5f03da427e708231d40b1558/html5/thumbnails/53.jpg)
Verification of Temporal Properties
53
![Page 54: Workshop in Verification of Distributed Protocolsodedp/workshop18/ivy-workshop18.pdf · State of the art in formal verification ess Automation Proof Assistants 22 Ultimately limited](https://reader033.vdocuments.mx/reader033/viewer/2022060207/5f03da427e708231d40b1558/html5/thumbnails/54.jpg)
Possible Projects
• Verify any distributed / shared memory algorithm
• Paxos variants
• Disk Paxos, Generalised Paxos, EPaxos (see http://paxos.systems/variants.html for ideas)
• Prove reconfiguration / failure recovery / log truncation / liveness
• Mutual Exclusion Algorithms
• Knuth’s Algorithm, Lamport’s Bakery, Patterson, …
• Prove safety and liveness
• Blockchain algorithms
• Algorand, HoneyBadgerBFT, Bitcoin-NG, …
• Improve Ivy
• Experiment with other SMT solvers (e.g. iProver, CVC4, Vampire, SPASS)
54