interpolation and widening
DESCRIPTION
Interpolation and Widening. Ken McMillan Microsoft Research. TexPoint fonts used in EMF: A A A A A. Interpolation and Widening. Widening/Narrowing and Craig Interpolation are two approaches to computing inductive invariants of transition systems. - PowerPoint PPT PresentationTRANSCRIPT
Interpolation and Widening
Ken McMillanMicrosoft Research
Interpolation and Widening• Widening/Narrowing and Craig Interpolation are two approaches to
computing inductive invariants of transition systems.• Both are essentially methods of generalizing from proofs about bounded
executions to proofs about unbounded executions.• In this talk, we'll consider the relationship between these two
approaches, from both theoretical and practical points of view.• Consider only property proving applications, since interpolation only
applies with a property to prove.
Intuitive comparison
stronger
weaker
iterations𝜏1 (⊥ )𝜏2 (⊥ )
...
lfp
stronger
weaker
iterations𝜏1 (⊥ )𝜏2 (⊥ )
...
lfp
𝛻 Δ 𝑆
inductive𝑆
widening/narrowing
interpolation
Abstractions as proof systems• We will view both widening/narrowing and interpolation as proof
systems– In particular, local proof systems
• A proof system (or abstraction) consists of:– A logical language L (abstract domain)– A set of sound deduction rules
• A choice of proof system constitutes a bias, or domain knowledge– Rich proof system = weak bias– Impoverished proof system = strong bias
By restricting the logical language and deduction rules, the analysisdesigner expresses a space of possible proofs in which the analysistool should search.
Fundamental problems
• Relevance– We must avoid a combinatorial explosion of deductions– Thus, deduction must be restricted to facts relevant to the property
• Convergence– Eventually the proofs for bounded executions must generalize to a proof of
unbounded executions.
Different approaches
• Widening/narrowing relies on a restricted proof system– Relevance is enforced by strong bias– Convergence is also enforced in this way, but proof of a property is not
guaranteed• Interpolation uses a rich proof system
– Relevance is determined by Occam's razor• relevant deductions occur in simple property proofs
– Convergence is not guaranteed, but• approached heuristically again using Occam's razor
We will see that the two methods have many aspects in common, but take different approaches to these fundamental problems.
In the interpolation approach, we rely on well-developed theoremproving approaches to search large spaces for simple proofs.
Proofs• A proof is a series of deductions, from premises to conclusions• Each deduction is an instance of an inference rule• Usually, we represent a proof as a tree...
P1 P2
P3 P4 P5
C
Premises
Conclusion
P1 P2
C
Inference rules• The inference rules depend on the theory we are reasoning in
p _ : p _ D
_ D
Resolution rule:
Boolean logic Linear arithmetic
x1 · y1
x2 · y2
x1+x2 · y1+y2
Sum rule:
Invariants from unwindings• A simple way to generalize from bounded to unbounded proofs:
– Consider just one program execution path, as straight-line program– Construct a proof for this straight-line program– See if this proof contains an inductive invariant proving the property
• Example program:
x = y = 0;while(*) x++; y++;while(x != 0) x--; y--;assert (y == 0);
{x == y}
invariant:
{x = 0 ^ y = 0}
{x = y}
{x = y}
{x = y}
{x = 0 ) y = 0}
{False}
{True}
{y = 0}
{y = 1}
{y = 2}
{y = 1}
{y = 0}
{False}
{True}
Unwind the loops
Proof of inline program contains invariants
for both loops
• Assertions may diverge as we unwind• A practical method must somehow
prevent this kind of divergence!
x = y = 0;
x++; y++;
x++; y++;
[x!=0];x--; y--;
[x!=0];x--; y--;
[x == 0][y != 0]
How can we find relevant proofs of program paths?
Interpolation Lemma• Let A and B be first order formulas, using
– some non-logical symbols (predicates, functions, constants)– the logical symbols ^, _, :, 9, 8, (), ...
• If A Ù B = false, there exists an interpolant A' for (A,B) such that:A Þ A'
A' ^ B = falseA’ uses only common vocabulary of A and B
[Craig,57]
A
p Ù qB
Øq Ù r
A’ = q
Interpolants as Floyd-Hoare proofs
False
x1=y0
True
y1>x1
))
)
1. Each formula implies the next
2. Each is over common symbols of prefix and suffix
3. Begins with true, ends with false
Proving in-line programs
SSAsequence Prover
Interpolation
HoareProof
proof
x=y;
y++;
[x=y]
x1= y0
y1=y0+1
x1=y1
{False}
{x=y}
{True}
{y>x}
x = y
y++
[x == y]
Local proofs and interpolants
x=y;
y++;
[y · x]
x1=y0
y1=y0+1
y1·x1
y0 · x1
x1+1 · y1 y1 · x1+1
y1 · y0+1
1 · 0FALSE
x1 · y0
y0+1 · y1
TRUE
x1 · y
x1+1 · y1
FALSE
This is an example of a local proof...
Definition of local proof
x1=y0
y1=y0+1
y1·x1
y0
scope of variable = range of frames it occurs in
y1
x1
vocabulary of frame = set of variables “in scope”
{x1,y0}
{x1,y0,y1}
{x1,y1}
x1+1 · y1
x1 · y0
y0+1 · y1 deduction “in scope” here
Local proof: Every deduction written in vocabulary of some frame.
Forward local proof
x1=y0
y1=y0+1
y1·x1
{x1,x0}
{x1,y0,y1}
{x1,y1}
Forward local proof: each deduction can be assigned a framesuch that all the deduction arrows go forward.
x1+1 · y1
1 · 0
FALSE
x1 · y0
y0+1 · y1
For a forward local proof, the (conjunction of) assertionscrossing frame boundary is an interpolant.
TRUE
x1 · y
x1+1 · y1
FALSE
Proofs and relevance
x1=y0+1
z1=x1+1
x1·y0
y0 · z1
{x1,y0}
{x1,y0,z1}
{x1,y0,z1}
TRUE
x1= y0 + 1
FALSE
z1 = y0 + 2
1·0
FALSE
x1= y0 + 1 Æ z1 = y0 + 2
• By dropping unneeded inferences, we weaken the interpolant and eliminate irrelevant predicates.
0 · 2
x1= y0 + 1
Interpolants are neither weakest pre not strongest post.
Applying Occam's Razor
• Define a (local) proof system– Can contain whatever proof rules you want
• Define a cost metric for proofs– For example, number of distinct predicates after dropping subscripts
• Exhaustive search for lowest cost proof– May restrict to forward or reverse proofs
x = e
[e/x]
FALSE unsat.
Allow simple arithmetic rewriting.
Simple proofs are more likely to generalize
Even this trivial proofs system allows useful flexibility
Loop example
x0 = 0y0 = 0
x1=x0+1y1=y0+1
TRUE
x0= 0Æ y0 = 0
...
x1=1 Æ y1 = 1x2=x1+1y2=y1+1
...
x1 = 1y1 = 1
x2 = 2y2 = 2
... ...
cost: 2N
x2=2 Æ y2 = 2
x0 = y0
x1 = y0+1
x1 = y1
x2 = y1+1
x2 = y2
TRUE
x0 = y0
...
x1= y1
cost: 2
x2= y2
Lowest cost proof is simpler, avoids divergence.
Interpolation• Generalize from bounded proofs to unbounded proofs• Weak bias
– Rich proof system (large space of proofs)– Apply Occam's razor (simple proofs more likely to generalize)
• Occam's razor is applied to – Avoid combinatorial explosion of deductions (relevance)– Eventually generalize to inductive proofs (convergence)
• Apply theorem proving technology to search large space of possible proofs for simple proofs– DPLL, SMT solvers, etc.
Widening operators• A widening operator is a function over partially ordered , satisfying the
following properties:
– Soundness: – Expansion: – Stability: for every ascending chain...
𝑥0 𝑥1 𝑥2⊑ ⊑ ⊑⋯𝛻
𝑦 0 𝑦 1 𝑦 2 ⋯
¿
𝛻
this chain eventually stabilizes.
Upward iteration sequence• Suppose we have a monotone transformer representing (or
approximating) our concrete semantics.• We use apply the widening operator to successive iterations of to
obtain an upward iteration sequence.
⊥ 𝜏
𝑦 1𝛻
𝑦 2𝛻𝜏
𝑦 3𝛻𝜏
• The widening properties guarantee– over-approximation– stabilization
𝑥1 𝑥2 𝑥3𝜏 𝜏 ⊑⊑ over-approximate
eventually stable!...
Narrowing similar but contracting
Widening as local deduction• Since widening loses information, we can think of it as a deduction rule• In fact, we may have several deduction rules at our disposal:
Sound if is an over-approximation
Sound if is an over-approximation
and is sound
Note we don't need the expansion and stability propertiesof to have a sound deduction rule.
Sound if is an over-approximation
and is sound
pre
abstract post
pre
join
pre
widen
{x=y, x,y }
{False}
{True}
Widening with octagons
x = y = 0;
x++; y++;
x++; y++;
[x!=0];x--; y--;
[x!=0];x--; y--;
[x == 0][y != 0]
{x=y, x,y }
Because we proved the property, we have computed an interpolant
{x=y, x,y }
{x=y, x,y }⊔
{x=y, x,y }𝛻
{x=y, x,y }
{x=y, x,y }
But note the irrelevant fact!
Our proof rules are too coarseto eliminate this fact.
{x,y }
{True}
Over-widening (with intervals)
x = y = 0;
x=1-x; y++;
x=1-x; y++;
[x==2];
{x,y }
{x,y }
{x,y }𝛻
Note if we had waited on step towiden we would have a proof.
{False}
Safe widening• Let us define a safe widening sequence as one that ends in a safe state.
⊥⊔⊔⊔⊔⊔⊔𝛻
Suppose we apply a sequence of rules and fail...
⊑𝑆
⊥⊔⊔⊔⊔⊔𝛻
We may postpone a widening to achieve a safety proof
⊑𝑆⊔
• This is a proof search problem!– If we have rules and steps, there are possible proofs
• Safe widening sequences may not stabilize– May not contain a long enough sequence of
• Safe widening sequences may not exist– That is, our proof system may be incomplete
Incompleteness
• Incomplete proof system on purpose• We restrict the proof system (strong bias) to enforce
– relevance focus– convergence
• These properties are obtained at the risk of over-widening
• Incompleteness derives only from incompleteness of underlying logic– For example, in Presburger arithmetic we have completeness
• Relevance focus and convergence rely on general heuristics– Occam's razor (simple proofs tend to generalize)– Rely on theorem proving techniques– Choice of logic and axioms also represents a weak bias
Widening/narrowing
Interpolation
Consequences of strong bias• Widening requires domain knowledge, which entails a careful choice of
the logical language L.– Octagons: easy– Unions of octagons: harder– Presburger arithmetic formulas: ???
• This entails incompleteness, as a restricted language implies loss of information.
• This also means we can tailor the representation for efficiency.– Octagons: use half-space representation, not convex hull of vertices– Polyhedra: mixed representation
Advantages of weak bias
• Boolean logic (e.g., hardware verification)– Language L is Boolean circuits over system state variables– There is no obvious a priori widening for this language– Interpolation techniques are the most effective known for this problem
• McMillan CAV 2003 (feasible interpolation using SAT solvers)• Bradley VMCAI 2011 (interpolation by local proof)
– Note rapid convergence is very important here • Infinite state cases requiring disjunctions
– Hard to formula a widening a priori– Weak bias can be used to avoid combinatorial explosion of disjuncts
• Example: IMPACT• Scaling to large number of variables
– Weak bias can allow focus just on relevant variables
Weak bias can be used in cases where domain knowledge is lacking.
Simple example
for(i = 0; i < N; i++) a[i] = i;
for(j = 0; j < N; j++) assert a[j] = j;
{8 x. 0 · x ^ x < i ) a[x] = x}
invariant:
Partial Axiomatization• Axioms of the theory of arrays (with select and update)
8 (A, I, V) (select(update(A,I,V), I) = V
8 (A,I,J,V) (I J ! select(update(A,I,V), J) = select(A,J))
• Axioms for arithmetic
8 (X,Y,Z) (X Y Y Z ! X Z)
8 (X) X X
8 (X,Y) (Y X Y succ(X))
[ integer axiom]etc...
We use a (local) first-order superposition prover to generateinterpolants, with a simple metric for proof complexity.
i = 0;
[i < N];a[i] = i; i++;
[i < N];a[i] = i; i++;
[i >= N]; j = 0;
[j < N]; j++;
[j < N];a[j] != j;
Unwinding simple example• Unwind the loops twice
i0 = 0
i0 < Na1 = update(a0,i0,i0)i1 = i0 + 1
i1 < Na2 = update(a1,i1,i1)i2 = i+1 + 1
i ¸ N ^ j0 = 0
j0 < N ^ j1 = j0 + 1
j1 < Nselect(a2,j1) j1
invariant
invariant
{i0 = 0}
{0 · U ^ U < i1 ) select(a1,U)=U}
{0 · U ^ U < i2 ) select(a2,U)=U}
{j · U ^ U < N ) select(a2,U)=U}
{j · U ^ U < N ) select(a2,U) = U}
weak bias prevents constants divergingas 0, succ(0), succ(succ(0)), ...
i = 0;
[i < N];a[i] = i; i++;
[i < N];a[i] = i; i++;
[i >= N]; j = 0;
[j < N]; j++;
[j < N];a[j] != j;
With strong bias• Something like array segmentation functor of C + C + Logozzo
{0,i} | i = 0
{0,i-1} {1,i} | i = 1, i
{0} {i-1}? {i} | i , iN
note: it so happened here our first try a wideningwas safe, but this may not always be so.
{0} {i}? | i ⊔
{0} {i}? | i 𝛻
...
Comparison
• Language L, operators and carefully chosen to throw away information at just the right places– This represents strong domain knowledge
• Carefully crafted representation yields high performance
• Axioms and proof bias are generic– Little domain knowledge is represented
• Uses a generic theorem prover to generate local proofs– No domain specific tuning
• Not as scalable as the strong bias approach
Widening/narrowing
Interpolation
List deletion example
• Add a few axioms about reachability• Invariant synthesized with 3 unwindings (after some: simplification):
a = create_list(); while(a){ tmp = a->next; free(a); a = tmp;}
{rea(next,a,nil) ^8 x (rea(next,a,x)! x = nil _ alloc(x))}
• No need to craft a new specialized domain for linked lists.• Weak bias can be used in cases where domain knowledge is lacking.
Are interpolants widenings?• A safe widening sequence is an interpolant.• An interpolant is not necessarily a widening sequence, however.
– Does not satisfy the expansion property– Does not satisfy the eventual stability property as we increase the sequence
length.• A consequence of giving up stabilization is that inductive invariants
(post-fixed points) are typically found in the middle of the sequence, not at an eventual stabilization point.– Early formulas tend to be too strong (influenced by initial condition)– Late formulas tend to be too weak (influenced by final condition)
Typical interpolant sequence
x = y = 0;
x++; y++;
x++; y++;
[x!=0];x--; y--;
[x!=0];x--; y--;
[x == 0][y != 0]
{}
{False}
{True}
{}
{}
{}
{}
Too strong
Too weak
Weakened, but not expansive
Does not stabilize at invariant
No matter how far we unwind, we may not get stabilization
Conclusion• Widening/narrowing and interpolation are methods of generalizing from
bounded to unbounded proofs• Formally, widening/narrowing satisfies stronger conditions
soundnessexpanding/contractingstabilizing
widening/narrowing
soundness
interpolation
stabilization is not obtained when proving properties, however
Conclusion, cont.• Heuristically, the difference is weak v. strong bias
restricted proof systemincompletenesssmaller search spacedomain knowledgeefficient representations
strong bias
rich proof systemcompletenesslarge search spaceOccam's razorgeneric representations
weak bias
• Can we combine strong and weak heuristics?– Fall back on weak heuristics when strong fails– Use weak heuristics to handle combinatorial complexity– Build known widenings into theory solvers in SMT?