exploiting sat solvers in unbounded model checking k. l. mcmillan cadence berkeley labs

Exploiting SAT solvers in unbounded model checking

K. L. McMillanCadence Berkeley Labs

Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification.

Overview

• We will examine the performance of two SAT-based abstraction methods– Counterexample-based abstraction– Proof-based abstraction

• This tells us something about– The nature of decision heuristics in SAT solvers– The strengths are weaknesses of SAT solvers

for bounded model checking and related applications.


Outline

• Background: SAT and BMC• Localization abstraction

– Cex-based– Proof-based

• Performance study– what it tells us about SAT


DPLL-style SAT solvers

• Objective:– Check satisfiability of a CNF formula

• literal: v or v• clause: disjunction of literals• CNF: conjunction of clauses

• Approach:– Branch: make arbitrary decisions– Propagate implication graph– Use conflicts to guide inference steps

SATO,GRASP,CHAFF,BERKMIN


The Implication Graph (BCP)

(a b) (b c d)

a

c

Decisions

b

Assignment: a b c d

d


Resolution

a b c a c d

b c d

When a conflict occurs, the implication graph isused to guide the resolution of clauses, so that thesame conflict will not occur again.


Conflict Clauses

(a b) (b c d) (b d)

a

c

Decisions

b

Assignment: a b c d

d

Conflict!

(b c )

resolve

Conflict!(a c)

resolve

Conflict!


Conflict Clauses (cont.)

• Conflict clauses:– Are generated by resolution– Are implied by existing clauses– Are in conflict in the current assignment– Are safely added to the clause set

Many heuristics are available for determiningwhen to terminate the resolution process.


Basic SAT algorithm

A =

empty clause?

yUNSAT

conflict?Deduce conflict

clause andbacktrack

y

n

is Atotal?

ySAT

Branch:add some literal

to A


Generating refutations

• Refutation = a proof of the null clause– Record a DAG containing all resolution steps

performed during conflict clause generation.– When null clause is generated, we can extract a

proof of the null clause as a resolution DAG.

Original clauses

Derived clauses

Null clause


Circuit SAT

ab

c p

g

Can the circuit output be 1?

inputvariables output

variable

(a g) (b g)(a b g)

(g p) (c p)(g c p)

CNF(p)

p is satisfiable when theformula CNF(p) pis satisfiable


Bounded Model Checking

• Given– A finite transition system M– A property p

• Determine– Does M allow a counterexample to p of k

transitions of fewer?

This problem can be translated to a SAT problem

BCCZ99


Models

Transition system described by a set of constraints

ab cp

g

Each circuit element is a constraintnote: a = at and a' = at+1

g = a b

p = g c

c' = p

Model:

C = { g = a b, p = g c, c' = p }


Properties

• We restrict our attention to safety properties.

• Characterized by:– Initial condition I– Final condition F (representing "bad" states)

• A counterexample is a path from a state satisfying I to state satisfying F, where every transition satisfies C.


Unfolding

• Unfold the model k times: Uk = C0 C1 ... Ck-1

ab

cp

g ab

cp

g ab

cp

g

...I0 Fk

• Use SAT solver to check satisfiability of I0 Uk Fk

• A satisfying assignment is a counterexample of k steps


BMC applications

• Debugging:– Can find counterexamples using a SAT solver

• Proving properties:– Only possible if a bound on the length of the

shortest counterexample is known.• I.e., we need a diameter bound. The diameter is the

maximum lenth of the shortest path between any two states.

– Worst case is exponential. Obtaining better bounds is sometimes possible, but generally intractable.


Localization abstraction

• Property: G (c X c)

ab cp

g

Model:

C = { g = a b, p = g c, c' = p }

'

free variable

C'property, C C' C property

Kurshan


Constraint granularity

ab cp

g Model:

C = { c' = (a b) c }

Most authors use constraints at "latch" granularity...

...however, techniques we will consider can be applied atboth "gate" and "latch" granularity.


Localization, cont• C' may refer to fewer state variables than C

– reduction in the state explosion problem

• Key issue: how to choose constraints in C'– counterexample-based– proof-based


Algorithm

Model checkabstraction C'

Choose initial C'

Can extend Cexfrom C'to C?

Add constraintsto C'

true, done

Cex

yes, Cex

no

SAT uses

Kurshan


Abstract counterexamples

• Assume simple safety property:– initial condition I and final condition F– w.l.o.g., assume I and F are atomic formulas

• to make this true, add constraints in C: vI I vF F

• Abstract variables V' = support(C',I,F)• Abstract counterexample A' is a truth

assignment to: { vt | v in V', t in 0..k }

where k is the number of steps.


Counterexample extension

• Abstract counterexample A' satisfies: I0 U'k Fk where U'k = C'0 C'1 ... C'k-1

• Find A consistent with A', satisfying: I0 Uk Fk where Uk = C0 C1 ... Ck-1

• That is, A is any satisfying assignment to:

A' I0 Uk Fk

I.e., to extend an abstract counterexample, we justapply it as a constraint in BMC. If unsat, abstractcounterexample is "false".

CGJLV 2000


Abstraction refinement

• Refinement = adding constraints to C' to eliminate false counterexamples.

• Many heuristsics used for this.

– Too many to cover here.

– Recall that a SAT solver can produce a resolution-based refutation in the UNSAT case....


Proof-based refinement

• Recall, to extend abstract Cex A', we check: A' I0 Uk Fk

• If UNSAT, we obtain refutation proof P– proof that A' cannot be extended to concrete Cex

• Let E be set of constraints used in proof P:E = { c C | some ci occurs in P }

• A' cannot be extended to a Cex for E– P is the proof of this.

Thus, add E to C' and continue...


In other words...

The refutation of the formula: A' I0 Uk Fk

gives us a sufficient set of constraints to rule out the abstract counterexample.

We continue ruling out counterexamples until either theabstraction C' proves the property or we can extend anabstract counterexample to a concrete one.


CCKSVW approach (FMCAD02)• Find the shortest prefix of Cex A' that cannot be extended.

• That is, A' I0 Uk Fk

is feasible for all k < i, but not for k=i.

s0 s1 s2 si-1 si...

OK OK OK OK NO!


CCKSVW approach cont.

• Let P be a refutation of A' I0 Ui Fi

• Let E be set of constraints used in proof P only on state si-1:

E = { c C | ci-2 occurs in P }

s0 s1 s2 si-1 si...

OK OK OK OK NO!

add constraints used here


Weakness of Cex-based approach

• Arbitrarily chosen abstract Cex may be refutable for many reasons not related to property.– Thus, may add irrelevant constraints.– To remedy, may try to characterize a set of

Cex's rather than just one (e.g., GKM-HFV,TACAS03).

Alternative: don't use counterexamples


Proof-based abstraction

BMCat depth k

Cex?done

No Cex?

Use refutation to choose abstraction

MC abstraction doneTrue?

False?

Incr

ease

kMA,TACAS03


BMC phase

• Unfold the model k times: U = C0 C1 ... Ck-1

• Use SAT solver to check satisfiability of I0 U Fk

• If unsatisfiable:• property has no Cex of length k• produce a refutation proof P


Abstraction phase

• Let C' be set of constraints used in proof P:C' = { c C | some ci occurs in P }

• C' admits no counterexample of length k– let U' = C'0 C'1 ... C'k-1

– P is a refutation of I0 U' Fk

• Model check property on C'– property true for C' implies true for C– else Cex of length k' > k (why?)

• restart for k = k'


Algorithm

BMCC at depth k

Cex?done

No Cex?

Refutation P inducesabstraction C'

Model check C' doneTrue?

Cex of depth k'?

let

k =

k'

Notice: MC counterexample is thrown away!


Termination

• Depth k increases at each iteration• Eventually k > d, diameter of C'• If k > d, no counterexample is possible

In practice, termination uses occurs when k d/2

Usually, diameter C' << diameter of C


Weakness of proof-based abs

• BMC must refute all counterexamples of length k, while in Cex-based, BMC must refute only one (partial) counterexample.


PicoJavaII benchmarks

• Hardware Java virtual machine implementation• Properties derived from verification of ICU

– handles cache, instruction prefetch and decode

• Original abstraction was manual• Added neigboring IFU to make problem harder

ICU IFUMem,Cache

Integerunit

properties

No properties can be verified by standard model checking!


Abstraction results

solid = original, gray = manual, open = proof-based abstraction

345

305 306 306 305

104

307

73

97

52 54

292

312

285

126

354

289

212

151

51

0

50

100

150

200

250

300

350

400S

tate

va

ria

ble

s


Inference

• SAT solver seems to be very effective at narrowing down the proof to relevant facts.

In most cases, it did better than manual abstraction.


Comparing CBA and PBA

• Apples-apples comparison– same SAT solver– same model checker– only differences are:

• For CBA previous A' is kept as a constriaint for BMC, C' is cumulative.

• For PBA previous A' and C' are thrown away each iteration.

Note these are my implementations. This says nothingabout performance of specific tools!


Run time comparison

0.01

0.1

1

10

100

1000

0.01 0.1 1 10 100 1000

Counterexample-based abstraction

Proo

f-b

ased

ab

stra

ctio

n


Abstraction comparison

0

10

20

30

40

50

60

0 10 20 30 40 50 60

Counterexample-based abstraction

Proo

f-b

ased

ab

stra

ctio

n


Possible explanation

• Internally, SAT solver is really doing CBA

a=0b=1c=0d=1

decision stack= abstract Cex A'

refutation of A'

decision heuristicmoves proof variablesup, into A'


PBA run-time breakdown

0%

50%

100%

solid = BMC time, open = MC time


CBA run-time breakdown

0%

50%

100%

solid = BMC time, open = MC time


A (fuzzy) hypothesis

• Parameterized models allowing no abstraction

SAT-based BMC "succeeds" when number of relevant variables is small, and fails otherwise.

"success" is BMC for k = diameter of relevant logic

Model Max state vars

German protocol 42

"swap" 21


Industrial benchmarks

0

100

200

300

400

500

600

700

0 100 200 300 400 500 600 700

Original state variables

Ab

stra

ctio

n s

tate

var

iab

les


Implications for model checking

• Most of the time if bounded model checking succeeds, unbounded model checking also succeeds using abstraction.

• No need to settle for time bounded result


Conclusions

• SAT solvers are very effective at ignoring irrelevant facts– Can think of decision heuristic as a form of CBA

• implications for improving heuristics?

– Hence very similar performance of CBA and PBA for localization abstraction

• SAT solver performance is tied to number of relevant variables– Performs well if there is a small UNSAT "core"– Performs badly when all variables relevant.

exploiting sat solvers in unbounded model checking k. l. mcmillan cadence berkeley labs

Documents

b c p g

c d b c d

g c x c

b b c d b d

c c c property kurshan

b c d d slide

b g g p c p g c p cnfp

b c resolve conflict