A brief history ofmodel checking

Ken McMillanCadence Berkeley Labsmcmillan@cadence.com

Outline

• Part I -- Introduction to model checking– Automatic formal verification of finite-state

systems– Applications

• Commercial hardware design• Avionics, chemical plant control, automotive, etc.

• Part II -- A brief history of model checking– Influence of many abstract ideas from logic on the

development of model checking

The Verification Problem

• Debugging chips by simulation...– consumes greater than half of design time,– is unreliable

• “Escapes” can cost up to $500M,– is increasing in cost as chip densities scale up

2

Model Checking• input:

– temporal logicspec

– finite-state model• output

– yes– no +

counterexample

(look ma, no testvectors!)

MC

G(p ⇒ F q)yes

nop

q

pq

5

Temporal logic (LTL)• A logical notation that allows to:

– specify relations in time– conveniently express finite control

properties• Temporal operators

– G p “henceforth p”– F p “eventually p”– X p “p at the next time”– p W q “p unless q”

6

Types of temporal properties

• Safety (nothing bad happens)

G ~(ack1 & ack2) “mutual exclusion”G (req ⇒ (req W ack)) “req must hold

until ack”• Liveness (something good happens)

G (req ⇒F ack) “if req, eventually ack”

• FairnessGF req ⇒ GF ack “if infinitely often req,

infinitely often ack”

7

Computation tree logic (CTL)• Branching time model• Path quantifiers

– A = “for all future paths”– E = “for some future path”

• Example: AF p = “inevitably p”

AFp

p

p

p

9

CTL model checking algorithm• Example: AF p = “inevitably p”

• Complexity– linear in size of model (FSM)– linear in size of specification formula

p

Note: LTL is exponential in formula size

AFpAFp

AFpAFp

10

Example: traffic light controller

• Guarantee no collisions• Guarantee eventual service

E

S

N

14

Specifications• Safety (no collisions)

AG ¬ (E_Go ∧ (N_Go | S_Go));

• LivenessAG (¬ N_Go ∧ N_Sense ⇒ AF N_Go);

AG (¬ S_Go ∧ S_Sense ⇒ AF S_Go); AG (¬ E_Go ∧ E_Sense ⇒ AF E_Go);

• Fairness constraintsinfinitely often ¬(N_Go ∧ N_Sense);

infinitely often ¬(S_Go ∧ S_Sense); infinitely often ¬(E_Go ∧ E_Sense);

(assume each sensor off infinitely often)

15

Counterexample

• East and North lights on at same time...E_Go

E_SenseNS_LockN_GoN_ReqN_SenseS_GoS_ReqS_Sense

E_ReqN light goes on atsame time S light goesoff.

S takes priority andresets NS_Lock

20

State explosion problem• What if the state space is too large?

– too much parallelism– data in model

• Approaches– Abstraction/reduction– “Symbolic” methods– Exploiting symmetry– “Partial order” methods

21

Binary Decision Diagrams• Ordered decision tree for f = ab + cd

0 0 0 1 0 0 0 1 0 0 0 1 1 1 1 1

d d d d d d d d

c c c c

0 1

0 1 0 1

0 1 0 1 0 1 0 1

b b

a

22

OBDD reduction• Reduced (OBDD) form:

0 1

d

c

01

0 1

0 1

b

a

0

1

Key idea: combine equivalent subcases

24

Symbolic model checking

• Basic idea:– Use BDD’s to represent sets and relations– Avoid explicitly representing states

• Transition relations

a,b a’,b’R(a,b,a’,b’)

25

Image computation• EX p = states that can reach p in one step

EXp p

EX p = ∃ v’. (R(v,v’) ∧ p(v’))

Note: ∃ a. f = f |a=0 + f |a=1

26

Fixed point iteration• EF p = states that can reach p

S0 = pS1...Sw

Si+1 = Si \/ EX Si

...Model checking without building state graph

33

Example: “Gigamax” cache protocol

• First commercial application• Method scales well with system size• Finds very subtle “escapes”

M P P . . .

cluster bus

M P P . . .

. . .

global bus

UIC

UIC

UIC

. . .

Genealogy of model checking

Logics ofPrograms

Temporal/Modal Logics

CTL ModelChecking

SymbolicModel Checking

ω-automataS1S

LTLMCATV

Tarski

µ-calc

QBF BDD

Many ideas from logic influence developmentof model checking...

Logics of programs

• Floyd/Hoare/Dijkstra– Give precise definitions of programming languages– Allows reasoning about programs

(proofs/derivations)– Pre-post conditions/ weakest precondition

• example: assignment axioms

{true} x :=y {x = y}

{P} x := y {P} (no x in P)

• Pnueli– Concurrent vs. sequential programming

– need to characterize execution sequences– proposes use of temporal logic

Concurrent programs

sequentialA B

concurrentA B

call

ret

Temporal and modal logics

• Roots in philosophical logic– Tense logic -- formalizing linguistic time

“If a, then b before c”– Modal logic -- reasoning about possibility

“If I had run I would have caught my plane”

• New use in computer science:– characterize the interactions of parallel processes

G req ⇒ F ack

GenealogyLogics ofPrograms

Temporal/Modal Logics

Pnueli, late 70’s

Floyd/Hoarelate ‘60’s

Aristotle 300’sBCEKripke ‘59

CTL Model checking

• Reasoning about properties of non-deterministic programs– branching time properties of programs– fixed point characterizations (Tarski)

• every monotonic function has least/greatest fixed point– key idea: apply to finite graphs, not infinite trees

• can directly calculate Tarski fixed points

• Applications– finite state machines in hardware– protocols– proved incorrectness of some published designs

Genealogy, contLogics ofPrograms

Temporal/Modal Logics

CTL ModelChecking

Tarski

Clarke/EmersonEarly 80’s

50’s

Some published circuits are proved incorrect

Decidable logics and automata

• Büchi– S1S -- reason about sets of natural numbers– Automata on infinite words

• characterize set of models of formula• example: sets that contain the odd numbers

– Deep connection between logics and automata

0,1

0,1

1

0

LTL model checking

• Vardi and Wolper– Apply Büchi’s technique to LTL– Automaton construction yields optimal decision

algorithm• Kurshan

– Specify properties directly as automata• example: infinitely often p (GFp)

p

true

¬p

GenealogyLogics ofPrograms

Temporal/Modal Logics

CTL ModelChecking

ω-automataS1S

LTLMCATV

TarskiBüchi, 60

Kurshan Vardi/Wolper

mid 80’s

Symbolic Model Checking

• State explosion problem– graph model guarantees worst-case complexity

• Characterize sets and relations by Booleanformulas– compute Tarski fixed points directly on formulas

– Use BDD’s to represent formulas• efficient canonical form

EXp = ∃v′. (R ∧ p′) (QBF)

Mu-calculus

• Park’s Mu-Calculus– Logic of relations with fixed point operator– Can express transitive closure– Nicely characterizes what SMC can compute

• SMC algorithm for Mu-calculus– Use to express symbolic algorithms for

• CTL, LTL model checking

• Automaton containment, etc...– Note: bad specification logic, but good for

describing algorithms

AFp = µQ. p ∨ AX Q

Genealogy, cont.

– Note first commercial application in 1990• Encore Gigamax cache protocols

Logics ofPrograms

Temporal/Modal Logics

CTL ModelChecking

SymbolicModel Checking

ω-automataS1S

LTLMCATV

Tarski

µ-calc

QBF BDD

Park60’s

Bryantmid 80’s

late 80’s

Applications

• Hardware Design– Encore Gigamax– Intel instruction decoder– SGI cache protocol chip

• Other areas– Avionics (TCAS)– Chemical plant control– Nuclear storage facilities (!)

• Commercial tools– Cadence, IBM, Synopsys

A convergence of research areasin logic

• Many areas of logic have shaped the discoursein model checking– Logics of programs– Temporal/Modal logics– Tarski fixed point theory– Decidable logics -- S1S/automata– Park’s mu-calculus

• Much of this work is quite abstract, but hasstrongly influenced practical work in modelchecking