a modular checker for multithreaded programs

1

A Modular Checker for Multithreaded Programs

Cormac FlanaganHP Systems Research Center

Joint work withShaz Qadeer Sanjit A. Seshia

2

Multithreaded programs

• Operating systems, databases, web servers, file systems, web browsers– Large programs, many threads

• Correctness problem:– for all inputs and interleavings, program

satisfies specification• Testing ?

– Frangipani filesystem took 1 year of testing– Linux kernel version 2.2 -> 2.4

• fine-grained locking to support multiprocessors• 2.4.* change log has 36 synchronization bug fixes

3

Verifying multithreaded programs

• Shared-memory• Many threads• Many procedures• Many synchronization

idioms– locks (mutex, reader-writer)– semaphores (binary, counting)– changes in locking discipline– interrupt levels

Ashcroft 75Owicki-Gries 76Lamport 77Jones 83...

• Many lines of code• Many language features

Sequential checkers:...Extended Static Checking Dijkstra’s weakest precond. Automatic theorem proving...

– dynamic creation heap objects, unbounded # objs

– pointers, pointer chasing, pointer smashing, ...

– aliasing, arrays, arithmetic– classes, method, inheritance, overriding, ...

4

Contribution

• New verification methodology for MT programs– thread-modular reasoning– procedure-modular reasoning

• novel procedure specs. that respect module boundaries

– generalization of Jones’ rely-guarantee method• Calvin

– implements our methodology– automatic checker for multithreaded Java

programs– leverages sequential checking technology (ESC)

5

Simple multithreaded program

Thread 1:

acquire( )x := x + 1assert x > 0release( )

Thread 2:

acquire( )x := 0release( )

Main module: int x := 0

release( ) { m := 0}

int m := 0 // m = 0 => lock is not held // m = tid => lock held by thread tid

// tid is id of current thread (non-zero int)acquire( ) { int t := tid while (t != 0) CAS(m,0,t)}

Mutex module:

if (m = 0) then m,t := t,m

6

Problems

1. Many threads use thread-modular (assume-guarantee)

reasoning

7

Other threads

int t := 1

while (t != 0) CAS(m,0,t) x := x + 1 assert x > 0 m := 0

*

*

*

*

*

*

Env assumptionA1* int t := 1A1*while (t != 0) A1* CAS(m,0,t) A1* x := x + 1A1* assert x > 0A1* m := 0

Simplify A1 int t := 1A1

while (t != 0) A1

CAS(m,0,t) A1 x := x + 1A1 assert x > 0A1 m := 0

Thread 1

acquire()

x := x + 1

assert x > 0

release()

Inline body

int t := 1

while (t != 0)

CAS(m,0,t)

x := x + 1

assert x > 0

m := 0

Invariant I m = 0 x >= 0

Env. assumption Atid I I’ m = tid m’ = m x’ = xm,x

• Check using ESC

• Also check actions of Thread1 satisfies A2

• Similar checks for Thread2

Program is OK

8

Calvin version 1

• Verifies multithreaded Java programs• Inlines procedures• Thread-modular reasoning

– requires environment assumption– reduces correctness of MT program to

correctness of sequential programs• Leverages Extended Static Checker for Java

– generates VC for sequential programs– uses automatic theorem prover to check VC

9

The Apprentice Challenge

• Proposed by Moore and Porter– 50 lines of Java code– requires reasoning about many language

features• pointers, aliasing, arithmetic, infinite state• dynamic allocation of new objects & threads

• ACL2: 1 week• Calvin: 1 afternoon, 6 annotations

10

Problems

1. Many threads use thread-modular (assume-guarantee)

reasoning

2. Many procedures use procedure-modular reasoning

11

What are the specifications?

Thread 1:

acquire( )x := x + 1assert x > 0release( )

Thread 2:

acquire( )x := 0release( )

Main module: int x := 0

acquire( ) { int t := tid while (t != 0) CAS(m,0,t)}

release( ) { m := 0}

Mutex module: int m := 0

spec. for acquire( ) spec. for release( )

12

Procedure specification (Jones)

procedurebody

precondition

postcondition

environmentassumption

guarantee

13

Simple multithreaded program

Thread 1:

acquire( )x := x + 1assert x > 0release( )

Thread 2:

acquire( )x := 0release( )

Main module: int x := 0

acquire( ) { int t := tid while (t != 0) CAS(m,0,t)}

release( ) { m := 0}

Mutex module: int m := 0 precondition: Ienv_assumption: I I’ m = tid m’ = m x’ = xm,x

guarantee: I I’ m != 0 m’ = mm postcondition: m = tid x >= 0

14

Procedure specifications (Calvin)

procedurebody

precondition

postcondition

environmentassumption

guarantee(action)

abstraction(program)

15

Simple multithreaded program

Thread 1:

acquire( )x := x + 1assert x > 0release( )

Thread 2:

acquire( )x := 0release( )

Main module: int x := 0

acquire( ) { int t := tid while (t != 0) CAS(m,0,t)}

release( ) { m := 0}

Mutex module: int m := 0 env_assumption: true abstraction: skip* m = 0 m’ = tidm

skip* env_assumption: true abstraction: m’ = 0m

16

Other threads

skip*

m=0m’=tidm skip* x := x + 1 assert x > 0 m’ = 0m

*

*

*

*

*

*

Env assumptionA1* skip*A1*m=0m’=tidm

A1*skip* A1* x := x + 1A1* assert x > 0A1* m’ = 0m

Simplify A1

m=0m’=tidm A1

x := x + 1A1 assert x > 0A1 m’ = 0m

Thread 1

acquire()

x := x + 1

assert x > 0

release()

Inline spec

skip*

m=0m’=tidm

skip*

x := x + 1

assert x > 0

m’ = 0m

Invariant I m = 0 x >= 0

Env. assumption Atid I I’ m = tid m’ = m x’ = xm,x

• Check using ESC

• Theorem prover for additional checks

• Ditto for Thread2

Program is OK

17

Verifying Mutex module

Implementation:acquire( ) { int t := tid while (t != 0) { CAS(m,0,t) }}

Implementation is simulated by abstractionwith respect to environment assumption

env_assumption: trueabstraction: skip* m = 0 m’ = tidm

skip*

18

Simulation WitnessAbstraction:Implemetation:

m!=tid

m=tid

int t := tid

t != 0

end

CAS(m,0,t)yes

no

skip

skip

m==0 m’==tidm

m!=0

m=0

19

Calvin version 2

• Verifies multithreaded Java programs• Thread-modular reasoning• Procedure-modular reasoning

– novel procedure specification mechanism• specifications respect module boundaries

– check procedure body is simulated by abstraction– use procedure abstraction at call site

• Reduces to correctness of sequential program– leverage ESC

20

Producer-consumer

Mutex

Queue

Producer-consumer Producer || Consumer

increasingintegers

checkincreasing

acquire( ) { ...}

release( ) { ... }

spec spec

put(int n) int get()spec spec

21

Real multithreaded programs• java.util.Vector

– 400 lines of code– 2 environment assumptions

• specify locking discipline

– Calvin reported 1 race condition

• Mercator web crawler– uses reader-writer locks – specified and verified 4 locking primitives

• acquire_read(), release_read(), acquire_write(), release_write()

– checked 1500 LOC that uses reader-writer locks– verified absence of races on many data

structures

22

Daisy file system• Multithreaded

– synchronization as in high-performance FS• Verifiable file system

– small, 1500 LOC– straightforward data structures

• Functional– can mount as NFS partition

• Verified absence of race conditions• Verified specifications of many procedures

– ialloc is correct– create is atomic

23

Conclusion

• Checking multithreaded programs requires scalable and automated tools

interactive theorem provers:

PVS, ACL2, ...

model checkers:JPF, Bandera,

SPIN, ...

Calvin

Automation

Sca

lab

ility

Note: graph is not to scale