a modular checker for multithreaded programs
DESCRIPTION
A Modular Checker for Multithreaded Programs. Cormac Flanagan HP Systems Research Center Joint work with Shaz Qadeer Sanjit A. Seshia. Multithreaded programs. Operating systems, databases, web servers, file systems, web browsers Large programs, many threads Correctness problem: - PowerPoint PPT PresentationTRANSCRIPT
1
A Modular Checker for Multithreaded Programs
Cormac FlanaganHP Systems Research Center
Joint work withShaz Qadeer Sanjit A. Seshia
2
Multithreaded programs
• Operating systems, databases, web servers, file systems, web browsers– Large programs, many threads
• Correctness problem:– for all inputs and interleavings, program
satisfies specification• Testing ?
– Frangipani filesystem took 1 year of testing– Linux kernel version 2.2 -> 2.4
• fine-grained locking to support multiprocessors• 2.4.* change log has 36 synchronization bug fixes
3
Verifying multithreaded programs
• Shared-memory• Many threads• Many procedures• Many synchronization
idioms– locks (mutex, reader-writer)– semaphores (binary, counting)– changes in locking discipline– interrupt levels
Ashcroft 75Owicki-Gries 76Lamport 77Jones 83...
• Many lines of code• Many language features
Sequential checkers:...Extended Static Checking Dijkstra’s weakest precond. Automatic theorem proving...
– dynamic creation heap objects, unbounded # objs
– pointers, pointer chasing, pointer smashing, ...
– aliasing, arrays, arithmetic– classes, method, inheritance, overriding, ...
4
Contribution
• New verification methodology for MT programs– thread-modular reasoning– procedure-modular reasoning
• novel procedure specs. that respect module boundaries
– generalization of Jones’ rely-guarantee method• Calvin
– implements our methodology– automatic checker for multithreaded Java
programs– leverages sequential checking technology (ESC)
5
Simple multithreaded program
Thread 1:
acquire( )x := x + 1assert x > 0release( )
Thread 2:
acquire( )x := 0release( )
Main module: int x := 0
release( ) { m := 0}
int m := 0 // m = 0 => lock is not held // m = tid => lock held by thread tid
// tid is id of current thread (non-zero int)acquire( ) { int t := tid while (t != 0) CAS(m,0,t)}
Mutex module:
if (m = 0) then m,t := t,m
6
Problems
1. Many threads use thread-modular (assume-guarantee)
reasoning
7
Other threads
int t := 1
while (t != 0) CAS(m,0,t) x := x + 1 assert x > 0 m := 0
*
*
*
*
*
*
Env assumptionA1* int t := 1A1*while (t != 0) A1* CAS(m,0,t) A1* x := x + 1A1* assert x > 0A1* m := 0
Simplify A1 int t := 1A1
while (t != 0) A1
CAS(m,0,t) A1 x := x + 1A1 assert x > 0A1 m := 0
Thread 1
acquire()
x := x + 1
assert x > 0
release()
Inline body
int t := 1
while (t != 0)
CAS(m,0,t)
x := x + 1
assert x > 0
m := 0
Invariant I m = 0 x >= 0
Env. assumption Atid I I’ m = tid m’ = m x’ = xm,x
• Check using ESC
• Also check actions of Thread1 satisfies A2
• Similar checks for Thread2
Program is OK
8
Calvin version 1
• Verifies multithreaded Java programs• Inlines procedures• Thread-modular reasoning
– requires environment assumption– reduces correctness of MT program to
correctness of sequential programs• Leverages Extended Static Checker for Java
– generates VC for sequential programs– uses automatic theorem prover to check VC
9
The Apprentice Challenge
• Proposed by Moore and Porter– 50 lines of Java code– requires reasoning about many language
features• pointers, aliasing, arithmetic, infinite state• dynamic allocation of new objects & threads
• ACL2: 1 week• Calvin: 1 afternoon, 6 annotations
10
Problems
1. Many threads use thread-modular (assume-guarantee)
reasoning
2. Many procedures use procedure-modular reasoning
11
What are the specifications?
Thread 1:
acquire( )x := x + 1assert x > 0release( )
Thread 2:
acquire( )x := 0release( )
Main module: int x := 0
acquire( ) { int t := tid while (t != 0) CAS(m,0,t)}
release( ) { m := 0}
Mutex module: int m := 0
spec. for acquire( ) spec. for release( )
12
Procedure specification (Jones)
procedurebody
precondition
postcondition
environmentassumption
guarantee
13
Simple multithreaded program
Thread 1:
acquire( )x := x + 1assert x > 0release( )
Thread 2:
acquire( )x := 0release( )
Main module: int x := 0
acquire( ) { int t := tid while (t != 0) CAS(m,0,t)}
release( ) { m := 0}
Mutex module: int m := 0 precondition: Ienv_assumption: I I’ m = tid m’ = m x’ = xm,x
guarantee: I I’ m != 0 m’ = mm postcondition: m = tid x >= 0
14
Procedure specifications (Calvin)
procedurebody
precondition
postcondition
environmentassumption
guarantee(action)
abstraction(program)
15
Simple multithreaded program
Thread 1:
acquire( )x := x + 1assert x > 0release( )
Thread 2:
acquire( )x := 0release( )
Main module: int x := 0
acquire( ) { int t := tid while (t != 0) CAS(m,0,t)}
release( ) { m := 0}
Mutex module: int m := 0 env_assumption: true abstraction: skip* m = 0 m’ = tidm
skip* env_assumption: true abstraction: m’ = 0m
16
Other threads
skip*
m=0m’=tidm skip* x := x + 1 assert x > 0 m’ = 0m
*
*
*
*
*
*
Env assumptionA1* skip*A1*m=0m’=tidm
A1*skip* A1* x := x + 1A1* assert x > 0A1* m’ = 0m
Simplify A1
m=0m’=tidm A1
x := x + 1A1 assert x > 0A1 m’ = 0m
Thread 1
acquire()
x := x + 1
assert x > 0
release()
Inline spec
skip*
m=0m’=tidm
skip*
x := x + 1
assert x > 0
m’ = 0m
Invariant I m = 0 x >= 0
Env. assumption Atid I I’ m = tid m’ = m x’ = xm,x
• Check using ESC
• Theorem prover for additional checks
• Ditto for Thread2
Program is OK
17
Verifying Mutex module
Implementation:acquire( ) { int t := tid while (t != 0) { CAS(m,0,t) }}
Implementation is simulated by abstractionwith respect to environment assumption
env_assumption: trueabstraction: skip* m = 0 m’ = tidm
skip*
18
Simulation WitnessAbstraction:Implemetation:
m!=tid
m=tid
int t := tid
t != 0
end
CAS(m,0,t)yes
no
skip
skip
m==0 m’==tidm
m!=0
m=0
19
Calvin version 2
• Verifies multithreaded Java programs• Thread-modular reasoning• Procedure-modular reasoning
– novel procedure specification mechanism• specifications respect module boundaries
– check procedure body is simulated by abstraction– use procedure abstraction at call site
• Reduces to correctness of sequential program– leverage ESC
20
Producer-consumer
Mutex
Queue
Producer-consumer Producer || Consumer
increasingintegers
checkincreasing
acquire( ) { ...}
release( ) { ... }
spec spec
put(int n) int get()spec spec
21
Real multithreaded programs• java.util.Vector
– 400 lines of code– 2 environment assumptions
• specify locking discipline
– Calvin reported 1 race condition
• Mercator web crawler– uses reader-writer locks – specified and verified 4 locking primitives
• acquire_read(), release_read(), acquire_write(), release_write()
– checked 1500 LOC that uses reader-writer locks– verified absence of races on many data
structures
22
Daisy file system• Multithreaded
– synchronization as in high-performance FS• Verifiable file system
– small, 1500 LOC– straightforward data structures
• Functional– can mount as NFS partition
• Verified absence of race conditions• Verified specifications of many procedures
– ialloc is correct– create is atomic
23
Conclusion
• Checking multithreaded programs requires scalable and automated tools
interactive theorem provers:
PVS, ACL2, ...
model checkers:JPF, Bandera,
SPIN, ...
Calvin
Automation
Sca
lab
ility
Note: graph is not to scale