testing and verifying atomicity of composed concurrent operations ohad shacham tel aviv university...
TRANSCRIPT
![Page 1: Testing and Verifying Atomicity of Composed Concurrent Operations Ohad Shacham Tel Aviv University Nathan Bronson Stanford University Alex Aiken Stanford](https://reader036.vdocuments.mx/reader036/viewer/2022062423/56649e9f5503460f94ba1e9b/html5/thumbnails/1.jpg)
Testing and VerifyingAtomicity of Composed Concurrent Operations
Ohad Shacham Tel Aviv University
Nathan Bronson Stanford University
Alex Aiken Stanford University
Mooly Sagiv Tel Aviv University
Martin Vechev ETH
Eran Yahav Technion
![Page 2: Testing and Verifying Atomicity of Composed Concurrent Operations Ohad Shacham Tel Aviv University Nathan Bronson Stanford University Alex Aiken Stanford](https://reader036.vdocuments.mx/reader036/viewer/2022062423/56649e9f5503460f94ba1e9b/html5/thumbnails/2.jpg)
Specifying and Verifying Software Composition
• Efficient libraries are widely available• Composing software in a way which
guarantee correctness:– Specification– Verification– Synthesis– Performance
![Page 3: Testing and Verifying Atomicity of Composed Concurrent Operations Ohad Shacham Tel Aviv University Nathan Bronson Stanford University Alex Aiken Stanford](https://reader036.vdocuments.mx/reader036/viewer/2022062423/56649e9f5503460f94ba1e9b/html5/thumbnails/3.jpg)
Concurrent Data Structures
• Writing highly concurrent data structures is complicated
• Modern programming languages provide efficient concurrent collections with atomic operations
.
.
…
……
…
…
………
![Page 4: Testing and Verifying Atomicity of Composed Concurrent Operations Ohad Shacham Tel Aviv University Nathan Bronson Stanford University Alex Aiken Stanford](https://reader036.vdocuments.mx/reader036/viewer/2022062423/56649e9f5503460f94ba1e9b/html5/thumbnails/4.jpg)
attr = new HashMap();…
Attribute removeAttribute(String name){ Attribute val = null; synchronized(attr) { found = attr.containsKey(name) ; if (found) { val = attr.get(name); attr.remove(name); } } return val;}
attr = new ConcurrentHashMap();…
Attribute removeAttribute(String name){ Attribute val = null; /* synchronized(attr) { */ found = attr.containsKey(name) ; if (found) { val = attr.get(name); attr.remove(name); } /* } */ return val;}
TOMCAT Motivating ExampleTOMCAT 5*.TOMCAT 6*.
Invariant: removeAttribute(name) returns the removed value or null if it does not exist
![Page 5: Testing and Verifying Atomicity of Composed Concurrent Operations Ohad Shacham Tel Aviv University Nathan Bronson Stanford University Alex Aiken Stanford](https://reader036.vdocuments.mx/reader036/viewer/2022062423/56649e9f5503460f94ba1e9b/html5/thumbnails/5.jpg)
removeAttribute(“A”) {Attribute val = null;
found = attr.containsKey(“A”); if (found) {
val = attr.get(“A”);
attr.remove(“A”); } return val;
o
attr.put(“A”, o);
attr.remove(“A”);
Invariant: removeAttribute(name) returns the removed value or null if it does not exist
![Page 6: Testing and Verifying Atomicity of Composed Concurrent Operations Ohad Shacham Tel Aviv University Nathan Bronson Stanford University Alex Aiken Stanford](https://reader036.vdocuments.mx/reader036/viewer/2022062423/56649e9f5503460f94ba1e9b/html5/thumbnails/6.jpg)
Challenge
Testing and Verifying the atomicity of composed operations
…
……
…
…
………
![Page 7: Testing and Verifying Atomicity of Composed Concurrent Operations Ohad Shacham Tel Aviv University Nathan Bronson Stanford University Alex Aiken Stanford](https://reader036.vdocuments.mx/reader036/viewer/2022062423/56649e9f5503460f94ba1e9b/html5/thumbnails/7.jpg)
Challenges in Testing
• Specifying software correctness• Bugs occur in rarely executed traces
– Especially true in concurrent systems
• Scalability of dynamic checking– large traces
• Hard to find programs to test
![Page 8: Testing and Verifying Atomicity of Composed Concurrent Operations Ohad Shacham Tel Aviv University Nathan Bronson Stanford University Alex Aiken Stanford](https://reader036.vdocuments.mx/reader036/viewer/2022062423/56649e9f5503460f94ba1e9b/html5/thumbnails/8.jpg)
Challenges in Verification
• Specifying software correctness• Many sources of unboundedness
– Data• Integers• Stack• Heap• …
– Interleavings• Scalability of static checking
– Large programs• Hard to find programs to verify
![Page 9: Testing and Verifying Atomicity of Composed Concurrent Operations Ohad Shacham Tel Aviv University Nathan Bronson Stanford University Alex Aiken Stanford](https://reader036.vdocuments.mx/reader036/viewer/2022062423/56649e9f5503460f94ba1e9b/html5/thumbnails/9.jpg)
Testing atomicity ofcomposed operations
OOPSLA’11
![Page 10: Testing and Verifying Atomicity of Composed Concurrent Operations Ohad Shacham Tel Aviv University Nathan Bronson Stanford University Alex Aiken Stanford](https://reader036.vdocuments.mx/reader036/viewer/2022062423/56649e9f5503460f94ba1e9b/html5/thumbnails/10.jpg)
Challenge 1: Long traces
• Assume that composed operations are written inside encapsulated methods
• Modular testing– Unit testing in all contexts– Composed operations need to be correct in all
contexts
• May lead to false warnings
![Page 11: Testing and Verifying Atomicity of Composed Concurrent Operations Ohad Shacham Tel Aviv University Nathan Bronson Stanford University Alex Aiken Stanford](https://reader036.vdocuments.mx/reader036/viewer/2022062423/56649e9f5503460f94ba1e9b/html5/thumbnails/11.jpg)
False Warning
if (m.contains(k))
return m.get(k);
else
return k;
• False warning in clients without remove• Sometimes indicate “future bugs”
m.remove(k);
![Page 12: Testing and Verifying Atomicity of Composed Concurrent Operations Ohad Shacham Tel Aviv University Nathan Bronson Stanford University Alex Aiken Stanford](https://reader036.vdocuments.mx/reader036/viewer/2022062423/56649e9f5503460f94ba1e9b/html5/thumbnails/12.jpg)
Challenge 2: Specification
• Check that composed operations are Linearizable [Herlihy & Wing, TOPLAS’90]– Returns the same result as some sequential run
![Page 13: Testing and Verifying Atomicity of Composed Concurrent Operations Ohad Shacham Tel Aviv University Nathan Bronson Stanford University Alex Aiken Stanford](https://reader036.vdocuments.mx/reader036/viewer/2022062423/56649e9f5503460f94ba1e9b/html5/thumbnails/13.jpg)
Linearizability removeAttribute(“A”) {
Attribute val = null;
found = attr.containsKey(“A”); if (found) {
val = attr.get(“A”);
attr.remove(“A”); } return val;
attr.put(“A”, o);
attr.put(“A”, o);
attr.remove(“A”);
removeAttribute(“A”) {Attribute val = null;
found = attr.containsKey(“A”); if (found) { return val;
removeAttribute(“A”) {Attribute val = null;
found = attr.containsKey(“A”); if (found) { return val; attr.put(“A”, o);
attr.remove(“A”);
attr.put(“A”, o);
attr.remove(“A”);
removeAttribute(“A”) {Attribute val = null;
found = attr.containsKey(“A”) ; if (found) {
val = attr.get(“A”); attr.remove(“A”);
} return val;
o
o
null
null
o
null
null
o
null
null
o
null
attr.remove(“A”);
![Page 14: Testing and Verifying Atomicity of Composed Concurrent Operations Ohad Shacham Tel Aviv University Nathan Bronson Stanford University Alex Aiken Stanford](https://reader036.vdocuments.mx/reader036/viewer/2022062423/56649e9f5503460f94ba1e9b/html5/thumbnails/14.jpg)
But Linearizability errors only occur in rarely executed paths
removeAttribute(“A”) {Attribute val = null;
found = attr.containsKey(“A”); if (found) {
val = attr.get(“A”);
attr.remove(“A”); } return val;
attr.put(“A”, o);
attr.remove(“A”);
![Page 15: Testing and Verifying Atomicity of Composed Concurrent Operations Ohad Shacham Tel Aviv University Nathan Bronson Stanford University Alex Aiken Stanford](https://reader036.vdocuments.mx/reader036/viewer/2022062423/56649e9f5503460f94ba1e9b/html5/thumbnails/15.jpg)
Linearizability errors only occur in rarely executed paths
• Only consider “atomic” executions of base collection operations [TACAS’10, Ball et. al.]
• Employ commutativity/influence of base collection operations– Operations on different key commute– Partial order reduction using the collection interface
![Page 16: Testing and Verifying Atomicity of Composed Concurrent Operations Ohad Shacham Tel Aviv University Nathan Bronson Stanford University Alex Aiken Stanford](https://reader036.vdocuments.mx/reader036/viewer/2022062423/56649e9f5503460f94ba1e9b/html5/thumbnails/16.jpg)
Influence table
Operation Condition Potential Action
get(k) get(k) == null put(k,*)
get(k) get(k) != null remove(k)
containsKey(k) get(k) == null put(k,*)
containsKey(k) get(k) != null remove(k)
remove(k) get(k) == null put(k,*)
remove(k) get(k) != null remove(k)
![Page 17: Testing and Verifying Atomicity of Composed Concurrent Operations Ohad Shacham Tel Aviv University Nathan Bronson Stanford University Alex Aiken Stanford](https://reader036.vdocuments.mx/reader036/viewer/2022062423/56649e9f5503460f94ba1e9b/html5/thumbnails/17.jpg)
COLT Tester
program
CO extractor
candidateCOs
Timeout
instrument linearizability
checking
CO
key/value driver
Non-Lin
Execution
library spec
influence driver
![Page 18: Testing and Verifying Atomicity of Composed Concurrent Operations Ohad Shacham Tel Aviv University Nathan Bronson Stanford University Alex Aiken Stanford](https://reader036.vdocuments.mx/reader036/viewer/2022062423/56649e9f5503460f94ba1e9b/html5/thumbnails/18.jpg)
removeAttribute(“A”) {Attribute val = null;
found = attr.containsKey(“A”); if (found) {
val = attr.get(“A”);
attr.remove(“A”); } return val;
Attribute removeAttribute(String name){ Attribute val = null; found = attr.containsKey(name) ; if (found) { val = attr.get(name); attr.remove(name); } return val;}
attr.put(“A”, o);
attr.remove(“A”);
o
o
null
![Page 19: Testing and Verifying Atomicity of Composed Concurrent Operations Ohad Shacham Tel Aviv University Nathan Bronson Stanford University Alex Aiken Stanford](https://reader036.vdocuments.mx/reader036/viewer/2022062423/56649e9f5503460f94ba1e9b/html5/thumbnails/19.jpg)
attr.put(“A”, o);
attr.remove(“A”);
removeAttribute(“A”) {Attribute val = null;
found = attr.containsKey(“A”); if (found) { return val;
removeAttribute(“A”) {Attribute val = null;
found = attr.containsKey(“A”); if (found) { return val; attr.put(“A”, o);
attr.remove(“A”);
attr.put(“A”, o);
attr.remove(“A”);
removeAttribute(“A”) {Attribute val = null;
found = attr.containsKey(“A”) ; if (found) {
val = attr.get(“A”); attr.remove(“A”);
} return val;
o
o
null
null
o
null
null
o
null
null
o
null
removeAttribute(“A”) {Attribute val = null;
found = attr.containsKey(“A”); if (found) {
val = attr.get(“A”);
attr.remove(“A”); } return val;
attr.put(“A”, o);
attr.remove(“A”);
![Page 20: Testing and Verifying Atomicity of Composed Concurrent Operations Ohad Shacham Tel Aviv University Nathan Bronson Stanford University Alex Aiken Stanford](https://reader036.vdocuments.mx/reader036/viewer/2022062423/56649e9f5503460f94ba1e9b/html5/thumbnails/20.jpg)
Evaluation
• Use Google code search and Koders to search for collection operations methods with at least two operations
• Used simple static analysis to extract composed operations– 29% needed manual modification
• Check Linearizability of all public domain composed operations
• Extracted 112 composed operations from 55 applications– Apache Tomcat, Cassandra, MyFaces – Trinidad, …
• Each run took less than a second• Without influence timeout always occur
![Page 21: Testing and Verifying Atomicity of Composed Concurrent Operations Ohad Shacham Tel Aviv University Nathan Bronson Stanford University Alex Aiken Stanford](https://reader036.vdocuments.mx/reader036/viewer/2022062423/56649e9f5503460f94ba1e9b/html5/thumbnails/21.jpg)
112Unknown
![Page 22: Testing and Verifying Atomicity of Composed Concurrent Operations Ohad Shacham Tel Aviv University Nathan Bronson Stanford University Alex Aiken Stanford](https://reader036.vdocuments.mx/reader036/viewer/2022062423/56649e9f5503460f94ba1e9b/html5/thumbnails/22.jpg)
59Non
Linearizable
53Unknown
![Page 23: Testing and Verifying Atomicity of Composed Concurrent Operations Ohad Shacham Tel Aviv University Nathan Bronson Stanford University Alex Aiken Stanford](https://reader036.vdocuments.mx/reader036/viewer/2022062423/56649e9f5503460f94ba1e9b/html5/thumbnails/23.jpg)
53Unknown
42Non
Linearizable
17Open Non
Linearizable
![Page 24: Testing and Verifying Atomicity of Composed Concurrent Operations Ohad Shacham Tel Aviv University Nathan Bronson Stanford University Alex Aiken Stanford](https://reader036.vdocuments.mx/reader036/viewer/2022062423/56649e9f5503460f94ba1e9b/html5/thumbnails/24.jpg)
17Open Non
Linearizable
42Non
Linearizable
31Linearizable
22Globals
![Page 25: Testing and Verifying Atomicity of Composed Concurrent Operations Ohad Shacham Tel Aviv University Nathan Bronson Stanford University Alex Aiken Stanford](https://reader036.vdocuments.mx/reader036/viewer/2022062423/56649e9f5503460f94ba1e9b/html5/thumbnails/25.jpg)
31Linearizable
81Non-Linearizable
![Page 26: Testing and Verifying Atomicity of Composed Concurrent Operations Ohad Shacham Tel Aviv University Nathan Bronson Stanford University Alex Aiken Stanford](https://reader036.vdocuments.mx/reader036/viewer/2022062423/56649e9f5503460f94ba1e9b/html5/thumbnails/26.jpg)
Results
• Reported the bugs with fixes• Even bugs in open environment• As a result of the paper the Java library is being
changed“A preliminary version is in the pre-java8 "jsr166e" packageas ConcurrentHashMapV8. We can't release the actual versionyet because it relies on Java8 lambda (closure) syntax support.See links fromhttp://gee.cs.oswego.edu/dl/concurrency-interest/index.htmlincluding:http://gee.cs.oswego.edu/dl/jsr166/dist/jsr166edocs/jsr166e/ConcurrentHashMapV8.html
Good luck continuing to find errors and misuses that canhelp us create better concurrency components!”
![Page 27: Testing and Verifying Atomicity of Composed Concurrent Operations Ohad Shacham Tel Aviv University Nathan Bronson Stanford University Alex Aiken Stanford](https://reader036.vdocuments.mx/reader036/viewer/2022062423/56649e9f5503460f94ba1e9b/html5/thumbnails/27.jpg)
Verifying atomicity ofcomposed operations
![Page 28: Testing and Verifying Atomicity of Composed Concurrent Operations Ohad Shacham Tel Aviv University Nathan Bronson Stanford University Alex Aiken Stanford](https://reader036.vdocuments.mx/reader036/viewer/2022062423/56649e9f5503460f94ba1e9b/html5/thumbnails/28.jpg)
Motivation
• Unbounded number of potential composed operations– There exists no “thick” interface
• Automatically prove Linearizability for composed operations beyond the ones provided– Already supports the existing interface– No higher order functions
• Zero false alarms– beyond modularity
![Page 29: Testing and Verifying Atomicity of Composed Concurrent Operations Ohad Shacham Tel Aviv University Nathan Bronson Stanford University Alex Aiken Stanford](https://reader036.vdocuments.mx/reader036/viewer/2022062423/56649e9f5503460f94ba1e9b/html5/thumbnails/29.jpg)
Attribute removeAttribute(String name){ Attribute val = null; found = attr.containsKey(name) ; if (found) { val = attr.get(name); attr.remove(name); } return val;}
Attribute removeAttribute(String name){ Attribute val = null; found = attr.containsKey(name) ; if (found) { val = attr.get(name); attr.remove(name); } return val;}
Attribute removeAttribute(String name){ Attribute val = null; found = attr.containsKey(name) ; if (found) { val = attr.get(name); attr.remove(name); } return val;}
Data independent [Wolper, POPL’86]
Attribute removeAttribute(String name){ Attribute val = null; found = attr.containsKey(name) ; if (found) { val = attr.get(name); attr.remove(name); } return val;}
![Page 30: Testing and Verifying Atomicity of Composed Concurrent Operations Ohad Shacham Tel Aviv University Nathan Bronson Stanford University Alex Aiken Stanford](https://reader036.vdocuments.mx/reader036/viewer/2022062423/56649e9f5503460f94ba1e9b/html5/thumbnails/30.jpg)
Verifying data independent operations using Linearization points in the code
Data independent Verified using single input
Influence CO adds one value
Map elements are bounded
Single Mutation
![Page 31: Testing and Verifying Atomicity of Composed Concurrent Operations Ohad Shacham Tel Aviv University Nathan Bronson Stanford University Alex Aiken Stanford](https://reader036.vdocuments.mx/reader036/viewer/2022062423/56649e9f5503460f94ba1e9b/html5/thumbnails/31.jpg)
Verifying data independent operations
• Small model reduction• Decidable when the local state is bounded• Explore all possible executions using:
– One input key and finite number of values– Influenced based environment uses single value
• Employ SPIN
![Page 32: Testing and Verifying Atomicity of Composed Concurrent Operations Ohad Shacham Tel Aviv University Nathan Bronson Stanford University Alex Aiken Stanford](https://reader036.vdocuments.mx/reader036/viewer/2022062423/56649e9f5503460f94ba1e9b/html5/thumbnails/32.jpg)
![Page 33: Testing and Verifying Atomicity of Composed Concurrent Operations Ohad Shacham Tel Aviv University Nathan Bronson Stanford University Alex Aiken Stanford](https://reader036.vdocuments.mx/reader036/viewer/2022062423/56649e9f5503460f94ba1e9b/html5/thumbnails/33.jpg)
![Page 34: Testing and Verifying Atomicity of Composed Concurrent Operations Ohad Shacham Tel Aviv University Nathan Bronson Stanford University Alex Aiken Stanford](https://reader036.vdocuments.mx/reader036/viewer/2022062423/56649e9f5503460f94ba1e9b/html5/thumbnails/34.jpg)
program
Composed Operation extractor
candidateCOs
CO
Library spec
Data Independentverifier
SCM/FCMInput keys/values
Lin
Linearizability verifier
generator
Non-Lin
CO
SPIN
Promela
Input keys/values
Linearizability tester
generator
CO
Execution
Java
No SCM
key/value driverInfluence driver
Unknown
Non-Lin
Influence driver
![Page 35: Testing and Verifying Atomicity of Composed Concurrent Operations Ohad Shacham Tel Aviv University Nathan Bronson Stanford University Alex Aiken Stanford](https://reader036.vdocuments.mx/reader036/viewer/2022062423/56649e9f5503460f94ba1e9b/html5/thumbnails/35.jpg)
31Linearizable
81Non-Linearizable
![Page 36: Testing and Verifying Atomicity of Composed Concurrent Operations Ohad Shacham Tel Aviv University Nathan Bronson Stanford University Alex Aiken Stanford](https://reader036.vdocuments.mx/reader036/viewer/2022062423/56649e9f5503460f94ba1e9b/html5/thumbnails/36.jpg)
Summary
• Writing concurrent data structures is hard• Employing atomic library operations is error prone• Modular linearizability checking• Leverage influence• Leverage data independence
• Sweet spot– Identify important bugs together with a traces showing and
explaining the violations– Hard to find– Prove the linearizability of several composed operations– Simple and efficient technique