model counting a quest for nails 2 willem visser stellenbosch university joint work with matt dwyer...
TRANSCRIPT
![Page 1: Model Counting A Quest for Nails 2 Willem Visser Stellenbosch University Joint work with Matt Dwyer (UNL, USA) Jaco Geldenhuys (SU, RSA) Corina Pasareanu](https://reader036.vdocuments.mx/reader036/viewer/2022070400/56649f135503460f94c26a2b/html5/thumbnails/1.jpg)
Model CountingA Quest for Nails 2
Willem VisserStellenbosch University
Joint work with Matt Dwyer (UNL, USA)
Jaco Geldenhuys (SU, RSA) Corina Pasareanu (NASA, USA)
Antonio Filieri (Stuttgart, Germany)
![Page 2: Model Counting A Quest for Nails 2 Willem Visser Stellenbosch University Joint work with Matt Dwyer (UNL, USA) Jaco Geldenhuys (SU, RSA) Corina Pasareanu](https://reader036.vdocuments.mx/reader036/viewer/2022070400/56649f135503460f94c26a2b/html5/thumbnails/2.jpg)
Stellenbosch?
![Page 3: Model Counting A Quest for Nails 2 Willem Visser Stellenbosch University Joint work with Matt Dwyer (UNL, USA) Jaco Geldenhuys (SU, RSA) Corina Pasareanu](https://reader036.vdocuments.mx/reader036/viewer/2022070400/56649f135503460f94c26a2b/html5/thumbnails/3.jpg)
![Page 4: Model Counting A Quest for Nails 2 Willem Visser Stellenbosch University Joint work with Matt Dwyer (UNL, USA) Jaco Geldenhuys (SU, RSA) Corina Pasareanu](https://reader036.vdocuments.mx/reader036/viewer/2022070400/56649f135503460f94c26a2b/html5/thumbnails/4.jpg)
Saving the Whooping Crane
![Page 5: Model Counting A Quest for Nails 2 Willem Visser Stellenbosch University Joint work with Matt Dwyer (UNL, USA) Jaco Geldenhuys (SU, RSA) Corina Pasareanu](https://reader036.vdocuments.mx/reader036/viewer/2022070400/56649f135503460f94c26a2b/html5/thumbnails/5.jpg)
PC = C1 & C2 & … & Cn
PC solutions PC feasibility
>0
![Page 6: Model Counting A Quest for Nails 2 Willem Visser Stellenbosch University Joint work with Matt Dwyer (UNL, USA) Jaco Geldenhuys (SU, RSA) Corina Pasareanu](https://reader036.vdocuments.mx/reader036/viewer/2022070400/56649f135503460f94c26a2b/html5/thumbnails/6.jpg)
Resources• ISSTA 2012
– Probabilistic Symbolic Execution
• FSE 2012– Green: Reduce, Reuse and Recycle Constraints…
• ICSE 2013– Software Reliability with Symbolic PathFinder
• PLDI 2014– Compositional Solution Space Quantification for Probabilistic Software Analysis
• FSE 2014– Statistical Symbolic Execution with Informed Sampling
• ASE 2014 Submitted– Exact and Approximate Probabilistic Symbolic Execution for Nondeterministic Programs
• Implemented in Symbolic PathFinder– Using LattE
![Page 7: Model Counting A Quest for Nails 2 Willem Visser Stellenbosch University Joint work with Matt Dwyer (UNL, USA) Jaco Geldenhuys (SU, RSA) Corina Pasareanu](https://reader036.vdocuments.mx/reader036/viewer/2022070400/56649f135503460f94c26a2b/html5/thumbnails/7.jpg)
In a perfect world…
only linear integer constraints
and only uniform distributions
![Page 8: Model Counting A Quest for Nails 2 Willem Visser Stellenbosch University Joint work with Matt Dwyer (UNL, USA) Jaco Geldenhuys (SU, RSA) Corina Pasareanu](https://reader036.vdocuments.mx/reader036/viewer/2022070400/56649f135503460f94c26a2b/html5/thumbnails/8.jpg)
void test(int x, int y) { if (y == x*10) S0; else S1; if (x > 3 && y > 10) S2; else S3;}
Symbolic Execution
[ Y=X*10 ] S0
[ X>3 & 10<Y=X*10] S2
[ true ] test (X,Y)
[ Y!=X*10 & !(X>3 & Y>10) ] S3
[ Y!=X*10 ] S1
[ Y=X*10 & !(X>3 & Y>10) ] S3
[ X>3 & 10<Y!=X*10] S2
Test(1,10) reaches S0,S3Test(0,1) reaches S1,S3Test(4,11) reaches S1,S2
![Page 9: Model Counting A Quest for Nails 2 Willem Visser Stellenbosch University Joint work with Matt Dwyer (UNL, USA) Jaco Geldenhuys (SU, RSA) Corina Pasareanu](https://reader036.vdocuments.mx/reader036/viewer/2022070400/56649f135503460f94c26a2b/html5/thumbnails/9.jpg)
![Page 10: Model Counting A Quest for Nails 2 Willem Visser Stellenbosch University Joint work with Matt Dwyer (UNL, USA) Jaco Geldenhuys (SU, RSA) Corina Pasareanu](https://reader036.vdocuments.mx/reader036/viewer/2022070400/56649f135503460f94c26a2b/html5/thumbnails/10.jpg)
void test(int x, int y) { if (y == x*10) S0; else S1; if (x > 3 && y > 10) S2; else S3;}
Paths
[ Y=X*10 ] S0
[ X>3 & 10<Y=X*10] S2
[ true ] test (X,Y)
[ Y!=X*10 & !(X>3 & Y>10) ] S3
[ Y!=X*10 ] S1
[ Y=X*10 & !(X>3 & Y>10) ] S3
[ X>3 & 10<Y!=X*10] S2
![Page 11: Model Counting A Quest for Nails 2 Willem Visser Stellenbosch University Joint work with Matt Dwyer (UNL, USA) Jaco Geldenhuys (SU, RSA) Corina Pasareanu](https://reader036.vdocuments.mx/reader036/viewer/2022070400/56649f135503460f94c26a2b/html5/thumbnails/11.jpg)
void test(int x, int y) { if (y == x*10) S0; else S1; if (x > 3 && y > 10) S2; else S3;}
Paths and Rivers
[ Y=X*10 ] [ Y!=X*10 ]
[ X>3 & 10<Y=X*10]
[ X>3 & 10<Y!=X*10]
[ Y!=X*10 & !(X>3 & Y>10) ]
[ true ]
[ Y=X*10 & !(X>3 & Y>10) ]
![Page 12: Model Counting A Quest for Nails 2 Willem Visser Stellenbosch University Joint work with Matt Dwyer (UNL, USA) Jaco Geldenhuys (SU, RSA) Corina Pasareanu](https://reader036.vdocuments.mx/reader036/viewer/2022070400/56649f135503460f94c26a2b/html5/thumbnails/12.jpg)
void test(int x, int y: 0..99) { if (y == x*10) S0; else S1; if (x > 3 && y > 10) S2; else S3;}
Almost Rivers
[ Y=X*10 ] [ Y!=X*10 ]
[ X>3 & 10<Y=X*10] [ X>3 & 10<Y!=X*10] [ Y!=X*10 & !(X>3 & Y>10) ]
[ true ]
[ Y=X*10 & !(X>3 & Y>10) ]
y=10xy=10x
x>3 & y>10x>3 & y>10x>3 & y>10x>3 & y>10
1 2 43
Which of 1, 2, 3 or 4 is the most likely?
![Page 13: Model Counting A Quest for Nails 2 Willem Visser Stellenbosch University Joint work with Matt Dwyer (UNL, USA) Jaco Geldenhuys (SU, RSA) Corina Pasareanu](https://reader036.vdocuments.mx/reader036/viewer/2022070400/56649f135503460f94c26a2b/html5/thumbnails/13.jpg)
void test(int x, int y: 0..99) { if (y == x*10) S0; else S1; if (x > 3 && y > 10) S2; else S3;}
Rivers
[ Y=X*10 ] [ Y!=X*10 ]
[ X>3 & 10<Y=X*10] [ X>3 & 10<Y!=X*10] [ Y!=X*10 & !(X>3 & Y>10) ]
[ true ]
[ Y=X*10 & !(X>3 & Y>10) ]
y=10xy=10x
x>3 & y>10x>3 & y>10x>3 & y>10x>3 & y>10
![Page 14: Model Counting A Quest for Nails 2 Willem Visser Stellenbosch University Joint work with Matt Dwyer (UNL, USA) Jaco Geldenhuys (SU, RSA) Corina Pasareanu](https://reader036.vdocuments.mx/reader036/viewer/2022070400/56649f135503460f94c26a2b/html5/thumbnails/14.jpg)
LattE Model Counter
http://www.math.ucdavis.edu/~latte/
Count solutions for conjunction
of Linear Inequalities
![Page 15: Model Counting A Quest for Nails 2 Willem Visser Stellenbosch University Joint work with Matt Dwyer (UNL, USA) Jaco Geldenhuys (SU, RSA) Corina Pasareanu](https://reader036.vdocuments.mx/reader036/viewer/2022070400/56649f135503460f94c26a2b/html5/thumbnails/15.jpg)
void test(int x, int y: 0..99) { if (y == x*10) S0; else S1; if (x > 3 && y > 10) S2; else S3;}
Rivers of Values
[ Y=X*10 ] [ Y!=X*10 ]
[ X>3 & 10<Y=X*10] [ X>3 & 10<Y!=X*10] [ Y!=X*10 & !(X>3 & Y>10) ]
[ true ]
[ Y=X*10 & !(X>3 & Y>10) ]
y=10xy=10x
x>3 & y>10x>3 & y>10x>3 & y>10x>3 & y>10
104
9990
8538
10
6 4 1452
![Page 16: Model Counting A Quest for Nails 2 Willem Visser Stellenbosch University Joint work with Matt Dwyer (UNL, USA) Jaco Geldenhuys (SU, RSA) Corina Pasareanu](https://reader036.vdocuments.mx/reader036/viewer/2022070400/56649f135503460f94c26a2b/html5/thumbnails/16.jpg)
[ Y=X*10 ]
[ Y!=X*10 ]
[ X>3 & 10<Y=X*10] [ X>3 & 10<Y!=X*10] [ Y!=X*10 & !(X>3 & Y>10) ]
[ true ]
[ Y=X*10 & !(X>3 & Y>10) ]
y=10xy=10x
x>3 & y>10x>3 & y>10x>3 & y>10x>3 & y>10
104
9990
8538
10
6 4 1452
Progra
m Underst
anding
![Page 17: Model Counting A Quest for Nails 2 Willem Visser Stellenbosch University Joint work with Matt Dwyer (UNL, USA) Jaco Geldenhuys (SU, RSA) Corina Pasareanu](https://reader036.vdocuments.mx/reader036/viewer/2022070400/56649f135503460f94c26a2b/html5/thumbnails/17.jpg)
How likely is a PC to be satisfied?
A Path Condition defines the constraints on the inputs
to execute a path
# solutions to the PC
Domain Size
Assuming uniform distribution of values
![Page 18: Model Counting A Quest for Nails 2 Willem Visser Stellenbosch University Joint work with Matt Dwyer (UNL, USA) Jaco Geldenhuys (SU, RSA) Corina Pasareanu](https://reader036.vdocuments.mx/reader036/viewer/2022070400/56649f135503460f94c26a2b/html5/thumbnails/18.jpg)
PC
c
P
= Prob (c & PC)
Prob (PC)
Pc
!c1-Pc
= Prob (c & PC)
P
Conditional and Path Probabilities
P’’ = (1-Pc) x P P’ = Pc x P
Pc = Prob (c | PC)
![Page 19: Model Counting A Quest for Nails 2 Willem Visser Stellenbosch University Joint work with Matt Dwyer (UNL, USA) Jaco Geldenhuys (SU, RSA) Corina Pasareanu](https://reader036.vdocuments.mx/reader036/viewer/2022070400/56649f135503460f94c26a2b/html5/thumbnails/19.jpg)
[ X>3 & 10<Y=X*10] [ X>3 & 10<Y!=X*10] [ Y!=X*10 & !(X>3 & Y>10) ][ Y=X*10 & !(X>3 & Y>10) ]
y=10xy=10x
x>3 & y>10x>3 & y>10x>3 & y>10x>3 & y>10
1
0.999
0.855
0.001
0.6 0.4 0.145
Probabilit
ies
0.0006 0.0004 0.8538 0.1452
![Page 20: Model Counting A Quest for Nails 2 Willem Visser Stellenbosch University Joint work with Matt Dwyer (UNL, USA) Jaco Geldenhuys (SU, RSA) Corina Pasareanu](https://reader036.vdocuments.mx/reader036/viewer/2022070400/56649f135503460f94c26a2b/html5/thumbnails/20.jpg)
[ X>3 & 10<Y=X*10] [ X>3 & 10<Y!=X*10] [ Y!=X*10 & !(X>3 & Y>10) ][ Y=X*10 & !(X>3 & Y>10) ]
y=10xy=10x
x>3 & y>10x>3 & y>10x>3 & y>10x>3 & y>10
1
0.999
0.855
0.001
0.6 0.4 0.145
Reliability
0.0006 0.0004 0.8538 0.1452
0.9996 Reliable
![Page 21: Model Counting A Quest for Nails 2 Willem Visser Stellenbosch University Joint work with Matt Dwyer (UNL, USA) Jaco Geldenhuys (SU, RSA) Corina Pasareanu](https://reader036.vdocuments.mx/reader036/viewer/2022070400/56649f135503460f94c26a2b/html5/thumbnails/21.jpg)
void test(int x,y: 0..99) { boolean error = false; if (x > 0) { if (y == hash(x)) error = true; else … if (x > 3 && y > 10) … else assert !error; }}
What is the reliability?
Reliability with Symbolic Execution
Uniform Distribution: 0.9908
int hash(x) { if (0<=x<=10) return x*10; else return 0;}
![Page 22: Model Counting A Quest for Nails 2 Willem Visser Stellenbosch University Joint work with Matt Dwyer (UNL, USA) Jaco Geldenhuys (SU, RSA) Corina Pasareanu](https://reader036.vdocuments.mx/reader036/viewer/2022070400/56649f135503460f94c26a2b/html5/thumbnails/22.jpg)
Usage Profiles
domain{x : 0,99;y : 0,99;
};
usageProfile{ x > y : 1/10; x <= y : 9/10;};
Constraints must be disjoint and cover the complete domain
Probabilities must add to 1
![Page 23: Model Counting A Quest for Nails 2 Willem Visser Stellenbosch University Joint work with Matt Dwyer (UNL, USA) Jaco Geldenhuys (SU, RSA) Corina Pasareanu](https://reader036.vdocuments.mx/reader036/viewer/2022070400/56649f135503460f94c26a2b/html5/thumbnails/23.jpg)
void test(int x,y) { boolean error = false; if (x > 0) { if (y == hash(x)) error = true; else … if (x > 3 && y > 10) … else assert !error; }}
Reliability with Symbolic Execution
Profile Reliability
Uniform 0.99080x > y : 0.1 0.99766y > x : 0.1 0.98407x > 10 & y > 10: 0.99 0.99995x > 10 & y > 10: 1 1.00000int hash(x) { if (0<=x<=10) return x*10; else return 0;}
![Page 24: Model Counting A Quest for Nails 2 Willem Visser Stellenbosch University Joint work with Matt Dwyer (UNL, USA) Jaco Geldenhuys (SU, RSA) Corina Pasareanu](https://reader036.vdocuments.mx/reader036/viewer/2022070400/56649f135503460f94c26a2b/html5/thumbnails/24.jpg)
PC
…c1 c2
cn
Prob(PC | UP) = i=1,n Prob(PC | ci) x pi
Prob(PC | ci) = Prob (PC & ci)
Prob (ci)
Calculate Probabilities AFTER
Symbolic Execution
c1 : p1
c2 : p2
…
cn : pn
UP
![Page 25: Model Counting A Quest for Nails 2 Willem Visser Stellenbosch University Joint work with Matt Dwyer (UNL, USA) Jaco Geldenhuys (SU, RSA) Corina Pasareanu](https://reader036.vdocuments.mx/reader036/viewer/2022070400/56649f135503460f94c26a2b/html5/thumbnails/25.jpg)
n Failure Paths m Success Paths
ProbS(P) = i=1..m Prob(PCm | UP)
NON Looping Programs
Reliability(P) = ProbS(P)
![Page 26: Model Counting A Quest for Nails 2 Willem Visser Stellenbosch University Joint work with Matt Dwyer (UNL, USA) Jaco Geldenhuys (SU, RSA) Corina Pasareanu](https://reader036.vdocuments.mx/reader036/viewer/2022070400/56649f135503460f94c26a2b/html5/thumbnails/26.jpg)
n Failure m Success ProbS(P) = i=1..m Prob(PCm
| UP)
ProbF(P) = i=1..n Prob(PCn | UP)
ProbG(P) = 1 - (ProbS(P) + ProbF(P))
Looping Programs => Bounded Analysis
Unknown
Reliability(P) >= ProbS(P)
Confidence = 1 – ProbG(P)
![Page 27: Model Counting A Quest for Nails 2 Willem Visser Stellenbosch University Joint work with Matt Dwyer (UNL, USA) Jaco Geldenhuys (SU, RSA) Corina Pasareanu](https://reader036.vdocuments.mx/reader036/viewer/2022070400/56649f135503460f94c26a2b/html5/thumbnails/27.jpg)
Time for a
new example
![Page 28: Model Counting A Quest for Nails 2 Willem Visser Stellenbosch University Joint work with Matt Dwyer (UNL, USA) Jaco Geldenhuys (SU, RSA) Corina Pasareanu](https://reader036.vdocuments.mx/reader036/viewer/2022070400/56649f135503460f94c26a2b/html5/thumbnails/28.jpg)
void unlikely(int x, int y, int z : 1..1000) { if (x <= 50) { S0 } else { if (x == 500 && y == 500 && z == 500) { assert false; } S1 }}
10-9 probability
![Page 29: Model Counting A Quest for Nails 2 Willem Visser Stellenbosch University Joint work with Matt Dwyer (UNL, USA) Jaco Geldenhuys (SU, RSA) Corina Pasareanu](https://reader036.vdocuments.mx/reader036/viewer/2022070400/56649f135503460f94c26a2b/html5/thumbnails/29.jpg)
Statistical Symbolic Execution
Monte Carlo Sampling of Symbolic Paths+
Confidence and Error Boundsbased on Bayesian Estimation
Inform
ed
Confidence = 1, i.e. exact incremental analysis
![Page 30: Model Counting A Quest for Nails 2 Willem Visser Stellenbosch University Joint work with Matt Dwyer (UNL, USA) Jaco Geldenhuys (SU, RSA) Corina Pasareanu](https://reader036.vdocuments.mx/reader036/viewer/2022070400/56649f135503460f94c26a2b/html5/thumbnails/30.jpg)
Monte Carlo Sampling of Symbolic Paths
PC
c
#PC
= Prob (c & PC)
Prob (PC)
Pc
!c1-Pc
= # (c & PC)
#PC
Pc = Prob (c | PC)
Step 1: Calculate Conditional Probability for a branch
![Page 31: Model Counting A Quest for Nails 2 Willem Visser Stellenbosch University Joint work with Matt Dwyer (UNL, USA) Jaco Geldenhuys (SU, RSA) Corina Pasareanu](https://reader036.vdocuments.mx/reader036/viewer/2022070400/56649f135503460f94c26a2b/html5/thumbnails/31.jpg)
Monte Carlo Sampling of Symbolic Paths
PC
c
#PC
Pc
!c1-Pc
rand = throwDice();If (rand <= Pc) pick c; //thenelse pick !c; //else
Step 2: Take random value and pick c or !c direction
![Page 32: Model Counting A Quest for Nails 2 Willem Visser Stellenbosch University Joint work with Matt Dwyer (UNL, USA) Jaco Geldenhuys (SU, RSA) Corina Pasareanu](https://reader036.vdocuments.mx/reader036/viewer/2022070400/56649f135503460f94c26a2b/html5/thumbnails/32.jpg)
void unlikely(int x, int y, int z : 1..1000) { if (x <= 50) { S0 } else { if (x == 500 && y == 500 && z == 500) { assert false; } S1 }}
109
50*106
[ X>50 ]
x<=50x<=50
950*106[ X<=50 ]
More likely to be picked
![Page 33: Model Counting A Quest for Nails 2 Willem Visser Stellenbosch University Joint work with Matt Dwyer (UNL, USA) Jaco Geldenhuys (SU, RSA) Corina Pasareanu](https://reader036.vdocuments.mx/reader036/viewer/2022070400/56649f135503460f94c26a2b/html5/thumbnails/33.jpg)
void unlikely(int x, int y, int z : 1..1000) { if (x <= 50) { S0 } else { if (x == 500 && y == 500 && z == 500) { assert false; } S1 }}
109
50*106
[ X>50 ]
950*106[ X<=50 ]
[ X=500 ]
106 949*106
[ X>50 & X!=500 ]
More likely to be picked
After 1 sampleCovered only S1
After 100 samplesWill likely also cover S0
[ X<=50 ]
y==500y==500
x==500x==500
x<=50x<=50
After 105 samplesWill likely hit x==500
but Eagles will have to reunitebefore hitting the violation
![Page 34: Model Counting A Quest for Nails 2 Willem Visser Stellenbosch University Joint work with Matt Dwyer (UNL, USA) Jaco Geldenhuys (SU, RSA) Corina Pasareanu](https://reader036.vdocuments.mx/reader036/viewer/2022070400/56649f135503460f94c26a2b/html5/thumbnails/34.jpg)
void unlikely(int x, int y, int z : 1..1000) { if (x <= 50) { S0 } else { if (x == 500 && y == 500 && z == 500) { assert false; } S1 }}
109
50*106
x<=50x<=50
[ X>50 ]
950*106[ X<=50 ]
[ X=500 ]
106 949*106
[ X>50 & X!=500 ]
Informed Sampling[Draining the river]
After every pathsampled remove the path cleverlyx==500x==500
![Page 35: Model Counting A Quest for Nails 2 Willem Visser Stellenbosch University Joint work with Matt Dwyer (UNL, USA) Jaco Geldenhuys (SU, RSA) Corina Pasareanu](https://reader036.vdocuments.mx/reader036/viewer/2022070400/56649f135503460f94c26a2b/html5/thumbnails/35.jpg)
void unlikely(int x, int y, int z : 1..1000) { if (x <= 50) { S0 } else { if (x == 500 && y == 500 && z == 500) { assert false; } S1 }}
51*106
50*106
x<=50x<=50
[ X>50 ]
106[ X<=50 ]
[ X=500 ]
1060
[ X>50 & X!=500 ]
Informed Sample 2
x==500x==500
![Page 36: Model Counting A Quest for Nails 2 Willem Visser Stellenbosch University Joint work with Matt Dwyer (UNL, USA) Jaco Geldenhuys (SU, RSA) Corina Pasareanu](https://reader036.vdocuments.mx/reader036/viewer/2022070400/56649f135503460f94c26a2b/html5/thumbnails/36.jpg)
void unlikely(int x, int y, int z : 1..1000) { if (x <= 50) { S0 } else { if (x == 500 && y == 500 && z == 500) { assert false; } S1 }}
106
0
x<=50x<=50
[ X>50 ]
106[ X<=50 ]
x==500x==500
[ X=500 ]
1060
[ X>50 & X!=500 ]
Informed Sample 3
[ X<=50 ]
y==500y==500
![Page 37: Model Counting A Quest for Nails 2 Willem Visser Stellenbosch University Joint work with Matt Dwyer (UNL, USA) Jaco Geldenhuys (SU, RSA) Corina Pasareanu](https://reader036.vdocuments.mx/reader036/viewer/2022070400/56649f135503460f94c26a2b/html5/thumbnails/37.jpg)
void unlikely(int x, int y, int z : 1..1000) { if (x <= 50) { S0 } else { if (x == 500 && y == 500 && z == 500) { assert false; } S1 }}
106
[ X>50 ]
Informed Sample 4
x<=50x<=50
106
x==500x==500
y==500y==500
106
[ X==500 & Y!=500 ]
[ X==500 ]
[ X,Y==500 ]
999*1031*103
![Page 38: Model Counting A Quest for Nails 2 Willem Visser Stellenbosch University Joint work with Matt Dwyer (UNL, USA) Jaco Geldenhuys (SU, RSA) Corina Pasareanu](https://reader036.vdocuments.mx/reader036/viewer/2022070400/56649f135503460f94c26a2b/html5/thumbnails/38.jpg)
void unlikely(int x, int y, int z : 1..1000) { if (x <= 50) { S0 } else { if (x == 500 && y == 500 && z == 500) { assert false; } S1 }}
103
[ X>50 ]
Informed Sample 5
x<=50x<=50
103
x==500x==500
103
[ X==500 & Y!=500 ]
[ X==500 ]
[ X,Y==500 ]
0y==500y==500
z==500z==500
103
[ X,Y==500 & Z!=500 ]
9991
![Page 39: Model Counting A Quest for Nails 2 Willem Visser Stellenbosch University Joint work with Matt Dwyer (UNL, USA) Jaco Geldenhuys (SU, RSA) Corina Pasareanu](https://reader036.vdocuments.mx/reader036/viewer/2022070400/56649f135503460f94c26a2b/html5/thumbnails/39.jpg)
void unlikely(int x, int y, int z : 1..1000) { if (x <= 50) { S0 } else { if (x == 500 && y == 500 && z == 500) { assert false; } S1 }}
1
[ X>50 ]
x<=50x<=50
1
x==500x==500
1 [ X==500 ]
[ X,Y==500 ]
y==500y==500
1
[ X,Y==500 & Z!=500 ]
0
z==500z==500
1[ X,Y,Z==500 ]
After 6 Informed Sampleswe hit the 10-9 event
Confindence = 1, since we explored the complete space
![Page 40: Model Counting A Quest for Nails 2 Willem Visser Stellenbosch University Joint work with Matt Dwyer (UNL, USA) Jaco Geldenhuys (SU, RSA) Corina Pasareanu](https://reader036.vdocuments.mx/reader036/viewer/2022070400/56649f135503460f94c26a2b/html5/thumbnails/40.jpg)
Cool Feature of Informed Sampling
First samples the most likely paths
Then the slightly less likely paths
Until you get to the very unlikely paths
Then the even less likely paths
![Page 41: Model Counting A Quest for Nails 2 Willem Visser Stellenbosch University Joint work with Matt Dwyer (UNL, USA) Jaco Geldenhuys (SU, RSA) Corina Pasareanu](https://reader036.vdocuments.mx/reader036/viewer/2022070400/56649f135503460f94c26a2b/html5/thumbnails/41.jpg)
Multithreaded Informed Sampling => Symbolic Execution
[ X>3 & 10<Y=X*10] [ X>3 & 10<Y!=X*10] [ Y!=X*10 & !(X>3 & Y>10) ][ Y=X*10 & !(X>3 & Y>10) ]
y=10xy=10x
y!=10x & x>3 & y>10
y!=10x & x>3 & y>10
y=10x & x>3 & y>10
y=10x & x>3 & y>10
104
9990
8538
10
6 4 1452
Only shared structurePC => count
Only shared structurePC => count
Run n threads, eachdoing informed sampling
to reach a leave
Run n threads, eachdoing informed sampling
to reach a leave
When you update, first check if any
value will become <= 0, if so, terminate and
pick a new path from the top
When you update, first check if any
value will become <= 0, if so, terminate and
pick a new path from the top
![Page 42: Model Counting A Quest for Nails 2 Willem Visser Stellenbosch University Joint work with Matt Dwyer (UNL, USA) Jaco Geldenhuys (SU, RSA) Corina Pasareanu](https://reader036.vdocuments.mx/reader036/viewer/2022070400/56649f135503460f94c26a2b/html5/thumbnails/42.jpg)
Multithreaded Informed Sampling => Symbolic Execution
[ X>3 & 10<Y=X*10] [ X>3 & 10<Y!=X*10] [ Y!=X*10 & !(X>3 & Y>10) ][ Y=X*10 & !(X>3 & Y>10) ]
y=10xy=10x
y!=10x & x>3 & y>10
y!=10x & x>3 & y>10
y=10x & x>3 & y>10
y=10x & x>3 & y>10
104
9990
8538
10
6 4 1452T1T1 T2T2
![Page 43: Model Counting A Quest for Nails 2 Willem Visser Stellenbosch University Joint work with Matt Dwyer (UNL, USA) Jaco Geldenhuys (SU, RSA) Corina Pasareanu](https://reader036.vdocuments.mx/reader036/viewer/2022070400/56649f135503460f94c26a2b/html5/thumbnails/43.jpg)
Multithreaded Informed Sampling => Symbolic Execution
[ X>3 & 10<Y=X*10] [ X>3 & 10<Y!=X*10] [ Y!=X*10 & !(X>3 & Y>10) ][ Y=X*10 & !(X>3 & Y>10) ]
y=10xy=10x
y!=10x & x>3 & y>10
y!=10x & x>3 & y>10
y=10x & x>3 & y>10
y=10x & x>3 & y>10
104
1452
0
10
6 4 1452
T1T1
T2T2
T2T2
T2T2
T2T2
![Page 44: Model Counting A Quest for Nails 2 Willem Visser Stellenbosch University Joint work with Matt Dwyer (UNL, USA) Jaco Geldenhuys (SU, RSA) Corina Pasareanu](https://reader036.vdocuments.mx/reader036/viewer/2022070400/56649f135503460f94c26a2b/html5/thumbnails/44.jpg)
Multithreaded Informed Sampling => Symbolic Execution
[ X>3 & 10<Y=X*10] [ X>3 & 10<Y!=X*10] [ Y!=X*10 & !(X>3 & Y>10) ][ Y=X*10 & !(X>3 & Y>10) ]
y=10xy=10x
y!=10x & x>3 & y>10
y!=10x & x>3 & y>10
y=10x & x>3 & y>10
y=10x & x>3 & y>10
104
0
0
10
6 4 0
T1T1 T2T2
![Page 45: Model Counting A Quest for Nails 2 Willem Visser Stellenbosch University Joint work with Matt Dwyer (UNL, USA) Jaco Geldenhuys (SU, RSA) Corina Pasareanu](https://reader036.vdocuments.mx/reader036/viewer/2022070400/56649f135503460f94c26a2b/html5/thumbnails/45.jpg)
Multithreaded Informed Sampling => Symbolic Execution
[ X>3 & 10<Y=X*10] [ X>3 & 10<Y!=X*10] [ Y!=X*10 & !(X>3 & Y>10) ][ Y=X*10 & !(X>3 & Y>10) ]
y=10xy=10x
y!=10x & x>3 & y>10
y!=10x & x>3 & y>10
y=10x & x>3 & y>10
y=10x & x>3 & y>10
104
0
0
10
6 4 0
T1T1T2T2
![Page 46: Model Counting A Quest for Nails 2 Willem Visser Stellenbosch University Joint work with Matt Dwyer (UNL, USA) Jaco Geldenhuys (SU, RSA) Corina Pasareanu](https://reader036.vdocuments.mx/reader036/viewer/2022070400/56649f135503460f94c26a2b/html5/thumbnails/46.jpg)
Multithreaded Informed Sampling => Symbolic Execution
[ X>3 & 10<Y=X*10] [ X>3 & 10<Y!=X*10] [ Y!=X*10 & !(X>3 & Y>10) ][ Y=X*10 & !(X>3 & Y>10) ]
y=10xy=10x
y!=10x & x>3 & y>10
y!=10x & x>3 & y>10
y=10x & x>3 & y>10
y=10x & x>3 & y>10
104
0
0
0
0 0 0
T1T1 T2T2
![Page 47: Model Counting A Quest for Nails 2 Willem Visser Stellenbosch University Joint work with Matt Dwyer (UNL, USA) Jaco Geldenhuys (SU, RSA) Corina Pasareanu](https://reader036.vdocuments.mx/reader036/viewer/2022070400/56649f135503460f94c26a2b/html5/thumbnails/47.jpg)
Informed Samplingas a search heuristic
for Concolic execution
when negating constraints pick the path with the most values flowing down it next
![Page 48: Model Counting A Quest for Nails 2 Willem Visser Stellenbosch University Joint work with Matt Dwyer (UNL, USA) Jaco Geldenhuys (SU, RSA) Corina Pasareanu](https://reader036.vdocuments.mx/reader036/viewer/2022070400/56649f135503460f94c26a2b/html5/thumbnails/48.jpg)
Green: Reduce, Reuse and Recycle Constraints in Program Analysis
Willem Visser
Stellenbosch University
Joint work with Jaco Geldenhuys and Matt Dwyer
![Page 49: Model Counting A Quest for Nails 2 Willem Visser Stellenbosch University Joint work with Matt Dwyer (UNL, USA) Jaco Geldenhuys (SU, RSA) Corina Pasareanu](https://reader036.vdocuments.mx/reader036/viewer/2022070400/56649f135503460f94c26a2b/html5/thumbnails/49.jpg)
What is Symbolic Execution
• Executing a program with symbolic inputs• Collect all constraints to execute a path
through code, called Path Condition– Stop when Path Condition becomes infeasible
• Many uses– Checking for errors, without running the code– Solve feasible constraints to get inputs for test
cases
![Page 50: Model Counting A Quest for Nails 2 Willem Visser Stellenbosch University Joint work with Matt Dwyer (UNL, USA) Jaco Geldenhuys (SU, RSA) Corina Pasareanu](https://reader036.vdocuments.mx/reader036/viewer/2022070400/56649f135503460f94c26a2b/html5/thumbnails/50.jpg)
Decision Procedures
• Huge advances in the last 15 years• Many great tools
– Z3, Yices, CVC3, STP, …
• Satisfiability is NP-complete• Worst case complexity is exponential in the
size of the formula• Our goal is to make these tools even better,
without changing a line of code inside them!
![Page 51: Model Counting A Quest for Nails 2 Willem Visser Stellenbosch University Joint work with Matt Dwyer (UNL, USA) Jaco Geldenhuys (SU, RSA) Corina Pasareanu](https://reader036.vdocuments.mx/reader036/viewer/2022070400/56649f135503460f94c26a2b/html5/thumbnails/51.jpg)
int m(int x,y) { if (x < 0) x = -x; if (y < 0) y = -y; if (x < 10) { return 1; } else if (9 < y) { return -1; } else { return 0; }}
[ X < 0 ]
[ Y < 0 ] [ Y < 0 ]
[ X < 10 ] [ X < 10 ]
X < 0
Y < 0 !(Y < 0)
[ 9 < Y ]
-X < 10 !(-X < 10)
9 < -Y !(9 < -Y)
-X < 10
[ 9 < Y ]
9 < Y !(9 < Y)
!(-X < 10)
!(X < 0)
![Page 52: Model Counting A Quest for Nails 2 Willem Visser Stellenbosch University Joint work with Matt Dwyer (UNL, USA) Jaco Geldenhuys (SU, RSA) Corina Pasareanu](https://reader036.vdocuments.mx/reader036/viewer/2022070400/56649f135503460f94c26a2b/html5/thumbnails/52.jpg)
[ X < 0 ]
[ Y < 0 ] [ Y < 0 ]
[ X < 10 ] [ X < 10 ]
X < 0
Y < 0
[ 9 < Y ]
-X < 10
9 < -Y !(9 < -Y)
-X < 10
[ 9 < Y ]
9 < Y
!(X < 0)
!(-X < 10)
[ X < 10 ] [ X < 10 ]
[ 9 < Y ]
-X < 10
9 < -Y !(9 < -Y)
X < 10
[ 9 < Y ]
9 < Y !(9 < Y)
!(X < 10)
Y < 0 !(Y < 0)!(Y < 0)
X < 0 /\ Y < 0
X < 0 /\ Y < 0 /\ !(-X < 10)
X < 0 /\ Y < 0 /\ !(-X < 10) /\ 9 < -Y
X < 10 !(X < 10)
Don’t need the complete constraint to decide feasibility
9 < -Y
X < 0
![Page 53: Model Counting A Quest for Nails 2 Willem Visser Stellenbosch University Joint work with Matt Dwyer (UNL, USA) Jaco Geldenhuys (SU, RSA) Corina Pasareanu](https://reader036.vdocuments.mx/reader036/viewer/2022070400/56649f135503460f94c26a2b/html5/thumbnails/53.jpg)
[ X < 0 ]
[ Y < 0 ] [ Y < 0 ]
[ X < 10 ] [ X < 10 ]
X < 0
Y < 0
[ 9 < Y ]
-X < 10
9 < -Y !(9 < -Y)
-X<10
[ 9 < Y ]
9 < Y
!(X < 0)
!(-X < 10)
[ X < 10 ] [ X < 10 ]
[ 9 < Y ]
!(-X<10)
9 < -Y !(9 < -Y)
X < 10
[ 9 < Y ]
9 < Y !(9 < Y)
!(X < 10)
Y < 0 !(Y < 0)!(Y < 0)
Y < 0
X < 0 /\ !(-X < 10)
Y < 0 /\ 9 < -Y
X < 10 !(X < 10)
!(Y < 0)
!(Y < 0)
Y < 0
X < 0 !(X < 0)
X < 0 /\ !(-X < 10) !(X < 0) /\ !(X < 10) !(X < 0) /\ !(X < 10)
Slicing constraints leads to the same constraints in different places
9 < -Y
These two constraints are the same!
![Page 54: Model Counting A Quest for Nails 2 Willem Visser Stellenbosch University Joint work with Matt Dwyer (UNL, USA) Jaco Geldenhuys (SU, RSA) Corina Pasareanu](https://reader036.vdocuments.mx/reader036/viewer/2022070400/56649f135503460f94c26a2b/html5/thumbnails/54.jpg)
Canonization of ConstraintsX < 0 /\ !(-X < 10)
X < 0 /\ -X >= 10
X < 0 /\ X <= -10
X + 1 <= 0 /\ X + 10 <= 0
Y < 0 /\ 9 < -Y
Y < 0 /\ Y < - 9
Y < 0 /\ Y + 9 < 0
Y + 1 <= 0 /\ Y + 10 <= 0
ax + by + cz +…+ k {<=,=,!=} 0
Canonical Form
• Scale by -1 to transform > and >= to < and <=• Add 1 to transform < to <=
V0 + 1 <= 0 /\ V0 + 10 <= 0
![Page 55: Model Counting A Quest for Nails 2 Willem Visser Stellenbosch University Joint work with Matt Dwyer (UNL, USA) Jaco Geldenhuys (SU, RSA) Corina Pasareanu](https://reader036.vdocuments.mx/reader036/viewer/2022070400/56649f135503460f94c26a2b/html5/thumbnails/55.jpg)
[ X < 0 ]
[ Y < 0 ] [ Y < 0 ]
[ X < 10 ] [ X < 10 ]
[ 9 < Y ] [ 9 < Y ]
[ X < 10 ] [ X < 10 ]
[ 9 < Y ] [ 9 < Y ]
V0+1 <= 0
V0+1 <= 0 /\ V0+10 <= 0
-V0 <= 0 -V0 <= 0
V0+1 <= 0
V0+1 <= 0 -V0 <= 0
V0+1<=0 /\ V0+10<=0 -V0<=0/\-V0+10<=0 -V0<=0/\-V0+10<=0
V0+1<=0 /\ V0+10<=0
V0+1<=0 /\ -V0-9<=0
V0+1<=0 /\ -V0 - 9 <=0
V0+1<=0 /\ -V0 - 9 <=0
-V0<=0 /\ V0-9 <=0
-V0<=0 /\ -V0+10<=0
-V0<=0 /\ V0-9<=0
-V0<=0 /\ V0-9 <=0
V0+1<=0 /\ V0+10<=0
V0+1<=0 /\ -V0-9<=0
-V0<=0 /\ -V0+10<=0
-V0<=0 /\ V0-9<=0
![Page 56: Model Counting A Quest for Nails 2 Willem Visser Stellenbosch University Joint work with Matt Dwyer (UNL, USA) Jaco Geldenhuys (SU, RSA) Corina Pasareanu](https://reader036.vdocuments.mx/reader036/viewer/2022070400/56649f135503460f94c26a2b/html5/thumbnails/56.jpg)
What if we store the results?
and reuse them to avoid recalculation
![Page 57: Model Counting A Quest for Nails 2 Willem Visser Stellenbosch University Joint work with Matt Dwyer (UNL, USA) Jaco Geldenhuys (SU, RSA) Corina Pasareanu](https://reader036.vdocuments.mx/reader036/viewer/2022070400/56649f135503460f94c26a2b/html5/thumbnails/57.jpg)
[ Y < 0 ]
[ X < 10 ] [ X < 10 ]
[ 9 < Y ] [ 9 < Y ]
-V0 <= 0
V0+1 <= 0
-V0 <= 0
-V0<=0/\-V0+10<=0 -V0<=0/\-V0+10<=0
-V0<=0 /\ V0-9 <=0
-V0<=0 /\ V0-9 <=0
V0+1<=0 /\ V0+10<=0
V0+1<=0 /\ -V0-9<=0
-V0<=0 /\ -V0+10<=0
-V0<=0 /\ V0-9<=0
4
1 4
6 6
5 5
3 2 5 6
-V0<=0 /\ V0-9<=0
6
-V0<=0 /\ -V0+10<=0
5
-V0 <= 0
4
V0+1<=0 /\ -V0-9<=0
2
V0+1<=0 /\ V0+10<=0
3
V0+1 <= 01
V0+1 <= 0
1
V0+1<=0 /\ -V0 - 9 <=0
2V0+1<=0 /\ -V0 - 9 <=0
2
V0+1<=0 /\ V0+10<=0
3
V0+1 <= 0 /\ V0+10 <= 0
3
[ X < 0 ]
![Page 58: Model Counting A Quest for Nails 2 Willem Visser Stellenbosch University Joint work with Matt Dwyer (UNL, USA) Jaco Geldenhuys (SU, RSA) Corina Pasareanu](https://reader036.vdocuments.mx/reader036/viewer/2022070400/56649f135503460f94c26a2b/html5/thumbnails/58.jpg)
Let’s change the program!
int m(int x,y) { if (x < 0) x = -x; if (y < 0) y = -y; if (x < 10) { return 1; } else if (9 < y) { return -1; } else { return 0; }}
If (10 < y)
Only the last 8 constraints are changed in the symbolic execution tree and 4 of themare reused.
Reusing the stored resultsfrom the first analysis eliminates14 decision procedure calls!
![Page 59: Model Counting A Quest for Nails 2 Willem Visser Stellenbosch University Joint work with Matt Dwyer (UNL, USA) Jaco Geldenhuys (SU, RSA) Corina Pasareanu](https://reader036.vdocuments.mx/reader036/viewer/2022070400/56649f135503460f94c26a2b/html5/thumbnails/59.jpg)
Green
• Reduce– Slicing + Canonization
• Reuse– Storing results
• Recycle– Across Analyses of Programs and even Tools
![Page 60: Model Counting A Quest for Nails 2 Willem Visser Stellenbosch University Joint work with Matt Dwyer (UNL, USA) Jaco Geldenhuys (SU, RSA) Corina Pasareanu](https://reader036.vdocuments.mx/reader036/viewer/2022070400/56649f135503460f94c26a2b/html5/thumbnails/60.jpg)
PC = knownPC /\ newPC
Known to be SAT
Slicing Algorithm
1.Build a constraint graph for knownPC /\ newPC1. Vertices are symbolic variables2. Edges between them if they are in the same constraint
2.Find all variables R reachable from variables in newPC3.Return the conjunction of all the constraints containing variables R
Classic Symbolic ExecutionnewPC is the last decision on the pathknownPC is all the rest
Dynamic Symbolic ExecutionnewPC is the negated conjunctknownPC are all the other conjuncts
![Page 61: Model Counting A Quest for Nails 2 Willem Visser Stellenbosch University Joint work with Matt Dwyer (UNL, USA) Jaco Geldenhuys (SU, RSA) Corina Pasareanu](https://reader036.vdocuments.mx/reader036/viewer/2022070400/56649f135503460f94c26a2b/html5/thumbnails/61.jpg)
Factorizing Slicer
PC = C1 & C2 & … & Cn
Returns independent sub-constraints
PC = (C1 & C2) & (C3 & C4 & C5) & (… & Cn)
![Page 62: Model Counting A Quest for Nails 2 Willem Visser Stellenbosch University Joint work with Matt Dwyer (UNL, USA) Jaco Geldenhuys (SU, RSA) Corina Pasareanu](https://reader036.vdocuments.mx/reader036/viewer/2022070400/56649f135503460f94c26a2b/html5/thumbnails/62.jpg)
Pre-Heuristic lexicographic reorderingX > Y vs Y < X => X > Y
Three Parts to Canonization
Normal Formax + by + cz +…+ k {<=,=,!=} 0
Post-Heuristic 1. lexicographic order of
constraints2. Renaming based on order
in constraints
![Page 63: Model Counting A Quest for Nails 2 Willem Visser Stellenbosch University Joint work with Matt Dwyer (UNL, USA) Jaco Geldenhuys (SU, RSA) Corina Pasareanu](https://reader036.vdocuments.mx/reader036/viewer/2022070400/56649f135503460f94c26a2b/html5/thumbnails/63.jpg)
NoSQL In-memory key-value store
First hack took about 10 mins:1.Download Redis, make, start2.Find Java wrapper…Jedis3.Add 5 lines of code4.Viola!
Simply get(“PC”) and if not found put(“PC”,”T | F”)
![Page 64: Model Counting A Quest for Nails 2 Willem Visser Stellenbosch University Joint work with Matt Dwyer (UNL, USA) Jaco Geldenhuys (SU, RSA) Corina Pasareanu](https://reader036.vdocuments.mx/reader036/viewer/2022070400/56649f135503460f94c26a2b/html5/thumbnails/64.jpg)
Storage is layered
Localhost Colleague
Offshore Store
What you don’t find locally, look for in other storesResults are pushed backNew local results are pushed out
![Page 65: Model Counting A Quest for Nails 2 Willem Visser Stellenbosch University Joint work with Matt Dwyer (UNL, USA) Jaco Geldenhuys (SU, RSA) Corina Pasareanu](https://reader036.vdocuments.mx/reader036/viewer/2022070400/56649f135503460f94c26a2b/html5/thumbnails/65.jpg)
Current State
• Green– Services– Slicing, Canonizer, … [Filters]– (Redis) Store– Z3, CVC3, etc. [Solvers]– LattE [Model Counters]
![Page 66: Model Counting A Quest for Nails 2 Willem Visser Stellenbosch University Joint work with Matt Dwyer (UNL, USA) Jaco Geldenhuys (SU, RSA) Corina Pasareanu](https://reader036.vdocuments.mx/reader036/viewer/2022070400/56649f135503460f94c26a2b/html5/thumbnails/66.jpg)
ResultsWhy Slice and Canonize?
-store +store
-canon +canon -canon +canon
-slice 95506 94739 96448 50467
+slice 27129 27369 20410 5603
Binomial Heap with all add/remove sequences of length 5time in milliseconds
![Page 67: Model Counting A Quest for Nails 2 Willem Visser Stellenbosch University Joint work with Matt Dwyer (UNL, USA) Jaco Geldenhuys (SU, RSA) Corina Pasareanu](https://reader036.vdocuments.mx/reader036/viewer/2022070400/56649f135503460f94c26a2b/html5/thumbnails/67.jpg)
Reuse between programs
BinomialHeap
TreeMap BinaryTree
155
04
1
38 154 133
80.6% reused 54.5% reused
Only 3.1% reused
![Page 68: Model Counting A Quest for Nails 2 Willem Visser Stellenbosch University Joint work with Matt Dwyer (UNL, USA) Jaco Geldenhuys (SU, RSA) Corina Pasareanu](https://reader036.vdocuments.mx/reader036/viewer/2022070400/56649f135503460f94c26a2b/html5/thumbnails/68.jpg)
Future Work• Extending Model Counting to other types
– Reference Types, Strings, Floats, etc.
• Green– Are the number of actually occurring constraints
in code “finite”?– How far can one push the Big Data idea?– Main goal now is to get as many people as
possible to use Green
• Ultimate Goal: Real-time developer feedback
![Page 69: Model Counting A Quest for Nails 2 Willem Visser Stellenbosch University Joint work with Matt Dwyer (UNL, USA) Jaco Geldenhuys (SU, RSA) Corina Pasareanu](https://reader036.vdocuments.mx/reader036/viewer/2022070400/56649f135503460f94c26a2b/html5/thumbnails/69.jpg)
The Green Framework
http://green-solver.googlecode.com
Already integrated into Symbolic PathFinder