constraint based synthesis · synthesis: modern view = ∧ ∧ ∧ space of programs safety...
TRANSCRIPT
![Page 1: Constraint Based Synthesis · Synthesis: modern view = ∧ ∧ ∧ Space of programs Safety Properties Test Harnesses Input/Output Examples](https://reader034.vdocuments.mx/reader034/viewer/2022051604/6005ae4897d1fc360b54fce7/html5/thumbnails/1.jpg)
Constraint Based Synthesis
Armando Solar-Lezama
![Page 2: Constraint Based Synthesis · Synthesis: modern view = ∧ ∧ ∧ Space of programs Safety Properties Test Harnesses Input/Output Examples](https://reader034.vdocuments.mx/reader034/viewer/2022051604/6005ae4897d1fc360b54fce7/html5/thumbnails/2.jpg)
Synthesis: 1980s view
Complete Formal Specification
![Page 3: Constraint Based Synthesis · Synthesis: modern view = ∧ ∧ ∧ Space of programs Safety Properties Test Harnesses Input/Output Examples](https://reader034.vdocuments.mx/reader034/viewer/2022051604/6005ae4897d1fc360b54fce7/html5/thumbnails/3.jpg)
Synthesis: modern view
= ∧∧∧Space of programs
Safety Properties
Test Harnesses
Input/Output Examples
![Page 4: Constraint Based Synthesis · Synthesis: modern view = ∧ ∧ ∧ Space of programs Safety Properties Test Harnesses Input/Output Examples](https://reader034.vdocuments.mx/reader034/viewer/2022051604/6005ae4897d1fc360b54fce7/html5/thumbnails/4.jpg)
SketchC-like languagewith holes and
assertions
![Page 5: Constraint Based Synthesis · Synthesis: modern view = ∧ ∧ ∧ Space of programs Safety Properties Test Harnesses Input/Output Examples](https://reader034.vdocuments.mx/reader034/viewer/2022051604/6005ae4897d1fc360b54fce7/html5/thumbnails/5.jpg)
/* Average of x and y without overflow */int avg(int x, int y){int t = expr({x/2, y/2, x%2, y%2, 2 }, {PLUS, DIV});assert t == (x+y)/2;return t;
}
/* Operands */ /*Operators*/
![Page 6: Constraint Based Synthesis · Synthesis: modern view = ∧ ∧ ∧ Space of programs Safety Properties Test Harnesses Input/Output Examples](https://reader034.vdocuments.mx/reader034/viewer/2022051604/6005ae4897d1fc360b54fce7/html5/thumbnails/6.jpg)
/* Average of x and y without overflow */int avg(int x, int y){int t = y / 2 + x / 2 + ((x % 2 + y % 2) / 2);assert t == (x+y)/2;return t;
}
![Page 7: Constraint Based Synthesis · Synthesis: modern view = ∧ ∧ ∧ Space of programs Safety Properties Test Harnesses Input/Output Examples](https://reader034.vdocuments.mx/reader034/viewer/2022051604/6005ae4897d1fc360b54fce7/html5/thumbnails/7.jpg)
SketchC-like languagewith holes and
assertions
High-LevelLanguage Compiler
UnrollInlineEnumerate
AutomatedTutoring
![Page 8: Constraint Based Synthesis · Synthesis: modern view = ∧ ∧ ∧ Space of programs Safety Properties Test Harnesses Input/Output Examples](https://reader034.vdocuments.mx/reader034/viewer/2022051604/6005ae4897d1fc360b54fce7/html5/thumbnails/8.jpg)
Program+
Pre/Post cond+
Invariants
VCGenerator
SketchC-like languagewith holes and
assertions
UnrollInlineEnumerate
AutomatedTutoring
ProvablyCorrectSynthesis
ProgramOptimization
![Page 9: Constraint Based Synthesis · Synthesis: modern view = ∧ ∧ ∧ Space of programs Safety Properties Test Harnesses Input/Output Examples](https://reader034.vdocuments.mx/reader034/viewer/2022051604/6005ae4897d1fc360b54fce7/html5/thumbnails/9.jpg)
AutomatedTutoring
Program+
Properties
ProvablyCorrectSynthesis
ProgramOptimization
SketchC-like languagewith holes and
assertions
UnrollInlineEnumerate
AbstractDomainCompiler
VisualProgramming
![Page 10: Constraint Based Synthesis · Synthesis: modern view = ∧ ∧ ∧ Space of programs Safety Properties Test Harnesses Input/Output Examples](https://reader034.vdocuments.mx/reader034/viewer/2022051604/6005ae4897d1fc360b54fce7/html5/thumbnails/10.jpg)
AutomatedTutoring
Program+
Properties
ProvablyCorrectSynthesis
ProgramOptimization
SketchC-like languagewith holes and
assertions
UnrollInlineEnumerate
AbstractDomainCompiler
VisualProgramming
![Page 11: Constraint Based Synthesis · Synthesis: modern view = ∧ ∧ ∧ Space of programs Safety Properties Test Harnesses Input/Output Examples](https://reader034.vdocuments.mx/reader034/viewer/2022051604/6005ae4897d1fc360b54fce7/html5/thumbnails/11.jpg)
Automated TutoringWith Rishabh Singh and Sumit Gulwani
![Page 12: Constraint Based Synthesis · Synthesis: modern view = ∧ ∧ ∧ Space of programs Safety Properties Test Harnesses Input/Output Examples](https://reader034.vdocuments.mx/reader034/viewer/2022051604/6005ae4897d1fc360b54fce7/html5/thumbnails/12.jpg)
12
Pinpoint student errors
def computeDeriv(poly):deriv = []zero = 0if (len(poly) == 1):
return derivfor e in range(0, len(poly)):
if (poly[e] == 0):zero += 1
else:deriv.append(poly[e]*e)
return deriv
replace derive by [0]
Should be a 1
Get rid of this
def computeDeriv(poly):length = int(len(poly)-1)i = lengthderiv = range(1,length)
if len(poly) == 1:deriv = [0.0]
else:while i >= 0:
new = round((poly[i] * i),2)i -= 1deriv[i] = new
return deriv
Should be a 0
Should be >
This is a synthesis problem
![Page 13: Constraint Based Synthesis · Synthesis: modern view = ∧ ∧ ∧ Space of programs Safety Properties Test Harnesses Input/Output Examples](https://reader034.vdocuments.mx/reader034/viewer/2022051604/6005ae4897d1fc360b54fce7/html5/thumbnails/13.jpg)
SketchC-like languagewith holes and
assertions
High-LevelLanguage Compiler
UnrollInlineEnumerate
Python
![Page 14: Constraint Based Synthesis · Synthesis: modern view = ∧ ∧ ∧ Space of programs Safety Properties Test Harnesses Input/Output Examples](https://reader034.vdocuments.mx/reader034/viewer/2022051604/6005ae4897d1fc360b54fce7/html5/thumbnails/14.jpg)
SketchC-like languagewith holes and
assertions
CompilerUnrollInlineEnumerate
Python
Error Model
![Page 15: Constraint Based Synthesis · Synthesis: modern view = ∧ ∧ ∧ Space of programs Safety Properties Test Harnesses Input/Output Examples](https://reader034.vdocuments.mx/reader034/viewer/2022051604/6005ae4897d1fc360b54fce7/html5/thumbnails/15.jpg)
SketchPython CompilerUnrollInlineEnumerate
struct MultiType{int type;int ival;bit bval;MTString str; MTList lst;MTDict dict; MTTuple tup;
}
ival bval
lst…
Type
int bool
list
Dynamic Types
![Page 16: Constraint Based Synthesis · Synthesis: modern view = ∧ ∧ ∧ Space of programs Safety Properties Test Harnesses Input/Output Examples](https://reader034.vdocuments.mx/reader034/viewer/2022051604/6005ae4897d1fc360b54fce7/html5/thumbnails/16.jpg)
SketchCompilerUnrollInlineEnumerate
[1,2]
1 -
- -INT
int
list
bool
2 -
- -INT
int
list
bool
- -
len=2, lVals = {*, * } -
LISTint
list
bool
Python
![Page 17: Constraint Based Synthesis · Synthesis: modern view = ∧ ∧ ∧ Space of programs Safety Properties Test Harnesses Input/Output Examples](https://reader034.vdocuments.mx/reader034/viewer/2022051604/6005ae4897d1fc360b54fce7/html5/thumbnails/17.jpg)
SketchCompilerUnrollInlineEnumerate
Python
x + y addMT(x,y)
15 -
- -
INTint
list
bool
10 -
- -
INTint
list
bool
5 -
- -
INTint
list
bool
![Page 18: Constraint Based Synthesis · Synthesis: modern view = ∧ ∧ ∧ Space of programs Safety Properties Test Harnesses Input/Output Examples](https://reader034.vdocuments.mx/reader034/viewer/2022051604/6005ae4897d1fc360b54fce7/html5/thumbnails/18.jpg)
SketchCompilerUnrollInlineEnumerate
Python
x + y addMT(x,y)
- -
[1,2] -
LISTint
list
bool
- -
[3] -
LISTint
list
bool
- -
[1,2,3] -
LISTint
list
bool
![Page 19: Constraint Based Synthesis · Synthesis: modern view = ∧ ∧ ∧ Space of programs Safety Properties Test Harnesses Input/Output Examples](https://reader034.vdocuments.mx/reader034/viewer/2022051604/6005ae4897d1fc360b54fce7/html5/thumbnails/19.jpg)
SketchCompilerUnrollInlineEnumerate
Python
x + y addMT(x,y)
- -
[3] -
LISTint
list
bool
5 -
- -
INTint
list
bool
assert false;
![Page 20: Constraint Based Synthesis · Synthesis: modern view = ∧ ∧ ∧ Space of programs Safety Properties Test Harnesses Input/Output Examples](https://reader034.vdocuments.mx/reader034/viewer/2022051604/6005ae4897d1fc360b54fce7/html5/thumbnails/20.jpg)
SketchCompilerUnrollInlineEnumerate
Python
Error Model
• return a return [0]
• range(a1, a2) range(a1+1,a2)
• a0 == a1 False
![Page 21: Constraint Based Synthesis · Synthesis: modern view = ∧ ∧ ∧ Space of programs Safety Properties Test Harnesses Input/Output Examples](https://reader034.vdocuments.mx/reader034/viewer/2022051604/6005ae4897d1fc360b54fce7/html5/thumbnails/21.jpg)
SketchCompilerUnrollInlineEnumerate
Python
Error Model
range(0, len(poly))
range(a1,a2) range(a1+1, a2)
range({0 ,1}, len(poly))
default choice
![Page 22: Constraint Based Synthesis · Synthesis: modern view = ∧ ∧ ∧ Space of programs Safety Properties Test Harnesses Input/Output Examples](https://reader034.vdocuments.mx/reader034/viewer/2022051604/6005ae4897d1fc360b54fce7/html5/thumbnails/22.jpg)
SketchCompilerUnrollInlineEnumerate
Python
Error Model
{ , } modifyMT0( )
MultiType modifyMT0( ){if(??) returnelse
choice0 = truereturn
}
- -
- -
- -
- -
- -
- -
- -
- -
- -
- -
// default choice
// non-default choice- -
- -
![Page 23: Constraint Based Synthesis · Synthesis: modern view = ∧ ∧ ∧ Space of programs Safety Properties Test Harnesses Input/Output Examples](https://reader034.vdocuments.mx/reader034/viewer/2022051604/6005ae4897d1fc360b54fce7/html5/thumbnails/23.jpg)
Benchmark Test SetevalPoly-6.00 13
compBal-stdin-6.00 52compDeriv-6.00 103hangman2-6.00x 218prodBySum-6.00 268oddTuples-6.00 344hangman1-6.00x 351evalPoly-6.00x 541
compDeriv-6.00x 918oddTuples-6.00x 1756iterPower-6.00x 2875
recurPower-6.00x 2938iterGCD-6.00x 2988
![Page 24: Constraint Based Synthesis · Synthesis: modern view = ∧ ∧ ∧ Space of programs Safety Properties Test Harnesses Input/Output Examples](https://reader034.vdocuments.mx/reader034/viewer/2022051604/6005ae4897d1fc360b54fce7/html5/thumbnails/24.jpg)
Average Running Time (in s)
05
101520253035
Tim
e (in
s)
![Page 25: Constraint Based Synthesis · Synthesis: modern view = ∧ ∧ ∧ Space of programs Safety Properties Test Harnesses Input/Output Examples](https://reader034.vdocuments.mx/reader034/viewer/2022051604/6005ae4897d1fc360b54fce7/html5/thumbnails/25.jpg)
Feedback Generated (Percentage)
0%10%20%30%40%50%60%70%80%90%
Feed
back
Per
cent
age
![Page 26: Constraint Based Synthesis · Synthesis: modern view = ∧ ∧ ∧ Space of programs Safety Properties Test Harnesses Input/Output Examples](https://reader034.vdocuments.mx/reader034/viewer/2022051604/6005ae4897d1fc360b54fce7/html5/thumbnails/26.jpg)
Invariant Synthesis forOptimizationWork with Alvin Cheung and Shoaib Kamil
![Page 27: Constraint Based Synthesis · Synthesis: modern view = ∧ ∧ ∧ Space of programs Safety Properties Test Harnesses Input/Output Examples](https://reader034.vdocuments.mx/reader034/viewer/2022051604/6005ae4897d1fc360b54fce7/html5/thumbnails/27.jpg)
Optimization then and now
Naïve source code
Optimal executableKind-of-OK executable
ATLAS Pochoir
Domain specific problem description
Close to optimal implementation
Halide
![Page 28: Constraint Based Synthesis · Synthesis: modern view = ∧ ∧ ∧ Space of programs Safety Properties Test Harnesses Input/Output Examples](https://reader034.vdocuments.mx/reader034/viewer/2022051604/6005ae4897d1fc360b54fce7/html5/thumbnails/28.jpg)
What about existingsource code?
Source Code DSL ProgramProof ofEquivalence
Synthesis
![Page 29: Constraint Based Synthesis · Synthesis: modern view = ∧ ∧ ∧ Space of programs Safety Properties Test Harnesses Input/Output Examples](https://reader034.vdocuments.mx/reader034/viewer/2022051604/6005ae4897d1fc360b54fce7/html5/thumbnails/29.jpg)
Java to SQL
Application
SQL Queries
Database
Relations
ORMlibraries
Methods
Objects
![Page 30: Constraint Based Synthesis · Synthesis: modern view = ∧ ∧ ∧ Space of programs Safety Properties Test Harnesses Input/Output Examples](https://reader034.vdocuments.mx/reader034/viewer/2022051604/6005ae4897d1fc360b54fce7/html5/thumbnails/30.jpg)
Java to SQL
Application
SQL Queries
Database
Relations
ORMlibraries
Methods
Objects
![Page 31: Constraint Based Synthesis · Synthesis: modern view = ∧ ∧ ∧ Space of programs Safety Properties Test Harnesses Input/Output Examples](https://reader034.vdocuments.mx/reader034/viewer/2022051604/6005ae4897d1fc360b54fce7/html5/thumbnails/31.jpg)
Java to SQL
List getUsersWithRoles () {List users = User.getAllUsers();List roles = Role.getAllRoles();
List results = new ArrayList();
for (User u : users) {
for (Role r : roles) {
if (u.roleId == r.id)
results.add(u); }}
return results; }
SELECT * FROM user
SELECT * FROM role
List getUsersWithRoles () {return executeQuery(“SELECT u FROM user u, role r WHERE u.roleId == r.id
ORDER BY u.roleId, r.id”; }
convert to
![Page 32: Constraint Based Synthesis · Synthesis: modern view = ∧ ∧ ∧ Space of programs Safety Properties Test Harnesses Input/Output Examples](https://reader034.vdocuments.mx/reader034/viewer/2022051604/6005ae4897d1fc360b54fce7/html5/thumbnails/32.jpg)
Java to SQL
List getUsersWithRoles () {List users = User.getAllUsers();List roles = Role.getAllRoles();
List results = new ArrayList();
for (User u : users) {
for (Role r : roles) {
if (u.roleId == r.id)
results.add(u); }}
return results; }results = outputExpr(users, roles)
Verificationconditions
preConditionouterInvariant(users/query(…), results/[], …)
outerInvariant(…) ∧ outer loop terminatesresults = outputExpr(users, roles) …
outerInvariant(users, roles, u, results, …)
innerInvariant(users, roles, u, r, results, …)
![Page 33: Constraint Based Synthesis · Synthesis: modern view = ∧ ∧ ∧ Space of programs Safety Properties Test Harnesses Input/Output Examples](https://reader034.vdocuments.mx/reader034/viewer/2022051604/6005ae4897d1fc360b54fce7/html5/thumbnails/33.jpg)
Program+
Unkown Post cond+
Unknown Invariants
VCGenerator
SketchC-like languagewith holes and
assertions
UnrollInlineEnumerate
![Page 34: Constraint Based Synthesis · Synthesis: modern view = ∧ ∧ ∧ Space of programs Safety Properties Test Harnesses Input/Output Examples](https://reader034.vdocuments.mx/reader034/viewer/2022051604/6005ae4897d1fc360b54fce7/html5/thumbnails/34.jpg)
6/17/2013 PLDI 2013 34
Real-world Evaluation
Wilos (project management application) – 62k LOC
Operation type # Fragments found # Fragments converted
Projection 1 1
Selection 13 10
Join 7 7
Aggregation 11 10
Total 33 28
![Page 35: Constraint Based Synthesis · Synthesis: modern view = ∧ ∧ ∧ Space of programs Safety Properties Test Harnesses Input/Output Examples](https://reader034.vdocuments.mx/reader034/viewer/2022051604/6005ae4897d1fc360b54fce7/html5/thumbnails/35.jpg)
6/17/2013 PLDI 2013 35
Real-world Evaluation
iTracker (bug tracking system) – 61k LOC
Operation type # Fragments found # Fragments converted
Projection 3 2
Selection 3 2
Join 1 1
Aggregation 9 7
Total 16 12
![Page 36: Constraint Based Synthesis · Synthesis: modern view = ∧ ∧ ∧ Space of programs Safety Properties Test Harnesses Input/Output Examples](https://reader034.vdocuments.mx/reader034/viewer/2022051604/6005ae4897d1fc360b54fce7/html5/thumbnails/36.jpg)
6/17/2013 PLDI 2013 36
Selection Query
50% selectivity 10% selectivity
100
1K
10K
0 50K 100K
Page
load
tim
e (m
s)
Total number of users in DB
original
inferred
100
1K
10K
0 50K 100K
Page
load
tim
e (m
s)
Total number of users in DB
originalinferred
![Page 37: Constraint Based Synthesis · Synthesis: modern view = ∧ ∧ ∧ Space of programs Safety Properties Test Harnesses Input/Output Examples](https://reader034.vdocuments.mx/reader034/viewer/2022051604/6005ae4897d1fc360b54fce7/html5/thumbnails/37.jpg)
100
1K
10K
100K
1000K
0 20K 40K 60K 80K 100K
Page
load
tim
e (m
s)
Number of roles / users in DB
original inferred
6/17/2013 PLDI 2013 37
Join Query
Nested-loop join Hash join!O(n2) O(n)
![Page 38: Constraint Based Synthesis · Synthesis: modern view = ∧ ∧ ∧ Space of programs Safety Properties Test Harnesses Input/Output Examples](https://reader034.vdocuments.mx/reader034/viewer/2022051604/6005ae4897d1fc360b54fce7/html5/thumbnails/38.jpg)
What about HPC?
LegacyFortran/C++Code
Stencil DLS(Halide)
Proof ofEquivalence
Synthesis
![Page 39: Constraint Based Synthesis · Synthesis: modern view = ∧ ∧ ∧ Space of programs Safety Properties Test Harnesses Input/Output Examples](https://reader034.vdocuments.mx/reader034/viewer/2022051604/6005ae4897d1fc360b54fce7/html5/thumbnails/39.jpg)
Legacy to Halide
for (k=y_min-2;k<=y_max+2;k++) {for (j=x_min-2;j<=x_max+2;j++) {
post_vol[((x_max+5)*(k-(y_min-2))+(j)-(x_min-2))]=volume[((x_max+4)*(k-(y_min-2))+(j)-(x_min-2))]
+ vol_flux_y[((x_max+4)*(k+1 -(y_min-2))+(j)-(x_min-2))]- vol_flux_y[((x_max+4)*(k-(y_min-2))+(j)-(x_min-2))];
}}
∀ , ∈ post_vol[j,k] = volume[j,k]+ vol_flux[j, k+1] + vol_flux[j, k]
![Page 40: Constraint Based Synthesis · Synthesis: modern view = ∧ ∧ ∧ Space of programs Safety Properties Test Harnesses Input/Output Examples](https://reader034.vdocuments.mx/reader034/viewer/2022051604/6005ae4897d1fc360b54fce7/html5/thumbnails/40.jpg)
Invariants
∀ , ∈ ∶, = ( { , , , } )
![Page 41: Constraint Based Synthesis · Synthesis: modern view = ∧ ∧ ∧ Space of programs Safety Properties Test Harnesses Input/Output Examples](https://reader034.vdocuments.mx/reader034/viewer/2022051604/6005ae4897d1fc360b54fce7/html5/thumbnails/41.jpg)
for(int i=0; i<n-1; ++i)for(int j=0; j<m-1; ++j)
out[i,j] = sin(in[i,j])
Invariants
∀ , ∈ ∶, = ( { , , , } )
![Page 42: Constraint Based Synthesis · Synthesis: modern view = ∧ ∧ ∧ Space of programs Safety Properties Test Harnesses Input/Output Examples](https://reader034.vdocuments.mx/reader034/viewer/2022051604/6005ae4897d1fc360b54fce7/html5/thumbnails/42.jpg)
for(int i=0; i<n-1; ++i)for(int j=0; j<m-1; ++j)
out[i,j] = sin(in[i,j])
Invariants
∀ , ∈ ∶, = ( { , , , } )
![Page 43: Constraint Based Synthesis · Synthesis: modern view = ∧ ∧ ∧ Space of programs Safety Properties Test Harnesses Input/Output Examples](https://reader034.vdocuments.mx/reader034/viewer/2022051604/6005ae4897d1fc360b54fce7/html5/thumbnails/43.jpg)
Challenges
• Big invariants
• Complex floating point arithmetic
• Universal Quantifiers
![Page 44: Constraint Based Synthesis · Synthesis: modern view = ∧ ∧ ∧ Space of programs Safety Properties Test Harnesses Input/Output Examples](https://reader034.vdocuments.mx/reader034/viewer/2022051604/6005ae4897d1fc360b54fce7/html5/thumbnails/44.jpg)
Quantifiers
∃ ∀ ( )outerInvariant(…) ∧ outer loop terminates→ output= outputExpr(in)
if(outerInvariant(…) && cond()){assert out == outputExpr(in) ;
}
The ∀ leads to a ∃∀∃ constraint!
![Page 45: Constraint Based Synthesis · Synthesis: modern view = ∧ ∧ ∧ Space of programs Safety Properties Test Harnesses Input/Output Examples](https://reader034.vdocuments.mx/reader034/viewer/2022051604/6005ae4897d1fc360b54fce7/html5/thumbnails/45.jpg)
Quantifiers
∃ ∀ ( )if(outerInvariant(…) && cond()){
assert out == outputExpr(in) ;}
Always safe to weaken this condition (more true)
∀ , ∈ ∶ , = ( { , , , } ), = ( { , , , } ), ∈Let the synthesizer discover !
![Page 46: Constraint Based Synthesis · Synthesis: modern view = ∧ ∧ ∧ Space of programs Safety Properties Test Harnesses Input/Output Examples](https://reader034.vdocuments.mx/reader034/viewer/2022051604/6005ae4897d1fc360b54fce7/html5/thumbnails/46.jpg)
Does it work?
• Can compute invariants for all stencils in Cloverleaf
• Most complex one takes 20 hrs on 15 Cores
• Currently pushing new benchmarks
![Page 47: Constraint Based Synthesis · Synthesis: modern view = ∧ ∧ ∧ Space of programs Safety Properties Test Harnesses Input/Output Examples](https://reader034.vdocuments.mx/reader034/viewer/2022051604/6005ae4897d1fc360b54fce7/html5/thumbnails/47.jpg)
??SketchC-like languagewith holes and
assertions
UnrollInlineEnumerate
• There is more to synthesis than “synthesis”
• You can do this too!• All the sketch infrastructure is available in open source