program synthesis with types peter-michael osera posera [email protected]
TRANSCRIPT
Program Synthesis with TypesPeter-Michael Osera
http://www.cis.upenn.edu/[email protected]
Programming is hard!
[3, 5, 2]
[3, 3, 5, 5, 2, 2]
?
[3, 5, 2]
[3, 3, 5, 5, 2, 2]
public static List<Integer> stutter(List<Integer> l) { List<Integer> ret = new LinkedList<Integer>(); for (int i = 0; i < l.size(); i++) { ret.add(l.get(i)); ret.add(l.get(i)); } return ret;}
Program SynthesisThe automatic generation of
programs from specification
(Input from sensors)
Brake (or not)
?
Correctness by Construction
PacemakersBanking SystemsMissile LaunchersAvionicsInsulin InjectorsLaser Eye Surgery MachinesCompilers and OptimizersDrug Delivery SystemsFinance and Trading SoftwareSilicon DesignsLunar Rover Navigation SystemsInfrastructure SoftwarePassword DatabasesCritical Runtime SystemsConference Management SystemsOS KernelsMedical Record DatabasesLife Support RegulatorsNuclear Reactor Management SystemsSupply Chain Management SystemsOnline Payment SystemsNetwork ProtocolsRocket ShipsMRI MachinesAcademic Record Databases
00101101010...
00010110...
?
Program Assistance
Bitsketching[Solar-Lezama
2005]
?
Broadening Participation
Flashfill [Gulwani
2011]
// ...List<Integer> ret = new LinkedList<Integer>();for (int i = 0; i < l.size(); i++) { ret.add(l.get(i)); ret.add(l.get(i));}return ret;// ...
// ... List<Integer> ret = new LinkedList<Integer>();for (int i : l) { ret.add(i); ret.add(i);}return ret;// ...
public static List<Integer>stutter(List<Integer> l) { return l; }
public static List<Integer>stutter(List<Integer> l) { return 5; }
// ... List<Integer> ret = new LinkedList<Integer>();for (int i = 0; i < l.size(); i++) { int x = l.get(i); ret.add(x); ret.add(x); i = 0;}return ret;// ...
Specification(i.e., “intent”)
Logical Specification
Input-output Examples[3, 5, 2] -> [3, 3, 5, 5, 2, 2]
Complete, hard to specify
Incomplete, easy to specify
Version space algebras, genetic prog. (Maching learning)[Lau 2000, Weimer 2009]
SMT solvers (Formal methods)[Solar-Lezama 2008, Kneuss 2013,
Torlak 2014]
Satisfiability Modulo Theories
Brute-force search (AI/logic)[Green 1969, Summers 1976,
Kitzelmann 2010, Albarghouthi 2013]
• Type theory,• Proof theory, and• Functional
programming[Osera 2015]
Functional Programming Logic
2. Exploit the inherent structure of typed, FPs
1. Synthesize programs using higher-order data(lambda x: x + 1)
Structure and Synthesis
Why does functional programming matter? [Huges 1990]
??
?? ?? ??
Pattern Matching
Higher-order Functions
“This is the key to functional programming’s power — it
allows improved modularization.”
Structure and Synthesis
Why does functional programming matter for synthesis?
??
?? ?? ??
Independent Reasoning
Restricted Search Space
Modularity simple, tractable synthesis techniques
1 2 3 4 5 6 7 8 9 10 111
10
100
1000
10000
100000
1000000
10000000
100000000
1000000000
10000000000
Untyped Programs Typed ProgramsProgram Size (AST nodes)
Nu
mb
er
of
Pro
gra
ms
Why Types?1,949,031,274
Untyped Programs
201,998Typed
Programs
21,704Typed, Normal
Programs
stutter
How do we write functional programs?
How do we synthesize functional programs?
Program Synthesis With Types
stutter = ?
type list =| Nil| Cons of int * list
Algebraic Datatype
Constructors
Constructor ArgumentsExample values of type
listNil
Cons (0, Nil)
Cons (1, Nil)
Cons (0, Cons (0, Nil))
Nil
Nil0
Nil1
Nil00
stutter : list -> list
type list =| Nil| Cons of int * list
Example values of type listNil
Cons (0, Nil)
Cons (1, Nil)
Cons (0, Cons (0, Nil))
[]
[0]
[1]
[0, 0]
stutter takes a list and produces a list
stutter : list -> list
: list -> list
Context:
Available Variables
stutter : list -> list
let rec stutter (l:list) : list = : list
Context:l : list
stutter : list -> list
stutter : list -> list
let rec stutter (l:list) : list = l
Context:l : list
stutter : list -> list
[3, 5, 2]
[3, 5, 2]
[3, 3, 5, 5, 2, 2]
stutter : list -> list
let rec stutter (l:list) : list = stutter l
Context:l : list
stutter : list -> list
[3, 5, 2]
[3, 3, 5, 5, 2, 2]
Loop!
stutter : list -> list
let rec stutter (l:list) : list = match l with | Nil -> : list | Cons (x, l’) -> : list
Context:l : list
stutter : list -> list
stutter : list -> list
let rec stutter (l:list) : list = match l with | Nil -> l | Cons (x, l’) -> : list
Context:l : list
stutter : list -> list
[]
[]
[]
stutter : list -> list
let rec stutter (l:list) : list = match l with | Nil -> [] | Cons (x, l’) -> : list
Context:l : list
stutter : list -> list
stutter : list -> list
let rec stutter (l:list) : list = match l with | Nil -> [] | Cons (x, l’) -> : list
Context:l : list
stutter : list -> list
x : int
l’ : list
[3, 5, 2]
Cons (3, Cons (5, Cons 2, Nil)))
stutter : list -> list
let rec stutter (l:list) : list = match l with | Nil -> [] | Cons (x, l’) -> : list
Context:l : list
stutter : list -> list
x : int
l’ : list
[3, 5, 2]
3 [5, 2]
Cons (x, Cons (x, )) stutter(l’)
[3, 3, 5, 5, 2, 2]
stutterCons (x2)
stutter : list -> list
let rec stutter (l:list) : list = match l with | Nil -> [] | Cons (x, l’) -> Cons (x, Cons (x, stutter l’))
kambing-mobile:synml posera$ ./synml.native tests/natlist/stutter.mllet stutter (l1:list) : list = match l1 with | Nil -> [] | Cons (n1, l2) -> Cons (n1, Cons (n1, stutter l2))
Type-directed Program Synthesiswith Examples
1. Refine program (and examples) by type
2. Guess a variable or function application
3. Learn more by pattern matching
stutter : list -> list
: list -> list
Goal Examples:[] => []
[0] => [0, 0]
[1, 0] => [1, 1, 0, 0]
Context
(Refine by type)
stutter : list -> list
let rec stutter (l:list) : list = : list
Goal Examples:[] => []
[0] => [0, 0]
[1, 0] => [1, 1, 0, 0]
Context
(Refine by type)
stutter : list -> list
let rec stutter (l:list) : list = : list
Goal Examples:[]
[0, 0]
[1, 1, 0, 0]
Contextl=[]
l=[0]
l=[1, 0]
(Guess a variable or application)
Example “World”
stutter : list -> list
let rec stutter (l:list) : list = l
Goal Examples:[]
[0, 0]
[1, 1, 0, 0]
Contextl=[]
l=[0]
l=[1, 0]
(Guess a variable or application)
1. Generate an expression*
2. Evaluate and check against each example world*We don’t generate stutter l – syntactic restriction on recursive
calls.
stutter : list -> list
let rec stutter (l:list) : list =
Goal Examples:[]
[0, 0]
[1, 1, 0, 0]
Contextl=[]
l=[0]
l=[1, 0]
(Learn more by pattern matching)let rec stutter (l:list) : list =
match l with | Nil -> : list | Cons (x, l’) -> : list
stutter : list -> list
let rec stutter (l:list) : list =
Goal Examples:[]
Contextl=[]
(Refine by type)
let rec stutter (l:list) : list = match l with | Nil -> : list | Cons (x, l’) -> : list
stutter : list -> list
let rec stutter (l:list) : list =
Goal Examples:[]
Contextl=[]
(Refine by type)
let rec stutter (l:list) : list = match l with | Nil -> Nil | Cons (x, l’) -> : list
stutter : list -> list
let rec stutter (l:list) : list =
Goal Examples:[0, 0]
[1, 1, 0, 0]
Contextl=[0], ...
l=[1, 0], ...
let rec stutter (l:list) : list = match l with | Nil -> Nil | Cons (x, l’) -> : list
stutter : list -> list
let rec stutter (l:list) : list =
Goal Examples:[0, 0]
[1, 1, 0, 0]
Contextl=[0], x=0, l’=[]
l=[1, 0], x=1, l’=[0]
let rec stutter (l:list) : list = match l with | Nil -> Nil | Cons (x, l’) -> : list
stutter : list -> list
let rec stutter (l:list) : list =
Goal Examples:Cons (0, Cons (0, ...))
Cons (1, Cons (1, ...))
Contextl=[0], x=0, l’=[]
l=[1, 0], x=1, l’=[0]
(Refine by type + guess a variable x2)let rec stutter (l:list) : list =
match l with | Nil -> Nil | Cons (x, l’) -> : list
Cons (x, Cons (x, ...))
stutter : list -> list
let rec stutter (l:list) : list =
Goal Examples:[]
[0, 0]
Contextl=[0], x=0, l’=[]
l=[1, 0], x=1, l’=[0]
(Guess a variable or application)let rec stutter (l:list) : list =
match l with | Nil -> Nil | Cons (x, l’) -> Cons (x, Cons (x, : list))
want: stutter l’
try: l, l’, ...
stutter : list -> list
let rec stutter (l:list) : list =
Goal Examples:[]
[0, 0]
Contextl=[0], x=0, l’=[]
l=[1, 0], x=1, l’=[0]
(Guess a variable or application)let rec stutter (l:list) : list =
match l with | Nil -> Nil | Cons (x, l’) -> Cons (x, Cons (x, stutter l’))
stutter=( [] => [] | [0] => [0, 0] | [1, 0] => [1, 1, 0, 0] )
stutter : list -> list
let rec stutter (l:list) : list =
Goal Examples:[]
[0, 0]
Contextl=[0], x=0, l’=[]
l=[1, 0], x=1, l’=[0]
(Guess a variable or application)let rec stutter (l:list) : list =
match l with | Nil -> Nil | Cons (x, l’) -> Cons (x, Cons (x, stutter l’))
stutter [] = []
stutter=( [] => [] | [0] => [0, 0] | [1, 0] => [1, 1, 0, 0] )
stutter : list -> list
let rec stutter (l:list) : list =
Goal Examples:[]
[0, 0]
Contextl=[0], x=0, l’=[]
l=[1, 0], x=1, l’=[0]
(Guess a variable or application)let rec stutter (l:list) : list =
match l with | Nil -> Nil | Cons (x, l’) -> Cons (x, Cons (x, stutter l’))
stutter [0] = [0, 0]
stutter=( [] => [] | [0] => [0, 0] | [1, 0] => [1, 1, 0, 0] )
stutter : list -> list
let rec stutter (l:list) : list =
Goal Examples:[]
[0, 0]
Contextl=[0], x=0, l’=[]
l=[1, 0], x=1, l’=[0]
let rec stutter (l:list) : list = match l with | Nil -> Nil | Cons (x, l’) -> Cons (x, Cons (x, stutter l’))
stutter=( [] => [] | [0] => [0, 0] | [1, 0] => [1, 1, 0, 0] )
1. Type Refinement
2. Guessing3. Pattern
Matching
...
Program Synthesis With Types
P -> P
x:P|-P
x:P|-P
x:P, n:Qx’:P|-P
x:P, n:Qx’:P|-P
x:P|-P
(Case Analysis
on x)
(Curry-Howard Isomorphism)
Synthesis=
Proof Search+
Example Refinement
λsyn, a logical foundation for program synthesis
Myth, a program synthesizer for typed, functional programs (OCaml)
let map : (nat -> nat) -> list -> list = fun (f2:nat -> nat) -> let rec f3 (l1:list) : list = match l1 with | Nil -> [] | Cons (n1, l2) -> Cons (f2 n1, f3 l2) in f3;;
real 0m0.040suser 0m0.023ssys 0m0.012s
map: higher-order functions
f2 is a function value!
let fvs_large : exp -> list = let rec f1 (e1:exp) : list = match e1 with | Unit -> [] | BVar n1 -> [n1] | FVar n1 -> [] | Lam (n1, e2) -> f1 e2 | App (e2, e3) -> append (f1 e2) (f1 e3) | Pair (e2, e3) -> append (f1 e2) (f1 e3) | Fst e2 -> f1 e2 | Snd e2 -> f1 e2 | Inl e2 -> f1 e2 | Inr e2 -> f1 e2 | Match (e2, n1, e3, n2, e4) -> (match f1 e2 with | Nil -> append (f1 e4) (f1 e3) | Cons (n3, l1) -> Cons (n3, append (f1 e3) (f1 e4))) | Const n1 -> [] | Binop (e2, b1, e3) -> append (f1 e3) (f1 e2) in f1;;
real 0m3.282suser 0m2.848ssys 0m0.413s
fvs_large: really big functions
75 AST nodes!(3.28s)
let pairwise_swap : list -> list = let rec f1 (l1:list) : list = match l1 with | Nil -> [] | Cons (n1, l2) -> (match f1 l2 with | Nil -> (match l2 with | Nil -> [] | Cons (n2, l3) -> Cons (n2, Cons (n1, f1 l3))) | Cons (n2, l3) -> []) in f1;;
real 0m1.315suser 0m1.212ssys 0m0.089s
pairwise_swap: surprising results
Ex. [0, 1, 0, 1] => [1, 0, 1, 0][0, 1, 0] => []
Inside-out recursion
• Evaluation: 43 benchmark tests + extended examples
• Compared to previous work: Benchmarks contains tests of similar complexity/size
Myth operates at comparable-or-better speeds
• Context size (number of variables) matters!
Test #/Examples
Size (AST)
T (min, s) T (ctx, s)
bool_xor 4 9 0.002 0.003
list_rev_snoc 5 11 0.006 0.018
list_fold 9 13 0.139 0.504
list_pairwise_swap
20 19 0.007 10.207
nat_iseven 4 10 0.001 0.014
tree_binsert 20 31 0.374 9.034
tree_nodes_at_level
24 22 1.093 4.917
arith 22 47 11.139 ---
...
My Mission
Help people harness the power of computation in its many forms, in particular, through computer
programming.
Programming Language Theory<->
Practical Tools and Implementation
Language interoperability
Low-level Memory Safety
Program Synthesis
PL + Computer Science
Education
Program Synthesis and Polymorphism
type ‘a list =| Nil| Cons of ‘a * list
Context:l1 : ‘a list
l2 : ‘a list
append : ‘a list -> ‘a list -> ‘a list
Terms of type list:l1l2
append l1 l2append l1 (append l1 l2)
...
append l1 [0]append l1 [1]append l2 [0]
...
Polymorphic list
Program Synthesis and Richer Types
How can richer type systems:
• Make synthesis more efficient?
• Allow us to synthesize richer classes of correct-by-construction programs?
Polymorphic Types
Linear Types
Dependent Types
Generic libraries, DSLs
Concurrency, security
Arbitrary properties
Semi-automated Programming
How do we integrate this into an editor?
Type-Directed Programming
module FreeMonad where data Free f a = Pure a | Free (f (Free f a)) instance Functor f => Monad (Free f) where return a = Pure a Pure a >>= f = _ Free f >>= g = _
(Example from https://wiki.haskell.org/GHC/Typed_holes)
FreeMonad.hs:9:18: Found hole ‘_’ with type: Free f b Where: ‘f’ is a rigid type variable bound by the instance declaration at FreeMonad.hs:7:10 ‘b’ is a rigid type variable bound by the type signature for (>>=) :: Free f a -> (a -> Free f b) -> Free f b at FreeMonad.hs:9:10 Relevant bindings include f :: a -> Free f b (bound at FreeMonad.hs:9:14) a :: a (bound at FreeMonad.hs:9:8) (>>=) :: Free f a -> (a -> Free f b) -> Free f b (bound at FreeMonad.hs:9:3) In the expression: _ In an equation for ‘>>=’: (Pure a) >>= f = _ In the instance declaration for ‘Monad (Free f)’
Type-Directed Programming
module FreeMonad where data Free f a = Pure a | Free (f (Free f a)) instance Functor f => Monad (Free f) where return a = Pure a Pure a >>= f = _ Free f >>= g = _
Found hole ‘_’ with type: Free f b...f :: a -> Free f ba :: a(>>=) :: Free f a -> (a -> Free f b) -> Free f b
f a :: Free f b
Type-Directed Programming
module FreeMonad where data Free f a = Pure a | Free (f (Free f a)) instance Functor f => Monad (Free f) where return a = Pure a Pure a >>= f = f a Free f >>= g = Free (fmap (>>= g) f)
How do we surface type-directed synthesis to the user?
How effective are these tools vs. “regular” programming?
InductFun: Synthesis and Education
How can synthesis be applied to programming education tools?
InductFun!
[Osera 2013]
An online tutorial system for learning proof by induction in the context of functional
programming.
http://inductfun.org
Typed program synthesis feedback for proofs
(e.g., introductory programs [Singh 2013])
Steve Zdancewic
David Walker(Princeton)
Jonathan Frankle
(Princeton)
Rohan Shah
Help people harness the power of computation in its many forms, in particular, through computer
programming.
http://www.cis.upenn.edu/[email protected]
Thanks!
My Mission
Programming Language Theory<->
Practical Tools and Implementation
Synthesizing Normal Forms
e ::= x | e1 e2 | fun x -> e | C (e1, .., en) | match e with p1 -> e1 .. pn -> en
VariablesApplicationFunctionsConstructo
rs
Pattern Matching
e ::= x | e1 e2 | fun x -> e | C (e1, .., en) | match e with p1 -> e1 .. pn -> en
t1 -> t2Datatypes
Introduction Forms (type-directed)
Types direct intro form and example decomposition
e ::= x | e1 e2 | fun x -> e | C (e1, .., en) | match e with p1 -> e1 .. pn -> en
????
Elimination Forms (Not type-directed)
Must guess-and-check all possible (well-typed) elim forms
E ::= x | E II ::= E | fun x -> I | C (I1, .., In) | match E with p1 -> I1 .. pn -> In
λsyn: synthesis foundations
Elimination Forms
Introduction Forms
• Soundness: we always synthesis something that is well-typed and agrees with the examples
• Synthesized terms are always in normal form
(Inspired by proof search andbidirectional typechecking [Pfenning 2004])
An expression in normal form cannot evaluate any further
Synthesizing Normal Forms
Not in normal form Normal form
(fun x -> x) 5 5
match Nil with| Nil -> 0...
0
x
stutter l’
An expression in normal form cannot evaluate any further
Synthesizing Normal Forms
Not in normal form Normal form
(fun x -> x) 5 5
match Nil with| Nil -> 0...
0
We only synthesize these!
x
stutter l’
1 2 3 4 5 6 7 8 9 10 111
10
100
1000
10000
100000
1000000
10000000
100000000
1000000000
10000000000
Untyped Programs Typed ProgramsProgram Size (AST nodes)
Nu
mb
er
of
Pro
gra
ms
1,949,031,274
201,998
21,704
Refinement Trees
Dictated by types and examples
Generating and evaluating many
terms (exponential wall)
Pre-compute type refinements and pattern
matches
Localize efficient typed-
term generation