program synthesis for automating education sumit gulwani (sumitg@microsoft.com) microsoft research,...

Program Synthesis for Automating Education

Sumit Gulwani(sumitg@microsoft.com)

Microsoft Research, Redmond

Program Synthesis = Synthesis of executable code from user intent expressed using some constraints.

• Enabling Technology is now available – Better search/logical reasoning techniques

• SAT/SMT solvers– Faster machines (Good application for multi-cores)

• A natural goal to enable people to easily leverage computational power.

2

Program Synthesis: Why Today?

Our techniques can synthesize a wide variety of algorithms/programs from logic and examples.

• Undergraduate book algorithms (e.g., sorting, dynamic prog)– [Srivastava/Gulwani/Foster, POPL 2010]

• Program Inverses (e.g, deserializers from serializers)– [Srivastava/Gulwani/Chaudhuri/Foster, MSR-TR-2010-34]

• Graph Algorithms (e.g., bi-partiteness check)– [Itzhaky/Gulwani/Immerman/Sagiv, OOPSLA 2010]

• String Manipulating Programs(e.g. ”Viktor Kuncak”-> “Kuncak, V.”)– [Gulwani, POPL 2011]

• Bit-vector algorithms (e.g., turn-off rightmost one bit)– [Jha/Gulwani/Seshia/Tiwari, ICSE 2010]

3

Program Synthesis: Recent Success

• Straight-line programs that use – Arithmetic Operators: +,-,*,/– Logical Operators: Bitwise and/or/not, Shift left/right

• Challenge: Combination of arithmetic + logical operators leads to unintuitive algorithms

• Application: Provides most-efficient way to accomplish a given task on a given architecture

4

Bitvector Algorithms

1 0 1 0 1 1 0 0

Turn-off rightmost 1-bit

5

Examples of Bitvector Algorithms

1 0 1 0 1 1 0 0

1 0 1 0 1 0 0 0

Z

Z & (Z-1)

1 0 1 0 1 0 1 1

Z

Z-1

1 0 1 0 1 0 0 0

&

Z & (Z-1)

6

Examples of Bitvector Algorithms

P25: Higher order half

of product of x and yo1 := and(x,0xFFFF);o2 := shr(x,16);o3 := and(y,0xFFFF);o4 := shr(y,16);o5 := mul(o1,o3);o6 := mul(o2,o3);o7 := mul(o1,o4);o8 := mul(o2,o4);o9 := shr(o5,16);o10 := add(o6,o9);o11 := and(o10,0xFFFF);o12 := shr(o10,16);o13 := add(o7,o11);o14 := shr(o13,16);o15 := add(o14,o12);res := add(o15,o8);

P24: Round up to next highest power of 2

o1 := sub(x,1);o2 := shr(o1,1);o3 := or(o1,o2);o4 := shr(o3,2);o5 := or(o3,o4);o6 := shr(o5,4);o7 := or(o5,o6);o8 := shr(o7,8);o9 := or(o7,o8);o10 := shr(o9,16);o11 := or(o9,o10);res := add(o10,1);

[ICSE 2010] Joint work with Susmit Jha, Sanjit Seshia (UC-Berkeley), Ashish Tiwari (SRI) and Venkie (MSR Redmond)

Students and Teachers

End-Users

Algorithm Designers

Software Developers

Most Transformational Target

Pyramid of Technology Users

7

Consumers of Program Synthesis Technology

Automating Education

Make education interactive and fun

• Automated problem solving (for students)– Provide hints– Point out mistakes and suggest fixes

• Creation of teaching material (for teachers)– Authoring tools– Problem construction

• Group interaction (for teachers/students)– Ask questions– Share annotations

Domains: Geometry, Algebra, Probability, Mechanics, Electrical Circuits, etc. 8

What is the role of PL + Logic + Synthesis?• Programming Language for Geometry

– Objects: Point, Line, Circle– Constructors

• Ruler(Point, Point) -> Line • Compass(Point, Point) -> Circle• Intersect(Circle, Circle) -> Pair of Points• Intersect(Line, Circle) -> Pair of Points• Intersect(Line, Line) -> Point

• Logic for Geometry– Inequality predicates over arithmetic expressions

• Distance(Point, Point), Angle(Line, Line), …• Automated Problem Solving

– Given pre/postcondition, synthesize a straight-line program 9

Geometry Constructions Domain

Automated Problem Solving• Given pre/postcondition, synthesize a straight-line

program

Example: Draw a line L’ perpendicular to a given line L.Precondition: truePostcondition: Angle(L’,L) = 90

ProgramStep 1: P1, P2 = ChoosePoint(L);Step 2: C1 = Circle(P1,P2);Step 3: C2 = Circle(P2,P1);Step 4: <P3, P4> = Intersect(C1,C2);Step 5: L’ = Line(P3,P4); 10

Geometry Domain: Automated Problem Solving

11

Constructing line L’ perpendicular to given line L

P1 P2

P3

P4

C1C2

L

L’

Step 1: P1, P2 = ChoosePoint(L);Step 2: C1 = Circle(P1,P2);Step 3: C2 = Circle(P2,P1);Step 4: <P3, P4> = Intersect(C1,C2);Step 5: L’ = Line(P3,P4);

• Bisect a given line.• Bisect an angle.• Copy an angle.• Draw a line parallel to a given line.• Draw an equilateral triangle given two points.• Draw a regular hexagon given a side.• Given 4 points, draw a square with each of the

sides passing through a different point.

Other Applications: • New approximate geometric constructions• 2D/3D planning problems

12

Examples of Geometry Constructions

• Synthesis, in general, is harder than verification.– Synthesis Problem: Given pre/postcondition, synthesize a

straight-line program– Verification Problem: Given pre/postcondition, and a

straight-line program, determine whether the Hoare triple holds.

• Decision procedures for verification of geometry constructions are known, but are complex.– Because of symbolic reasoning. 13

Synthesis Algorithm for Geometry Constructions

Precondition: TruePostcondition: Angle(L,L’) = 90

Step 1: P1, P2 = ChoosePoint(L);Step 2: C1 = Circle(P1,P2);Step 3: C2 = Circle(P2,P1);Step 4: <P3, P4> =

Intersect(C1,C2);Step 5: L’ = Line(P3,P4);

• Symbolic reasoning based decision procedures are complex.

• How about property testing?

Theorem: A construction that works (i.e., satisfies the postcondition) for a randomly chosen model of precondition also works for all models (w.h.p.).

Proof: • Objects constructed using ruler/compass can be

described using polynomial ops (+,-,*), square-root & division operator.

• The randomized polynomial identity testing algorithm lifts to square-root and division operators as well !

14

A simpler strategy for verification of Constructions

• Problem: Given two polynomials P1 and P2, determine whether they are equivalent.

• The naïve deterministic algorithm of expanding polynomials to compare them term-wise is exponential.

• A simple randomized test is probabilistically sufficient:– Choose random values r for polynomial variables x– If P1(r) ≠ P2(r), then P1 is not equivalent to P2.– Otherwise P1 is equivalent to P2 with high

probability,

15

Randomized Polynomial Identity Testing

Problem: Symbolic reasoning is hard.

Idea #1: Leverage Property Testing to reduce symbolic reasoning to concrete reasoning.• Construct a random input-output example (I,O) for the

problem and find a construction that can generate O from I.• Example: Construct incenter of a triangle.

– If I chose my input triangle to be an equilateral one, then the circumcenter construction also appears to work!• Since incenter = circumcenter for an equilateral traingle.

– But what are the chances of choosing an random triangle to be an equilateral one?

16

Synthesis Algorithm for Geometry Constructions

Exhuastive Search Strategy: Given input objects I and desired objects O, keep constructing new objects from I using ruler and compass until objects O are obtained.

Problem: Search blows up, i.e., too many (useless) objects get constructed.

– Example: n points lead to O(n^2) lines, which leads to O(n^4) points, and so on…

17

Synthesis Algorithm for Geometry Constructions

Problem: Search space is huge.

• Idea #2: Perform goal-directed reasoning.– Example: If an operation leads to construction of a line L

that passes through a desired output point, it is worthwhile constructing line L.

– Mimics human intelligence.– For this to be effective, we need solutions with small

depth.

• Idea #3: Work with a richer library of primitives.– Common constructions picked up from chapters of text-

books.– A search space of (small width, large depth) is converted

into one of (large width, small depth).– Mimics human experience/knowledge.

18

Synthesis Algorithm for Geometry Constructions

19

Search space Exploration: With/without goal-directness

20

Problem Solving Engine with Natural Interfaces

Natural Language Processing

Paraphrasing

Synthesis Engine

Problem Description in English

Problem Description as Logical Relation

Solution as Functional Program

Solution in English

Joint work with: Kalika Bali, Monojit Chaudhuri (MSR Bangalore) Vijay Korthikanti (UIUC), Ashish Tiwari (SRI)

Useful modules powered by problem solving engine

The next step is to architect several useful modules on top of the problem-solving architecture such as:

• Interactive feedback to students– Provide hints– Point out mistakes and suggest fixes

• Creation of teaching material (for teachers)– Problem construction– Authoring tools

21

What domains should we prioritize for automation?

• Mathematics– Algebra– Probability

• Physics– Mechanics– Electrical Circuits– Optics

• Chemistry– Quantitative Chemistry– Organic Chemistry

22

Other Domains

• Consider the problem of computing effective resistance between two nodes in a graph of resistances.

• MATLAB implements Kirchoff’s law based decision procedure– Algebraic sum of the currents at any circuit junction = 0– Sum of changes in potential in any complete loop = 0

23

Electrical Circuits: Concept-specific solutions

Joint work with: Swarat Chaudhuri (Penn State University)

• Consider the problem of computing effective resistance between two nodes in a graph of resistances.

• Kirchoff’s law based decision procedure is not useful for students who are expected to know only simpler concepts.

• Solutions need be parameterized by specific concepts such as:– Series/Parallel composition of resistances– Symmetry Reduction– Wheatstone Bridge

24

Electrical Circuits: Concept-specific solutions

Joint work with: Swarat Chaudhuri (Penn State University)

25

Resistance Reduction Concepts

If R3/R1 = R4/R2, then VD = VB

Parallel Combination

Series Combination

Wheat-stone Bridge

Students and Teachers

End-Users

Algorithm Designers

Software Developers

Most Transformational Target

Pyramid of Technology Users

26

Consumers of Program Synthesis Technology

Automating Education: Long-term Goals

• Ultra-intelligent computer

• Model of human mind

• Inter-stellar travel

27

• Tutorial at FMCAD 2010 (Oct 20, Wednesday, 9 am)– “Dimensions in Program Synthesis”– (More) applications and methodologies/techniques

for synthesis– A cool demo

• Technical report– “Synthesizing Geometry Constructions” by Sumit

Gulwani, Vijay Korthikanti, Ashish Tiwari – Send email to sumitg@microsoft for copy of the

draft.

28

References