synthesizing number transformations from input-output examples rishabh singh and sumit gulwani

Synthesizing Number Transformations from

Input-Output Examples

Rishabh Singh and Sumit Gulwani

Number

• One of the most commonly used data type

• Number Formatting– Every language has its own format #.00? (C#, Excel), 05.2f (Python, C)– End-users

Help-forums

Help-forums

Excel Help-forums

Excel Help-forums

upper 45 or 95

Observations from Help-forums

• Input-Output Examples – Specification mechanism

• Additional inputs for removing ambiguity

Excel add-in with same interface

Talk Outline

1. Number Transformation Language

2. Synthesis Algorithm

3. String & Number Language Combination

4. Synthesis Algorithm for Combination

5. Experiments

Generic Framework

• Expression Language L– Expressive and succinct

• Efficient Data structures for set of expressions– Version-space algebra

• GenerateStr – All sets of expressions from I-O example

• Intersect– Intersect two sets of expressions

Number Transformations

Number Transformation Language

Number Transformation Language

Number Transformation Language

Format String

• : minimum number of significant digits

• : maximum number of significant digits

• : number of whitespaces at end

Number Format

123.4567

123.46

123.4 123.40

(𝛼=2 , 𝛽=2,𝛾=0).NET :

Exactly 2 decimal places

Semantics

(𝛼=2 , 𝛽=2,𝛾=0)

Exactly 2 decimal places

24.589 24.59

24.2 24.20

GenerateStr

• Interval domains for

123.4567

123.46

(~𝛼=[0,2] ,~𝛽=[2,2] ,~𝛾=[0,2])

123.4 123.40

(~𝛼=[2,2] ,~𝛽=[2 ,∞ ] ,~𝛾=[0,0])

invariant

Intersect

(~𝛼=[0,2] ,~𝛽=[2,2] ,~𝛾=[0,2])

(~𝛼=[2,2] ,~𝛽=[2 ,∞ ] ,~𝛾=[0,0])

∧

(~𝛼=[2,2] ,~𝛽=[2,2 ] ,~𝛾=[0,0])

Extension to Integer parts12.4567 012.46

123.4 123.40

Dec(u) ( Int(u), Frac(u))

GenerateStr(Frac(u1), Frac(u2))

GenerateStr(Int(u1)R, Int(u2)R)

Rounding Numbers

• zero of interval

• size of interval

• nearest, upper, lower

𝛿

𝑥

Round Format

11 45

46 95

(𝑧=45 ,𝛿=50 ,𝑚=↑)=Min(Roundup(A1/45, 0)*45, Roundup(A1/95,0)*95)

Round off to upper 45

GenerateStr

Not enough info. to learn precise

Intersect

• Intersect((n1,n’1),(n2,n’2))

Intersect

• Intersect((31,45),(86,95))

Intersect

• Intersect((31,45),(86,95))

Intersect

• Intersect((31,45),(86,95))

Intersect

• Intersect((31,45),(86,95))

Intersect

• Intersect((31,45),(86,95))

Intersect

• Intersect((31,45),(86,95))

GenerateStr

𝛿∈𝐷𝑖𝑣𝑖𝑠𝑜𝑟𝑠 (1,000 ,000 )

Only need to maintain greatest value

Intersect

𝛿1∈𝐷𝑖𝑣𝑖𝑠𝑜𝑟𝑠 (𝑘1 )

𝛿2∈𝐷𝑖𝑣𝑖𝑠𝑜𝑟𝑠 (𝑘2)∧

𝛿∈𝐷𝑖𝑣𝑖𝑠𝑜𝑟𝑠¿

Combining String and Number Transformations

String Language [GulwaniPOPL11]

Combined Language

Combination Examples

Synthesis Algorithm

2004.07.08

7/8/2004

1)(-5, -4)2)(-5, dot -1)3)(-5, dot 2)4)(-5, 7)5)(6, -4)6)….

Program P1 to extract 7 in input (P1, 7 ->

7)

Synthesis Algorithm

2004.07.08

7/8/2004

(P1, 7 -> 7)

Program P2 to extract 07 in input1)(dot 1, dot 2)

2)(dot 1, dot -1)3)(-6, -4)4)(-6, dot -1)5)(-6, dot 2)6)….

(P2, 07 -> 7)

GenerateStr

2004.07.08

7/8/2004

(P1, 7 -> 7)(P2, 07 -> 7)

GenerateStr

2004.07.08

7/8/2004

(P3, 8 -> 8)(P4, 08 -> 8)

Constructing the DAG

071/283/425060748

0 1 2 3 4

567

(P1, 7 -> 7)

(P2, 07 -> 7)

8

Intersection

• DAG intersection

• Edge-wise program intersection

Experiments

Benchmarks

• 50 benchmark problems– Help forums– Excel product team

Number of I/O examples

1 2 30

5

10

15

20

25

30

35

Number of Input-Output Ex-amples

Nu

mb

er

of

Be

nch

ma

rks

Performance

1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 43 45 47 490

0.5

1

1.5

2

2.5

3

3.5

Benchmarks

Ru

nn

ing

Tim

e (

in s

eco

nd

s)

Related work

• String Transformations [Gulwani POPL11]

• Table Transformations [Singh,Gulwani VLDB12]

• Spreadsheet Data Manipulation using Examples[Gulwani, Harris, Singh CACM 2012]

Conclusion

• Number Transformation Language– Synthesis algorithm

• String + Number Transformations– Combined Synthesis algorithm

Thanks!

End-Users

Algorithm Designers

Software Developers

CAV

Large potential

Questions & Comments

Backup slides

Number Format

123.4567

123.46

123.4 123.4

(𝛼=0 , 𝛽=2 ,𝛾=0).NET :

Number Format

123.4567

123.46

123.4 123.4_

(𝛼=0 , 𝛽=2 ,𝛾=2).NET :

a b

c d

e f

g h

i j

k l

Help-forums

Number Format

123.4567

123.46

123.4 123.4

(𝛼=0 , 𝛽=2 ,𝛾=0).NET :

At most 2 decimal places

Number Format

123.4567

123.46

123.4 123.4_

(𝛼=0 , 𝛽=2 ,𝛾=2).NET :

Exactly 2 decimal places with space