Saucy3: Fast Symmetry Discovery in GraphsSaucy3: Fast Symmetry Discovery in Graphs

Hadi KatebiKarem A. SakallahIgor L. Markov

The University of Michigan

2

OutlineOutline

Graph symmetry Implicit representation of permutation sets:

Ordered Partition Pairs (OPPs) Basic permutation search tree Pruning via partition refinement:

– Non-isomorphic OPP pruning– Matching OPP pruning

Group-theoretic pruning:– Coset pruning– Orbit pruning

Algorithm trace Experimental results Conclusions

3

6

1

3

2

4

7

8

5

Graph SymmetryGraph Symmetry

6

1

3

2

4

7

8

5 (1 4)(2 3)

4

1

4

2

3

1

6

4

2

3

1

7

8

5

is a symmetry!

The set of edges is unchanged.

(1 2 3 4)(5 6)is not a symmetry!

2

4

3

1

5

2

4

3

1

6 5

2

4

3

1

7

8

6 5

2

4

3

1

7

8

6The set of edges is different.

4

Problem StatementProblem Statement

Given a graph G– with n vertices– and a partition p of its vertices (colors),– with unknown set of symmetries Sym(G)p,

Find a set of symmetries S Sym(G)p

– such that S generates Sym(G)p– and |S| ≤ n - 1

5

Graph Symmetry ToolsGraph Symmetry Tools

Nauty (McKay, ’81)– Blazed the trail– Tuned to quickly find the symmetries of

large sets of small graphs Saucy (Darga et al, DAC ’04)

– Graph symmetry can be fast for large yetsparse graphs

– > 1000x speedup over nauty for graphs with tens of thousands of vertices

Bliss (Junttila & Kaski, ’07)– Efficient canonical labeling of sparse graphs– Some improvements on Saucy

Traces (Piperno, ’08)

Primarily

canonical labelin

g tools;

symmetries p

roduced as a byproduct

6

PermutationsPermutations

Permutation: bijection from V to V

– Tabular representation– Cycle notation

Graph Symmetry: permutation that preserves edge relation

Permutation Composition: Symmetric group on m-element set T: Sm(T )

– |Sm(T )| = m !

1 2V , , ,n

v v v

7

Ordered PartitionsOrdered Partitions

1 2 mW W W

Unit OP: m = 1 Discrete OP: m = n

i i iW V , W

i j i jW W

1 i m iW V

8

Ordered Partition Pair (OPP)Ordered Partition Pair (OPP)

1 2

1 2

mT

B k

T T T

B B B

Isomorphic OPP: m = k and

Non-isomorphic OPP: m ≠ k or

Matching OPP: isomorphic and

Unit OPP: top and bottom ordered partitions are unit

Discrete OPP: top and bottom ordered partitions are discrete

for all i iT B i

for all such that 1i i iT B i T

for some i iT B i

9

Implicit Representation of Permutations Sets using OPPsImplicit Representation of Permutations Sets using OPPs

2 01021

1 2 0

Discrete OPP(single permutation)

10

Implicit Representation of Permutations Sets using OPPsImplicit Representation of Permutations Sets using OPPs

0 1 201 02 12 012 021

0 1 2

, ,, , , , ,

, ,

Unit OPP(m ! permutations)

11

Implicit Representation of Permutations Sets using OPPsImplicit Representation of Permutations Sets using OPPs

2 0 112 021

1 2 0

,,

,

Isomorphic OPP

12

31 0 2 4 3

13 0 2 43 0 2 4 1

, ,S , ,

, ,

Implicit Representation of Permutations Sets using OPPsImplicit Representation of Permutations Sets using OPPs

Matching OPP

13

Implicit Representation of Permutations Sets using OPPsImplicit Representation of Permutations Sets using OPPs

0 21

12 0

,

,

Non-isomorphic OPP

14

Basic Search for SymmetriesBasic Search for Symmetries

1

3

2

4

6

7

8

5

67

8

6 7 8

6 7 8

, ,

, ,

7 8 6

6 8 7

,

,

7 8 6

7 8 6

,

,

7 8 6

6 7 8

,

,

7 8 6

6 8 7

7 8 6

8 6 7

8 7 6

7 8 6

8 7 6

8 7 6

8 7 6

7 6 8

8 7 6

6 7 8

67

88 86

76

8X 7

8

6X

66

78

68

7ü

77

67

8ü

68

76

86

7X

77

87

6X

15

Vertex Partition RefinementVertex Partition Refinement

For each vertex v, compute a neighbor-count tuple Partition the vertices based on these tuples Repeat until the partition stabilizes

Try to distinguish vertices that are not symmetric

1 2 3 4 6 7 8

1 2 3 54 6 7 8

5, , , , , ,

, , , , , ,

1 2 3 4 5 6 7 8

1 2 3 4 5 6 7 8

, , , , , , ,

, , , , , , ,

R

61

3

2

4

7

85

(3)

(3) (3)

(3) (3) (3)

(3)(7)5

(2,1)

(2,1)(2,1)

(2,1)(2,1) (2,1)

(2,1)

16

Search-Tree Pruning via Vertex RefinementSearch-Tree Pruning via Vertex Refinement

67

8

6 7 8

6 7 8

, ,

, ,

7 8 6

7 8 6

,

,

77

67

8ü

78

68

7ü

R

8 7 6

8 7 6

8 7 6

7 8 6

17

Isomorphic RefinementIsomorphic Refinement

61

3

2

4

7

85

1 2 3 4 6 7 8

1 2 3 54 6 7 8

5, , , , , ,

, , , , , ,

2 3 4 6 7 8

1 3 4 6

1

28

5

7 5

, , , , ,

, , , , ,

2 3

1

6 7 4

3

1

28 54

8

6 7

5,

,

, ,

, ,

61

3

2

4

7

85

12

1 2

R

2

3 4

1

43

ü

Top Bottom

18

Non-Isomorphic RefinementNon-Isomorphic Refinement

61

3

2

4

7

85

1 2 3 4 6 7 8

1 2 3 54 6 7 8

5, , , , , ,

, , , , , ,

2 3 4 6 7 8

1 2 3 4

1

68

5

7 5

, , , , ,

, , , , ,

61

3

2

4

7

85

16

16

R

2

3 4

7

8

û2 3

7

6 7 8

1 2 3

5

8 64

14

5

,

,

, ,

, , ,

Top Bottom

19

Group GeneratorsGroup Generators

8 vertices: 8! = 40320 permutations

48 symmetries8 for square and 6 for triangle

Basic enumeration is inefficient Fundamental concept: symmetry group can be represented

implicitly by an exponentially smaller set of generators

6

1

3

2

4

7

8

5

identity(1 2 4 3)(1 4)(2 3)(1 3 4 2)(1 2)(3 4)(1 3)(2 4)(1 4)(2 3)

identity(6 7 8)(6 8 7)

(6 7)(6 8)(7 8)

20

Group GeneratorsGroup Generators

6

1

3

2

4

7

8

5

Generators:

g1 = (1 2)(3 4)

g2 = (2 3)

g3 = (6 7)

g4 = (6 8)

(1 3 4 2)(7 8) = g2g1g3g4g3

is a symmetry

g4g3 = (6 8) ◦ (6 7) = (6 7 8)

g3g4g3 = (6 7) ◦ (6 7 8) = (7 8)

g1g3g4g3 = (1 2)(3 4) ◦ (7 8) = (1 2)(3 4)(7 8)

g2g1g3g4g3 = (2 3) ◦ (1 2)(3 4)(7 8) = (1 3 4 2)(7 8)

21

Orbit PartitionOrbit Partition

Initial partition:{{1,2,3,4,5,6,7,8}}

Orbit partition:{{1,2,3,4},{5},

{6,7,8}} After degree refinement:

{{1,2,3,4,6,7,8},{5}}

6

1

3

2

4

7

8

5

Symmetry group induces an equivalence relation on vertices:the orbit partition

Refinement provides an approximation of the orbit partition Orbit partition:

– Built up incrementally from discovered symmetries– Used to prune search for redundant symmetries

22

CosetsCosets

A subgroup H of a group G partitions it into cosets Each coset has the same number of elements as

H G can be generated by composing a single

representative from each coset with H Used to prune search for redundant symmetries

23

Structure of Permutation Search TreeStructure of Permutation Search Tree

24

Algorithm OutlineAlgorithm Outline

Phase 1: Recursive subgroup decomposition Phase 2: Search for coset representatives …

surprisingly like SAT solving! Four pruning mechanisms:

– Group-theoretic Coset pruning: stop after coset representative is

found Orbit pruning: avoid looking for coset

representative– Algorithmic (due to OPP data structure):

Matching OPP pruning: identify candidate permutation before reaching leaves

Non-isomorphic OPP pruning: detect absence of coset representative in current subtree

25

Saucy 2.1 Search TreeSaucy 2.1 Search Tree

≈

61

3

2

4

7

85

{7,8}

{6,7,8}

{2,3}{6,7,8}

{1,2,3,4}{6,7,8}

Orbit Partition

1,2,3,4,6,7,8 5

1,2,3,4,6,7,8 5

2,3,4,6,7,8 1 5

2,3,4,6,7,8 1 5

6,7,8 4 2,3 1 5

6,7,8 4 2,3 1 5

6,7,8 4 3 2 1 5

6,7,8 4 3 2 1 5

7,8 6 4 3 2 1 5

7,8 6 4 3 2 1 5

2,3,4,6,7,8 1 5

1,3,4,6,7,8 2 5

6,7,8 4 2,3 1 5

6,7,8 3 1,4 2 5

6,7,8 4 3 2 1 5

6,7,8 4 2 3 1 5

7,8 6 4 3 2 1 5

6,8 7 4 3 2 1 5

(7 8)id (6 7)

6,7,8 4 3 2 1 5

6,7,8 3 4 1 2 5

2,3,4,6,7,8 1 5

1,2,3,4,7,8 6 5

6,7,8 | 4 2,3 1 5

1,2,3,4 7,8 6 5

//

≈

// //x

(2 3)=

(1 2)(3 4)=

≈ Coset pruning// Orbit Pruning= Matching OPPx Non-isomorphic OPP

11 121314

16

1718

22 23

66 67 68

21 24

77 78 76 78

R R R

26

Experimental EvaluationExperimental Evaluation

1183 SAT 2009 competition benchmarks– Application– Crafted – Random

Saucy on all 1183 Shatter on 47 most difficult benchmarks Experiments on SUN workstation

– 3GHz Intel Dual-Core CPU– 6MB cache– 8GB RAM– 64-bit Redhat Linux

27

Saucy Time vs. Graph VerticesSaucy Time vs. Graph Vertices

0.001

0.01

0.1

1

10

100

1000

1.E+01 1.E+02 1.E+03 1.E+04 1.E+05 1.E+06 1.E+07 1.E+08

Graph Vertices

Tim

e (s

)

Crafted Application Random

28

Saucy Time vs. Graph VerticesSaucy Time vs. Graph Vertices

Time out = 500 sec. Sacuy finished on all but 18

– connum: 6 (solved by varying branching heuristics)

– equilarge: 3– mod2-rand3bip: 9

Crafted category is the most challenging Weak trend towards larger run times for larger

graphs Saucy is really fast (runtime < 1 sec.) on 93%

(1101) of all benchmarks

29

Saucy Group Order vs. TestcaseSaucy Group Order vs. Testcase

1.E+00

1.E+06

1.E+12

1.E+18

1.E+24

1.E+30

1.E+36

1.E+42

1.E+48

1.E+54

1.E+60

0 50 100 150 200 250 300

Testcase

Gro

up O

rder

Crafted Application Random

30

Saucy Group Order vs. TestcaseSaucy Group Order vs. Testcase 323 benchmarks exhibited non-trivial symmetries Random category:

– 606 had no symmetry– 4 had one symmetry

Crafted category:– 175 out of 263 (66%) had symmetry– 18 timed out

Application category:– 144 out of 292 (50%) had symmetry

31

Saucy Group Order vs. #generatorsSaucy Group Order vs. #generators

1.E+00

1.E+35

1.E+70

1.E+105

1.E+140

1.E+175

1.E+210

1.E+245

1.E+280

1 10 100 1000

# Generators

Gro

up O

rder

Crafted Application Random

32

Saucy Group Order vs. #generatorsSaucy Group Order vs. #generators Guarantee to produce no more than n - 1

generators for n-vertex graph The number of reported generators is significantly

less than n - 1

33

Shatter +SBP VariablesShatter +SBP Variables

1.E+01

1.E+02

1.E+03

1.E+04

1.E+05

1.E+06

1.E+07

1.E+01 1.E+03 1.E+05 1.E+07Original Variables

+SB

P V

aria

bles

Crafted Application

34

Shatter +SBP ClausesShatter +SBP Clauses

1.E+01

1.E+02

1.E+03

1.E+04

1.E+05

1.E+06

1.E+07

1.E+01 1.E+03 1.E+05 1.E+07

Original Clauses

+SB

P C

laus

esCrafted Application

35

Shatter Variables and ClausesShatter Variables and Clauses

Shatter on 47 benchmarks – Unsolved benchmarks or benchmarks with

run time > 1000 sec. Application: 13 Crafted: 34

# added SBP clauses – Less than 4% for 29 benchmarks– Ranged from 25% to 133% for 18 benchmarks

# added SBP variables– Less than 1% for 23 benchmarks– Ranged from 9% to an order of magnitude for

24 benchmarks

36

Shatter (Symmetry-Breaking) FlowShatter (Symmetry-Breaking) Flow Use shatter to generate SBPs Add SBPs to the original CNF formula Pass the augmented CNF formula to the SAT

solver Statistical data:

– We used a re-ordering script to Reorder variables Reorder clauses

– 20 re-ordered versions of each benchmark 10 for the original benchmarks 10 for the SBP augmented benchmarks

Time-outs– Crafted: 5000 sec.– Application: 10000 sec.

37

SAT Solver Run Time SAT Solver Run Time

0100020003000400050006000700080009000

10000

0 1 2 3 4 5 6 7 8 9 10 11 12 13Benchmark

Tim

e (s

)

Original +SPB

1 2 3 4 5 6 7 8 9 10 11 12 13

1. mod3block_4vars_11gates_b2_restricted2. mod4block_2vars_8gates_u23. phnf-size10-exclusive-FIFO.used-as. sat04-991.sat05-4192.reshuffled-074. sgp_5-5-6.sat05-2675.reshuffled-075. sgp_5-6-8.sat05-2669.reshuffled-076. 9dlx_vliw_at_b_iq57. 9dlx_vliw_at_b_iq68. 9dlx_vliw_at_b_iq79. 9dlx_vliw_at_b_iq810. 9dlx_vliw_at_b_iq911. clauses-812. cube-11-h14-sat13. post-cbmc-aes-ee-r3-noholes

5 1 9

5

10

5

2

1

6 5

99

1

35

4

5 5

13 out of 47 benchmarked finished within time-outs

38

SAT Solver Run TimeSAT Solver Run Time

SBP augmented versions led to fewer time-outs All but 3 benchmarks were solved faster Four benchmarks which were reported to be

unsolvable in SAT 2009 competition were solved with the addition of SBPs

39

Conclusions and Future WorkConclusions and Future Work

For SAT: symmetry discovery is practically free Static symmetry breaking

– Many CNF instances possess no or little symmetry

– CNF instances with a lot of symmetry may or may not benefit from static symmetry breaking

Future work:– SAT-inspired algorithmic enhancements:

Branching heuristics Learning

– Dynamic symmetry breaking: Integrating symmetry breaking within the SAT

solver Uncovering hidden/conditional symmetries