![Page 1: Accelerating Simulated Annealing for the Permanent and Combinatorial Counting Problems](https://reader035.vdocuments.mx/reader035/viewer/2022062804/56649d7a5503460f94a5f15d/html5/thumbnails/1.jpg)
Accelerating Simulated Annealing for the Permanent and Combinatorial Counting
Problems
![Page 2: Accelerating Simulated Annealing for the Permanent and Combinatorial Counting Problems](https://reader035.vdocuments.mx/reader035/viewer/2022062804/56649d7a5503460f94a5f15d/html5/thumbnails/2.jpg)
Accelerating Simulated Annealing for the Permanent and Combinatorial Counting
ProblemsTalk outline:1. The Permanent problem
2. Simulated annealing for the Permanent aia(MCMC algorithm by JSV ‘01)
3. New simulated annealing schedule
![Page 3: Accelerating Simulated Annealing for the Permanent and Combinatorial Counting Problems](https://reader035.vdocuments.mx/reader035/viewer/2022062804/56649d7a5503460f94a5f15d/html5/thumbnails/3.jpg)
Permanent of an nxn matrix A
History & motivation:
• defined by Cauchy [1812]
• used in a variety of areas: statistical physics,
statistics, vision, anonymization systems, …
A =
![Page 4: Accelerating Simulated Annealing for the Permanent and Combinatorial Counting Problems](https://reader035.vdocuments.mx/reader035/viewer/2022062804/56649d7a5503460f94a5f15d/html5/thumbnails/4.jpg)
Permanent of an nxn matrix A
History & motivation:
• defined by Cauchy [1812]
• used in a variety of areas: statistical physics,
statistics, vision, anonymization systems, …
A =
![Page 5: Accelerating Simulated Annealing for the Permanent and Combinatorial Counting Problems](https://reader035.vdocuments.mx/reader035/viewer/2022062804/56649d7a5503460f94a5f15d/html5/thumbnails/5.jpg)
A – binary (entries 0 or 1):
rows
columns
Permanent of an nxn matrix A
adjacency matrix of a bipartite graph
![Page 6: Accelerating Simulated Annealing for the Permanent and Combinatorial Counting Problems](https://reader035.vdocuments.mx/reader035/viewer/2022062804/56649d7a5503460f94a5f15d/html5/thumbnails/6.jpg)
rows
columns
Permanent of an nxn matrix A
A – binary (entries 0 or 1):adjacency matrix of a bipartite graph
![Page 7: Accelerating Simulated Annealing for the Permanent and Combinatorial Counting Problems](https://reader035.vdocuments.mx/reader035/viewer/2022062804/56649d7a5503460f94a5f15d/html5/thumbnails/7.jpg)
rows
columns
The permanent counts the number of perfect matchings.
Permanent of an nxn matrix A
A – binary (entries 0 or 1):adjacency matrix of a bipartite graph
![Page 8: Accelerating Simulated Annealing for the Permanent and Combinatorial Counting Problems](https://reader035.vdocuments.mx/reader035/viewer/2022062804/56649d7a5503460f94a5f15d/html5/thumbnails/8.jpg)
O*(n26) fpras for any bipartite graph, later O*(n10)
Previous Work on the Permanent Problem
[Kasteleyn ’67]
#P-complete for non-planar graphs[Valiant ‘79]
poly-time for planar graphs (bipartite or not)
[Jerrum-Sinclair ’89]fpras for special graphs, e.g. the dense graphs, based on a Markov chain by Broder ’88
[Jerrum-Sinclair-Vigoda ‘01 & ‘05]
Our result: O*(n7)
![Page 9: Accelerating Simulated Annealing for the Permanent and Combinatorial Counting Problems](https://reader035.vdocuments.mx/reader035/viewer/2022062804/56649d7a5503460f94a5f15d/html5/thumbnails/9.jpg)
Broder chain
uniform sampling of perfect matchings of a given graph
At a perfect matching:
• remove a random edge
At a near-matching:
• pick a vertex at random
- if a hole, try to match with the other hole
- otherwise slide (if can)
![Page 10: Accelerating Simulated Annealing for the Permanent and Combinatorial Counting Problems](https://reader035.vdocuments.mx/reader035/viewer/2022062804/56649d7a5503460f94a5f15d/html5/thumbnails/10.jpg)
Broder chain
uniform sampling of perfect matchings of a given graph
At a perfect matching:
• remove a random edge
At a near-matching:
• pick a vertex at random
- if a hole, try to match with the other hole
- otherwise slide (if can)
![Page 11: Accelerating Simulated Annealing for the Permanent and Combinatorial Counting Problems](https://reader035.vdocuments.mx/reader035/viewer/2022062804/56649d7a5503460f94a5f15d/html5/thumbnails/11.jpg)
Broder chain
uniform sampling of perfect matchings of a given graph
At a perfect matching:
• remove a random edge
At a near-matching:
• pick a vertex at random
- if a hole, try to match with the other hole
- otherwise slide (if can)
![Page 12: Accelerating Simulated Annealing for the Permanent and Combinatorial Counting Problems](https://reader035.vdocuments.mx/reader035/viewer/2022062804/56649d7a5503460f94a5f15d/html5/thumbnails/12.jpg)
Broder chain
uniform sampling of perfect matchings of a given graph
At a perfect matching:
• remove a random edge
At a near-matching:
• pick a vertex at random
- if a hole, try to match with the other hole
- otherwise slide (if can)
![Page 13: Accelerating Simulated Annealing for the Permanent and Combinatorial Counting Problems](https://reader035.vdocuments.mx/reader035/viewer/2022062804/56649d7a5503460f94a5f15d/html5/thumbnails/13.jpg)
Broder chain
uniform sampling of perfect matchings of a given graph
At a perfect matching:
• remove a random edge
At a near-matching:
• pick a vertex at random
- if a hole, try to match with the other hole
- otherwise slide (if can)
![Page 14: Accelerating Simulated Annealing for the Permanent and Combinatorial Counting Problems](https://reader035.vdocuments.mx/reader035/viewer/2022062804/56649d7a5503460f94a5f15d/html5/thumbnails/14.jpg)
Broder chain
uniform sampling of perfect matchings of a given graph
At a perfect matching:
• remove a random edge
At a near-matching:
• pick a vertex at random
- if a hole, try to match with the other hole
- otherwise slide (if can)
![Page 15: Accelerating Simulated Annealing for the Permanent and Combinatorial Counting Problems](https://reader035.vdocuments.mx/reader035/viewer/2022062804/56649d7a5503460f94a5f15d/html5/thumbnails/15.jpg)
Broder chain
uniform sampling of perfect matchings of a given graph
At a perfect matching:
• remove a random edge
At a near-matching:
• pick a vertex at random
- if a hole, try to match with the other hole
- otherwise slide (if can)
![Page 16: Accelerating Simulated Annealing for the Permanent and Combinatorial Counting Problems](https://reader035.vdocuments.mx/reader035/viewer/2022062804/56649d7a5503460f94a5f15d/html5/thumbnails/16.jpg)
Does the Broder chain mix in polynomial time?
![Page 17: Accelerating Simulated Annealing for the Permanent and Combinatorial Counting Problems](https://reader035.vdocuments.mx/reader035/viewer/2022062804/56649d7a5503460f94a5f15d/html5/thumbnails/17.jpg)
Does the Broder chain mix in polynomial time?
![Page 18: Accelerating Simulated Annealing for the Permanent and Combinatorial Counting Problems](https://reader035.vdocuments.mx/reader035/viewer/2022062804/56649d7a5503460f94a5f15d/html5/thumbnails/18.jpg)
Does the Broder chain mix in polynomial time?
![Page 19: Accelerating Simulated Annealing for the Permanent and Combinatorial Counting Problems](https://reader035.vdocuments.mx/reader035/viewer/2022062804/56649d7a5503460f94a5f15d/html5/thumbnails/19.jpg)
Does the Broder chain mix in polynomial time?
Perfect matchings
State space
Exponentially smaller!
![Page 20: Accelerating Simulated Annealing for the Permanent and Combinatorial Counting Problems](https://reader035.vdocuments.mx/reader035/viewer/2022062804/56649d7a5503460f94a5f15d/html5/thumbnails/20.jpg)
Theorem[JS]: Rapid mixing if perfect matchings polynomially related to near-perfect matchings.
![Page 21: Accelerating Simulated Annealing for the Permanent and Combinatorial Counting Problems](https://reader035.vdocuments.mx/reader035/viewer/2022062804/56649d7a5503460f94a5f15d/html5/thumbnails/21.jpg)
Idea[JSV]: Weight the states so that the weighted ratio is always polynomially bounded.
Theorem[JS]: Rapid mixing if perfect matchings polynomially related to near-perfect matchings.
![Page 22: Accelerating Simulated Annealing for the Permanent and Combinatorial Counting Problems](https://reader035.vdocuments.mx/reader035/viewer/2022062804/56649d7a5503460f94a5f15d/html5/thumbnails/22.jpg)
Theorem[JS]: Rapid mixing if perfect matchings polynomially related to near-perfect matchings.
n2+1 regions, very different weight
Idea[JSV]: Weight the states so that the weighted ratio is always polynomially bounded.
u,v
![Page 23: Accelerating Simulated Annealing for the Permanent and Combinatorial Counting Problems](https://reader035.vdocuments.mx/reader035/viewer/2022062804/56649d7a5503460f94a5f15d/html5/thumbnails/23.jpg)
n2+1 regions, each about the same weight
Theorem[JS]: Rapid mixing if perfect matchings polynomially related to near-perfect matchings.
Idea[JSV]: Weight the states so that the weighted ratio is always polynomially bounded.
Ideal weights (for a matching with holes u,v):(# perfects) / (# nears with holes u,v)
u,v
![Page 24: Accelerating Simulated Annealing for the Permanent and Combinatorial Counting Problems](https://reader035.vdocuments.mx/reader035/viewer/2022062804/56649d7a5503460f94a5f15d/html5/thumbnails/24.jpg)
Ideal weights (for a matching with holes u,v):(# perfects) / (# nears with holes u,v)
A perfect matching sampled with prob. 1/(n2+1)
Computing ideal weights as hard as original problem?
Good:
Bad:
u,v
![Page 25: Accelerating Simulated Annealing for the Permanent and Combinatorial Counting Problems](https://reader035.vdocuments.mx/reader035/viewer/2022062804/56649d7a5503460f94a5f15d/html5/thumbnails/25.jpg)
Solution: Approximate the ideal weights
Ideal weights (for a matching with holes u,v):(# perfects) / (# nears with holes u,v)
A perfect matching sampled with prob. 1/(n2+1)
Computing ideal weights as hard as original problem?
Good:
Bad:
u,v
![Page 26: Accelerating Simulated Annealing for the Permanent and Combinatorial Counting Problems](https://reader035.vdocuments.mx/reader035/viewer/2022062804/56649d7a5503460f94a5f15d/html5/thumbnails/26.jpg)
Solution: Approximate the ideal weights
Simulated Annealing
Start with an easy instance,
gradually get to the target instance.
Kn,n
GIdeal weights (for a matching with holes u,v):(# perfects) / (# nears with holes u,v)
![Page 27: Accelerating Simulated Annealing for the Permanent and Combinatorial Counting Problems](https://reader035.vdocuments.mx/reader035/viewer/2022062804/56649d7a5503460f94a5f15d/html5/thumbnails/27.jpg)
Ideal weights (for a matching with holes u,v):(# perfects) / (# nears with holes u,v)
How?
![Page 28: Accelerating Simulated Annealing for the Permanent and Combinatorial Counting Problems](https://reader035.vdocuments.mx/reader035/viewer/2022062804/56649d7a5503460f94a5f15d/html5/thumbnails/28.jpg)
How?
• start with the complete graphs (weights easy to compute)
Ideal weights (for a matching with holes u,v):(# perfects) / (# nears with holes u,v)
![Page 29: Accelerating Simulated Annealing for the Permanent and Combinatorial Counting Problems](https://reader035.vdocuments.mx/reader035/viewer/2022062804/56649d7a5503460f94a5f15d/html5/thumbnails/29.jpg)
How?
• start with the complete graphs (weights easy to compute)
• fade away non-edges
Ideal weights (for a matching with holes u,v):(# perfects) / (# nears with holes u,v)
![Page 30: Accelerating Simulated Annealing for the Permanent and Combinatorial Counting Problems](https://reader035.vdocuments.mx/reader035/viewer/2022062804/56649d7a5503460f94a5f15d/html5/thumbnails/30.jpg)
How?
• start with the complete graphs (weights easy to compute)
• fade away non-edges
Ideal weights (for a matching with holes u,v):(# perfects) / (# nears with holes u,v)
![Page 31: Accelerating Simulated Annealing for the Permanent and Combinatorial Counting Problems](https://reader035.vdocuments.mx/reader035/viewer/2022062804/56649d7a5503460f94a5f15d/html5/thumbnails/31.jpg)
How?
• start with the complete graphs (weights easy to compute)
• fade away non-edges
Ideal weights (for a matching with holes u,v):(# perfects) / (# nears with holes u,v)
![Page 32: Accelerating Simulated Annealing for the Permanent and Combinatorial Counting Problems](https://reader035.vdocuments.mx/reader035/viewer/2022062804/56649d7a5503460f94a5f15d/html5/thumbnails/32.jpg)
How?
• start with the complete graphs (weights easy to compute)
• fade away non-edges
Ideal weights (for a matching with holes u,v):(# perfects) / (# nears with holes u,v)
![Page 33: Accelerating Simulated Annealing for the Permanent and Combinatorial Counting Problems](https://reader035.vdocuments.mx/reader035/viewer/2022062804/56649d7a5503460f94a5f15d/html5/thumbnails/33.jpg)
How?
• start with the complete graphs (weights easy to compute)
• fade away non-edges
Ideal weights (for a matching with holes u,v):(# perfects) / (# nears with holes u,v)
![Page 34: Accelerating Simulated Annealing for the Permanent and Combinatorial Counting Problems](https://reader035.vdocuments.mx/reader035/viewer/2022062804/56649d7a5503460f94a5f15d/html5/thumbnails/34.jpg)
How?
• start with the complete graphs (weights easy to compute)
• fade away non-edges
Ideal weights (for a matching with holes u,v):(# perfects) / (# nears with holes u,v)
![Page 35: Accelerating Simulated Annealing for the Permanent and Combinatorial Counting Problems](https://reader035.vdocuments.mx/reader035/viewer/2022062804/56649d7a5503460f94a5f15d/html5/thumbnails/35.jpg)
How?
• start with the complete graphs (weights easy to compute)
• fade away non-edges
Ideal weights (for a matching with holes u,v):(# perfects) / (# nears with holes u,v)
![Page 36: Accelerating Simulated Annealing for the Permanent and Combinatorial Counting Problems](https://reader035.vdocuments.mx/reader035/viewer/2022062804/56649d7a5503460f94a5f15d/html5/thumbnails/36.jpg)
How?
• start with the complete graphs (weights easy to compute)
• fade away non-edges
Ideal weights (for a matching with holes u,v):(# perfects) / (# nears with holes u,v)
![Page 37: Accelerating Simulated Annealing for the Permanent and Combinatorial Counting Problems](https://reader035.vdocuments.mx/reader035/viewer/2022062804/56649d7a5503460f94a5f15d/html5/thumbnails/37.jpg)
How?
• start with the complete graphs (weights easy to compute)
• fade away non-edges
Ideal weights (for a matching with holes u,v):(# perfects) / (# nears with holes u,v)
![Page 38: Accelerating Simulated Annealing for the Permanent and Combinatorial Counting Problems](https://reader035.vdocuments.mx/reader035/viewer/2022062804/56649d7a5503460f94a5f15d/html5/thumbnails/38.jpg)
The edges have activities:
- 1 for a real edge
- 2 [0,1] for a non-edge
Ideal weights (for a matching with holes u,v):(# perfects) / (# nears with holes u,v)
![Page 39: Accelerating Simulated Annealing for the Permanent and Combinatorial Counting Problems](https://reader035.vdocuments.mx/reader035/viewer/2022062804/56649d7a5503460f94a5f15d/html5/thumbnails/39.jpg)
How activities help?
• start with = 1
• compute corresponding weights n! / (n-1)!
![Page 40: Accelerating Simulated Annealing for the Permanent and Combinatorial Counting Problems](https://reader035.vdocuments.mx/reader035/viewer/2022062804/56649d7a5503460f94a5f15d/html5/thumbnails/40.jpg)
and 4-apx of weights
and 2-apx of
weights
2-apx = 4-apx for new
Repeat until < 1/n!
How activities help?
• start with = 1
• compute corresponding weights n! / (n-1)!
![Page 41: Accelerating Simulated Annealing for the Permanent and Combinatorial Counting Problems](https://reader035.vdocuments.mx/reader035/viewer/2022062804/56649d7a5503460f94a5f15d/html5/thumbnails/41.jpg)
We need:
O*(n6) per sample
O*(n2) samples (boosting from 4-apx to 2-apx )
O*(n2) -decrements (phases)
O*(n10) total to get a 2-apx of the ideal weights
Running Time [JSV]
Thm: The (,hole-weights)-Broder chain mixes in time O*(n6).
![Page 42: Accelerating Simulated Annealing for the Permanent and Combinatorial Counting Problems](https://reader035.vdocuments.mx/reader035/viewer/2022062804/56649d7a5503460f94a5f15d/html5/thumbnails/42.jpg)
Running Time [BSVV]
Thm: The (,hole-weights)-Broder chain mixes in time O*(n6).
O*(n4)
O*(n)
v
O*(n7)
O*(n4)
O*(n2)
We need:
O*(n6) per sample
O*(n2) samples (boosting from 4-apx to 2-apx )
O*(n2) -decrements (phases)
O*(n10) total to get a 2-apx of the ideal weights
![Page 43: Accelerating Simulated Annealing for the Permanent and Combinatorial Counting Problems](https://reader035.vdocuments.mx/reader035/viewer/2022062804/56649d7a5503460f94a5f15d/html5/thumbnails/43.jpg)
Reformulation of the problem
Promise: a set of polynomials of degree n such that
Goal: -sequence (from 1 to 1/n!) such that
• polynomials have a low-degree term
• non-negative integer coefficients sum to · n!
Tricky part:
Don’t know the coefficients!
0.2 0.4 0.6 0.8 1
1
2
3
4
5
6
for every polynomial ratio of consecutive values · 2
![Page 44: Accelerating Simulated Annealing for the Permanent and Combinatorial Counting Problems](https://reader035.vdocuments.mx/reader035/viewer/2022062804/56649d7a5503460f94a5f15d/html5/thumbnails/44.jpg)
Problem: no low-degree terms and xn “dominates”
if the value of some xj drops below 1/n!, ignore the polynomial
Intuition
The worst case is the set of polynomials xj, j=1,..,n
Fix:
0.2 0.4 0.6 0.8 1
0.2
0.4
0.6
0.8
1
O(n log n) points for xn
O(1/j log n) points for xj, j<n
TOTAL:
O(n log2 n) points
![Page 45: Accelerating Simulated Annealing for the Permanent and Combinatorial Counting Problems](https://reader035.vdocuments.mx/reader035/viewer/2022062804/56649d7a5503460f94a5f15d/html5/thumbnails/45.jpg)
Conclusions
Open Problems
• new cooling schedule: a blackbox, ijapplicable to other problems
• improved analysis of the weighted Broder chain
• interest of practical comunity
• other applications of the cooling schedule
• faster mixing result
• do we need n2 weights?
• non-bipartite graphs