adaptive annealing: a near-optimal connection between sampling and counting
DESCRIPTION
Adaptive annealing: a near-optimal connection between sampling and counting. Daniel Štefankovič (University of Rochester) Santosh Vempala Eric Vigoda (Georgia Tech). Counting. independent sets spanning trees matchings perfect matchings k-colorings. Compute the number of. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Adaptive annealing: a near-optimal connection between sampling and counting](https://reader034.vdocuments.mx/reader034/viewer/2022051517/56815870550346895dc5cf1d/html5/thumbnails/1.jpg)
Adaptive annealing: a near-optimal connection between
sampling and counting
Daniel Štefankovič(University of Rochester)
Santosh VempalaEric Vigoda(Georgia Tech)
![Page 2: Adaptive annealing: a near-optimal connection between sampling and counting](https://reader034.vdocuments.mx/reader034/viewer/2022051517/56815870550346895dc5cf1d/html5/thumbnails/2.jpg)
independent sets
spanning trees
matchings
perfect matchings
k-colorings
Counting
![Page 3: Adaptive annealing: a near-optimal connection between sampling and counting](https://reader034.vdocuments.mx/reader034/viewer/2022051517/56815870550346895dc5cf1d/html5/thumbnails/3.jpg)
independent sets
Compute the number of
(hard-core gas model)
independent set subset S of vertices, of a graph no two in S are neighbors
=
![Page 4: Adaptive annealing: a near-optimal connection between sampling and counting](https://reader034.vdocuments.mx/reader034/viewer/2022051517/56815870550346895dc5cf1d/html5/thumbnails/4.jpg)
# independent sets = 7
independent set = subset S of vertices no two in S are neighbors
![Page 5: Adaptive annealing: a near-optimal connection between sampling and counting](https://reader034.vdocuments.mx/reader034/viewer/2022051517/56815870550346895dc5cf1d/html5/thumbnails/5.jpg)
# independent sets = 5598861
independent set = subset S of vertices no two in S are neighbors
![Page 6: Adaptive annealing: a near-optimal connection between sampling and counting](https://reader034.vdocuments.mx/reader034/viewer/2022051517/56815870550346895dc5cf1d/html5/thumbnails/6.jpg)
#P-complete
#P-complete even for 3-regular graphs
graph G # independent sets in G
(Dyer, Greenhill, 1997)
![Page 7: Adaptive annealing: a near-optimal connection between sampling and counting](https://reader034.vdocuments.mx/reader034/viewer/2022051517/56815870550346895dc5cf1d/html5/thumbnails/7.jpg)
graph G # independent sets in G
approximation
randomization
?
![Page 8: Adaptive annealing: a near-optimal connection between sampling and counting](https://reader034.vdocuments.mx/reader034/viewer/2022051517/56815870550346895dc5cf1d/html5/thumbnails/8.jpg)
We would like to know Q
Goal: random variable Y such that
P( (1-)Q Y (1+)Q ) 1-
“Y gives (1-estimate”
![Page 9: Adaptive annealing: a near-optimal connection between sampling and counting](https://reader034.vdocuments.mx/reader034/viewer/2022051517/56815870550346895dc5cf1d/html5/thumbnails/9.jpg)
(approx) counting sampling Valleau,Card’72 (physical chemistry), Babai’79 (for matchings and colorings), Jerrum,Valiant,V.Vazirani’86
random variables: X1 X2 ... Xt
E[X1 X2 ... Xt]
= O(1)V[Xi]
E[Xi]2
the Xi are easy to estimate
= “WANTED”
the outcome of the JVV reduction:
such that1)
2)
squared coefficient of variation (SCV)
![Page 10: Adaptive annealing: a near-optimal connection between sampling and counting](https://reader034.vdocuments.mx/reader034/viewer/2022051517/56815870550346895dc5cf1d/html5/thumbnails/10.jpg)
E[X1 X2 ... Xt]
= O(1)V[Xi]
E[Xi]2
the Xi are easy to estimate
= “WANTED” 1)
2)
O(t2/2) samples (O(t/2) from each Xi)
give 1 estimator of “WANTED” with prob3/4
Theorem (Dyer-Frieze’91)
(approx) counting sampling
![Page 11: Adaptive annealing: a near-optimal connection between sampling and counting](https://reader034.vdocuments.mx/reader034/viewer/2022051517/56815870550346895dc5cf1d/html5/thumbnails/11.jpg)
JVV for independent sets
P( ) 1
# independent sets =
GOAL: given a graph G, estimate the number of independent sets of G
![Page 12: Adaptive annealing: a near-optimal connection between sampling and counting](https://reader034.vdocuments.mx/reader034/viewer/2022051517/56815870550346895dc5cf1d/html5/thumbnails/12.jpg)
JVV for independent sets
P( )
P( ) =
?
?
?
?
?P( )?P( )P( )
X1 X2 X3 X4
Xi [0,1] and E[Xi] ½ = O(1)V[Xi]
E[Xi]2
P(AB)=P(A)P(B|A)
![Page 13: Adaptive annealing: a near-optimal connection between sampling and counting](https://reader034.vdocuments.mx/reader034/viewer/2022051517/56815870550346895dc5cf1d/html5/thumbnails/13.jpg)
Self-reducibility for independent sets
?
?
?P( ) 57=
![Page 14: Adaptive annealing: a near-optimal connection between sampling and counting](https://reader034.vdocuments.mx/reader034/viewer/2022051517/56815870550346895dc5cf1d/html5/thumbnails/14.jpg)
?
?
?P( ) 57=
57=
Self-reducibility for independent sets
![Page 15: Adaptive annealing: a near-optimal connection between sampling and counting](https://reader034.vdocuments.mx/reader034/viewer/2022051517/56815870550346895dc5cf1d/html5/thumbnails/15.jpg)
?
?
?P( ) 57=
57=
57=
Self-reducibility for independent sets
![Page 16: Adaptive annealing: a near-optimal connection between sampling and counting](https://reader034.vdocuments.mx/reader034/viewer/2022051517/56815870550346895dc5cf1d/html5/thumbnails/16.jpg)
?
?P( ) 35=
35=
Self-reducibility for independent sets
![Page 17: Adaptive annealing: a near-optimal connection between sampling and counting](https://reader034.vdocuments.mx/reader034/viewer/2022051517/56815870550346895dc5cf1d/html5/thumbnails/17.jpg)
?
?P( ) 35=
35= 3
5=
Self-reducibility for independent sets
![Page 18: Adaptive annealing: a near-optimal connection between sampling and counting](https://reader034.vdocuments.mx/reader034/viewer/2022051517/56815870550346895dc5cf1d/html5/thumbnails/18.jpg)
35
57= 5
7=
35
57= 2
3 = 7
Self-reducibility for independent sets
![Page 19: Adaptive annealing: a near-optimal connection between sampling and counting](https://reader034.vdocuments.mx/reader034/viewer/2022051517/56815870550346895dc5cf1d/html5/thumbnails/19.jpg)
SAMPLERORACLEgraph G
random independentset of G
JVV: If we have a sampler oracle:
then FPRAS using O(n2) samples.
![Page 20: Adaptive annealing: a near-optimal connection between sampling and counting](https://reader034.vdocuments.mx/reader034/viewer/2022051517/56815870550346895dc5cf1d/html5/thumbnails/20.jpg)
SAMPLERORACLEgraph G
random independentset of G
JVV: If we have a sampler oracle:
then FPRAS using O(n2) samples.
SAMPLERORACLE
graph G set fromgas-model Gibbs at
ŠVV: If we have a sampler oracle:
then FPRAS using O*(n) samples.
![Page 21: Adaptive annealing: a near-optimal connection between sampling and counting](https://reader034.vdocuments.mx/reader034/viewer/2022051517/56815870550346895dc5cf1d/html5/thumbnails/21.jpg)
O*( |V| ) samples suffice for counting
Application – independent sets
Cost per sample (Vigoda’01,Dyer-Greenhill’01) time = O*( |V| ) for graphs of degree 4.
Total running time: O* ( |V|2 ).
![Page 22: Adaptive annealing: a near-optimal connection between sampling and counting](https://reader034.vdocuments.mx/reader034/viewer/2022051517/56815870550346895dc5cf1d/html5/thumbnails/22.jpg)
Other applications
matchings O*(n2m) (using Jerrum, Sinclair’89)
spin systems: Ising model O*(n2) for <C
(using Marinelli, Olivieri’95)
k-colorings O*(n2) for k>2 (using Jerrum’95)
total running time
![Page 23: Adaptive annealing: a near-optimal connection between sampling and counting](https://reader034.vdocuments.mx/reader034/viewer/2022051517/56815870550346895dc5cf1d/html5/thumbnails/23.jpg)
easy = hot
hard = cold
![Page 24: Adaptive annealing: a near-optimal connection between sampling and counting](https://reader034.vdocuments.mx/reader034/viewer/2022051517/56815870550346895dc5cf1d/html5/thumbnails/24.jpg)
1
2
4Hamiltonian
0
![Page 25: Adaptive annealing: a near-optimal connection between sampling and counting](https://reader034.vdocuments.mx/reader034/viewer/2022051517/56815870550346895dc5cf1d/html5/thumbnails/25.jpg)
Hamiltonian H : {0,...,n}
Big set =
Goal: estimate |H-1(0)|
|H-1(0)| = E[X1] ... E[Xt ]
![Page 26: Adaptive annealing: a near-optimal connection between sampling and counting](https://reader034.vdocuments.mx/reader034/viewer/2022051517/56815870550346895dc5cf1d/html5/thumbnails/26.jpg)
Distributions between hot and cold
(x) exp(-H(x))
= inverse temperature
= 0 hot uniform on = cold uniform on H-1(0)
(Gibbs distributions)
![Page 27: Adaptive annealing: a near-optimal connection between sampling and counting](https://reader034.vdocuments.mx/reader034/viewer/2022051517/56815870550346895dc5cf1d/html5/thumbnails/27.jpg)
(x)
Normalizing factor = partition function
exp(-H(x))
Z()= exp(-H(x)) x
Z()
Distributions between hot and cold
(x) exp(-H(x))
![Page 28: Adaptive annealing: a near-optimal connection between sampling and counting](https://reader034.vdocuments.mx/reader034/viewer/2022051517/56815870550346895dc5cf1d/html5/thumbnails/28.jpg)
Partition function
Z()= exp(-H(x)) x
have: Z(0) = ||want: Z() = |H-1(0)|
![Page 29: Adaptive annealing: a near-optimal connection between sampling and counting](https://reader034.vdocuments.mx/reader034/viewer/2022051517/56815870550346895dc5cf1d/html5/thumbnails/29.jpg)
(x) exp(-H(x))Z()
Assumption: we have a sampler oracle for
SAMPLERORACLE
graph G
subset of Vfrom
![Page 30: Adaptive annealing: a near-optimal connection between sampling and counting](https://reader034.vdocuments.mx/reader034/viewer/2022051517/56815870550346895dc5cf1d/html5/thumbnails/30.jpg)
(x) exp(-H(x))Z()
Assumption: we have a sampler oracle for
W
![Page 31: Adaptive annealing: a near-optimal connection between sampling and counting](https://reader034.vdocuments.mx/reader034/viewer/2022051517/56815870550346895dc5cf1d/html5/thumbnails/31.jpg)
(x) exp(-H(x))Z()
Assumption: we have a sampler oracle for
W X = exp(H(W)( - ))
![Page 32: Adaptive annealing: a near-optimal connection between sampling and counting](https://reader034.vdocuments.mx/reader034/viewer/2022051517/56815870550346895dc5cf1d/html5/thumbnails/32.jpg)
(x) exp(-H(x))Z()
Assumption: we have a sampler oracle for
W X = exp(H(W)( - ))
E[X] = (s) X(s) s
= Z()Z()
can obtain the following ratio:
![Page 33: Adaptive annealing: a near-optimal connection between sampling and counting](https://reader034.vdocuments.mx/reader034/viewer/2022051517/56815870550346895dc5cf1d/html5/thumbnails/33.jpg)
Partition function
Z() = exp(-H(x)) x
Our goal restated
Goal: estimate Z()=|H-1(0)|
Z() =Z(1) Z(2) Z(t)
Z(0) Z(1) Z(t-1)Z(0)
0 = 0 < 1 < 2 < ... < t =
...
![Page 34: Adaptive annealing: a near-optimal connection between sampling and counting](https://reader034.vdocuments.mx/reader034/viewer/2022051517/56815870550346895dc5cf1d/html5/thumbnails/34.jpg)
Our goal restated
Z() =Z(1) Z(2) Z(t)
Z(0) Z(1) Z(t-1)Z(0)...
How to choose the cooling schedule?
Cooling schedule:
E[Xi] =Z(i)
Z(i-1)
V[Xi]
E[Xi]2O(1)
minimize length, while satisfying
0 = 0 < 1 < 2 < ... < t =
![Page 35: Adaptive annealing: a near-optimal connection between sampling and counting](https://reader034.vdocuments.mx/reader034/viewer/2022051517/56815870550346895dc5cf1d/html5/thumbnails/35.jpg)
Parameters: A and n
Z() =AH: {0,...,n}
Z() = exp(-H(x)) x
Z() = ak e- kk=0
n
ak = |H-1(k)|
![Page 36: Adaptive annealing: a near-optimal connection between sampling and counting](https://reader034.vdocuments.mx/reader034/viewer/2022051517/56815870550346895dc5cf1d/html5/thumbnails/36.jpg)
Parameters
Z() =A H: {0,...,n}
independent sets
matchings
perfect matchings
k-colorings
2V
V!
kV
A
E
V
V
E
n
V!
![Page 37: Adaptive annealing: a near-optimal connection between sampling and counting](https://reader034.vdocuments.mx/reader034/viewer/2022051517/56815870550346895dc5cf1d/html5/thumbnails/37.jpg)
Previous cooling schedules
Z() =A H: {0,...,n}
+ 1/n (1 + 1/ln A) ln A
“Safe steps”
O( n ln A)Cooling schedules of length
O( (ln n) (ln A) )
0 = 0 < 1 < 2 < ... < t =
(Bezáková,Štefankovič, Vigoda,V.Vazirani’06)
(Bezáková,Štefankovič, Vigoda,V.Vazirani’06)
![Page 38: Adaptive annealing: a near-optimal connection between sampling and counting](https://reader034.vdocuments.mx/reader034/viewer/2022051517/56815870550346895dc5cf1d/html5/thumbnails/38.jpg)
No better fixed schedule possible
Z() =A H: {0,...,n}
Za() = (1 + a e ) A1+a
- n
A schedule that works for all
(with a[0,A-1])
has LENGTH ( (ln n)(ln A) )
![Page 39: Adaptive annealing: a near-optimal connection between sampling and counting](https://reader034.vdocuments.mx/reader034/viewer/2022051517/56815870550346895dc5cf1d/html5/thumbnails/39.jpg)
Parameters
Z() =A H: {0,...,n}Our main result:
non-adaptive schedules of length *( ln A )
Previously:
can get adaptive scheduleof length O* ( (ln A)1/2 )
![Page 40: Adaptive annealing: a near-optimal connection between sampling and counting](https://reader034.vdocuments.mx/reader034/viewer/2022051517/56815870550346895dc5cf1d/html5/thumbnails/40.jpg)
Related work
can get adaptive scheduleof length O* ( (ln A)1/2 )
Lovász-Vempala Volume of convex bodies in O*(n4) schedule of length O(n1/2)
(non-adaptive cooling schedule)
![Page 41: Adaptive annealing: a near-optimal connection between sampling and counting](https://reader034.vdocuments.mx/reader034/viewer/2022051517/56815870550346895dc5cf1d/html5/thumbnails/41.jpg)
Existential part
for every partition function there exists a cooling schedule of length O*((ln A)1/2)
Lemma:
can get adaptive scheduleof length O* ( (ln A)1/2 )
there exists
![Page 42: Adaptive annealing: a near-optimal connection between sampling and counting](https://reader034.vdocuments.mx/reader034/viewer/2022051517/56815870550346895dc5cf1d/html5/thumbnails/42.jpg)
W X = exp(H(W)( - ))
E[X2]
E[X]2
Z(2-) Z()
Z()2= C
E[X]Z() Z()=
Express SCV using partition function
(going from to )
![Page 43: Adaptive annealing: a near-optimal connection between sampling and counting](https://reader034.vdocuments.mx/reader034/viewer/2022051517/56815870550346895dc5cf1d/html5/thumbnails/43.jpg)
f()=ln Z()Proof:
E[X2]
E[X]2
Z(2-) Z()
Z()2= C
C’=(ln C)/2
2-
![Page 44: Adaptive annealing: a near-optimal connection between sampling and counting](https://reader034.vdocuments.mx/reader034/viewer/2022051517/56815870550346895dc5cf1d/html5/thumbnails/44.jpg)
f()=ln Z() f is decreasing f is convex f’(0) –n f(0) ln A
Proof:
either f or f’changes a lot
Let K:=f
(ln |f’|) 1K
1
![Page 45: Adaptive annealing: a near-optimal connection between sampling and counting](https://reader034.vdocuments.mx/reader034/viewer/2022051517/56815870550346895dc5cf1d/html5/thumbnails/45.jpg)
f:[a,b] R, convex, decreasingcan be “approximated” using
f’(a)f’(b)
(f(a)-f(b))
segments
![Page 46: Adaptive annealing: a near-optimal connection between sampling and counting](https://reader034.vdocuments.mx/reader034/viewer/2022051517/56815870550346895dc5cf1d/html5/thumbnails/46.jpg)
Proof:
2-
Technicality: getting to 2-
![Page 47: Adaptive annealing: a near-optimal connection between sampling and counting](https://reader034.vdocuments.mx/reader034/viewer/2022051517/56815870550346895dc5cf1d/html5/thumbnails/47.jpg)
Proof:
2-
i
i+1
Technicality: getting to 2-
![Page 48: Adaptive annealing: a near-optimal connection between sampling and counting](https://reader034.vdocuments.mx/reader034/viewer/2022051517/56815870550346895dc5cf1d/html5/thumbnails/48.jpg)
Proof:
2-
i
i+1i+2
Technicality: getting to 2-
![Page 49: Adaptive annealing: a near-optimal connection between sampling and counting](https://reader034.vdocuments.mx/reader034/viewer/2022051517/56815870550346895dc5cf1d/html5/thumbnails/49.jpg)
Proof:
2-
i
i+1i+2
Technicality: getting to 2-
i+3
ln ln A
extrasteps
![Page 50: Adaptive annealing: a near-optimal connection between sampling and counting](https://reader034.vdocuments.mx/reader034/viewer/2022051517/56815870550346895dc5cf1d/html5/thumbnails/50.jpg)
Existential Algorithmic
can get adaptive scheduleof length O* ( (ln A)1/2 )
there exists
can get adaptive scheduleof length O* ( (ln A)1/2 )
![Page 51: Adaptive annealing: a near-optimal connection between sampling and counting](https://reader034.vdocuments.mx/reader034/viewer/2022051517/56815870550346895dc5cf1d/html5/thumbnails/51.jpg)
Algorithmic construction
(x) exp(-H(x))Z()
using a sampler oracle for
we can construct a cooling schedule of length
38 (ln A)1/2(ln ln A)(ln n)
Our main result:
Total number of oracle calls
107 (ln A) (ln ln A+ln n)7 ln (1/)
![Page 52: Adaptive annealing: a near-optimal connection between sampling and counting](https://reader034.vdocuments.mx/reader034/viewer/2022051517/56815870550346895dc5cf1d/html5/thumbnails/52.jpg)
current inverse temperature
ideally move to such that
E[X] =Z()
Z()
E[X2]
E[X]2B2B1
Algorithmic construction
![Page 53: Adaptive annealing: a near-optimal connection between sampling and counting](https://reader034.vdocuments.mx/reader034/viewer/2022051517/56815870550346895dc5cf1d/html5/thumbnails/53.jpg)
current inverse temperature
ideally move to such that
E[X] =Z()
Z()
E[X2]
E[X]2B2B1
Algorithmic construction
X is “easy to estimate”
![Page 54: Adaptive annealing: a near-optimal connection between sampling and counting](https://reader034.vdocuments.mx/reader034/viewer/2022051517/56815870550346895dc5cf1d/html5/thumbnails/54.jpg)
current inverse temperature
ideally move to such that
E[X] =Z()
Z()
E[X2]
E[X]2B2B1
Algorithmic construction
we make progress (assuming B11)
![Page 55: Adaptive annealing: a near-optimal connection between sampling and counting](https://reader034.vdocuments.mx/reader034/viewer/2022051517/56815870550346895dc5cf1d/html5/thumbnails/55.jpg)
current inverse temperature
ideally move to such that
E[X] =Z()
Z()
E[X2]
E[X]2B2B1
Algorithmic construction
need to construct a “feeler” for this
![Page 56: Adaptive annealing: a near-optimal connection between sampling and counting](https://reader034.vdocuments.mx/reader034/viewer/2022051517/56815870550346895dc5cf1d/html5/thumbnails/56.jpg)
Algorithmic construction
current inverse temperature
ideally move to such that
E[X] =Z()
Z()
E[X2]
E[X]2B2B1
need to construct a “feeler” for this
= Z()
Z()
Z(2)
Z()
![Page 57: Adaptive annealing: a near-optimal connection between sampling and counting](https://reader034.vdocuments.mx/reader034/viewer/2022051517/56815870550346895dc5cf1d/html5/thumbnails/57.jpg)
Algorithmic construction
current inverse temperature
ideally move to such that
E[X] =Z()
Z()
E[X2]
E[X]2B2B1
need to construct a “feeler” for this
= Z()
Z()
Z(2)
Z()
bad “feeler”
![Page 58: Adaptive annealing: a near-optimal connection between sampling and counting](https://reader034.vdocuments.mx/reader034/viewer/2022051517/56815870550346895dc5cf1d/html5/thumbnails/58.jpg)
Rough estimator for Z()
Z()
Z() = ak e- kk=0
n
For W we have P(H(W)=k) = ak e- k
Z()
![Page 59: Adaptive annealing: a near-optimal connection between sampling and counting](https://reader034.vdocuments.mx/reader034/viewer/2022051517/56815870550346895dc5cf1d/html5/thumbnails/59.jpg)
Rough estimator for
Z() = ak e- kk=0
n
For W we have P(H(W)=k) = ak e- k
Z()
For U we have P(H(U)=k) = ak e- k
Z()
If H(X)=k likely at both , rough estimator
Z()
Z()
![Page 60: Adaptive annealing: a near-optimal connection between sampling and counting](https://reader034.vdocuments.mx/reader034/viewer/2022051517/56815870550346895dc5cf1d/html5/thumbnails/60.jpg)
Rough estimator for
For W we have P(H(W)=k) = ak e- k
Z()
For U we have P(H(U)=k) = ak e- k
Z()
P(H(U)=k)P(H(W)=k)ek(-)=
Z()
Z()
Z()
Z()
![Page 61: Adaptive annealing: a near-optimal connection between sampling and counting](https://reader034.vdocuments.mx/reader034/viewer/2022051517/56815870550346895dc5cf1d/html5/thumbnails/61.jpg)
Rough estimator for
Z() = ak e- kk=0
n
For W we have P(H(W)[c,d]) =
ak e- k
Z()
k=c
d
Z()
Z()
![Page 62: Adaptive annealing: a near-optimal connection between sampling and counting](https://reader034.vdocuments.mx/reader034/viewer/2022051517/56815870550346895dc5cf1d/html5/thumbnails/62.jpg)
P(H(U)[c,d])P(H(W)[c,d])
e ec(-)
e 1
If |-| |d-c| 1 then
Rough estimator for
We also need P(H(U) [c,d]) P(H(W) [c,d])to be large.
Z()
Z()
Z()
Z()
Z()
Z()
![Page 63: Adaptive annealing: a near-optimal connection between sampling and counting](https://reader034.vdocuments.mx/reader034/viewer/2022051517/56815870550346895dc5cf1d/html5/thumbnails/63.jpg)
Split {0,1,...,n} into h 4(ln n) ln Aintervals [0],[1],[2],...,[c,c(1+1/ ln A)],...
for any inverse temperature there exists a interval with P(H(W) I) 1/8h
We say that I is HEAVY for
![Page 64: Adaptive annealing: a near-optimal connection between sampling and counting](https://reader034.vdocuments.mx/reader034/viewer/2022051517/56815870550346895dc5cf1d/html5/thumbnails/64.jpg)
Algorithm
find an interval I which is heavy for the current inverse temperature
see how far I is heavy (until some *)
use the interval I for the feeler
repeat
Z()
Z()
Z(2)
Z()either * make progress, or * eliminate the interval I
![Page 65: Adaptive annealing: a near-optimal connection between sampling and counting](https://reader034.vdocuments.mx/reader034/viewer/2022051517/56815870550346895dc5cf1d/html5/thumbnails/65.jpg)
Algorithm
find an interval I which is heavy for the current inverse temperature
see how far I is heavy (until some *)
use the interval I for the feeler
repeat
Z()
Z()
Z(2)
Z()either * make progress, or * eliminate the interval I * or make a “long move”
![Page 66: Adaptive annealing: a near-optimal connection between sampling and counting](https://reader034.vdocuments.mx/reader034/viewer/2022051517/56815870550346895dc5cf1d/html5/thumbnails/66.jpg)
if we have sampler oracles for
then we can get adaptive scheduleof length t=O* ( (ln A)1/2 )
independent sets O*(n2) (using Vigoda’01, Dyer-Greenhill’01)
matchings O*(n2m) (using Jerrum, Sinclair’89)
spin systems: Ising model O*(n2) for <C
(using Marinelli, Olivieri’95) k-colorings O*(n2) for k>2 (using Jerrum’95)
![Page 67: Adaptive annealing: a near-optimal connection between sampling and counting](https://reader034.vdocuments.mx/reader034/viewer/2022051517/56815870550346895dc5cf1d/html5/thumbnails/67.jpg)
![Page 68: Adaptive annealing: a near-optimal connection between sampling and counting](https://reader034.vdocuments.mx/reader034/viewer/2022051517/56815870550346895dc5cf1d/html5/thumbnails/68.jpg)
O(t2/2) samples (O(t/2) from each Xi)
give 1 estimator of “WANTED” with prob3/4
Theorem (Dyer-Frieze’91)
Appendix – proof of:
E[X1 X2 ... Xt]
= O(1)V[Xi]
E[Xi]2
the Xi are easy to estimate
= “WANTED” 1)
2)
![Page 69: Adaptive annealing: a near-optimal connection between sampling and counting](https://reader034.vdocuments.mx/reader034/viewer/2022051517/56815870550346895dc5cf1d/html5/thumbnails/69.jpg)
P( Y gives (1)-estimate )
1 - V[Y]E[Y]2
1
Y=X1 + X2 + ... + Xn
n
The Bienaymé-Chebyshev inequality
![Page 70: Adaptive annealing: a near-optimal connection between sampling and counting](https://reader034.vdocuments.mx/reader034/viewer/2022051517/56815870550346895dc5cf1d/html5/thumbnails/70.jpg)
P( Y gives (1)-estimate )
1 -
Y=X1 + X2 + ... + Xn
nV[Y]E[Y]2
=1 V[X]
E[X]2n
The Bienaymé-Chebyshev inequality
squared coefficient of variation SCV
V[Y]E[Y]2
1
![Page 71: Adaptive annealing: a near-optimal connection between sampling and counting](https://reader034.vdocuments.mx/reader034/viewer/2022051517/56815870550346895dc5cf1d/html5/thumbnails/71.jpg)
P( Y gives (1)-estimate of Q )
Let X1,...,Xn,X be independent, identically distributed random variables, Q=E[X]. Let
The Bienaymé-Chebyshev inequality
1 - V[X]
n E[X]2
1
Then
Y=X1 + X2 + ... + Xn
n
![Page 72: Adaptive annealing: a near-optimal connection between sampling and counting](https://reader034.vdocuments.mx/reader034/viewer/2022051517/56815870550346895dc5cf1d/html5/thumbnails/72.jpg)
P( Y gives (1)-estimate of Q )
- 2 . n . E[X] / 3 1 –
Let X1,...,Xn,X be independent, identically distributed random variables, 0 X 1, Q=E[X]. Let
Chernoff’s bound
Y=X1 + X2 + ... + Xn
nThen
e
![Page 73: Adaptive annealing: a near-optimal connection between sampling and counting](https://reader034.vdocuments.mx/reader034/viewer/2022051517/56815870550346895dc5cf1d/html5/thumbnails/73.jpg)
n V[X]E[X]2
1
1
n 1
E[X]
3
ln (1/)
0X1
![Page 74: Adaptive annealing: a near-optimal connection between sampling and counting](https://reader034.vdocuments.mx/reader034/viewer/2022051517/56815870550346895dc5cf1d/html5/thumbnails/74.jpg)
n 1
E[X]
1
1
n 1
E[X]
3
ln (1/)
0X1
0X1
![Page 75: Adaptive annealing: a near-optimal connection between sampling and counting](https://reader034.vdocuments.mx/reader034/viewer/2022051517/56815870550346895dc5cf1d/html5/thumbnails/75.jpg)
Median “boosting trick”
P( ) 3/4
n 1
E[X]
4
(1-)Q (1+)Q
Y=X1 + X2 + ... + Xn
n
Y
=
![Page 76: Adaptive annealing: a near-optimal connection between sampling and counting](https://reader034.vdocuments.mx/reader034/viewer/2022051517/56815870550346895dc5cf1d/html5/thumbnails/76.jpg)
Median trick – repeat 2T times
(1-)Q (1+)Q
P( ) 3/4
P( ) 1 - e-T/4> T out of 2T
median is in
P( ) 1 - e
-T/4
![Page 77: Adaptive annealing: a near-optimal connection between sampling and counting](https://reader034.vdocuments.mx/reader034/viewer/2022051517/56815870550346895dc5cf1d/html5/thumbnails/77.jpg)
n 1
E[X]
32
n 1
E[X]
3
ln (1/)
0X1
0X1
+ median trick
ln (1/)
![Page 78: Adaptive annealing: a near-optimal connection between sampling and counting](https://reader034.vdocuments.mx/reader034/viewer/2022051517/56815870550346895dc5cf1d/html5/thumbnails/78.jpg)
n V[X]
E[X]2
32
n 1
E[X]
3
ln (1/)
0X1
+ median trick
ln (1/)
![Page 79: Adaptive annealing: a near-optimal connection between sampling and counting](https://reader034.vdocuments.mx/reader034/viewer/2022051517/56815870550346895dc5cf1d/html5/thumbnails/79.jpg)
O(t2/2) samples (O(t/2) from each Xi)
give 1 estimator of “WANTED” with prob3/4
Theorem (Dyer-Frieze’91)
Appendix – proof of:
E[X1 X2 ... Xt]
= O(1)V[Xi]
E[Xi]2
the Xi are easy to estimate
= “WANTED” 1)
2)
![Page 80: Adaptive annealing: a near-optimal connection between sampling and counting](https://reader034.vdocuments.mx/reader034/viewer/2022051517/56815870550346895dc5cf1d/html5/thumbnails/80.jpg)
How precise do the Xi have to be?
First attempt – Chernoff’s bound
![Page 81: Adaptive annealing: a near-optimal connection between sampling and counting](https://reader034.vdocuments.mx/reader034/viewer/2022051517/56815870550346895dc5cf1d/html5/thumbnails/81.jpg)
How precise do the Xi have to be?
First attempt – Chernoff’s bound
(1 )(1 )(1 )... (1 ) 1t
t
t
t
Main idea:
![Page 82: Adaptive annealing: a near-optimal connection between sampling and counting](https://reader034.vdocuments.mx/reader034/viewer/2022051517/56815870550346895dc5cf1d/html5/thumbnails/82.jpg)
How precise do the Xi have to be?
First attempt – Chernoff’s bound
(1 )(1 )(1 )... (1 ) 1t
t
t
t
Main idea:
each term (t2) samples (t3) total
n 1
E[X]
1
ln (1/)
![Page 83: Adaptive annealing: a near-optimal connection between sampling and counting](https://reader034.vdocuments.mx/reader034/viewer/2022051517/56815870550346895dc5cf1d/html5/thumbnails/83.jpg)
How precise do the Xi have to be?
Bienaymé-Chebyshev is better(Dyer-Frieze’1991)
P( X gives (1)-estimate )
1 - V[X]E[X]2
1
squared coefficient of variation (SCV)
GOAL: SCV(X) 2/4
X=X1 X2 ... Xt
![Page 84: Adaptive annealing: a near-optimal connection between sampling and counting](https://reader034.vdocuments.mx/reader034/viewer/2022051517/56815870550346895dc5cf1d/html5/thumbnails/84.jpg)
How precise do the Xi have to be?
Bienaymé-Chebyshev is better(Dyer-Frieze’1991)
SCV(X) = (1+SCV(X1)) ... (1+SCV(Xt)) - 1
Main idea:
SCV(Xi) t SCV(X) <
SCV(X)=V[X]
E[X]2
E[X2]
E[X]2= -1
![Page 85: Adaptive annealing: a near-optimal connection between sampling and counting](https://reader034.vdocuments.mx/reader034/viewer/2022051517/56815870550346895dc5cf1d/html5/thumbnails/85.jpg)
How precise do the Xi have to be?
Bienaymé-Chebyshev is better(Dyer-Frieze’1991)
X1 X2 ... Xt
X =Main idea:
SCV(Xi) t SCV(X) <
each term (t /2) samples (t2/2) total
![Page 86: Adaptive annealing: a near-optimal connection between sampling and counting](https://reader034.vdocuments.mx/reader034/viewer/2022051517/56815870550346895dc5cf1d/html5/thumbnails/86.jpg)
if we have sampler oracles for
then we can get adaptive scheduleof length t=O* ( (ln A)1/2 )
independent sets O*(n2) (using Vigoda’01, Dyer-Greenhill’01)
matchings O*(n2m) (using Jerrum, Sinclair’89)
spin systems: Ising model O*(n2) for <C
(using Marinelli, Olivieri’95) k-colorings O*(n2) for k>2 (using Jerrum’95)