An algorithmic proof of theLovasz Local Lemma

via resampling oracles

Jan VondrakIBM Almaden

Nick Harvey University of British Columbia

Unlikely, but possible

• Alex Tsipras replaces Mario Draghi as head of ECB

• Random assignment to a k-SAT formula is satisfiable?

• O(congestion + dilation) packet routing

• O(1)-approx for Santa Claus (max-min allocation) problem

• Other uses in rounding linear program relaxations

Lovasz Local Lemma: [Erdos-Lovasz ‘75]Let E1,E2,…,En be events in a probability space.

Suppose• Ei is jointly independent of all but d events

• P[Ei] · 1/ed for all i

Then P[Åi Ei] > 0.

Simultaneously avoiding many events

“Needle in a haystack”• It is possible to

avoid all events E1,E2,…,En.• But the

probability of this may be exponentially small.

Lovasz Local Lemma: [Erdos-Lovasz 1975]Let E1,E2,…,En be events in a probability space.

Suppose• Ei is jointly independent of all but d events

• P[Ei] · 1/ed for all i

Then P[Åi Ei] > 0.

Simultaneously avoiding many events

Beyond what nature intended:• Finding a very

rare object• Methods are

simple, but consequences are dramatic.

Example: 2-coloring hypergraphs

• Let Ei be event ith set is all-red or all-blue. Note P[Ei]=21-k.

• Ei is independent of all but 2k-1/e=d sets.

• Since P[Ei]· 1/ed, LLL implies P[Åi Ei]>0.

• So, there is a coloring where no set is all-red or all-blue.

• Given: System of sets of size k, each intersecting · 2k-1/e sets.

• Goal: Color vertices red/blue so that each set has both colors.

• Random approach: color each vertex independently red/blue.

• Under the original distribution P, it is unlikely but possible to avoid all events E1,E2,…,En.

• Can we find another distribution (i.e. a randomized algorithm)in which it is likely to avoid E1,E2,…,En?

• Limited scenarios, weaker parameters: [Beck ’91], [Alon ’91], [Molloy-Reed ’98], [Czumaj-Scheideler’00], [Srinivasan ’08]

Algorithmic LLL

• Under the original distribution P, it is unlikely but possible to avoid all events E1,E2,…,En.

• Can we find another distribution (i.e. a randomized algorithm)in which it is likely to avoid E1,E2,…,En?

Algorithmic LLL

Moser’s Algorithm: [Moser ‘08]While some set is all-red or all-blue

Pick an arbitrary one andre-color it randomly.

• Theorem: [Moser-Tardos ‘08] If each set intersects ·2k-1/e sets, the algorithm terminates in expected polynomial time.

The Moser-Tardos Framework(probability spaces with a product measure)

• Independent random variables Y1,…,Ym

• “Bad” events E1,E2,…,En

• Ei depends on variables var(Ei)• A dependency graph G:

i»j if var(Ei)Åvar(Ej);.• 9 x1,...,xn 2 (0,1) such that

P[Ei] · xi ¢ j2¡(i) (1-xj) 8i.

Moser-Tardos Algorithm: While some event Ei occurs,resample all variables in var(Ei).

• Theorem: [Moser-Tardos ‘08] The algorithm finds ! 2 Åi

Ei

after resampling operations, in expectation.

Y1 Y2 Y3 Y4

Y5 Y6 Y7 Y8

Y9 Y10 Y11 Y12

Y13 Y10 Y11 Y12

Algorithmic Improvements• Many extensions of Moser-Tardos:– deterministic LLL algorithm [Chandrasekaran et al. ’10]– exponentially many events [Haupler-Saha-Srinivasan

’10]– better conditions on probabilities [Kolipaka-Szegedy’11],

[Pegden ’13]• Require independent RVs Y1,…,Ym with product

measure

We consider events determined by subsets of underlying mutually independent random variables ... this appears to be necessary in order to get any algorithmic access to the problem [Moser-Tardos ’08]

• Beyond independent RVs– Permutations [Harris-Srinivasan ‘14]– Abstract flaw-correction framework [Achlioptas-Iliopoulos

‘14, next talk]

Algorithmic Local Lemmafor general probability spaces?

• For any application of LLL on a probability space , can we design a randomized algorithm to quickly find ! 2

Åi Ei ?– [Achlioptas-Iliopoulos ’14] partially answers

this; goes beyond LLL too.

• How can algorithm “move about” in general probability space?

• Our key definition: resampling oracles

• Consider probability space , measure ¹, events E1,E2,…,En and dependency relation denoted ».

• A resampling oracle for Ei is a random function ri : ! Measure Regeneration: If ! has

distribution ¹ conditioned on Ei,then ri(!) has distribution ¹. (Removes conditioning on Ei)

Local Causation: If Ek¿Ei and !Ek, then ri(!)Ek. (Cannot cause non-neighbor events)

Resampling Oracles

!

Ei ri(!)

• Consider probability space , measure ¹, events E1,E2,…,En and dependency relation denoted ».

• A resampling oracle for Ei is a random function ri : ! Measure Regeneration: If ! has

distribution ¹ conditioned on Ei,then ri(!) has distribution ¹. (Removes conditioning on Ei)

Local Causation: If Ek¿Ei and !Ek, then ri(!)Ek. (Cannot cause non-neighbor events)

Resampling Oracles

Our Main Result:An Algorithmic LLL in a General Setting

• Arbitrary probability space • Events E1,E2,…,En

• A dependency graph G for whichEi independent of non-neighbors(more generally, positive association with non-nbrs)

• A resampling oracle for each Ei

• 9 x1,...,xn 2 (0,1) such thatP[Ei] · xi ¢ j2¡(i) (1-xj) 8i.

• Theorem: [H-Vondrak ‘15] There is an algorithm that findsa point ! 2 Åi Ei after resampling operations, with high probability.

Rainbow trees

• Application: finding many disjoint rainbow-colored trees in edge-colored complete graphs.

Rainbow-tree packing conjecture• Conjecture: [Brualdi-Hollingsworth ’96] For

any proper edge-coloring of K2n with 2n-1 colors, there is a decomposition of its edges into n rainbow spanning trees.

Theorem: [Carraher et al ’13] If each color appears ·n times (and n¸106), then there are n/(1000 log n) disjoint rainbow spanning trees.Our Result: If each color appears ·n/50 times, then we can efficiently construct n/50 disjoint rainbow spanning trees.

Disjoint rainbow trees via the LLL Sample n/50 random spanning trees,

independently. Hope that they are (a) edge-disjoint, and (b)

rainbow.

The magic of the Lovász Local Lemma... this actually works!

Two types of bad events Eeik: the same edge e appears in two spanning trees

Ti and Tk

Fefi: two edges with same color appear in spanning tree Ti

Dependencies only happen if edges are incident ) P[Eeik], P[Fefi] · n-2, d · n2/3, so LLL applies

Resampling Spanning Trees in Kn

• Let T be a uniformly random spanning tree in Kn

• For edge set A, let EA = { AµT }.

• Dependency Graph: Make EA a neighbor of EB, unless A and B are vertex-disjoint.

• Resampling oracle rA:

If T uniform conditioned on EA, want rA(T ) uniform.

But, should not disturb edges that are vtx-disjoint from A.

• EA = { AµT }.

• Resampling oracle rA(T ):

– If T uniform conditioned on EA, want rA(T ) uniform.

–But, should not disturb edges that are vtx-disjoint from A.A

• Lemma: rA(T ) is uniformly random.

• EA = { AµT }.

• Resampling oracle rA(T ):

– If T uniform conditioned on EA, want rA(T ) uniform.

–But, should not disturb edges that are vtx-disjoint from A.

T

–Contract edges of Tvtx-disjoint from A

• Lemma: rA(T ) is uniformly random.

• EA = { AµT }.

• Resampling oracle rA(T ):

– If T uniform conditioned on EA, want rA(T ) uniform.

–But, should not disturb edges that are vtx-disjoint from A.

–Contract edges of Tvtx-disjoint from A–Delete edges

adjacent to A

• Lemma: rA(T ) is uniformly random.

• EA = { AµT }.

• Resampling oracle rA(T ):

– If T uniform conditioned on EA, want rA(T ) uniform.

–But, should not disturb edges that are vtx-disjoint from A.

–Contract edges of Tvtx-disjoint from A–Delete edges

adjacent to A–Let rA(T ) be a

uniformly random spanning tree in resulting graph.

rA(T )

• Lemma: rA(T ) is uniformly random.

LLL via Resampling Oracles

Given resampling oracles, how to get algorithm for LLL?

MIS ResampleDraw ! from ¹While any bad events occur in !

Pick a maximal independent set J of events that occur

For each j 2 J, let ! Ã ri(!)Output !

LLL via Resampling Oracles

Note: In iteration i, we resample a maximal independent set Ji of events.

In iteration i+1, all violated events are in ¡ +(Ji) = ¡(Ji) [ Ji.

Thus, Ji+1 µ ¡ +(Ji) for all i.

Similar to Moser & Tardos’ parallel algorithm

Analysis

• Def: Seq = {I1,…,It : Is indep, Is;, Ii+1µ¡ +

(Ii) }

• Two Ingredients1. Coupling Lemma:

For any sequence I1,…,It of independent sets,P[ algorithm resamples I1,…,It ] ·

2. Summation Bound: is small if t0 is sufficiently large.

1. Coupling Lemma

• Resampling oracle ri for Ei satisfies If ! has distr ¹ conditioned on Ei, then

ri(!) has distr ¹. Does algorithm’s ! always have

distribution ¹? No: ! is also conditioned on some events

not occurring. Couple Alg with Alg(I1,…,It) which

checks in sequence if events I1,…,It occur and resamples them. P[ Alg(I1,…,It) succeeds ] ·

Draw ! from ¹While any bad events occur in !

Pick a maximal independent set J of events that occur

For each j 2 J, let ! Ã ri(!)

2. Summation Bound• Let pi = P[Ei]. Suppose

pi · (1-²)¢xi ¢ j2¡(i) (1-xj) 8 i

• Key Idea:(LLL) ) pi · (1-²) xi

“Binomial Formula”

by (LLL)

(LLL)

This is ¼ 90% of the proof!

• Our algorithm works under the cluster expansion condition and under Shearer’s condition

• Works for directed graphs if the LLL condition has ² slack

• We don’t know how to parallelize or derandomize our algorithm

–Witness Tree Lemma is demonstrably false in our setting!

Technical Details

Summary

• Algorithmic proof of LLL that works for any probability space and any events, under usual LLL conditions,so long as you can design resampling oracles.

• Efficiency is similar to Moser-Tardos, but slightly worse.

• Proof is similar to Moser-Tardos but actually simpler:no witness trees or branching processes