iterative rounding and relaxation - cornell university...delete an xe variable modify (1) constraint...

Iterative Rounding and Relaxation

James Davis

Department of Computer ScienceRutgers University–Camden

[email protected]

March 11, 2010

James Davis (Rutgers Camden) Iterative Rounding 1 / 58

Iterative Rounding and Relaxation

Ingredients:Linear ProgramTheorem about individual variable values in LP solution

Technique:Solve LPRound some variablesRemove variables, relax constraintsIterate


Brief History

Survivable Network DesignJain (1998)

MBDSTGoemans (2006)Singh and Lau (2007, 2008)Bansal, Khandekar, Nagarajan (2008)


Outline

1 Introduction: Vertex Cover

2 LP Formulation

3 Algorithm

4 AnalysisBounding CostBounding Degrees

5 Main TheoremLaminar Lemma Proof

6 Improvement


Vertex Cover

Input:A graph G = (V ,E)

Non-negative costs on vertices cv

Output:A minimum-cost collection of vertices so that each edge in G isincident on at least one vertex in the collection


Vertex Cover

min∑v∈V

cv xv

xu + xv ≥ 1 ∀e = (u, v)

xv ≥ 0 ∀v ∈ V


Vertex Cover: Main Theorem

Theorem (Nemhauser-Trotter)

In a basic optimal LP solution, each xv ∈ {12 ,1,0}

Simple 2-appx algorithm:Solve the Vertex Cover LPInclude all vertices with xv 6= 0 in our cover


MBDST: Problem Statement

Input:A graph G = (V ,E)

Costs ce ≥ 0 for all e ∈ EA set W ⊆ VDegree bounds bv for all v ∈W

Output:Find a min-cost spanning tree (V ,F ) that doesn’t violate degreebounds.


Example

MST

Cost = 3

MBDST

Cost = 7


Outline


2 LP Formulation

3 Algorithm



6 Improvement


MBDST Properties

Notation:S: any subset of verticesE(S): edges with both endpoints in SF : set of edges in MBDST

Properties:

Spanning: Exactly |V | − 1 edges in F

Acyclic: For |S| ≥ 2, at most |S| − 1 edges of F in E(S)

Degree Bounds: At most bv edges of F incident on v


Integer Program

xe = 1 if e ∈ F and xe = 0 otherwise

min∑e∈E

cexe (Objective)∑e∈E

xe = |V | − 1 (1)∑e∈E(S)

xe ≤ |S| − 1 ∀S ⊂ V , |S| ≥ 2 (2)

∑e∈δ(v)

xe ≤ bv ∀v ∈W (3)

xe ∈ {0,1} ∀e ∈ E


Linear Program

xe = 1 if e ∈ F and xe = 0 otherwise

min∑e∈E


xe = |V | − 1 (1)∑e∈E(S)

xe ≤ |S| − 1 ∀S ⊂ V , |S| ≥ 2 (2)

∑e∈δ(v)

xe ≤ bv ∀v ∈W (3)

xe ≥ 0 ∀e ∈ E


LP Properties

There are exponentially many constraints (2)

Ellipsoid method

Separation oracleI (1) and (3) are easy to checkI (2) requires work

Skip Oracle


Separation Oracle: Flow Network

0

1

1

1

0



0

1

1

1

0

s

t



0

1/2

3/2

1

1

1

1/2

1/2

0

s

t



0

1/2

3/2

1/2

1\2

1/2

1/2

1/2

0

s

t



0

1/2

3/2

1/2

1\2

1/2

1/2

1/2

0

s

t

11 1

1


Separation Oracle:s-t cut

Capacity = 2 + 12 · 2 + 1

2 · 4


Separation Oracle

The capacity across S is |V |+ (|S| − 1)−∑

e∈E(S)

xe

The capacity across S is at least |V | iff∑

e∈E(S)

xe ≤ |S| − 1

The max-flow from s to t is |V | iff (2) are satisfied


Outline


2 LP Formulation

3 Algorithm



6 Improvement


Main Theorem

x̄ =< x1, x2, . . . , x|E | >: solution to LPSupport(x̄): set of edges s.t. xe > 0

TheoremFor any basic solution x̄ to the linear program either:

1 I ∃v with exactly one incident edge e ∈ Support(x̄)I xe = 1

2 ∃v ∈W with at most 3 edges of Support(x̄) incident on v

Condition 1 identifies a leaf in the treeCondition 2 identifies a vertex with sufficiently small number ofnonzero incident edges


Algorithm

F = ∅While |V | > 1

x̄ ← LP solution on < G,W >

Remove all edges e with xe = 0If condition 1 is satisfied by x̄

I Add (u, v) to FI Remove v and (u, v) from GI If u ∈W reduce bu by 1

If condition 2 is satisfied by x̄I Remove v from W


Linear Program

min∑e∈E


xe = |V | − 1 (1)∑e∈E(S)

xe ≤ |S| − 1 ∀S ⊂ V , |S| ≥ 2 (2)

∑e∈δ(v)

xe ≤ bv ∀v ∈W (3)

xe ≥ 0 ∀e ∈ E


LP Relationships

In each iteration LP is in the same familySame separation oracleThe Main Theorem applies to each LP

If condition 1 is satisfied LP is incrementally modified:Delete an xe variableModify (1) constraintRemove some (2) constraintsModify some (3) constraints

If condition 2 is satisfied LP is incrementally modified:Remove a (3) constraint


Outline


2 LP Formulation

3 Algorithm



6 Improvement


Bounding Cost

TheoremThe tree returned by our algorithm has cost at most LP OPT

e

G'

v1

LP: current lin. prog.LP′: new lin. prog.F ′: MBDST in G′

IH: cost(F ′) ≤ LP′(G′)

cost(F ′) + ce ≤ LP′(G′) + cexe

≤ LP(G′) + cexe

= LP(G)


Bounding Cost

LemmaLP(G′) is a feasible solution to LP′(G′)

Changes:

1 −1 on RHS, −1 on LHS2 Remove constraints3 −1 on RHS, −1 on LHS;

Remove constraints

∑e∈E

xe = |V | − 1 (1)∑e∈E(S)

xe ≤ |S| − 1 (2)

∑e∈δ(v)

xe ≤ bv (3)


Min-Cost Spanning Trees

Recap:Spanning tree has optimal costDegree bounds?

Implications:





Min-Cost Spanning Trees

Recap:Spanning tree has optimal costDegree bounds?

Implications:




When W = ∅ we have OPT


Outline


2 LP Formulation

3 Algorithm



6 Improvement


Degree Bounds

vu 1

bv ,bu ≥ 1bv never violatedbu “adjusted”

Algorithmx̄ ← LP solution on < G,W >Remove e /∈ Support(x̄)If condition 1 is satisfied by x̄

Add (u, v) to F

Remove v and (u, v) from G

If u ∈ W reduce bu by 1

If condition 2 is satisfied by x̄

Remove v from W


Degree Bounds

bv ≥ 1All 3 edges may be in Fbv violated by at most 2

Algorithmx̄ ← LP solution on < G,W >Remove e /∈ Support(x̄)If condition 1 is satisfied by x̄

Add (u, v) to F

Remove v and (u, v) from G

If u ∈ W reduce bu by 1

If condition 2 is satisfied by x̄

Remove v from W


Analysis Summary

Cost:Spanning tree has optimal cost

Degree Bounds:No degree bound violated by more than 2

Theorem (Goemans)The algorithm for MBDST produces a spanning tree in which thedegree of v is at most bv + 2 for v ∈W and has cost no greater thanOPT


Outline


2 LP Formulation

3 Algorithm



6 Improvement


Main Theorem





Linear Program

min∑e∈E


xe = |V | − 1 (1)∑e∈E(S)

xe ≤ |S| − 1 ∀S ⊂ V , |S| ≥ 2 (2)

∑e∈δ(v)

xe ≤ bv ∀v ∈W (3)

xe ≥ 0 ∀e ∈ E


Laminar Lemma

LemmaFor any basic LP solution x̄ there is a Z ⊆W and a collection L ofS ⊆ V where:

1 ∀S ∈ L, S is tight; ∀v ∈ Z, v is tight2 The vectors χE(S) and χδ(v) are independent3 |L|+ |Z | = |Support(x̄)|4 L is laminar


Characteristic Vector

χE(S) =< 1,1,1,0,0,0,0 > χδ(v) =< 0,1,1,0,1,0,0 >


Laminar Sets

Intersecting Sets

AB

A ∩ B 6= ∅A− B 6= ∅B − A 6= ∅

Laminar Sets

No intersecting sets


Laminar Lemma




Property of L

LemmaIf all S ∈ L contain at least 2 vertices then |L| ≤ |V | − 1

Use induction on |V |Base case: |V | = 2Induction step

I Shrink smallest set to vertexI Generates L′ and V ′

I L′ is laminarI |L′| = |L| − 1I |V ′| ≤ |V | − 1I |L′| ≤ |V ′| − 1I |L| ≤ |V | − 1


Property of Support(x̄)

Lemma|Support(x̄)| < |V |+ |W |

Recall property 3 of Laminar Lemma: |L|+ |Z | = |Support(x̄)|

|Support(x̄)| = |L|+ |Z |≤ |L|+ |W |< |V |+ |W | (Previous Lemma)


From Laminar Lemma to Main Theorem




Suppose Main Theorem wasn’t true:For every v ∈ V there are at least 2 edges incident on itFor every v ∈W there are at least 4 edges incident on it

|Support(x̄)| ≥ 12

(2(|V | − |W |) + 4(|W |))

= |V |+ |W | (Contradiction!)






∑e∈E(S) xe ≤ |V | − 2∑e∈E xe = |V | − 1∑e∈δ(v) xe ≥ 1

xe ≥ 1xe ≤ 1xe = 1






S=V-v

v∑

e∈E(S) xe ≤ |V | − 2∑e∈E xe = |V | − 1∑e∈δ(v) xe ≥ 1

xe ≥ 1xe ≤ 1xe = 1






S={u,v}u v

∑e∈E(S) xe ≤ |V | − 2∑e∈E xe = |V | − 1∑e∈δ(v) xe ≥ 1

xe ≥ 1xe ≤ 1xe = 1


Outline


2 LP Formulation

3 Algorithm



6 Improvement


Laminar Lemma




LP Background

min∑

cixi (Objective)

a11x1 + a12x2 + ...+ a1nxn ≥ b1 (1)a21x1 + a22x2 + ...+ a2nxn ≥ b2 (2)

... = ...

am1x1 + am2x2 + ...+ amnxn ≥ bm (m)xi ≥ 0 (Non-Negative)


LP Background

Constraints definehalf-spacesObjective is a hyperplaneSolution always a corner≥ n tight constraintsConstraints lin. ind.

Linear Program

min∑

ci xi

a11x1 + a12x2 + ...+ a1nxn ≥ b1 (1)

a21x1 + a22x2 + ...+ a2nxn ≥ b2 (2)

... = ...

am1x1 + am2x2 + ...+ amnxn ≥ bm (m)

xi ≥ 0


LP Background

Constraints definehalf-spacesObjective is a hyperplaneSolution always a corner≥ |E | tight constraintsConstraints lin. ind.

MBDST LP

min∑e∈E

cexe

∑e∈E

xe = |V | − 1 (1)

∑e∈E(S)

xe ≤ |S| − 1 (2)

∑e∈δ(v)

xe ≤ bv (3)

xe ≥ 0


Laminar Lemma




Laminar Lemma Proof

Lemma∑e∈E(S) xe is supermodular∑

e∈E(S)

xe +∑

e∈E(T )

xe ≤∑

e∈E(S∩T )

xe +∑

e∈E(S∪T )

xe

S T


Laminar Lemma Proof

LemmaS,T are tight, S and T cross, then S ∩ T ,S ∪ T are tight and

χE(S) + χE(T ) = χE(S∩T ) + χE(S∪T )

(|S| − 1) + (|T | − 1) = (|S ∩ T | − 1) + (|S ∪ T | − 1)

≥∑

E(S∩T )

xe +∑

E(S∪T )

xe (feasibility)

≥∑E(S)

xe +∑E(T )

xe (supermodularity)

Since S and T are tight, these are all equalities


Laminar Lemma Proof: Finding L

Lemma∃L that is laminar and Span(T ) ⊆ Span(L), where T contains all tightsets

Let L be a maximal laminar collection of TRecall that χE(S) + χE(T ) = χE(S∩T ) + χE(S∪T )

S T

Span(T ) Span(L)

S (least int. in L) T (int. S)S ∪ T and S ∩ T S ∪ T and S ∩ T

S ∩ T S ∪ TS ∪ T S ∩ T


Laminar Lemma Proof: Finding L

Lemma∃L that is laminar and Span(T ) ⊆ Span(L), where T contains all tightsets

Let L be a maximal laminar collection of TRecall that χE(S) + χE(T ) = χE(S∩T ) + χE(S∪T )

T

Span(T ) Span(L)

S (least int. in L) T (int. S)S ∪ T and S ∩ T S ∪ T and S ∩ T

S ∩ T S ∪ TS ∪ T S ∩ T


Laminar Lemma Proof: Finding Z



(T ,Y ) spans R|Support(x̄)|

(L,Y ) spans R|Support(x̄)|

To obtain (L,Z ) remove v ∈ Y that are dependent


Recap

LP formulationMain TheoremAlgorithmCost no more than OPTDegree bounds violated by at most 2Main Theorem ProofLaminar Lemma Proof


Outline


2 LP Formulation

3 Algorithm



6 Improvement


Improved Main Theorem

TheoremIf x̄ is a basic solution to LP where W 6= ∅ then there is a v, s.t.

|δ(v) ∩ Support(x̄)| ≤ bv + 1


Algorithm

Phase 1:While W 6= ∅

x̄ ← LP solution on < G,W >

For all xe = 0, remove e from ERemove v from W if there are at most bv + 1 edges of δ(v) inSupport(x̄)

Phase 2:Run algorithm on < G′, ∅ >


Analysis

Theorem (Singh and Lau)The improved algorithm for MBDST produces a spanning tree in whichthe degree of v is at most bv + 1 for v ∈W and has cost no greaterthan OPT


References

Kamal Jain. A factor 2 approximation algorithm for the generalizedSteiner network problem. Combinatorica, 21:39-60, 2001.

Michel X. Goemans. Minimum bounded-degree spanning trees. FOCS’06

Mohit Singh and Lap Chi Lau. Approximating minimum boundeddegree spanning trees to within one of optimal. STOC ’07.


Acknowledgements

Thanks to David Shmoys and David Williamson for letting us use themanuscript of their forthcoming book, “The Design of ApproximationAlgorithms.”


Thank You!

Return to Oracle


iterative rounding and relaxation - cornell university...delete an xe variable modify (1) constraint...

Documents