Graph Partitioning using Single Commodity Flows

Rohit KhandekarUC Berkeley

Joint work with Satish Rao and Umesh Vazirani

Graph Partitioning

Graph Partitioning

Outline

• The sparsest cut problem

• Previous work

• Our algorithm and outline of analysis

• Open questions

The Sparsest Cut Problem

Given a graph G=(V,E)

ST=V \ S

Find a cut that minimizes the ratio of the number of edges across and the size of the smaller side

Minimize (S,T)

|E(S,T)|

min |S|,|T|

Sparsity ф(S)

The Sparsest Cut Problem

• Fundamental NP-hard combinatorial problem

• Central objects of study in theory of Markov chains, geometric embeddings

• Algorithmic primitive in clustering, divide-conquer, packet routing in distributed networks, etc.

The Sparsest Cut Problem

• Related to the conductance

• 0 ≤ conductance ≤ 1• If degree is bounded, sparsity ≈ conductance

|E(S,T)|

min ∑vεSd(v), ∑vεT d(v)

Previous workThree approaches in theory

Spectral Method based

[Alon-Milman’85]

Sparsity = √ ф

Multi-Commodity Flow based

[Leighton-Rao’88]

Sparsity = O( ф · log n

)

Semi-definite Programming based

[Arora-Rao-Vazirani’04]

Sparsity = O( ф · √log n

)

Previous work

In practice, most successful graph partitioning heuristics use eigenvector based approach, multi-level clustering, max-flows

• Chaco [Hendrickson-Leland’94]• METIS [Karypis-Kumar’98]• Eigenvector + single commodity max-

flow [Lang’04]

Multi-commodity Flows

Send flow between multiple source-sink pairs simultaneously

Examples:• Leighton-Rao send flow between every pair

of vertices (embed complete graph)• Arora-Rao-Vazirani generalize this approach

to embedding an expander graph

Computing multi-commodity flows (currently) takes Ω(n2) time (n = # vertices) even approximately

Question

Can we get good approximations using a few single commodity flow computations?

Answer: YES

There exists an algorithm that finds a O(log2 n) approximation using O(log2 n) single commodity max-flow computations.

It runs in time O*(n3/2).

Approximation vs. Running time

log n√log n log2 n “quadratic”

n2

n3/2

n

LRARVAHK

AM,LS,ST

this

Approximation ratio

Ru

nn

ing

tim

e

Expanders

We call a weighted graph H=(V,F,w) an “α-expander” if sparsity of any cut is at least α.

S T = V \ S

w(S,T)

min |S|,|T|ф(S) = ≥ α

Embedding a Graph into another

G=(V,E) H=(V,F,w)

Embedding a Graph into another

G=(V,E) H=(V,F,w)

we

If an α-expander can be embedded in G, then G is also an α-expander.

f1

f2

f3f1 + f2 + f3 = we

Route such a flow for each edge e ε H without violating (unit) edge-capacities in G.

Main Theorem

Given a graph G=(V,E) on n vertices and α ≤ 1, there exists an algorithm that

• either outputs a cut of sparsity at most α,

• or proves that every cut has sparsity at least . log2 n

The algorithm does O(log2 n) single commodity max-flow computations and runs in time O*(n3/2).

embeds a (α/log2 n)-expander in G

α

Algorithm

Assume α = 1

Output a cut (S,T=V \ S) such that|E(S,T)| ≤ min |S|,|T|

OR

Embed a (1/log2 n)-expander in G

Algorithm tries to do this

We vs. Adversary

G=(V,E) H=(V,F,w)

n/2n/2

We vs. Adversary

G=(V,E) H=(V,F,w)

We vs. Adversary

G=(V,E) H=(V,F,w)

We vs. Adversary

G=(V,E) H=(V,F,w)

We vs. Adversary

G=(V,E) H=(V,F,w)

We vs. Adversary

G=(V,E) H=(V,F,w)G=(V,E) H=(V,F,w)

If we output a cut …

k

Cut-size = n/2 – k + l +|E(S,T)| < n/2

T

S

Assume |S| ≤ |T|

l

Therefore,

|E(S,T)| < k – l ≤ |S| = min |S|,|T|

On the other hand …

Lemma:

After O(log2 n) iterations,H becomes an Ω(1)-expander.

Proof:

Later.

Lemma implies Main Theorem

• H is a “sum” of O(log2 n) matchings.

• Each matching is routable in G.

• Therefore H/O(log2 n) is routable in G.

• Since H is an Ω(1)-expander, H/O(log2 n) is an Ω(1/log2 n)-expander.

How to prove that H becomes an expander?

A graph is an expander if and only if

the random walk from every vertex mixes rapidly.

This is NOT an expander.

“Simulating” Random Walks

1

1/2

1/2

1/4

1/4

1/4

1/4

1/8

1/81/4

1/4

“Simulating” Random Walks

1/8

1/8

H is an expander if all such distributions become uniform …

How does adversary find a cut in H

Adversary would like to find a balanced cut across which very small amount of probability has crossed over.

How does adversary find a cut in H

How does adversary find a cut in H

= +1 charge

= –1 charge

Mix the charge along the matchings …

Random assignment of charge

How does adversary find a cut in H

Order the vertices according to the final charge presentand cut in half.

n/2 n/2

Analysis

For a vertex v, let Pv be the vector of probabilities present at v from all the n walks.

Initially, Pv = (0,…,0,1,0,…,0) where 1 is at co-ordinate v.

If we add an edge (u,v), we update these vectors as

Pu := Pv :=

Pu + Pv

Outline of the Analysis

2

Outline of the Analysis

We prove that after O(log2 n) iterations,Pv ≈ π = (1/n,1/n,…,1/n) for all v.

This, in turn, implies that after O(log2 n) iterations, the graph H becomes an Ω(1)-expander.

Potential function

Ψ = ∑v |Pv – π|2

Initial potential = ∑v |(0,…,1,…,0) – π|2 = n – 1

If (u,v) is matched, reduction in potential of u and v is

|Pu – π|2 + |Pv – π|2 – 2 |(Pu+Pv)/2 – π|2

Potential function

If (u,v) is matched, reduction in potential of u and v is

|Pu – π|2 + |Pv – π|2 – 2 |(Pu+Pv)/2 – π|2

= ½ |Pu – Pv|2

Pu – π Pv – π

Therefore to reduce the potential fast, we should match u and v if |Pu – Pv| is large.

Random Projections

Random Projections

Pv – π

n/2 n/2

Random Projections≈

Mixing Random Charges

Taking projections on a random vector r = (r1, r2, …, rn)

is equivalent to

Mixing the initial charges r1, r2, …, rn

Random Projections≈

Mixing Random Charges

“Probability spread matrix” P =

Projections on r are given by P · r

Note that P = Mt · Mt-1 · … · M1 · I

Thus P · r = Mt ( Mt-1 ( … (M1 · r)) … )

P1

P2

…Pn

Potential function

The projected lengths are “faithful” to the actual lengths within a factor of log n.

Therefore we can argue that the potential decreases by a factor of (1 – 1/log n) in each iteration.

Thus after O(log2 n) iterations, the potential becomes negligible; and the random walks mix.

Running time

• Number of iterations = O(log2 n)• Each iteration = 1 max-flow

= O*(m3/2)

• [Benczur-Karger’96] In O*(m) time, we can transform any graph G on n vertices into G’ on same vertices:– G’ has O(n log (n)/ε2) edges– All cuts in G’ have size within (1 ± ε) of those in G

• Overall running time = O*(m + n3/2)

Remarks

Finally, when all random walks mix,

P =

In fact, P can be routed in G. Thus we in fact embed a complete graph.

1/n … 1/n1/n … 1/n …1/n … 1/n

Extensions to Balanced Separator

• Balanced Separator:Partition V into S and T = V \ S such that– |S|, |T| ≥ n/3, and– |E(S,T)| is minimized.

• The techniques can be extended to yield O(log2 n) approximation for this problem in similar running times.

Open Questions

Improve approximation ratio and/or running time.

log n√log n polylog n quadratic

n2

n3/2

n

LRARVAHK

AM,LS,STthis

Approximation ratio

Ru

nnin

g t

ime n3

?

?

Open Questions

Our algorithm can be thought of as a “primal”- “dual” algorithm.

Is there a more general framework?

Can we extend this technique to other problems?

Thank You

Graph Partitioning