approximations for isoperimetric and spectral profile and related parameters

Approximations for Isoperimetric and

Spectral Profile and Related Parameters

Prasad RaghavendraMSR New England

S

David SteurerPrinceton University

Prasad TetaliGeorgia Tech

joint work with

Graph Expansion

d-regular graph Gwith n vertices

d

expansion(S) = # edges leaving S

d |S|

vertex set S

A random neighbor of a random vertex in S is outside of S with probability expansion(S)

ФG = expansion(S)minimum|S| ≤ n/2

Conductance of Graph G

Uniform Sparsest Cut Problem Given a graph G compute ФG and find the set S achieving it.

Extremely well-studied, many different contexts pseudo-randomness, group theory, online routing, Markov chains, metric

embeddings, …

S

Measuring Graph Expansion

d-regular graph Gwith n vertices

d


d |S|

vertex set S ФG = expansion(S)minimum|S| ≤ n/2


CompleteGraph

CompleteGraph

Path Complete graphs with a perfect matching

Typically, small sets expand to a greater extent.

Isoperimetric Profile

Ф(δ) = expansion(S)minimum|S| ≤ δn

Isoperimetric Profile of Graph G


d |S|

• Conductance function – defined by [Lovasz-Kannan] used to obtain better mixing time bounds for Markov chains.

•Decreasing function of δSet Size δ 0.5

Ф(δ)

1

S

Approximating Isoperimetric Profile

Ф(δ) = expansion(S)minimum|S| ≤ δn

Isoperimetric Profile of Graph G


d |S|

S

Uniform Sparsest Cut: Determine the lowest point on the curve.

Set Size δ

0.5

Ф(δ)

1

What is the value of Ф(δ) for a given graph G and a constant δ > 0? Gap Small Set Expansion Problem (GapSSE)(η, δ)Given a graph G and constants δ, η > 0,

Is Ф(δ) < η OR Ф(δ) > 1- η?----Closely tied to Unique Games Conjecture

(Last talk of the day “Graph Expansion and the Unique Games Conjecture” [R-Steurer 10])

AlgorithmTheorem

For every constant δ > 0, there exists a poly-time algorithm that finds a set S of size O(δ) such that

expansion (S) ≤

))/1log()(( O

-- A (Ф(δ) vs ) approximation

)/1log()(

• For small enough δ, the algorithm cannot distinguish betweenФ(δ) < η OR Ф(δ) > 1- η

Theorem [R-Steurer-Tulsiani 10]An improvement over above algorithm by better than constant

factor, would help distinguish Ф(δ) < η OR Ф(δ) > 1- η

A Spectral RelaxationLet x = (x1 , x2 , .. xn) be the indicator function of the unknown least expanding set S

S

SC

1

0

Eji

ji xx),(

2)(Number of edges leaving S =

i

ix2|S| =

xxdLxx

xd

xx

T

T

x

ii

Ejiji

x nn

}1,0{2),(

2

}1,0{min

)(min

|Support(x)| < δn

)(nx nx

Relaxing 0,1 to real numbers

Why is it spectral?Spectral Profile: [Goel-Montenegro-Tetali]

xxdLxx

xd

xx

T

T

x

ii

Ejiji

x nn

min)(

min)( 2),(

2

xxdLxx

xd

xx

T

T

xx

ii

Ejiji

xx nn

1

2),(

2

1

min)(

min

Smallest Eigen Value of Laplacian:

|Support(x)| < δn |Support(x)| < δn

Observation: Λ(δ) is the smallest possible eigenvalue of a submatrix L(S,S) of size at most < δn of the Laplacian L .

Rounding EigenvectorsCheeger’s inequality There is a sparse cut of value at mostxxd

LxxT

T

xx n

1

min

Smallest Eigen Value of Laplacian

2

Rounding x

xxdLxx

xd

xx

T

T

x

ii

Ejiji

x nn

min)(

min)( 2),(

2


Lemma: There exists a set S of volume at most δ whose expansion is at most )(2

0

Spectral Profile [Goel-Montenegro-Tetali]

xxdLxx

xd

xx

T

T

x

ii

Ejiji

x nn

min)(

min)( 2),(

2


Unlike eigen values, Λ(δ) is not the optimum of a convex program. We show an efficiently computable SDP that gives good guarantee.

Theorem: There exists an efficiently computable SDP relaxation Λ* (δ) of Λ(δ) such that for every graph G

Λ* (δ) ≤ Λ(δ) ≤ Λ* (δ/2) O(log 1/∙ δ)

RecapTheorem: (Approximating Spectral Profile)There exists an efficiently computable SDP relaxation Λ* (δ) of Λ(δ) such that for every graph G

Λ* (δ) ≤ Λ(δ) ≤ Λ* (δ/2) O(log 1/∙ δ)

Lemma: (Cheeger Style Rounding)There exists a set S of volume at most δ whose expansion is at most )(2

Theorem (Approximating Isoperimetric Profile)For every constant δ > 0, there exists a poly-time algorithm that finds a set S of size O(δ) such that

expansion (S) ≤

))/1log()(( O

Restricted Eigenvalue Problem

Given a matrix A, find a submatrix A[S,S] of size at most δn X δn matrix with the least eigenvalue.

-- Our algorithm is applicable to diagonally dominant matrices

(yields a log(1/δ) approximation).

Approximating Spectral Profile

ii

Ejiji

x xd

xx

n 2),(

2)(min)(

|Support(x)| < δn

SDP Relaxation for

ii

ii xxSupportx 22

|)(|

222

|| nvnvi

ii

i

ii

Ejiji

x xd

xx

n 2),(

2)(min)(

|Support(x)| < δn

Replace each xi by a vector vi:

Eji

ji vvNumerator),(

2||

Ejiiv

),(

2||Denominator = n

Without loss of generality, xi can be assumed positive.This yields the constraint: vi v∙ j ≥ 0

Enforcing Sparsity:

By Cauchy-Schwartz inequality:

SDP Relaxation for

2

,

nvvji

ji

ii

Ejiji

x xd

xx

n 2),(

2)(min)(

|Support(x)| < δn

Minimize (Sum of Squared Edge Lengths)

Subject to

Eji

ji vv),(

2||

nvEjii

),(

2||

Positive Inner Products: vi v∙ j ≥ 0

Average squared length =1

Average Pairwise Correlation < δ

Rounding

Two Phase Rounding:

• Transform SDP vectors in to a SDP solution with only non-negative coordinates.

• Use thresholding to convert non-negative vectors in to sparse vectors.

Making SDP solution nonnegativeLet v be a n-dimensional real vector.Let v* denote the unit vector along direction v.

Map the vector v to the following function over Rn :fv = ||v|| (Square Root of Probability Density Function∙

of n-dimensional Gaussian centered at v*)

)()( *vxvxfv Formally,

Where: Ф(x) = probability density function of a

mean 0, variance σ spherical Gaussian in n-dimensions.

Properties

24/1

2

),(

)(||2

nefEjii

)()( *vxvxfv Where:

Ф(x) = probability density function of a mean 0, variance σ spherical Gaussian in n-dimensions.

Lemma: (Pairwise correlation remains low if σ is small)

2

,

nvvji

ji Average Pairwise Correlation < δSDP Constraint

Pick σ = 1/sqrt{log(1/δ)}

Properties

Lemma: (Squared Distances get stretched by at most 1/σ2) |f1 – f2 |2 ≤ O(1/σ2) |v1 – v2 |2

)()( *vxvxfv Where:

Ф(x) = probability density function of a mean 0, variance σ spherical Gaussian in n-dimensions.

For our choice of σ, squared distances are stretched by log(1/δ) .

With a log(1/δ) factor loss, we obtaining a non-negative SDP solution.

Rounding a positive vector solutionLet us pretend the vectors vi are non-negative. i.e., vi(t) ≥ 0 for all t

Rounding Non-Negative Vectors1. Sample t2. Compute threshold

θ = (average of vi(t) over i) * (2/ δ )3. Set xi = max{ vi(t) – θ, 0 } for all i

Observation: |Support(x)| < δ/2 n∙

Observation:

Eji

jiji vvxxE),(

22 ])([

Open Problem

Find integrality gaps for the SDP relaxation:

Current best have δ = 1/poly(logn),

Do there exist integrality gaps with δ = 1/poly(n)?

2

,

nvvji

ji

Minimize (Sum of Squared Edge Lengths)

Subject to

Eji

ji vv),(

2||

nvEjii

),(

2||

Positive Inner Products: vi v∙ j ≥ 0

Average squared length =1

Average Pairwise Correlation < δ

Thank You

Spectral ProfileSecond Eigen Value:

xxdLxx

xd

xx

T

T

x

ii

Ejiji

x nn

min)(

min 2),(

2

|Support(x)| < δn

)(

Spectral Profile

Remarks:• Λ(δ) ≤ minimum of Ф(δ0) over all δ0 ≤ δ• Unlike second eigen value, Λ(δ) is not the optimum of a convex program.• Λ(δ) is the smallest possible eigenvalue of a submatrix of size at most < δn of the matrix L • xi can be assumed to be all positive.

Small Sets via Spectral Profile

Using an analysis along the lines of analysis of Cheeger’s inequality, it yields:

Theorem [Raghavendra-Steurer-Tetali 10]There exists a poly-time algorithm that finds a set S of

size O(δ) such thatexpansion (S) ≤ sqrt (Λ* (δ) O(log 1/∙ δ))

So if there is a set S with expansion(S) = ε, thenthe algorithm finds a cut of size )/1log( O

Similar behaviour as the Gaussian expansion profile

Spectral ProfileSecond Eigen Value:

xxdLxx

xd

xx

T

T

x

ii

Ejiji

x nn

min)(

min 2),(

2

|Support(x)| < δn

)(

Spectral Profile

Replace xi by vectors vi

Subject tovi v∙ j ≥ 0

i

ii

i vnv 22

||

ii

Ejiji

v vd

vv

i2

),(

2

*

||

||min)(

Graph Expansion

d-regular graph G

d


d |S|

vertex set S

A random neighbor of a random vertex in S is outside of S with probability expansion(S)

ФG = expansion(S)minimum|S| ≤ n/2


Uniform Sparsest Cut Problem Given a graph G compute ФG and find the set S achieving it.

Approximation Algorithms:

•Cheeger’s Inequality [Alon][Alon-Milman] Given a graph G, if the normalized adjacency matrix has second eigen value λ2 then,

•A log n approximation algorithm [Leighton-Rao].•A sqrt(log n) approximation algorithm using semidefinite programming [Arora-Rao-Vazirani].

)1(22)1(

22

G

Extremely well-studied, many different contexts

pseudo-randomness, group theory, online routing,

Markov chains, metric embeddings, …

Limitations of Eigenvalues• The best lower bound that Cheeger’s inequality gives on expansion is

(1-λ2)/2 < ½, while Ф(δ) can be close to 1.

• Consider graph G Connect pairs of points on {0,1}n that are εn Hamming distance away.Then Second eigenvalue ≈ 1- ε and ф(1/2) ≈ εyet Ф(δ) ≈ 1 (small sets have near-perfect expansion)

• A SIMPLE SDP RELAXATION cannot distinguish between- all small sets expand almost completely- exists small set with almost no expansion

A Conjecture

Small-Set Expansion Conjecture:

8η>0, 9 ± >0 such that GapSSE(η, ± ) is NP-hard, i.e.,

Given a graph G, it is NP-hard to distinguish YES: expansion(S) < η for some S of volume ¼ ±NO: expansion(S) > 1- η for all S of volume ¼ ±

Road Map

• Algorithm:– Spectral Profile.

• Reductions within Expansion

• Relationship with Unique Games Conjecture

Gaussian Curve

Gaussian Graph

Vertices: all points in Rd (d dimensional real space)

Edges: Weights according to the following sampling procedure:

• Sample a random Gaussian variable x in Rd

• Perturb x as follows to get y in Rd

Add an edge between x an y

zxy 22)1(

Γε (δ) = Gaussian noise sensitivity of a set of measure δ

= least expanding sets are caps/thresholds of measure δ

= )/1log(

Set Size δ 0.5

Ф(δ)

1

Approximating Expansion Profile

Reductions within Expansion

Reductions within Expansion

Theorem [Raghavendra-Steurer-Tulsiani 10]For every positive integer q, and constants ε,δ,γ, given

a graph it is SSE-hard to distinguish between:

• There exists q disjoint small sets S1 , S2 , .. Sq of size close to 1/q, such that expansion(Si) ≤ ε

• No set of size μ> δ has expansion less than size Γε/2 (μ) -- expansion of set of size μ in Gaussian graph with parameter ε/2

expansion (S) < sqrt (Λ* (δ) O(log 1/∙ μ))

Informal Statement

Set Size δ

1

Set Size δ 0.5

Ф(δ)

1

QualitativeAssumption

GapSSE is NPhard

Quantitative Statement

Given a graph G , it is SSE-hard to distinguish whether, •There is a small set of size whose expansion is ε.

•Every small set of size μ (in a certain range) expands at least as much as the corresponding Gaussian graph with noise ε )/1log( O

Corollaries

Corollary: The algorithm in [Raghavendra-Steurer Tetali 10] has near-optimal guarantee assuming SSE conjecture.

Corollary: Assuming GapSSE, there is no constant factor approximation for Balanced Separator or Uniform Sparsest Cut.

Relation with Unique Games Conjecture

Unique GamesUnique game ¡ :

Each edge (A,B) has an associated map ¼ ¼ is bijection from L(A) to L(B)

graph of size n

AB

label set L(A) ¼

Goal: Find an assignment of labels to vertices such that maximum number of edges are satisfied.

A labelling satisfies (A,B) if ¼ (label of A) = label of B

Referee

sample (A,B,¼)

BA

Player 1 Player 2pick b in L(B)pick a in L(A)

ba

Refereeplayers win if ¼(a) = b

no communication between players

value( ¡ ):maximum success probabilityover all strategies of the players

Unique Games Conjecture [Khot02]

Unique Games Conjecture: [Khot ‘02]8²>0, 9 q >0:

NP-hard to distinguish for ¡ with label set size qYES: value( ¡ ) > 1-² NO: value( ¡ ) < ²

Implications of UGC

UGC

BASIC SDP is optimal for …

Constraint Satisfaction Problems [Raghavendra`08]MAX CUT, MAX 2SAT

Ordering CSPs [GMR`08]MAX ACYCLIC SUBGRAPH, BETWEENESS

Grothendieck Problems [KNS`08, RS`09]

Metric Labeling Problems [MNRS`08]MULTIWAY CUT, 0-EXTENSION

Kernel Clustering Problems [KN`08,10]

Strict Monotone CSPs [KMTV`10]VERTEX COVER, HYPERGRAPH VERTEX COVER

…

If we could show , then a refutation of UGC would imply an improved algorithm for PROBLEM X

“Reverse Reductions”

UGCBASIC SDP is optimal forlots of optimization problems,e.g.: MAX CUT and VERTEX COVER

BASIC SDP optimal

for PROBLEM X

?*

*

Parallel Repetition is natural candidate reduction for [FeigeKinderO’Donnell’07]*

Win-Win Situation

Bad news: this reduction cannot work [Raz’08, BHHRRS’08]

PROBLEM X = MAX CUT

Small Set Expansion and Unique Games

• Solving Unique Games Finding a small non-expanding set in the “label extended graph”

Theorem [Raghavendra-Steurer 10]

Small Set Expansion Conjecture Unique Games Conjecture

Establishes a reverse connection from a natural problem.

Implications of UGC

UGC

BASIC SDP is optimal for …

Constraint Satisfaction Problems [Raghavendra`08]MAX CUT, MAX 2SAT

Ordering CSPs [GMR`08]MAX ACYCLIC SUBGRAPH, BETWEENESS

Grothendieck Problems [KNS`08, RS`09]

Metric Labeling Problems [MNRS`08]MULTIWAY CUT, 0-EXTENSION

Kernel Clustering Problems [KN`08,10]

Strict Monotone CSPs [KMTV`10]VERTEX COVER, HYPERGRAPH VERTEX COVER

…

Uniform Sparsest Cut [KNS`08, RS`09]

Minimum Linear Arrangement[KNS`08, RS`09]

Gap SSE

UGCWith expansion

Most known SDP integrality gap instances for problems like MaxCut, Vertex Cover, Unique games have graphs that are “small set expanders”

Theorem [Raghavendra-Steurer-Tulsiani 10]

Small Set Expansion Conjecture MaxCut or Unique Games on Small Set Expanders is hard.

Reverse Connections?

Approximating Spectral Profile

RoadmapIntroductionGraph Expansion: Cheeger’s Inequality, Leighton Rao, ARVExpansion Profile: Small Sets expand more than large ones.Cheeger’s inequality and SDPs failGapSSE ProblemRelation to Unique Games Conjecture: Unique Games definition, Applications, lack of reverse reductions. Label extended graph Small setsSmall Set Expansion Conjecture UGCUGC with SSE is easyAlgorithm for SSE: Spectral Profile, SDP for Spectral Profile, Rounding algorithmRelations within expansion:GapSSE Balanced Separator Hardness

NP-hard OptimizationExample:

MAX CUT: partition vertices of a graph into two setsso as to maximize number of cut edges

fundamental graph partitioning problem

benchmark for algorithmic techniques

ApproximationMAX CUT

Trivial approximation

First non-trivial approximation

based on a semidefinite relaxation (BASIC SDP)

Random assignment, cut ½ of the edges

Goemans–Williamson algorithm (‘95)

approx-ratio 0.878

Beyond Max Cut:

Analogous BASIC SDP relaxation for many other problems

Almost always, BASIC SDP gives best known approximation(often strictly better than non-SDP methods)






approx-ratio 0.878

Can we beat the approximation guarantee of BASIC SDP?


Unique Games Conjecture

Unique Games Conjecture [Khot’02] (roughly):it is NP-hard to approximate the value of Unique Games

certain optimization problem:given equations of the form

xi – xj = cij mod qsatisfy as many as possible

Can we beat the approximation guarantee of BASIC SDP for Max Cut?

No, assuming Khot’s Unique Games Conjecture! [KKMO`05, MOO`05, OW’08]

UGC: Is it true?

UGC

?

UGCon expanding

instances

UGCon product instances

[AKKSTV’08, AIMS’10]

[BHHRRS’08,Raz’08]

UGCfor certain SDP

hierarchies

[RaghavendraS’09,KhotSaket’09]

Any non-trivial consequences if UGC is false?

What are hard instances?

“Reverse Reductions”


Win-Win SituationA refutation of UGC implies an improved algorithm for SMALL-SET EXPANSION (better than BASIC SDP!)

SMALL-SET EXPANSION

First reduction from natural combinatorial problem to UNIQUE GAMES

BASIC SDP optimal

for PROBLEM X

Approximating Small-Set Expansion

Expansion profile of G at ±:minimum expansion(S) over all S with volume < ±

SHow well can we approximate the

expansion profile of G for small ± ?

BASIC SDP cannot distinguish between- all small sets expand almost completely- exists small set with almost no expansion

Small-Set Expansion Conjecture:8²>0, 9 ± >0:

NP-hard to distinguish YES: expansion(S) < ² for some S of volume ¼ ±NO: expansion(S) > 1-² for all S of volume ¼ ±

Unique 2-Prover Games

Referee

sample (A,B,¼) from DBA

Player 1 Player 2pick b in L(B)pick a in L(A)

ba

Refereeplayers win if ¼(a) = b

Unique game ¡ :

no communication between players


distribution D over triples (A,B,¼)- A and B are from U- ¼ is bijection from L(A) to L(B)

universe of size n

AB

label set L(A)

label set L(B)

¼

Approximating Unique Games

Unique Games Conjecture: [Khot ‘02]

8²>0, 9 q >0:NP-hard to distinguish for ¡ with label set size q

YES: value( ¡ ) > 1-² NO: value( ¡ ) < ²

How well can we approximate the value of a unique gamefor large label sets?

BASIC SDP cannot distinguish between games with value ¼ 1 and ¼ 0for large label sets

Approximating Unique Games

Unique Games Conjecture:8²>0, 9 q >0:

NP-hard to distinguish for ¡ with label set size qYES: value( ¡ ) > 1-² NO: value( ¡ ) < ²

Our main theorem:

Small-Set Expansion Conjecture ) Unique Games Conjecture

Reduction: Small-Set Expansion Unique Games

Task: find non-expanding set of volume ¼ ±

graph G

A

B

Refereesample R = 1/± random edges M

A = one half of each edgeB = other half of each edge

BA

Player 1 Player 2pick b 2 Bpick a 2 A

ba

Refereeplayers win if (a,b) 2 M

P( ) < ²

CompletenessSmall Non-Expanding Set (Partial) Strategy

graph G

AB

S

Suppose expansion(S) < ²

Strategy for Player 1:pick a 2 A if a is unique intersection with Sotherwise, refuse to answer

With constant probability, |A Å S |= {a}Conditioned on this event:

other half of a’s edge outside of S

partial game value > 1 - ²

referee allows players to refuse for few queries

SoundnessStrategy Small Non-Expanding Set

standard trick: can assume both players have same strategy

Suppose:players win with prob > 1-²

Idea: strategy distribution over sets- sample R-1 vertices U- output S = { x | Player 1 picks x if A=U+x }

Easy to show:

E volume(S) = 1/R = ±E # edges leaving S

E d |S|< ²

Problem:volume(S) might not be concentrated around ±

Are we done?No!

SoundnessStrategy Small Non-Expanding Set

Idea: Referee adds ²-noise to A and B+ ²-noise+ ²-noise

New distribution over sets:- sample R-1 vertices U- S = { x | players pick x if A = U+x+noise

with probability > ½ }

Intuition:players cannot distinguish x and noise

Can show: 8 U. volume(S) <

= ± / ²

² R random vertices

# noise vertices1

Suppose:players win with prob > 1-²

Summary


BASIC SDP optimal for SMALL-SET

EXPANSION

more reverse reductions?

Open Questions (+ Subsequent Work)

Hardness results based on Small-Set Expansion Conjecture? Reductions between Expansion Problems [Raghavendra S Tulsiani’10]

Thanks!Questions?

noise here vs. noise in other hardness reductions?

Better algorithms for UNIQUE GAMES via SMALL-SET EXPANSION? 2npoly(²) algorithm for (1-², ½)-UG via SSE [Arora Barak S’10]

APPENDIX

Rounding a UG strategy to a small non-expanding set

Unique Games

decorated bipartite graph

uv

label set L(v)label set L(u)

bijection ¼uv: L(u) L(v)

Referee

random edge (u,v)

vu

Player 1 Player 2pick j in L(v)pick i in L(u)

ji

Refereeplayers win if ¼uv(i) = j

Unique Games instance ¡ :

each player knows only half of referee’s edge








approx-ratio 0.878

General constraint satisfaction problems (CSPs):

for every CSP: approximation matches integrality gap of certain SDPnear-linear running time [S’10]

simple generic approximation algorithm [RaghavendraS’09]

Can we beat the approximation guarantee of BASIC SDP (for Max Cut)?

Combinatorial OptimizationExample:

MAX CUT: partition vertices of a graph into two setsso as to maximize number of cut edges

CAIDA at UCSD analyzes business relationshipsamong Autonomous Systems in the internet using MAX 2SAT

¼ MAX CUTA concrete practical application:

approximations for isoperimetric and spectral profile and related parameters

Documents