approximations for isoperimetric and spectral profile and related parameters prasad raghavendra msr...
TRANSCRIPT
Approximations for Isoperimetric and
Spectral Profile and Related Parameters
Prasad RaghavendraMSR New England
S
David SteurerPrinceton University
Prasad TetaliGeorgia Tech
joint work with
Graph Expansion
d-regular graph Gwith n vertices
d
expansion(S) = # edges leaving S
d |S|
vertex set S
A random neighbor of a random vertex in S is outside of S with probability expansion(S)
ФG = expansion(S)minimum|S| ≤ n/2
Conductance of Graph G
Uniform Sparsest Cut Problem Given a graph G compute ФG and find the set S achieving it.
Extremely well-studied, many different contexts pseudo-randomness, group theory, online routing, Markov chains, metric
embeddings, …
S
Measuring Graph Expansion
d-regular graph Gwith n vertices
d
expansion(S) = # edges leaving S
d |S|
vertex set S ФG = expansion(S)minimum|S| ≤ n/2
Conductance of Graph G
CompleteGraph
CompleteGraph
Path Complete graphs with a perfect matching
Typically, small sets expand to a greater extent.
Isoperimetric Profile
Ф(δ) = expansion(S)minimum|S| ≤ δn
Isoperimetric Profile of Graph G
expansion(S) = # edges leaving S
d |S|
• Conductance function – defined by [Lovasz-Kannan] used to obtain better mixing time bounds for Markov chains.
•Decreasing function of δSet Size δ 0.5
Ф(δ)
1
S
Approximating Isoperimetric Profile
Ф(δ) = expansion(S)minimum|S| ≤ δn
Isoperimetric Profile of Graph G
expansion(S) = # edges leaving S
d |S|
S
Uniform Sparsest Cut: Determine the lowest point on the curve.
Set Size δ
0.5
Ф(δ)
1
What is the value of Ф(δ) for a given graph G and a constant δ > 0?
Gap Small Set Expansion Problem (GapSSE)(η, δ)Given a graph G and constants δ, η > 0,
Is Ф(δ) < η OR Ф(δ) > 1- η?----Closely tied to Unique Games Conjecture
(Last talk of the day “Graph Expansion and the Unique Games Conjecture” [R-Steurer 10])
AlgorithmTheorem
For every constant δ > 0, there exists a poly-time algorithm that finds a set S of size O(δ) such that
expansion (S) ≤
))/1log()(( O
-- A (Ф(δ) vs ) approximation
)/1log()(
• For small enough δ, the algorithm cannot distinguish betweenФ(δ) < η OR Ф(δ) > 1- η
Theorem [R-Steurer-Tulsiani 10]An improvement over above algorithm by better than constant
factor, would help distinguish Ф(δ) < η OR Ф(δ) > 1- η
A Spectral RelaxationLet x = (x1 , x2 , .. xn) be the indicator function of the unknown least expanding set S
S
SC
1
0
Eji
ji xx),(
2)(Number of edges leaving S =
i
ix2|S| =
xxd
Lxx
xd
xx
T
T
x
ii
Ejiji
x nn
}1,0{2
),(
2
}1,0{min
)(
min
|Support(x)| < δn
)(nx nx
Relaxing 0,1 to real numbers
Why is it spectral?Spectral Profile: [Goel-Montenegro-Tetali]
xxd
Lxx
xd
xx
T
T
x
ii
Ejiji
x nn
min
)(
min)( 2
),(
2
xxd
Lxx
xd
xx
T
T
xx
ii
Ejiji
xx nn
1
2
),(
2
1
min
)(
min
Smallest Eigen Value of Laplacian:
|Support(x)| < δn |Support(x)| < δn
Observation: Λ(δ) is the smallest possible eigenvalue of a submatrix L(S,S) of size at most < δn of the Laplacian L .
Rounding EigenvectorsCheeger’s inequality There is a sparse cut of value at mostxxd
LxxT
T
xx n
1
min
Smallest Eigen Value of Laplacian
2
Rounding x
xxd
Lxx
xd
xx
T
T
x
ii
Ejiji
x nn
min
)(
min)( 2
),(
2
|Support(x)| < δn |Support(x)| < δn
Lemma: There exists a set S of volume at most δ whose expansion is at most )(2
0
Spectral Profile [Goel-Montenegro-Tetali]
xxd
Lxx
xd
xx
T
T
x
ii
Ejiji
x nn
min
)(
min)( 2
),(
2
|Support(x)| < δn |Support(x)| < δn
Unlike eigen values, Λ(δ) is not the optimum of a convex program. We show an efficiently computable SDP that gives good guarantee.
Theorem: There exists an efficiently computable SDP relaxation Λ* (δ) of Λ(δ) such that for every graph G
Λ* (δ) ≤ Λ(δ) ≤ Λ* (δ/2) O(log 1/∙ δ)
RecapTheorem: (Approximating Spectral Profile)There exists an efficiently computable SDP relaxation Λ* (δ) of Λ(δ) such that for every graph G
Λ* (δ) ≤ Λ(δ) ≤ Λ* (δ/2) O(log 1/∙ δ)
Lemma: (Cheeger Style Rounding)There exists a set S of volume at most δ whose expansion is at most )(2
Theorem (Approximating Isoperimetric Profile)For every constant δ > 0, there exists a poly-time algorithm that finds a set S of size O(δ) such that
expansion (S) ≤
))/1log()(( O
Restricted Eigenvalue Problem
Given a matrix A, find a submatrix A[S,S] of size at most δn X δn matrix with the least eigenvalue.
-- Our algorithm is applicable to diagonally dominant matrices
(yields a log(1/δ) approximation).
Approximating Spectral Profile
ii
Ejiji
x xd
xx
n 2
),(
2)(
min)(|Support(x)| < δn
SDP Relaxation for
ii
ii xxSupportx 22
|)(|
22
2
|| nvnvi
ii
i
ii
Ejiji
x xd
xx
n 2
),(
2)(
min)(|Support(x)| < δn
Replace each xi by a vector vi:
Eji
ji vvNumerator),(
2||
Ejiiv
),(
2||Denominator = n
Without loss of generality, xi can be assumed positive.This yields the constraint: vi v∙ j ≥ 0
Enforcing Sparsity:
By Cauchy-Schwartz inequality:
SDP Relaxation for
2
,
nvvji
ji
ii
Ejiji
x xd
xx
n 2
),(
2)(
min)(|Support(x)| < δn
Minimize (Sum of Squared Edge Lengths)
Subject to
Eji
ji vv),(
2||
nvEjii
),(
2||
Positive Inner Products: vi v∙ j ≥ 0
Average squared length =1
Average Pairwise Correlation < δ
Rounding
Two Phase Rounding:
• Transform SDP vectors in to a SDP solution with only non-negative coordinates.
• Use thresholding to convert non-negative vectors in to sparse vectors.
Making SDP solution nonnegative
Let v be a n-dimensional real vector.Let v* denote the unit vector along direction v.
Map the vector v to the following function over Rn :fv = ||v|| (Square Root of Probability Density Function∙
of n-dimensional Gaussian centered at v*)
)()( *vxvxfv Formally,
Where: Ф(x) = probability density function of a
mean 0, variance σ spherical Gaussian in n-dimensions.
Properties
24/1
2
),(
)(||2
nefEjii
)()( *vxvxfv Where:
Ф(x) = probability density function of a mean 0, variance σ spherical Gaussian in n-dimensions.
Lemma: (Pairwise correlation remains low if σ is small)
2
,
nvvji
ji Average Pairwise Correlation < δSDP Constraint
Pick σ = 1/sqrt{log(1/δ)}
Properties
Lemma: (Squared Distances get stretched by at most 1/σ2) |f1 – f2 |2 ≤ O(1/σ2) |v1 – v2 |2
)()( *vxvxfv Where:
Ф(x) = probability density function of a mean 0, variance σ spherical Gaussian in n-dimensions.
For our choice of σ, squared distances are stretched by log(1/δ) .
With a log(1/δ) factor loss, we obtaining a non-negative SDP solution.
Rounding a positive vector solution
Let us pretend the vectors vi are non-negative.
i.e., vi(t) ≥ 0 for all t
Rounding Non-Negative Vectors1. Sample t2. Compute threshold
θ = (average of vi(t) over i) * (2/ δ )3. Set xi = max{ vi(t) – θ, 0 } for all i
Observation: |Support(x)| < δ/2 n∙
Observation:
Eji
jiji vvxxE),(
22 ])([
Open Problem
Find integrality gaps for the SDP relaxation:
Current best have δ = 1/poly(logn),
Do there exist integrality gaps with δ = 1/poly(n)?
2
,
nvvji
ji
Minimize (Sum of Squared Edge Lengths)
Subject to
Eji
ji vv),(
2||
nvEjii
),(
2||
Positive Inner Products: vi v∙ j ≥ 0
Average squared length =1
Average Pairwise Correlation < δ
Thank You
Spectral Profile
Second Eigen Value:
xxd
Lxx
xd
xx
T
T
x
ii
Ejiji
x nn
min
)(
min 2
),(
2
|Support(x)| < δn
)(
Spectral Profile
Remarks:• Λ(δ) ≤ minimum of Ф(δ0) over all δ0 ≤ δ• Unlike second eigen value, Λ(δ) is not the optimum of a convex program.• Λ(δ) is the smallest possible eigenvalue of a submatrix of size at most < δn of the matrix L • xi can be assumed to be all positive.
Small Sets via Spectral Profile
Using an analysis along the lines of analysis of Cheeger’s inequality, it yields:
Theorem [Raghavendra-Steurer-Tetali 10]There exists a poly-time algorithm that finds a set S of
size O(δ) such thatexpansion (S) ≤ sqrt (Λ* (δ) O(log 1/∙ δ))
So if there is a set S with expansion(S) = ε, thenthe algorithm finds a cut of size
)/1log( O
Similar behaviour as the Gaussian expansion profile
Spectral Profile
Second Eigen Value:
xxd
Lxx
xd
xx
T
T
x
ii
Ejiji
x nn
min
)(
min 2
),(
2
|Support(x)| < δn
)(
Spectral Profile
Replace xi by vectors vi
Subject tovi v∙ j ≥ 0
i
ii
i vnv 2
2
||
ii
Ejiji
v vd
vv
i2
),(
2
*
||
||
min)(
Graph Expansion
d-regular graph G
d
expansion(S) = # edges leaving S
d |S|
vertex set S
A random neighbor of a random vertex in S is outside of S with probability expansion(S)
ФG = expansion(S)minimum|S| ≤ n/2
Conductance of Graph G
Uniform Sparsest Cut Problem Given a graph G compute ФG and find the set S achieving it.
Approximation Algorithms:
•Cheeger’s Inequality [Alon][Alon-Milman] Given a graph G, if the normalized adjacency matrix has second eigen value λ2 then,
•A log n approximation algorithm [Leighton-Rao].•A sqrt(log n) approximation algorithm using semidefinite programming [Arora-Rao-Vazirani].
)1(22
)1(2
2
G
Extremely well-studied, many different contexts
pseudo-randomness, group theory, online routing,
Markov chains, metric embeddings, …
Limitations of Eigenvalues• The best lower bound that Cheeger’s inequality gives on expansion
is (1-λ2)/2 < ½, while Ф(δ) can be close to 1.
• Consider graph G Connect pairs of points on {0,1}n that are εn Hamming
distance away.Then Second eigenvalue ≈ 1- ε and ф(1/2) ≈ εyet Ф(δ) ≈ 1 (small sets have near-perfect expansion)
• A SIMPLE SDP RELAXATION cannot distinguish between- all small sets expand almost completely- exists small set with almost no expansion
A Conjecture
Small-Set Expansion Conjecture:
8η>0, 9 ± >0 such that GapSSE(η, ± ) is NP-hard, i.e.,
Given a graph G, it is NP-hard to distinguish YES: expansion(S) < η for some S of volume ¼ ±NO: expansion(S) > 1- η for all S of volume ¼ ±
Road Map
• Algorithm:– Spectral Profile.
• Reductions within Expansion
• Relationship with Unique Games Conjecture
Gaussian Curve
Gaussian Graph
Vertices: all points in Rd (d dimensional real space)
Edges: Weights according to the following sampling procedure:
• Sample a random Gaussian variable x in Rd
• Perturb x as follows to get y in Rd
Add an edge between x an y
zxy 22)1(
Γε (δ) = Gaussian noise sensitivity of a set of measure δ
= least expanding sets are caps/thresholds of measure δ
= )/1log(
Set Size δ 0.5
Ф(δ)
1
Approximating Expansion Profile
Reductions within Expansion
Reductions within Expansion
Theorem [Raghavendra-Steurer-Tulsiani 10]For every positive integer q, and constants ε,δ,γ, given
a graph it is SSE-hard to distinguish between:
• There exists q disjoint small sets S1 , S2 , .. Sq of size close to 1/q, such that expansion(Si) ≤ ε
• No set of size μ> δ has expansion less than size Γε/2 (μ) -- expansion of set of size μ in Gaussian graph with parameter ε/2
expansion (S) < sqrt (Λ* (δ) O(log 1/∙ μ))
Informal Statement
Set Size δ
1
Set Size δ 0.5
Ф(δ)
1
QualitativeAssumption
GapSSE is NPhard
Quantitative Statement
Given a graph G , it is SSE-hard to distinguish whether, •There is a small set of size whose expansion is ε.
•Every small set of size μ (in a certain range) expands at least as much as the corresponding Gaussian graph with noise ε )/1log( O
Corollaries
Corollary: The algorithm in [Raghavendra-Steurer Tetali 10] has near-optimal guarantee assuming SSE conjecture.
Corollary: Assuming GapSSE, there is no constant factor approximation for Balanced Separator or Uniform Sparsest Cut.
Relation with Unique Games Conjecture
Unique GamesUnique game ¡ :
Each edge (A,B) has an associated map ¼ ¼ is bijection from L(A) to L(B)
graph of size n
AB
label set L(A) ¼
Goal: Find an assignment of labels to vertices such that maximum number of edges are satisfied.
A labelling satisfies (A,B) if ¼ (label of A) = label of B
Referee
sample (A,B,¼)
BA
Player 1 Player 2pick b in L(B)pick a in L(A)
ba
Refereeplayers win if ¼(a) = b
no communication between players
value( ¡ ):maximum success probabilityover all strategies of the players
Unique Games Conjecture [Khot02]
Unique Games Conjecture: [Khot ‘02]8²>0, 9 q >0:
NP-hard to distinguish for ¡ with label set size qYES: value( ¡ ) > 1-² NO: value( ¡ ) < ²
Implications of UGC
UGC
BASIC SDP is optimal for …
Constraint Satisfaction Problems [Raghavendra`08]MAX CUT, MAX 2SAT
Ordering CSPs [GMR`08]MAX ACYCLIC SUBGRAPH, BETWEENESS
Grothendieck Problems [KNS`08, RS`09]
Metric Labeling Problems [MNRS`08]MULTIWAY CUT, 0-EXTENSION
Kernel Clustering Problems [KN`08,10]
Strict Monotone CSPs [KMTV`10]VERTEX COVER, HYPERGRAPH VERTEX COVER
…
If we could show , then a refutation of UGC would imply an improved algorithm for PROBLEM X
“Reverse Reductions”
UGCBASIC SDP is optimal forlots of optimization problems,e.g.: MAX CUT and VERTEX COVER
BASIC SDP optimal
for PROBLEM X
?*
*
Parallel Repetition is natural candidate reduction for [FeigeKinderO’Donnell’07]*
Win-Win Situation
Bad news: this reduction cannot work [Raz’08, BHHRRS’08]
PROBLEM X = MAX CUT
Small Set Expansion and Unique Games
• Solving Unique Games Finding a small non-expanding set in the “label extended graph”
Theorem [Raghavendra-Steurer 10]
Small Set Expansion Conjecture Unique Games Conjecture
Establishes a reverse connection from a natural problem.
Implications of UGC
UGC
BASIC SDP is optimal for …
Constraint Satisfaction Problems [Raghavendra`08]MAX CUT, MAX 2SAT
Ordering CSPs [GMR`08]MAX ACYCLIC SUBGRAPH, BETWEENESS
Grothendieck Problems [KNS`08, RS`09]
Metric Labeling Problems [MNRS`08]MULTIWAY CUT, 0-EXTENSION
Kernel Clustering Problems [KN`08,10]
Strict Monotone CSPs [KMTV`10]VERTEX COVER, HYPERGRAPH VERTEX COVER
…
Uniform Sparsest Cut [KNS`08, RS`09]
Minimum Linear Arrangement[KNS`08, RS`09]
Gap SSE
UGCWith expansion
Most known SDP integrality gap instances for problems like MaxCut, Vertex Cover, Unique games have graphs that are “small set expanders”
Theorem [Raghavendra-Steurer-Tulsiani 10]
Small Set Expansion Conjecture MaxCut or Unique Games on Small Set Expanders is hard.
Reverse Connections?
Approximating Spectral Profile
RoadmapIntroductionGraph Expansion: Cheeger’s Inequality, Leighton Rao, ARVExpansion Profile: Small Sets expand more than large ones.Cheeger’s inequality and SDPs failGapSSE ProblemRelation to Unique Games Conjecture: Unique Games definition, Applications, lack of reverse reductions. Label extended graph Small setsSmall Set Expansion Conjecture UGCUGC with SSE is easyAlgorithm for SSE: Spectral Profile, SDP for Spectral Profile, Rounding algorithmRelations within expansion:GapSSE Balanced Separator Hardness
NP-hard OptimizationExample:
MAX CUT: partition vertices of a graph into two setsso as to maximize number of cut edges
fundamental graph partitioning problem
benchmark for algorithmic techniques
ApproximationMAX CUT
Trivial approximation
First non-trivial approximation
based on a semidefinite relaxation (BASIC SDP)
Random assignment, cut ½ of the edges
Goemans–Williamson algorithm (‘95)
approx-ratio 0.878
Beyond Max Cut:
Analogous BASIC SDP relaxation for many other problems
Almost always, BASIC SDP gives best known approximation(often strictly better than non-SDP methods)
ApproximationMAX CUT
Trivial approximation
based on a semidefinite relaxation (BASIC SDP)
Random assignment, cut ½ of the edges
Goemans–Williamson algorithm (‘95)
approx-ratio 0.878
Can we beat the approximation guarantee of BASIC SDP?
First non-trivial approximation
Unique Games Conjecture
Unique Games Conjecture [Khot’02] (roughly):it is NP-hard to approximate the value of Unique Games
certain optimization problem:given equations of the form
xi – xj = cij mod qsatisfy as many as possible
Can we beat the approximation guarantee of BASIC SDP for Max Cut?
No, assuming Khot’s Unique Games Conjecture! [KKMO`05, MOO`05, OW’08]
UGC: Is it true?
UGC
?
UGCon expanding
instances
UGCon product instances
[AKKSTV’08, AIMS’10]
[BHHRRS’08,Raz’08]
UGCfor certain SDP
hierarchies
[RaghavendraS’09,KhotSaket’09]
Any non-trivial consequences if UGC is false?
What are hard instances?
“Reverse Reductions”
UGCBASIC SDP is optimal forlots of optimization problems,e.g.: MAX CUT and VERTEX COVER
Win-Win SituationA refutation of UGC implies an improved algorithm for SMALL-SET EXPANSION (better than BASIC SDP!)
SMALL-SET EXPANSION
First reduction from natural combinatorial problem to UNIQUE GAMES
BASIC SDP optimal
for PROBLEM X
Approximating Small-Set Expansion
Expansion profile of G at ±:minimum expansion(S) over all S with volume < ±
SHow well can we approximate the
expansion profile of G for small ± ?
BASIC SDP cannot distinguish between- all small sets expand almost completely- exists small set with almost no expansion
Small-Set Expansion Conjecture:8²>0, 9 ± >0:
NP-hard to distinguish YES: expansion(S) < ² for some S of volume ¼ ±NO: expansion(S) > 1-² for all S of volume ¼ ±
Unique 2-Prover Games
Referee
sample (A,B,¼) from D
BA
Player 1 Player 2pick b in L(B)pick a in L(A)
ba
Refereeplayers win if ¼(a) = b
Unique game ¡ :
no communication between players
value( ¡ ):maximum success probabilityover all strategies of the players
distribution D over triples (A,B,¼)- A and B are from U- ¼ is bijection from L(A) to L(B)
universe of size n
AB
label set L(A)
label set L(B)
¼
Approximating Unique Games
Unique Games Conjecture: [Khot ‘02]
8²>0, 9 q >0:NP-hard to distinguish for ¡ with label set size q
YES: value( ¡ ) > 1-² NO: value( ¡ ) < ²
How well can we approximate the value of a unique gamefor large label sets?
BASIC SDP cannot distinguish between games with value ¼ 1 and ¼ 0for large label sets
Approximating Unique Games
Unique Games Conjecture:8²>0, 9 q >0:
NP-hard to distinguish for ¡ with label set size qYES: value( ¡ ) > 1-² NO: value( ¡ ) < ²
Our main theorem:
Small-Set Expansion Conjecture ) Unique Games Conjecture
Reduction: Small-Set Expansion Unique Games
Task: find non-expanding set of volume ¼ ±
graph G
A
B
Refereesample R = 1/± random edges M
A = one half of each edgeB = other half of each edge
BA
Player 1 Player 2pick b 2 Bpick a 2 A
ba
Refereeplayers win if (a,b) 2 M
P( ) < ²
CompletenessSmall Non-Expanding Set (Partial) Strategy
graph G
AB
S
Suppose expansion(S) < ²
Strategy for Player 1:pick a 2 A if a is unique intersection with Sotherwise, refuse to answer
With constant probability, |A Å S |= {a}Conditioned on this event:
other half of a’s edge outside of S
partial game value > 1 - ²
referee allows players to refuse for few queries
SoundnessStrategy Small Non-Expanding Set
standard trick: can assume both players have same strategy
Suppose:players win with prob > 1-²
Idea: strategy distribution over sets- sample R-1 vertices U- output S = { x | Player 1 picks x if A=U+x }
Easy to show:
E volume(S) = 1/R = ±
E # edges leaving S
E d |S|< ²
Problem:volume(S) might not be concentrated around ±
Are we done?No!
SoundnessStrategy Small Non-Expanding Set
Idea: Referee adds ²-noise to A and B+ ²-noise+ ²-noise
New distribution over sets:- sample R-1 vertices U- S = { x | players pick x if A = U+x+noise
with probability > ½ }
Intuition:players cannot distinguish x and noise
Can show: 8 U. volume(S) <
= ± / ²
² R random vertices
# noise vertices1
Suppose:players win with prob > 1-²
Summary
UGCBASIC SDP is optimal forlots of optimization problems,e.g.: MAX CUT and VERTEX COVER
BASIC SDP optimal for SMALL-SET
EXPANSION
more reverse reductions?
Open Questions (+ Subsequent Work)
Hardness results based on Small-Set Expansion Conjecture? Reductions between Expansion Problems [Raghavendra S Tulsiani’10]
Thanks!Questions?
noise here vs. noise in other hardness reductions?
Better algorithms for UNIQUE GAMES via SMALL-SET EXPANSION? 2npoly(²) algorithm for (1-², ½)-UG via SSE [Arora Barak S’10]
APPENDIX
Rounding a UG strategy to a small non-expanding set
Unique Games
decorated bipartite graph
uv
label set L(v)label set L(u)
bijection ¼uv: L(u) L(v)
Referee
random edge (u,v)
vu
Player 1 Player 2pick j in L(v)pick i in L(u)
ji
Refereeplayers win if ¼uv(i) = j
Unique Games instance ¡ :
each player knows only half of referee’s edge
value( ¡ ):maximum success probabilityover all strategies of the players
ApproximationMAX CUT
Trivial approximation
First non-trivial approximation
based on a semidefinite relaxation (BASIC SDP)
Random assignment, cut ½ of the edges
Goemans–Williamson algorithm (‘95)
approx-ratio 0.878
General constraint satisfaction problems (CSPs):
for every CSP: approximation matches integrality gap of certain SDPnear-linear running time [S’10]
simple generic approximation algorithm [RaghavendraS’09]
Can we beat the approximation guarantee of BASIC SDP (for Max Cut)?
Combinatorial OptimizationExample:
MAX CUT: partition vertices of a graph into two setsso as to maximize number of cut edges
CAIDA at UCSD analyzes business relationshipsamong Autonomous Systems in the internet using MAX 2SAT
¼ MAX CUTA concrete practical application: