yuan zhou carnegie mellon university
DESCRIPTION
Approximability and Proof Complexity. Yuan Zhou Carnegie Mellon University Joint works with Boaz Barak, Fernando G.S.L. Brandão, Aram W. Harrow, Jonathan Kelner, Ryan O'Donnell and David Steurer. Constraint Satisfaction Problems. Given: a set of variables: V - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Yuan Zhou Carnegie Mellon University](https://reader036.vdocuments.mx/reader036/viewer/2022062806/56814e58550346895dbbf146/html5/thumbnails/1.jpg)
Yuan ZhouCarnegie Mellon University
Joint works with Boaz Barak, Fernando G.S.L. Brandão, Aram W. Harrow, Jonathan Kelner, Ryan
O'Donnell and David Steurer
![Page 2: Yuan Zhou Carnegie Mellon University](https://reader036.vdocuments.mx/reader036/viewer/2022062806/56814e58550346895dbbf146/html5/thumbnails/2.jpg)
Constraint Satisfaction Problems
• Given:– a set of variables: V– a set of values: Ω– a set of "local constraints": E
• Goal: find an assignment σ : V -> Ω to maximize #satisfied constraints in E
• α-approximation algorithm: always outputs a solution of value at least α*OPT
![Page 3: Yuan Zhou Carnegie Mellon University](https://reader036.vdocuments.mx/reader036/viewer/2022062806/56814e58550346895dbbf146/html5/thumbnails/3.jpg)
Example 1: Max-Cut
• Vertex set: V = {1, 2, 3, ..., n}• Value set: Ω = {0, 1}• Typical local constraint: (i, j) э E wants σ(i) ≠
σ(j)
• Alternative description:– Given G = (V, E), divide V into two parts,– to maximize #edges across the cut
• Best approx. alg.: 0.878-approx. [GW'95]• Best NP-hardness: 0.941 [Has'01, TSSW'00]
![Page 4: Yuan Zhou Carnegie Mellon University](https://reader036.vdocuments.mx/reader036/viewer/2022062806/56814e58550346895dbbf146/html5/thumbnails/4.jpg)
Example 2: Balanced Seperator
• Vertex set: V = {1, 2, 3, ..., n}• Value set: Ω = {0, 1}• Minimize #satisfied local constraints: (i, j) э E : σ(i) ≠ σ(j)• Global constraint: n/3 ≤ |{i : σ(i) = 0}| ≤
2n/3
• Alternative description:– given G = (V, E)– divide V into two "balanced" parts,– to minimize #edges across the cut
![Page 5: Yuan Zhou Carnegie Mellon University](https://reader036.vdocuments.mx/reader036/viewer/2022062806/56814e58550346895dbbf146/html5/thumbnails/5.jpg)
Example 2: Balanced Seperator (cont'd)
• Vertex set: V = {1, 2, 3, ..., n}• Value set: Ω = {0, 1}• Minimize #satisfied local constraints: (i, j) э E : σ(i) ≠ σ(j)• Global constraint: n/3 ≤ |{i : σ(i) = 0}| ≤
2n/3
• Best approx. alg.: sqrt{log n}-approx. [ARV'04]
• Only (1+ε)-approx. alg. is ruled out even assuming 3-SAT does not have subexp time alg. [AMS'07]
![Page 6: Yuan Zhou Carnegie Mellon University](https://reader036.vdocuments.mx/reader036/viewer/2022062806/56814e58550346895dbbf146/html5/thumbnails/6.jpg)
Example 3: Unique Games• Vertex set: V = {1, 2, 3, ..., n}• Value set: Ω = {0, 1, 2, ..., q - 1}• Maximize #satisfied local constraints: (i, j) э E : σ(i) - σ(j) = c (mod q)
• Unique Games Conjecture (UGC) [Kho'02, KKMO'07]
No poly-time algorithm, given an instance where optimal solution satisfies (1-ε) constraints, finds a solution satisfying ε constraints
• Stronger than (implies) "no constant approx. alg."
![Page 7: Yuan Zhou Carnegie Mellon University](https://reader036.vdocuments.mx/reader036/viewer/2022062806/56814e58550346895dbbf146/html5/thumbnails/7.jpg)
Example 3: Unique Games (cont'd)
• Vertex set: V = {1, 2, 3, ..., n}• Value set: Ω = {0, 1, 2, ..., q - 1}• Maximize #satisfied local constraints: (i, j) э E : σ(i) - σ(j) = c (mod q)
• UG(ε): to tell whether an instance has a solution satisfying (1-ε) constraints, or no solution satisfying ε constraints
• Unique Games Conjecture (UGC). UG(ε) is hard for sufficiently large q
![Page 8: Yuan Zhou Carnegie Mellon University](https://reader036.vdocuments.mx/reader036/viewer/2022062806/56814e58550346895dbbf146/html5/thumbnails/8.jpg)
Example 3: Unique Games (cont'd)
• Implications of UGC– For large class of problems, BASIC-SDP
(semidefinite programming relaxation) achieves optimal approximation ratio
Max-Cut: 0.878-approx. Vertex-Cover: 2-approx. Max-CSP [KKMO '07, MOO '10, KV '03, Rag '08]
![Page 9: Yuan Zhou Carnegie Mellon University](https://reader036.vdocuments.mx/reader036/viewer/2022062806/56814e58550346895dbbf146/html5/thumbnails/9.jpg)
Open questions
• Is UGC true?
• Are the implications of UGC true?– Is Max-Cut hard to approximate better than
0.878?
– Is Balanced Seperator hard to approximate with in constant factor?
![Page 10: Yuan Zhou Carnegie Mellon University](https://reader036.vdocuments.mx/reader036/viewer/2022062806/56814e58550346895dbbf146/html5/thumbnails/10.jpg)
SDP Relaxation hierarchies
• A systematic way to write tighter and tighter SDP relaxations
• Examples– Sherali-Adams+SDP [SA'90]– Lasserre hierarchy [Par'00, Las'01]
…
?
UG(ε)
r rounds SDP relaxation in roughly time
)(rOn
BASIC-SDP
GW SDP for Maxcut (0.878-approx.)ARV SDP for Balanced Seperator
![Page 11: Yuan Zhou Carnegie Mellon University](https://reader036.vdocuments.mx/reader036/viewer/2022062806/56814e58550346895dbbf146/html5/thumbnails/11.jpg)
How many rounds of tighening suffice?• Upperbounds
– rounds of SA+SDP suffice for UG(ε) [ABS'10,
BRS'11]
• Lowerbounds [KV'05, DKSV'06, RS'09, BGHMRS '12]
(also known as constructing integrality gap instances)
– rounds of SA+SDP needed for UG(ε)
– rounds of SA+SDP needed for better-than-0.878 approx for Max-Cut
– rounds for SA+SDP needed for constant approx. for Balanced Seperator
)1(n
))logexp((log )1(n
)1()log(log n
))logexp((log )1(n
![Page 12: Yuan Zhou Carnegie Mellon University](https://reader036.vdocuments.mx/reader036/viewer/2022062806/56814e58550346895dbbf146/html5/thumbnails/12.jpg)
Our Results
• We study the performance of Lasserre SDP hierarchy against known lowerbound instances for SA+SDP hierarchy, and show that
• 8-round Lasserre solves the Unique Games lowerbound instances [BBHKSZ'12]
• 4-round Lasserre solves the Balanced Seperator lowerbound instances [OZ'12]
• Constant-round Lasserre gives better-than-0.878 approximation for Max-Cut lowerbound instances [OZ'12]
![Page 13: Yuan Zhou Carnegie Mellon University](https://reader036.vdocuments.mx/reader036/viewer/2022062806/56814e58550346895dbbf146/html5/thumbnails/13.jpg)
Proof overview
• Integrality gap instance– SDP completeness: a good vector solution– Integral soundness: no good integral
solution
• A common method to construct gaps (e.g. [RS'09])
– Use the instance derived from a hardness reduction
– Lift the completeness proof to vector world– Use the soundness proof directly
![Page 14: Yuan Zhou Carnegie Mellon University](https://reader036.vdocuments.mx/reader036/viewer/2022062806/56814e58550346895dbbf146/html5/thumbnails/14.jpg)
Proof overview (cont'd)
• Our goal: to prove there is no good vector solution– Rounding algorithms?
• Instead, – we bound the value of the dual of the SDP
– interpret the dual of the SDP as a proof system ("Sum-of-squares proof system")
– lift the soundness proof to the proof system
![Page 15: Yuan Zhou Carnegie Mellon University](https://reader036.vdocuments.mx/reader036/viewer/2022062806/56814e58550346895dbbf146/html5/thumbnails/15.jpg)
Remarks
• Connection between SDP hierarchies and algebraic proof systems
• New insight in designing integrality gap instances– should avoid soundness proofs that can be
lifted to Sum-of-Squares proof system
• Lasserre is strictly stronger than other hierarchies on UG and related problems (as it was believed to be)
![Page 16: Yuan Zhou Carnegie Mellon University](https://reader036.vdocuments.mx/reader036/viewer/2022062806/56814e58550346895dbbf146/html5/thumbnails/16.jpg)
Outline of the rest of the talk
Sum-of-Squares proof system
Relation between SoS proof system and Lasserre SDP hierarchy
Lift the soundness proofs to the SoS proof system
![Page 17: Yuan Zhou Carnegie Mellon University](https://reader036.vdocuments.mx/reader036/viewer/2022062806/56814e58550346895dbbf146/html5/thumbnails/17.jpg)
Sum-of-Squares proof system
![Page 18: Yuan Zhou Carnegie Mellon University](https://reader036.vdocuments.mx/reader036/viewer/2022062806/56814e58550346895dbbf146/html5/thumbnails/18.jpg)
Polynomial optimization
• Maximize/Minimize• Subject to
all functions are low-degree n-variate polynomial functions
• Max-Cut example: Maximize
s.t.
)(xp
0)(,0)(,0)( 21 xqxqxq m0)(,0)(,0)( '21 xrxrxr m
2)(E jiE(i,j)
xx
ixx ii ,0)1(
![Page 19: Yuan Zhou Carnegie Mellon University](https://reader036.vdocuments.mx/reader036/viewer/2022062806/56814e58550346895dbbf146/html5/thumbnails/19.jpg)
Polynomial optimization (cont'd)
• Maximize/Minimize• Subject to
all functions are low-degree n-variate polynomial functions
• Balanced Seperator example: Minimize
s.t.
)(xp
0)(,0)(,0)( 21 xqxqxq m0)(,0)(,0)( '21 xrxrxr m
32
31 ][E,][E
,0)1(
ii
ii
ii
xx
ixx
2)(E jiE(i,j)
xx
![Page 20: Yuan Zhou Carnegie Mellon University](https://reader036.vdocuments.mx/reader036/viewer/2022062806/56814e58550346895dbbf146/html5/thumbnails/20.jpg)
Certifying no good solution
• Maximize• Subject to
• To certify that there is no solution better than , simply say that the following equations & inequalities are infeasible
)(xp
0)(,0)(,0)( 21 xqxqxq m0)(,0)(,0)( '21 xrxrxr m
)(xp
0)(,0)(,0)( 21 xqxqxq m0)(,0)(,0)( '21 xrxrxr m
![Page 21: Yuan Zhou Carnegie Mellon University](https://reader036.vdocuments.mx/reader036/viewer/2022062806/56814e58550346895dbbf146/html5/thumbnails/21.jpg)
The Sum-of-Squares proof system
• To show the following equations & inequalities are infeasible,
• Show that
• where is a sum of squared polynomials, including 's
• A degree-d "Sum-of-Squares" refutation, where
0)(,0)(,0)( 21 xqxqxq m0)(,0)(,0)( '21 xrxrxr m
)}deg(),deg(){deg(max hqfd iii
)()()(1...1
xhxqxfmi
ii
)(xh)(xri
![Page 22: Yuan Zhou Carnegie Mellon University](https://reader036.vdocuments.mx/reader036/viewer/2022062806/56814e58550346895dbbf146/html5/thumbnails/22.jpg)
Example 1
• To refute
• We simply write
• A degree-2 SoS refutation
2)1()2()1(1 xxxx
0)1(
2
xx
x
![Page 23: Yuan Zhou Carnegie Mellon University](https://reader036.vdocuments.mx/reader036/viewer/2022062806/56814e58550346895dbbf146/html5/thumbnails/23.jpg)
Example 2: Max-Cut on triangle graph
• To refute
• We "simply" write ... ...
0)1(,0)1(,0)1( 332211 xxxxxx
2)()()( 213
232
221 xxxxxx
![Page 24: Yuan Zhou Carnegie Mellon University](https://reader036.vdocuments.mx/reader036/viewer/2022062806/56814e58550346895dbbf146/html5/thumbnails/24.jpg)
)12)(1(
)3222)(1(
)12)(1(
)1()1()(
2)()()(
212133
313123122
3223
2211
232
221
22313221
213
232
221
xxxxxx
xxxxxxxx
xxxxxx
xxxxxxxxxxx
xxxxxx
Example 2: Max-Cut on triangle graph (cont'd)
• A degree-4 SoS refutation
![Page 25: Yuan Zhou Carnegie Mellon University](https://reader036.vdocuments.mx/reader036/viewer/2022062806/56814e58550346895dbbf146/html5/thumbnails/25.jpg)
Relation between SoS proof system and Lasserre SDP hierarchy
![Page 26: Yuan Zhou Carnegie Mellon University](https://reader036.vdocuments.mx/reader036/viewer/2022062806/56814e58550346895dbbf146/html5/thumbnails/26.jpg)
Finding SoS refutation by SDP
• A degree-d SoS refutation corresponds to solution of an SDP with variables
• The SDP is the same as the dual of -round Lasserre relaxation
• An SoS refutation => upperbound on the dual of optimum of Lasserre => upperbound on the value of Lasserre– e.g. 4-round Lasserre says that Max-Cut of
the triangle graph is at most 2 (BASIC-SDP gives 9/4)
)( dnO
)(d
Bounding SDP value by SoS refutation
![Page 27: Yuan Zhou Carnegie Mellon University](https://reader036.vdocuments.mx/reader036/viewer/2022062806/56814e58550346895dbbf146/html5/thumbnails/27.jpg)
Remarks• Positivestellensatz. [Krivine'64, Stengle'73] If
the given equalities & inequalities are infeasible, there is always an SoS refutation (degree not bounded).
• The degree-d SoS proof system was first proposed by Grigoriev and Vorobjov in 1999
• Grigoriev showed degree is needed to refute unsatisfiable sparse -linear equations– later rediscovered by Schoenbeck in
Lasserre world
)(n2F
![Page 28: Yuan Zhou Carnegie Mellon University](https://reader036.vdocuments.mx/reader036/viewer/2022062806/56814e58550346895dbbf146/html5/thumbnails/28.jpg)
Lift the proofs to SoS proof system
![Page 29: Yuan Zhou Carnegie Mellon University](https://reader036.vdocuments.mx/reader036/viewer/2022062806/56814e58550346895dbbf146/html5/thumbnails/29.jpg)
Unique Games
![Page 30: Yuan Zhou Carnegie Mellon University](https://reader036.vdocuments.mx/reader036/viewer/2022062806/56814e58550346895dbbf146/html5/thumbnails/30.jpg)
Components of the soundness proof
• Cauchy-Schwarz/Hölder's inequality• Hypercontractivity inequality• Smallsets expand in the noisy hypercube• Invariance Principle• Influence decoding
(of known UG instances)
![Page 31: Yuan Zhou Carnegie Mellon University](https://reader036.vdocuments.mx/reader036/viewer/2022062806/56814e58550346895dbbf146/html5/thumbnails/31.jpg)
Hypercontractivity Inequality
• 2->4 hypercontractivity inequality: for low degree polynomial
we have
• Goal of an SoS proof: write
Note that 's are indeterminates
dSnSi
SiS xxf
||],[
)(
22
}1,1{
4
}1,1{])([E9])([E
xfxf
nn x
d
x
ixx
d hxfxfnn
2}2{}1{
4
}1,1{
22
}1,1{),,,(])([E])([E9
S
![Page 32: Yuan Zhou Carnegie Mellon University](https://reader036.vdocuments.mx/reader036/viewer/2022062806/56814e58550346895dbbf146/html5/thumbnails/32.jpg)
Hypercontractivity Inequality (cont'd)• 2->4 hypercontractivity inequality: for low degree polynomial
we have
• Goal of an SoS proof: write
• Prove by induction (very similar to the well-known inductive proof of the inequality itself)...
dSnSi
SiS xxf
||],[
)(
22
}1,1{
4
}1,1{])([E9])([E
xfxf
nn x
d
x
ixx
d hxfxfnn
2}2{}1{
4
}1,1{
22
}1,1{),,,(])([E])([E9
![Page 33: Yuan Zhou Carnegie Mellon University](https://reader036.vdocuments.mx/reader036/viewer/2022062806/56814e58550346895dbbf146/html5/thumbnails/33.jpg)
Components of the soundness proof
• Cauchy-Schwarz/Hölder's inequality• Hypercontractivity inequality• Smallsets expand in the noisy hypercube• Invariance Principle• Influence decoding
(of known UG instances)
![Page 34: Yuan Zhou Carnegie Mellon University](https://reader036.vdocuments.mx/reader036/viewer/2022062806/56814e58550346895dbbf146/html5/thumbnails/34.jpg)
A few words on Invariance Principle• trickier • "bump function" is used in the original proof
--- not a polynomial!
• but... a polynomial substitution is enough for UG
![Page 35: Yuan Zhou Carnegie Mellon University](https://reader036.vdocuments.mx/reader036/viewer/2022062806/56814e58550346895dbbf146/html5/thumbnails/35.jpg)
Max-Cut and Balanced Seperator• An SoS proof for "Majority Is Stablest" theorem
is needed for Max-Cut instances– We don't know how to get around the bump
function issue in the invariance step– Instead, we proved a weaker theorem: "2/pi
theorem" -- suffices to give better-than-0.878 algorithms for known Max-Cut instances
• Balanced Seperator. Key is to SoS-ize the proof for KKL theorem– Hypercontractivity and SSE is also useful
there – Some more issues to be handled
![Page 36: Yuan Zhou Carnegie Mellon University](https://reader036.vdocuments.mx/reader036/viewer/2022062806/56814e58550346895dbbf146/html5/thumbnails/36.jpg)
Summary
• SoS/Lasserre hierarchy refutes all known UG instances and Balanced Seperator instances, gives better-than-0.878 approximation for known Max-Cut instances,– certain types of soundness proof does not
work for showing a gap of SoS/Lasserre hierarchy
![Page 37: Yuan Zhou Carnegie Mellon University](https://reader036.vdocuments.mx/reader036/viewer/2022062806/56814e58550346895dbbf146/html5/thumbnails/37.jpg)
Open problems
• Show that SoS/Lasserre hierarchy fully refutes Max-Cut instances?– SoS-ize Majority Is Stablest theorem...
• More lowerbound instances for SoS/Lasserre hierarchy?
![Page 38: Yuan Zhou Carnegie Mellon University](https://reader036.vdocuments.mx/reader036/viewer/2022062806/56814e58550346895dbbf146/html5/thumbnails/38.jpg)
Thank you!