1 a convex polynomial that is not sos-convex amir ali ahmadi pablo a. parrilo laboratory for...
TRANSCRIPT
![Page 1: 1 A Convex Polynomial that is not SOS-Convex Amir Ali Ahmadi Pablo A. Parrilo Laboratory for Information and Decision Systems Massachusetts Institute of](https://reader036.vdocuments.mx/reader036/viewer/2022081518/5514c6f8550346b0338b4b99/html5/thumbnails/1.jpg)
1
A Convex Polynomial that is not SOS-Convex
Amir Ali AhmadiPablo A. Parrilo
Laboratory for Information and Decision SystemsMassachusetts Institute of Technology
FRG: Semidefinite Optimization and Convex Algebraic Geometry
May 2009 - MIT
![Page 2: 1 A Convex Polynomial that is not SOS-Convex Amir Ali Ahmadi Pablo A. Parrilo Laboratory for Information and Decision Systems Massachusetts Institute of](https://reader036.vdocuments.mx/reader036/viewer/2022081518/5514c6f8550346b0338b4b99/html5/thumbnails/2.jpg)
2
Deciding Convexity
42
321
22
212
31
41 1632248)( xxxxxxxxxp
Given a multivariate polynomial p(x):=p(x1,…, xn ) of even degree, how to decide if it is convex?
A concrete example:
Most direct application: global optimization
Global minimization of polynomials is NP-hard even when the degree is 4
But in presence of convexity, no local minima exist, and simple gradient methods can find a global min
![Page 3: 1 A Convex Polynomial that is not SOS-Convex Amir Ali Ahmadi Pablo A. Parrilo Laboratory for Information and Decision Systems Massachusetts Institute of](https://reader036.vdocuments.mx/reader036/viewer/2022081518/5514c6f8550346b0338b4b99/html5/thumbnails/3.jpg)
3
Other Applications
In many problems, we would like to parameterize a family of convex polynomials that perhaps:
serve as a convex envelope to a non-convex function
approximate a more complicated function
fit data samples with “small” error
[Magnani, Lall, Boyd]
To address these questions, we need an understanding of the algebraic structure of the set of convex polynomials
![Page 4: 1 A Convex Polynomial that is not SOS-Convex Amir Ali Ahmadi Pablo A. Parrilo Laboratory for Information and Decision Systems Massachusetts Institute of](https://reader036.vdocuments.mx/reader036/viewer/2022081518/5514c6f8550346b0338b4b99/html5/thumbnails/4.jpg)
4
Convexity and the Second Derivative
2221
21
2221
21
2221
21
2221
21
19219248969624
969624484812)(
xxxxxxxx
xxxxxxxxxH
But can we efficiently check if H(x) is PSD for all x?
Equivalently, H(x) is PSD if and only if the scalar polynomial yTH(x)y in 2n variables [x;y] is positive semidefinite (psd)
Back to our example:
Fact: a polynomial p(x) is convex if and only if its Hessian H(x) is positive semidefinite (PSD)
![Page 5: 1 A Convex Polynomial that is not SOS-Convex Amir Ali Ahmadi Pablo A. Parrilo Laboratory for Information and Decision Systems Massachusetts Institute of](https://reader036.vdocuments.mx/reader036/viewer/2022081518/5514c6f8550346b0338b4b99/html5/thumbnails/5.jpg)
5
Complexity of Deciding Convexity
Checking polynomial nonnegativity is NP-hard for degree 4 or larger
However, there is additional structure in the polynomial yTH(x)y:
Quadratic in y (a “biform”)
H(x) is a matrix of second derivatives partial derivatives commute
Pardalos and Vavasis (’92) included the following question proposed by Shor on a list of the seven most important open problems in complexity theory for numerical optimization:
“What is the complexity of deciding convexity of a multivariate polynomial of degree four?”
To the best of our knowledge: still open
![Page 6: 1 A Convex Polynomial that is not SOS-Convex Amir Ali Ahmadi Pablo A. Parrilo Laboratory for Information and Decision Systems Massachusetts Institute of](https://reader036.vdocuments.mx/reader036/viewer/2022081518/5514c6f8550346b0338b4b99/html5/thumbnails/6.jpg)
6
SOS-Convexity
p(x) sos-convex
p(x) convex
))(())(()()()( yxMyxMyxMxMyyxHy TTTT
As we will see, checking sos-convexity can be cast as the feasibility of a semidefinite program (SDP), which can be solved in polynomial time using interior-point methods.
Defn. ([Helton, Nie]): a polynomial p(x) is sos-convex if its Hessian factors as
for a possibly nonsquare polynomial matrix M(x).
)()()( xMxMxH T
![Page 7: 1 A Convex Polynomial that is not SOS-Convex Amir Ali Ahmadi Pablo A. Parrilo Laboratory for Information and Decision Systems Massachusetts Institute of](https://reader036.vdocuments.mx/reader036/viewer/2022081518/5514c6f8550346b0338b4b99/html5/thumbnails/7.jpg)
7
SOS-convexity (Ctnd.)
sos-convexity in the literature: Semidefinite representability of semialgebraic sets [Helton, Nie]
Generalization of Jensen’s inequality [Lasserre]
Polynomial fitting, minimum volume convex sets [Magnani, Lall, Boyd]
Question that has been raised:
Q: must every convex polynomial be sos-convex?
NoOur main contribution
(via an explicit counterexample)
![Page 8: 1 A Convex Polynomial that is not SOS-Convex Amir Ali Ahmadi Pablo A. Parrilo Laboratory for Information and Decision Systems Massachusetts Institute of](https://reader036.vdocuments.mx/reader036/viewer/2022081518/5514c6f8550346b0338b4b99/html5/thumbnails/8.jpg)
8
AgendaNonnegativity and sum of squares
A bit of history
Connection to semidefinite programming
SOS-matrices
Other (equivalent?) notions for sos-convexity
Our counterexample (convex but not sos-convex)
Ideas behind the proof
Several remarks
How did we find it?
Conclusions
![Page 9: 1 A Convex Polynomial that is not SOS-Convex Amir Ali Ahmadi Pablo A. Parrilo Laboratory for Information and Decision Systems Massachusetts Institute of](https://reader036.vdocuments.mx/reader036/viewer/2022081518/5514c6f8550346b0338b4b99/html5/thumbnails/9.jpg)
9
Nonnegative and Sum of Squares Polynomials
m
ii xqxp
1
2 )()(
Defn. A polynomial p(x) is nonnegative or positive semidefinite (psd) if
Defn. A polynomial p(x) is a sum of squares (sos) if there exist some other polynomials q1(x),…, qm(x) such that
p(x) sos p(x) psd (obvious)
When is the converse true?
nxxp 0)(
![Page 10: 1 A Convex Polynomial that is not SOS-Convex Amir Ali Ahmadi Pablo A. Parrilo Laboratory for Information and Decision Systems Massachusetts Institute of](https://reader036.vdocuments.mx/reader036/viewer/2022081518/5514c6f8550346b0338b4b99/html5/thumbnails/10.jpg)
10
Hilbert’s 1888 Paper
In 1888, Hilbert proved that a nonnegative polynomial p(x) of degree d in n variables must be sos only in the following cases:
n=1 (univariate polynomials of any degree)
d=2 (quadratic polynomials in any number of variables)
n=2 and d=4 (bivariate quartics)
In all other cases, there are polynomials that are psd but not sos
![Page 11: 1 A Convex Polynomial that is not SOS-Convex Amir Ali Ahmadi Pablo A. Parrilo Laboratory for Information and Decision Systems Massachusetts Institute of](https://reader036.vdocuments.mx/reader036/viewer/2022081518/5514c6f8550346b0338b4b99/html5/thumbnails/11.jpg)
11
The Celebrated Example of Motzkin
The first concrete counterexample was found about 80 years later!
This polynomial is psd but not sos
13),( 22
21
41
22
42
2121 xxxxxxxxM
![Page 12: 1 A Convex Polynomial that is not SOS-Convex Amir Ali Ahmadi Pablo A. Parrilo Laboratory for Information and Decision Systems Massachusetts Institute of](https://reader036.vdocuments.mx/reader036/viewer/2022081518/5514c6f8550346b0338b4b99/html5/thumbnails/12.jpg)
12
Sum of Squares and Semidefinite Programming
Unlike nonnegativity, checking whether a polynomial is SOS is a tractable problem
,)( Qzzxp T
Thm: A polynomial p(x) of degree 2d is SOS if and only if there exists a PSD matrix Q such that
where z is the vector of monomials of degree up to d
].,...,,,...,,,1[ 2121dnn xxxxxxz
Feasible set is the intersection of an affine subspace and the PSD cone, and thus is a semidefinite program.
![Page 13: 1 A Convex Polynomial that is not SOS-Convex Amir Ali Ahmadi Pablo A. Parrilo Laboratory for Information and Decision Systems Massachusetts Institute of](https://reader036.vdocuments.mx/reader036/viewer/2022081518/5514c6f8550346b0338b4b99/html5/thumbnails/13.jpg)
13
SOS matrices
Therefore, can solve an SDP to check if P(x) is an sos-matrix.
Defn. ([Kojima],[Gatermann-Parrilo]):
A symmetric polynomial matrix P(x) is an sos-matrix if
for a possibly nonsquare polynomial matrix M(x).
)()()( xMxMxP T
Lemma: P(x) is an sos-matrix if and only if the scalar polynomial yTP(x)y in [x;y] is sos.
![Page 14: 1 A Convex Polynomial that is not SOS-Convex Amir Ali Ahmadi Pablo A. Parrilo Laboratory for Information and Decision Systems Massachusetts Institute of](https://reader036.vdocuments.mx/reader036/viewer/2022081518/5514c6f8550346b0338b4b99/html5/thumbnails/14.jpg)
14
PSD matrices may not be SOS
Explicit “biform” examples of Choi, Reznick (and others), yield PSD matrices that are not SOS.
For instance, the biquadratic Choi form can be rewritten as:
However this example (and all others we’ve found), is not a valid Hessian:
![Page 15: 1 A Convex Polynomial that is not SOS-Convex Amir Ali Ahmadi Pablo A. Parrilo Laboratory for Information and Decision Systems Massachusetts Institute of](https://reader036.vdocuments.mx/reader036/viewer/2022081518/5514c6f8550346b0338b4b99/html5/thumbnails/15.jpg)
15
Equivalent notions for convexity
)2
1(]1,0[,
))1(()()1()(
enoughyx
yxpypxp
yxxyxpxpyp T ,))(()()(
yxyxHyT ,0)(
Basic definition:
First order condition:
Second order condition:
![Page 16: 1 A Convex Polynomial that is not SOS-Convex Amir Ali Ahmadi Pablo A. Parrilo Laboratory for Information and Decision Systems Massachusetts Institute of](https://reader036.vdocuments.mx/reader036/viewer/2022081518/5514c6f8550346b0338b4b99/html5/thumbnails/16.jpg)
16
Each condition can be SOS-ified
]1,0[))1(()()1()(),( SOSyxpypxpyxg
SOSxyxpxpypyxg T ))(()()(),(
SOSyxHyyxg T )(),(2
Basic definition:
First order condition:
Second order condition:
SOSyxpypxpyxg )()()(),(2
1
2
1
2
1
2
1
2
1 A’
B
C
A
AThm: CBA’
Proof: mimics the “standard” proof closely and uses closedness of the SOS cone
![Page 17: 1 A Convex Polynomial that is not SOS-Convex Amir Ali Ahmadi Pablo A. Parrilo Laboratory for Information and Decision Systems Massachusetts Institute of](https://reader036.vdocuments.mx/reader036/viewer/2022081518/5514c6f8550346b0338b4b99/html5/thumbnails/17.jpg)
17
A convex polynomial that is not sos-convex
Without further ado...
]1,0[))1(()()1()(),( yxpypxpyxg
))(()()(),( xyxpxpypyxg T
yxHyyxg T )(),(2
B
C
A
Need a polynomial p(x) such that all the following polynomials
are psd but not sos.
![Page 18: 1 A Convex Polynomial that is not SOS-Convex Amir Ali Ahmadi Pablo A. Parrilo Laboratory for Information and Decision Systems Massachusetts Institute of](https://reader036.vdocuments.mx/reader036/viewer/2022081518/5514c6f8550346b0338b4b99/html5/thumbnails/18.jpg)
18
Our Counterexample
83
63
22
43
42
23
62
82
63
21
43
22
21
23
42
21
43
41
23
22
41
42
41
23
61
22
61
81
30607044
162416335
43254011832)(
xxxxxxx
xxxxxxxxxxx
xxxxxxxxxxxp
Claim:
p(x) is convex: H(x) is PSD
p(x) is not sos-convex: H(x) ≠ MT(x)M(x)
A homogeneous polynomial in three variables, of degree 8.
![Page 19: 1 A Convex Polynomial that is not SOS-Convex Amir Ali Ahmadi Pablo A. Parrilo Laboratory for Information and Decision Systems Massachusetts Institute of](https://reader036.vdocuments.mx/reader036/viewer/2022081518/5514c6f8550346b0338b4b99/html5/thumbnails/19.jpg)
19
Proof: H(x) is PSD
yxHyxxx T )()( 23
22
21
)()()()( 23
22
21 xMxMxHxxx T
Or equivalently the scalar polynomial
is sos.
Claim:
Proof: Exact sos decomposition, with rational coefficients.
Exploiting symmetries of this polynomial, we solve SDPs of significantly reduced size
44433322211123
22
21 )()( zQzzQzzQzzQzyxHyxxx TTTTT
![Page 20: 1 A Convex Polynomial that is not SOS-Convex Amir Ali Ahmadi Pablo A. Parrilo Laboratory for Information and Decision Systems Massachusetts Institute of](https://reader036.vdocuments.mx/reader036/viewer/2022081518/5514c6f8550346b0338b4b99/html5/thumbnails/20.jpg)
20
Rational SOS Decomposition
![Page 21: 1 A Convex Polynomial that is not SOS-Convex Amir Ali Ahmadi Pablo A. Parrilo Laboratory for Information and Decision Systems Massachusetts Institute of](https://reader036.vdocuments.mx/reader036/viewer/2022081518/5514c6f8550346b0338b4b99/html5/thumbnails/21.jpg)
21
Rational SOS Decomposition
![Page 22: 1 A Convex Polynomial that is not SOS-Convex Amir Ali Ahmadi Pablo A. Parrilo Laboratory for Information and Decision Systems Massachusetts Institute of](https://reader036.vdocuments.mx/reader036/viewer/2022081518/5514c6f8550346b0338b4b99/html5/thumbnails/22.jpg)
22
Proof: H(x)≠MT(x)M(x)
Lemma: if H(x) is an sos-matrix, then all its 2n-1 principal minors are sos polynomials. In particular, all diagonal elements are sos.
Proof: follows from the Cauchy-Binet formula.
Therefore, it suffices to show that
is not sos.
We do this by a duality argument.
63
43
22
23
42
43
21
23
22
21
42
21
23
41
22
41
61
48326420516
300120035401792)1,1(
xxxxxxxxxx
xxxxxxxH
![Page 23: 1 A Convex Polynomial that is not SOS-Convex Amir Ali Ahmadi Pablo A. Parrilo Laboratory for Information and Decision Systems Massachusetts Institute of](https://reader036.vdocuments.mx/reader036/viewer/2022081518/5514c6f8550346b0338b4b99/html5/thumbnails/23.jpg)
23
Separating Hyperplane
SOS
PSDH(1,1)
µ
SOSww
H
0,
0)1,1(,
![Page 24: 1 A Convex Polynomial that is not SOS-Convex Amir Ali Ahmadi Pablo A. Parrilo Laboratory for Information and Decision Systems Massachusetts Institute of](https://reader036.vdocuments.mx/reader036/viewer/2022081518/5514c6f8550346b0338b4b99/html5/thumbnails/24.jpg)
24
A few remarks
)1,,(),( 2121 xxpxxp
Our counterexample is robust to small perturbations
Follows from inequalities being strict
A dehomogenized version is still convex but not sos-convex
Minimal in the number of variables
“Almost” minimal in the degree
![Page 25: 1 A Convex Polynomial that is not SOS-Convex Amir Ali Ahmadi Pablo A. Parrilo Laboratory for Information and Decision Systems Massachusetts Institute of](https://reader036.vdocuments.mx/reader036/viewer/2022081518/5514c6f8550346b0338b4b99/html5/thumbnails/25.jpg)
25
How did we find this polynomial?
sosyxHyxxx
H
r
Tr )()(
0)1,1(,
,fix
23
22
21
parameterize H(x)
add Hessian constraints (partial derivatives must commute)
solve this sos-programSOS
PSDH(1,1)
µ
M
![Page 26: 1 A Convex Polynomial that is not SOS-Convex Amir Ali Ahmadi Pablo A. Parrilo Laboratory for Information and Decision Systems Massachusetts Institute of](https://reader036.vdocuments.mx/reader036/viewer/2022081518/5514c6f8550346b0338b4b99/html5/thumbnails/26.jpg)
26
Messages to take home…
SOS-relaxation is a tractable technique for certifying positive semidefiniteness of scalar or matrix polynomials
We specialized to convexity and sos-convexity
Three natural notions for sos-convexity are equivalent
Not always exact
But very powerful (at least for low degrees and dimensions)
Proposed a convex relaxation to search over a restricted family of psd polynomials that are not sos
Open: what’s the complexity of deciding convexity?
Our result further supports the hypothesis that it must be a hard problem
![Page 27: 1 A Convex Polynomial that is not SOS-Convex Amir Ali Ahmadi Pablo A. Parrilo Laboratory for Information and Decision Systems Massachusetts Institute of](https://reader036.vdocuments.mx/reader036/viewer/2022081518/5514c6f8550346b0338b4b99/html5/thumbnails/27.jpg)
27
• Want to know more? Preprint at http://arxiv.org/abs/0903.1287
Questions?