wolfe’s method - uc davis mathematicsjhaddock/slides/wolfe/uc...wolfe’s method jamie haddock...
TRANSCRIPT
Wolfe’s Method
Jamie Haddock
April 6, 2017
Graduate Group in Applied Mathematics
UC Davis
Minimum Norm Point in Polytope
We are interested in solving the problem (MNP(P)):
minx∈P||x ||2
where P is a polytope.
Reminder: A polytope, P, is the convex hull of points p1, p2, ..., pm,
P =
{ m∑i=1
λipi :m∑i=1
λi = 1, λi ≥ 0 for all i = 1, 2, ...,m
}.
1
Minimum Norm Point in Polytope
We are interested in solving the problem (MNP(P)):
minx∈P||x ||2
where P is a polytope.
Reminder: A polytope, P, is the convex hull of points p1, p2, ..., pm,
P =
{ m∑i=1
λipi :m∑i=1
λi = 1, λi ≥ 0 for all i = 1, 2, ...,m
}.
1
Minimum Norm Point in Polytope
p1p2
p3
p4
p5
P
O
2
Minimum Norm Point in Polytope
p1p2
p3
p4
p5
P
O
2
Formulation as QP
This is a quadratic programming problem (QP).
Given p1, p2, ..., pm ∈ Rn, define P = [p1p2...pm] ∈ Rn×m.
Then our problem is which can be solved using interior-point methods.
3
Formulation as QP
This is a quadratic programming problem (QP).
Given p1, p2, ..., pm ∈ Rn, define P = [p1p2...pm] ∈ Rn×m.
Then our problem is which can be solved using interior-point methods.
3
Formulation as QP
This is a quadratic programming problem (QP).
Given p1, p2, ..., pm ∈ Rn, define P = [p1p2...pm] ∈ Rn×m.
Then our problem is
min ||Py ||22s.t. 1T y = 1
y ≥ 0
which can be solved using interior-point methods.
3
Optimality Condition
Theorem (Wolfe)
Let P = {p1, p2, ..., pm}. Then x ∈ conv(P) is MNP(P) if and only if
xTpj ≥ ||x ||22 for all j = 1, 2, ...,m.
4
Optimality Condition
Theorem (Wolfe)
Let P = {p1, p2, ..., pm}. Then x ∈ conv(P) is MNP(P) if and only if
xTpj ≥ ||x ||22 for all j = 1, 2, ...,m.
p1p2
p3
p4
p5
P
O
4
Optimality Condition
Theorem (Wolfe)
Let P = {p1, p2, ..., pm}. Then x ∈ conv(P) is MNP(P) if and only if
xTpj ≥ ||x ||22 for all j = 1, 2, ...,m.
p1p2
p3
p4
p5
P
O
4
Optimality Condition
Theorem (Wolfe)
Let P = {p1, p2, ..., pm}. Then x ∈ conv(P) is MNP(P) if and only if
xTpj ≥ ||x ||22 for all j = 1, 2, ...,m.
p1p2
p3
p4
p5
P
O
{x : pT4 x = ||p4||22}
4
Intuition and Definitions
Idea: Exploit linear information about the problem in order to progress
towards the nonlinear (quadratic) solution.
Def: An affinely independent set of points Q = {q1, q2, ..., qk} is a corral
if MNP(conv(Q)) ∈ relint(conv(Q)).
q1 q2
O
q1 q2
q3
O
Note: Singletons are corrals.
Note: There is a corral of points in P whose convex hull contains
MNP(P).
5
Intuition and Definitions
Idea: Exploit linear information about the problem in order to progress
towards the nonlinear (quadratic) solution.
Def: An affinely independent set of points Q = {q1, q2, ..., qk} is a corral
if MNP(conv(Q)) ∈ relint(conv(Q)).
q1 q2
O
q1 q2
q3
O
Note: Singletons are corrals.
Note: There is a corral of points in P whose convex hull contains
MNP(P).
5
Intuition and Definitions
Idea: Exploit linear information about the problem in order to progress
towards the nonlinear (quadratic) solution.
Def: An affinely independent set of points Q = {q1, q2, ..., qk} is a corral
if MNP(conv(Q)) ∈ relint(conv(Q)).
q1 q2
O
q1 q2
q3
O
Note: Singletons are corrals.
Note: There is a corral of points in P whose convex hull contains
MNP(P).
5
Intuition and Definitions
Idea: Exploit linear information about the problem in order to progress
towards the nonlinear (quadratic) solution.
Def: An affinely independent set of points Q = {q1, q2, ..., qk} is a corral
if MNP(conv(Q)) ∈ relint(conv(Q)).
q1 q2
O
q1 q2
q3
O
Note: Singletons are corrals.
Note: There is a corral of points in P whose convex hull contains
MNP(P).
5
Intuition and Definitions
Idea: Exploit linear information about the problem in order to progress
towards the nonlinear (quadratic) solution.
Def: An affinely independent set of points Q = {q1, q2, ..., qk} is a corral
if MNP(conv(Q)) ∈ relint(conv(Q)).
q1 q2
O
q1 q2
q3
O
Note: Singletons are corrals.
Note: There is a corral of points in P whose convex hull contains
MNP(P).
5
Intuition and Definitions
Idea: Exploit linear information about the problem in order to progress
towards the nonlinear (quadratic) solution.
Def: An affinely independent set of points Q = {q1, q2, ..., qk} is a corral
if MNP(conv(Q)) ∈ relint(conv(Q)).
q1 q2
O
q1 q2
q3
O
Note: Singletons are corrals.
Note: There is a corral of points in P whose convex hull contains
MNP(P).5
Sketch of Method
x ∈ P = {p1, p2, ..., pm}C = {x}while x is not MNP(P)
pj ∈ {p ∈ P : xTp < ||x ||22}C = C ∪ {pj}y = MNP(aff(C ))
while y 6∈ conv(C )
z = argminz∈conv(C)∩xy
||z − y ||2
C = C − {pi} where pi , z
are on different faces of
conv(C )
x = z
y = MNP(aff(C ))
x = y
return x
p1 = (0, 2)
p2 = (3, 0)
p3 = (−2, 1)
p1
p3
p2
O
P
6
Sketch of Method
x ∈ P = {p1,p2, ...,pm}C = {x}while x is not MNP(P)
pj ∈ {p ∈ P : xTp < ||x ||22}C = C ∪ {pj}y = MNP(aff(C ))
while y 6∈ conv(C )
z = argminz∈conv(C)∩xy
||z − y ||2
C = C − {pi} where pi , z
are on different faces of
conv(C )
x = z
y = MNP(aff(C ))
x = y
return x
p1 = (0, 2)
p2 = (3, 0)
p3 = (−2, 1)
p1 = x
p3
p2
O
P
6
Sketch of Method
x ∈ P = {p1, p2, ..., pm}C = {x}while x is not MNP(P)
pj ∈ {p ∈ P : xTp < ||x ||22}C = C ∪ {pj}y = MNP(aff(C ))
while y 6∈ conv(C )
z = argminz∈conv(C)∩xy
||z − y ||2
C = C − {pi} where pi , z
are on different faces of
conv(C )
x = z
y = MNP(aff(C ))
x = y
return x
p1 = (0, 2)
p2 = (3, 0)
p3 = (−2, 1)
p1 = x
p3
p2
O
P
C = {p1}
6
Sketch of Method
x ∈ P = {p1, p2, ..., pm}C = {x}while x is not MNP(P)
pj ∈ {p ∈ P : xTp < ||x ||22}C = C ∪ {pj}y = MNP(aff(C ))
while y 6∈ conv(C )
z = argminz∈conv(C)∩xy
||z − y ||2
C = C − {pi} where pi , z
are on different faces of
conv(C )
x = z
y = MNP(aff(C ))
x = y
return x
p1 = (0, 2)
p2 = (3, 0)
p3 = (−2, 1)
p1 = x
p3
p2
O
P
C = {p1}
6
Sketch of Method
x ∈ P = {p1, p2, ..., pm}C = {x}while x is not MNP(P)
pj ∈ {p ∈ P : xTp < ||x||22}C = C ∪ {pj}y = MNP(aff(C ))
while y 6∈ conv(C )
z = argminz∈conv(C)∩xy
||z − y ||2
C = C − {pi} where pi , z
are on different faces of
conv(C )
x = z
y = MNP(aff(C ))
x = y
return x
p1 = (0, 2)
p2 = (3, 0)
p3 = (−2, 1)
p1 = x
p3
p2
O
P
C = {p1}
6
Sketch of Method
x ∈ P = {p1, p2, ..., pm}C = {x}while x is not MNP(P)
pj ∈ {p ∈ P : xTp < ||x ||22}C = C∪ {pj}y = MNP(aff(C ))
while y 6∈ conv(C )
z = argminz∈conv(C)∩xy
||z − y ||2
C = C − {pi} where pi , z
are on different faces of
conv(C )
x = z
y = MNP(aff(C ))
x = y
return x
p1 = (0, 2)
p2 = (3, 0)
p3 = (−2, 1)
p1 = x
p3
p2
O
P
C = {p1,p2}
6
Sketch of Method
x ∈ P = {p1, p2, ..., pm}C = {x}while x is not MNP(P)
pj ∈ {p ∈ P : xTp < ||x ||22}C = C ∪ {pj}y = MNP(aff(C))
while y 6∈ conv(C )
z = argminz∈conv(C)∩xy
||z − y ||2
C = C − {pi} where pi , z
are on different faces of
conv(C )
x = z
y = MNP(aff(C ))
x = y
return x
p1 = (0, 2)
p2 = (3, 0)
p3 = (−2, 1)
p1 = x
p3
p2
O
Py
C = {p1, p2}
6
Sketch of Method
x ∈ P = {p1, p2, ..., pm}C = {x}while x is not MNP(P)
pj ∈ {p ∈ P : xTp < ||x ||22}C = C ∪ {pj}y = MNP(aff(C ))
while y 6∈ conv(C)
z = argminz∈conv(C)∩xy
||z − y ||2
C = C − {pi} where pi , z
are on different faces of
conv(C )
x = z
y = MNP(aff(C ))
x = y
return x
p1 = (0, 2)
p2 = (3, 0)
p3 = (−2, 1)
p1 = x
p3
p2
O
Py
C = {p1, p2}
6
Sketch of Method
x ∈ P = {p1, p2, ..., pm}C = {x}while x is not MNP(P)
pj ∈ {p ∈ P : xTp < ||x ||22}C = C ∪ {pj}y = MNP(aff(C ))
while y 6∈ conv(C )
z = argminz∈conv(C)∩xy
||z − y ||2
C = C − {pi} where pi , z
are on different faces of
conv(C )
x = z
y = MNP(aff(C ))
x = y
return x
p1 = (0, 2)
p2 = (3, 0)
p3 = (−2, 1)
p1
p3
p2
O
Px = y
C = {p1, p2}
6
Sketch of Method
x ∈ P = {p1, p2, ..., pm}C = {x}while x is not MNP(P)
pj ∈ {p ∈ P : xTp < ||x ||22}C = C ∪ {pj}y = MNP(aff(C ))
while y 6∈ conv(C )
z = argminz∈conv(C)∩xy
||z − y ||2
C = C − {pi} where pi , z
are on different faces of
conv(C )
x = z
y = MNP(aff(C ))
x = y
return x
p1 = (0, 2)
p2 = (3, 0)
p3 = (−2, 1)
p1
p3
p2
O
Px = y
C = {p1, p2}
6
Sketch of Method
x ∈ P = {p1, p2, ..., pm}C = {x}while x is not MNP(P)
pj ∈ {p ∈ P : xTp < ||x||22}C = C ∪ {pj}y = MNP(aff(C ))
while y 6∈ conv(C )
z = argminz∈conv(C)∩xy
||z − y ||2
C = C − {pi} where pi , z
are on different faces of
conv(C )
x = z
y = MNP(aff(C ))
x = y
return x
p1 = (0, 2)
p2 = (3, 0)
p3 = (−2, 1)
p1
p3
p2
O
Px = y
C = {p1, p2}
6
Sketch of Method
x ∈ P = {p1, p2, ..., pm}C = {x}while x is not MNP(P)
pj ∈ {p ∈ P : xTp < ||x ||22}C = C∪ {pj}y = MNP(aff(C ))
while y 6∈ conv(C )
z = argminz∈conv(C)∩xy
||z − y ||2
C = C − {pi} where pi , z
are on different faces of
conv(C )
x = z
y = MNP(aff(C ))
x = y
return x
p1 = (0, 2)
p2 = (3, 0)
p3 = (−2, 1)
p1
p3
p2
O
Px = y
C = {p1,p2,p3}
6
Sketch of Method
x ∈ P = {p1, p2, ..., pm}C = {x}while x is not MNP(P)
pj ∈ {p ∈ P : xTp < ||x ||22}C = C ∪ {pj}y = MNP(aff(C))
while y 6∈ conv(C )
z = argminz∈conv(C)∩xy
||z − y ||2
C = C − {pi} where pi , z
are on different faces of
conv(C )
x = z
y = MNP(aff(C ))
x = y
return x
p1 = (0, 2)
p2 = (3, 0)
p3 = (−2, 1)
p1
p3
p2
O = y
Px
C = {p1, p2, p3}
6
Sketch of Method
x ∈ P = {p1, p2, ..., pm}C = {x}while x is not MNP(P)
pj ∈ {p ∈ P : xTp < ||x ||22}C = C ∪ {pj}y = MNP(aff(C ))
while y 6∈ conv(C)
z = argminz∈conv(C)∩xy
||z − y ||2
C = C − {pi} where pi , z
are on different faces of
conv(C )
x = z
y = MNP(aff(C ))
x = y
return x
p1 = (0, 2)
p2 = (3, 0)
p3 = (−2, 1)
p1
p3
p2
O = y
Px
C = {p1, p2, p3}
6
Sketch of Method
x ∈ P = {p1, p2, ..., pm}C = {x}while x is not MNP(P)
pj ∈ {p ∈ P : xTp < ||x ||22}C = C ∪ {pj}y = MNP(aff(C ))
while y 6∈ conv(C )
z = argminz∈conv(C)∩xy
||z− y||2
C = C − {pi} where pi , z
are on different faces of
conv(C )
x = z
y = MNP(aff(C ))
x = y
return x
p1 = (0, 2)
p2 = (3, 0)
p3 = (−2, 1)
z
p1
p3
p2
O = y
Px
C = {p1, p2, p3}
6
Sketch of Method
x ∈ P = {p1, p2, ..., pm}C = {x}while x is not MNP(P)
pj ∈ {p ∈ P : xTp < ||x ||22}C = C ∪ {pj}y = MNP(aff(C ))
while y 6∈ conv(C )
z = argminz∈conv(C)∩xy
||z − y ||2
C = C− {pi} where pi, z
are on different faces of
conv(C)
x = z
y = MNP(aff(C ))
x = y
return x
p1 = (0, 2)
p2 = (3, 0)
p3 = (−2, 1)
z
p1
p3
p2
O = y
Px
C = {p2,p3}
6
Sketch of Method
x ∈ P = {p1, p2, ..., pm}C = {x}while x is not MNP(P)
pj ∈ {p ∈ P : xTp < ||x ||22}C = C ∪ {pj}y = MNP(aff(C ))
while y 6∈ conv(C )
z = argminz∈conv(C)∩xy
||z − y ||2
C = C − {pi} where pi , z
are on different faces of
conv(C )
x = z
y = MNP(aff(C ))
x = y
return x
p1 = (0, 2)
p2 = (3, 0)
p3 = (−2, 1)
x = z
p1
p3
p2
O = y
P
C = {p2, p3}
6
Sketch of Method
x ∈ P = {p1, p2, ..., pm}C = {x}while x is not MNP(P)
pj ∈ {p ∈ P : xTp < ||x ||22}C = C ∪ {pj}y = MNP(aff(C ))
while y 6∈ conv(C )
z = argminz∈conv(C)∩xy
||z − y ||2
C = C − {pi} where pi , z
are on different faces of
conv(C )
x = z
y = MNP(aff(C))
x = y
return x
p1 = (0, 2)
p2 = (3, 0)
p3 = (−2, 1)
x = z
p1
p3
p2
O
P
y
C = {p2, p3}
6
Sketch of Method
x ∈ P = {p1, p2, ..., pm}C = {x}while x is not MNP(P)
pj ∈ {p ∈ P : xTp < ||x ||22}C = C ∪ {pj}y = MNP(aff(C ))
while y 6∈ conv(C)
z = argminz∈conv(C)∩xy
||z − y ||2
C = C − {pi} where pi , z
are on different faces of
conv(C )
x = z
y = MNP(aff(C ))
x = y
return x
p1 = (0, 2)
p2 = (3, 0)
p3 = (−2, 1)
x = z
p1
p3
p2
O
P
y
C = {p2, p3}
6
Sketch of Method
x ∈ P = {p1, p2, ..., pm}C = {x}while x is not MNP(P)
pj ∈ {p ∈ P : xTp < ||x ||22}C = C ∪ {pj}y = MNP(aff(C ))
while y 6∈ conv(C )
z = argminz∈conv(C)∩xy
||z − y ||2
C = C − {pi} where pi , z
are on different faces of
conv(C )
x = z
y = MNP(aff(C ))
x = y
return x
p1 = (0, 2)
p2 = (3, 0)
p3 = (−2, 1)
p1
p3
p2
O
P
x = y
C = {p2, p3}
6
Sketch of Method
x ∈ P = {p1, p2, ..., pm}C = {x}while x is not MNP(P)
pj ∈ {p ∈ P : xTp < ||x ||22}C = C ∪ {pj}y = MNP(aff(C ))
while y 6∈ conv(C )
z = argminz∈conv(C)∩xy
||z − y ||2
C = C − {pi} where pi , z
are on different faces of
conv(C )
x = z
y = MNP(aff(C ))
x = y
return x
p1 = (0, 2)
p2 = (3, 0)
p3 = (−2, 1)
p1
p3
p2
O
P
x
C = {p2, p3}
6
Sketch of Method
x ∈ P = {p1, p2, ..., pm}C = {x}while x is not MNP(P)
pj ∈ {p ∈ P : xTp < ||x ||22}C = C ∪ {pj}y = MNP(aff(C ))
while y 6∈ conv(C )
z = argminz∈conv(C)∩xy
||z − y ||2
C = C − {pi} where pi , z
are on different faces of
conv(C )
x = z
y = MNP(aff(C ))
x = y
return x
p1 = (0, 2)
p2 = (3, 0)
p3 = (−2, 1)
p1
p3
p2
O
P
x
C = {p2, p3}
6
LP reduces to MNP
Proposition (De Loera, H., Rademacher)
An instance of LP can be reduced to an instance of MNP in strongly
polynomial time.
This means that if there is a polynomial time (or even better, strongly
polynomial time) algorithm for MNP then we have a polynomial time (or
strongly polynomial time) algorithm for LP.
7
LP reduces to MNP
Proposition (De Loera, H., Rademacher)
An instance of LP can be reduced to an instance of MNP in strongly
polynomial time.
This means that if there is a polynomial time (or even better, strongly
polynomial time) algorithm for MNP then we have a polynomial time (or
strongly polynomial time) algorithm for LP.
7
Experiments
8
Experiments
8