convex duaity and kantorovich duality theoremhomepages.wmich.edu/~ledyaev/zhu-talk2-sp2016.pdf ·...
TRANSCRIPT
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Convex sets and functionsConvex duality
Kantorovich Duality theorem
Convex Duaity and Kantorovich Duality Theorem
Qiji Zhu
Analysis Seminar
Feburary 12, 2016
Qiji Zhu Convex Duaity and Kantorovich Duality Theorem
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Convex sets and functionsConvex duality
Kantorovich Duality theorem
Introduction
In the last talk I mentioned that the Kantorovich duality for masstransport problem is, in fact, a special case of the convex duality.Today we go into some of the details by
• First introduce the basic concepts of convex analysis: convexsets and functions, subdifferentials, and convex conjugate;
• next we discuss the Fenchel duality theorem and outline theproof.
• finally culminating in a proof of the Kantorovich dualitytheorem by converting the Kantorovich mass transportproblem into a Fenchel problem.
Qiji Zhu Convex Duaity and Kantorovich Duality Theorem
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Convex sets and functionsConvex duality
Kantorovich Duality theorem
Convex setsConvex functionsSubdifferential
Convex sets
Convex sets
We say C ⊂ X is convex if for any x, y ∈ C and λ ∈ [0, 1],
λx+ (1− λ)y ∈ C.
Note that the sum and difference of convex sets are convex and sois the intersection of any class of convex sets. However, the unionof two convex sets may not be convex.
Qiji Zhu Convex Duaity and Kantorovich Duality Theorem
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Convex sets and functionsConvex duality
Kantorovich Duality theorem
Convex setsConvex functionsSubdifferential
Convex functions
Convex functions
We say f : X 7→ R ∪ {+∞} is convex if, for any x, y ∈ X andλ ∈ [0, 1],
f(λx+ (1− λ)y) ≤ λf(x) + (1− λ)f(y).
Qiji Zhu Convex Duaity and Kantorovich Duality Theorem
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Convex sets and functionsConvex duality
Kantorovich Duality theorem
Convex setsConvex functionsSubdifferential
Convex functions: properties
A convex function is always below any secant line in between thetwo intersection points.
x
y
Qiji Zhu Convex Duaity and Kantorovich Duality Theorem
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Convex sets and functionsConvex duality
Kantorovich Duality theorem
Convex setsConvex functionsSubdifferential
Convex functions: properties
Also, a convex function is always above its tangent lines.Moreover, a differentiable convex function has an increasingderivative function.
x
y
Qiji Zhu Convex Duaity and Kantorovich Duality Theorem
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Convex sets and functionsConvex duality
Kantorovich Duality theorem
Convex setsConvex functionsSubdifferential
Epigraph characterizations
Epigraph characterizations
Function f : X 7→ R ∪ {+∞} is convex iff epi f is a convex set inX × R.
Qiji Zhu Convex Duaity and Kantorovich Duality Theorem
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Convex sets and functionsConvex duality
Kantorovich Duality theorem
Convex setsConvex functionsSubdifferential
Subdifferential
Subdifferential is a substitute for the derivative.
Subdifferential
The subdifferential of a lower semi-continuous convex function ϕat x ∈ dom ϕ is defined by
∂ϕ(x) = {x∗ ∈ X∗ : ϕ(y)− ϕ(x) ≥ ⟨x∗, y − x⟩}.
Qiji Zhu Convex Duaity and Kantorovich Duality Theorem
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Convex sets and functionsConvex duality
Kantorovich Duality theorem
Convex setsConvex functionsSubdifferential
Nonemptyness of subdifferential
The most useful property of a convex function related to Lagrangemultipliers is
Nonemptyness of subdifferential
Let f : X 7→ R ∪ {+∞} be a convex function. Then for anyx ∈ int dom f ,
∂f(x) = ∅.
This result follows directly from the Hahn-Banach separationtheorem.
Qiji Zhu Convex Duaity and Kantorovich Duality Theorem
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Convex sets and functionsConvex duality
Kantorovich Duality theorem
Convex setsConvex functionsSubdifferential
Nonemptyness of subdifferential: graph
Nonemptyness of subdifferential follows from separation theorem
x
y
Qiji Zhu Convex Duaity and Kantorovich Duality Theorem
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Convex sets and functionsConvex duality
Kantorovich Duality theorem
Convex setsConvex functionsSubdifferential
Subdifferential characterizations
Subdifferential characterizations
Function f : X 7→ R ∪ {+∞} is convex iff, for anyx∗ ∈ ∂f(x), y∗ ∈ ∂f(y),
⟨y∗ − x∗, y − x⟩ ≥ 0.
Qiji Zhu Convex Duaity and Kantorovich Duality Theorem
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Convex sets and functionsConvex duality
Kantorovich Duality theorem
Convex setsConvex functionsSubdifferential
Corollary
Derivative characterizations
Function f : R 7→ R ∪ {+∞} is convex iff f ′ is increasing orf ′′ ≥ 0.
Qiji Zhu Convex Duaity and Kantorovich Duality Theorem
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Convex sets and functionsConvex duality
Kantorovich Duality theorem
Fenchel-Young inequalityFenchel formulationWeak and strong duality
Fenchel-Legendre transform
Fenchel-Legendre transform
Let f : X → R ∪+∞ be a lsc function. The Fenchel-Legendretransform f∗ : X∗ → R ∪+∞ is defined by
f∗(x∗) = supx∈X
[⟨x∗, x⟩ − f(x)]
.
Note that f is not necessarily convex but f∗ is always convex.When f ′−1 exists we have
f∗(x∗) = x∗f ′−1(x∗)− f(f ′−1(x∗)).
Qiji Zhu Convex Duaity and Kantorovich Duality Theorem
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Convex sets and functionsConvex duality
Kantorovich Duality theorem
Fenchel-Young inequalityFenchel formulationWeak and strong duality
Fenchel-Legendre transform: examples
f(x) dom f f∗(y) dom f∗
0 R 0 {0}
0 R+ 0 −R+
0 [−1, 1] |y| R
0 [0, 1] y+ R
Table: Conjugate pairs of convex functions on R.
Qiji Zhu Convex Duaity and Kantorovich Duality Theorem
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Convex sets and functionsConvex duality
Kantorovich Duality theorem
Fenchel-Young inequalityFenchel formulationWeak and strong duality
Fenchel-Legendre transform: examples
|x|p/p, p > 1 R |y|q/q (1p +1q = 1) R
|x|p/p, p > 1 R+ |y+|q/q (1p +1q = 1) R
−xp/p, 0<p<1 R+ −(−y)q/q (1p +1q = 1) −int R+
− log x int R+ −1− log(−y) −int R+
ex R{y log y − y (y > 0)0 (y = 0)
R+
Table: Conjugate pairs of convex functions on R.
Qiji Zhu Convex Duaity and Kantorovich Duality Theorem
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Convex sets and functionsConvex duality
Kantorovich Duality theorem
Fenchel-Young inequalityFenchel formulationWeak and strong duality
Fenchel-Young inequality
Fenchel-Young inequality follows directly from definition.
Fenchel-Young inequality
f(x) + f∗(x∗) ≥ ⟨x∗, x⟩.
Qiji Zhu Convex Duaity and Kantorovich Duality Theorem
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Convex sets and functionsConvex duality
Kantorovich Duality theorem
Fenchel-Young inequalityFenchel formulationWeak and strong duality
Holder inequality
Let f(x) = |x|p/p in the Fenchel -Young inequality we get
Holder inequality
For 1/p+ 1/q = 1,|x|p
p+
|y|q
q≥ |xy|.
When p = q = 2 we get
Cauchy inequality
|x|2
2+
|y|2
2≥ |xy|.
Qiji Zhu Convex Duaity and Kantorovich Duality Theorem
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Convex sets and functionsConvex duality
Kantorovich Duality theorem
Fenchel-Young inequalityFenchel formulationWeak and strong duality
Fenchel-Young equality
In general,
Fenchel-Young equality
f(x) + f∗(x∗) = ⟨x∗, x⟩.
iffx∗ ∈ ∂f(x).
This also follows from the definition.
Qiji Zhu Convex Duaity and Kantorovich Duality Theorem
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Convex sets and functionsConvex duality
Kantorovich Duality theorem
Fenchel-Young inequalityFenchel formulationWeak and strong duality
Fenchel-Young inequality: graphic illustration
For increasing function ϕ, ϕ(0) = 0, f(x) =∫ x0 ϕ(s)ds is convex
and f∗(x∗) =∫ x∗0 ϕ−1(t)dt.
s
t
O x
x∗
ϕϕ−1
Fenchel-Young inequality
Qiji Zhu Convex Duaity and Kantorovich Duality Theorem
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Convex sets and functionsConvex duality
Kantorovich Duality theorem
Fenchel-Young inequalityFenchel formulationWeak and strong duality
Fenchel-Young inequality: graphic illustration
s
t
O x
x∗
ϕϕ−1
Fenchel-Young inequality
Qiji Zhu Convex Duaity and Kantorovich Duality Theorem
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Convex sets and functionsConvex duality
Kantorovich Duality theorem
Fenchel-Young inequalityFenchel formulationWeak and strong duality
Fenchel-Young equality
s
t
O x
x∗
ϕϕ−1
Fenchel-Young equality
We see x∗ = ϕ(x), x = ϕ−1(x∗).
Qiji Zhu Convex Duaity and Kantorovich Duality Theorem
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Convex sets and functionsConvex duality
Kantorovich Duality theorem
Fenchel-Young inequalityFenchel formulationWeak and strong duality
Fenchel duality
Let v(y) = infx[f(x) + g(x+ y)]. The Fenchel primal problem is
p = v(0) = infx[f(x) + g(x)]. (1)
The dual problem is
d = v∗∗(0) = supy∗
[−f∗(y∗)− g∗(−y∗)]. (2)
Qiji Zhu Convex Duaity and Kantorovich Duality Theorem
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Convex sets and functionsConvex duality
Kantorovich Duality theorem
Fenchel-Young inequalityFenchel formulationWeak and strong duality
Verify the dual
We calculate v∗(−y∗) = supx,y[⟨−y∗, y⟩ − f(x)− g(x+ y)].Letting u = x+ y we have
v∗(−y∗) = supx,u
⟨−y∗, u− x⟩ − f(x)− g(u)
= supx[⟨y∗, x⟩ − f(x)] + sup
u[⟨−y∗, u⟩ − g(u)]
= f∗(y∗) + g∗(−y∗).
Thus,
d = v∗∗(0) = sup−y∗
[0− v∗(−y∗)] = sup−y∗
[−f∗(y∗)− g∗(−y∗)].
Qiji Zhu Convex Duaity and Kantorovich Duality Theorem
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Convex sets and functionsConvex duality
Kantorovich Duality theorem
Fenchel-Young inequalityFenchel formulationWeak and strong duality
Weak duality
Weak duality p ≥ d follows directly from the definition.Strong duality asserting p = d needs additional condition which wediscuss below.
Qiji Zhu Convex Duaity and Kantorovich Duality Theorem
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Convex sets and functionsConvex duality
Kantorovich Duality theorem
Fenchel-Young inequalityFenchel formulationWeak and strong duality
Strong duality
• If both f and g are convex functions it is easy to see that so is
v(y) = infx[f(x) + g(x+ y)].
• We can directly check that dom v = dom g − dom f .
• The sufficient condition for ∂v(0) = ∅, is
0 ∈ int dom v = int[dom g − dom f ]. (3)
A condition (3) is often referred to as a constraint qualification.
Qiji Zhu Convex Duaity and Kantorovich Duality Theorem
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Convex sets and functionsConvex duality
Kantorovich Duality theorem
Fenchel-Young inequalityFenchel formulationWeak and strong duality
Strong duality
Duality and constraint qualification
If l.s.c. convex functions f , g satisfy the constraint qualificationconditions
0 ∈ int dom v = int[dom g − dom f ]
then p = d, and the dual problem has a solution when finite.
Qiji Zhu Convex Duaity and Kantorovich Duality Theorem
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Convex sets and functionsConvex duality
Kantorovich Duality theorem
Fenchel-Young inequalityFenchel formulationWeak and strong duality
Strong duality: Proof
The condition
0 ∈ int dom v = int[dom g − dom f ]
implies that ∂v(0) = ∅. Let −y∗ ∈ ∂v(0) we have
f(x) + g(x+ y) ≥ v(y) ≥ p− ⟨y∗, y⟩
Letting u = x+ y we have
p ≤ f(x)− ⟨y∗, x⟩+ g(u) + ⟨y∗, u⟩
Taking inf on x, u we have
p ≤ −f∗(y∗)− g∗(−y∗) ≤ d ≤ p.
Qiji Zhu Convex Duaity and Kantorovich Duality Theorem
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Convex sets and functionsConvex duality
Kantorovich Duality theorem
Kantorovich DualityKantorovich DualityProof of Kantorovich DualityChecking constraint qualification condition
Setting
We will prove the Kantorovich duality theorem under the followingsetting:
• Assume X and Y are compact sets.
• Consider Banach space E = C(X × Y ), continuous functionson X × Y endowed with the sup norm.
• Then E∗ =M(X × Y ) is the space of regular Radonmeasures endowed with the total variation norm.
Qiji Zhu Convex Duaity and Kantorovich Duality Theorem
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Convex sets and functionsConvex duality
Kantorovich Duality theorem
Kantorovich DualityKantorovich DualityProof of Kantorovich DualityChecking constraint qualification condition
Kantorovich Duality
Kantorovich Duality
The function c satisfies the constraint qualification conditions thereexists a(x), b(y) integrable such that
c(x, y) ≥ a(x) + b(y).
Then
infπ∈Π(µ,ν)
∫X×Y
c(x, y)dπ(x, y)
= sup(φ,ψ)∈Φc
{∫Xφ(x)dµ(x) +
∫Yψ(y)dν(y)
}.
where Φc := {(φ,ψ) ∈ C(X)× C(Y ) : φ(x) + ψ(y) ≤ c(x, y)}.
Qiji Zhu Convex Duaity and Kantorovich Duality Theorem
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Convex sets and functionsConvex duality
Kantorovich Duality theorem
Kantorovich DualityKantorovich DualityProof of Kantorovich DualityChecking constraint qualification condition
Converting to Fenchel problem
For u ∈ E = C(X × Y ) define
g(u) :=
{0 u(x, y) ≥ −c(x, y)+∞ otherwise.
f(u) :=
{∫X φ(x)dµ+
∫Y ψ(y)dν u(x, y) = φ(x) + ψ(y)
+∞ otherwise.
Qiji Zhu Convex Duaity and Kantorovich Duality Theorem
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Convex sets and functionsConvex duality
Kantorovich Duality theorem
Kantorovich DualityKantorovich DualityProof of Kantorovich DualityChecking constraint qualification condition
Converting to Fenchel problem
We have
infu∈E
[f(u) + g(u)]
= inf{∫Xφ1(x)dµ+
∫Yψ1(y)dν : φ1(x) + ψ1(y) ≥ −c(x, y)}
= − sup(φ,ψ)∈Φc
{∫Xφ(x)dµ+
∫Yψ(y)dν}.
Qiji Zhu Convex Duaity and Kantorovich Duality Theorem
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Convex sets and functionsConvex duality
Kantorovich Duality theorem
Kantorovich DualityKantorovich DualityProof of Kantorovich DualityChecking constraint qualification condition
Calculating the conjugate
We have
g∗(−π) = supu∈E
[−∫X×Y
udπ − g(u)]
= supu∈E
[−∫X×Y
udπ : −u(x, y) ≤ c(x, y)]
=
{∫X×Y c(x, y)dπ π ∈M+(X × Y )
+∞ otherwise.
Qiji Zhu Convex Duaity and Kantorovich Duality Theorem
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Convex sets and functionsConvex duality
Kantorovich Duality theorem
Kantorovich DualityKantorovich DualityProof of Kantorovich DualityChecking constraint qualification condition
Calculating the conjugate
f∗(π) = supu∈E
[
∫X×Y
u(x, y)dπ − f(u)]
= supu∈E
[
∫X×Y
φ(x) + ψ(y)dπ −∫Xφ(x)dµ−
∫Yψ(y)dν]
=
{0 π ∈ Π(µ, ν)
+∞ otherwise.
Qiji Zhu Convex Duaity and Kantorovich Duality Theorem
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Convex sets and functionsConvex duality
Kantorovich Duality theorem
Kantorovich DualityKantorovich DualityProof of Kantorovich DualityChecking constraint qualification condition
Calculating the conjugate
Thus,
supπ∈E∗
[−f∗(π)− g∗(−π)]
= supπ∈Π(µ,ν)
−∫X×Y
c(x, y)dπ
= − infπ∈Π(µ,ν)
∫X×Y
c(x, y)dπ.
Qiji Zhu Convex Duaity and Kantorovich Duality Theorem
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Convex sets and functionsConvex duality
Kantorovich Duality theorem
Kantorovich DualityKantorovich DualityProof of Kantorovich DualityChecking constraint qualification condition
Constraint qualification condition
We observe
[a(x)− 1] + [b(y)− 1] ∈ int[dom g ∩ dom f ].
Thus
0 ∈ intdom f − dom g.
Qiji Zhu Convex Duaity and Kantorovich Duality Theorem
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Convex sets and functionsConvex duality
Kantorovich Duality theorem
Kantorovich DualityKantorovich DualityProof of Kantorovich DualityChecking constraint qualification condition
References
G. Monge, Memoire sur la theo rie des deeblais et des remblais, InHistorie de l’Academie Royale des Sciences de Paris (1781) 666-704.A. Galichon, Optimal Transport Methods in Economics, preprint 2015.L. V. Kantorovhich, Mathematical methods in the organization andplanning of production. Leningrad Univ. 1939.L. V. Kantorovhich, On the translocation of masses. Dokl. Akad. Nauk.USSR 37 (1942) 199-201.C. Villani, Optimal Transport, Old and New, Springer 2006
C. Villani, Topics in Optimal Transportation, AMS 2003.
Qiji Zhu Convex Duaity and Kantorovich Duality Theorem