duality - u-m personal world wide web servermepelman/teaching/ioe611/handouts/duali… · duality i...

Duality

I Lagrange dual problem

I weak and strong duality

I geometric interpretation

I optimality conditions

I perturbation and sensitivity analysis

I examples

I generalized inequalities

IOE 611: Nonlinear Programming, Fall 2017 6. Duality Page 6–1

Lagrangian

Standard form problem (not necessarily convex)

minimize f0(x)subject to f

i

(x) 0, i = 1, . . . ,mhi

(x) = 0, i = 1, . . . , p

variable x 2 Rn, domain D 6= ;, optimal value p?

Lagrangian: L : Rn ⇥Rm ⇥Rp ! R, with dom L = D ⇥Rm ⇥Rp,

L(x , �, ⌫) = f0(x) +mX

i=1

�i

fi

(x) +pX

i=1

⌫i

hi

(x)

I weighted sum of objective and constraint functions

I �i

is Lagrange multiplier associated with fi

(x) 0

I ⌫i

is Lagrange multiplier associated with hi

(x) = 0


Lagrange dual function

Lagrange dual function: g : Rm ⇥ Rp ! R,

g(�, ⌫) = infx2D

L(x , �, ⌫)

= infx2D

f0(x) +

mX

i=1

�i

fi

(x) +pX

i=1

⌫i

hi

(x)

!

g is concave, can be �1 for some �, ⌫

Lower bound property: if � ⌫ 0, then g(�, ⌫) p?

Proof: if x is feasible and � ⌫ 0, then

f0(x) � L(x , �, ⌫) � infx2D

L(x , �, ⌫) = g(�, ⌫).

Minimizing over all feasible x gives p? � g(�, ⌫)


Example: Least-norm solution of linear equations

minimize xT xsubject to Ax = b

dual function

I Lagrangian is L(x , ⌫) = xT x + ⌫T (Ax � b)

I to minimize L over x , set gradient equal to zero:

rx

L(x , ⌫) = 2x + AT⌫ = 0 =) x(⌫) = �(1/2)AT⌫

I plug it into L to obtain g :

g(⌫) = L((�1/2)AT⌫, ⌫) = �1

4⌫TAAT⌫ � bT⌫

— a concave function of ⌫

lower bound property: p? � �(1/4)⌫TAAT⌫ � bT⌫ for all ⌫IOE 611: Nonlinear Programming, Fall 2017 6. Duality Page 6–4

Example: Standard form LP

minimize cT xsubject to Ax = b, x ⌫ 0

dual functionI Lagrangian is

L(x , �, ⌫) = cT x + ⌫T (Ax � b) � �T x

= �bT⌫ + (c + AT⌫ � �)T x

I L is linear in x , hence

g(�, ⌫) = infx

L(x , �, ⌫) =

⇢ �bT⌫ AT⌫ � � + c = 0�1 otherwise

g is linear on a�ne domain {(�, ⌫) | AT⌫ � � + c = 0}, henceconcave

lower bound property: p? � �bT⌫ if AT⌫ + c ⌫ 0IOE 611: Nonlinear Programming, Fall 2017 6. Duality Page 6–5

Example: Equality constrained norm minimization

minimize kxksubject to Ax = b

dual function

g(⌫) = infx

(kxk � ⌫TAx + bT⌫) =

⇢bT⌫ kAT⌫k⇤ 1�1 otherwise

,

where kvk⇤ = supkuk1 uT v is the dual norm of k · k

Proof: consider infx

(kxk � yT x), where y = AT⌫.

I If kyk⇤ 1, then kxk � yT x � 0 for all x , with equality ifx = 0, so inf

x

(kxk � yT x) = 0.I If kyk⇤ > 1, choose x = tu where kuk 1, uT y = kyk⇤ > 1:

kxk � yT x = t(kuk � kyk⇤) ! �1 as t ! 1,

so infx

(kxk � yT x) = �1.

lower bound property: p? � bT⌫ if kAT⌫k⇤ 1IOE 611: Nonlinear Programming, Fall 2017 6. Duality Page 6–6

Two-way partitioning

minimize xTWxsubject to x2

i

= 1, i = 1, . . . , n

I a nonconvex problem; feasible set contains 2n discrete points

I interpretation: partition {1, . . . , n} in two sets; Wij

is cost ofassigning i , j to the same set; �W

ij

is cost of assigning todi↵erent sets

dual function

g(⌫) = infx

(xTWx +X

i

⌫i

(x2i

� 1)) = infx

xT (W + diag(⌫))x � 1T⌫

=

⇢ �1T⌫ W + diag(⌫) ⌫ 0�1 otherwise

lower bound property: p? � �1T⌫ if W + diag(⌫) ⌫ 0example: ⌫ = ��

min

(W )1 gives bound p? � n�min

(W )IOE 611: Nonlinear Programming, Fall 2017 6. Duality Page 6–7

The conjugate function

the conjugate of a function f is

f ⇤(y) = supx2dom f

(yT x � f (x))

The conjugate function

the conjugate of a function f is

f∗(y) = supx∈dom f

(yTx − f(x))

PSfrag replacements

f(x)

(0,−f∗(y))

xy

x

• f∗ is convex (even if f is not)

• will be useful in chapter 5

Convex functions 3–21

I f ⇤ is convex (even if f is not)


Examples

I strictly convex quadratic f (x) = (1/2)xTQx with Q 2 Sn

++

f ⇤(y) = supx

(yT x � (1/2)xTQx)

=1

2yTQ�1y

I negative logarithm f (x) = � log x

f ⇤(y) = supx>0

(xy + log x)

=

⇢ �1 � log(�y) y < 01 otherwise

I log-determinant: f (X ) = log detX�1 on Sn

++.f ⇤(Y ) = log det(�Y )�1 � n with dom f ⇤ = �Sn

++.


Lagrange dual function and conjugate functionRecall: conjugate function f ⇤(y) = sup

x2dom f

(yT x � f (x))

minimize f0(x)subject to Ax � b, Cx = d

dual function

g(�, ⌫) = infx2dom f0

⇣f0(x) + (AT� + CT⌫)T x � bT� � dT⌫

⌘

= �f ⇤0 (�AT� � CT⌫) � bT� � dT⌫

example: minimum-volume covering ellipsoid

minimize f0(X ) = log detX�1 subject to aTi

Xai

1, i = 1, . . . ,m

I f ⇤0 (Y ) = log det(�Y )�1 � n with dom f ⇤0 = �Sn

++

I aTi

Xai

1 , tr((ai

aTi

)X ) 1

g(�) =

(log det(

Pm

i=1 �i

ai

aTi

) � 1T� + n,P

i

�i

ai

aTi

� 0,

�1 otherwise.


The dual problemLagrange dual problem

maximize g(�, ⌫)subject to � ⌫ 0

I finds best lower bound on p?, obtained from Lagrange dualfunction

I a convex optimization problem; optimal value denoted d?

I �, ⌫ are dual feasible if � ⌫ 0, (�, ⌫) 2 dom g

I often simplified by making implicit constraint (�, ⌫) 2 dom gexplicit

example: standard form LP and its dual

minimize cT xsubject to Ax = b

x ⌫ 0

maximize �bT⌫subject to AT⌫ + c ⌫ 0

IOE 611: Nonlinear Programming, Fall 2017 6. Duality –11

Weak and strong dualityweak duality: d? p?

I always holds (for convex and nonconvex problems), even if p?

and/or d? are infinite.

I can be used to find nontrivial lower bounds for di�cultproblemsfor example, solving the SDP

maximize �1T⌫subject to W + diag(⌫) ⌫ 0

gives a lower bound for the two-way partitioning problem

strong duality: d? = p?

I does not hold in general

I (usually) holds for convex problems

I conditions that guarantee strong duality in convex problemsare called constraint qualifications


Slater’s constraint qualification

strong duality holds for a convex problem


i

(x) 0, i = 1, . . . ,mAx = b

if it is strictly feasible, i.e.,

9x 2 intD : fi

(x) < 0, i = 1, . . . ,m, Ax = b

I also guarantees that the dual optimum is attained (ifp? > �1)

I can be sharpened: e.g., can replace intD with rel intD(interior relative to a�ne hull); linear inequalities do not needto hold with strict inequality, . . .

I there exist many other types of constraint qualifications


Inequality form LP

primal problemminimize cT xsubject to Ax � b

dual function

g(�) = infx

⇣(c + AT�)T x � bT�

⌘=

⇢ �bT� AT� + c = 0�1 otherwise

dual problem

maximize �bT�subject to AT� + c = 0, � ⌫ 0

I Convex problem with linear constraints, so p? = d?

I ... except when both primal and dual are infeasible


Quadratic program

primal problem (assume P 2 Sn

++)

minimize xTPxsubject to Ax � b

dual function

g(�) = infx

⇣xTPx + �T (Ax � b)

⌘= �1

4�TAP�1AT� � bT�

dual problem

maximize �(1/4)�TAP�1AT� � bT�subject to � ⌫ 0

I Convex problem with linear constraints, so p? = d?


A nonconvex problem with strong duality

minimize xTAx + 2bT xsubject to xT x 1

nonconvex if A 6⌫ 0dual function: g(�) = inf

x

(xT (A + �I )x + 2bT x � �)

I unbounded below if A + �I 6⌫ 0 or if A + �I ⌫ 0 andb 62 R(A + �I )

I minimized by x = �(A + �I )†b otherwise:g(�) = �bT (A + �I )†b � �

dual problem and equivalent SDP:

maximize �bT (A + �I )†b � �subject to A + �I ⌫ 0

b 2 R(A + �I )

maximize �t � �

subject to

A + �I bbT t

�⌫ 0

strong duality although primal problem is not convex (not easy toshow)


Geometric interpretationfor simplicity, consider problem with one constraint f1(x) 0interpretation of dual function:

g(�) = inf(u,t)2G

(t + �u), where G = {(f1(x), f0(x)) | x 2 D}

Geometric interpretation

for simplicity, consider problem with one constraint f1(x) ≤ 0

interpretation of dual function:

g(λ) = inf(u,t)∈G

(t + λu), where G = {(f1(x), f0(x)) | x ∈ D}

PSfrag replacementsG

f1(x) p⋆

g(λ)λu + t = g(λ)

t

u

PSfrag replacements

G

p⋆

d⋆

t

u

• λu + t = g(λ) is (non-vertical) supporting hyperplane to G• hyperplane intersects t-axis at t = g(λ)

Duality 5–15

I G is the set of values taken on by constraint and objectivefunctions

I For � ⌫ 0, �u + t = g(�) is (non-vertical) supportinghyperplane to G

I hyperplane intersects t-axis at t = g(�)IOE 611: Nonlinear Programming, Fall 2017 6. Duality Page 6–17

Epigraph variationSame interpretation if G is replaced with

A = {(u, t) | f1(x) u, f0(x) t for some x 2 D}epigraph variation: same interpretation if G is replaced with

A = {(u, t) | f1(x) u, f0(x) t for some x 2 D}

PSfrag replacementsA

f1(x)p⋆

g(λ)

λu + t = g(λ)

t

u

strong duality

• holds if there is a non-vertical supporting hyperplane to A at (0, p?)

• for convex problem, A is convex, hence has supp. hyperplanes at (0, p?)

• Slater’s condition: if there exist (u, t) 2 A with u < 0, then supportinghyperplanes at (0, p?) must be non-vertical

Duality 5–16

I A = {(u, v , t) | f (x) � u, h(x) = v , f0(x) t for some x 2 D}I p? = inf{t | (0, 0, t) 2 A}I g(�, ⌫) = inf{(�, ⌫, 1)T (u, v , t) | (u, v , t) 2 A}

I if the inf is finite, (�, ⌫, 1)T (u, v , t) � g(�, ⌫) is a non-verticalsupporting hyperplane to A (weak duality).


Strong duality

From the geometric interpretation with one inequality:

I Strong duality holds if there is a non-vertical supportinghyperplane to A at (0, p?)

I For convex problem, A is convex, hence has suppoprtinghyperplane at (0, p?)

I Slater’s condition: if there exist (u, t) 2 A with u < 0, thensupporting hyperplanes at (0, p?) must be non-vertical


Proof of strong duality under convexity and Slater’s CQTo simplify, assume:

I A has full row rank (wolog for feasible problems)

I 9x 2 intD: Ax = b, fi

(x) < 0, i = 1, . . . ,m (Slater point)

I p? is finite

Outline of the proof:

I A is convex; B = {(0, 0, s) | s < p?} is convex, A \ B = ;.

I There exists a hyperplane separating A and BI Interpretation: 9� � 0, µ � 0, ⌫:

�T f (x) + ⌫T (Ax � b) + µf0(x) � µp? 8x 2 D

I If µ > 0, g(�/µ, ⌫/µ) = p? — strong duality! (and we foundthe solution of the dual)

I If µ = 0, use the Slater point and the first assumption toderive the contradiction.


Complementary slacknessAssume strong duality holds, x? is primal optimal, (�?, ⌫?) is dualoptimal (not assuming primal problem is convex!)

f0(x?) = g(�?, ⌫?) = inf

x

f0(x) +

mX

i=1

�?i

fi

(x) +pX

i=1

⌫?i

hi

(x)

!

f0(x?) +

mX

i=1

�?i

fi

(x?) +pX

i=1

⌫?i

hi

(x?)

f0(x?)

hence, the two inequalities hold with equalityI x? minimizes L(x , �?, ⌫?)I �?

i

fi

(x?) = 0 for i = 1, . . . ,m (known as complementaryslackness):

�?i

> 0 =) fi

(x?) = 0, fi

(x?) < 0 =) �?i

= 0


Karush-Kuhn-Tucker (KKT) conditionsFor a problem with di↵erentiable f

i

’s and hi

’s:Assume strong duality holds, x is primal optimal, (�, ⌫) is dualoptimal (still no convexity assumed in the primal). Then:

1. primal constraints: fi

(x) 0, i = 1, . . . ,m, hi

(x) = 0,i = 1, . . . , p

2. dual constraints: � ⌫ 0

3. complementary slackness: �i

fi

(x) = 0, i = 1, . . . ,m

4. gradient of Lagrangian with respect to x vanishes:

rf0(x) +mX

i=1

�i

rfi

(x) +pX

i=1

⌫i

rhi

(x) = 0

The above four conditions are called KKT conditions.

If strong duality holds and x , �, ⌫ are optimal, then they mustsatisfy the KKT conditions


Implication of KKT conditions for convex problems

if x , �, ⌫ satisfy KKT for a convex problem, then they are optimal:

I from complementary slackness: f0(x) = L(x , �, ⌫)

I from 4th condition (and convexity): g(�, ⌫) = L(x , �, ⌫)

hence, f0(x) = g(�, ⌫)

If Slater’s condition is satisfied:x is optimal if and only if there exist �, ⌫ that satisfy KKTconditions

I recall that Slater implies strong duality, and dual optimum isattained

I generalizes optimality condition rf0(x) = 0 for unconstrainedproblem


Example(assume ↵

i

> 0)

minimize �Pn

i=1 log(xi

+ ↵i

)subject to x ⌫ 0, 1T x = 1

x is optimal i↵ x ⌫ 0, 1T x = 1, and there exist � 2 Rn, ⌫ 2 Rsuch that

� ⌫ 0, �i

xi

= 0,1

xi

+ ↵i

+ �i

= ⌫

I if ⌫ < 1/↵i

: �i

= 0 and xi

= 1/⌫ � ↵i

I if ⌫ � 1/↵i

: �i

= ⌫ � 1/↵i

and xi

= 0I determine ⌫ from 1T x =

Pn

i=1 max{0, 1/⌫ � ↵i

} = 1interpretation: water-filling

I n patches; level of patch i is at height ↵i

I flood area with unit amount of water

I resulting level is 1/⌫?

example: water-filling (assume αi > 0)

minimize −!n

i=1 log(xi + αi)subject to x ≽ 0, 1Tx = 1

x is optimal iff x ≽ 0, 1Tx = 1, and there exist λ ∈ Rn, ν ∈ R such that

λ ≽ 0, λixi = 0,1

xi + αi+ λi = ν

• if ν < 1/αi: λi = 0 and xi = 1/ν − αi

• if ν ≥ 1/αi: λi = ν − 1/αi and xi = 0

• determine ν from 1Tx =!n

i=1 max{0, 1/ν − αi} = 1

interpretation

• n patches; level of patch i is at height αi

• flood area with unit amount of water

• resulting level is 1/ν⋆

PSfrag replacements

i

1/ν⋆

xi

αi

Duality 5–20IOE 611: Nonlinear Programming, Fall 2017 6. Duality Page 6–24

Feasibility problems and alternative systemsfeasibility problem A (variables x 2 Rn)

fi

(x) < 0, i = 1, . . . ,m, hi

(x) = 0, i = 1, . . . , p

feasibility problem B (variables � 2 Rm, ⌫ 2 Rp)

� ⌫ 0, � 6= 0, g(�, ⌫) � 0

where g(�, ⌫) = infx

�Pm

i=1 �i

fi

(x) +P

p

i=1 ⌫i

hi

(x)�

I feasibility problem B is convex even if problem A is notI A and B are always weak alternatives: at most one is

feasibleproof: assume x satisfies A, �, ⌫ satisfy B

0 g(�, ⌫) mX

i=1

�i

fi

(x) +pX

i=1

⌫i

hi

(x) < 0

I A and B are strong alternatives if exactly one of the two isfeasible

I can prove infeasibility of A by producing solution of B andvice-versa


Strong alternatives: convex case

(A) fi

(x) < 0, i = 1, . . . ,m, hi

(x) = 0, i = 1, . . . , p(B) � ⌫ 0, � 6= 0, g(�, ⌫) � 0

I Suppose A is convex: fi

’s are convex, h(x) = Ax � b 2 Rp

I Suppose further 9x 2 rel intD : Ax = bI Theorem: A and B are strong alternatives

I Weak alternatives — already established.I Consider optimization problem (p? < 0 i↵ A has a solution)

minimizex,s s

subject to f (x) � 1s � 0Ax = b

I Dual (g(�, ⌫) as in B): attains d? = p?, due to Slater’s CQ

maximize�,⌫ g(�, ⌫)subject to � ⌫ 0, 1T� = 1

I If A is infeasible, d? � 0 and 9(�, ⌫) that satisfy BIOE 611: Nonlinear Programming, Fall 2017 6. Duality Page 6–26

Example: homogeneous linear inequalities

I (A) Fx < 0, Cx 0, Ax = 0I g(�, µ, ⌫) = inf

x

(FT� + CTµ + AT⌫)T x =(0 FT� + CTµ + AT⌫ = 0

�1 otherwise.

I (B) (�, µ) ⌫ 0, � 6= 0, FT� + CTµ + AT⌫ = 0

I Farkas’ lemma: Ax 0, cT x < 0 and AT y + c = 0, y ⌫ 0are strong alternatives


Example: first-order optimality conditions

minimize f0(x) subj. to fi

(x) 0, i = 1, . . . ,m, hi

(x) = 0, i = 1, . . . , p

I Assume di↵erentiable functions, do not assume convexityI Fritz-John necessary optimality conditions: if x is a local

minimizer, then 9(�0, �, ⌫) 6= 0 : (�0, �) ⌫ 0, �i

fi

(x) = 0 8i :

�0rf0(x) +mX

i=1

�i

rfi

(x) +pX

i=1

⌫i

rhi

(x) = 0

I If rhi

(x), i = 1, . . . , p are linearly dependent, set (�0, �) = 0I If rh

i

(x), i = 1, . . . , p are linearly independent,I let I ⌘ {i > 0 : f

i

(x) = 0}I then 6 9d : rf

i

(x)Td < 0, i 2 I [ {0}; rh

i

(x)Td = 0, 1 i p

I True i↵ 9(�0,�, ⌫) : (�0,�) ⌫ 0, (�0,�) 6= 0,

�0rf0(x) +mX

i=1

�i

rf

i

(x) +

pX

i=1

⌫i

rh

i

(x) = 0

I If also rhi

(x), i = 1, . . . , p, rfi

(x), i 2 I are lin. ind., then�0 > 0

I Result: KKT necessary optimality conditions (with a lin. ind. CQ)IOE 611: Nonlinear Programming, Fall 2017 6. Duality Page 6–28

Duality and problem reformulations

I equivalent formulations of a problem can lead to very di↵erentduals

I reformulating the primal problem can be useful when the dualis di�cult to derive, or uninteresting

Common reformulations

I introduce new variables and equality constraints

I make explicit constraints implicit or vice-versa

I transform objective or constraint functionse.g., replace f0(x) by �(f0(x)) with � convex, increasing


Introducing new variables and equality constraints

minimize f0(Ax + b)

I dual function is constant:g = inf

x

L(x) = infx

f0(Ax + b) = p?

I we have strong duality, but dual is quite useless

Reformulated problem and its dual

minimize f0(y)subject to Ax + b � y = 0

maximize bT⌫ � f ⇤0 (⌫)subject to AT⌫ = 0

Dual function follows from

g(⌫) = infx ,y

(f0(y) � ⌫T y + ⌫TAx + bT⌫)

=

⇢ �f ⇤0 (⌫) + bT⌫ AT⌫ = 0�1 otherwise


Norm approximation problem:minimize kAx � bk

minimize kyksubject to y = Ax � b

can look up conjugate of k · k, or derive dual directly

g(⌫) = infx ,y

(kyk + ⌫T y � ⌫TAx + bT⌫)

=

⇢bT⌫ + inf

y

(kyk + ⌫T y) AT⌫ = 0�1 otherwise

=

⇢bT⌫ AT⌫ = 0, k⌫k⇤ 1�1 otherwise

(see page 5-4)Dual of norm approximation problem

maximize bT⌫subject to AT⌫ = 0, k⌫k⇤ 1


Implicit constraintsLP with box constraints: primal and dual problem

minimize cT xsubject to Ax = b

�1 � x � 1

maximize �bT⌫ � 1T�1 � 1T�2

subject to c + AT⌫ + �1 � �2 = 0�1 ⌫ 0, �2 ⌫ 0

Reformulation with box constraints made implicit

minimize f0(x) =

⇢cT x �1 � x � 11 otherwise

subject to Ax = b

Dual function

g(⌫) = inf�1�x�1

(cT x + ⌫T (Ax � b))

= �bT⌫ � kAT⌫ + ck1Dual problem: maximize �bT⌫ � kAT⌫ + ck1


Problems with generalized inequalities


i

(x) �K

i

0, i = 1, . . . ,mhi

(x) = 0, i = 1, . . . , p

�K

i

is generalized inequality on Rk

i

Definitions are parallel to scalar case:

I Lagrange multiplier for fi

(x) �K

i

0 is vector �i

2 Rk

i

I Lagrangian L : Rn ⇥ Rk1 ⇥ · · · ⇥ Rk

m ⇥ Rp ! R, is defined as

L(x , �1, · · · , �m

, ⌫) = f0(x) +mX

i=1

�T

i

fi

(x) +pX

i=1

⌫i

hi

(x)

I dual function g : Rk1 ⇥ · · · ⇥ Rk

m ⇥ Rp ! R, is defined as

g(�1, . . . , �m

, ⌫) = infx2D

L(x , �1, · · · , �m

, ⌫)


Lower bound property: if �i

⌫K

⇤i

0, then g(�1, . . . , �m

, ⌫) p?

Proof: if x is feasible and � ⌫K

⇤i

0, then

f0(x) � f0(x) +mX

i=1

�T

i

fi

(x) +pX

i=1

⌫i

hi

(x)

� infx2D

L(x , �1, . . . , �m

, ⌫)

= g(�1, . . . , �m

, ⌫)

minimizing over all feasible x gives p? � g(�1, . . . , �m

, ⌫)Dual problem

maximize g(�1, . . . , �m

, ⌫)subject to �

i

⌫K

⇤i

0, i = 1, . . . ,m

I weak duality: p? � d? alwaysI strong duality: p? = d? for convex problem with constraint

qualification(for example, Slater’s: primal problem is strictly feasible)


Semidefinite programprimal SDP (F

i

,G 2 Sk)

minimize cT xsubject to x1F1 + · · · + x

n

Fn

� G

I Lagrange multiplier is matrix Z 2 Sk

I Lagrangian L(x ,Z ) = cT x + tr (Z (x1F1 + · · · + xn

Fn

� G ))I dual function

g(Z ) = infx

L(x ,Z ) =

(� tr(GZ ) tr(F

i

Z ) + ci

= 0, i = 1, . . . , n,

�1 otherwise.

dual SDP

maximize � tr(GZ )subject to Z ⌫ 0, tr(F

i

Z ) + ci

= 0, i = 1, . . . , n

p? = d? if primal SDP is strictly feasible (9x withx1F1 + · · · + x

n

Fn

� G )IOE 611: Nonlinear Programming, Fall 2017 6. Duality Page 6–35

Alternative systems with generalized inequalitiesI Suppose f

i

(x)’s are Ki

-convex (Ki

’s are proper cones) andhi

(x) are a�ne:

(A) fi

(x) �K

i

0, i = 1, . . . ,m, Ax = b

I g(�, ⌫) = infx2D(

Pm

i=1 �T

i

fi

(x) + ⌫T (Ax � b))

(B) �i

⌫K

?i

0, i = 1, . . . ,m, � 6= 0, g(�, ⌫) � 0

I A and B are weak alternatives (easy to show)I If, in addition, 9x 2 rel intD : Ax = b, they are strong

alternativesI To prove, consider optimization problem (e

i

�K

i

0 8i)minimize

x ,s ssubject to f

i

(x) �K

i

sei

, i = 1, . . . ,mAx = b


Example: feasibility of an LMI

(A) F (x) = x1F1 + · · · + xn

Fn

+ G � 0

(B) Z ⌫ 0, Z 6= 0, tr(GZ ) � 0, tr(Fi

Z ) = 0, i = 1, . . . , n

I The above systems are strong alternativesI Similar analysis can be done for A with non-strict inequality:

I Need conditions on matrices Fi

, e.g.,

nX

i=1

vi

Fi

⌫ 0 =)nX

i=1

vi

Fi

= 0

I If this condition holds, the following are strong alternatives:

(A) F (x) = x1F1 + · · · + xn

Fn

+ G � 0

(B) Z ⌫ 0, tr(GZ ) > 0, tr(Fi

Z ) = 0, i = 1, . . . , n


Perturbation and sensitivity analysis(Unperturbed) optimization problem and its dual


i

(x) 0, i = 1, . . . ,mhi

(x) = 0, i = 1, . . . , p

maximize g(�, ⌫)subject to � ⌫ 0

Perturbed problem and its dual

min. f0(x)s.t. f

i

(x) ui

, i = 1, . . . ,mhi

(x) = vi

, i = 1, . . . , p

max. g(�, ⌫) � uT� � vT⌫s.t. � ⌫ 0

I x is primal variable; u, v are parametersI p?(u, v) is optimal value as a function of (u, v)

(p?(0, 0) = p?)I we are interested in information about p?(u, v) that we can

obtain from the solution of the unperturbed problem and itsdual


Global sensitivity resultAssume strong duality holds for unperturbed problem, and that �?,⌫? are dual optimal for unperturbed problem.Apply weak duality to perturbed problem:

p?(u, v) � g(�?, ⌫?) � uT�? � vT⌫?

= p?(0, 0) � uT�? � vT⌫?

sensitivity interpretationI if �?

i

large: p? increases greatly if we tighten constraint i(u

i

< 0)I if �?

i

small: p? does not decrease much if we loosen constrainti (u

i

> 0)I if ⌫?

i

large and positive: p? increases greatly if we take vi

< 0;if ⌫?

i

large and negative: p? increases greatly if we take vi

> 0I if ⌫?

i

small and positive: p? does not decrease much if we takevi

> 0;if ⌫?

i

small and negative: p? does not decrease much if wetake v

i

< 0IOE 611: Nonlinear Programming, Fall 2017 6. Duality Page 6–39

Local sensitivity:If (in addition) p?(u, v) is di↵erentiable at (0, 0), then

�?i

= �@p?(0, 0)

@ui

, ⌫?i

= �@p?(0, 0)

@vi

Proof (for �?i

): from global sensitivity result,

@p?(0, 0)

@ui

= limt&0

p?(tei

, 0) � p?(0, 0)

t� ��?

i

@p?(0, 0)

@ui

= limt%0

p?(tei

, 0) � p?(0, 0)

t ��?

i

hence, equality.

p?(u) for a problem with one(inequality) constraint:

local sensitivity: if (in addition) p⋆(u, v) is differentiable at (0, 0), then

λ⋆i = −∂p⋆(0, 0)

∂ui, ν⋆

i = −∂p⋆(0, 0)

∂vi

proof (for λ⋆i ): from global sensitivity result,

∂p⋆(0, 0)

∂ui= lim

t↘0

p⋆(tei, 0) − p⋆(0, 0)

t≥ −λ⋆

i

∂p⋆(0, 0)

∂ui= lim

t↗0

p⋆(tei, 0) − p⋆(0, 0)

t≤ −λ⋆

i

hence, equality

p⋆(u) for a problem with one (inequality)constraint:

PSfrag replacements

up⋆(u)

p⋆(0) − λ⋆u

u = 0

Duality 5–23IOE 611: Nonlinear Programming, Fall 2017 6. Duality Page 6–40

duality - u-m personal world wide web servermepelman/teaching/ioe611/handouts/duali… · duality i...

Documents