lecture 5: conjugate duality · duality theorems for linear cp • theorem 3 (conic duality theorem...
TRANSCRIPT
![Page 1: LECTURE 5: CONJUGATE DUALITY · Duality theorems for linear CP • Theorem 3 (Conic Duality Theorem ): (i) If problems (CP) and (CD) are both feasible, then they . have optimal solutions](https://reader033.vdocuments.mx/reader033/viewer/2022052612/5f11d7ff2425a142cf725fb0/html5/thumbnails/1.jpg)
LECTURE 5: CONJUGATE DUALITY 1. Primal problem and its conjugate dual 2. Duality theory and optimality conditions 3. Relation to other type of dual problems 4. Linear conic programming problems
![Page 2: LECTURE 5: CONJUGATE DUALITY · Duality theorems for linear CP • Theorem 3 (Conic Duality Theorem ): (i) If problems (CP) and (CD) are both feasible, then they . have optimal solutions](https://reader033.vdocuments.mx/reader033/viewer/2022052612/5f11d7ff2425a142cf725fb0/html5/thumbnails/2.jpg)
Motivation of conjugate duality Min f(x) (over R) = Max g(y) (over R) and h(y) = - g(y) • f(x) + h(y) = f(x) – g(y) can be viewed as “duality gap” • Would like to have (i) weak duality 0 (ii) strong duality 0 = f(x*) + h(y*)
![Page 3: LECTURE 5: CONJUGATE DUALITY · Duality theorems for linear CP • Theorem 3 (Conic Duality Theorem ): (i) If problems (CP) and (CD) are both feasible, then they . have optimal solutions](https://reader033.vdocuments.mx/reader033/viewer/2022052612/5f11d7ff2425a142cf725fb0/html5/thumbnails/3.jpg)
Where is the duality information • Recall the fundamental result of Fenchel’s conjugate inequality
• Need a structure such that in general
and at optimal solutions 0 = <x*, y*> = f(x*) + h(y*)
![Page 4: LECTURE 5: CONJUGATE DUALITY · Duality theorems for linear CP • Theorem 3 (Conic Duality Theorem ): (i) If problems (CP) and (CD) are both feasible, then they . have optimal solutions](https://reader033.vdocuments.mx/reader033/viewer/2022052612/5f11d7ff2425a142cf725fb0/html5/thumbnails/4.jpg)
Concept of dual cone • Let X be a cone in
• Define its dual cone • Properties: (i) Y is a cone. (ii) Y is a convex set. (iii) Y is a closed set.
![Page 5: LECTURE 5: CONJUGATE DUALITY · Duality theorems for linear CP • Theorem 3 (Conic Duality Theorem ): (i) If problems (CP) and (CD) are both feasible, then they . have optimal solutions](https://reader033.vdocuments.mx/reader033/viewer/2022052612/5f11d7ff2425a142cf725fb0/html5/thumbnails/5.jpg)
Observations
![Page 6: LECTURE 5: CONJUGATE DUALITY · Duality theorems for linear CP • Theorem 3 (Conic Duality Theorem ): (i) If problems (CP) and (CD) are both feasible, then they . have optimal solutions](https://reader033.vdocuments.mx/reader033/viewer/2022052612/5f11d7ff2425a142cf725fb0/html5/thumbnails/6.jpg)
Conjugate (Geometric) duality
![Page 7: LECTURE 5: CONJUGATE DUALITY · Duality theorems for linear CP • Theorem 3 (Conic Duality Theorem ): (i) If problems (CP) and (CD) are both feasible, then they . have optimal solutions](https://reader033.vdocuments.mx/reader033/viewer/2022052612/5f11d7ff2425a142cf725fb0/html5/thumbnails/7.jpg)
Dual side information • Conjugate dual function
• Dual cone
Properties: 1. Y is a cone in 2. Y is closed and convex 3. both are closed and convex.
![Page 8: LECTURE 5: CONJUGATE DUALITY · Duality theorems for linear CP • Theorem 3 (Conic Duality Theorem ): (i) If problems (CP) and (CD) are both feasible, then they . have optimal solutions](https://reader033.vdocuments.mx/reader033/viewer/2022052612/5f11d7ff2425a142cf725fb0/html5/thumbnails/8.jpg)
Conjugate (Geometric) dual problem
![Page 9: LECTURE 5: CONJUGATE DUALITY · Duality theorems for linear CP • Theorem 3 (Conic Duality Theorem ): (i) If problems (CP) and (CD) are both feasible, then they . have optimal solutions](https://reader033.vdocuments.mx/reader033/viewer/2022052612/5f11d7ff2425a142cf725fb0/html5/thumbnails/9.jpg)
Observations
![Page 10: LECTURE 5: CONJUGATE DUALITY · Duality theorems for linear CP • Theorem 3 (Conic Duality Theorem ): (i) If problems (CP) and (CD) are both feasible, then they . have optimal solutions](https://reader033.vdocuments.mx/reader033/viewer/2022052612/5f11d7ff2425a142cf725fb0/html5/thumbnails/10.jpg)
Conjugate duality theory
![Page 11: LECTURE 5: CONJUGATE DUALITY · Duality theorems for linear CP • Theorem 3 (Conic Duality Theorem ): (i) If problems (CP) and (CD) are both feasible, then they . have optimal solutions](https://reader033.vdocuments.mx/reader033/viewer/2022052612/5f11d7ff2425a142cf725fb0/html5/thumbnails/11.jpg)
Conjugate duality theory
![Page 12: LECTURE 5: CONJUGATE DUALITY · Duality theorems for linear CP • Theorem 3 (Conic Duality Theorem ): (i) If problems (CP) and (CD) are both feasible, then they . have optimal solutions](https://reader033.vdocuments.mx/reader033/viewer/2022052612/5f11d7ff2425a142cf725fb0/html5/thumbnails/12.jpg)
Proof
![Page 13: LECTURE 5: CONJUGATE DUALITY · Duality theorems for linear CP • Theorem 3 (Conic Duality Theorem ): (i) If problems (CP) and (CD) are both feasible, then they . have optimal solutions](https://reader033.vdocuments.mx/reader033/viewer/2022052612/5f11d7ff2425a142cf725fb0/html5/thumbnails/13.jpg)
Conjugate duality theory
![Page 14: LECTURE 5: CONJUGATE DUALITY · Duality theorems for linear CP • Theorem 3 (Conic Duality Theorem ): (i) If problems (CP) and (CD) are both feasible, then they . have optimal solutions](https://reader033.vdocuments.mx/reader033/viewer/2022052612/5f11d7ff2425a142cf725fb0/html5/thumbnails/14.jpg)
Example – Standard form LP
![Page 15: LECTURE 5: CONJUGATE DUALITY · Duality theorems for linear CP • Theorem 3 (Conic Duality Theorem ): (i) If problems (CP) and (CD) are both feasible, then they . have optimal solutions](https://reader033.vdocuments.mx/reader033/viewer/2022052612/5f11d7ff2425a142cf725fb0/html5/thumbnails/15.jpg)
Conjugate dual problem
![Page 16: LECTURE 5: CONJUGATE DUALITY · Duality theorems for linear CP • Theorem 3 (Conic Duality Theorem ): (i) If problems (CP) and (CD) are both feasible, then they . have optimal solutions](https://reader033.vdocuments.mx/reader033/viewer/2022052612/5f11d7ff2425a142cf725fb0/html5/thumbnails/16.jpg)
Dual LP
![Page 17: LECTURE 5: CONJUGATE DUALITY · Duality theorems for linear CP • Theorem 3 (Conic Duality Theorem ): (i) If problems (CP) and (CD) are both feasible, then they . have optimal solutions](https://reader033.vdocuments.mx/reader033/viewer/2022052612/5f11d7ff2425a142cf725fb0/html5/thumbnails/17.jpg)
Example – Karmarkar form LP
![Page 18: LECTURE 5: CONJUGATE DUALITY · Duality theorems for linear CP • Theorem 3 (Conic Duality Theorem ): (i) If problems (CP) and (CD) are both feasible, then they . have optimal solutions](https://reader033.vdocuments.mx/reader033/viewer/2022052612/5f11d7ff2425a142cf725fb0/html5/thumbnails/18.jpg)
Example – Karmarkar form LP
![Page 19: LECTURE 5: CONJUGATE DUALITY · Duality theorems for linear CP • Theorem 3 (Conic Duality Theorem ): (i) If problems (CP) and (CD) are both feasible, then they . have optimal solutions](https://reader033.vdocuments.mx/reader033/viewer/2022052612/5f11d7ff2425a142cf725fb0/html5/thumbnails/19.jpg)
Example – Karmarkar form LP • Conjugate dual problem becomes
which is an unconstrained convex programming problem.
![Page 20: LECTURE 5: CONJUGATE DUALITY · Duality theorems for linear CP • Theorem 3 (Conic Duality Theorem ): (i) If problems (CP) and (CD) are both feasible, then they . have optimal solutions](https://reader033.vdocuments.mx/reader033/viewer/2022052612/5f11d7ff2425a142cf725fb0/html5/thumbnails/20.jpg)
Illustration
![Page 21: LECTURE 5: CONJUGATE DUALITY · Duality theorems for linear CP • Theorem 3 (Conic Duality Theorem ): (i) If problems (CP) and (CD) are both feasible, then they . have optimal solutions](https://reader033.vdocuments.mx/reader033/viewer/2022052612/5f11d7ff2425a142cf725fb0/html5/thumbnails/21.jpg)
Example – Posynomial programming • Nonconvex programming problem
• Transformation
![Page 22: LECTURE 5: CONJUGATE DUALITY · Duality theorems for linear CP • Theorem 3 (Conic Duality Theorem ): (i) If problems (CP) and (CD) are both feasible, then they . have optimal solutions](https://reader033.vdocuments.mx/reader033/viewer/2022052612/5f11d7ff2425a142cf725fb0/html5/thumbnails/22.jpg)
Posynomial programming • Primal problem: Conjugate dual problem:
![Page 23: LECTURE 5: CONJUGATE DUALITY · Duality theorems for linear CP • Theorem 3 (Conic Duality Theorem ): (i) If problems (CP) and (CD) are both feasible, then they . have optimal solutions](https://reader033.vdocuments.mx/reader033/viewer/2022052612/5f11d7ff2425a142cf725fb0/html5/thumbnails/23.jpg)
Conjugate dual problem
![Page 24: LECTURE 5: CONJUGATE DUALITY · Duality theorems for linear CP • Theorem 3 (Conic Duality Theorem ): (i) If problems (CP) and (CD) are both feasible, then they . have optimal solutions](https://reader033.vdocuments.mx/reader033/viewer/2022052612/5f11d7ff2425a142cf725fb0/html5/thumbnails/24.jpg)
Degree of difficulties • When degree of difficulty = 0, we have a system of linear
equations:
• When degree of difficulty = k, we have
![Page 25: LECTURE 5: CONJUGATE DUALITY · Duality theorems for linear CP • Theorem 3 (Conic Duality Theorem ): (i) If problems (CP) and (CD) are both feasible, then they . have optimal solutions](https://reader033.vdocuments.mx/reader033/viewer/2022052612/5f11d7ff2425a142cf725fb0/html5/thumbnails/25.jpg)
Duality gap • Definition:
• Observation:
![Page 26: LECTURE 5: CONJUGATE DUALITY · Duality theorems for linear CP • Theorem 3 (Conic Duality Theorem ): (i) If problems (CP) and (CD) are both feasible, then they . have optimal solutions](https://reader033.vdocuments.mx/reader033/viewer/2022052612/5f11d7ff2425a142cf725fb0/html5/thumbnails/26.jpg)
Extremality conditions • Definition:
• Corollary:
![Page 27: LECTURE 5: CONJUGATE DUALITY · Duality theorems for linear CP • Theorem 3 (Conic Duality Theorem ): (i) If problems (CP) and (CD) are both feasible, then they . have optimal solutions](https://reader033.vdocuments.mx/reader033/viewer/2022052612/5f11d7ff2425a142cf725fb0/html5/thumbnails/27.jpg)
Proof of Corollary
![Page 28: LECTURE 5: CONJUGATE DUALITY · Duality theorems for linear CP • Theorem 3 (Conic Duality Theorem ): (i) If problems (CP) and (CD) are both feasible, then they . have optimal solutions](https://reader033.vdocuments.mx/reader033/viewer/2022052612/5f11d7ff2425a142cf725fb0/html5/thumbnails/28.jpg)
Necessary and sufficient conditions • Corollary
• Observation
![Page 29: LECTURE 5: CONJUGATE DUALITY · Duality theorems for linear CP • Theorem 3 (Conic Duality Theorem ): (i) If problems (CP) and (CD) are both feasible, then they . have optimal solutions](https://reader033.vdocuments.mx/reader033/viewer/2022052612/5f11d7ff2425a142cf725fb0/html5/thumbnails/29.jpg)
When will the duality gap vanish?
![Page 30: LECTURE 5: CONJUGATE DUALITY · Duality theorems for linear CP • Theorem 3 (Conic Duality Theorem ): (i) If problems (CP) and (CD) are both feasible, then they . have optimal solutions](https://reader033.vdocuments.mx/reader033/viewer/2022052612/5f11d7ff2425a142cf725fb0/html5/thumbnails/30.jpg)
Nonlinear complementarity problem
![Page 31: LECTURE 5: CONJUGATE DUALITY · Duality theorems for linear CP • Theorem 3 (Conic Duality Theorem ): (i) If problems (CP) and (CD) are both feasible, then they . have optimal solutions](https://reader033.vdocuments.mx/reader033/viewer/2022052612/5f11d7ff2425a142cf725fb0/html5/thumbnails/31.jpg)
Lagrangian function and duality • Definition:
![Page 32: LECTURE 5: CONJUGATE DUALITY · Duality theorems for linear CP • Theorem 3 (Conic Duality Theorem ): (i) If problems (CP) and (CD) are both feasible, then they . have optimal solutions](https://reader033.vdocuments.mx/reader033/viewer/2022052612/5f11d7ff2425a142cf725fb0/html5/thumbnails/32.jpg)
Observations
![Page 33: LECTURE 5: CONJUGATE DUALITY · Duality theorems for linear CP • Theorem 3 (Conic Duality Theorem ): (i) If problems (CP) and (CD) are both feasible, then they . have optimal solutions](https://reader033.vdocuments.mx/reader033/viewer/2022052612/5f11d7ff2425a142cf725fb0/html5/thumbnails/33.jpg)
Observations
![Page 34: LECTURE 5: CONJUGATE DUALITY · Duality theorems for linear CP • Theorem 3 (Conic Duality Theorem ): (i) If problems (CP) and (CD) are both feasible, then they . have optimal solutions](https://reader033.vdocuments.mx/reader033/viewer/2022052612/5f11d7ff2425a142cf725fb0/html5/thumbnails/34.jpg)
Saddle point theorem
![Page 35: LECTURE 5: CONJUGATE DUALITY · Duality theorems for linear CP • Theorem 3 (Conic Duality Theorem ): (i) If problems (CP) and (CD) are both feasible, then they . have optimal solutions](https://reader033.vdocuments.mx/reader033/viewer/2022052612/5f11d7ff2425a142cf725fb0/html5/thumbnails/35.jpg)
Observations
![Page 36: LECTURE 5: CONJUGATE DUALITY · Duality theorems for linear CP • Theorem 3 (Conic Duality Theorem ): (i) If problems (CP) and (CD) are both feasible, then they . have optimal solutions](https://reader033.vdocuments.mx/reader033/viewer/2022052612/5f11d7ff2425a142cf725fb0/html5/thumbnails/36.jpg)
Observations
![Page 37: LECTURE 5: CONJUGATE DUALITY · Duality theorems for linear CP • Theorem 3 (Conic Duality Theorem ): (i) If problems (CP) and (CD) are both feasible, then they . have optimal solutions](https://reader033.vdocuments.mx/reader033/viewer/2022052612/5f11d7ff2425a142cf725fb0/html5/thumbnails/37.jpg)
Proof of saddle point theorem
![Page 38: LECTURE 5: CONJUGATE DUALITY · Duality theorems for linear CP • Theorem 3 (Conic Duality Theorem ): (i) If problems (CP) and (CD) are both feasible, then they . have optimal solutions](https://reader033.vdocuments.mx/reader033/viewer/2022052612/5f11d7ff2425a142cf725fb0/html5/thumbnails/38.jpg)
Linear conic programming problems • Linear Conic Programming (LCoP) A general conic optimization problem is as follows:
( )
where is a closed and convex co
minimize subject to
" " is a linear operator line an
ke "inner product."d
P c xA x b
x∈=
![Page 39: LECTURE 5: CONJUGATE DUALITY · Duality theorems for linear CP • Theorem 3 (Conic Duality Theorem ): (i) If problems (CP) and (CD) are both feasible, then they . have optimal solutions](https://reader033.vdocuments.mx/reader033/viewer/2022052612/5f11d7ff2425a142cf725fb0/html5/thumbnails/39.jpg)
Dual linear conic dual problem
![Page 40: LECTURE 5: CONJUGATE DUALITY · Duality theorems for linear CP • Theorem 3 (Conic Duality Theorem ): (i) If problems (CP) and (CD) are both feasible, then they . have optimal solutions](https://reader033.vdocuments.mx/reader033/viewer/2022052612/5f11d7ff2425a142cf725fb0/html5/thumbnails/40.jpg)
Duality theorems for Linear CP
![Page 41: LECTURE 5: CONJUGATE DUALITY · Duality theorems for linear CP • Theorem 3 (Conic Duality Theorem ): (i) If problems (CP) and (CD) are both feasible, then they . have optimal solutions](https://reader033.vdocuments.mx/reader033/viewer/2022052612/5f11d7ff2425a142cf725fb0/html5/thumbnails/41.jpg)
Duality theorems for linear CP • Theorem 3 (Conic Duality Theorem): (i) If problems (CP) and (CD) are both feasible, then they have optimal solutions. (ii) If one of the two problems has an interior feasible solution with a finite optimal objective value, then the other one is feasible and has the same optimal objective value. (iii) If one of the two problems is unbounded, then the other has no feasible solution. (iv) If (CP) and (CD) both have interior feasible solutions, then they have optimal solutions with zero duality gap.
![Page 42: LECTURE 5: CONJUGATE DUALITY · Duality theorems for linear CP • Theorem 3 (Conic Duality Theorem ): (i) If problems (CP) and (CD) are both feasible, then they . have optimal solutions](https://reader033.vdocuments.mx/reader033/viewer/2022052612/5f11d7ff2425a142cf725fb0/html5/thumbnails/42.jpg)
Examples of conic programs
![Page 43: LECTURE 5: CONJUGATE DUALITY · Duality theorems for linear CP • Theorem 3 (Conic Duality Theorem ): (i) If problems (CP) and (CD) are both feasible, then they . have optimal solutions](https://reader033.vdocuments.mx/reader033/viewer/2022052612/5f11d7ff2425a142cf725fb0/html5/thumbnails/43.jpg)
Example of conic programs
![Page 44: LECTURE 5: CONJUGATE DUALITY · Duality theorems for linear CP • Theorem 3 (Conic Duality Theorem ): (i) If problems (CP) and (CD) are both feasible, then they . have optimal solutions](https://reader033.vdocuments.mx/reader033/viewer/2022052612/5f11d7ff2425a142cf725fb0/html5/thumbnails/44.jpg)
Example of conic programs
![Page 45: LECTURE 5: CONJUGATE DUALITY · Duality theorems for linear CP • Theorem 3 (Conic Duality Theorem ): (i) If problems (CP) and (CD) are both feasible, then they . have optimal solutions](https://reader033.vdocuments.mx/reader033/viewer/2022052612/5f11d7ff2425a142cf725fb0/html5/thumbnails/45.jpg)
Semi-definite Programming (SDP)
![Page 46: LECTURE 5: CONJUGATE DUALITY · Duality theorems for linear CP • Theorem 3 (Conic Duality Theorem ): (i) If problems (CP) and (CD) are both feasible, then they . have optimal solutions](https://reader033.vdocuments.mx/reader033/viewer/2022052612/5f11d7ff2425a142cf725fb0/html5/thumbnails/46.jpg)
Duality theorems for SDP
![Page 47: LECTURE 5: CONJUGATE DUALITY · Duality theorems for linear CP • Theorem 3 (Conic Duality Theorem ): (i) If problems (CP) and (CD) are both feasible, then they . have optimal solutions](https://reader033.vdocuments.mx/reader033/viewer/2022052612/5f11d7ff2425a142cf725fb0/html5/thumbnails/47.jpg)
Quadratic programming problem
![Page 48: LECTURE 5: CONJUGATE DUALITY · Duality theorems for linear CP • Theorem 3 (Conic Duality Theorem ): (i) If problems (CP) and (CD) are both feasible, then they . have optimal solutions](https://reader033.vdocuments.mx/reader033/viewer/2022052612/5f11d7ff2425a142cf725fb0/html5/thumbnails/48.jpg)
Conjugate dual QP
![Page 49: LECTURE 5: CONJUGATE DUALITY · Duality theorems for linear CP • Theorem 3 (Conic Duality Theorem ): (i) If problems (CP) and (CD) are both feasible, then they . have optimal solutions](https://reader033.vdocuments.mx/reader033/viewer/2022052612/5f11d7ff2425a142cf725fb0/html5/thumbnails/49.jpg)
Conjugate dual QP
![Page 50: LECTURE 5: CONJUGATE DUALITY · Duality theorems for linear CP • Theorem 3 (Conic Duality Theorem ): (i) If problems (CP) and (CD) are both feasible, then they . have optimal solutions](https://reader033.vdocuments.mx/reader033/viewer/2022052612/5f11d7ff2425a142cf725fb0/html5/thumbnails/50.jpg)
Conjugate dual QP