cs672: approximation algorithms spring 14 introduction to...
TRANSCRIPT
![Page 1: CS672: Approximation Algorithms Spring 14 Introduction to ...shaddin/cs672sp14/slides/LP1.pdfIntroduction to Linear Programming I Instructor: Shaddin Dughmi. Outline 1 Linear Programming](https://reader034.vdocuments.mx/reader034/viewer/2022050307/5f6fd48a7d5b0c46f4252f29/html5/thumbnails/1.jpg)
CS672: Approximation AlgorithmsSpring 14
Introduction to Linear Programming I
Instructor: Shaddin Dughmi
![Page 2: CS672: Approximation Algorithms Spring 14 Introduction to ...shaddin/cs672sp14/slides/LP1.pdfIntroduction to Linear Programming I Instructor: Shaddin Dughmi. Outline 1 Linear Programming](https://reader034.vdocuments.mx/reader034/viewer/2022050307/5f6fd48a7d5b0c46f4252f29/html5/thumbnails/2.jpg)
Outline
1 Linear Programming
2 Application to Combinatorial Problems
3 Duality and Its Interpretations
4 Properties of Duals
![Page 3: CS672: Approximation Algorithms Spring 14 Introduction to ...shaddin/cs672sp14/slides/LP1.pdfIntroduction to Linear Programming I Instructor: Shaddin Dughmi. Outline 1 Linear Programming](https://reader034.vdocuments.mx/reader034/viewer/2022050307/5f6fd48a7d5b0c46f4252f29/html5/thumbnails/3.jpg)
Outline
1 Linear Programming
2 Application to Combinatorial Problems
3 Duality and Its Interpretations
4 Properties of Duals
![Page 4: CS672: Approximation Algorithms Spring 14 Introduction to ...shaddin/cs672sp14/slides/LP1.pdfIntroduction to Linear Programming I Instructor: Shaddin Dughmi. Outline 1 Linear Programming](https://reader034.vdocuments.mx/reader034/viewer/2022050307/5f6fd48a7d5b0c46f4252f29/html5/thumbnails/4.jpg)
A Brief History
The forefather of convex optimization problems, and the mostubiquitous.
Best understood in that contextBut this is not a convex optimization class
Developed by Kantorovich during World War II (1939) for planningthe Soviet army’s expenditures and returns. Kept secret.Discovered a few years later by George Dantzig, who in 1947developed the simplex method for solving linear programsJohn von Neumann developed LP duality in 1947, and applied it togame theoryPolynomial-time solvable under fairly general conditions
Ellipsoid method (Khachiyan 1979)Interior point methods (Karmarkar 1984).
Linear Programming 1/24
![Page 5: CS672: Approximation Algorithms Spring 14 Introduction to ...shaddin/cs672sp14/slides/LP1.pdfIntroduction to Linear Programming I Instructor: Shaddin Dughmi. Outline 1 Linear Programming](https://reader034.vdocuments.mx/reader034/viewer/2022050307/5f6fd48a7d5b0c46f4252f29/html5/thumbnails/5.jpg)
LP General Form
minimize (or maximize) cᵀxsubject to aᵀi x ≤ bi, for i ∈ C1.
aᵀi x ≥ bi, for i ∈ C2.aᵀi x = bi, for i ∈ C3.
Decision variables: x ∈ Rn
Parameters:c ∈ Rn defines the linear objective functionai ∈ Rn and bi ∈ R define the i’th constraint
Linear Programming 2/24
![Page 6: CS672: Approximation Algorithms Spring 14 Introduction to ...shaddin/cs672sp14/slides/LP1.pdfIntroduction to Linear Programming I Instructor: Shaddin Dughmi. Outline 1 Linear Programming](https://reader034.vdocuments.mx/reader034/viewer/2022050307/5f6fd48a7d5b0c46f4252f29/html5/thumbnails/6.jpg)
Standard Forms
Packing Formmaximize cᵀxsubject to Ax � b
x � 0
Covering Formminimize cᵀxsubject to Ax � b
x � 0
Every LP can be transformed to either formminimizing cᵀx is equivalent to maximizing −cᵀxinequality constraints can be flipped by multiplying by −1Each equality constraint can be replaced by two inequalitiesUconstrained variable xj can be replaced by x+j − x
−j , where both
x+j and x−j are constrained to be nonnegative.
Linear Programming 3/24
![Page 7: CS672: Approximation Algorithms Spring 14 Introduction to ...shaddin/cs672sp14/slides/LP1.pdfIntroduction to Linear Programming I Instructor: Shaddin Dughmi. Outline 1 Linear Programming](https://reader034.vdocuments.mx/reader034/viewer/2022050307/5f6fd48a7d5b0c46f4252f29/html5/thumbnails/7.jpg)
Geometric View
Linear Programming 4/24
![Page 8: CS672: Approximation Algorithms Spring 14 Introduction to ...shaddin/cs672sp14/slides/LP1.pdfIntroduction to Linear Programming I Instructor: Shaddin Dughmi. Outline 1 Linear Programming](https://reader034.vdocuments.mx/reader034/viewer/2022050307/5f6fd48a7d5b0c46f4252f29/html5/thumbnails/8.jpg)
Geometric View
Linear Programming 4/24
![Page 9: CS672: Approximation Algorithms Spring 14 Introduction to ...shaddin/cs672sp14/slides/LP1.pdfIntroduction to Linear Programming I Instructor: Shaddin Dughmi. Outline 1 Linear Programming](https://reader034.vdocuments.mx/reader034/viewer/2022050307/5f6fd48a7d5b0c46f4252f29/html5/thumbnails/9.jpg)
A 2-D example
maximize x1 + x2subject to x1 + 2x2 ≤ 2
2x1 + x2 ≤ 2x1, x2 ≥ 0
Linear Programming 5/24
![Page 10: CS672: Approximation Algorithms Spring 14 Introduction to ...shaddin/cs672sp14/slides/LP1.pdfIntroduction to Linear Programming I Instructor: Shaddin Dughmi. Outline 1 Linear Programming](https://reader034.vdocuments.mx/reader034/viewer/2022050307/5f6fd48a7d5b0c46f4252f29/html5/thumbnails/10.jpg)
Economic Interpretation: Optimal Production
n products, m raw materialsEvery unit of product j uses aij units of raw material iThere are bi units of material i availableProduct j yields profit cj per unitFacility wants to maximize profit subject to available raw materials
maximize cᵀxsubject to aᵀi x ≤ bi, for i = 1, . . . ,m.
xj ≥ 0, for j = 1, . . . , n.
Linear Programming 6/24
![Page 11: CS672: Approximation Algorithms Spring 14 Introduction to ...shaddin/cs672sp14/slides/LP1.pdfIntroduction to Linear Programming I Instructor: Shaddin Dughmi. Outline 1 Linear Programming](https://reader034.vdocuments.mx/reader034/viewer/2022050307/5f6fd48a7d5b0c46f4252f29/html5/thumbnails/11.jpg)
Terminology
Hyperplane: The region defined by a linear equalityHalfspace: The region defined by a linear inequality aᵀi x ≤ bi.Polyhedron: The intersection of a set of linear inequalities inEuclidean space
Feasible region of an LP is a polyhedronPolytope: A bounded polyhedron
Equivalently: convex hull of a finite set of pointsVertex: A point x is a vertex of polyhedron P if 6 ∃y 6= 0 withx+ y ∈ P and x− y ∈ PFace of P : The intersection with P of a hyperplane H disjoint fromthe interior of P
Linear Programming 7/24
![Page 12: CS672: Approximation Algorithms Spring 14 Introduction to ...shaddin/cs672sp14/slides/LP1.pdfIntroduction to Linear Programming I Instructor: Shaddin Dughmi. Outline 1 Linear Programming](https://reader034.vdocuments.mx/reader034/viewer/2022050307/5f6fd48a7d5b0c46f4252f29/html5/thumbnails/12.jpg)
Basic Facts about LPs and Polytopes
FactFeasible regions of LPs (i.e. polyhedrons) are convex
FactSet of optimal solutions of an LP is convex
In fact, a face of the polyhedronintersection of P with hyperplane cᵀx = OPT
FactAt a vertex, n linearly independent constraints are satisfied withequality (a.k.a. tight)
Linear Programming 8/24
![Page 13: CS672: Approximation Algorithms Spring 14 Introduction to ...shaddin/cs672sp14/slides/LP1.pdfIntroduction to Linear Programming I Instructor: Shaddin Dughmi. Outline 1 Linear Programming](https://reader034.vdocuments.mx/reader034/viewer/2022050307/5f6fd48a7d5b0c46f4252f29/html5/thumbnails/13.jpg)
Basic Facts about LPs and Polytopes
FactFeasible regions of LPs (i.e. polyhedrons) are convex
FactSet of optimal solutions of an LP is convex
In fact, a face of the polyhedronintersection of P with hyperplane cᵀx = OPT
FactAt a vertex, n linearly independent constraints are satisfied withequality (a.k.a. tight)
Linear Programming 8/24
![Page 14: CS672: Approximation Algorithms Spring 14 Introduction to ...shaddin/cs672sp14/slides/LP1.pdfIntroduction to Linear Programming I Instructor: Shaddin Dughmi. Outline 1 Linear Programming](https://reader034.vdocuments.mx/reader034/viewer/2022050307/5f6fd48a7d5b0c46f4252f29/html5/thumbnails/14.jpg)
Basic Facts about LPs and Polytopes
FactFeasible regions of LPs (i.e. polyhedrons) are convex
FactSet of optimal solutions of an LP is convex
In fact, a face of the polyhedronintersection of P with hyperplane cᵀx = OPT
FactAt a vertex, n linearly independent constraints are satisfied withequality (a.k.a. tight)
Linear Programming 8/24
![Page 15: CS672: Approximation Algorithms Spring 14 Introduction to ...shaddin/cs672sp14/slides/LP1.pdfIntroduction to Linear Programming I Instructor: Shaddin Dughmi. Outline 1 Linear Programming](https://reader034.vdocuments.mx/reader034/viewer/2022050307/5f6fd48a7d5b0c46f4252f29/html5/thumbnails/15.jpg)
Basic Facts about LPs and Polyhedrons
FactAn LP either has an optimal solution, or is unbounded or infeasible
Linear Programming 9/24
![Page 16: CS672: Approximation Algorithms Spring 14 Introduction to ...shaddin/cs672sp14/slides/LP1.pdfIntroduction to Linear Programming I Instructor: Shaddin Dughmi. Outline 1 Linear Programming](https://reader034.vdocuments.mx/reader034/viewer/2022050307/5f6fd48a7d5b0c46f4252f29/html5/thumbnails/16.jpg)
Fundamental Theorem of LPIf an LP in standard form has an optimal solution, then it has a vertexoptimal solution.
ProofAssume not, and take a non-vertex optimal solution x with themaximum number of tight constraintsThere is y 6= 0 s.t. x± y are feasibley is perpendicular to the objective function and the tightconstraints at x.
i.e. cᵀy = 0, and aᵀi y = 0 whenever the i’th constraint is tight for x.
Can choose y s.t. yj < 0 for some jLet α be the largest constant such that x+ αy is feasible
Such an α exists
An additional constraint becomes tight at x+ αy, a contradiction.
Linear Programming 10/24
![Page 17: CS672: Approximation Algorithms Spring 14 Introduction to ...shaddin/cs672sp14/slides/LP1.pdfIntroduction to Linear Programming I Instructor: Shaddin Dughmi. Outline 1 Linear Programming](https://reader034.vdocuments.mx/reader034/viewer/2022050307/5f6fd48a7d5b0c46f4252f29/html5/thumbnails/17.jpg)
Fundamental Theorem of LPIf an LP in standard form has an optimal solution, then it has a vertexoptimal solution.
ProofAssume not, and take a non-vertex optimal solution x with themaximum number of tight constraintsThere is y 6= 0 s.t. x± y are feasibley is perpendicular to the objective function and the tightconstraints at x.
i.e. cᵀy = 0, and aᵀi y = 0 whenever the i’th constraint is tight for x.
Can choose y s.t. yj < 0 for some jLet α be the largest constant such that x+ αy is feasible
Such an α exists
An additional constraint becomes tight at x+ αy, a contradiction.
Linear Programming 10/24
![Page 18: CS672: Approximation Algorithms Spring 14 Introduction to ...shaddin/cs672sp14/slides/LP1.pdfIntroduction to Linear Programming I Instructor: Shaddin Dughmi. Outline 1 Linear Programming](https://reader034.vdocuments.mx/reader034/viewer/2022050307/5f6fd48a7d5b0c46f4252f29/html5/thumbnails/18.jpg)
Counting non-zero Variables
CorollaryIf an LP in standard form has an optimal solution, then there is anoptimal solution with at most m non-zero variables.
maximize cᵀxsubject to aᵀi x ≤ bi, for i = 1, . . . ,m.
xj ≥ 0, for j = 1, . . . , n.
e.g. for optimal production with n products and m raw materials,there is an optimal plan with at most m products.
Linear Programming 11/24
![Page 19: CS672: Approximation Algorithms Spring 14 Introduction to ...shaddin/cs672sp14/slides/LP1.pdfIntroduction to Linear Programming I Instructor: Shaddin Dughmi. Outline 1 Linear Programming](https://reader034.vdocuments.mx/reader034/viewer/2022050307/5f6fd48a7d5b0c46f4252f29/html5/thumbnails/19.jpg)
Outline
1 Linear Programming
2 Application to Combinatorial Problems
3 Duality and Its Interpretations
4 Properties of Duals
![Page 20: CS672: Approximation Algorithms Spring 14 Introduction to ...shaddin/cs672sp14/slides/LP1.pdfIntroduction to Linear Programming I Instructor: Shaddin Dughmi. Outline 1 Linear Programming](https://reader034.vdocuments.mx/reader034/viewer/2022050307/5f6fd48a7d5b0c46f4252f29/html5/thumbnails/20.jpg)
Application to Combinatorial Problems
Linear programs often encode combinatorial problems eitherexactly or approximatelySince our focus is on NP-hard problems, we encounter mostly thelatter
An LP often relaxes the problemAllows “better than optimal” solutions which are fractional
Uses1 Rounding a solution of the LP2 Analysis via primal/dual paradigm
Application to Combinatorial Problems 12/24
![Page 21: CS672: Approximation Algorithms Spring 14 Introduction to ...shaddin/cs672sp14/slides/LP1.pdfIntroduction to Linear Programming I Instructor: Shaddin Dughmi. Outline 1 Linear Programming](https://reader034.vdocuments.mx/reader034/viewer/2022050307/5f6fd48a7d5b0c46f4252f29/html5/thumbnails/21.jpg)
Application to Combinatorial Problems
Linear programs often encode combinatorial problems eitherexactly or approximatelySince our focus is on NP-hard problems, we encounter mostly thelatter
An LP often relaxes the problemAllows “better than optimal” solutions which are fractional
Uses1 Rounding a solution of the LP2 Analysis via primal/dual paradigm
Application to Combinatorial Problems 12/24
![Page 22: CS672: Approximation Algorithms Spring 14 Introduction to ...shaddin/cs672sp14/slides/LP1.pdfIntroduction to Linear Programming I Instructor: Shaddin Dughmi. Outline 1 Linear Programming](https://reader034.vdocuments.mx/reader034/viewer/2022050307/5f6fd48a7d5b0c46f4252f29/html5/thumbnails/22.jpg)
Example: Shortest Path
Given a directed network G = (V,E) where edge e has length `e ∈ R+,find the minimum cost path from s to t.
s t
1 11
2
2 2
2
3
330
50
1
Shortest Path LPminimize
∑e∈E `exe
subject to∑e→v
xe −∑v→e
xe = δv, for v ∈ V.
xe ≥ 0, for e ∈ E.
Where δv = −1 if v = s, 1 if v = t, and 0 otherwise.
Application to Combinatorial Problems 13/24
![Page 23: CS672: Approximation Algorithms Spring 14 Introduction to ...shaddin/cs672sp14/slides/LP1.pdfIntroduction to Linear Programming I Instructor: Shaddin Dughmi. Outline 1 Linear Programming](https://reader034.vdocuments.mx/reader034/viewer/2022050307/5f6fd48a7d5b0c46f4252f29/html5/thumbnails/23.jpg)
Example: Shortest Path
Given a directed network G = (V,E) where edge e has length `e ∈ R+,find the minimum cost path from s to t.
s t
1 11
2
2 2
2
3
330
50
1
Shortest Path LPminimize
∑e∈E `exe
subject to∑e→v
xe −∑v→e
xe = δv, for v ∈ V.
xe ≥ 0, for e ∈ E.
Where δv = −1 if v = s, 1 if v = t, and 0 otherwise.Application to Combinatorial Problems 13/24
![Page 24: CS672: Approximation Algorithms Spring 14 Introduction to ...shaddin/cs672sp14/slides/LP1.pdfIntroduction to Linear Programming I Instructor: Shaddin Dughmi. Outline 1 Linear Programming](https://reader034.vdocuments.mx/reader034/viewer/2022050307/5f6fd48a7d5b0c46f4252f29/html5/thumbnails/24.jpg)
Example: Vertex Cover
Given an undirected graph G = (V,E), with weights wi for i ∈ V , findminimum-weight S ⊆ V “covering” all edges.
Vertex Cover LPminimize
∑i∈V wixi
subject to xi + xj ≥ 1, for (i, j) ∈ E.xi ≥ 0, for i ∈ V.
Application to Combinatorial Problems 14/24
![Page 25: CS672: Approximation Algorithms Spring 14 Introduction to ...shaddin/cs672sp14/slides/LP1.pdfIntroduction to Linear Programming I Instructor: Shaddin Dughmi. Outline 1 Linear Programming](https://reader034.vdocuments.mx/reader034/viewer/2022050307/5f6fd48a7d5b0c46f4252f29/html5/thumbnails/25.jpg)
Example: Vertex Cover
Given an undirected graph G = (V,E), with weights wi for i ∈ V , findminimum-weight S ⊆ V “covering” all edges.
Vertex Cover LPminimize
∑i∈V wixi
subject to xi + xj ≥ 1, for (i, j) ∈ E.xi ≥ 0, for i ∈ V.
Application to Combinatorial Problems 14/24
![Page 26: CS672: Approximation Algorithms Spring 14 Introduction to ...shaddin/cs672sp14/slides/LP1.pdfIntroduction to Linear Programming I Instructor: Shaddin Dughmi. Outline 1 Linear Programming](https://reader034.vdocuments.mx/reader034/viewer/2022050307/5f6fd48a7d5b0c46f4252f29/html5/thumbnails/26.jpg)
Example: Knapsack
Given n items with sizes s1, . . . , sn and values v1, . . . , vn, and aknapsack of capacity C, find the maximum value set of items which fitsin the knapsack.
Knapsack LP
maximize∑n
i=1 vixisubject to
∑ni=1 sixi ≤ C
xi ≤ 1, for i ∈ [n].xi ≥ 0, for i ∈ [n].
Application to Combinatorial Problems 15/24
![Page 27: CS672: Approximation Algorithms Spring 14 Introduction to ...shaddin/cs672sp14/slides/LP1.pdfIntroduction to Linear Programming I Instructor: Shaddin Dughmi. Outline 1 Linear Programming](https://reader034.vdocuments.mx/reader034/viewer/2022050307/5f6fd48a7d5b0c46f4252f29/html5/thumbnails/27.jpg)
Example: Knapsack
Given n items with sizes s1, . . . , sn and values v1, . . . , vn, and aknapsack of capacity C, find the maximum value set of items which fitsin the knapsack.
Knapsack LP
maximize∑n
i=1 vixisubject to
∑ni=1 sixi ≤ C
xi ≤ 1, for i ∈ [n].xi ≥ 0, for i ∈ [n].
Application to Combinatorial Problems 15/24
![Page 28: CS672: Approximation Algorithms Spring 14 Introduction to ...shaddin/cs672sp14/slides/LP1.pdfIntroduction to Linear Programming I Instructor: Shaddin Dughmi. Outline 1 Linear Programming](https://reader034.vdocuments.mx/reader034/viewer/2022050307/5f6fd48a7d5b0c46f4252f29/html5/thumbnails/28.jpg)
Outline
1 Linear Programming
2 Application to Combinatorial Problems
3 Duality and Its Interpretations
4 Properties of Duals
![Page 29: CS672: Approximation Algorithms Spring 14 Introduction to ...shaddin/cs672sp14/slides/LP1.pdfIntroduction to Linear Programming I Instructor: Shaddin Dughmi. Outline 1 Linear Programming](https://reader034.vdocuments.mx/reader034/viewer/2022050307/5f6fd48a7d5b0c46f4252f29/html5/thumbnails/29.jpg)
Linear Programming Duality
Primal LP
maximize cᵀxsubject to Ax � b
Dual LP
minimize bᵀysubject to Aᵀy = c
y � 0
A ∈ Rm×n, c ∈ Rn, b ∈ Rm
yi is the dual variable corresponding to primal constraint Aix ≤ biAT
j y ≥ cj is the dual constraint corresponding to primal variable xj
Duality and Its Interpretations 16/24
![Page 30: CS672: Approximation Algorithms Spring 14 Introduction to ...shaddin/cs672sp14/slides/LP1.pdfIntroduction to Linear Programming I Instructor: Shaddin Dughmi. Outline 1 Linear Programming](https://reader034.vdocuments.mx/reader034/viewer/2022050307/5f6fd48a7d5b0c46f4252f29/html5/thumbnails/30.jpg)
Linear Programming Duality: Standard Form, andVisualization
Primal LP
maximize cᵀxsubject to Ax � b
x � 0
Dual LP
minimize yᵀbsubject to Aᵀy � c
y � 0
x1 x2 x3 x4y1 a11 a12 a13 a14 b1y2 a21 a22 a23 a24 b2y3 a31 a32 a33 a34 b3
c1 c2 c3 c4
yi is the dual variable corresponding to primal constraint Aix ≤ biAT
j y ≥ cj is the dual constraint corresponding to primal variable xj
Duality and Its Interpretations 17/24
![Page 31: CS672: Approximation Algorithms Spring 14 Introduction to ...shaddin/cs672sp14/slides/LP1.pdfIntroduction to Linear Programming I Instructor: Shaddin Dughmi. Outline 1 Linear Programming](https://reader034.vdocuments.mx/reader034/viewer/2022050307/5f6fd48a7d5b0c46f4252f29/html5/thumbnails/31.jpg)
Linear Programming Duality: Standard Form, andVisualization
Primal LP
maximize cᵀxsubject to Ax � b
x � 0
Dual LP
minimize yᵀbsubject to Aᵀy � c
y � 0
x1 x2 x3 x4y1 a11 a12 a13 a14 b1y2 a21 a22 a23 a24 b2y3 a31 a32 a33 a34 b3
c1 c2 c3 c4
yi is the dual variable corresponding to primal constraint Aix ≤ biAT
j y ≥ cj is the dual constraint corresponding to primal variable xj
Duality and Its Interpretations 17/24
![Page 32: CS672: Approximation Algorithms Spring 14 Introduction to ...shaddin/cs672sp14/slides/LP1.pdfIntroduction to Linear Programming I Instructor: Shaddin Dughmi. Outline 1 Linear Programming](https://reader034.vdocuments.mx/reader034/viewer/2022050307/5f6fd48a7d5b0c46f4252f29/html5/thumbnails/32.jpg)
Linear Programming Duality: Standard Form, andVisualization
Primal LP
maximize cᵀxsubject to Ax � b
x � 0
Dual LP
minimize yᵀbsubject to Aᵀy � c
y � 0
x1 x2 x3 x4y1 a11 a12 a13 a14 b1y2 a21 a22 a23 a24 b2y3 a31 a32 a33 a34 b3
c1 c2 c3 c4
yi is the dual variable corresponding to primal constraint Aix ≤ biAT
j y ≥ cj is the dual constraint corresponding to primal variable xjDuality and Its Interpretations 17/24
![Page 33: CS672: Approximation Algorithms Spring 14 Introduction to ...shaddin/cs672sp14/slides/LP1.pdfIntroduction to Linear Programming I Instructor: Shaddin Dughmi. Outline 1 Linear Programming](https://reader034.vdocuments.mx/reader034/viewer/2022050307/5f6fd48a7d5b0c46f4252f29/html5/thumbnails/33.jpg)
Interpretation 1: Economic Interpretation
Recall the Optimal Production problemn products, m raw materialsEvery unit of product j uses aij units of raw material iThere are bi units of material i availableProduct j yields profit cj per unitFacility wants to maximize profit subject to available raw materials
Primal LP
max∑n
j=1 cjxjs.t.
∑nj=1 aijxj ≤ bi, for i ∈ [m].
xj ≥ 0, for j ∈ [n].
Dual LP
min∑m
i=1 biyis.t.
∑mi=1 aijyi ≥ cj , for j ∈ [n].
yi ≥ 0, for i ∈ [m].
x1 x2 x3 x4y1 a11 a12 a13 a14 b1y2 a21 a22 a23 a24 b2y3 a31 a32 a33 a34 b3
c1 c2 c3 c4
Dual variable yi is a proposed price per unit of raw material iDual price vector is feasible if facility has incentive to sell materialsBuyer wants to spend as little as possible to buy materials
Duality and Its Interpretations 18/24
![Page 34: CS672: Approximation Algorithms Spring 14 Introduction to ...shaddin/cs672sp14/slides/LP1.pdfIntroduction to Linear Programming I Instructor: Shaddin Dughmi. Outline 1 Linear Programming](https://reader034.vdocuments.mx/reader034/viewer/2022050307/5f6fd48a7d5b0c46f4252f29/html5/thumbnails/34.jpg)
Interpretation 1: Economic Interpretation
Primal LP
max∑n
j=1 cjxjs.t.
∑nj=1 aijxj ≤ bi, for i ∈ [m].
xj ≥ 0, for j ∈ [n].
Dual LP
min∑m
i=1 biyis.t.
∑mi=1 aijyi ≥ cj , for j ∈ [n].
yi ≥ 0, for i ∈ [m].
x1 x2 x3 x4y1 a11 a12 a13 a14 b1y2 a21 a22 a23 a24 b2y3 a31 a32 a33 a34 b3
c1 c2 c3 c4
Dual variable yi is a proposed price per unit of raw material iDual price vector is feasible if facility has incentive to sell materialsBuyer wants to spend as little as possible to buy materials
Duality and Its Interpretations 18/24
![Page 35: CS672: Approximation Algorithms Spring 14 Introduction to ...shaddin/cs672sp14/slides/LP1.pdfIntroduction to Linear Programming I Instructor: Shaddin Dughmi. Outline 1 Linear Programming](https://reader034.vdocuments.mx/reader034/viewer/2022050307/5f6fd48a7d5b0c46f4252f29/html5/thumbnails/35.jpg)
Interpretation 1: Economic Interpretation
Primal LP
max∑n
j=1 cjxjs.t.
∑nj=1 aijxj ≤ bi, for i ∈ [m].
xj ≥ 0, for j ∈ [n].
Dual LP
min∑m
i=1 biyis.t.
∑mi=1 aijyi ≥ cj , for j ∈ [n].
yi ≥ 0, for i ∈ [m].
x1 x2 x3 x4y1 a11 a12 a13 a14 b1y2 a21 a22 a23 a24 b2y3 a31 a32 a33 a34 b3
c1 c2 c3 c4
Dual variable yi is a proposed price per unit of raw material iDual price vector is feasible if facility has incentive to sell materialsBuyer wants to spend as little as possible to buy materials
Duality and Its Interpretations 18/24
![Page 36: CS672: Approximation Algorithms Spring 14 Introduction to ...shaddin/cs672sp14/slides/LP1.pdfIntroduction to Linear Programming I Instructor: Shaddin Dughmi. Outline 1 Linear Programming](https://reader034.vdocuments.mx/reader034/viewer/2022050307/5f6fd48a7d5b0c46f4252f29/html5/thumbnails/36.jpg)
Interpretation 1: Economic Interpretation
Primal LP
max∑n
j=1 cjxjs.t.
∑nj=1 aijxj ≤ bi, for i ∈ [m].
xj ≥ 0, for j ∈ [n].
Dual LP
min∑m
i=1 biyis.t.
∑mi=1 aijyi ≥ cj , for j ∈ [n].
yi ≥ 0, for i ∈ [m].
x1 x2 x3 x4y1 a11 a12 a13 a14 b1y2 a21 a22 a23 a24 b2y3 a31 a32 a33 a34 b3
c1 c2 c3 c4
Dual variable yi is a proposed price per unit of raw material iDual price vector is feasible if facility has incentive to sell materialsBuyer wants to spend as little as possible to buy materials
Duality and Its Interpretations 18/24
![Page 37: CS672: Approximation Algorithms Spring 14 Introduction to ...shaddin/cs672sp14/slides/LP1.pdfIntroduction to Linear Programming I Instructor: Shaddin Dughmi. Outline 1 Linear Programming](https://reader034.vdocuments.mx/reader034/viewer/2022050307/5f6fd48a7d5b0c46f4252f29/html5/thumbnails/37.jpg)
Interpretation 1: Economic Interpretation
Primal LP
max∑n
j=1 cjxjs.t.
∑nj=1 aijxj ≤ bi, for i ∈ [m].
xj ≥ 0, for j ∈ [n].
Dual LP
min∑m
i=1 biyis.t.
∑mi=1 aijyi ≥ cj , for j ∈ [n].
yi ≥ 0, for i ∈ [m].
x1 x2 x3 x4y1 a11 a12 a13 a14 b1y2 a21 a22 a23 a24 b2y3 a31 a32 a33 a34 b3
c1 c2 c3 c4
Dual variable yi is a proposed price per unit of raw material iDual price vector is feasible if facility has incentive to sell materialsBuyer wants to spend as little as possible to buy materials
Duality and Its Interpretations 18/24
![Page 38: CS672: Approximation Algorithms Spring 14 Introduction to ...shaddin/cs672sp14/slides/LP1.pdfIntroduction to Linear Programming I Instructor: Shaddin Dughmi. Outline 1 Linear Programming](https://reader034.vdocuments.mx/reader034/viewer/2022050307/5f6fd48a7d5b0c46f4252f29/html5/thumbnails/38.jpg)
Interpretation 2: Finding the Best Upperbound
Recall the simple LP
maximize x1 + x2subject to x1 + 2x2 ≤ 2
2x1 + x2 ≤ 2x1, x2 ≥ 0
We found that the optimal solution was at (23 ,23), with an optimal
value of 4/3.
What if, instead of finding the optimal solution, we saught to findan upperbound on its value by combining inequalities?
Each inequality implies an upper bound of 2Multiplying each by 1
3 and summing gives x1 + x2 ≤ 4/3.
Duality and Its Interpretations 19/24
![Page 39: CS672: Approximation Algorithms Spring 14 Introduction to ...shaddin/cs672sp14/slides/LP1.pdfIntroduction to Linear Programming I Instructor: Shaddin Dughmi. Outline 1 Linear Programming](https://reader034.vdocuments.mx/reader034/viewer/2022050307/5f6fd48a7d5b0c46f4252f29/html5/thumbnails/39.jpg)
Interpretation 2: Finding the Best Upperbound
Recall the simple LP
maximize x1 + x2subject to x1 + 2x2 ≤ 2
2x1 + x2 ≤ 2x1, x2 ≥ 0
We found that the optimal solution was at (23 ,23), with an optimal
value of 4/3.What if, instead of finding the optimal solution, we saught to findan upperbound on its value by combining inequalities?
Each inequality implies an upper bound of 2Multiplying each by 1
3 and summing gives x1 + x2 ≤ 4/3.
Duality and Its Interpretations 19/24
![Page 40: CS672: Approximation Algorithms Spring 14 Introduction to ...shaddin/cs672sp14/slides/LP1.pdfIntroduction to Linear Programming I Instructor: Shaddin Dughmi. Outline 1 Linear Programming](https://reader034.vdocuments.mx/reader034/viewer/2022050307/5f6fd48a7d5b0c46f4252f29/html5/thumbnails/40.jpg)
Interpretation 2: Finding the Best Upperbound
x1 x2 x3 x4y1 a11 a12 a13 a14 b1y2 a21 a22 a23 a24 b2y3 a31 a32 a33 a34 b3
c1 c2 c3 c4
Multiplying each row i by yi and summing gives the inequality
yTAx ≤ yT b
When yTA ≥ cT , the right hand side of the inequality is an upperbound on cTx.The dual LP can be thought of as trying to find the bestupperbound on the primal that can be achieved this way.
Duality and Its Interpretations 20/24
![Page 41: CS672: Approximation Algorithms Spring 14 Introduction to ...shaddin/cs672sp14/slides/LP1.pdfIntroduction to Linear Programming I Instructor: Shaddin Dughmi. Outline 1 Linear Programming](https://reader034.vdocuments.mx/reader034/viewer/2022050307/5f6fd48a7d5b0c46f4252f29/html5/thumbnails/41.jpg)
Interpretation 3: Physical Forces
Apply force field c to a ball inside polytope Ax ≤ b.
Eventually, ball will come to rest against the walls of the polytope.Wall aix ≤ bi applies some force −yiai to the ballSince the ball is still, cT =
∑i yiai = yTA.
Dual can be thought of as trying to minimize “work”∑
i yibi tobring ball back to origin by moving polytopeWe will see that, at optimality, only the walls adjacent to the ballpush (Complementary Slackness)
Duality and Its Interpretations 21/24
![Page 42: CS672: Approximation Algorithms Spring 14 Introduction to ...shaddin/cs672sp14/slides/LP1.pdfIntroduction to Linear Programming I Instructor: Shaddin Dughmi. Outline 1 Linear Programming](https://reader034.vdocuments.mx/reader034/viewer/2022050307/5f6fd48a7d5b0c46f4252f29/html5/thumbnails/42.jpg)
Interpretation 3: Physical Forces
Apply force field c to a ball inside polytope Ax ≤ b.Eventually, ball will come to rest against the walls of the polytope.
Wall aix ≤ bi applies some force −yiai to the ballSince the ball is still, cT =
∑i yiai = yTA.
Dual can be thought of as trying to minimize “work”∑
i yibi tobring ball back to origin by moving polytopeWe will see that, at optimality, only the walls adjacent to the ballpush (Complementary Slackness)
Duality and Its Interpretations 21/24
![Page 43: CS672: Approximation Algorithms Spring 14 Introduction to ...shaddin/cs672sp14/slides/LP1.pdfIntroduction to Linear Programming I Instructor: Shaddin Dughmi. Outline 1 Linear Programming](https://reader034.vdocuments.mx/reader034/viewer/2022050307/5f6fd48a7d5b0c46f4252f29/html5/thumbnails/43.jpg)
Interpretation 3: Physical Forces
Apply force field c to a ball inside polytope Ax ≤ b.Eventually, ball will come to rest against the walls of the polytope.Wall aix ≤ bi applies some force −yiai to the ball
Since the ball is still, cT =∑
i yiai = yTA.Dual can be thought of as trying to minimize “work”
∑i yibi to
bring ball back to origin by moving polytopeWe will see that, at optimality, only the walls adjacent to the ballpush (Complementary Slackness)
Duality and Its Interpretations 21/24
![Page 44: CS672: Approximation Algorithms Spring 14 Introduction to ...shaddin/cs672sp14/slides/LP1.pdfIntroduction to Linear Programming I Instructor: Shaddin Dughmi. Outline 1 Linear Programming](https://reader034.vdocuments.mx/reader034/viewer/2022050307/5f6fd48a7d5b0c46f4252f29/html5/thumbnails/44.jpg)
Interpretation 3: Physical Forces
Apply force field c to a ball inside polytope Ax ≤ b.Eventually, ball will come to rest against the walls of the polytope.Wall aix ≤ bi applies some force −yiai to the ballSince the ball is still, cT =
∑i yiai = yTA.
Dual can be thought of as trying to minimize “work”∑
i yibi tobring ball back to origin by moving polytopeWe will see that, at optimality, only the walls adjacent to the ballpush (Complementary Slackness)
Duality and Its Interpretations 21/24
![Page 45: CS672: Approximation Algorithms Spring 14 Introduction to ...shaddin/cs672sp14/slides/LP1.pdfIntroduction to Linear Programming I Instructor: Shaddin Dughmi. Outline 1 Linear Programming](https://reader034.vdocuments.mx/reader034/viewer/2022050307/5f6fd48a7d5b0c46f4252f29/html5/thumbnails/45.jpg)
Interpretation 3: Physical Forces
Apply force field c to a ball inside polytope Ax ≤ b.Eventually, ball will come to rest against the walls of the polytope.Wall aix ≤ bi applies some force −yiai to the ballSince the ball is still, cT =
∑i yiai = yTA.
Dual can be thought of as trying to minimize “work”∑
i yibi tobring ball back to origin by moving polytopeWe will see that, at optimality, only the walls adjacent to the ballpush (Complementary Slackness)
Duality and Its Interpretations 21/24
![Page 46: CS672: Approximation Algorithms Spring 14 Introduction to ...shaddin/cs672sp14/slides/LP1.pdfIntroduction to Linear Programming I Instructor: Shaddin Dughmi. Outline 1 Linear Programming](https://reader034.vdocuments.mx/reader034/viewer/2022050307/5f6fd48a7d5b0c46f4252f29/html5/thumbnails/46.jpg)
Outline
1 Linear Programming
2 Application to Combinatorial Problems
3 Duality and Its Interpretations
4 Properties of Duals
![Page 47: CS672: Approximation Algorithms Spring 14 Introduction to ...shaddin/cs672sp14/slides/LP1.pdfIntroduction to Linear Programming I Instructor: Shaddin Dughmi. Outline 1 Linear Programming](https://reader034.vdocuments.mx/reader034/viewer/2022050307/5f6fd48a7d5b0c46f4252f29/html5/thumbnails/47.jpg)
Duality is an Inversion
Primal LP
maximize cᵀxsubject to Ax � b
x � 0
Dual LP
minimize bᵀysubject to Aᵀy � c
y � 0
Duality is an InversionGiven a primal LP in standard form, the dual of its dual is itself.
Properties of Duals 22/24
![Page 48: CS672: Approximation Algorithms Spring 14 Introduction to ...shaddin/cs672sp14/slides/LP1.pdfIntroduction to Linear Programming I Instructor: Shaddin Dughmi. Outline 1 Linear Programming](https://reader034.vdocuments.mx/reader034/viewer/2022050307/5f6fd48a7d5b0c46f4252f29/html5/thumbnails/48.jpg)
Correspondance Between Variables and Constraints
Primal LP
max∑n
j=1 cjxjs.t.
yi :
∑nj=1 aijxj ≤ bi, for i ∈ [m].
xj ≥ 0, for j ∈ [n].
Dual LP
min∑m
i=1 biyis.t.
xj :
∑mi=1 aijyi ≥ cj , for j ∈ [n].
yi ≥ 0, for i ∈ [m].
The i’th primal constraint gives rise to the i’th dual variable yiThe j’th primal variable xj gives rise to the j’th dual constraint
Properties of Duals 23/24
![Page 49: CS672: Approximation Algorithms Spring 14 Introduction to ...shaddin/cs672sp14/slides/LP1.pdfIntroduction to Linear Programming I Instructor: Shaddin Dughmi. Outline 1 Linear Programming](https://reader034.vdocuments.mx/reader034/viewer/2022050307/5f6fd48a7d5b0c46f4252f29/html5/thumbnails/49.jpg)
Correspondance Between Variables and Constraints
Primal LP
max∑n
j=1 cjxjs.t.yi :
∑nj=1 aijxj ≤ bi, for i ∈ [m].
xj ≥ 0, for j ∈ [n].
Dual LP
min∑m
i=1 biyis.t.
xj :
∑mi=1 aijyi ≥ cj , for j ∈ [n].
yi ≥ 0, for i ∈ [m].
The i’th primal constraint gives rise to the i’th dual variable yi
The j’th primal variable xj gives rise to the j’th dual constraint
Properties of Duals 23/24
![Page 50: CS672: Approximation Algorithms Spring 14 Introduction to ...shaddin/cs672sp14/slides/LP1.pdfIntroduction to Linear Programming I Instructor: Shaddin Dughmi. Outline 1 Linear Programming](https://reader034.vdocuments.mx/reader034/viewer/2022050307/5f6fd48a7d5b0c46f4252f29/html5/thumbnails/50.jpg)
Correspondance Between Variables and Constraints
Primal LP
max∑n
j=1 cjxjs.t.yi :
∑nj=1 aijxj ≤ bi, for i ∈ [m].
xj ≥ 0, for j ∈ [n].
Dual LP
min∑m
i=1 biyis.t.xj :
∑mi=1 aijyi ≥ cj , for j ∈ [n].
yi ≥ 0, for i ∈ [m].
The i’th primal constraint gives rise to the i’th dual variable yiThe j’th primal variable xj gives rise to the j’th dual constraint
Properties of Duals 23/24
![Page 51: CS672: Approximation Algorithms Spring 14 Introduction to ...shaddin/cs672sp14/slides/LP1.pdfIntroduction to Linear Programming I Instructor: Shaddin Dughmi. Outline 1 Linear Programming](https://reader034.vdocuments.mx/reader034/viewer/2022050307/5f6fd48a7d5b0c46f4252f29/html5/thumbnails/51.jpg)
Syntactic Rules
Primal LP
max cᵀxs.t.yi : aix ≤ bi, for i ∈ C1.yi : aix = bi, for i ∈ C2.
xj ≥ 0, for j ∈ D1.xj ∈ R, for j ∈ D2.
Dual LP
min bᵀys.t.xj : aᵀj y ≥ cj , for j ∈ D1.
xj : aᵀj y = cj , for j ∈ D2.
yi ≥ 0, for i ∈ C1.yi ∈ R, for i ∈ C2.
Rules of ThumbLoose constraint (i.e. inequality)⇒ tight dual variable (i.e.nonnegative)Tight constraint (i.e. equality)⇒ loose dual variable (i.e.unconstrained)
Properties of Duals 24/24