ee/econ 458 duality j. mccalley 1. our example (and hw) problem 2 after one iteration, we obtained...

EE/Econ 458Duality

J. McCalley

1

Our example (and HW) problem

2

0,0,0,0,0

1823

122

4

s.t.

53max

54321

521

42

31

21

xxxxx

xxx

xx

xx

xxF

1823

122

4

053

521

42

31

21

xxx

xx

xx

xxF

After one iteration, we obtained the following Tableau:

Coefficients of Basic variable

Eq. # F x1 x2 x3 x4 x5

Right side

F 0 1 -3 0 0 2.5 0 30 x3 1 0 1 0 1 0 0 4 x2 2 0 0 1 0 0.5 0 6 x5 3 0 3 0 0 -1 1 6

Add 5 × pivot row

Add -2 × pivot row

Optimality test indicates we need to do another iteration (since coefficient of x1is negative in above tableau).


3

In performing the second iteration, the only choice for the entering variable is x1 (coefficients of all other variables in the tableau are non-negative).

And so the constraint identifying the leaving variable is the last one, since its ratio is smallest (2<4).

To determine the leaving variable, we identify the constraints that most limit the increase in x1, as shown below.


Eq. # F x1 x2 x3 x4 x5

Right side

F 0 1 -3 0 0 2.5 0 30 x3 1 0 1 0 1 0 0 4 x2 2 0 0 1 0 0.5 0 6 x5 3 0 3 0 0 -1 1 6 2

3

6

41

4

And so what is the leaving variable?


4

The leaving variable is x5, since it is the variable in the last constraint which gets pushed to 0 as x1 increases.

Make the pivot element “1” by dividing the pivot row by 3.

This is equivalent to dividing both sides of the following by 3.3x1+0x2+0x3-x4+x5=6 (1/3)*[3x1+0x2+0x3-x4+x5] =6*(1/3) x1+0x2+0x3-0.3333x4+0.3333x5=2


Eq. # F x1 x2 x3 x4 x5

Right side

F 0 1 -3 0 0 2.5 0 30 x3 1 0 1 0 1 0 0 4 x2 2 0 0 1 0 0.5 0 6 x1 3 0 3 0 0 -1 1 6

Pivot column

Pivot row

Pivot element


Eq. # F x1 x2 x3 x4 x5

Right side

F 0 1 -3 0 0 2.5 0 30 x3 1 0 1 0 1 0 0 4 x2 2 0 0 1 0 0.5 0 6 x1 3 0 1 0 0 -0.333 0.333 2

Now what is the next step?


5

Use the last equation to eliminate the -3 in the top equation.

This is equivalent to multiplying the bottom equation by 3 and adding it to the top equation:3*[x1+0x2+0x3-0.3333x4+0.3333x5] =2*3

-3x1+0x2+0x3 -x4 +x5 =6 +{F+3x1+0x2+0x3+2.5x4+0x5=30}

------------------------------------------ F+0x1+0x2+0x3+1.5x4+x5=36

Basic variable

Eq. #

Coefficients of Right side F x1 x2 x3 x4 x5

F 0 1 -3 0 0 2.5 0 30 x3 1 0 1 0 1 0 0 4 x2 2 0 0 1 0 0.5 0 6 x1 3 0 1 0 0 -0.333 0.333 2


Eq. # F x1 x2 x3 x4 x5

Right side

F 0 1 0 0 0 1.5 1 36 x3 1 0 1 0 1 0 0 4 x2 2 0 0 1 0 0.5 0 6 x1 3 0 1 0 0 -0.333 0.333 2


6

Use the last equation to eliminate the 1 from the 2nd equation:

This is equivalent to multiplying the bottom equation by -1 and adding it to the 2nd equation:-1*[x1+0x2+0x3-0.3333x4+0.3333x5] =2*-1 -x1+0x2+0x3 +0.3333x4 -0.3333x5 =-2 +{x1+0x2+1x3 +0x4 +0x5 =4}

------------------------------------------------------ 0x1+0x2+1x3+0.3333x4-0.3333x5=2

Basic variable

Eq. #

Coefficients of Right side F x1 x2 x3 x4 x5

F 0 1 -3 0 0 2.5 0 30 x3 1 0 1 0 1 0 0 4 x2 2 0 0 1 0 0.5 0 6 x1 3 0 1 0 0 -0.333 0.333 2


Eq. # F x1 x2 x3 x4 x5

Right side

F 0 1 0 0 0 1.5 1 36 x3 1 0 0 0 1 0.333 -0.333 2 x2 2 0 0 1 0 0.5 0 6 x1 3 0 1 0 0 -0.333 0.333 2

Introduction to duality

7

Assume the problem we have been working on is actually a resource allocation problem:

0,0

1823

12

4

53max

21

21

1

21

xx

xx

x2

x s.t.

xxF

(1) Problem

2 Resources

Question: How to gain the most, in terms of the value of our optimized objective function, by increasing one resource or another?

0,0

1823

12

5

53max

21

21

1

21

xx

xx

x2

x s.t.

xxF

Problem(2)

2

0,0

1823

12

4

53max

21

21

1

21

xx

xx

x2

x s.t.

xxF

(1) Problem

2

CPLEX yields F1*=36 CPLEX yields F2

*=36(increases by 0)


8

0,0

1823

12

4

53max

21

21

1

21

xx

xx

x2

x s.t.

xxF

(1) Problem

2

CPLEX yields F1*=36 CPLEX yields F3

*=37.5(increases by 1.5)

0,0

1823

13

4

53max

21

21

1

21

xx

xx

x2

x s.t.

xxF

Problem(3)

2

CPLEX yields F1*=36

0,0

1823

12

4

53max

21

21

1

21

xx

xx

x2

x s.t.

xxF

(1) Problem

2

0,0

1923

12

4

53max

21

21

1

21

xx

xx

x2

x s.t.

xxF

(4) Problem

2

CPLEX yields F4*=37

(increases by 1.0)

Summary

1*

5.1*

0*

3

2

1

b

F

b

F

b

F


9

1*

5.1*

0*

3

2

1

b

F

b

F

b

F

Assume F* represents my optimal profits. Assume also that I have a little extra money to spend. Then this information tells me my optimal objective will improve most if I spend that money to increase a unit of resource 2 (b2), less if I spend it to increase a unit of resource 3 (b3), and it will do me no good at all to spend that money to increase resource 1 (b1) .

Go back and inspect the Tableau for the solution to Problem 1.


Eq. # F x1 x2 x3 x4 x5

Right side

F 0 1 0 0 0 1.5 1 36 x3 1 0 0 0 1 0.333 -0.333 2 x2 2 0 0 1 0 0.5 0 6 x1 3 0 1 0 0 -0.333 0.333 2

The coefficients of the slack variables in this final tableau are exactly the same as the right-hand-sides of the above: 0, 1.5, and 1 !

(Important note: ΔF*/Δbi is how the optimal value of F changes with bi.)


10

1*

5.1*

0*

3

2

1

b

F

b

F

b

F The coefficients of the slack variables in the objective function expression of the final tableau give the improvement in the objective for a unit increase in the right-hand-sides of the corresponding constraints.


Eq. # F x1 x2 x3 x4 x5

Right side

F 0 1 0 0 0 1.5 1 36 x3 1 0 0 0 1 0.333 -0.333 2 x2 2 0 0 1 0 0.5 0 6 x1 3 0 1 0 0 -0.333 0.333 2

What if your objective function was profits per hr, measured in $/hr?And what if one of your equations expressed a constraint on MW output for a certain generator? Units would be $/MW-hr. Where have we seen this before?

This always happens for linear programs. Here is a general rule:

x3 is SV for constr 1.x4 is SV for constr 2.X5 is SV for constr 3.


11

1*

5.1*

0*

3

2

1

b

F

b

F

b

FThese slack variable coefficients are called dual variables, dual prices, or shadow prices (implicit rather than explicit values associated with obtaining additional resources). We denote them as λi corresponding to the ith constraint.


Eq. # F x1 x2 x3 x4 x5

Right side

F 0 1 0 0 0 1.5 1 36 x3 1 0 0 0 1 0.333 -0.333 2 x2 2 0 0 1 0 0.5 0 6 x1 3 0 1 0 0 -0.333 0.333 2


12

1*

5.1*

0*

3

2

1

b

F

b

F

b

F From the left and below, we see that λ3 =1, which we now understand to mean that if we increase b3 by 1 unit, our objective function will increase by 1, from 36 to 37, and this, as we have learned, is the case.


Eq. # F x1 x2 x3 x4 x5

Right side

F 0 1 0 0 0 1.5 1 36 x3 1 0 0 0 1 0.333 -0.333 2 x2 2 0 0 1 0 0.5 0 6 x1 3 0 1 0 0 -0.333 0.333 2

But we need to be a little careful…

But what if we increase b3 by 6, to 24? Will we see an increase in F* by 6 as well, to 42? Will increasing b3 to 26 see F* increase to 44?

CPLEX yields F1*=36

0,0

1823

12

4

53max

21

21

1

21

xx

xx

x2

x s.t.

xxF

(1) Problem

2

0,0

2423

12

4

53max

21

21

1

21

xx

xx

x2

x s.t.

xxF

(5) Problem

2

CPLEX yields F5*=42

0,0

2623

12

4

53max

21

21

1

21

xx

xx

x2

x s.t.

xxF

(6) Problem

2

CPLEX yields F6*=42


13

1*

5.1*

0*

3

2

1

b

F

b

F

b

F

There are two observations to be made here:

1. Changing some bi do not influence F*.Only a subset of resources limit the objective. You may have labor hours, trucks, & pipe fittings as

resources, each of which are individually constrained. You have a large truck fleet and a whole warehouse of pipe fittings, but you don’t have enough labor. And so you increase labor until you hit the limit on pipe fitting inventory. You then increase pipe fitting inventory, but soon you hit the limit on trucks. 2. ΔF*/Δbi is not accurate if Δbi gets too large.

A better definition of the dual variables follows:

ii b

F

* This says that the λi = ΔF*/Δbi only for incrementally small Δbi. In

an LP, λi = ΔF*/Δbi is no longer true when Δbi becomes so large that its corresponding constraint is no longer binding.

Motivating the dual problem

14

Consider our now very familiar example problem (call it P):

0,0

1823

12

4

53max

21

21

1

21

xx

xx

x2

x s.t.

xxF

2

Let’s express linear combinations of multiples of the constraints, where the constraint multipliers are denoted λi on the ith constraint. We assume that λi≥0 to avoid having to change the inequality.

32132223111

321

2

11

21

18124)22()3(

)1823

)12

)4

53max

xxxx

xx(

x(2

x( s.t.

xxF

2

321232131

321

22

11

21

18124)22()3(

)1823(

)12 (2

)4 ( s.t.

53max

xx

xx

x

x

xxF

The composite inequality

P


15

Composite inequality:321232131 18124)22()3( xx

The left-hand-side of the composite inequality, being a linear combination of our original inequalities, must hold at any feasible solution (x1,x2), and in particular, at the optimal solution. That is, it is a necessary condition for satisfying the inequalities from which it came.It is not, however, a sufficient condition (see notes for example) unless we constrain how we choose the λi.

The composite inequality. 321232131

21

18124)22()3(

53

xx

xxFThe objective function.

Let’s develop criteria for selecting λ1, λ2, λ3. Concept 1: Make sure our choices of λ1, λ2, λ3 are such that each coefficient of xi in the composite inequality is at least as great as the corresponding coefficient in the objective function expression:

522

33

32

31

Any solution (x1,x2) results in a value F less than or

equal to the left-hand-side of the composite inequality.


16


Concept 2: Because F is less than or equal to the left-hand side of the composite inequality (by concept 1), and because the left-hand-side of the composite inequality is less than or equal to the right hand side of the composite inequality, we can also say that any solution (x1,x2) results in a value F which is less than or equal to the right-hand-side of the composite inequality.Concept 3: Concept 2 implies the right-hand-side of the composite inequality is an upper bound on the value that F may take. This is true for any value of F, even the maximum value F*.

F* Right-hand-side of composite inequality

0

Δ


17


Concept 4: Now choose λ1, λ2, λ3 to minimize the right-hand-side of the composite inequality, subject to the constraints we imposed in choosing the λi ‘s.

F* Right-hand-side of composite inequality

0

Δ

This creates a least upper bound to F*, i.e., it pushes right-hand-side of the composite inequality as far left as possible, while guaranteeing right-hand-side remains ≥ F* (due to enforcement of above constraints).

522

33

32

31

F* 0

Δ

F* 0

Δ


18


Concept 5: Given that the right-hand-side of the composite inequality is an upper bound to F*, then finding its minimum, subject to constraints, implies Δ=0. So we have a new problem to solve:

0,0,0

522

33

subject to

18124min

D Problem

321

32

31

321

G

What does concept 5 tell us about the above problem, if we solve it?The value of the obtained objective function, at the optimum, will be the same as the value of the original objective function at its optimum, i.e., F*=G*.


19

In other words, solving problem D is equivalent to solving problem P.

Problem P is called the primal problem. Problem D is called the dual problem. They are precisely equivalent. To show this, let’s use CPLEX to solve Problem D.

Problem Primal

21

21

2

1

21

0,0

1823

12 2

4 s.t.

53max

P Problem

xx

xx

x

x

xxF

Problem Dual

321

32

31

321

0,0,0

522

33

subject to

18124min

D Problem

G


20

The CPLEX code to do it is below (note I use y instead of λ in code):

Problem Dual

321

32

31

321

0,0,0

522

33

subject to

18124min

D Problem

G

minimize 1 y1 + 12 y2 + 18 y3subject to 1 y1 + 3 y3 >= 3 2 y2 + 2 y3 >=5 y1 >= 0 y2 >= 0 y3 >= 0end

The solution givesG*=36λ 1=0λ 2=1.5λ 3=1

Using CPLEX, following solution of the above dual problem, I typed the following command: display solution dual – CPLEX gave: Constraint Name Dual Pricec1 2.000000c2 6.000000

What are these?

They are:1.Coefficients of the SVs λ 4 , λ 5 of the objective function expression in last tableau of dual problem (not the values of the SVs). They are sensitivities.2.Values of the DVs at the optimal solution to the dual of this problem – “dual to the dual” which is the primal! So these are the values x1 and x2.


21

The CPLEX code to do it is below (note I use y instead of λ in code):

Problem Dual

321

32

31

321

0,0,0

522

33

subject to

18124min

D Problem

GThe solution givesG*=36λ 1=0λ 2=1.5λ 3=1

0,0,0

422

33

18124min

321

32

31

321

to subject

G

D1 Problem

The solution givesG*=30λ 1=0λ 2=1λ 3=1

Using CPLEX, following solution of the above dual problem, I typed the following command: display solution dual – CPLEX gave: Constraint Name Dual Pricec1 2.000000c2 6.000000

6*

2*

2

1

c

G

c

Gc1, c2 are right-hand-sides of the two constraints in the dual


22

There is a circular relationship here, which can be stated as:The dual of the dual to a primal is the primal.

If you called Problem D our primal problem, and took its dual, you would get our original primal problem P back, as illustrated below.

Problem Primal

321

32

31

321

0,0,0

522

33

18124min

to subject

G

D Problem

Problem Dual

21

21

2

1

21

0,0

1823

12 2

4 s.t.

53max

P Problem

xx

xx

x

x

xxF

Problem Primal

21

21

2

1

21

0,0

1823

12 2

4 s.t.

53max

P Problem

xx

xx

x

x

xxF

Problem Dual

321

32

31

321

0,0,0

522

33

subject to

18124min

D Problem

G

Primal-dual relationships

23

1. Number of decision variables and constraints:• # of dual decision variables = # of primal constraints.• # of dual constraints = # of primal decision variables.

2. Coefficients of decision variables in dual objective are right-hand-sides of primal constraints.

Problem Primal

21

21

2

1

21

0,0

1823

12 2

4 s.t.

53max

P Problem

xx

xx

x

x

xxF

Problem Dual

321

32

31

321

0,0,0

522

33

subject to

18124min

D Problem

G

3. Coefficients of decision variables in primal objective are right-hand-sides of dual constraints.

Problem Primal

21

21

2

1

21

0,0

1823

12 2

4 s.t.

53max

P Problem

xx

xx

x

x

xxF

Problem Dual

321

32

31

321

0,0,0

522

33

subject to

18124min

D Problem

G


24

4. Coefficients of one variable across multiple primal constraints are coefficients of multiple variables in one dual constraint.

Problem Primal

21

21

2

1

21

0,0

1823

12 2

4 s.t.

53max

P Problem

xx

xx

x

x

xxF

Problem Dual

to subject

G

D Problem

0,0,0

522

33

18124min

321

32

31

321

Problem Primal

21

21

2

1

21

0,0

1823

12 2

4 s.t.

53max

P Problem

xx

xx

x

x

xxF

Problem Dual

to subject

G

D Problem

0,0,0

522

33

18124min

321

32

31

321


25

5. Coefficients of one variable across multiple dual constraints are coefficients of multiple variables in one primal constraint.

Problem Primal

21

21

2

1

21

0,0

1823

12 2

4 s.t.

53max

P Problem

xx

xx

x

x

xxF

Problem Dual

to subject

G

D Problem

0,0,0

522

33

18124min

321

32

31

321

Problem Primal

21

21

2

1

21

0,0

1823

12 2

4 s.t.

53max

P Problem

xx

xx

x

x

xxF

Problem Dual

to subject

G

D Problem

0,0,0

522

33

18124min

321

32

31

321

Problem Primal

21

21

2

1

21

0,0

1823

12 2

4 s.t.

53max

P Problem

xx

xx

x

x

xxF

Problem Dual

to subject

G

D Problem

0,0,0

522

33

18124min

321

32

31

321

6. If primal objective is maximization, dual objective is minimization7. If primal constraints are ≤, dual constraints are ≥.

Obtaining the dual

26

We can immediately write down the dual given the primal.

Example: Let’s return to the example we used to illustrate use of CPLEX in the notes called “Intro_CPLEX.”

321 345max xxxF Subject to:

532 321 xxx1124 321 xxx

8243 321 xxx

0,0,0 321 xxx

By inspection:

321 8115min G

Subject to:

5342 321 443 321 322 321

0,0,0 321

Let’s check it using CPLEX….(if we got it right above, they should have the same objective function value at the optimal solution).

Obtaining the dual

27

Primal

maximize 5 x1 + 4 x2 + 3 x3subject to 2 x1 + 3 x2 + x3 <= 5 4 x1 + x2 + 2x3 <= 11 3 x1 + 4 x2 + 2 x3 <= 8 x1 >= 0 x2 >= 0 x3 >= 0endThe solution is (x1,x2,x3)=(2,0,1), F*=13.The primal SV coefficients, i.e., the values of the dual variables are: (λ1, λ2, λ3)=(1,0,1)

minimize 5 y1 + 11 y2 + 8 y3subject to 2 y1 + 4 y2 + 3 y3 >= 5 3 y1 + 1 y2 + 4 y3 >= 4 1 y1 + 2 y2 + 2 y3 >= 3 y1 >= 0 y2 >= 0 y3 >= 0end

Dual

The solution is (λ1, λ2, λ3)=(1,0,1), G*=13.The dual SV coefficients, i.e., the values of the dual variables are: (x1,x2,x3)=(2,0,1).

Viewing primal-dual relationship

28

Generalize the primal-dual problems:

0,...,,

...

...

...

..

...max

21

2211

22222121

11212111

2211

n

mnmnmm

nn

nn

nn

xxx

bxaxaxa

bxaxaxa

bxaxaxa

ts

xcxcxcF

0,...,,

...

...

...

..

...min

21

2211

22222112

11221111

2211

n

nmmnnn

mm

mm

mm

caaa

caaa

caaa

ts

bbbG


29


30

The duality theorem

31

Duality theorem: If the primal problem has an optimal solution x*, then the dual problem has an optimal solution λ* such that

*)(*)( xFG

But what if the primal does not have an optimal solution?Then what happens in the dual?

To answer this, we must first consider what are the alternatives for finding an optimal solution to the primal? There are two:1. The primal is unbounded.2. The primal is infeasible.

Unbounded vs. infeasibility

32

Infeasible

0,0

18 3

12 4

4 s.t.

53max

21

1

1

1

21

xx

x

x

x

xxF

0,0

7

1823

12

4

53max

21

2

21

1

21

xx

x

xx

x2

x s.t.

xxF

2

UnboundedCPLEX> read ex.lpProblem 'ex.lp' read.Read time = 0.00 sec.CPLEX> primoptBound infeasibility column 'x2'.Presolve time = 0.00 sec.Presolve - Unbounded or infeasible.Solution time = 0.00 sec.CPLEX>

CPLEX> read ex1.lpProblem 'ex1.lp' read.Read time = 0.00 sec.CPLEX> primoptDual infeasible due to empty column 'x2'.Presolve time = 0.00 sec.Presolve - Unbounded or infeasible.Solution time = 0.00 sec.CPLEX>

An unbounded primal

33

Let’s inspect the unbounded problem.

0,0

18 3

12 4

4 s.t.

53max

21

1

1

1

21

xx

x

x

x

xxF

Recall that the objective function for the dual establishes an upper bound for the objective function of the primal, i.e.,. )()( xFG

for any sets of feasible solutions λ and x. If F(x) is unbounded, then the only possibility for G(λ) is that it must be infeasible.

An unbounded primal

34

Write down the dual.

0,0

18 3

12 4

4 s.t.

53max

21

1

1

1

21

xx

x

x

x

xxF

Likewise, we can show that if the dual is unbounded, the primal must be infeasible.

However, it is not necessarily true that if the primal (or dual) is infeasible, then the dual (or primal) is unbounded. It is possible for an infeasible primal to have an infeasible dual and vice-versa, that is, both the primal and the dual may be both be infeasible.

Finally, if both primal and dual have feasible solutions, both have optimal solutions.

0,0,0

5000

334

..

18124min

321

321

321

321

ts

G

This constraint cannot be satisfied, therefore this problem is infeasible.

Primal-Dual relations

DUAL

Optimal Infeasible Unbounded

PRIMAL

Optimal Possible Impossible Impossible

Infeasible Impossible Possible Possible

Unbounded Impossible Possible Impossible

A comment on using CPLEX

maximize 5 x1 + 4 x2 + 3 x3subject to 2 x1 + 3 x2 + x3 <= 5 4 x1 + x2 + 2x3 <= 11 3 x1 + 4 x2 + 2 x3 <= 8 x1 >= 0 x2 >= 0 x3 >= 0end

CPLEX> primopt… Primal simplex - Optimal: Objective = 1.3000000000e+01… CPLEX> display solution variables -Variable Name Solution Valuex1 2.000000x3 1.000000All other variables in the range 1-3 are 0.CPLEX> display solution slacks -Constraint Name Slack Valueslack c2 1.000000slack c4 -2.000000slack c6 -1.000000All other slacks in the range 1-6 are 0.CPLEX> display solution dual -Constraint Name Dual Pricec1 1.000000c3 1.000000All other dual prices in the range 1-6 are 0.CPLEX> quit

CPLEX will name the slack variables c1, c2, …, cm, cm+1,…, cm+n

where there are m constraints and n decision variables. Therefore the first m slack variables (c1, c2, …, cm) correspond to the explicit inequality constraints, and the last n slack variables (cm+1,…, cm+n) correspond to the nonnegativity constraints on the decision variables.



The values of the slack variables at the solution were c1=0, c2=1, c3=0, c4=-2, c5=0, c6=-1.

Note that CPLEX does not print the values of slack variables that are zero.




The fact that c1=0 and c3=0 indicates that the first and third constraints are binding. That c2=1 indicates the left-hand side of the second constraint is less than the right-hand-side by 1.Let’s check:


51)0(3)2(2

532 321

xxxConstraint 1:

Constraint 2:

Constraint 3:10)1(20)2(4

1124 321

xxx

8)1(2)0(4)2(3

8243 321

xxx



The fact that c5=0 indicates that the second inequality constraint is binding, i.e., which is consistent with the fact that x2=0.


02 x



The facts that c4=-2, c6=-1 is interesting because these slacks are negative. This is a result of the fact that the corresponding constraints are actually “greater than or less to” constraints instead of “less than or equal to constraints.” The way they are treated in CPLEX is as follows:


00 411 cxx

00 633 cxxso that when x1=2,, c4=-2, andwhen x3=1, , c6=-1..

ee/econ 458 duality j. mccalley 1. our example (and hw) problem 2 after one iteration, we obtained...

Documents

f 3x1 0x2 0x3

x1 coefficients

f 0x1 0x2 0x3

3x1 0x2 0x3x4 x5

3x1 0x2 0x3 x4 x5

hw problem

x1 increases

0x1 0x2 1x3