duality for mixed integer linear...

55
lehigh-logo Duality for Mixed Integer Linear Programming TED RALPHS MENAL G ¨ UZELSOY ISE Department COR@L Lab Lehigh University [email protected] University of Newcastle, Newcastle, Australia 13 May 2009 Thanks: Work supported in part by the National Science Foundation Ralphs and G ¨ uzelsoy (Lehigh University) Duality for MILP 18 February 2009 1 / 56

Upload: others

Post on 05-Mar-2020

43 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Duality for Mixed Integer Linear Programmingcoral.ie.lehigh.edu/~ted/files/talks/DUALITY09.pdfDuality for Mixed Integer Linear Programming ... In mathematical programming, the input

lehigh-logo

Duality for Mixed Integer Linear Programming

TED RALPHSMENAL GUZELSOY

ISE DepartmentCOR@L Lab

Lehigh [email protected]

University of Newcastle, Newcastle, Australia13 May 2009

Thanks:Work supported in part by the National Science Foundation

Ralphs and Guzelsoy (Lehigh University) Duality for MILP 18 February 2009 1 / 56

Page 2: Duality for Mixed Integer Linear Programmingcoral.ie.lehigh.edu/~ted/files/talks/DUALITY09.pdfDuality for Mixed Integer Linear Programming ... In mathematical programming, the input

lehigh-logo

Outline

1 Duality TheoryIntroductionIP Duality

2 The Subadditive DualFormulationProperties

3 Constructing Dual FunctionsThe Value FunctionGomory’s ProcedureBranch-and-Bound MethodBranch-and-Cut

Ralphs and Guzelsoy (Lehigh University) Duality for MILP 18 February 2009 2 / 56

Page 3: Duality for Mixed Integer Linear Programmingcoral.ie.lehigh.edu/~ted/files/talks/DUALITY09.pdfDuality for Mixed Integer Linear Programming ... In mathematical programming, the input

lehigh-logo

What is Duality?

It is difficult to give a general definition of mathematical duality, thoughmathematics is replete with various notions of it.

Set Theory and Logic (De Morgan Laws)Geomety (Pascal’s Theorem & Brianchon’s Theorem)Combinatorics (Graph Coloring)

The duality we are interested in is a sort offunctional duality.

We define a generic optimization problem to be a mappingf : X → R, whereXis the set of possibleinputsandf (x) is theresult.

Duality may then be defined as a method of transforming a givenprimal problemto an associateddual problemsuch that

the dual problem yields a bound on the primal problem, andapplying a related transformation to the dual produces the primal again.

In many case, we would also like to require that the dual boundbe “close” to theprimal result for a specific input of interest.

Ralphs and Guzelsoy (Lehigh University) Duality for MILP 18 February 2009 3 / 56

Page 4: Duality for Mixed Integer Linear Programmingcoral.ie.lehigh.edu/~ted/files/talks/DUALITY09.pdfDuality for Mixed Integer Linear Programming ... In mathematical programming, the input

lehigh-logo

Duality in Mathematical Programming

In mathematical programming, the input is the problem data (e.g., the constraintmatrix, right-hand side, and cost vector for a linear program).

We view the primal and the dual as parametric problems, but some data is heldconstant.

Uses of the Dual in Mathematical Programing

If the dual is easier to evaluate, we can use it to obtain a bound on theprimal optimal value.

We can also use the dual to perform sensitivity analysis on theparameterized primal input data.

Finally, we can also use the dual to warm start solution procedure basedon evaluation of the dual.

Ralphs and Guzelsoy (Lehigh University) Duality for MILP 18 February 2009 4 / 56

Page 5: Duality for Mixed Integer Linear Programmingcoral.ie.lehigh.edu/~ted/files/talks/DUALITY09.pdfDuality for Mixed Integer Linear Programming ... In mathematical programming, the input

lehigh-logo

Duality in Integer Programming

We will initially be interested in the mixed integer linear program (MILP)instance

zIP = minx∈S

cx, (P)

where,c ∈ Rn, S= x ∈ Zr+ × Rn−r

+ | Ax = b with A ∈ Qm×n, b ∈ Rm.

We call this instance thebase primal instance.

To construct a dual, we need a parameterized version of this instance.

For reasons that will become clear, the most relevant parameterization is of theright-hand side.

Thevalue function(or primal function) of the base primal instance (P) is

z(d) = minx∈S(d)

cx,

where for a givend ∈ Rm, S(d) = x ∈ Zr+ × Rn−r

+ | Ax = d.

We letz(d) = ∞ if d ∈ Ω = d ∈ Rm | S(d) = ∅.

Ralphs and Guzelsoy (Lehigh University) Duality for MILP 18 February 2009 5 / 56

Page 6: Duality for Mixed Integer Linear Programmingcoral.ie.lehigh.edu/~ted/files/talks/DUALITY09.pdfDuality for Mixed Integer Linear Programming ... In mathematical programming, the input

lehigh-logo

Example: Value Function

zIP = min 12x1 + 2x3 + x4

s.t x1 −32x2 + x3 − x4 = b and

x1, x2 ∈ Z+, x3, x4 ∈ R+.

0

z(d)

d1-1-2-3 3 42-4

32 −

12−

52−

72

52

32

12

12

32

52

72

1

2

3

Ralphs and Guzelsoy (Lehigh University) Duality for MILP 18 February 2009 6 / 56

Page 7: Duality for Mixed Integer Linear Programmingcoral.ie.lehigh.edu/~ted/files/talks/DUALITY09.pdfDuality for Mixed Integer Linear Programming ... In mathematical programming, the input

lehigh-logo

Dual Functions

A dual functionF : Rm → R is one that satisfiesF(d) ≤ z(d) for all d ∈ Rm.

How to select such a function?

We choose may choose one that is easy to construct/evaluate and/or for whichF(b) ≈ z(b).

This results in thebase dual instance

zD = maxF(b) : F(d) ≤ z(d), d ∈ Rm, F ∈ Υm

whereΥm ⊆ f | f : Rm→R

We callF∗ strongfor this instance ifF∗ is afeasibledual function andF∗(b) = z(b).

This dual instance always has a solutionF∗ that is strong if the value function isbounded andΥm ≡ f | f : Rm→R. Why?

Ralphs and Guzelsoy (Lehigh University) Duality for MILP 18 February 2009 7 / 56

Page 8: Duality for Mixed Integer Linear Programmingcoral.ie.lehigh.edu/~ted/files/talks/DUALITY09.pdfDuality for Mixed Integer Linear Programming ... In mathematical programming, the input

lehigh-logo

The LP Relaxation Dual Function

It is easy to obtain a feasible dual function for any MILP.

Consider the value function of the LP relaxation of the primal problem:

FLP(d) = maxv∈Rm

vd : vA≤ c.

By linear programming duality theory, we haveFLP(d) ≤ z(d) for all d ∈ Rm.

Of course,FLP is not necessarily strong.

Ralphs and Guzelsoy (Lehigh University) Duality for MILP 18 February 2009 8 / 56

Page 9: Duality for Mixed Integer Linear Programmingcoral.ie.lehigh.edu/~ted/files/talks/DUALITY09.pdfDuality for Mixed Integer Linear Programming ... In mathematical programming, the input

lehigh-logo

Example: LP Dual Function

FLP(d) = min vd,s.t 0≥ v ≥ − 1

2, andv ∈ R,

which can be written explicitly as

FLP(d) =

0, d ≤ 0− 1

2d, d > 0.

0d

1-1-2-3 3 42-4−

32 −

12−

52−

72

52

32

12

12

32

52

72

1

2

3

z(d)FLP(d)

Ralphs and Guzelsoy (Lehigh University) Duality for MILP 18 February 2009 9 / 56

Page 10: Duality for Mixed Integer Linear Programmingcoral.ie.lehigh.edu/~ted/files/talks/DUALITY09.pdfDuality for Mixed Integer Linear Programming ... In mathematical programming, the input

lehigh-logo

The Subadditive Dual

By considering that

F(d) ≤ z(d), d ∈ Rm ⇐⇒ F(d) ≤ cx , x ∈ S(d), d ∈ Rm

⇐⇒ F(Ax) ≤ cx , x ∈ Zn+,

the generalized dual problem can be rewritten as

zD = maxF(b) : F(Ax) ≤ cx, x ∈ Zr+ × Rn−r

+ , F ∈ Υm.

Can we further restrictΥm and still guarantee a strong dual solution?

The class of linear functions? NO!

The class of convex functions? NO!

The class of sudadditive functions? YES!

Ralphs and Guzelsoy (Lehigh University) Duality for MILP 18 February 2009 10 / 56

Page 11: Duality for Mixed Integer Linear Programmingcoral.ie.lehigh.edu/~ted/files/talks/DUALITY09.pdfDuality for Mixed Integer Linear Programming ... In mathematical programming, the input

lehigh-logo

The Subadditive Dual

Let a functionF be defined over a domainV. ThenF is subadditive ifF(v1) + F(v2) ≥ F(v1 + v2)∀v1, v2, v1 + v2 ∈ V.

Note that the value functionz is subadditive overΩ. Why?

If Υm ≡ Γm ≡ F is subadditive| F : Rm→R, F(0) = 0, we can rewrite thedual problem above as thesubadditive dual

zD = max F(b)

F(aj) ≤ cj j = 1, ..., r,

F(aj) ≤ cj j = r + 1, ..., n, and

F ∈ Γm,

where the functionF is defined by

F(d) = lim supδ→0+

F(δd)

δ∀d ∈ Rm.

Here,F is theupper d-directional derivativeof F at zero.

Ralphs and Guzelsoy (Lehigh University) Duality for MILP 18 February 2009 11 / 56

Page 12: Duality for Mixed Integer Linear Programmingcoral.ie.lehigh.edu/~ted/files/talks/DUALITY09.pdfDuality for Mixed Integer Linear Programming ... In mathematical programming, the input

lehigh-logo

Example: Upper D-directional Derivative

The upper d-directional derivative can be interpreted as the slope of the valuefunction in directiond at 0.

For the example, we have

0d

1-1-2-3 3 42-4−

32 −

12−

52−

72

52

32

12

12

32

52

72

1

2

3

z(d)z(d)

Ralphs and Guzelsoy (Lehigh University) Duality for MILP 18 February 2009 12 / 56

Page 13: Duality for Mixed Integer Linear Programmingcoral.ie.lehigh.edu/~ted/files/talks/DUALITY09.pdfDuality for Mixed Integer Linear Programming ... In mathematical programming, the input

lehigh-logo

Weak Duality

Weak Duality Theorem

Let x be a feasible solution to the primal problem and letF be a feasible solutionto the subadditive dual. Then,F(b) ≤ cx.

Proof.

Corollary

For the primal problem and its subadditive dual:1 If the primal problem (resp., the dual) is unbounded then thedual problem

(resp., the primal) is infeasible.2 If the primal problem (resp., the dual) is infeasible, then the dual problem

(resp., the primal) is infeasible or unbounded.

Ralphs and Guzelsoy (Lehigh University) Duality for MILP 18 February 2009 13 / 56

Page 14: Duality for Mixed Integer Linear Programmingcoral.ie.lehigh.edu/~ted/files/talks/DUALITY09.pdfDuality for Mixed Integer Linear Programming ... In mathematical programming, the input

lehigh-logo

Strong Duality

Strong Duality Theorem

If the primal problem (resp., the dual) has a finite optimum, then so does thesubadditive dual problem (resp., the primal) and they are equal.

Outline of the Proof. Show that the value functionzor an extension toz is a feasibledual function.

Note thatzsatisfies the dual constraints.

Ω ≡ Rm: z∈ Γm.

Ω ⊂ Rm: ∃ ze ∈ Γm with ze(d) = z(d) ∀ d ∈ Ω andze(d) < ∞ ∀d ∈ Rm.

Ralphs and Guzelsoy (Lehigh University) Duality for MILP 18 February 2009 14 / 56

Page 15: Duality for Mixed Integer Linear Programmingcoral.ie.lehigh.edu/~ted/files/talks/DUALITY09.pdfDuality for Mixed Integer Linear Programming ... In mathematical programming, the input

lehigh-logo

Example: Subadditive Dual

For our IP instance, the subadditive dual problem is

max F(b)F(1) ≤ 1

2F(− 3

2) ≤ 0F(1) ≤ 2

F(−1) ≤ 1F ∈ Γ1.

.

and we have the following feasible dual functions:1 F1(d) = d

2 is an optimal dual function forb ∈ 0, 1, 2, ....

2 F2(d) = 0 is an optimal function forb ∈ ...,−3,− 32, 0.

3 F3(d) = max 12⌈d− ⌈⌈d⌉−d⌉

4 ⌉, 2d − 32⌈d− ⌈⌈d⌉−d⌉

4 ⌉ is an optimal function forb ∈ [0, 1

4] ∪ [1, 54] ∪ [2, 9

4] ∪ ....

4 F4(d) = max 32⌈

2d3 −

2⌈⌈ 2d3 ⌉− 2d

3 ⌉

3 ⌉ − d,− 34⌈

2d3 −

2⌈⌈ 2d3 ⌉− 2d

3 ⌉

3 ⌉ + d2 is an optimal

function forb ∈ ... ∪ [− 72,−3] ∪ [−2,− 3

2] ∪ [− 12, 0]

Ralphs and Guzelsoy (Lehigh University) Duality for MILP 18 February 2009 15 / 56

Page 16: Duality for Mixed Integer Linear Programmingcoral.ie.lehigh.edu/~ted/files/talks/DUALITY09.pdfDuality for Mixed Integer Linear Programming ... In mathematical programming, the input

lehigh-logo

Example: Feasible Dual Functions

0d

1-1-2-3 3 42-4−

32 −

12−

52−

72

52

32

12

12

32

52

72

1

2

3

z(d)F(d)

Notice how different dual solutions are optimal for some right-hand sides andnot for others.

Only the value function is optimal for all right-hand sides.

Ralphs and Guzelsoy (Lehigh University) Duality for MILP 18 February 2009 16 / 56

Page 17: Duality for Mixed Integer Linear Programmingcoral.ie.lehigh.edu/~ted/files/talks/DUALITY09.pdfDuality for Mixed Integer Linear Programming ... In mathematical programming, the input

lehigh-logo

Farkas’ Lemma (Pure Integer)

For the primal problem, exactly one of the following holds:1 S 6= ∅

2 There is anF ∈ Γm with F(aj) ≥ 0, j = 1, ..., n, andF(b) < 0.

Proof. Let c = 0 and apply strong duality theorem to subadditive dual.

Ralphs and Guzelsoy (Lehigh University) Duality for MILP 18 February 2009 17 / 56

Page 18: Duality for Mixed Integer Linear Programmingcoral.ie.lehigh.edu/~ted/files/talks/DUALITY09.pdfDuality for Mixed Integer Linear Programming ... In mathematical programming, the input

lehigh-logo

Complementary Slackness (Pure Integer)

For a given right-hand sideb, let x∗ andF∗ be feasible solutions to the primaland the subadditive dual problems, respectively. Thenx∗ andF∗ are optimalsolutions if and only if

1 x∗j (cj − F∗(aj)) = 0, j = 1, ..., n and

2 F∗(b) =∑n

j=1 F∗(aj)x∗j .

Proof. For an optimal pair we have

F∗(b) = F∗(Ax∗) =n

j=1

F∗(aj)x∗j = cx∗.

Ralphs and Guzelsoy (Lehigh University) Duality for MILP 18 February 2009 18 / 56

Page 19: Duality for Mixed Integer Linear Programmingcoral.ie.lehigh.edu/~ted/files/talks/DUALITY09.pdfDuality for Mixed Integer Linear Programming ... In mathematical programming, the input

lehigh-logo

Example: Pure Integer Case

If we require integrality of all variables in our previous example, then the valuefunction becomes

0d

1-1-2-3 3 42-4−

32 −

12−

52−

72

52

32

12

12

32

52

72

1

2

3

z(d)

Ralphs and Guzelsoy (Lehigh University) Duality for MILP 18 February 2009 19 / 56

Page 20: Duality for Mixed Integer Linear Programmingcoral.ie.lehigh.edu/~ted/files/talks/DUALITY09.pdfDuality for Mixed Integer Linear Programming ... In mathematical programming, the input

lehigh-logo

Constructing Dual Functions

Explicit constructionThe Value FunctionGenerating Functions

RelaxationsLagrangian RelaxationQuadratic Lagrangian RelaxationCorrected Linear Dual Functions

Primal Solution AlgorithmsCutting Plane MethodBranch-and-Bound MethodBranch-and-Cut Method

Ralphs and Guzelsoy (Lehigh University) Duality for MILP 18 February 2009 20 / 56

Page 21: Duality for Mixed Integer Linear Programmingcoral.ie.lehigh.edu/~ted/files/talks/DUALITY09.pdfDuality for Mixed Integer Linear Programming ... In mathematical programming, the input

lehigh-logo

Properties of the Value Function

It is subadditive overΩ.

It is piecewise polyhedral.

For an ILP, it can be obtained by a finite number of limited operations onelements of the RHS:

(i) rational multiplication(ii) nonnegative combination(iii) rounding

Chvatal fcns.

(iv) taking the minimum

Gomory fcns.

Ralphs and Guzelsoy (Lehigh University) Duality for MILP 18 February 2009 21 / 56

Page 22: Duality for Mixed Integer Linear Programmingcoral.ie.lehigh.edu/~ted/files/talks/DUALITY09.pdfDuality for Mixed Integer Linear Programming ... In mathematical programming, the input

lehigh-logo

The Value Function for MILPs

There is a one-to-one correspondence between ILP instancesand Gomoryfunctions.

TheJeroslow Formulashows that the value function of a MILP can also becomputed by taking the minimum of different values of a single Gomoryfunction with a correction term to account for the continuous variables.

The value function of the earlier example is

z(d) = min

32 max

⌊2d⌋3

,⌈

⌊2d⌋2

+ 3⌈2d⌉2 + 2d,

32 max

⌈2d⌉3

,⌈

⌈2d⌉2

− d,

Ralphs and Guzelsoy (Lehigh University) Duality for MILP 18 February 2009 22 / 56

Page 23: Duality for Mixed Integer Linear Programmingcoral.ie.lehigh.edu/~ted/files/talks/DUALITY09.pdfDuality for Mixed Integer Linear Programming ... In mathematical programming, the input

lehigh-logo

Jeroslow Formula

Let the setE consist of the index sets of dual feasible bases of the linearprogram

min1M

cCxC :1M

ACxC = b, x ≥ 0

whereM ∈ Z+ such that for anyE ∈ E , MA−1E aj ∈ Zm for all j ∈ I .

Theorem (Jeroslow Formula)There is ag ∈ G m such that

z(d) = minE∈E

g(⌊d⌋E) + vE(d− ⌊d⌋E) ∀d ∈ Rm with S(d) 6= ∅,

where forE ∈ E , ⌊d⌋E = AE⌊A−1E d⌋ andvE is the corresponding basic feasible

solution.

Ralphs and Guzelsoy (Lehigh University) Duality for MILP 18 February 2009 23 / 56

Page 24: Duality for Mixed Integer Linear Programmingcoral.ie.lehigh.edu/~ted/files/talks/DUALITY09.pdfDuality for Mixed Integer Linear Programming ... In mathematical programming, the input

lehigh-logo

The Single Constraint Case

Let us now consider an MILP with a single constraint for the purposes ofillustration:

minx∈S

cx, (P1)

c ∈ Rn, S= x ∈ Zr+ × Rn−r

+ | a′x = b with a ∈ Qn, b ∈ R.

Thevalue functionof (P) is

z(d) = minx∈S(d)

cx,

where for a givend ∈ R, S(d) = x ∈ Zr+ × Rn−r

+ | a′x = d.

Assumptions: LetI = 1, . . . , r, C = r + 1, . . . , n, N = I ∪ C.

z(0) = 0 =⇒ z : R → R ∪ +∞,N+ = i ∈ N | ai > 0 6= ∅ andN− = i ∈ N | ai < 0 6= ∅,r < n, that is,|C| ≥ 1 =⇒ z : R → R.

Ralphs and Guzelsoy (Lehigh University) Duality for MILP 18 February 2009 24 / 56

Page 25: Duality for Mixed Integer Linear Programmingcoral.ie.lehigh.edu/~ted/files/talks/DUALITY09.pdfDuality for Mixed Integer Linear Programming ... In mathematical programming, the input

lehigh-logo

Example

min 3x1 + 72x2 + 3x3 + 6x4 + 7x5 + 5x6

s.t 6x1 + 5x2 − 4x3 + 2x4 − 7x5 + x6 = b andx1, x2, x3 ∈ Z+, x4, x5, x6 ∈ R+.

(SP)

0 2 4 6 8 10 12 14 16 18 20 22 24 26 28-2-4-6-8-10-12-14-16

2

4

6

8

10

12

14

16

18

d

z

Ralphs and Guzelsoy (Lehigh University) Duality for MILP 18 February 2009 25 / 56

Page 26: Duality for Mixed Integer Linear Programmingcoral.ie.lehigh.edu/~ted/files/talks/DUALITY09.pdfDuality for Mixed Integer Linear Programming ... In mathematical programming, the input

lehigh-logo

Example (cont’d)

η = 12, ζ = − 3

4, ηC = 3 andζC = −1:

0 2 4 6 8 10 12 14 16 18 20 22 24 26 28-2-4-6-8-10-12-14-16

2

4

6

8

10

12

14

16

18

d

FL

FU

η

z

ζC

ζ

ηC

η = ηC⇐⇒z(d) = FU(d) = FL(d) ∀d ∈ R+

ζ = ζC⇐⇒z(d) = FU(d) = FL(d) ∀d ∈ R−

Ralphs and Guzelsoy (Lehigh University) Duality for MILP 18 February 2009 26 / 56

Page 27: Duality for Mixed Integer Linear Programmingcoral.ie.lehigh.edu/~ted/files/talks/DUALITY09.pdfDuality for Mixed Integer Linear Programming ... In mathematical programming, the input

lehigh-logo

Observations

Considerd+U , d−

U , d+L , d−

L :

0 2 4 6 8 10 12 14 16 18 20 22 24 26 28-2-4-6-8-10-12-14-16

2

4

6

8

10

12

14

16

18

dd−U

d−L d+U d+L2d+L 3d+L 4d+L2d−L2d−L

The relation betweenFU and the linear segments ofz: ηC, ζC

Ralphs and Guzelsoy (Lehigh University) Duality for MILP 18 February 2009 27 / 56

Page 28: Duality for Mixed Integer Linear Programmingcoral.ie.lehigh.edu/~ted/files/talks/DUALITY09.pdfDuality for Mixed Integer Linear Programming ... In mathematical programming, the input

lehigh-logo

Redundant Variables

Let T ⊆ C be such that

t+ ∈ T if and only if ηC < ∞ andηC =ct+

at+and similarly,

t− ∈ T if and only if ζC > −∞ andζC =ct−

at−.

and define

ν(d) = min cI xI + cTxT

s.t. aI xI + aTxT = dxI ∈ ZI

+, xT ∈ RT+

Then

ν(d) = z(d) for all d ∈ R.

The variables inC\T areredundant.

zcan be represented withat most 2 continuous variables.

Ralphs and Guzelsoy (Lehigh University) Duality for MILP 18 February 2009 28 / 56

Page 29: Duality for Mixed Integer Linear Programmingcoral.ie.lehigh.edu/~ted/files/talks/DUALITY09.pdfDuality for Mixed Integer Linear Programming ... In mathematical programming, the input

lehigh-logo

Example

min x1 − 3/4x2 + 3/4x3

s.t 5/4x1 − x2 + 1/2x3 = b, x1, x2 ∈ Z+, x3 ∈ R+.

ηc= 3/2, ζC

= −∞.

-1

-2

12

1

32

2

52

12−

32 -1−

52 -2-3−

72 11

2 2 352

32

72

12

32

52

For each discontinuous pointdi , we havedi − (5/4yi1 − yi

2) = 0 and each linear segment hasthe slope ofηC = 3/2.

Ralphs and Guzelsoy (Lehigh University) Duality for MILP 18 February 2009 29 / 56

Page 30: Duality for Mixed Integer Linear Programmingcoral.ie.lehigh.edu/~ted/files/talks/DUALITY09.pdfDuality for Mixed Integer Linear Programming ... In mathematical programming, the input

lehigh-logo

Jeroslow Formula

Let M ∈ Z+ be such that for anyt ∈ T, Maj

at∈ Z for all j ∈ I .

Then there is a Gomory functiong such that

z(d) = mint∈T

g(⌊d⌋t) +ct

at(d− ⌊d⌋t), ⌊d⌋t =

at

M

Mdat

, ∀d ∈ R

Such a Gomory function can be obtained from the value function of a relatedPILP.

For t ∈ T, setting

ωt(d) = g(⌊d⌋t) +ct

at(d− ⌊d⌋t) ∀d ∈ R,

we can writez(d) = min

t∈Tωt(d) ∀d ∈ R

Ralphs and Guzelsoy (Lehigh University) Duality for MILP 18 February 2009 30 / 56

Page 31: Duality for Mixed Integer Linear Programmingcoral.ie.lehigh.edu/~ted/files/talks/DUALITY09.pdfDuality for Mixed Integer Linear Programming ... In mathematical programming, the input

lehigh-logo

Piecewise Linearity and Continuity

For t ∈ T, ωt is piecewise linear with finitely many linear segments on anyclosed interval and each of those linear segments has a slopeof ηC if t = t+ orζC if t = t−.

ωt+ is continuous from the right, ωt− is continuous from the left.

ωt+ andωt− are bothlower-semicontinuous.

Theoremz is piecewise-linearwith finitelymany linear segments on any closed intervaland each of those linear segments has a slope ofηC or ζC.

(Meyer 1975)z is lower-semicontinuous.

ηC < ∞ if and only ifz is continuous from the right.

ζC > −∞ if and only ifz is continuous from the left.

BothηC andζC arefinite if and only ifz is continuous everywhere.

Ralphs and Guzelsoy (Lehigh University) Duality for MILP 18 February 2009 31 / 56

Page 32: Duality for Mixed Integer Linear Programmingcoral.ie.lehigh.edu/~ted/files/talks/DUALITY09.pdfDuality for Mixed Integer Linear Programming ... In mathematical programming, the input

lehigh-logo

Maximal Subadditive Extension

Let f : [0, h]→R, h > 0 besubadditiveandf (0) = 0.

Themaximal subadditive extensionof f from [0, h] to R+ is

fS(d) =

f (d) if d ∈ [0, h]

infC∈C(d)

ρ∈C

f (ρ) if d > h ,

C(d) is the set of all finite collectionsρ1, ..., ρR such thatρi ∈ [0, h], i = 1, ..., Rand

PRi=1 ρi = d.

Each collectionρ1, ..., ρR is called anh-partitionof d.We can also extend a subadditive functionf : [h, 0]→R, h < 0 to R

−similarly.

(Bruckner 1960)fS is subadditive and ifg is any other subadditive extension offfrom [0, h] to R+, theng ≤ fS (maximality).

Ralphs and Guzelsoy (Lehigh University) Duality for MILP 18 February 2009 32 / 56

Page 33: Duality for Mixed Integer Linear Programmingcoral.ie.lehigh.edu/~ted/files/talks/DUALITY09.pdfDuality for Mixed Integer Linear Programming ... In mathematical programming, the input

lehigh-logo

Extending the Value Function

Suppose we usez itself as theseedfunction.

Observe that we can change the“ inf” to “min” :

Lemma

Let the functionf : [0, h]→R be defined byf (d) = z(d) ∀d ∈ [0, h]. Then,

fS(d) =

z(d) if d ∈ [0, h]

minC∈C(d)

ρ∈C

z(ρ) if d > h .

For anyh > 0, z(d) ≤ fS(d) ∀d ∈ R+.

Observe that ford ∈ R+, fS(d)→z(d) while h→∞.

Is there anh < ∞ such thatfS(d) = z(d) ∀d ∈ R+?

Ralphs and Guzelsoy (Lehigh University) Duality for MILP 18 February 2009 33 / 56

Page 34: Duality for Mixed Integer Linear Programmingcoral.ie.lehigh.edu/~ted/files/talks/DUALITY09.pdfDuality for Mixed Integer Linear Programming ... In mathematical programming, the input

lehigh-logo

Extending the Value Function (cont.)

Yes! For large enoughh, maximal extension produces the value function itself.

Theorem

Letdr = maxai | i ∈ N anddl = minai | i ∈ N and let the functionsfr andfl bethe maximal subadditive extensions ofz from the intervals[0, dr ] and[dl , 0] to R+ andR−, respectively. Let

F(d) =

fr(d) d ∈ R+

fl(d) d ∈ R−

then,z = F.

Outline of the Proof.

z≤ F : By construction.

z≥ F : Using MILP duality,F is dual feasible.

In other words, the value function is completely encoded by the breakpoints in[dl , dr ]and 2 slopes.

Ralphs and Guzelsoy (Lehigh University) Duality for MILP 18 February 2009 34 / 56

Page 35: Duality for Mixed Integer Linear Programmingcoral.ie.lehigh.edu/~ted/files/talks/DUALITY09.pdfDuality for Mixed Integer Linear Programming ... In mathematical programming, the input

lehigh-logo

General Procedure

We will construct the value function in two stepsConstruct the value function on[dl , dr ].Extend the value function to the entire real line from[dl , dr ].

AssumptionsWe assumeηC < ∞ andζC < ∞.We construct the value function overR+ only.These assmuptions are only needed to simplify the presentation.

Constructing the Value Function on[0, dr ]

If both ηC andζC arefinite, the value function iscontinuousand the slopes of thelinear segments alternate betweenηC andζC.For d1, d2 ∈ [0, dr ], if z(d1) andz(d2) are connected by a line with slopeηC or ζC,thenz is linear over[d1, d2] with the respective slope (subadditivity).With these observations, we can formulate a finite algorithm to evaluatez in [dl , dr ].

Ralphs and Guzelsoy (Lehigh University) Duality for MILP 18 February 2009 35 / 56

Page 36: Duality for Mixed Integer Linear Programmingcoral.ie.lehigh.edu/~ted/files/talks/DUALITY09.pdfDuality for Mixed Integer Linear Programming ... In mathematical programming, the input

lehigh-logo

Example (cont’d)

dr = 6:

ηC —— ζC ——

0 2 4 6

2

4

6

8

Figure:Evaluatingz in [0, 6]

Ralphs and Guzelsoy (Lehigh University) Duality for MILP 18 February 2009 36 / 56

Page 37: Duality for Mixed Integer Linear Programmingcoral.ie.lehigh.edu/~ted/files/talks/DUALITY09.pdfDuality for Mixed Integer Linear Programming ... In mathematical programming, the input

lehigh-logo

Example (cont’d)

dr = 6:

ηC —— ζC ——

0 2 4 6

2

4

6

8

Figure:Evaluatingz in [0, 6]

Ralphs and Guzelsoy (Lehigh University) Duality for MILP 18 February 2009 37 / 56

Page 38: Duality for Mixed Integer Linear Programmingcoral.ie.lehigh.edu/~ted/files/talks/DUALITY09.pdfDuality for Mixed Integer Linear Programming ... In mathematical programming, the input

lehigh-logo

Example (cont’d)

dr = 6:

ηC —— ζC ——

0 2 4 6

2

4

6

8

Figure:Evaluatingz in [0, 6]

Ralphs and Guzelsoy (Lehigh University) Duality for MILP 18 February 2009 38 / 56

Page 39: Duality for Mixed Integer Linear Programmingcoral.ie.lehigh.edu/~ted/files/talks/DUALITY09.pdfDuality for Mixed Integer Linear Programming ... In mathematical programming, the input

lehigh-logo

Example (cont’d)

dr = 6:

ηC —— ζC ——

0 2 4 6

2

4

6

8

Figure:Evaluatingz in [0, 6]

Ralphs and Guzelsoy (Lehigh University) Duality for MILP 18 February 2009 39 / 56

Page 40: Duality for Mixed Integer Linear Programmingcoral.ie.lehigh.edu/~ted/files/talks/DUALITY09.pdfDuality for Mixed Integer Linear Programming ... In mathematical programming, the input

lehigh-logo

Example (cont’d)

dr = 6:

ηC —— ζC ——

0 2 4 6

2

4

6

8

Figure:Evaluatingz in [0, 6]

Ralphs and Guzelsoy (Lehigh University) Duality for MILP 18 February 2009 40 / 56

Page 41: Duality for Mixed Integer Linear Programmingcoral.ie.lehigh.edu/~ted/files/talks/DUALITY09.pdfDuality for Mixed Integer Linear Programming ... In mathematical programming, the input

lehigh-logo

Example (cont’d)

Extending the value function of (SP) from[0, 12] to [0, 18]

0 2 4 6 8 10 12 14 16 18 20 22 24 26 28-2-4-6-8-10-12-14-16

2

4

6

8

10

12

14

16

18

d

FL

FU

z

Υ

Ralphs and Guzelsoy (Lehigh University) Duality for MILP 18 February 2009 41 / 56

Page 42: Duality for Mixed Integer Linear Programmingcoral.ie.lehigh.edu/~ted/files/talks/DUALITY09.pdfDuality for Mixed Integer Linear Programming ... In mathematical programming, the input

lehigh-logo

Example (cont’d)

Extending the value function of (SP) from[0, 12] to [0, 18]

0 2 4 6 8 10 12 14 16 18 20 22 24 26 28-2-4-6-8-10-12-14-16

2

4

6

8

10

12

14

16

18

d

FL

FU

z

Υ

Ralphs and Guzelsoy (Lehigh University) Duality for MILP 18 February 2009 42 / 56

Page 43: Duality for Mixed Integer Linear Programmingcoral.ie.lehigh.edu/~ted/files/talks/DUALITY09.pdfDuality for Mixed Integer Linear Programming ... In mathematical programming, the input

lehigh-logo

Example (cont’d)

Extending the value function of (SP) from[0, 18] to [0, 24]

0 2 4 6 8 10 12 14 16 18 20 22 24 26 28-2-4-6-8-10-12-14-16

2

4

6

8

10

12

14

16

18

d

FL

FU

z

Υ

Ralphs and Guzelsoy (Lehigh University) Duality for MILP 18 February 2009 43 / 56

Page 44: Duality for Mixed Integer Linear Programmingcoral.ie.lehigh.edu/~ted/files/talks/DUALITY09.pdfDuality for Mixed Integer Linear Programming ... In mathematical programming, the input

lehigh-logo

Example (cont’d)

Extending the value function of (SP) from[0, 18] to [0, 24]

0 2 4 6 8 10 12 14 16 18 20 22 24 26 28-2-4-6-8-10-12-14-16

2

4

6

8

10

12

14

16

18

d

FL

FU

z

Υ

Ralphs and Guzelsoy (Lehigh University) Duality for MILP 18 February 2009 44 / 56

Page 45: Duality for Mixed Integer Linear Programmingcoral.ie.lehigh.edu/~ted/files/talks/DUALITY09.pdfDuality for Mixed Integer Linear Programming ... In mathematical programming, the input

lehigh-logo

General Case

For E ∈ E , setting

ωE(d) = g(⌊d⌋E) + vE(d− ⌊d⌋E) ∀d ∈ Rm with S(d) 6= ∅,

we can writez(d) = min

E∈E

ωE(d) ∀d ∈ Rm with S(d) 6= ∅.

Many of our previous results can be extended to general case in the obvious way.

Similarly, we can use maximal subadditive extensions to construct the valuefunction.

However, an obvious combinatorial explosion occurs.

Therefore, we consider using single row relaxations to get asubadditiveapproximation.

Ralphs and Guzelsoy (Lehigh University) Duality for MILP 18 February 2009 45 / 56

Page 46: Duality for Mixed Integer Linear Programmingcoral.ie.lehigh.edu/~ted/files/talks/DUALITY09.pdfDuality for Mixed Integer Linear Programming ... In mathematical programming, the input

lehigh-logo

Gomory’s Procedure

There is a Chvatal function that is optimal to the subadditive dual of an ILP withRHSb ∈ ΩIP andzIP(b) > −∞.

The procedure:In iterationk, we solve the following LP

zIP(b)k−1 = min cxs.t. Ax = b

∑nj=1 f i(aj)xj ≥ f i(b) i = 1, ..., k− 1

x ≥ 0

Thekth cut,k > 1, is dependent on the RHS and written as:

f k(d) =

m∑

i=1

λk−1i di +

k−1∑

i=1

λk−1m+i f

i(d)

where λk−1 = (λk−11 , ..., λk−1

m+k−1) ≥ 0

Ralphs and Guzelsoy (Lehigh University) Duality for MILP 18 February 2009 46 / 56

Page 47: Duality for Mixed Integer Linear Programmingcoral.ie.lehigh.edu/~ted/files/talks/DUALITY09.pdfDuality for Mixed Integer Linear Programming ... In mathematical programming, the input

lehigh-logo

Gomory’s Procedure (cont.)

Assume thatb ∈ ΩIP, zIP(b) > −∞ and the algorithm terminates afterk + 1iterations.

If uk is the optimal dual solution to the LP in the final iteration, then

Fk(d) =

m∑

i=1

uki di +

k∑

i=1

ukm+i f

i(d),

is a Chvatal function withFk(b) = zIP(b) and furthermore, it is optimal to thesubadditive ILP dual problem.

Ralphs and Guzelsoy (Lehigh University) Duality for MILP 18 February 2009 47 / 56

Page 48: Duality for Mixed Integer Linear Programmingcoral.ie.lehigh.edu/~ted/files/talks/DUALITY09.pdfDuality for Mixed Integer Linear Programming ... In mathematical programming, the input

lehigh-logo

Gomory’s Procedure (cont.)

Example:Consider an integer program with two constraints. Suppose that thefollowing is the first constraint:

2x1 + −2x2 + x3 − x4 = 3

At the first iteration, Gomory’s procedure can be used to derive the valid inequality

⌈2/2⌉x1 + ⌈ − 2/2⌉x2 + ⌈1/2⌉x3 + ⌈ − 1/2⌉x4 ≥ ⌈3/2⌉

by scaling the constraint withλ = 1/2. In other words, we obtainx1 − x2 + x3 ≥ 2.Then the corresponding function is:

F1(d) = ⌈d1/2⌉ + 0d2 = ⌈d1/2⌉

By continuing Gomory’s procedure, one can then build up an optimal dual function inthis way.

Ralphs and Guzelsoy (Lehigh University) Duality for MILP 18 February 2009 48 / 56

Page 49: Duality for Mixed Integer Linear Programmingcoral.ie.lehigh.edu/~ted/files/talks/DUALITY09.pdfDuality for Mixed Integer Linear Programming ... In mathematical programming, the input

lehigh-logo

Branch-and-Bound Method

Assume that the primal problem is solved to optimality.Let T be the set of leaf nodes of the search tree.Thus, we’ve solved the LP relaxation of the following problem at nodet ∈ T

zt(b) = min cxs.t x ∈ St(b)

,

whereSt(b) = Ax = b, x ≥ lt,−x ≥ −ut, x ∈ Zn andut, lt ∈ Zr are thebranching bounds applied to the integer variables.Let (vt, vt, vt) be

the dual feasible solution used to prune nodet, if t is feasibly pruneda dual feasible solution (that can be obtained from it parent) to nodet, if t isinfeasibly pruned

Then,

FBB(d) = mint∈T

vtd + vtlt − vtut

is an optimal solution to the generalized dual problem.

Ralphs and Guzelsoy (Lehigh University) Duality for MILP 18 February 2009 49 / 56

Page 50: Duality for Mixed Integer Linear Programmingcoral.ie.lehigh.edu/~ted/files/talks/DUALITY09.pdfDuality for Mixed Integer Linear Programming ... In mathematical programming, the input

lehigh-logo

Branch-and-Cut Method

It is thus easy to get a strong dual function from branch-and-bound.

Note, however, that it’s not subadditive in general.

To obtain a subaditive function, we can include the variablebounds explicitly asconstraints, but then the function may not be strong.

For branch-and-cut, we have to take care of the cuts.Case 1: Do we know the subadditive representation of each cut?Case 2: Do we know the RHS dependency of each cut?Case 3: Otherwise, we can use some proximity results or the variable bounds.

Ralphs and Guzelsoy (Lehigh University) Duality for MILP 18 February 2009 50 / 56

Page 51: Duality for Mixed Integer Linear Programmingcoral.ie.lehigh.edu/~ted/files/talks/DUALITY09.pdfDuality for Mixed Integer Linear Programming ... In mathematical programming, the input

lehigh-logo

Case 1

If we know the subadditive representation of each cut:At a nodet, we solve the LP relaxation of the following problem

zt(b) = min cxs.t Ax ≥ b

x ≥ lt

−x ≥ −gt

Htx ≥ ht

x ∈ Zr+ × Rn−r

+

wheregt, lt ∈ Rr are the branching bounds applied to the integer variables andHtx ≥ ht is the set of added cuts in the form

j∈I

Ftk(a

kj )xj +

j∈N\I

Ftk(a

kj )xj ≥ Ft

k(σk(b)) k = 1, ..., ν(t),

ν(t): the number of cuts generated so far,ak

j , j = 1, ..., n: the columns of the problem that thekth cut is constructed from,σk(b): is the mapping ofb to the RHS of the corresponding problem.

Ralphs and Guzelsoy (Lehigh University) Duality for MILP 18 February 2009 51 / 56

Page 52: Duality for Mixed Integer Linear Programmingcoral.ie.lehigh.edu/~ted/files/talks/DUALITY09.pdfDuality for Mixed Integer Linear Programming ... In mathematical programming, the input

lehigh-logo

Case 1

Let T be the set of leaf nodes,ut, ut, ut andwt be the dual feasible solution used toprunet ∈ T. Then,

F(d) = mint∈T

utd + utlt − utgt +

ν(t)∑

k=1

wtkF

tk(σk(d))

is an optimal dual function, that is,z(b) = F(b).

Again, we obtain a subaddite function if the variables are bounded.

However, we may not know the subadditive representation of each cut.

Ralphs and Guzelsoy (Lehigh University) Duality for MILP 18 February 2009 52 / 56

Page 53: Duality for Mixed Integer Linear Programmingcoral.ie.lehigh.edu/~ted/files/talks/DUALITY09.pdfDuality for Mixed Integer Linear Programming ... In mathematical programming, the input

lehigh-logo

Case 2

If we know the RHS dependencies of each cut:We know for

Gomory fractional cuts.

Knapsack cuts

Mixed-integer Gomory cuts

?

Then, we do the same analysis as before.

Ralphs and Guzelsoy (Lehigh University) Duality for MILP 18 February 2009 53 / 56

Page 54: Duality for Mixed Integer Linear Programmingcoral.ie.lehigh.edu/~ted/files/talks/DUALITY09.pdfDuality for Mixed Integer Linear Programming ... In mathematical programming, the input

lehigh-logo

Case 3

In the absence of subadditive representations or RHS dependencies:For each nodet ∈ T, let ht be such that

htk =

n∑

j=1

htkjxj with xj =

ltj if htkj ≥ 0

gtj otherwise

, k = 1, ..., ν(t)

wherehtkj is thekth entry of columnht

j . Furthermore, define

ht =n

j=1

htj xj with xj =

ltj if wthtj ≥ 0

gtj otherwise

.

Then the function

F(d) = mint∈T

utd + utlt − utgt + maxwtht, wtht

is a feasible dual solution and henceF(b) yields a lower bound.This approach is the easiest way and can be used for bounded MILPs (binaryprograms), however it is unlikely to get an effective dual feasible solution forgeneral MILPs.

Ralphs and Guzelsoy (Lehigh University) Duality for MILP 18 February 2009 54 / 56

Page 55: Duality for Mixed Integer Linear Programmingcoral.ie.lehigh.edu/~ted/files/talks/DUALITY09.pdfDuality for Mixed Integer Linear Programming ... In mathematical programming, the input

lehigh-logo

Conclusions

It is possible to generalize the duality concepts that are familiar to us from linearprogramming.

However, it is extremely difficult to make any of it practical.

There are some isolated cases where this theory has been applied in practice, butthey are far and few between.

We have a number of projects aimed in this direction, so stay tuned...

Ralphs and Guzelsoy (Lehigh University) Duality for MILP 18 February 2009 55 / 56