overview of mixed-integer (linear) programming: recent ...focapo-cpc.org/pdf/linderoth.pdf ·...

Mixed-Integer (Linear) Programming: Recent Advances,and Future Research Directions

Jeff Linderoth

Dept. of ISyEUniv. of Wisconsin-Madison

[email protected]

@JeffLinderoth

Linderoth (UW ISyE) Quo Vadis MIP FOCAPO 1 / 58

Introduction

Mixed Integer Linear Program: (MILP)

minx,z

c>x+ d>z

s.t. Ax+ Bz ≤ bx ∈ Rn+z ∈ Zp

Integer variables always have bounds (e.g. zi ∈ {0, 1})

Powerful Modeling Paradigm

“On-off” constraints

Logical conditions between constraints and variables

Cardinality constraints: Select at most k out of p alternatives

Nonlinearities: Fixed-charges, piecewise linear functions


Introduction Make Fun of Christos

Instructions

At this point, we would like to send

you some guidelines:

1) Please limit your talk to *35

minutes*. We would like to have ample

time for discussion.

...5) One of the major pieces of input we

received from the technical advisory

committee was that invited talks should

not only present new results, but also

succinctly (1) review the state-of-the

art, and (2) discuss new challenges and

open questions.



MISSION IMPOSSIBLE

Talk may be a bit basic for some of the audience

There will be some (biased) discussion of my own team’scontributions

If you aren’t a (recent) MILP user, I have only one take-away message:



Branch-And-Cut: The “Virginia Slims” of Algorithms

If you haven’t tried MILP because you believe that it is “impossible”to solve, you may be wrong


Progress in MIP Solvers Main Solvers

The Big Three

Three Primary Solvers for MILP

1 IBM CPLEX: (Version 12.7)

2 FICO XPRESS: (Version 8)

3 Gurobi: (Version 7)

There are other commercial solvers (e.g.) Mosek, but for MILP theseare the big three

Commercial Solvers remain head and shoulders1 abovefree/open-source solvers

SCIP: http://scip.zib.de/ Free for non-academic useMIP-CL: http://www.mipcl-cpp.appspot.com/Coin-Cbc: Part of open-source Coin-OR initiative coin-or.org

1Maybe 6-7 times fasterLinderoth (UW ISyE) Quo Vadis MIP FOCAPO 6 / 58

http://scip.zib.de/

http://www.mipcl-cpp.appspot.com/

coin-or.org

Progress in MIP Solvers NEOS

NEOS: www.neos-server.org

Allow users to solve optimization problems on remote resources

Started at Argonne National Lab in 1994

Moved to UW-Madison (Wisconsin Institute of Discovery) in 2010.

Web interface, API (XML-RPC)

Access to 11 different solvers for MILP


www.neos-server.org


NEOS Daily Jobs, 2016

Over one million optimizationproblems solved in 2016!



Most Popular Problems/Solvers


Progress in MIP Solvers MILP Speedups

CPLEX MILP Improvement: v1-v11



CPLEX MILP Improvement: v6-v12.6.1



Calculations

Improvement in MIP Software from 1988-2017

Algorithms: 147650x

Machines: 17120x

http://preshing.com/20120208/a-look-back-at-single-threaded-cpu-performance/

NET: (Algorithm × Machine): 2,527,768,000x

What Does This “Mean”?

A “typical” MILP that would have taken 124 years to solve in 1988will solve in 1 second now.

This is amazing, but your mileage may vary


http://preshing.com/20120208/a-look-back-at-single-threaded-cpu-performance/

Progress in MIP Solvers Stories

In case you don’t believe me...

Electrical Power Industry, ERPI GS-6401, June 1989:

Mixed-integer programming (MIP) is a powerful modelingtool. They are, however, theoretically complicated andcomputationally cumbersome.

As a result, the utility (power) industries did not use mixed-integerlinear programming for many years.



Join the New Decade

Bob Bixby, Founder of CPLEX, and now the“Bi” in GuRoBi:

“Give me your most difficult MILP model”

California 7-Day Model

m = 48939 constraints,n = 25755 variables, p = 2856integer variables

“Many Days”—No solution.

(> 1 hour to solve LPrelaxation)



Dateline: 9/27/1999

At the DIMACS/EPRI Workshop on Next Generation of UnitCommitment Models

http://dimacs.rutgers.edu/Workshops/NextGeneration/

program.html

Solved model in 22 mins to fulloptimality.

Using CPLEX v6.5

During his talk!


http://dimacs.rutgers.edu/Workshops/NextGeneration/program.html

http://dimacs.rutgers.edu/Workshops/NextGeneration/program.html


Now Even Faster!Gurobi Optimizer version 5.5.0

Read MPS format model from file unitcal_7.mps.bz2

Optimize a model with 48939 Rows, 25755 Columns and 127595 NonZeros

Presolved: 38804 Rows, 19960 Columns, 105627 Nonzeros

Root relaxation: objective 1.945018e+07, 18340 iterations, 0.60 seconds

Nodes | Current Node | Objective Bounds | Work

Expl Unexpl | Obj Depth IntInf | Incumbent BestBd Gap | It/Node Time

0 0 1.9450e+07 0 721 - 1.9450e+07 - - 2s

0 0 1.9596e+07 0 559 - 1.9596e+07 - - 16s

0 0 1.9598e+07 0 487 - 1.9598e+07 - - 20s

H 0 0 2.066856e+07 1.9598e+07 5.18% - 25s

13 11 1.9669e+07 6 217 2.0669e+07 1.9602e+07 5.16% 649 30s

36 28 1.9668e+07 9 219 2.0669e+07 1.9605e+07 5.15% 707 35s

H 93 84 1.998399e+07 1.9605e+07 1.90% 342 37s

100 74 1.9678e+07 17 204 1.9984e+07 1.9606e+07 1.89% 321 43s

* 271 111 54 1.972042e+07 1.9606e+07 0.58% 170 43s

H 417 29 1.964604e+07 1.9606e+07 0.21% 129 43s

858 178 1.9629e+07 11 141 1.9637e+07 1.9609e+07 0.14% 96.9 50s

H 924 187 1.963578e+07 1.9609e+07 0.13% 94.6 51s

H 987 221 1.963563e+07 1.9611e+07 0.12% 93.9 53s

1024 237 1.9630e+07 19 107 1.9636e+07 1.9611e+07 0.12% 93.5 56s

H 1034 237 1.963556e+07 1.9611e+07 0.12% 92.8 56s

1144 288 1.9630e+07 14 501 1.9636e+07 1.9611e+07 0.12% 89.0 63s

1245 294 1.9621e+07 30 399 1.9636e+07 1.9619e+07 0.09% 131 185s

1673 261 1.9623e+07 35 120 1.9636e+07 1.9623e+07 0.07% 117 190s

Explored 2167 nodes (274011 simplex iterations) in

194.37 seconds

Optimal solution found (tolerance 1.00e-04)


Progress in MIP Solvers ran14x18

Story #2: ran14x18

In 1998, there was no Twitter. The best search engine was Altavista2,and there were “newsgroups” on the Interwebs where you could askquestions.

Challenge post to sci.op-research, August 4, 1998: (From authorsof paper Sun et al. (1998))

“We tried CPLEX 5.0 on this instance, obtaining a solutionwith objective value 3712 but without proving optimality.The same solution was also found in some runs of thegenetic algorithm. In response to my posting onsci.op-research, some people tried exact solvers on thisinstance (fctp, mps), but without success. I also contactedCPLEX, but they also failed.”

2Google was incoporated in August 1998Linderoth (UW ISyE) Quo Vadis MIP FOCAPO 17 / 58


Solving ran14x18

Dateline, Nov 13, 1998:

“I proved the optimality of the solution3712 for the ran14x18 FCTP instance. Inorder to do this I ran my PARINO parallelMIP code for about 52 hours on 32Pentium II (300MHz) processors. 7624677nodes of the branch and bound tree wereevaluated in this time. PARINO adds flowcover inequalities which are usually quiteuseful in helping to improve the LP boundsfor these types of problems. Also, I used aparallel pseudocost-type branching rule



CPLEX Too

Dateline, Nov 25, 1998

We are in the process of developing anew version of CPLEX, so we decidedto try the ran14x18 problem with ournew version on a large number ofparallel processors. We used an SGIOrigin system with 64 processors. Ourparallel development code solved it in8982253 nodes, and 3 hours 12minutes wall clock time.

NB: This was when Gu (now the “Gu” in GuRoBi) just joined CPLEX



Now: gurobi v7.0 on my Windows Desktop



The Seymour Problem

To find a “short” proof of the Four-Color Theorem3, Robertson,Seymour, Thomas wanted to find the smallest set of “configurations”such that the Four-Color Theorem is true if none of theseconfigurations can exist in a minimal counterexample.

This can be posed as an integer program

Although Seymour claimed to have found a solution with objectivevalue 423, nobody (including Seymour himself) had been able eitherto reproduce this solution, or prove a strong lower bound on theoptimal value.

There was some skepticism in the integer programming community asto whether this was indeed the optimal value.

3Any map can be colored using four colors in such a way that regions sharing aboundary segment receive different colors.



Ferris + Linderoth versus Seymour

Still a young and energetic Postdoc, I visited Michael Ferris atUW-Madison in 1999.

After setting a varety of algorithmic parameters, we set CPLEX 6.5running: It ran (using grid-software computing tool called HTCondorthat could checkpoint, save, and relocate computation to differentmachines) for > 10 years in the UW-Madison HTCondor Pool.

BUMMER!

It never solved!

Hooray!

With a different method using disjunctive cuts, and some“extreme” enumeration (in parallel), Pataki, Schmieta, Ceria,Ferris, and Linderoth solved in in 9000 CPU hours.

Indeed the solution is 423.



Now It’s Officially Easy!

This is just amazing!


MILP Algorithms Components

How Did They Do It?

1 Hired very good people!

2 Mined the academic literature. (Read: stole all of our good ideas).

Key Idea!

Recognize and exploit commonly appearing structures:

Preprocessing techniques

Cutting planes

Improved search/branching

Heuristic methods for finding feasible solutions



(M)ILP Feasible Region



Convex Hull



Cutting Planes



Branching



Branching Makes A Tree

x1 =1

x2 = 0

x1 = 0

x2 = 1

Recursively solve IP at each node

Fathom is lower bound at node is larger than value of a knownfeasible solution



MIP Building Blocks

Presolve: Tighten formulation and reduce problem size

Solve continuous relaxations

Ignoring integralityGives a bound on the optimal integral objective

Cutting planes: Cut off relaxation solutions

Branching variable selection: Intelligently explore search space

Primal heuristics: Find integer feasible solutions



Branch And Cut Components



Some Papers

Branching

An old(ish) survey of methods: (Linderoth and Savelsbergh, 1999)

“Reliability Branching”: (Achterberg et al., 2004)

Cutting

Gomory Cuts: (Balas et al., 1999)

(0-1/2) Cuts: (Caprara and Fischetti, 1996)

Disjunctive Cuts: (Balas and Perregaard, 2003; Bonami andMinoux, 2005)

Flow Covers: Gu et al. (1999)

MIR: Marchand and Wolsey (2001)

Surveys: Cornuejols (2008); Conforti et al. (2009)



More Papers

Heuristics

Local Branching: (Fischetti and Lodi, 2003)

Feasibility Pump: (Fischetti et al., 2005; Achterberg and Berthold,2007)

RINS: (Danna et al., 2005)

Presolve

Basic reduction in MILP: (Savelsbergh, 1994)

Achterberg et al. (2016): A 70(!) page paper that explainspresolve operations in Gurobi



{1} is a Series!



Importance of Each Component


Orbital Branching

Improvement in Branching: Orbital Branching

One recent branching improvement is for solving symmetric MILPs


Orbital Branching

Symmetry is Pretty!

The A matrix of a symmetric integer program


Orbital Branching

Symmetry Background

(We assume all variables binary for this discussion)

Let Πn be collection of permutations of {1, 2, . . . , n} := [n]

π ∈ Πn is a symmetry of IP if...1 x feasible ⇔ π(x) feasible2 cTx = cTπ(x)

The set of symmetries of IP (with composition of permutations)forms the symmetry group of IP

G(IP) = {π ∈ Πn | π(x) ∈ F , cTx = cTπ(x) ∀x ∈ F },

where F = {x ∈ {0, 1}n | Ax ≤ b} is the set of feasible solutions


Orbital Branching

Why Symmetry is Bad – Wasted Search!

Suppose the permutation (1, 2) ∈ G

x1 =1

x2 = 0

x1 = 0

x2 = 1

You evaluate many completely equivalent (isomorphic) subtrees


Orbital Branching

Orbits

For a point z ∈ Rn, the orbit of z under G is the set of all elements towhich z can be sent by permutations in G:

orb(G, z) def= {π(z) | π ∈ G}.

The union of the orbits of each variable forms a partition of [n].

The orbits encode which variables are “equivalent” (symmetric) withrespect to the symmetry G.


Orbital Branching Basic Idea

Orbital Branching

Let O be an orbit of the symmetry group of the IP.

Surely we can branch as∑i∈O

xi ≥ 1 or∑i∈O

xi ≤ 0.

If at least one variable i ∈ O is going to be one, and they are all“equivalent”, then you may as well pick (i∗) one arbitrarily.

x∗i = 1 or∑i∈O

xi = 0

No, really. That’s it. :-)


Orbital Branching Basic Idea

An Alternative View of Orbital Branching

Suppose that you have found that the variables xe, xf, xg and xhshare an orbit at node a, O = {e, f, g, h}.

Then you can surely branch as:

a

e f g hxe = 1 xf = 1 xg = 1 xh = 1

∑j∈O xj = 0

But the best solution you can find from nodes f, g, and h will be thesame as the best solution you can find from node e

In fact, the solutions will be isomorphic⇒ Prune nodes f, g, and h


Orbital Branching Computational Results

Tale of the Tape—Gurobi v3.0

Symmetry = 0 Symmetry = 2Instance Time Gap% Nodes Time Gap% Nodes

ecc105 7200 50.0 150 173 0.0 7ecc83 7200 15.0 724601 6 0.0 372ecc93 7200 20.0 108572 905 0.0 54650

codbt05 7200 7.4 352025 7200 3.7 359268codbt33 8 0.0 604 6 0.0 401codbt42 159 0.0 75569 111 0.0 45912codbt61 10 0.0 1485 7 0.0 950cov1053 7200 5.9 919836 77 0.0 10958cov1054 7200 2.0 189645 2330 0.0 103657cov1075 7200 5.0 549355 17 0.0 665cov954 58 0.0 31950 1 0.0 166sts27 1 0.0 4044 0 0.0 78sts45 18 0.0 61194 23 0.0 34839sts63 7200 4.4 8698168 85 0.0 43135sts81 7200 16.4 3252747 70 0.0 6317


Orbital Branching Computational Results

Tale of the Tape—CPLEX v12.1

Symmetry = 0 Symmetry = 5Instance Time Gap% Nodes Time Gap% Nodescod105 7200 52.4 13201 606 0.0 1120cod83 7200 14.3 1418001 79 0.0 15452cod93 7200 18.9 389028 7200 6.3 639001

codbt05 7200 5.6 1035046 150 0.0 23059codbt33 8 0.0 1049 1 0.0 14codbt42 89 0.0 84039 4 0.0 2141codbt61 8 0.0 1833 1 0.0 61cov1053 7200 5.9 1495461 2234 0.0 448008cov1054 7200 2.0 191970 7200 2.0 169371cov1075 7200 6.4 1505168 57 0.0 12227cov954 64 0.0 36563 3 0.0 1351sts27 0 0.0 3532 0 0.0 1307sts45 10 0.0 59890 6 0.0 28775sts63 1585 0.0 7692765 736 0.0 3607609sts81 7200 13.1 23933498 7200 11.5 23415204


Future Directions: Nonlinear

The World Is Nonlinear

An anecdote: July, 1948. A young andfrightened George Dantzig, presents hisnewfangled “linear programming” to ameeting of the Econometric Society ofWisconsin, attended by distinguishedscientists like Hotelling, Koopmans, and VonNeumann. Following the lecture, Hotelling(in Dantzig’s words “a huge whale of aman”) pronounced to the audience:

But we all know the world isnonlinear!

The world is indeednonlinear

Physical Processes andProperties

EquilibriumEnthalpy

Abstract Measures

Economies of ScaleCovarianceUtility of decisions



Mixed-Integer Nonlinear Optimization

Mixed Integer Linear Program: (MILP)

zminlp = minx,η

η

s.t. gi(x) ≤ 0, i = 1, . . . ,m

f(x) ≤ ηxI ∈ Z|I|

f : Rn → R, g : Rn → Rm smooth, sometimes convex functions.

I ⊆ {1, . . . , n} subset of integer variables

We may assume that f is a linear function.

NP-Super-Hard

Combines challenges of handling nonlinearitieswith combinatorial explosion of integer variables



Specializations

Functional Form Problem Type

gi(x) = ‖Aix+ bi‖2 − pTi x+ qi MISOCP

gi(x) = xTQix+ p

Ti + qi MIQP

gi(x) = aTi x− bi MILP

MISOCP: Mixed Integer Second Order Cone Program

Are convex-MINLP

MIQP: Mixed Integer Quadratic Program

May be convex or nonconvexConvex MIQP is a special case of MISOCPIf f is convex quadratic and g is an affine mapping, then there arespecialized algorithms for convex-MIQP



Aside – MISOCP

The feasible regions of surprisingly many (convex) MINLP can beexpressed as MISOCP:

Hyperbolic constraints: Product of (non-negative) variables ≥ anorm2.

wTw ≤ xy, x ≥ 0, y ≥ 0 ⇔ ∥∥∥∥[ 2w

x− y

]∥∥∥∥ ≤ x+ yLubin et al. (2016) show that every convex MINLP in the availabletest libraries can be reformulated to be a conic MINLP

Most are quadratic cone, but need also “power cone” and “exponentialcone”



New Math

Once academics could no longer compete against the likes of CPLEX,they turned their attentions to MINLP.

Try to apply same MILP improvement techniques to MINLP

MI

+

NLP

6=

MINLP



MINLP Progress (via MILP)

Undaunted by initial failures, many MILP researchers continue towork in MINLP

At MIP 2014 and MIP 2015, 7 of the 22 talks each each were aboutMINLP

Some Work Product. (There’s LOTS More)

Cutting planes: (Atamturk and Narayanan, 2010; Kılınc-Karzan,2016; Kılınc et al., 2017; S. Modaresi and Vielma, 2016)

Heuristics: (Bonami et al., 2008)

Preprocessing: (Grossmann and Lee, 2003; Gunluk and Linderoth,2010)

Branching: (Bonami et al., 2011)



Leading To Improvements – Convex MIQP



Leading To Improvements – Nonconvex MIQP


Future Directions: Nonlinear Wrapping Up

Optima 101

I recently had the opportunity to interview Bob Bixby for Optima(The Mathematical Optimization Society Newsletter):http://www.mathopt.org/Optima-Issues/optima101.pdf

“In some sense, companies like CPLEX,XPRESS, and Gurobi put academics

doing computational IP kind of out ofbusiness. Many people are doing MINLP

now.”


http://www.mathopt.org/Optima-Issues/optima101.pdf


Bixby Response

“I think there is still plenty of room forMIP actually. The truth is that therehas to be relatively little thats come outof the research community in the last 10years or so.

Certainly RINS was a big thing. Thatshad a huge influence. Symmetry testingwas a big thing, but not a huge thing.That got put in Gurobi, based on orbitalbranchings.



Conversation Continued

“It seems harder and harder to generateand demonstrate good/impactful ideaswithout access to the internals ofcommercial IP solvers.”

“Its perfectly clear, its very hard to testwhether something is a good idea or not.The most you can hope for these days isthat it looks reasonable when you do alittle bit of testing. Then you knowpeople working for the commercialsolvers will look at it and try to dosomething.Managing cutting planes can be verydifficult, and we would actually be happyif there were some nice thing we coulddo.”



Conclusions: Misson (NOT) Accomplished

Take Away Messages!

1 Christos Asks The Impossible!2 MILP has come a long way baby!

A positive academic/commercial“partnership”If you are not using MILP, Take herfor a spin!

3 MINLP is a current “frontier” ofprimary improvement for commercialsoftware



Things We Still Can’t Do Well

Optimization solver so far onlyeffectively deal with algebraicsystems!

Paul is here to help!

We can’t effectively solveoptimization problems with bothinteger variables and uncertainparameters

Shabbir is the man!


References

T. Achterberg and T. Berthold. Improving the feasibility pump. Discrete Optimization, 4:77–86,2007.

T. Achterberg, T. Koch, and A. Martin. Branching rules revisited. Operations Research Letters,33:42–54, 2004.

T. Achterberg, R. E. Bixby, Z. Gu, E. Rothberg, and D. Weninger. Presolve reductions in mixedinteger programming. Technical Report ZIB-Report 16-44, Konrad-Zuse-Zentrum furInformationstechnik Berlin, 2016.

A. Atamturk and V. Narayanan. Conic mixed integer rounding cuts. MathematicalProgramming, 122:1–20, 2010.

E. Balas, S. Ceria, G. Cornuejols, and N.R. Natraj. Gomory cuts revisited. Operations ResearchLetters, 19:1–9, 1999.

Egon Balas and Michael Perregaard. A precise correspondence between lift-and-project cuts,simple disjunctive cuts, and mixed integer Gomory cuts for 0-1 programming. MathematicalProgramming, 94:221–245, 2003.

P. Bonami and M. Minoux. Using rank-1 lift-and-project closures to generate cuts for 0-1 MIPs,a computational investigation. Discrete Optimization, 2:288–307, 2005.

P. Bonami, G. Cornuejols, A. Lodi, and F. Margot. A feasibility pump for mixed integernonlinear programs. Mathematical Programming, 119:331–352, 2008.

P. Bonami, J. Lee, S. Leyffer, and A. Wachter. More branch-and-bound experiments in convexnonlinear integer programming. Preprint ANL/MCS-P1949-0911, Argonne NationalLaboratory, Mathematics and Computer Science Division, September 2011.

A. Caprara and M. Fischetti. {0, 12} chvatal-gomory cuts. Mathematical Programming, 74:

221–235, 1996.Linderoth (UW ISyE) Quo Vadis MIP FOCAPO 0 / 17

References

M. Conforti, G. Cornuejols, and G. Zambelli. Polyhedral approaches to mixed integer linearprogramming. In M. Junger, T. Liebling, D. Naddef, W. Pulleyblank, G. Reinelt, G. Rinaldi,and L. Wolsey, editors, 50 Years of Integer Programming 1958—2008. Springer, 2009.

G. Cornuejols. Valid inequalities for mixed integer linear programs. Mathematical Programming,112:3–44, 2008.

E. Danna, E. Rothberg, and C. LePape. Exploring relaxation induced neighborhoods to improveMIP solutions. Mathematical Programming, 102:71–90, 2005.

M. Fischetti and A. Lodi. Local branching. Mathematical Programming, 98:23–47, 2003.

M. Fischetti, F. Glover, and A. Lodi. The feasibility pump. Mathematical Programming, 104:91–104, 2005.

I. Grossmann and S. Lee. Generalized convex disjunctive programming: Nonlinear convex hullrelaxation. Computational Optimization and Applications, pages 83–100, 2003.

Z. Gu, G. L. Nemhauser, and M. W. P. Savelsbergh. Lifted flow covers for mixed 0-1 integerprograms. Mathematical Programming, 85:439–467, 1999.

O. Gunluk and J. Linderoth. Perspective relaxation of mixed integer nonlinear programs withindicator variables. Mathematical Programming Series B, 104:186–203, 2010.

M. Kılınc, J. Linderoth, and J. Luedtke. Effective separation of disjunctive cuts for convex mixedinteger nonlinear programs. Mathematical Programming Computation, 2017. To appear.

Fatma Kılınc-Karzan. On minimal inequalities for mixed integer conic programs. Mathematics ofOperations Research, 41:477–510, 2016.

J. T. Linderoth and M. W. P. Savelsbergh. A computational study of search strategies in mixedinteger programming. INFORMS Journal on Computing, 11:173–187, 1999.


Perspective Cuts

M. Lubin, E. Yamangil, R. Bent, and J. P. Vielma. Extended formulations in mixed-integerconvex programming. In IPCO 2016: The Eighteenth Conference on Integer Programmingand Combinatorial Optimization, volume 9682, pages 102–113. Springer, 2016.

H. Marchand and L. Wolsey. Aggregation and mixed integer rounding to solve MIPs.Operations Research, 49:363–371, 2001.

M. R. Kln S. Modaresi and J. P. Vielma. Intersection cuts for nonlinear integer programming:Convexification techniques for structured sets. Mathematical Programming, 155:575–611,2016.

M. W. P. Savelsbergh. Preprocessing and probing techniques for mixed integer programmingproblems. ORSA Journal on Computing, 6:445–454, 1994.

M. Sun, J.E. Aronson, P. McKeown, and D. Drinka. A tabu search heuristic procedure for thefixed charge transportation problem. European Journal of Operational Research, 106(2):441–456, 1998.

D. Vandenbussche and G. L. Nemhauser. A branch-and-cut algorithm for nonconvex quadraticprograms with box constraints. Mathematical Programming, 102:559–575, 2005.


Perspective Cuts

Preprocessing for MINLP

MILP Force: Exploit The Structure!

Mixed Integer Linear Programmers carefully study simple problemstructures to come up with “good” formulations for problems

Good formulations closely approximate convex hull of feasiblesolutions

We need to do this for MINLP


Perspective Cuts

Indicator MINLPs

Binary variables z are used as indicator variables.

If zi = 0, components of x controlled by zi collapse to a point

If zi = 1, components of x controlled by zi belong to a convex set

Process Flow Applications

z = 0⇒ x1 = x2 = x3 = x4 = 0

z = 1⇒ f(x1, x2, x3, x4) ≤ 0


Perspective Cuts Convex Hull Characterizations

A Very Simple Example

Rdef={(x, y, z) ∈ R2 × B | y ≥ x2, 0 ≤ x ≤ uz

}

z = 0⇒ x = 0, y ≥ 0z = 1⇒ x ≤ u, y ≥ x2

x

y

z = 1

z

y ≥ x2

Deep Insights

conv(R) ≡ line connecting (0, 0, 0) to y = x2 in the z = 1 plane


Perspective Cuts Convex Hull Characterizations

Characterization of Convex HullDeep Theorem #1

R ={(x, y, z) ∈ R2 × B | y ≥ x2, 0 ≤ x ≤ uz

}conv(R) =

{(x, y, z) ∈ R3 | yz ≥ x2, 0 ≤ x ≤ uz, 0 ≤ z ≤ 1, y ≥ 0

}

x2 ≤ yz, y, z ≥ 0 ≡

The Only NLP I like: Second Order Cone Programming

x2 − yz is not convex

There are effective, robust algorithms for optimizing over conv(R)


Perspective Cuts Perspective

Giving You Some Perspective

For a convex function f : Rn → R, the perspective functionP : Rn+1 → R of f is

P(x, z) def=

{0 if z = 0zf(x/z) if z > 0

The epigraph of P(x, z) is a cone pointed at the origin whose lowershape is f(x)

Exploiting Your Perspective

If zi is an indicator that the (nonlinear, convex) inequality f(x) ≤ 0must hold, (otherwise x = 0), replace the inequality with itsperspective version:

zif(x/zi) ≤ 0

The resulting (convex) inequality is a much tighter relaxation ofthe feasible region.


Perspective Cuts Computations

Nonlinear Facility Location Problem

Problem studied by Gunluk, Lee, and Weismantel (’07) and classes ofstrong cutting planes derived

M: Facilities

N: Customers

xij: percentage of customer j ∈ N demand met by facility i ∈Mzi = 1⇔ facility i ∈M is built

Fixed cost for opening death star i ∈MCost of serving j ∈ N from i ∈M is proportion to teh square of theamount/percentage



Nonlinear Facility Location Formulation

z∗def= min

∑i∈M

cizi +∑i∈M

∑j∈N

qijx2ijyij

subject to

xij ≤ zi ∀i ∈M, ∀j ∈ N∑i∈M

xij = 1 ∀j ∈ N

xij ≥ 0 ∀i ∈M, ∀j ∈ Nzi ∈ {0, 1} ∀i ∈M

x2ij − ziyij ≤ 0 ∀i ∈M, ∀j ∈ N



Strength of Relaxations

zR: Value of NLP relaxation

zGLW : Value of NLP relaxation after GLW cuts

zP: Value of perspective relaxation

z∗: Optimal solution value

|M| |N| zR zGLW zP z∗

10 30 140.6 326.4 346.5 348.715 50 141.3 312.2 380.0 384.120 65 122.5 248.7 288.9 289.325 80 121.3 260.1 314.8 315.830 100 128.0 327.0 391.7 393.2

Woo Hoo!



Impact of SOCP

m = 30, n = 100

Bonmin B&B, GLW, Original: 16697 CPU seconds, 45901 nodes

Bonmin B&B, GLW, w/ineq: 21206 CPU seconds, 29277 nodes

Bonmin B&B, Perspective, 4201 CPU seconds, 39 B&B nodes

Mosek SOCP, Perspective, 23 CPU seconds, 44 B&B nodes

Larger Instances

m n T N

30 200 141.9 6340 100 76.4 5440 200 101.3 4550 100 61.6 4950 200 140.4 47

“The Force isStrong with This

One”


Piecewise Linear

Piecewise Linear

Low-dimensional nonlinearities (even non-convex ones) can beeffectively modeled with piecewise linear functions

MILP methods are used to optimize!

Gurobi/CPLEX: Specialized interface for specifying piecewise linearfunctions

Gurobi: specialiez linear programming (simplex) method for solvingpiecewise linear programs


Piecewise Linear

Decomposition

CPLEX v12.7 has an automatic mechanism for detecting whether otnot the instance has a structure that is amenable to Benders’Decomposition


Piecewise Linear

Cloud/Parallel

Most solvers are building/have built parallel solvers

Multi-core for some time, but more recently they have distributedsolvers.

Gurobi

1 Distributed MIP: Solve a single MIP model across multiplemachines. Manager machine passes problem data to a set ofworker machines and coordinates the overall solution process.

2 Distributed concurrent MIP: Doesn’t divide work, but rather eachmachine uses a different strategy. This seems stupid, but it can infact be very effective.

3 Distributed tuning: Tests model with variety of parameter settings


Piecewise Linear

Today’s Problem

Nonconvex (Box) QP. Q 6� 0

(boxqp) z∗ := max{x ∈ H | 1/2 xTQx+ cTx}

H = {x ∈ R[n] | `i ≤ xi ≤ ui ∀i ∈ [n]}

xTQx =∑

(i,j)∈EQijxixj +∑i∈[n] 1/2 Qiix

2i

Why Would Anyone Care?

As a relaxation of

max{x ∈ (H ∩ P) | 1/2 xTQx+ cTx}

for some polyhedron P

Improve on solver for nonconvex QP that is in CPLEX v12.6


Piecewise Linear

Gu Ro Bi

Zhonghao Gu Ed Rothberg Bob Bixby

Gurobi

A leading commercial provider of state-of-the-art software forsolving optimization problems

Estimated Company Value: $15,000,000


Piecewise Linear

Gu Bo Li

Oktay Gunluk Pierre Bonami Jeff Linderoth

GuBoLi

A made-up company consisting of three also-ran optimizers

Estimated Company Value: $0.15


Piecewise Linear

GuBoLi—Showing OffAvg. CPU Time on small (n ≤ 40) instances for well-known test setof Vandenbussche and Nemhauser (2005).

Couenne v0.4CPLEX v12.6.1GuBoLi (Gunluk, Bonami, Linderoth) v0.00001

Couenne CPLEX GuBoLi

0

1,000

2,000

3,000(2535)

(1670)

(1.57)

CP

UT

ime

“There’s only one thingto do at a moment like

this: strut!”

Bart’s Girlfriend, S6:E7


Piecewise Linear


overview of mixed-integer (linear) programming: recent ...focapo-cpc.org/pdf/linderoth.pdf ·...

Documents