http://www.mosek.com
Solving Linear OptimizationProblems with MOSEK.
Bo Jensen MOSEK ApS,Fruebjergvej 3, Box 16, 2100 Copenhagen,
Denmark.Email: [email protected]
INFORMS Annual San Diego Tuesday Oct 13, 13:30 - 15:00, 2009
Introduction
2 / 26
What is MOSEK
Introduction
Aim of talk
Topic
Introduction of theoptimizers
Introduction of theoptimizers(Continued)
Computational testsetup
Improvements in thepresolve
Improvements in thesimplex optimizers
Improvements in theinterior pointoptimizer
Conclusions
3 / 26
■ A software package for solving large-scale optimizationproblems.
■ Solves linear, conic, and nonlinear convex problems.Mixed integer optimizer.
■ Used to solve problems with up to millions of constraintsand variables.
■ Stand-alone as well as embedded :
◆ C◆ Java◆ MATLAB◆ Python◆ .NET
What is MOSEK (Continued)
Introduction
Aim of talk
Topic
Introduction of theoptimizers
Introduction of theoptimizers(Continued)
Computational testsetup
Improvements in thepresolve
Improvements in thesimplex optimizers
Improvements in theinterior pointoptimizer
Conclusions
4 / 26
■ Third party interfaces :
◆ AIMMS◆ COIN-OR◆ Frontline Systems◆ GAMS◆ Microsoft Solver Foundation◆ OptimJ
■ Version 1 released 1999.■ Version 6 released October 2009 (Next week).
For further information see www.MOSEK.com.Prices public available directly on our website !
Aim of talk
Introduction
Aim of talk
Topic
Introduction of theoptimizers
Introduction of theoptimizers(Continued)
Computational testsetup
Improvements in thepresolve
Improvements in thesimplex optimizers
Improvements in theinterior pointoptimizer
Conclusions
5 / 26
■ Highlight recent key improvements in the linearoptimizers.
■ Show computational results on a larger set of benchmarkproblems.
Topic
Introduction
Aim of talk
Topic
Introduction of theoptimizers
Introduction of theoptimizers(Continued)
Computational testsetup
Improvements in thepresolve
Improvements in thesimplex optimizers
Improvements in theinterior pointoptimizer
Conclusions
6 / 26
■ The problem(P ) min cT x
st Ax = b,
x ≥ 0.
■ The linear optimizers.
◆ Interior-point based.◆ Primal and dual simplex based.◆ Primal network flow simplex.
■ What are the recent improvements?
Introduction of the optimizers
Introduction
Aim of talk
Topic
Introduction of theoptimizers
Introduction of theoptimizers(Continued)
Computational testsetup
Improvements in thepresolve
Improvements in thesimplex optimizers
Improvements in theinterior pointoptimizer
Conclusions
7 / 26
■ Interior point.
◆ Based on the homogeneous and self-dual model.◆ Basis identification.◆ Highly tuned implementation.◆ Can exploit multiple processors.
■ Primal and Dual simplex.
◆ Exploits hyper sparsity (LU, solves and loops).◆ Multiple pricing options : Dantzig, partial, devex,
exact steepest edge, approximate steepest edge andhybrid pricing.
◆ Crashes an advanced initial basis.◆ Aggressive anti-degeneracy schemes.
Introduction of the optimizers (Continued)
Introduction
Aim of talk
Topic
Introduction of theoptimizers
Introduction of theoptimizers(Continued)
Computational testsetup
Improvements in thepresolve
Improvements in thesimplex optimizers
Improvements in theinterior pointoptimizer
Conclusions
8 / 26
■ Network optimizer.
◆ Specialized primal simplex implementation.◆ Exploits the tree structure of the basis and other
tricks.◆ Can extract embedded network structure.
The network optimizer has not been changed since mosek 5and will be left out of benchmark.
Computational test setup
9 / 26
Computational test setup
Introduction
Computational testsetup
Computational testsetup
How do we measureimprovements ?
Improvements in thepresolve
Improvements in thesimplex optimizers
Improvements in theinterior pointoptimizer
Conclusions
10 / 26
All computational results are run on the same machine :
■ Intel Xeon 2.27 GHz 16 Cores.■ 16 GB RAM.■ Running Linux.
Selected about 1000 problems from our clients and public
ressources
How do we measure improvement ?
Introduction
Computational testsetup
Computational testsetup
How do we measureimprovements ?
Improvements in thepresolve
Improvements in thesimplex optimizers
Improvements in theinterior pointoptimizer
Conclusions
11 / 26
■ Maximum solution time allowed 10000 sec, timeouts areexcluded.
■ Group problems by solution times :
◆ Small problems => best solution time ≤ 6 sek.◆ Medium problems => 6 sek < best solution time ≤
60 sek.◆ Large problems => best solution time > 60
■ Sum of solution time Bad benchmarking criteria!.■ Geometric mean of ratios of optimizer time i.e
MOSEK6/MOSEK5.■ Number of wins.
Improvements in the presolve
12 / 26
Presolve overview
Introduction
Computational testsetup
Improvements in thepresolve
Presolve overview
Presolving hotstarts
Presolving hotstarts
Numerical examples
Improvements in thesimplex optimizers
Improvements in theinterior pointoptimizer
Conclusions
13 / 26
■ MOSEK has had an extensive presolve since version 1.See [AND:95].
■ It was only employed in cold start cases.■ Since MOSEK 5 we can also presolve hotstart.■ Presolving hotstarts has been improved in MOSEK 6.
Presolving hotstarts
Introduction
Computational testsetup
Improvements in thepresolve
Presolve overview
Presolving hotstarts
Presolving hotstarts
Numerical examples
Improvements in thesimplex optimizers
Improvements in theinterior pointoptimizer
Conclusions
14 / 26
■ Assume we have a ”good” (optimal?) primal x∗ or dualy∗ solution to P .But now the problem has been modified by the user into :
(P ) min cT x
st Ax = b,
x ≥ 0.
Presolving hotstarts
Introduction
Computational testsetup
Improvements in thepresolve
Presolve overview
Presolving hotstarts
Presolving hotstarts
Numerical examples
Improvements in thesimplex optimizers
Improvements in theinterior pointoptimizer
Conclusions
14 / 26
■ Assume we have a ”good” (optimal?) primal x∗ or dualy∗ solution to P .But now the problem has been modified by the user into :
(P ) min cT x
st Ax = b,
x ≥ 0.
■ Challenges :
◆ The problem may have been modified in any way.◆ Some presolve techniques destroy primal and dual
feasibility.
■ Ex : Assume x∗ is primal feasible and x∗
k = uk,but dual presolve says x∗
k = lk ?
Presolving hotstarts (Continued)
Introduction
Computational testsetup
Improvements in thepresolve
Presolve overview
Presolving hotstarts
Presolving hotstarts
Numerical examples
Improvements in thesimplex optimizers
Improvements in theinterior pointoptimizer
Conclusions
15 / 26
■ The MOSEK presolve strategy:
◆ Aimed at ”hard” hotstarts where presolve reallymakes a difference.
Presolving hotstarts (Continued)
Introduction
Computational testsetup
Improvements in thepresolve
Presolve overview
Presolving hotstarts
Presolving hotstarts
Numerical examples
Improvements in thesimplex optimizers
Improvements in theinterior pointoptimizer
Conclusions
15 / 26
■ The MOSEK presolve strategy:
◆ Aimed at ”hard” hotstarts where presolve reallymakes a difference.
◆ Preserve either primal (primal simplex) or dualfeasibility (dual simplex).
Presolving hotstarts (Continued)
Introduction
Computational testsetup
Improvements in thepresolve
Presolve overview
Presolving hotstarts
Presolving hotstarts
Numerical examples
Improvements in thesimplex optimizers
Improvements in theinterior pointoptimizer
Conclusions
15 / 26
■ The MOSEK presolve strategy:
◆ Aimed at ”hard” hotstarts where presolve reallymakes a difference.
◆ Preserve either primal (primal simplex) or dualfeasibility (dual simplex).
◆ Also try to keep the basis intact.
Presolving hotstarts (Continued)
Introduction
Computational testsetup
Improvements in thepresolve
Presolve overview
Presolving hotstarts
Presolving hotstarts
Numerical examples
Improvements in thesimplex optimizers
Improvements in theinterior pointoptimizer
Conclusions
15 / 26
■ The MOSEK presolve strategy:
◆ Aimed at ”hard” hotstarts where presolve reallymakes a difference.
◆ Preserve either primal (primal simplex) or dualfeasibility (dual simplex).
◆ Also try to keep the basis intact.◆ If your problem takes few iterations in reoptimizing,
then you should switch off presolve.
Numerical examples
Introduction
Computational testsetup
Improvements in thepresolve
Presolve overview
Presolving hotstarts
Presolving hotstarts
Numerical examples
Improvements in thesimplex optimizers
Improvements in theinterior pointoptimizer
Conclusions
16 / 26
Selected 82 hotstart instances
Warning biased problem selection ahead!
small medium6-nopre 6 5 6-nopre 6 5
Num. 74 74 74 8 8 8Firsts 9 51 18 1 6 1Total time 1058.49 170.90 323.95 602.72 98.20 618.54G. avg. 5.79 1.77 2.39 41.68 11.21 44.37G. avg. r 2.42 0.74 0.94 0.25
Table 1: Presolve hotstart effect on client problems (Dual sim-plex).
Improvements in the simplex optimizers
17 / 26
Key areas simplex optimizers
Introduction
Computational testsetup
Improvements in thepresolve
Improvements in thesimplex optimizers
Computationalresults-primalsimplex
Computationalresults-dual simplex
Improvements in theinterior pointoptimizer
Conclusions
18 / 26
■ Speed and stability has in general been greatly improved.
Key areas simplex optimizers
Introduction
Computational testsetup
Improvements in thepresolve
Improvements in thesimplex optimizers
Computationalresults-primalsimplex
Computationalresults-dual simplex
Improvements in theinterior pointoptimizer
Conclusions
18 / 26
■ Speed and stability has in general been greatly improved.■ Larger parts of the code has been rewritten.
Key areas simplex optimizers
Introduction
Computational testsetup
Improvements in thepresolve
Improvements in thesimplex optimizers
Computationalresults-primalsimplex
Computationalresults-dual simplex
Improvements in theinterior pointoptimizer
Conclusions
18 / 26
■ Speed and stability has in general been greatly improved.■ Larger parts of the code has been rewritten.■ Parameter tuning (i.e running many test with different
internal settings).
Key areas simplex optimizers
Introduction
Computational testsetup
Improvements in thepresolve
Improvements in thesimplex optimizers
Computationalresults-primalsimplex
Computationalresults-dual simplex
Improvements in theinterior pointoptimizer
Conclusions
18 / 26
■ Speed and stability has in general been greatly improved.■ Larger parts of the code has been rewritten.■ Parameter tuning (i.e running many test with different
internal settings).■ Hybrid pricing in primal simplex has been completely
rewritten.
◆ Improved the automatic switch between partial andapproximate steepest-edge pricing in primal simplex.
◆ Restricted pricing (i.e optimizing over a subset ofcolumns) is now used more intensively.
Key areas simplex optimizers
Introduction
Computational testsetup
Improvements in thepresolve
Improvements in thesimplex optimizers
Computationalresults-primalsimplex
Computationalresults-dual simplex
Improvements in theinterior pointoptimizer
Conclusions
18 / 26
■ Speed and stability has in general been greatly improved.■ Larger parts of the code has been rewritten.■ Parameter tuning (i.e running many test with different
internal settings).■ Hybrid pricing in primal simplex has been completely
rewritten.
◆ Improved the automatic switch between partial andapproximate steepest-edge pricing in primal simplex.
◆ Restricted pricing (i.e optimizing over a subset ofcolumns) is now used more intensively.
■ Better crash module in dual simplex.
Key areas simplex optimizers
Introduction
Computational testsetup
Improvements in thepresolve
Improvements in thesimplex optimizers
Computationalresults-primalsimplex
Computationalresults-dual simplex
Improvements in theinterior pointoptimizer
Conclusions
18 / 26
■ Speed and stability has in general been greatly improved.■ Larger parts of the code has been rewritten.■ Parameter tuning (i.e running many test with different
internal settings).■ Hybrid pricing in primal simplex has been completely
rewritten.
◆ Improved the automatic switch between partial andapproximate steepest-edge pricing in primal simplex.
◆ Restricted pricing (i.e optimizing over a subset ofcolumns) is now used more intensively.
■ Better crash module in dual simplex.■ The automatic refactor frequency in the LU module has
been improved.
Computational results-primal simplex
Introduction
Computational testsetup
Improvements in thepresolve
Improvements in thesimplex optimizers
Computationalresults-primalsimplex
Computationalresults-dual simplex
Improvements in theinterior pointoptimizer
Conclusions
19 / 26
■ Small problems are misleading, since time measurementin MOSEK 5 and 6 are different !.
■ At least 25% speed up on average.
Computational results-dual simplex
Introduction
Computational testsetup
Improvements in thepresolve
Improvements in thesimplex optimizers
Computationalresults-primalsimplex
Computationalresults-dual simplex
Improvements in theinterior pointoptimizer
Conclusions
20 / 26
■ Small problems are misleading, since time measurementin MOSEK 5 and 6 are different !.
■ At least 25% speed up on average.
Improvements in the interior pointoptimizer
21 / 26
Key areas interior point optimizers
Introduction
Computational testsetup
Improvements in thepresolve
Improvements in thesimplex optimizers
Improvements in theinterior pointoptimizer
Computationalresults-interior point
Conclusions
22 / 26
■ Improved speed in general.■ Improved numerical stability on hard problems.
Computational results-interior point
Introduction
Computational testsetup
Improvements in thepresolve
Improvements in thesimplex optimizers
Improvements in theinterior pointoptimizer
Computationalresults-interior point
Conclusions
23 / 26
■ The testset is too easy for interior point !■ Interior point was 3-4 times faster than dual simplex !■ About 5-10% speed up (more for hard problems).
Conclusions
24 / 26
Conclusions
Introduction
Computational testsetup
Improvements in thepresolve
Improvements in thesimplex optimizers
Improvements in theinterior pointoptimizer
Conclusions
Conclusions
References
25 / 26
■ Much improved presolve performance for hotstarts.■ All optimizers more numerical stable.■ Primal simplex optimizer 25% faster on average.■ Dual simplex optimizer 25% faster on average.■ Interior point optimizer 5-10% faster on average.
References
Introduction
Computational testsetup
Improvements in thepresolve
Improvements in thesimplex optimizers
Improvements in theinterior pointoptimizer
Conclusions
Conclusions
References
26 / 26
[AND:95] E. Andersen and K. Andersen, ”Presolve in linear
programming”, Mathematical Programming, Volume 71 ,
1995