2009 : solving linear optimization problems with mosek
DESCRIPTION
My slides from INFORMS San Diego 2009. Just a short benchmark talk about recent improvements in mosek 6.TRANSCRIPT
![Page 1: 2009 : Solving linear optimization problems with MOSEK](https://reader035.vdocuments.mx/reader035/viewer/2022081507/554a436cb4c9055a408b509d/html5/thumbnails/1.jpg)
http://www.mosek.com
Solving Linear OptimizationProblems with MOSEK.
Bo Jensen MOSEK ApS,Fruebjergvej 3, Box 16, 2100 Copenhagen,
Denmark.Email: [email protected]
INFORMS Annual San Diego Tuesday Oct 13, 13:30 - 15:00, 2009
![Page 2: 2009 : Solving linear optimization problems with MOSEK](https://reader035.vdocuments.mx/reader035/viewer/2022081507/554a436cb4c9055a408b509d/html5/thumbnails/2.jpg)
Introduction
2 / 26
![Page 3: 2009 : Solving linear optimization problems with MOSEK](https://reader035.vdocuments.mx/reader035/viewer/2022081507/554a436cb4c9055a408b509d/html5/thumbnails/3.jpg)
What is MOSEK
Introduction
Aim of talk
Topic
Introduction of theoptimizers
Introduction of theoptimizers(Continued)
Computational testsetup
Improvements in thepresolve
Improvements in thesimplex optimizers
Improvements in theinterior pointoptimizer
Conclusions
3 / 26
■ A software package for solving large-scale optimizationproblems.
■ Solves linear, conic, and nonlinear convex problems.Mixed integer optimizer.
■ Used to solve problems with up to millions of constraintsand variables.
■ Stand-alone as well as embedded :
◆ C◆ Java◆ MATLAB◆ Python◆ .NET
![Page 4: 2009 : Solving linear optimization problems with MOSEK](https://reader035.vdocuments.mx/reader035/viewer/2022081507/554a436cb4c9055a408b509d/html5/thumbnails/4.jpg)
What is MOSEK (Continued)
Introduction
Aim of talk
Topic
Introduction of theoptimizers
Introduction of theoptimizers(Continued)
Computational testsetup
Improvements in thepresolve
Improvements in thesimplex optimizers
Improvements in theinterior pointoptimizer
Conclusions
4 / 26
■ Third party interfaces :
◆ AIMMS◆ COIN-OR◆ Frontline Systems◆ GAMS◆ Microsoft Solver Foundation◆ OptimJ
■ Version 1 released 1999.■ Version 6 released October 2009 (Next week).
For further information see www.MOSEK.com.Prices public available directly on our website !
![Page 5: 2009 : Solving linear optimization problems with MOSEK](https://reader035.vdocuments.mx/reader035/viewer/2022081507/554a436cb4c9055a408b509d/html5/thumbnails/5.jpg)
Aim of talk
Introduction
Aim of talk
Topic
Introduction of theoptimizers
Introduction of theoptimizers(Continued)
Computational testsetup
Improvements in thepresolve
Improvements in thesimplex optimizers
Improvements in theinterior pointoptimizer
Conclusions
5 / 26
■ Highlight recent key improvements in the linearoptimizers.
■ Show computational results on a larger set of benchmarkproblems.
![Page 6: 2009 : Solving linear optimization problems with MOSEK](https://reader035.vdocuments.mx/reader035/viewer/2022081507/554a436cb4c9055a408b509d/html5/thumbnails/6.jpg)
Topic
Introduction
Aim of talk
Topic
Introduction of theoptimizers
Introduction of theoptimizers(Continued)
Computational testsetup
Improvements in thepresolve
Improvements in thesimplex optimizers
Improvements in theinterior pointoptimizer
Conclusions
6 / 26
■ The problem(P ) min cT x
st Ax = b,
x ≥ 0.
■ The linear optimizers.
◆ Interior-point based.◆ Primal and dual simplex based.◆ Primal network flow simplex.
■ What are the recent improvements?
![Page 7: 2009 : Solving linear optimization problems with MOSEK](https://reader035.vdocuments.mx/reader035/viewer/2022081507/554a436cb4c9055a408b509d/html5/thumbnails/7.jpg)
Introduction of the optimizers
Introduction
Aim of talk
Topic
Introduction of theoptimizers
Introduction of theoptimizers(Continued)
Computational testsetup
Improvements in thepresolve
Improvements in thesimplex optimizers
Improvements in theinterior pointoptimizer
Conclusions
7 / 26
■ Interior point.
◆ Based on the homogeneous and self-dual model.◆ Basis identification.◆ Highly tuned implementation.◆ Can exploit multiple processors.
■ Primal and Dual simplex.
◆ Exploits hyper sparsity (LU, solves and loops).◆ Multiple pricing options : Dantzig, partial, devex,
exact steepest edge, approximate steepest edge andhybrid pricing.
◆ Crashes an advanced initial basis.◆ Aggressive anti-degeneracy schemes.
![Page 8: 2009 : Solving linear optimization problems with MOSEK](https://reader035.vdocuments.mx/reader035/viewer/2022081507/554a436cb4c9055a408b509d/html5/thumbnails/8.jpg)
Introduction of the optimizers (Continued)
Introduction
Aim of talk
Topic
Introduction of theoptimizers
Introduction of theoptimizers(Continued)
Computational testsetup
Improvements in thepresolve
Improvements in thesimplex optimizers
Improvements in theinterior pointoptimizer
Conclusions
8 / 26
■ Network optimizer.
◆ Specialized primal simplex implementation.◆ Exploits the tree structure of the basis and other
tricks.◆ Can extract embedded network structure.
The network optimizer has not been changed since mosek 5and will be left out of benchmark.
![Page 9: 2009 : Solving linear optimization problems with MOSEK](https://reader035.vdocuments.mx/reader035/viewer/2022081507/554a436cb4c9055a408b509d/html5/thumbnails/9.jpg)
Computational test setup
9 / 26
![Page 10: 2009 : Solving linear optimization problems with MOSEK](https://reader035.vdocuments.mx/reader035/viewer/2022081507/554a436cb4c9055a408b509d/html5/thumbnails/10.jpg)
Computational test setup
Introduction
Computational testsetup
Computational testsetup
How do we measureimprovements ?
Improvements in thepresolve
Improvements in thesimplex optimizers
Improvements in theinterior pointoptimizer
Conclusions
10 / 26
All computational results are run on the same machine :
■ Intel Xeon 2.27 GHz 16 Cores.■ 16 GB RAM.■ Running Linux.
Selected about 1000 problems from our clients and public
ressources
![Page 11: 2009 : Solving linear optimization problems with MOSEK](https://reader035.vdocuments.mx/reader035/viewer/2022081507/554a436cb4c9055a408b509d/html5/thumbnails/11.jpg)
How do we measure improvement ?
Introduction
Computational testsetup
Computational testsetup
How do we measureimprovements ?
Improvements in thepresolve
Improvements in thesimplex optimizers
Improvements in theinterior pointoptimizer
Conclusions
11 / 26
■ Maximum solution time allowed 10000 sec, timeouts areexcluded.
■ Group problems by solution times :
◆ Small problems => best solution time ≤ 6 sek.◆ Medium problems => 6 sek < best solution time ≤
60 sek.◆ Large problems => best solution time > 60
■ Sum of solution time Bad benchmarking criteria!.■ Geometric mean of ratios of optimizer time i.e
MOSEK6/MOSEK5.■ Number of wins.
![Page 12: 2009 : Solving linear optimization problems with MOSEK](https://reader035.vdocuments.mx/reader035/viewer/2022081507/554a436cb4c9055a408b509d/html5/thumbnails/12.jpg)
Improvements in the presolve
12 / 26
![Page 13: 2009 : Solving linear optimization problems with MOSEK](https://reader035.vdocuments.mx/reader035/viewer/2022081507/554a436cb4c9055a408b509d/html5/thumbnails/13.jpg)
Presolve overview
Introduction
Computational testsetup
Improvements in thepresolve
Presolve overview
Presolving hotstarts
Presolving hotstarts
Numerical examples
Improvements in thesimplex optimizers
Improvements in theinterior pointoptimizer
Conclusions
13 / 26
■ MOSEK has had an extensive presolve since version 1.See [AND:95].
■ It was only employed in cold start cases.■ Since MOSEK 5 we can also presolve hotstart.■ Presolving hotstarts has been improved in MOSEK 6.
![Page 14: 2009 : Solving linear optimization problems with MOSEK](https://reader035.vdocuments.mx/reader035/viewer/2022081507/554a436cb4c9055a408b509d/html5/thumbnails/14.jpg)
Presolving hotstarts
Introduction
Computational testsetup
Improvements in thepresolve
Presolve overview
Presolving hotstarts
Presolving hotstarts
Numerical examples
Improvements in thesimplex optimizers
Improvements in theinterior pointoptimizer
Conclusions
14 / 26
■ Assume we have a ”good” (optimal?) primal x∗ or dualy∗ solution to P .But now the problem has been modified by the user into :
(P ) min cT x
st Ax = b,
x ≥ 0.
![Page 15: 2009 : Solving linear optimization problems with MOSEK](https://reader035.vdocuments.mx/reader035/viewer/2022081507/554a436cb4c9055a408b509d/html5/thumbnails/15.jpg)
Presolving hotstarts
Introduction
Computational testsetup
Improvements in thepresolve
Presolve overview
Presolving hotstarts
Presolving hotstarts
Numerical examples
Improvements in thesimplex optimizers
Improvements in theinterior pointoptimizer
Conclusions
14 / 26
■ Assume we have a ”good” (optimal?) primal x∗ or dualy∗ solution to P .But now the problem has been modified by the user into :
(P ) min cT x
st Ax = b,
x ≥ 0.
■ Challenges :
◆ The problem may have been modified in any way.◆ Some presolve techniques destroy primal and dual
feasibility.
■ Ex : Assume x∗ is primal feasible and x∗
k = uk,but dual presolve says x∗
k = lk ?
![Page 16: 2009 : Solving linear optimization problems with MOSEK](https://reader035.vdocuments.mx/reader035/viewer/2022081507/554a436cb4c9055a408b509d/html5/thumbnails/16.jpg)
Presolving hotstarts (Continued)
Introduction
Computational testsetup
Improvements in thepresolve
Presolve overview
Presolving hotstarts
Presolving hotstarts
Numerical examples
Improvements in thesimplex optimizers
Improvements in theinterior pointoptimizer
Conclusions
15 / 26
■ The MOSEK presolve strategy:
◆ Aimed at ”hard” hotstarts where presolve reallymakes a difference.
![Page 17: 2009 : Solving linear optimization problems with MOSEK](https://reader035.vdocuments.mx/reader035/viewer/2022081507/554a436cb4c9055a408b509d/html5/thumbnails/17.jpg)
Presolving hotstarts (Continued)
Introduction
Computational testsetup
Improvements in thepresolve
Presolve overview
Presolving hotstarts
Presolving hotstarts
Numerical examples
Improvements in thesimplex optimizers
Improvements in theinterior pointoptimizer
Conclusions
15 / 26
■ The MOSEK presolve strategy:
◆ Aimed at ”hard” hotstarts where presolve reallymakes a difference.
◆ Preserve either primal (primal simplex) or dualfeasibility (dual simplex).
![Page 18: 2009 : Solving linear optimization problems with MOSEK](https://reader035.vdocuments.mx/reader035/viewer/2022081507/554a436cb4c9055a408b509d/html5/thumbnails/18.jpg)
Presolving hotstarts (Continued)
Introduction
Computational testsetup
Improvements in thepresolve
Presolve overview
Presolving hotstarts
Presolving hotstarts
Numerical examples
Improvements in thesimplex optimizers
Improvements in theinterior pointoptimizer
Conclusions
15 / 26
■ The MOSEK presolve strategy:
◆ Aimed at ”hard” hotstarts where presolve reallymakes a difference.
◆ Preserve either primal (primal simplex) or dualfeasibility (dual simplex).
◆ Also try to keep the basis intact.
![Page 19: 2009 : Solving linear optimization problems with MOSEK](https://reader035.vdocuments.mx/reader035/viewer/2022081507/554a436cb4c9055a408b509d/html5/thumbnails/19.jpg)
Presolving hotstarts (Continued)
Introduction
Computational testsetup
Improvements in thepresolve
Presolve overview
Presolving hotstarts
Presolving hotstarts
Numerical examples
Improvements in thesimplex optimizers
Improvements in theinterior pointoptimizer
Conclusions
15 / 26
■ The MOSEK presolve strategy:
◆ Aimed at ”hard” hotstarts where presolve reallymakes a difference.
◆ Preserve either primal (primal simplex) or dualfeasibility (dual simplex).
◆ Also try to keep the basis intact.◆ If your problem takes few iterations in reoptimizing,
then you should switch off presolve.
![Page 20: 2009 : Solving linear optimization problems with MOSEK](https://reader035.vdocuments.mx/reader035/viewer/2022081507/554a436cb4c9055a408b509d/html5/thumbnails/20.jpg)
Numerical examples
Introduction
Computational testsetup
Improvements in thepresolve
Presolve overview
Presolving hotstarts
Presolving hotstarts
Numerical examples
Improvements in thesimplex optimizers
Improvements in theinterior pointoptimizer
Conclusions
16 / 26
Selected 82 hotstart instances
Warning biased problem selection ahead!
small medium6-nopre 6 5 6-nopre 6 5
Num. 74 74 74 8 8 8Firsts 9 51 18 1 6 1Total time 1058.49 170.90 323.95 602.72 98.20 618.54G. avg. 5.79 1.77 2.39 41.68 11.21 44.37G. avg. r 2.42 0.74 0.94 0.25
Table 1: Presolve hotstart effect on client problems (Dual sim-plex).
![Page 21: 2009 : Solving linear optimization problems with MOSEK](https://reader035.vdocuments.mx/reader035/viewer/2022081507/554a436cb4c9055a408b509d/html5/thumbnails/21.jpg)
Improvements in the simplex optimizers
17 / 26
![Page 22: 2009 : Solving linear optimization problems with MOSEK](https://reader035.vdocuments.mx/reader035/viewer/2022081507/554a436cb4c9055a408b509d/html5/thumbnails/22.jpg)
Key areas simplex optimizers
Introduction
Computational testsetup
Improvements in thepresolve
Improvements in thesimplex optimizers
Computationalresults-primalsimplex
Computationalresults-dual simplex
Improvements in theinterior pointoptimizer
Conclusions
18 / 26
■ Speed and stability has in general been greatly improved.
![Page 23: 2009 : Solving linear optimization problems with MOSEK](https://reader035.vdocuments.mx/reader035/viewer/2022081507/554a436cb4c9055a408b509d/html5/thumbnails/23.jpg)
Key areas simplex optimizers
Introduction
Computational testsetup
Improvements in thepresolve
Improvements in thesimplex optimizers
Computationalresults-primalsimplex
Computationalresults-dual simplex
Improvements in theinterior pointoptimizer
Conclusions
18 / 26
■ Speed and stability has in general been greatly improved.■ Larger parts of the code has been rewritten.
![Page 24: 2009 : Solving linear optimization problems with MOSEK](https://reader035.vdocuments.mx/reader035/viewer/2022081507/554a436cb4c9055a408b509d/html5/thumbnails/24.jpg)
Key areas simplex optimizers
Introduction
Computational testsetup
Improvements in thepresolve
Improvements in thesimplex optimizers
Computationalresults-primalsimplex
Computationalresults-dual simplex
Improvements in theinterior pointoptimizer
Conclusions
18 / 26
■ Speed and stability has in general been greatly improved.■ Larger parts of the code has been rewritten.■ Parameter tuning (i.e running many test with different
internal settings).
![Page 25: 2009 : Solving linear optimization problems with MOSEK](https://reader035.vdocuments.mx/reader035/viewer/2022081507/554a436cb4c9055a408b509d/html5/thumbnails/25.jpg)
Key areas simplex optimizers
Introduction
Computational testsetup
Improvements in thepresolve
Improvements in thesimplex optimizers
Computationalresults-primalsimplex
Computationalresults-dual simplex
Improvements in theinterior pointoptimizer
Conclusions
18 / 26
■ Speed and stability has in general been greatly improved.■ Larger parts of the code has been rewritten.■ Parameter tuning (i.e running many test with different
internal settings).■ Hybrid pricing in primal simplex has been completely
rewritten.
◆ Improved the automatic switch between partial andapproximate steepest-edge pricing in primal simplex.
◆ Restricted pricing (i.e optimizing over a subset ofcolumns) is now used more intensively.
![Page 26: 2009 : Solving linear optimization problems with MOSEK](https://reader035.vdocuments.mx/reader035/viewer/2022081507/554a436cb4c9055a408b509d/html5/thumbnails/26.jpg)
Key areas simplex optimizers
Introduction
Computational testsetup
Improvements in thepresolve
Improvements in thesimplex optimizers
Computationalresults-primalsimplex
Computationalresults-dual simplex
Improvements in theinterior pointoptimizer
Conclusions
18 / 26
■ Speed and stability has in general been greatly improved.■ Larger parts of the code has been rewritten.■ Parameter tuning (i.e running many test with different
internal settings).■ Hybrid pricing in primal simplex has been completely
rewritten.
◆ Improved the automatic switch between partial andapproximate steepest-edge pricing in primal simplex.
◆ Restricted pricing (i.e optimizing over a subset ofcolumns) is now used more intensively.
■ Better crash module in dual simplex.
![Page 27: 2009 : Solving linear optimization problems with MOSEK](https://reader035.vdocuments.mx/reader035/viewer/2022081507/554a436cb4c9055a408b509d/html5/thumbnails/27.jpg)
Key areas simplex optimizers
Introduction
Computational testsetup
Improvements in thepresolve
Improvements in thesimplex optimizers
Computationalresults-primalsimplex
Computationalresults-dual simplex
Improvements in theinterior pointoptimizer
Conclusions
18 / 26
■ Speed and stability has in general been greatly improved.■ Larger parts of the code has been rewritten.■ Parameter tuning (i.e running many test with different
internal settings).■ Hybrid pricing in primal simplex has been completely
rewritten.
◆ Improved the automatic switch between partial andapproximate steepest-edge pricing in primal simplex.
◆ Restricted pricing (i.e optimizing over a subset ofcolumns) is now used more intensively.
■ Better crash module in dual simplex.■ The automatic refactor frequency in the LU module has
been improved.
![Page 28: 2009 : Solving linear optimization problems with MOSEK](https://reader035.vdocuments.mx/reader035/viewer/2022081507/554a436cb4c9055a408b509d/html5/thumbnails/28.jpg)
Computational results-primal simplex
Introduction
Computational testsetup
Improvements in thepresolve
Improvements in thesimplex optimizers
Computationalresults-primalsimplex
Computationalresults-dual simplex
Improvements in theinterior pointoptimizer
Conclusions
19 / 26
■ Small problems are misleading, since time measurementin MOSEK 5 and 6 are different !.
■ At least 25% speed up on average.
![Page 29: 2009 : Solving linear optimization problems with MOSEK](https://reader035.vdocuments.mx/reader035/viewer/2022081507/554a436cb4c9055a408b509d/html5/thumbnails/29.jpg)
Computational results-dual simplex
Introduction
Computational testsetup
Improvements in thepresolve
Improvements in thesimplex optimizers
Computationalresults-primalsimplex
Computationalresults-dual simplex
Improvements in theinterior pointoptimizer
Conclusions
20 / 26
■ Small problems are misleading, since time measurementin MOSEK 5 and 6 are different !.
■ At least 25% speed up on average.
![Page 30: 2009 : Solving linear optimization problems with MOSEK](https://reader035.vdocuments.mx/reader035/viewer/2022081507/554a436cb4c9055a408b509d/html5/thumbnails/30.jpg)
Improvements in the interior pointoptimizer
21 / 26
![Page 31: 2009 : Solving linear optimization problems with MOSEK](https://reader035.vdocuments.mx/reader035/viewer/2022081507/554a436cb4c9055a408b509d/html5/thumbnails/31.jpg)
Key areas interior point optimizers
Introduction
Computational testsetup
Improvements in thepresolve
Improvements in thesimplex optimizers
Improvements in theinterior pointoptimizer
Computationalresults-interior point
Conclusions
22 / 26
■ Improved speed in general.■ Improved numerical stability on hard problems.
![Page 32: 2009 : Solving linear optimization problems with MOSEK](https://reader035.vdocuments.mx/reader035/viewer/2022081507/554a436cb4c9055a408b509d/html5/thumbnails/32.jpg)
Computational results-interior point
Introduction
Computational testsetup
Improvements in thepresolve
Improvements in thesimplex optimizers
Improvements in theinterior pointoptimizer
Computationalresults-interior point
Conclusions
23 / 26
■ The testset is too easy for interior point !■ Interior point was 3-4 times faster than dual simplex !■ About 5-10% speed up (more for hard problems).
![Page 33: 2009 : Solving linear optimization problems with MOSEK](https://reader035.vdocuments.mx/reader035/viewer/2022081507/554a436cb4c9055a408b509d/html5/thumbnails/33.jpg)
Conclusions
24 / 26
![Page 34: 2009 : Solving linear optimization problems with MOSEK](https://reader035.vdocuments.mx/reader035/viewer/2022081507/554a436cb4c9055a408b509d/html5/thumbnails/34.jpg)
Conclusions
Introduction
Computational testsetup
Improvements in thepresolve
Improvements in thesimplex optimizers
Improvements in theinterior pointoptimizer
Conclusions
Conclusions
References
25 / 26
■ Much improved presolve performance for hotstarts.■ All optimizers more numerical stable.■ Primal simplex optimizer 25% faster on average.■ Dual simplex optimizer 25% faster on average.■ Interior point optimizer 5-10% faster on average.
![Page 35: 2009 : Solving linear optimization problems with MOSEK](https://reader035.vdocuments.mx/reader035/viewer/2022081507/554a436cb4c9055a408b509d/html5/thumbnails/35.jpg)
References
Introduction
Computational testsetup
Improvements in thepresolve
Improvements in thesimplex optimizers
Improvements in theinterior pointoptimizer
Conclusions
Conclusions
References
26 / 26
[AND:95] E. Andersen and K. Andersen, ”Presolve in linear
programming”, Mathematical Programming, Volume 71 ,
1995