TP5027 Transport & Freight Terminal M tManagement

Optimization Solvers

Dr. MENG QiangOffice: E1-07-10Tel: 6516 5494

E-mail: cvemq@nus.edu.sg

1

Learning Objectives & ReferenceLearning Objectives & Reference

Master two optimization software packages: Microsoft Excel Solver

( h i l i ) AMPL (A Mathematical Programming Language)

References Fourer, R. D., Gay, D. and Kernighan (2003) AMPL, A Modeling

Language for Mathematical Programming, 2nd edition. Brooks/Cole-Thomson.

Handout for AMPL

Handout for Excel Solver

2

ContentsContents

Introduction to algorithms solving LP, NLP and IPoduc o o go s so v g , N d Microsoft Excel Solver AMPL AMPL

3

Linear Programming Models AlgorithmsLinear Programming Models – Algorithms

Simplex Method (G. B. Dantzig, 1947)

x25 2x1+x2=5

– An iterative method– From one vertex to another one

3

4

CD (1,3)

– Terminated in the limited iterations

Example

1 2max ( ) 3 2f x x x= +

4x x+ ≤s.t.

1

2 -x1+4x2=2

B

C

1 2 4x x+ ≤

1 22 5x x+ ≤

1 24 2x x− + ≥ -2

x1

2 3 4

1

1 5-1

x1+x2=4A

1 24 2x x+ ≥1 2, 0x x ≥

2 2 3 41 513x1+2x2=2 3x1+2x2=4 3x1+2x2=9

4

Linear Programming Models Algorithms (cont’d)Linear Programming Models – Algorithms (cont d)

Principle behind the simplex method

– Check that if a vertex is an optimal solution

– If it is not, find another vertex which is adjacent to the current vertex

x2feasible solution set - a polytope x2

feasible solution set - a polytope

I t i i t l ith (K k 1984)

x1 x1

Interior point algorithm (Karmarkar, 1984)N. Karmarkar. A new polynomial-time algorithm for linear programming. In Proceedings of the Sixteenth Annual ACM Symposium on Theory of

5

Proceedings of the Sixteenth Annual ACM Symposium on Theory of Computing, pp. 302-311, Washington, D.C., 1984.

N li P i M d l O ti l S l tiNonlinear Programming Models – Optimal Solutions

Global minimum of a NLP (minimization problems)x* is a feasible solution in the set Ω and we have f(x*)≤f(x) for any x in the set Ω

Local minimum of a NLP (minimization problems)x* is a feasible solution in the set Ω and we have f(x*)≤ f(x) for any x in

i hb h d f *neighborhood of x*

Optimality conditions Karush-Kuhn-Tucker (KKT) conditionsKuhn, H. W. and Tucker, A. W. (1951) Nonlinear Programming. J. Neyman (ed.) Proceedings of the Second Berkeley Symposium on Mathematical Statistics andProceedings of the Second Berkeley Symposium on Mathematical Statistics and Probabilities. University of California, Berkeley, pp. 481-492. Karush, W. (1939) Minima of Functions of Several Variables with Inequalities as Side Conditions MS Th i D t t f M th ti U i it f Chi

6

Side Conditions, MSc Thesis, Department of Mathematics, University of Chicago, Chicago.

Nonlinear Programming Models AlgorithmsNonlinear Programming Models – Algorithms

Gradient based algorithmsx2 Ω S f f ibl l i= +1 2min ( ) 3 2f x x x

1 2 4x x+ ≤s.t.

x2

3

4Ω: Set of feasible solutions

Direction: ( ) ( )(1) 3 2f x∇ =1 2

1 22 5x x+ ≤

1 24 2x x− + ≥2

3 Direction: ( ) ( )3,2f x∇

x(1)

1 2, 0x x ≥

x12 3 4

1

1d1Contour of the objective function

3x1+2x2=23x1+2x2=5 3x1+2x2=9

2 3 41 Idea behind an algorithm for a NLP

– Given a feasible solution x(1)= (0.5, 1.75), gradient of the objective function: ( ) ( )(1) 3 2f∇gradient of the objective function: ( ) ( )(1) 3,2f x∇ =

– Observation: the objective function value will decrease if we move along the negative gradient direction at point x(1) from that point

7

negative gradient direction at point x from that point

x(2)= x(1)+α1d1 where α1 is the step size

Integer Programming Models - AlgorithmsInteger Programming Models - Algorithms

Differences between solving a LP and an ILP

( ) 1 2max 5 4f x x x= +x2

5

Example 1. A LP problem Optimum: x1=3.75, x2=1.25, f=23.75

1 2

. .5

10 6 45

s tx x

x x+ ≤

+ ≤ 3

4

1 2

1 2

10 6 45, 0x x

x x+ ≤≥

1

2

E l 2 A ILP bl

( ) 1 2max 5 4f x x xt

= +x1

0 1 2 3 4

1

5

Example 2. An ILP problem

1 2

. .5

10 6 45

s tx x

x x+ ≤

+ ≤

Optimum: x1=3, x2=2, f = 23

It seems that rounding off the optimal solution for

8

1 2

1 2

10 6 45, 0 and integerx x

x x+ ≤≥

It seems that rounding off the optimal solution for Example 1 can yield the optimal solution for Example 2

Integer Programming Models Algorithms (cont’d)Integer Programming Models – Algorithms (cont d)

Example 3. A LP Problemp

1 2 1 2max ( , ). .

f x x x xs t

= + Optimum:

* 4 * 4 51 2

1 2

2 2 18 10 13

x xx x

− + ≥− + ≤

x1*=4, x2*=4.5

f(x1*,x2*)=8.5

1 2, 0x x ≥

Example 4. An ILP Problem

1 2 1 2max ( , ). .

f x x x xs t

= +Optimum: It is necessary to

develop dedicated1 2

1 2

2 2 18 10 13

x xx x

− + ≥− + ≤

x1*=1, x2*=2

f(x1*,x2*)=3

develop dedicated algorithms for IP

9

1 2, 0 and integerx x ≥

Integer Programming Models Algorithms (cont’d)Integer Programming Models – Algorithms (cont d)

Existing algorithmss g go s

Cutting plane methods

− Solve any MIP problems− have not been proved very successful on large-scale problems

Enumerative methods

− Solve the class of 0-1 PIP problems p− have been proved very successful with certain types of problem

and not very successful on others

Branch and bound methods

− Solve any MIP problems

10

− have proved most successful in general on practical MIP problems

O ti i ti S lOptimization Solvers

Software Packages CPLEX (http://www.ilog.com/products/cplex/) LINDO (http://www.lindo.com/)( p )AMPL (http://www.ampl.com/)Add-in Solver in Microsoft Excel

Optimization subroutines in programming languages

(http://www.solver.com/)

Optimization subroutines in programming languages

IMSL numerical library for Fortran (http://www absoft com/Products/Libraries/imsl html)(http://www.absoft.com/Products/Libraries/imsl.html)

Optimization tollbox in Matlab (http://www.mathworks.com/access/helpdesk/help/toolbox/optim/optim.shtml)

11

Add-in Solver of Microsoft ExcelAdd-in Solver of Microsoft Excel

Solver in Microsoft ExcelA powerful add-in tool in Microsoft Excel, which is of capability in solving

− Linear programming modelsp g g

− Nonlinear programming models

− Integer programming modelsInteger programming models

How to use Solver? I t ll S l Install Solver

To activate the Excel Solver for the first time, select Tools and then select Add-Ins Checking the Solver Add-in box will cause Excel to open SolverAdd Ins. Checking the Solver Add in box will cause Excel to open Solver whenever you check tools and then Solver.

Procedure to use Solver

12

Step 1: Set up a spreadsheet that tracks all the decision variables, objective function and constraints

Add-in Solver of Microsoft ExcelAdd-in Solver of Microsoft ExcelStep 2.1: From the Tools menu, select Solver. The dialog box in Figure 1

will appearwill appear.

Figure 1Figure 1

Step 2.2: Move the mouse to the Set Target Cell portion of the dialog box and

13

click on the target cell (objective function) defined in Step 1 and select min or max

Add-in Solver of Microsoft Excel (cont’d)Add-in Solver of Microsoft Excel (cont d)

Step 2.3: Move the mouse to the By Changing Cells portion of the dialog box p y g g p gand click the changing cells (decision variables) defined in Step 1.

Step 2 4: Click on the Add button to add constraints The following screen willStep 2.4: Click on the Add button to add constraints. The following screen will appear. Move the Cell Reference part of the Add constraint dialog box to add a constraint.

Step 2.5: Click on Options button in Figure 1 to configure parameters

S 2 6 Cli k h S l b f Fi 1

14

Step 2.6: Click on the Solve button of Figure 1

Add-in Solver of Microsoft Excel (cont’d)Add-in Solver of Microsoft Excel (cont d)

Three Examplesee p es

Example 1. LP Example 2. MIP

1 2max 5 4z x x= +

subject to1 2max 3z x x= +

bj t t1 26 4 24x x+ ≤

1x x− + ≤

subject to

1 217 11 86.5x x+ ≤

2 10 2+ ≤

subject to

1 2 1x x+ ≤

2 2x ≤1 22 10.2x x+ ≤

1 3.87x ≤

1 20, 0x x≥ ≥1 is a non-negative integerx

2 0x ≥

15

Add-in Solver of Microsoft Excel (cont’d)Add-in Solver of Microsoft Excel (cont d) Example 3. PIP Example 4. NLP

6

1min i

i

z x=

= ( ) 1 2 3min 0.04 0.1 0.07f x x x x= + +

subject to

1 2 1x x+ ≥subject to

1 2 3

22 2

100

0.01 0.001 0.5

x x x

x x

+ + =

+ ≤

subject to

1 2 6 1x x x+ + ≥

3 4 1x x+ ≥

21 2 1

2 21 2 3 1 2 2 3

0.35 0.04 0.002 1

0.1 0.06 0.3 0.01 0.05 0.001 9

x x x

x x x x x x x

− − ≥ −

+ − + + + ≥ −

3 4 5 1x x x+ + ≥

1x x x+ + ≥

2 21 2 3 1 2 2 3

1

0.1 0.06 0.3 0.01 0.05 0.001 125

x x x x x x xx

+ − + + + ≤≥

4 5 6 1x x x+ + ≥

2 5 6 1x x x+ + ≥

0 1 1 2 6

1

2

60050

xx

≤≥≥

16

0 or 1, 1,2, ,6ix i= = 3 50x ≥

AMPL - IntroductionAMPL - Introduction

A modeling language for mathematical programmingode g gu ge o e c p og g AMPL is notable for the similarity of its arithmetic expressions to

customary algebraic notation, AMPL is notable for the generality and power of its set and subscripting

expression. AMPL was designed and implemented by Fourer, R. (Northwestern AMPL was designed and implemented by Fourer, R. (Northwestern

University), Gay, G., and Kernighghan, B. W. (Princeton University) around 1985.

Edit Editor Notepad

Running environment Running environment DOS

17

AMPL A Rudimentary FormatAMPL – A Rudimentary Format

Example 1: LPp e :

( ) 1 2max 5 4z x x x= +var x1 >=0;

AMPL Code

1 26 4 24x x+ ≤subject

var x1 0;var x2 >=0;maximize z: 5*x1+4*x2;subject to

1 22 6x x+ ≤

1 2 1x x− + ≤

subject to c1: 6*x1+4*x2<=24;c2: x1+2*x2<=6;3 1+ 2< 11 2

0≥2 2x ≤

c3: -x1+x2<=1;c4: x2<=2;

solve;1 2, 0x x ≥ display z,x1,x2;

18

AMPL – An Algebraic Model

#--------------------------------------------AMPL modelparam m;param n;

1 Model

( ) 1 2max 5 4z x x x= +

param c1..n;param b1..m;

param a1..m,1..n;

1 26 4 24x x+ ≤subject

p , ;

var x1..n>=0;

i i j i 1 [j]* [j]

1 22 6x x+ ≤

1x x+ ≤

maximize z: sumj in 1..nc[j]*x[j]; subject to restri in 1..m:sumj in 1..na[i,j]*x[j]<=b[i];#------------------------------------------specify model data

1 2 1x x− + ≤

2 2x ≤

p ydata;param n:=2;param m:=4;

1 5 2 41 2, 0x x ≥ param c:= 1 5 2 4;

param b:=1 24 2 6 3 1 4 2;param a: 1 2 :=

1 6 4 2 1 23 -1 14 0 1;

# l e the ble

19

#------------------------------------------solve the problemsolve;display z, x;

AMPL MIP

param n;set I=1..n;set J=1 n; #I is the same as JAMPL – MIP

1 2 3min penalty 19 12 34s s s+ + += + +

set J 1..n; #I is the same as Jparam pI;param dI;param perDayPenaltyI;

1 2 12 20x x My− + ≥subject to

( )1 2 121 5x x M y− + + − ≥

param M=1000;var xJ>=0; #continuousvar yI,J binary; #0-1var sMinusJ>=0; # s=sMinus-sPlus( )1 2 121 5x x M y+ + ≥

1 3 13 15x x My− + ≥

( )1 5x x M y+ + ≥

var sMinusJ> 0; # s sMinus sPlusvar sPlusJ>=0; minimize penalty: sum j in J

perDayPenalty[j]*sPlus[j]; ( )1 3 131 5x x M y− + + − ≥

2 3 23 15x x My− + ≥

( )1 20M+ + ≥

subject toConstraintType1i in I,j in J:i<j:

M*y[i,j]+x[i]-x[j]>=p[j];ConstraintType2i in I j in J:i<j:( )2 3 231 20x x M y− + + − ≥

1 1 1 25 5x s s+ −+ − = −

ConstraintType2i in I,j in J:i<j:M*(1-y[i,j])+x[j]-x[i]>=p[i];

dueDatej in J:x[j]+sMinus[j]-sPlus[j]=d[j]-p[j];data;

2 2 2 22 20x s s− ++ − = −

3 3 3 35 15x s s− ++ − = −0− + − + − + ≥

param n:=3;param p:= 1 5 2 20 3 15;param d:= 1 25 2 22 3 35;param perDayPenalty := 1 19 2 12 3 34;

20

1 2 3 1 1 2 2 3 3, , , , , , , , 0x x x s s s s s s+ + + ≥ 12 13 23, , 0,1y y y ∈

param perDayPenalty : 1 19 2 12 3 34;option solver cplex; solve;display penalty,x;

AMPL NLPAMPL – NLP

( ) 1 2max z x x x= − #------- Example -NLP( ) 1 2

subject to43 243+ ≤

# Example NLPvar x1>=0 :=10; #inital value = 10var x2>=0 :=10; #initial value = 10

41 23 243x x+ ≤

21 22 32x x+ ≤

maximize z: x1-x2;subject to1 3* 1^4+ 2< 243

1 2.1x ≥

2 3.5x ≥

c1: 3*x1^4+x2<=243;c2: x1+2*x2^2<=32;c3: x1>=2.1;c4: x2>=3.5;

solve;Remark:

display z,x1,x2;An initial feasible solution should be provided by users

21