sebastian bitzer ([email protected])[email protected] seminar constraint logic programming university of...

Sebastian Bitzer ([email protected])Seminar Constraint Logic ProgrammingUniversity of OsnabrueckNovember 11th 2002

The Simplex Algorithm

An approach to optimization problems for linear real

arithmetic constraints

mailto:[email protected]

11 November, 2002 The Simplex Algorithm 2

Content

• Optimization – what is that?

• The Simplex Algorithm – background

• Simplex form

• Basic feasible solved form / basic feasible solution

• The algorithm

• Initial basic feasible solved form


Content



• Simplex form


• The algorithm



Optimization – what is that?

• In general

• Optimization problem (C,f) with constraint C and objective function f

e.g. 4: YXC



Objective function f:– expression over variables V in constraint C– evaluates to a real number– e.g.

²²: YXf



a valuation θ (substituting variables by values):

solution of objective function using θ:

valuesv

variables

n1

1

2211 ,,,

n

nnX

vXvXvX

nvvvff ,,,: 21



preferred valuations:– valuation θ is preferred to valuation θ',

if f(θ) < f(θ')

optimal solution:– θ is optimal,

if f(θ) < f(θ') for all solutions θ' θ

(there is no solution that is preferred to θ)



Do all problems have an optimal solution?

497 XX

XXfX )( with 77

Optimization Example

0 1 2 3 4

1

2

3

4

Y

X

X+Y=4

An optimization problem

),4( 22 YXfYXC

Find the closest point to the origin satisfying the C. Some solutions and f value

8}2,2{

18}3,3{

16}4,0{

YX

YX

YX

Optimal solution

}2,2{ YX


Content



• Simplex form


• The algorithm



Background

• George Dantzig• born 8.11.1914,

Portland• invented "Simplex

Method of Optimisation" in 1947

• this grew out of his work with the USAF


Background

• originates from planning tasks:– plans or schedules for training– logistical supply– deployment of men

• has in practice usually polynomial cost


Quotes

Eugene Lawler (1980):[Linear programming] is used to allocate

resources, plan production, schedule workers, plan investment portfolios and formulate marketing (and military) strategies. The versatility and economic impact of linear programming in today's industrial world is truly awesome.


Quotes

Dantzig I:The tremendous power of the simplex method is a

constant surprise to me.

Dantzig II:... it is interesting to note that the original problem that

started my research is still outstanding - namely the problem of planning or scheduling dynamically over time, particularly planning dynamically under uncertainty. If such a problem could be successfully solved it could eventually through better planning contribute to the well- being and stability of the world.


Content



• Simplex form


• The algorithm


The example

0 1 2 3 4

1

2

3

4

Y

X

-2 -1 0

1

2

preferred solutions

An optimization problem showing contours of the objective function

minimize subject toX Y

Y

X

X

Y X

0

1

3

2 3


• (C,f) is in simplex form,

if C has the form

• CE is a conjunction of linear arithmetic equations

• CI is a term that constrains all variables in C to be 0

Simplex form

IECC


Simplex form

allowed conversions to get simplex form:– X not constrained to be non-negative:

– inequality e r (e=expression and r=number)

0and0with XXXXX

0with SrSere

The example

0 1 2 3 4

1

2

3

4

Y

X

-2 -1 0

1

2

preferred solutions



Y

X

X

Y X

0

1

3

2 3

An equivalent simplex form is:

00000

32

3

1

321

1

3

2

SSSYX

SYX

SX

SX


Content



• Simplex form


• The algorithm



Basic feasible solved form

feasible = practicable, able to be carried out (durchführbar, anwendbar)

• a simplex form optimization problem is in basic feasible solved form,

if all equations in CE (of the simplex form) have the form:

nn XaXabX 110


Basic feasible solved form

• X0 is called basic variable, does not occur anywhere else (neither in objective function)

• X1...n are parameters

• b, a1...n are constants

• b0

nn XaXabX 110


Basic feasible solution

• corresponding basic feasible solution to a basic feasible solved form:– setting each X1...n = 0

nn XaXabX 110

bX 0

0 1 2 3 4

1

2

3

4

Y

X

-2 -1 0

1

2

preferred solutions

The example



00000

32

3

1

321

1

3

2

SSSYX

SYX

SX

SX


Y

X

X

Y X

0

1

3

2 3

minimize subject to0 0 5 05

3 0 5 05

2

3

1 3

1 3

2 3

3

. .

. .

S S

Y S S

S S

X S

Basic feasible solved form:


Content



• Simplex form


• The algorithm



The algorithm

• idea: optimal solution has to be in one of the vertices

• so: go from one vertex to the preferred next vertex

• end: if there is no preferred vertex, the actual has to be the optimal solution


The algorithm

in other words:– take a basic feasible solved form– look for an "adjacent" basic feasible solved

form whose basic feasible solution decreases the value of the objective function

– if there is no such adjacent basic feasible solved form, then the optimum has been found


The algorithm

• adjacent just one single pivot• pivoting move one variable out of basic

variables ( exit variable) and another in ( entry variable)

ZYX 21749

XZY 7137


Entering Variable:

– choose one YJ with dJ<0

pivoting on this YJ can only decrease f (see next slide)

– no such YJ optimum has been found

The algorithm

Problem: Which variables should be

exiting resp. entering?

0011

11

1

j

m

ji

n

i

m

jjijii

n

i

m

jjj

YX

YabX

Ydef


Why pivoting on a Yj with dj<0 decreases objective function f

• looking at the basic feasible solution (bfs) every parametric variable (Yj) is set to 0

• pivoting on such a variable (var. becomes basic) leads to an increase of this variable in the bfs: Yj≥0

a Yj with negative dj decreases f

JJYdYdef 11


The algorithm

Exiting variable:– we have to maintain basic feasible solved form

all bi's have to be 0

choose a Xi so that –bI/aIJ is a minimum of:

– M={Ø} optimization problem unbounded

niaa

bM iJ

iJ

i 1 and0

Sebastian Bitzer

you choose the minimum because only then it is sure that all from the pivot resulting b_i are positiveexample:x1 = 8 - 4yx2 = 10 - 2yby choosing x2 (the maximum) as exiting variable the resulting b in equation x1 will become -12

Simplex Example

minimize subject to10

3

4 2 2

0 0 0 0

Y Z

X Y

T Y Z

X Y Z T

Choose variable Y, the first eqn is only one with neg. coeff Y X 3

minimize subject to7

3

10 2 2

0 0 0 0

X Z

Y X

T X Z

X Y Z T

Choose variable Z, the 2nd eqn is only one with neg. coeff Z X T 5 05.

minimize subject to2 2 05

3

5 05

0 0 0 0

X T

Y X

Z X T

X Y Z T

.

.

No variable can be chosen, optimal value 2 is found


The algorithmstarting from a problem in bfs form

repeat

Choose a variable y with negative coefficient in the obj. func.

Find the equation x = b + cy + ... where c<0 and -b/c is minimal

Rewrite this equation with y the subject y = -b/c + 1/c x + ...

Substitute -b/c + 1/c x + ... for y in all other eqns and obj. func.

until no such variable y exists or no such equation exists

if no such y exists optimum is found

else there is no optimal solution

The example

Basic feasible solution form: circleminimize subject to0 05 05

3 05 05

2

3

1 3

1 3

2 3

3

. .

. .

S S

Y S S

S S

X S

Choose S3, replace using 2nd eq

2

23

21

21

1

2

5.05.02

subject to 5.05.01 minimize

SX

SS

SSY

SS

Optimal solution: box

0 1 2 3 4

1

2

3

4

Y

X

-2 -1 0

1

2


Content



• Simplex form


• The algorithm



Initial basic feasible solved form

idea:– solve a different optimization problem

this optimization problem should have an initial basic feasible solved form, which:– can be found trivially– has an optimal solution that leads to an initial

basic feasible solved form of the original problem



0:1

11

1

j

m

ji

m

jjij

n

i

m

jjj xbxaxdef

00

:

11

11

1

i

n

ij

m

j

m

jjijii

n

i

n

ii

zx

xabzzf

add artificial variables and minimize on them:



to get basic feasible solved form:

n

i

m

jjiji

n

ii xabzf

1 11

solve this problem



possible outcomes:– (f > 0) original problem unsatisfiable

– (f = 0) (zi...n parametric) got a basic feasible solved form for original problem

– (f = 0) (zi...n parametric) zi must occur in such an equation:

m

jjj

n

iii xazdz

11

''0

Sebastian Bitzer

z = 0 + ... , because we solved f = \sum_{i=1}^{n} z_i and f = 0 and all b >= 0


Such an equation is no problem, because

• if all a’j = 0 → equation is redundant

• if one a’j ≠ 0 → use according xj for pivoting z out of basic variables (this maintains basic feasible solved form since z = 0 +…)

all z become parametric

m

jjj

n

iii xazdz

11

''0

The example

0 1 2 3 4

1

2

3

4

Y

X

-2 -1 0

1

2

preferred solutions



Y

X

X

Y X

0

1

3

2 3


00000

32

3

1

321

1

3

2

SSSYX

SYX

SX

SX

The example

X S

X S

X Y S

2

3

1

1

3

2 3

A X S

A X S

A X Y S

1 2

2 3

3 1

1

3

3 2

With artificial vars in bfs form:

Objective function: minimizeA A A

X Y S S S1 2 3

1 2 37 2

Original simplex form equations

The example

minimize subject toA A A

Y S S A A

S S A A

X S A

1 2 3

1 3 2 3

2 3 1 2

3 2

3 05 05 05 05

2

3

. . . .

Problem after minimization of objective function

Removing the artificial variables, the original problem

Y S S

S S

X S

3 0 5 0 5

2

3

1 3

2 3

3

. .


Simplex solver

finding a basic feasible solution is exactly a constraint satisfaction problem

efficient constraint solver for linear inequalities


CyclingProblem:

– if for one of the basic variables is valid: Xi = 0 + … , a pivot could be performed which does not change the corresponding basic feasible solution

danger of pivoting back

Solution:– use e.g. Bland’s anti-cycling rule (always select

candidate with smallest index: x2 instead of x4)


Summary

We have seen that optimisations of linear real arithmetic constraints play an important role in many applications.

The Simplex Method which was introduced here provides a very efficient algorithm to determine whether there exists an optimal solution to linear real arithmetic constraints and if there exists one, to compute it.


Literature

• books:

George B. Dantzig, Mukund N. Thapa

"Linear Programming I: Introduction"

Springer Verlag

Kim Marriott & Peter J. Stuckey

"Programming with Constraints: An Introduction"

MIT Press


Literature

• examples are taken from a presentation of Marriott & Stuckey and could be accessed via internet:

http://www.cs.mu.oz.au/~pjs/book/course.html

• this presentation in the net:

http://www-lehre.inf.uos.de/~sbitzer/clp

http://www.cs.mu.oz.au/~pjs/book/course.html







sebastian bitzer ([email protected])[email protected] seminar constraint logic programming university of...

Documents

simplex algorithm12

simplex method of optimisation

usaf slide

f value optimal solution

f f optimal solution

optimization problems

optimization example

polynomial cost slide