1

OutlineOutline

secrets

equivalence between row operations & matrix multiplication

simplex tableau in matrix form

revised simplex method relationship with column generation

2

The Most Beautiful …The Most Beautiful …

linear algebra

3

Maybe the Most Beautiful of All…Maybe the Most Beautiful of All…

algebraic properties

geometric properties

matrix properties

familiar with the equivalence

be lazy keeping and working only with the essence

e.g., how much information to carry in solving

(sometimes) use logic, not eyes e.g.,

4

To be at Home with the MaterialTo be at Home with the Material

3 4 7 12

2 6 8

w x y

w x y

3 4 7 12

2 1 6 8

3 4 7 12 4 3 7 12

2 1 6 8 1 2 6 8

in some sense

let E = and A =

EA =

5

Equivalence Between Equivalence Between Row Operations & Matrix MultiplicationRow Operations & Matrix Multiplication

making w basic in (1)

3 4 8 12

2 1 4 8

w x y b(1)

(2) row operations: (a) (1) = (1)/3

(b) (2) = (2)-2(1)

1 4 / 3 8 / 3 4

0 11/ 3 28 / 3 0

(1)

(2)

1/ 3 0

2 / 3 1

3 4 8 12

2 1 4 8

1 4 / 3 8 / 3 4

0 11/ 3 28 / 3 0

let E = and A =

EA =

6

Equivalence Between Equivalence Between Row Operations & Matrix MultiplicationRow Operations & Matrix Multiplication

3 4 8 12

2 1 4 8

w x y bmaking y basic in

(2)(1)

(2) row operations:

1 2

0 1/ 4

3 4 8 12

2 1 4 8

(1)

(2)

7 2 0 28

0.5 0.25 1 2

7 2 0 28

0.5 0.25 1 2

(a) (2) = (2)/4(b) (1) = (1)+8(2)

what should E be to make “v basic in (3)”?

7

Equivalence Between Equivalence Between Row Operations & Matrix MultiplicationRow Operations & Matrix Multiplication

4 1 0 4 8

1 0 4 5 4

2 2 6 3 2

v w x y b

8

Simplex TableauSimplex Tableau

x xS -z RHS

B. V.

cT 0 1 0

xS A I 0 b

initial tableau at some intermediate tableau with xB as basic variables

(xB xN)

xS -z RHS

B. V. 0 1 0

xS (B N) I 0 b

T T B Nc c

initial tableau with columns of xB in the intermediate tableau separated out

T T T 1B B Bc = c c B B = 0 T T T 1N N Bc = c c B N

TBc T

Nc TSc

T T 1 T 1 S B Bc = c B I c B

T 1 Bc B b

short form

x RHS

B. V.

xB B-1A B-1b

T T T 1 Bc = c c B A T 1 Bc B b

Minimization

xB xN xS -z RHS

B. V. 1

xB 0I B-1N B-1 B-1b

an iteration before minimal: 1 Find the smallest if all are non-negative, the minimal has

been found and stop; else continue.

2 Identify the entering variable xenter as the xj with the smallest

3 Identify the leaving variable xleave as xi with the minimal ratio. Stop if the problem is unbounded; else continue.

4 Identify aleave,enter from xenter and xleave.

5 Pivot on element aleave,enter to update the whole tableau and go to step 1.

9

Simplex ProcedureSimplex Procedurex RHS

B. V.

xB B-1A B-1b

T T T 1 Bc = c c B A T 1 Bc B b

;jc jc

.jc

no guarantee that the smallest gives the least number of iterations

can arbitrarily pick an xj with negative reduced cost as the entering variable

no need to update the whole tableau

10

Inefficient Simplex ProcedureInefficient Simplex Procedure

jc

opt.

minimal information: the set of current basic variables xB to generate the WHOLE tableau

conceptually, from xB

known cB

known current basis Bcur and hence known (Bcur)-1

any clever (i.e., lazy) method to get (Bnew)-1 from (Bcur)-1 without inverting Bnew every time?

the whole tableau from B-1

11

Minimal Information Minimal Information for the Simplex Procedurefor the Simplex Procedure

x RHS

B. V.

xB B-1A B-1b

T T T 1 Bc = c c B A T 1 Bc B b

keeping track of xB and (Bcur)-1

entering variable from reduced costs

leaving variable from minimum ratio test

finding (Bnew)-1 from (Bcur)-1

12

Revised Simplex AlgorithmRevised Simplex Algorithm

x RHS

B. V.

xB B-1A B-1b

T T T 1 Bc = c c B A T 1 Bc B b

suppose we have the current basic variables xB,cur and the inverse of the basis (Bcur)-1

known entities of the tableau:

13

Revised Simplex AlgorithmRevised Simplex Algorithm

xB,cur xN,cur -z RHS

B. V. 0 ? 1

xB I ? 0 (Bcur)-1b

T 1,cur cur( ) Bc B b

to find the entering variable xe : calculate for non-basic variables

stop if all reduced costs are non-negative; else pick the first xj with negative reduced cost as the entering variable

14

Revised Simplex AlgorithmRevised Simplex Algorithm

xB,cur xN,cur z RHS

B. V. 0 ? -1

xB I ? 0 (Bcur)-1b

T 1,cur cur( ) Bc B b

T 1,cur cur( )j j jc c

Bc B A where is the th columnj jA

to find the leaving variable xl

known column (Bcur)-1Ae of the entering variable xe with known RHS, execution of minimal ratio test to

determine the leaving variable xl (if available) pivoting on al,e to turn column e into (0, .., 0, 1, 0..,

0)T, where “1” occurs at the lth row

15

Revised Simplex AlgorithmRevised Simplex Algorithm

xB,cur xe xN,cur -z RHS

B. V. 0 ? 1

xB I (Bcur)-1Ae ? 0 (Bcur)-1b

T 1,cur cur( ) Bc B bT 1

,cur cur( ) 0e e ec c Bc B A

what should E be to make “v basic in (3)”?

16

Equivalence Between Equivalence Between Row Operations & Matrix MultiplicationRow Operations & Matrix Multiplication

4 1 0 4 8

1 0 4 5 4

2 2 6 3 2

v w x y b

making v basic in (3)

row operations: (a) (3) = (3)/2

(b) (2) = (2)+(3)(c) (1) = (1)-2(3)

13232

0 5 12 2 4

0 1 7 5

1 1 3 1

v w x y b

elementary matrix E =

1212

1 0 2

0 1

0 0

to find the elementary matrix E that turns Ae into

row operations are equivalent to pre-multiplying by matrix E, where E = I except the lth column,

17

Revised Simplex AlgorithmRevised Simplex Algorithm

row operations

0

0

1

0

0

e e

A EA

1,

,

Suppose e

e

m e

a

a

A

.

.

.

1

,

, if , ( )

. .l e

i e

l e

a

i l aa

i l

o w

E

eA

to find (Bnew)-1 from (Bcur)-1

claim: (Bnew)-1 = E(Bcur)-1

18

Revised Simplex AlgorithmRevised Simplex Algorithm

xB xN xS -z RHS

B. V. … … … 1 …

xB I … (Bcur)-1 0 ...

xB xN xS -z RHS

B. V. … … … 1 …

xB I … (Bnew)-1 0 ...

row operations

pre-multiplied by E

max 2x1+x2 min 2x1x2,

s.t. –x1+x2 2,

x2 4,

x1+x2 8,

x1 6,

x1, x2 0.19

Example of Example of Revised Simplex AlgorithmRevised Simplex Algorithm

20

Solving the ExampleSolving the Exampleby Simplex Methodby Simplex Method

21

Solving the ExampleSolving the Exampleby Simplex Methodby Simplex Method

22

Solving the ExampleSolving the Exampleby Simplex Methodby Simplex Method

23

Example of Revised Simplex Algorithm

revised simplex method no need to generate the whole tableau only generating columns when searching for first

negative reduced cost

column generation method generating column of non-basic variables only when

necessary usually with additional complexity to determine the

best entering variable for a given situation

24

Relationship Between Relationship Between Revised Simplex and Column GenerationRevised Simplex and Column Generation