linear programming the simplex algorithm chapter...
TRANSCRIPT
Linear ProgrammingThe Simplex Algorithm
Chapter 5
University of ChicagoBooth School of Business
Kipp Martin
October 17, 2017
1
Outline
List of Files
Key Concepts
Geometry and Theory
The Simplex Algorithm
2
List of Files
I simplexpivot.m (does one pivot)
I basicsimplex.m (executes simplex algorithm)
I lptest1.m
I lptest2.m
File Location:
http://faculty.chicagobooth.edu/kipp.martin/root/
htmls/coursework/36900/matlab.html
Reading: See Chapter 5 of the text.
3
Key Concepts
I standard form
I basis and basic solution
I basic feasible solution
I extreme point
I fundamental theorem of linear programming
I the simplex algorithm
4
DISCLAIMER!
What follows in these notes is a textbook description of thesimplex algorithm!
These notes are designed to explain the simplex algorithm from aconceptual point of view.
We ignore how to actually implement the algorithm. This isdiscussed in part 2 of the notes.
5
Motivation
I Projection and inverse projection are fantastic when it comesto understanding linear programming theory.
I As presented, the projection and inverse projection algorithmsare not practical for solving linear programming problems.
I Not only do projection and inverse projection have worst caseexponential running time, but this is the “expected case” –these algorithms must run to termination before you have theanswer. They do not terminate early.
I We now present an algorithm with worst case exponentialbehavior. But in practice it works very well. It terminateswhen optimality conditions are satisfied.
6
Motivation
I We will return to projection and inverse projection basedalgorithms when we study decomposition approaches.
I We want to solve min{c>x |Ax ≥ b} where A has millions ofnonzero elements.
I Two approaches – launch a direct frontal attack or tryflanking decomposition maneuvers.
I We study the frontal attack first.
Kipp’s lemma – if you can read a linear program into main memoryyou can solve it reasonably fast.
7
Geometry and Theory
Standard Form: for mathematical convenience we are going toassume every linear program we work with is in standard form. Thestandard form is:
min c>x
s.t. Ax = b
x ≥ 0
Any linear program can be put into standard form.
8
Geometry and TheoryThe linear program
max 10x1 + 9x2
.7x1 + x2 ≤ 630
.5x1 + (5/6)x2 ≤ 600
x1 + (2/3)x2 ≤ 708
.1x1 + .25x2 ≤ 135
x1, x2 ≥ 0
in standard form is
min −10x1 −9x2.7x1 +x2 +x3 = 630.5x1 +(5/6)x2 +x4 = 600x1 +(2/3)x2 +x5 = 708
.1x1 +.25x2 +x6 = 135x1, x2, x3, x4, x5, x6 ≥ 0
9
Geometry and Theory
After writing the linear program in standard form assume that A isan m × n matrix – i.e. we have m constraints (rows) and nvariables (columns).
Let B index a set of m variables. If the columns of A indexed by Bare linearly independent then the columns of A indexed by B are abasis and the variables xi , i ∈ B are called basic variables.
10
Geometry and Theory
By xB we mean an m component vector of variables withcomponents xBi
for i = 1, . . . ,m. We often say variable xBiis basic
in row i .
Let AB be the m ×m submatrix of A consisting of the columns ofA indexed by B. The variables not indexed B are called nonbasicvariables. The nonbasic variables are indexed by N. The matrixAN is the m× (n−m) submatrix of A consisting of the columns ofA indexed by N. Similarly, xN is an (n −m) component vector ofnonbasic variables.
11
Geometry and Theory
Let’s look at an example using MATLAB. The constraint matrix ofthe sample linear program is
>> A = [.7 1 1 0 0 0;
.5 5/6 0 1 0 0;
1 2/3 0 0 1 0;
.1 .25 0 0 0 1]
Define a basis by B by
>> B = [2 4 1 6]
The basis matrix corresponding to B expressed in MATLAB is
>> AB = A(:, B)
What is an easy way in MATLAB to test if B is really a basis?
12
Geometry and Theory
Since
>> B = [2 4 1 6]
We would say
I Variable x2 is basic in row 1
I Variable x4 is basic in row 2
I Variable x1 is basic in row 3
I Variable x6 is basic in row 4
13
Geometry and Theory
>> inv(AB)*A
ans =
0 1.0000 1.8750 0 -1.3125 0
0.0000 0.0000 -0.9375 1.0000 0.1563 0
1.0000 0 -1.2500 0 1.8750 0
0 0 -0.3437 0 0.1406 1.0000
Stated another way
I Variable x1 is basic in row 3
I Variable x2 is basic in row 1
I Variable x4 is basic in row 2
I Variable x6 is basic in row 4
14
Geometry and Theory
If we set the non-basic variables to zero, what is the implied valueof the basic variables for a given b.
>> inv(AB)*b
=
252.0000
120.0000
540.0000
18.0000
I Variable x1 is basic in row 3 – value = ?
I Variable x2 is basic in row 1 – value = ?
I Variable x4 is basic in row 2 – value = ?
I Variable x6 is basic in row 4 – value = ?
15
Geometry and TheoryHere is the math. The constraint matrix is
Ax = b
Partitioned into basic and non-basic variables
ABxB + ANxN = b
Multiply by the basis inverse
A−1B ABxB + A−1B ANxN = A−1B b
xB + A−1B ANxN = A−1B b
Since xN = 0,
xB = A−1B b
16
Geometry and Theory
We have
xB = A−1B b
But we also require xB ≥ 0.
If, indeed, xB ≥ 0, then xB is a basic feasible solution.
Now, for some geometry.
17
Geometry and TheoryWe are working with the constraints:
.7x1 +x2 +x3 = 630
.5x1 +(5/6)x2 +x4 = 600x1 +(2/3)x2 +x5 = 708
.1x1 +.25x2 +x6 = 135x1, x2, x3, x4, x5, x6 ≥ 0
Which is equivalent to the following (and plotted on the followingpage).
.7x1 + x2 ≤ 630
.5x1 + (5/6)x2 ≤ 600
x1 + (2/3)x2 ≤ 708
.1x1 + .25x2 ≤ 135
x1, x2 ≥ 0
18
Geometry and Theory
19
Geometry and Theory
If you were to enumerate every basic feasible solution, they wouldcorrespond to (moving clockwise from the origin):
I (0, 0)
I (0, 540)
I (300, 420)
I (540, 252)
I (708, 0)
20
Geometry and Theory
These points are called extreme points of the feasible region. Apoint, x , is an extreme point of the feasible region if and only if itcannot be written as
x = λ1x1 + λ2x
2, λ1 + λ2 = 1, λ1, λ2 ≥ 0
where x1 and x2 are distinct points in the feasible region.
In math jargon we say x is an extreme point if and only if it is nota convex combination of two distinct feasible points.
Another way to state this is that x is an extreme point if and onlyif it is not on a line segment connecting two distinct points in thefeasible region.
21
Geometry and Theory
There are three famous theorem critical to the Simplex algorithm.
Theorem 1: A point x is an extreme point of the LP feasibleregion if and only if it is a basic feasible solution.
Theorem 2 (finite basis theorem): Any point in a bounded LPfeasible region can be written as a convex combination of extremepoints.
Theorem 3 (fundamental theorem of linear programming): Ifthere is an optimal solution to a linear program there is an optimalextreme point solution to the linear program.
Theorem 1 + Theorem 3 imply we can solve a linear by restrictingour search to basic feasible solutions. This is exactly what theSimplex algorithm does.
22
The Simplex Algorithm
The linear program is standard form is:
min c>x
s.t. Ax = b
x ≥ 0
Let z0 = c>x and write this as:
z0 − c>x = 0
Ax = b
x ≥ 0
23
The Simplex AlgorithmLet B index the basic variables and N the nonbasic variables. Then
c> = [c>B , c>N ]
A = [AB ,AN ]
and we rewrite
z0 − c>x = 0
Ax = b
x ≥ 0
as
z0 − c>B xB − c>N xN = 0
ABxB + ANxN = b
xB , xN ≥ 0
24
The Simplex AlgorithmIn the system below multiply the second set of constraints by A−1B
z0 − c>B xB − c>N xN = 0
ABxB + ANxN = b
xB , xN ≥ 0
this multiplication gives:
z0 − c>B xB − c>N xN = 0
A−1B ABxB + A−1B ANxN = A−1B b
xB , xN ≥ 0
which is
z0 − c>B xB − c>N xN = 0
xB + A−1B ANxN = A−1B b
xB , xN ≥ 0
25
The Simplex AlgorithmOkay, we have:
z0 − c>B xB − c>N xN = 0
xB + A−1B ANxN = A−1B b
xB , xN ≥ 0
Now multiply the second set of constraints by c>B to get:
c>B xB + c>B A−1B ANxN = c>B A−1B b
and add to
z0 − c>B xB − c>N xN = 0
The terms c>B xB and −c>B xB cancel when we add.26
The Simplex Algorithm
This gives:
z0 + (c>B A−1B AN − c>N )xN = c>B A−1B b
xB + A−1B ANxN = A−1B b
xB , xN ≥ 0
We can rewrite as
z0 = c>B A−1B b + (c>N − c>B A−1B AN)xN
xB = A−1B b − A−1B ANxN
xB , xN ≥ 0
27
The Simplex Algorithm
Reduced Cost: The reduced costs of the nonbasic variables aredefined by
w>N := c>N − c>B A−1B AN
and the system on the previous slide is
z0 = c>B A−1B b + w>NxN
xB = A−1B b − A−1B ANxN
xB , xN ≥ 0
28
The Simplex Algorithm
Assume the basis B indexes a feasible solution. Then xB ≥ 0 andxN = 0.
z0 = c>B A−1B b + w>NxN
xB = A−1B b − A−1B ANxN
xB , xN ≥ 0
1. If a component has xN has a negative reduced cost, increasingthe value of that variable will lower the value of z0 and wehave an improved feasible solution assuming xB remainsnonnegative.
2. If a component of xN has a negative reduced cost we want toincrease it as much as possible – until a basic variable hitszero.
29
The Simplex Algorithm
Example: Consider the linear program
min 2x1 −3x2x2 ≥ 1
x1 +x2 ≥ 2−.5x1 +x2 ≤ 8−x1 +x2 ≤ 6x1, x2 ≥ 0
In standard form this linear program is
min 2x1 −3x2x2 −x3 = 1
x1 +x2 −x4 = 2−.5x1 +x2 +x5 = 8−x1 +x2 +x6 = 6x1, x2, x3, x4, x5, x6 ≥ 0
30
The Simplex Algorithm
Index a set of basic variables with B = {1, 2, 5, 6}, the objectivefunction row and system the
z0 − w>NxN = c>B A−1B b
xB + A−1B ANxN = A−1B b
xB , xN ≥ 0
is
z0 +5x3 −2x4 = −1x1 +x3 −x4 = 1
x2 −x3 = 1x5 +1.5x3 −.5x4 = 7.5
x6 +2x3 −x4 = 6
Why is this feasible?
31
The Simplex Algorithm
Rewriting gives:
z0 = −1 −5x3 +2x4x1 = 1 −x3 +x4
x2 = 1 +x3x5 = 7.5 −1.5x3 +.5x4
x6 = 6 −2x3 +x4
The current solution implied by the tableau above is:
x1 = 1, x2 = 1, x5 = 7.5, x6 = 6 x3 = 0, x4 = 0
with solution value z0 = −1.
32
The Simplex Algorithm
Variable x3 has a negative reduced cost of −5.
z0 = −1 −5x3 +2x4x1 = 1 −x3 +x4
x2 = 1 +x3x5 = 7.5 −1.5x3 +.5x4
x6 = 6 −2x3 +x4
If x3 is increased from 0 to 1 this one unit increase decreases theobjective function value by 5, the amount of the reduced cost, andthe corresponding solution is
x1 = 0, x2 = 2, x5 = 6, x6 = 4 x3 = 1, x4 = 0
which is feasible!
33
The Simplex Algorithm
Increasing x3 is clearly a good thing. How much can x3 beincreased?
z0 = −1 −5x3 +2x4x1 = 1 −x3 +x4
x2 = 1 +x3x5 = 7.5 −1.5x3 +.5x4
x6 = 6 −2x3 +x4
The negative coefficients in the x3 column limit the increase of x3.In this case the increase in x3 is limited by the minimum ratiobelow (we take absolute values of the negative numbers and donot consider the positive numbers):
1/1 = 1.0, 7.5/1.5 = 5.0, 6/2 = 3.0
The limit on the increase in x3 is the minimum ratio of 1.0.
34
The Simplex Algorithm
Bringing x3 in at a value of 1.0 drives basic variable x1 to zero andout of the basis.
We are making x3 basic and x1 nonbasic. In simplex terminologywe pivot x3 into the basis and x1 out. By convention thetableau is written this way
z0 +5x3 −2x4 = −1x1 +x3 −x4 = 1
x2 −x3 = 1x5 +1.5x3 −.5x4 = 7.5
x6 +2x3 −x4 = 6
35
The Simplex Algorithm
In the tableau below, pivot in x3 and pivot out x1.
z0 +5x3 −2x4 = −1x1 +x3 −x4 = 1
x2 −x3 = 1x5 +1.5x3 −.5x4 = 7.5
x6 +2x3 −x4 = 6
This gives the new tableau with basis B = {3, 2, 5, 6}.
z0 −5x1 +3x4 = −6x3 +x1 −x4 = 1
x2 +x1 −x4 = 2x5 −1.5x1 +x4 = 6
x6 −2x1 +x4 = 4
The objective function value has moved from −1 to −6.
36
The Simplex Algorithm
The updated tableau is:
z0 −5x1 +3x4 = −6x3 +x1 −x4 = 1
x2 +x1 −x4 = 2x5 −1.5x1 +x4 = 6
x6 −2x1 +x4 = 4
Variable x4 has a reduced cost of −3 (why not 3?). This variablecan come in at a maximum of
6/1 = 6, 4/1 = 4
The minimum ratio is 4, and this corresponds to the row where x6is basic. Pivot in variable x4 and pivot out variable x6.
37
The Simplex Algorithm
In the tableau below, pivot in x4 and pivot out x6.
z0 −5x1 +3x4 = −6x3 +x1 −x4 = 1
x2 +x1 −x4 = 2x5 −1.5x1 +x4 = 6
x6 −2x1 +x4 = 4
This gives the new tableau with basis B = {3, 2, 5, 4}.
z0 x1 −3x6 = −18x3 −x1 +x6 = 5
x2 −x1 +x6 = 6x5 +.5x1 −x6 = 2
x4 −2x1 +x6 = 4
The objective function value has moved from −6 to −18.
38
The Simplex Algorithm
The updated tableau is:
z0 x1 −3x6 = −18x3 −x1 +x6 = 5
x2 −x1 +x6 = 6x5 +.5x1 −x6 = 2
x4 −2x1 +x6 = 4
Variable x1 has a reduced cost of −1 (why not 1?). This variablecan come in at a maximum of
2/.5 = 4
The minimum ratio is 4, and this corresponds to the row where x5is basic. Pivot in variable x1 and pivot out variable x5. Note:variable x1 was pivoted out in a previous iteration and is nowcoming back into the basis.
39
The Simplex Algorithm
In the tableau below, pivot in x1 and pivot out x5.
z0 x1 −3x6 = −18x3 −x1 +x6 = 5
x2 −x1 +x6 = 6x5 +.5x1 −x6 = 2
x4 −2x1 +x6 = 4
This gives the new tableau with basis B = {3, 2, 1, 4}.
z0 −2x5 −x6 = −22x3 +2x5 −x6 = 9
x2 +2x5 −x6 = 10x1 +2x5 −2x6 = 4
x4 +4x5 −3x6 = 12
The objective function value has moved from −18 to −22.
40
The Simplex Algorithm
The updated tableau is:
z0 −2x5 −x6 = −22x3 +2x5 −x6 = 9
x2 +2x5 −x6 = 10x1 +2x5 −2x6 = 4
x4 +4x5 −3x6 = 12
I claim we are optimal!
41
The Simplex Algorithm
Oh No! No! No! No! I am worried again.
I am willing to buy into the fact that at this local point there is nofeasible improving direction. But what if I were somewhere else inthe feasible region?
Optimality Conditions to the rescue!!!!
42
The Simplex Algorithm
The linear program of interest (i.e the primal) is:
min c>x
Ax = b
x ≥ 0
The dual is:
min b>u
u>A ≤ c>
Let’s derive the optimality conditions.
43
The Simplex Algorithm
Optimality Conditions: (x , u) are an optimal primal-dual pair ifand only if
Ax = b,
x ≥ 0
u>A ≤ c>,
(u>A− c>)x = 0
So where are we?
44
The Simplex AlgorithmLet u> = c>B A−1B . The dual constraints are u>A ≤ c> so dualfeasibility is:
c>B A−1B A ≤ c>
written another way
c>B A−1B [AB ,AN ] ≤ [c>B , c>N ]
[c>B I , c>B A−1B AN ] ≤ [c>B , c>N ]
[0, c>B A−1B AN − c>N ] ≤ 0
what is
w>N = c>N − c>B A−1B AN?
45
The Simplex Algorithm
The reduced costs are:
w>N = c>N − c>B A−1B AN?
The simplex stopping criterion is positive reduced costs whichcorresponds to dual feasibility.
46
The Simplex Algorithm
But the tableau is:
z0 + (c>B A−1B AN − c>N )xN = c>B A−1B b
xB + A−1B ANxN = A−1B b
xB , xN ≥ 0
We can rewrite as
z0 = c>B A−1B b + (c>N − c>B A−1B AN)xN
xB = A−1B b − A−1B ANxN
xB , xN ≥ 0
so where are we?
47
The Simplex Algorithm
At every step of the simplex algorithm we are primal feasible.
We terminate when we are dual feasible.
What about complementary slackness? The condition
(u>A− c>)x = 0
But in simplex
[0, c>B A−1B AN − c>N ] = (u>A− c>)
Do we have complementary slackness?
48
The Simplex Algorithm
Here is the geometry of what we have done.
2 4 6 8x1
2
4
6
8
10
12x2
Feasible Region
Iteration 1
Iteration 2
Iteration 3
Iteration 4
49
The Simplex Algorithm
Doing simplex pivots by hand is cumbersome and does notcontribute much to learning. However, it is critical to understand:
I Selecting a variable to enter the basis
I Selecting a variable to leave the basis
I Doing a pivot
In order to facilitate this process use simplexpivot.m.
50
The Simplex Algorithm
simplexpivot.m
Inputs:
I A – the constraint matrix
I b – the right-hand-side vector
I c – the objective function vector
I B – an index set of the basic variables
Outputs:
I tableau – the tableau associated with basis B
I B – the updated basis vector after the user makes a pivot
Assumptions: we are working with standard form and solving aminimization problem.
51
The Simplex Algorithm
Let’s attack with simplexpivot.m the following linear program(use lptest2.m to execute this).
min 2x1 −3x2x2 −x3 = 1
x1 +x2 −x4 = 2−.5x1 +x2 +x5 = 8−x1 +x2 +x6 = 6x1, x2, x3, x4, x5, x6 ≥ 0
We use the following basis matrix
B = [1 2 5 6]
52
The Simplex Algorithm
[tableau, B] = simplexpivot(A, b, c, B);
-0.00 -0.00 5.00 -2.00 -0.00 -0.00 -1.00
----------------------------------------------------------
1.00 0.00 1.00 -1.00 0.00 0.00 1.00
0.00 1.00 -1.00 0.00 0.00 0.00 1.00
0.00 0.00 1.50 -0.50 1.00 0.00 7.50
0.00 0.00 2.00 -1.00 0.00 1.00 6.00
Enter the index of a variable to enter the basis 3
Row 1 has ratio 1.00
Row 3 has ratio 5.00
Row 4 has ratio 3.00
Enter the ROW NUMBER (not variable index) of the variable
to leave the basis 1
53
The Simplex Algorithm
-0.00 -0.00 5.00 -2.00 -0.00 -0.00 -1.00
----------------------------------------------------------
1.00 0.00 1.00 -1.00 0.00 0.00 1.00
0.00 1.00 -1.00 0.00 0.00 0.00 1.00
0.00 0.00 1.50 -0.50 1.00 0.00 7.50
0.00 0.00 2.00 -1.00 0.00 1.00 6.00
The result of a pivot on column 3 row 1
-5.00 -0.00 -0.00 3.00 -0.00 -0.00 -6.00
----------------------------------------------------------
1.00 0.00 1.00 -1.00 0.00 0.00 1.00
1.00 1.00 0.00 -1.00 0.00 0.00 2.00
-1.50 0.00 0.00 1.00 1.00 0.00 6.00
-2.00 0.00 0.00 1.00 0.00 1.00 4.00
B = 3 2 5 6
54
The Simplex Algorithm
>> [tableau, B] = simplexpivot(A, b, c, B);
-5.00 -0.00 -0.00 3.00 -0.00 -0.00 -6.00
----------------------------------------------------------
1.00 0.00 1.00 -1.00 0.00 0.00 1.00
1.00 1.00 0.00 -1.00 0.00 0.00 2.00
-1.50 0.00 0.00 1.00 1.00 0.00 6.00
-2.00 0.00 0.00 1.00 0.00 1.00 4.00
Enter the index of a variable to enter the basis 4
Row 3 has ratio 6.00
Row 4 has ratio 4.00
Enter the ROW NUMBER (not variable index) of the variable
to leave the basis 4
55
The Simplex Algorithm
-5.00 -0.00 -0.00 3.00 -0.00 -0.00 -6.00
----------------------------------------------------------
1.00 0.00 1.00 -1.00 0.00 0.00 1.00
1.00 1.00 0.00 -1.00 0.00 0.00 2.00
-1.50 0.00 0.00 1.00 1.00 0.00 6.00
-2.00 0.00 0.00 1.00 0.00 1.00 4.00
The result of the pivot on variable 4 in row 4 is
1.00 -0.00 -0.00 -0.00 -0.00 -3.00 -18.00
----------------------------------------------------------
-1.00 0.00 1.00 0.00 0.00 1.00 5.00
-1.00 1.00 0.00 0.00 0.00 1.00 6.00
0.50 0.00 0.00 0.00 1.00 -1.00 2.00
-2.00 0.00 0.00 1.00 0.00 1.00 4.00
B = 3 2 5 4
56
The Simplex Algorithm
>> [tableau, B] = simplexpivot(A, b, c, B);
1.00 -0.00 -0.00 -0.00 -0.00 -3.00 -18.00
----------------------------------------------------------
-1.00 0.00 1.00 0.00 0.00 1.00 5.00
-1.00 1.00 0.00 0.00 0.00 1.00 6.00
0.50 0.00 0.00 0.00 1.00 -1.00 2.00
-2.00 0.00 0.00 1.00 0.00 1.00 4.00
Enter the index of a variable to enter the basis 1
Row 3 has ratio 4.00
Enter the ROW NUMBER (not variable index) of the variable
to leave the basis 3
57
The Simplex Algorithm
1.00 -0.00 -0.00 -0.00 -0.00 -3.00 -18.00
----------------------------------------------------------
-1.00 0.00 1.00 0.00 0.00 1.00 5.00
-1.00 1.00 0.00 0.00 0.00 1.00 6.00
0.50 0.00 0.00 0.00 1.00 -1.00 2.00
-2.00 0.00 0.00 1.00 0.00 1.00 4.00
The result of the pivot on variable 1 in row 3 is
-0.00 -0.00 -0.00 -0.00 -2.00 -1.00 -22.00
----------------------------------------------------------
0.00 0.00 1.00 0.00 2.00 -1.00 9.00
0.00 1.00 0.00 0.00 2.00 -1.00 10.00
1.00 0.00 0.00 0.00 2.00 -2.00 4.00
0.00 0.00 0.00 1.00 4.00 -3.00 12.00
B = 3 2 1 4
58
The Simplex Algorithm
The original decisions variables were x1 and x2.
What is the optimal value of x1?
What is the optimal value of x2?
-0.00 -0.00 -0.00 -0.00 -2.00 -1.00 -22.00
----------------------------------------------------------
0.00 0.00 1.00 0.00 2.00 -1.00 9.00
0.00 1.00 0.00 0.00 2.00 -1.00 10.00
1.00 0.00 0.00 0.00 2.00 -2.00 4.00
0.00 0.00 0.00 1.00 4.00 -3.00 12.00
59
The Simplex Algorithm
The key steps in the Simplex Algorithm (assume minimization):
1. Start with a basic feasible solution and initialize the tableau.
2. Find a variable (pivot column) with a negative reduced cost.
3. Perform the minimum ratio test and find the pivot row.
4. Pivot and update the tableau (including reduced costs andright-hand-side).
5. Update the basic variable (B) and nonbasic variable (N) indexsets.
60
The Simplex AlgorithmThe MATLAB file basicsimplex.m implements the simplexalgorithm as we have described it.Inputs:
I A – the constraint matrix
I b – the right-hand-side vector
I c – the objective function vector
I B – an index set of the basic variables
Outputs:
I X – the optimal solution values of the variables
I objVal – the optimal objective function value.
I B – the basis of the optimal solution.
Assumptions: we are working with standard form and solving aminimization problem.
61
The Simplex Algorithm
Step 1: Start with a basic feasible solution and initialize thetableau.
numRows = size( b);
numVars = size( c);
N = setdiff([1:numVars], B)
AB = A(:, B);
AN = A(:, N);
numNonbasicVars = size( N’);
cN = c(N);
cB = c(B);
negRedCost = -1;
T = inv(AB)*A;
w = c’ - (cB’*inv(AB))*A;
wN = w( N);
bBAR = inv(AB)*b;
objVal = cB’*bBAR;
tableau = [-w , objVal; inv(AB)*A, bBAR]
62
The Simplex Algorithm
Step 2: Find a variable (pivot column) with a negative reducedcost.
for k = 1:numNonbasicVars
% look for a negative reduced cost
if wN( k) < 0
negRedCost = wN( k);
%record the index
negRedCostIdx = k;
break
end
end
63
The Simplex Algorithm
Step 3: Perform the minimum ratio test and find the pivot row.
% do minimum ratio test
% get the minimum ratio index
minRatioVal = inf;
for k = 1:numRows
if T( k, N(negRedCostIdx)) > 0
if bBAR(k) / T( k, N(negRedCostIdx)) < minRatioVal
minRatioVal = bBAR(k) / T( k, N(negRedCostIdx));
%record the index
minRatioIdx = k;
end
end
end
64
The Simplex AlgorithmStep 4: Pivot and update the tableau (including reduced costsand right-hand-side).
pivEl = T(minRatioIdx, N(negRedCostIdx));
% multiply pivot row by (1/pivEl)
T(minRatioIdx,:) = (1/pivEl).*T(minRatioIdx,:);
% also multiply the rhs by the pivot element
bBAR(minRatioIdx) = (1/pivEl)*bBAR(minRatioIdx);
% now pivot on row minRatioIdx, column N(negRedCostIdx)
for k = 1:numRows
if k ~= minRatioIdx
multiplier = T(k, N(negRedCostIdx));
if multiplier ~= 0
T(k,:) = T(k,:) - multiplier .* T(minRatioIdx,:);
bBAR(k) = bBAR(k) - multiplier*bBAR( minRatioIdx);
end
end
end
objVal = objVal + wN( negRedCostIdx)*bBAR( minRatioIdx );
w = w -wN( negRedCostIdx) .* T(minRatioIdx,:);
65
The Simplex Algorithm
Step 5: Update the basic variable (B) and nonbasic variable (N)index sets.
tmpIdx = B( minRatioIdx) ;
B( minRatioIdx) = N( negRedCostIdx);
N( negRedCostIdx) = tmpIdx;
% update Basic and Non-Basic matrices
AB = A(:, B);
AN = A(:, N);
cN = c(N);
cB = c(B);
wN = w(N);
66
The Simplex Algorithm
Here is a formal mathematical statement of the Simplex Algorithm.
Step 1: (Initialization) Initialize with a basis matrix AB such thatA−1B b ≥ 0 and create the initial tableau
z0 − wNxN = c>B A−1B b
xB + A−1B ANxN = A−1B b
Step 2: (Pivot Column Selection) If wN ≥ 0, stop, the currentsolution is optimal; else, select q ∈ N such that wq < 0.
67
The Simplex Algorithm
Step 3: (Minimum Ratio Test and Pivot Row Selection) Findthe maximum allowable increase of variable xq by the minimumratio test
i∗ ← argmin {xBi/aiq | aiq > 0, i = 1, . . .m}
Step 4: (Pivot and Update Tableau) Pivot on element ai∗q andupdate the tableau.
Step 5: (Update the Basic and Non-Basic Index Sets)
B ← (B\{Bi∗}) ∪ {q} N ← (N\{q}) ∪ {Bi∗}
Go to Step 2.
68