OR-1 2015 1

Chapter 2. Simplex method

Geometric view : max , s.t.

(2,0)

(2,2/3)

(1,2)(0,2)

𝑥2

𝑥1

OR-1 2015 2

Let .

Geometric intuition for the solution sets of

OR-1 2015 3

{𝑥 :𝑎’𝑥 0 }

{𝑥 :𝑎′ 𝑥≤ 0 }

Geometry in 2-D

𝑎

{𝑥 :𝑎′ 𝑥=0 }

0

OR-1 2015 4

Let be a (any) point satisfying . Then

Hence , where is any solution to , or .

So can be obtained by adding to every point satisfying . Sim-ilarly, for , and .

{𝑥 :𝑎′ 𝑥≥𝑏}

{𝑥 :𝑎′ 𝑥≤𝑏}

𝑎

{𝑥 :𝑎′ 𝑥=0 }

0{𝑥 :𝑎′ 𝑥=𝑏}

𝑧

OR-1 2015 5

Points satisfying (halfspace)

0)',)(3,4(

034

21

21

xx

xx

(4,3)

x2

x1

1034 21 xx

0)',)(3,4(

034

21

21

xx

xx

(2,2/3)

OR-1 2015 6

Def: The set of points which can be described in the form is called a polyhedron. ( Intersection of finite number of half-spaces)

Hence, linear programming is the problem of optimizing (maxi-mize, minimize) a linear function over a polyhedron.

Thm: Polyhedron is a convex set.

Pf) HW earlier.

To understand the simplex method, we will consider geometric intuition and algebraic logic together throughout the lectures.

OR-1 2015 7

Solving LP graphically

(2,0)

(2,2/3)

(1,2)(0,2)

x2

x1

02 21 xx

22 21 xx

21 2 max xx

0,

1034

2

2 t.s.

21

21

2

1

xx

xx

x

x

42 21 xx

52 21 xx

OR-1 2015 8

Properties of optimal solutions Thm: If LP has a unique optimal solution, the unique optimal solu-

tion is an extreme point.

Pf) Suppose is a unique optimal solution and it is not an extreme point of the feasible set. Then there exist feasible points such that for some .

Then .

If , then either or , hence contradiction to being an optimal so-lution.

If , is also optimal solution. Contradiction to being unique opti-mal solution.

Thm: Suppose polyhedron has at least one extreme point. If LP over has an optimal solution, it has an extreme point optimal solu-tion.

Pf) not given here.

Above theorem indicates that we need to examine extreme points only to find an optimal solution of an LP.

OR-1 2015 9

Multiple optimal solutions

21 2 max xx

0,

1034

2

2 t.s.

21

21

2

1

xx

xx

x

x

(2,0)

(2,2/3)

(1,2)(0,2)

x2

x1

52 21 xx

21 34 max xx

1034 21 xx

OR-1 2015 10

Obtaining extreme point algebraically

21 2 max xx

0,

1034

2

2 t.s.

21

21

2

1

xx

xx

x

x

(2,0)

(2,2/3)

(1,2)(0,2)

x2

x1

Extreme point (0, 2) can be identified by solving system of eq. Similarly, (1, 2) can be identified by .Note that gives (5/2, 0), which is not an extreme point since it is not in thepolyhedron.

Thm: Suppose polyhedron is given and has at least one extreme point (the space is -dimensional).

Let be the index set of constraints.

Let and be the set of constraints which hold at equality by , i.e., , and .

Then is an extreme point of if and only if the rank of is , where is the sub-matrix of obtained by choosing the rows of .

Pf) not given here. Using the above theorem, an extreme point of a polyhedron can be identified

by setting some of the inequalities as equalities and obtaining the solution satisfying the equalities (coefficient vectors (chosen rows of matrix) must be linearly independent so that the system gives a unique solution.).

If the obtained point satisfies other inequalities, it is in and it is an extreme point of the polyhedron. If the obtained point is not in , it is not an extreme point. See the earlier example.

OR-1 2015 11

OR-1 2015 12

Note: If is given as , where , the rows of matrix (linearly inde-pendent row vectors) should be included in equalities to iden-tify an extreme point.

Above result needs rigorous proof, but we use it in the class without proof since the proof is quite involved and the result it-self is crucial in understanding the behavior of the simplex method.

From the previous theorems, we know that we only need to con-sider extreme points of the polyhedron to find an optimal solu-tion.

The previous theorem gives a means of how to identify an ex-treme point algebraically.

Idea of algorithm?

Enumeration of all extreme points? :

maximum of choices for standard LP () ( , ( constraints and variables), which is quite large.

( the number of ways to choose inequalities (which hold at equalities) out of inequalities.)

Algorithm strategy : from an extreme point, move to the neighboring extreme point which gives a better (precisely speaking, not worse) solution in each it-eration of the algorithm.

OR-1 2015 13

OR-1 2015 14

Remark: There exists a polyhedron which is not full-dimen-

sional. (extreme point is defined same as before.)

0

0

0

1

3

2

1

321

x

x

x

xxx

x2

x1

x3

1

1

1This polyhedron is 2-dimensional.

OR-1 2015 15

Algebraic Derivation of Extreme Points for LP

Any LP problem must be converted to a problem having only equa-tions and no inequalities except the nonnegativity constraints if simplex method can be applied (details later)

Consider the LP problem max (we solve this form throughout)

, full row rank ()

An extreme point of can be obtained as the solution satisfying for some , and the coefficient matrix is nonsingular. We choose non-singular matrix which includes matrix as submatrix (note that the number of equations can be > ). Let be the index set such that holds (there are of them).

OR-1 2015 16

(continued)

Let , , and nonsingular, , where is the submatrix of having col-umns associated with variables set to 0 (.

Then an extreme point can be found by solving

. . (or )

multiplying on both sides, we obtain .

or ()

Solution is

This is the basic solution we mentioned earlier (refer note-2, p22, p29). By the choice of the variables we set at 0, we obtain differ-ent basic solutions.

are called basic variables, and are called nonbasic variables.

If the obtained solution satisfies nonnegativity, , we have a basic and feasible solution (satisfies nonnegativity of variables). So this point is in the polyhedron and hence an extreme point.

OR-1 2015 17

Ex: extreme point ( 1, 0, 0 ) can be obtained from , , . Since ( 1, 0, 0 ) satis-fies , it is an extreme point.

0

0

0

1

3

2

1

321

x

x

x

xxx

x2

x1

x3

1

1

1

Why do we need nonsingular?

Coefficient matrix (which is nonsingular) defining an extreme point ( is in-cluded)

OR-1 2015 18

=

𝑏

0

𝐵 𝑁 𝑥𝐵

𝑥𝑁− 𝐼0

𝐴𝑥=𝑏

−𝑥𝑁=0

𝑚 𝑛−𝑚

𝑚

𝑛−𝑚

Suppose , then (determinant of ) is given as

for any

for any ,

where .

is an square submatrix of obtained by deleting -th row and -th column of (called minor of ).

is called the cofactor of . Also is nonsingular if and only if . square matrix is nonsingular has a unique solution.

Hence the condition that the system in the previous slide has a unique solu-tion is equivalent to the condition that the coefficient matrix is nonsingular determinant of the coefficient matrix

determinant of matrix is nonsingular.

Hence we need the condition that B matrix is nonsingular when we define an extreme point, in which case the corresponding solution is a basic feasible solution.

OR-1 2015 19

Simplex method searches only basic feasible solutions, which is tantamount to searching the extreme points of the corresponding polyhedron until it finds an optimal solution.

OR-1 2015 20