basics on linear programmingerodri/webpage/cps/theory/lp... · 2021. 4. 6. · theorem (fundamental...

Basics on Linear Programming

Combinatorial Problem Solving (CPS)

Javier Larrosa Albert Oliveras Enric Rodrıguez-Carbonell

April 6, 2021

Linear Programs (LP’s)

2 / 22

■ A linear program is an optimization problem of the form

min cTx

A1x ≤ b1A2x = b2A3x ≥ b3x ∈ R

n

c ∈ Rn, bi ∈ R

mi , Ai ∈ Rmi×n, i = 1, 2, 3

■ x is the vector of variables

■ cTx is the cost or objective function

■ A1x ≤ b1, A2x = b2 and A3x ≥ b3 are the constraints

Linear Programs (LP’s)

2 / 22

■ A linear program is an optimization problem of the form

min cTx

A1x ≤ b1A2x = b2A3x ≥ b3x ∈ R

n

c ∈ Rn, bi ∈ R

mi , Ai ∈ Rmi×n, i = 1, 2, 3

■ x is the vector of variables

■ cTx is the cost or objective function

■ A1x ≤ b1, A2x = b2 and A3x ≥ b3 are the constraints

■ Example:min x+ y + z

x+ y = 30 ≤ x ≤ 20 ≤ y ≤ 2

Notes on the Definition of LP

3 / 22

■ Solving minimization or maximization is equivalent:

max{ f(x) | x ∈ S } = −min{ −f(x) | x ∈ S }

■ Satisfiability problems are a particular case:take arbitrary cost function, e.g., c = 0

Equivalent Forms of LP’s

4 / 22

■ This form is not the most convenient for algorithms

■ We will assume problems to be in canonical form:

min cTx

Ax = b

x ≥ 0

c ∈ Rn, b ∈ R

m, A ∈ Rm×n, n ≥ m, rank(A) = m

■ Often variables are identified with columns of the matrix,and constraints are identified with rows

Methods for Solving LP’s

5 / 22

■ Simplex algorithms

■ Interior-point algorithms

Basic Definitions (1)

6 / 22

min cTxAx = b

x ≥ 0

■ Any vector x such that Ax = b is called a solution

■ A solution x satisfying x ≥ 0 is called a feasible solution

■ An LP with feasible solutions is called feasible;otherwise it is called infeasible

■ A feasible solution x∗ is called optimalif cTx∗ ≤ cTx for all feasible solution x

■ A feasible LP with no optimal solution is unbounded


7 / 22

maxx+ 2y

x+ y + s1 = 3x+ s2 = 2y + s3 = 2x, y, s1, s2, s3 ≥ 0

■ (x, y, s1, s2, s3) = (−1,−1, 5, 3, 3) is a solution but not feasible

■ (x, y, s1, s2, s3) = (1, 1, 1, 1, 1) is a feasible solution


8 / 22

maxx+ βy

x+ y + s1 = α

x+ s2 = 2y + s3 = 2x, y, s1, s2, s3 ≥ 0

■ If α = −1 the LP is not feasible

■ If α = 3, β = 2 then(x, y, s1, s2, s3) = (1, 2, 0, 1, 0) is the (only) optimal solution


8 / 22

maxx+ βy

x+ y + s1 = α

x+ s2 = 2y + s3 = 2x, y, s1, s2, s3 ≥ 0

■ If α = −1 the LP is not feasible

■ If α = 3, β = 2 then(x, y, s1, s2, s3) = (1, 2, 0, 1, 0) is the (only) optimal solution

■ There may be more than one optimal solution:If α = 3 and β = 1 then{(1, 2, 0, 1, 0), (2, 1, 0, 0, 1), (3

2, 32, 0, 1

2, 12)} are optimal


9 / 22

max x + y

x

y

x ≤ 2

x + y ≤ 3

maxx + 2y

y ≤ 2

y ≥ 0

x ≥ 0(2, 1)

(1, 2)


10 / 22

minx+ 2y

x+ y ≤ 30 ≤ x ≤ 2y ≤ 2

Unbounded LPx

x ≤ 2

y ≤ 2

x ≥ 0

x + y ≤ 3

y

min x + 2y


11 / 22

maxx+ 2y

x+ y ≤ 30 ≤ x ≤ 2y ≤ 2

LP is bounded,but set of feasiblesolutions is not x

y

x ≤ 2

maxx + 2y

y ≤ 2

x ≥ 0

(1, 2)x + y ≤ 3

Bases (1)

12 / 22

Let us denote by a1, ..., an the columns of A

Recall that n ≥ m, rank(A) = m.

■ A matrix of m columns (ak1 , ..., akm) is a basisif the columns are linearly independent

■ Note that a basis is a square matrix!

■ If (ak1 , ..., akm) is a basis,then the variables (xk1 , ..., xkm) are called basic

■ We usually denote

by B the list of indices (k1, ..., km), andby R the list of indices (1, 2, ..., n)− B; and

by B the matrix (ai | i ∈ B), andby R the matrix (ai | i ∈ R)

xB the basic variables, xR the non-basic ones

Bases (2)

13 / 22

maxx+ 2yx+ y + s1 = 3x+ s2 = 2y + s3 = 2x, y, s1, s2, s3 ≥ 0

x y s1 s2 s3

A =

1 1 1 0 01 0 0 1 00 1 0 0 1

■ (x, s1, s2) do not form a basis:

1 1 01 0 10 0 0

does not have linearly independent columns

■ (s1, s2, s3) form a basis, where xB = (s1, s2, s3), xR = (x, y)

B =

1 0 00 1 00 0 1

R =

1 11 00 1

Bases (3)

14 / 22

■ If B is a basis, then the following holds

BxB +RxR = b

Hence:

xB = B−1b−B−1RxR

Non-basic variables determine values of basic ones

■ If non-basic variables are set to 0, we get the solution

xR = 0, xB = B−1b

Such a solution is called a basic solution

■ If a basic solution satisfies xB ≥ 0 then it is called abasic feasible solution, and the basis is feasible

Bases (4)

15 / 22

Consider basis (s1, s2, s3)


B =

1 0 00 1 00 0 1

R =

1 11 00 1

Equations xB = B−1b−B−1RxR are

s1 = 3− x− y

s2 = 2− x

s3 = 2− y

Basic solution is

σB =

322

σR =

(

00

)

So basis (s1, s2, s3) is feasible

Bases (5)

16 / 22

Basis (x, y, s1) is not feasible


B =

1 1 11 0 00 1 0

R =

0 01 00 1

x = 2− s2y = 2− s3s1 = −1 + s2 + s3

σB =

22−1

σR =

(

00

)

Bases (6)

17 / 22

A basis is called degeneratewhen at least one component of its basic solution xB is null

For example:


B =

1 1 01 0 10 1 0

R =

1 00 00 1

x = 2 + s3 − s1y = 2− s3s2 = s1 − s3

σB =

220

Geometry of LP’s (1)

18 / 22

■ Set of feasible solutions of an LP is a convex polyhedron

■ Basic feasible solutions are vertices of the convex polyhedron


19 / 22

■


■ xB1= (y, s1, s2)

■ xB2= (x, y, s2)

■ xB3= (x, y, s3)

■ xB4= (x, s1, s3)

■ xB5= (s1, s2, s3)

x ≤ 2x ≤ 2

x

y

x ≤ 2

y ≤ 2

(2, 0)(0, 0)

(2, 1)

(1, 2)(0, 2)

x + y ≤ 3

y ≥ 0

x ≥ 0

2

3

5 4

1


20 / 22

■ Theorem (Minkowski-Weyl)

Let P be a feasible LP.

There exist basic feasible solutions v1, ..., vr ∈ Rn and vectors

r1, ..., rs ∈ Rn such that a point x is a feasible solution to P iff

x =r

∑

i=1

λivi +s

∑

j=1

µjrj

for certain λi, µj such that∑r

i=1λi = 1 and λi, µj ≥ 0.

Possible Outcomes of an LP (1)

21 / 22

■ Theorem (Fundamental Theorem of Linear Programming)

Let P be an LP.

Then exactly one of the following holds:

1. P is infeasible

2. P is unbounded

3. P has an optimal basic feasible solution

It is sufficient to investigate basic feasible solutions!


22 / 22

Proof: Assume P feasible and with optimal solution x∗.

Let us see we can find a basic feasible solution as good as x∗.

By Minkowski-Weyl theorem, we can write

x∗ =∑r

i=1λ∗i vi +

∑sj=1

µ∗jrj

where∑r

i=1λ∗i = 1 and λ∗

i , µ∗j ≥ 0. Then

cTx∗ =∑r

i=1λ∗i c

T vi +∑s

j=1µ∗jc

T rj


22 / 22




x∗ =∑r

i=1λ∗i vi +

∑sj=1

µ∗jrj

where∑r



cTx∗ =∑r

i=1λ∗i c

T vi +∑s

j=1µ∗jc

T rj

■ If there is j such that cT rj < 0 then objective valuecan be decreased by taking µ∗

j larger. Contradiction!


22 / 22




x∗ =∑r

i=1λ∗i vi +

∑sj=1

µ∗jrj

where∑r



cTx∗ =∑r

i=1λ∗i c

T vi +∑s

j=1µ∗jc

T rj



■ Otherwise cT rj ≥ 0 for all j. Assume cTx∗ < cT vi for all i.

cTx∗ ≥∑r

i=1λ∗i c

T vi >∑r

i=1λ∗i c

Tx∗ = cTx∗∑r

i=1λ∗i = cTx∗


22 / 22




x∗ =∑r

i=1λ∗i vi +

∑sj=1

µ∗jrj

where∑r



cTx∗ =∑r

i=1λ∗i c

T vi +∑s

j=1µ∗jc

T rj



■ Otherwise cT rj ≥ 0 for all j. Assume cTx∗ < cT vi for all i.

cTx∗ ≥∑r

i=1λ∗i c

T vi >∑r

i=1λ∗i c

Tx∗ = cTx∗∑r

i=1λ∗i = cTx∗

Contradiction! Thus there is i such that cTx∗ ≥ cT vi;in fact, cTx∗ = cT vi by the optimality of x∗.

basics on linear programmingerodri/webpage/cps/theory/lp... · 2021. 4. 6. · theorem (fundamental...

Documents