coursopt-07-08

8/18/2019 CoursOPT-07-08

http://slidepdf.com/reader/full/coursopt-07-08 1/68

OPTIMIZ TION

COURSE NOTES

By

Prof. Atidel B. Hadj-Alouane

2007-2008

8/18/2019 CoursOPT-07-08


OPTIMIZATION 3A GI

2

CONTENTS

Chapter 1 Introduction to Network Programming ................................................................... ......................... 3

1.1 Introduction.......................................................................... ..................................................................... ......... 3

1.2 Definitions and basic concepts........ ............................................................................ ....................................... 4

1.3 Incidence and adjacency ........................................................................ ............................................................ 7

Chapter 2 The shortest path problem and its extensions..................................... .............................................. 9

2.1 Introduction.......................................................................... ..................................................................... ......... 9

2.2 Applications.............................................................. ....................................................................... ................... 9

2.3 The principle of optimality ..................................................................... .......................................................... 11

2.4 Shortest path algorithms .................................................................... .............................................................. 12

Chapter 3 Network Flow problems............................................................... ..................................................... 20

3.1 Introduction.......................................................................... ................................................................... ......... 20

3.2 Definitions .................................................................. ................................................................... ................... 20

3.3 The maximum value flow problem....................................................... ............................................................. 21

3.4 Concept of cuts ............................................................... ............................................................... ................... 27

3.5 Case of networks having lower bounds ......................................................................... ................................... 28

3.6 Minimum cost flow problems ................................................................................................................... ........ 30

Chapter 4 Dynamic Programming......................................................... ............................................................ 42

4.1 Introduction.......................................................................... ................................................................... ......... 42

4.2 Dynamic Programming formulations............................ ........................................................................... ........ 43

4.3 Dynamic programming complexity .................................................................................................................. 48 4.4 Characteristics of dynamic programming applications ............................................................................ ....... 49

4.5 Production and inventory control ............................................................................................................ ........ 50

4.6 Equipment replacement.................... ..................................................................... ........................................... 57

4.7 Equipment reliability............ ................................................................ ............................................................ 61

4.8 Summary of dynamic programming general aspects................................................................................. ....... 64

8/18/2019 CoursOPT-07-08


OPTIMIZATION 3A GI

3

Chapter 1 Introduction to Network Programming

1.1 Introduction

Network programming is a field of mathematics, which constitutes one of the most

used instruments for solving a wide class of discrete optimization problems in the

domain of operations research. A network allows for representing, in a simple way, a

variety of problems. Generally, network programming is a tool for modeling problems

involving connections or transshipment in complex systems, through the expression of

the relations that exist between its components. Examples of applications include

transportation networks and communication networks.

Networks can also be used to model problems which do not appear to have a link to

physical networks, such as project scheduling, where nodes of a the networks

represent activities and the arcs represent the precedence relations between the

activities. Other examples include production and inventory planning, resource

management and financial planning.

We distinguish 5 major classes of network problems:

1. Shortest path (or chain),2. Minimum cost spanning tree,3. Maximum value flow,4. Minimum cost flow,5. Project scheduling: CPM and PERT.

Example:

Consider an amusement park with O as the entrance and T the tramway station.Pb 1: shortest chain problem to go from O to T.

Pb 2: minmum spanning tree: design an electrical network using minimal cable

length.

Pb 3: maximum flow: transport the maximum of tramways from O to T.

8/18/2019 CoursOPT-07-08


OPTIMIZATION 3A GI

4

Figure 1.1: Examples

1.2 Definitions and basic concepts

1.2.1 Networks

A network, G, is defined by:

A set N of points, also called vertices or nodes ;

A set A ⊂ N ×N of lines, called arcs, each joining a pair of nodes together with

some associated data. Nodes i and j are said to be adjacent if there is a line

joining them. Two lines are said to be adjacent if they share at least one

node.

The network is denoted G=(N, A).

Let N= n: the number of nodes in G ; Then G is said to be of order n.

1.2.2 Ar cs and dir ect ed net wor ks

A line joining i and j that can only be used in the direction from i to j is called an arc,

and denoted by the ordered pair (i, j). It is incident out of i and into j. The arc (i,j)

has i as its tail and j as its head. If the arc is denoted by u=(i,j), then tail(u)=i and

head(u)=j. Graphically, the arc u is represented by an arrow:

The node i is said to be a precedent or a predecessor of j ; and j is called a successor

of i. The set of successors of i is denoted by Γ(i) ; The set of predecessors of i is

denoted by Γ-1(i).

Rq: The arc (i,i) is called a loop.

1

4

2

O

A

B

C

D

E

T

7

2

5

4

4

1

5

7

3

i ju

8/18/2019 CoursOPT-07-08


OPTIMIZATION 3A GI

5

A line joining i and j that can be used either from i to j or from j to i is called an edge

and is denoted by the unordered pair (i;j). It is incident at i and j.

A network is said to be directed if it contains only arcs. It is undirected if all its lines

are edges.

The degree of a node i d(i) is the number of lines (arcs or edges) incident at it. In a

directed network, the indegree d-(i) (outdegree d+(i) ) of a node is the number of arcs

incident into (out of) it. We have then: d(i) = d+(i) + d-(i)

Rq: d-(i) = |Γ-1(i)| and d+(i) = |Γ(i)|.

A p-Network is a network in which there exist at most p lines between any pair of

nodes.

For a 1-network, G can be perfectly determined by the set of nodes and the set of

successors of each node.

1. 2.3 Chai ns and Pat hs

A chain linking two nodes i1 (origin) and ik (destination) in G is a sequence of nodes

and lines alternately i1, u1, i2, u2, …, uk-1, ik, such that for each r=1 to k-1, ur is either

the arc (ir, ir+1) or the arc (ir+1, ir), or the edge (ir ; ir+1), with some orientation

selected for it. In the network represented in Figure 1.2, the arcs (A, B), (B, D) and

(E, D) constitute a chain from A to E.

An arc whose orientation coincides with (is opposite to) the direction of travel from

origin to destination is called a forward (reverse) arc of the chain.

A path is a chain which all the arcs are forward arcs. In Figure 1.2, the arcs (C, A), (A,

B) and (B, D) constitute a path from C to D.

8/18/2019 CoursOPT-07-08


OPTIMIZATION 3A GI

6

Figure 1.2 : Example of chains and paths

The length (or cardinality) of a chain or a path is the number of its arcs.

Rq: It is important to note that, in some references, the definitions of path and chain

are reversed.

1.2. 4 Cycles and Cir cuit s

A cycle (circuit) is a chain (path) whose end nodes coincide.

In Figure 1.2, the arcs (C, B), (B, D), (E, D) et (C, E) form a cycle; and the arcs (A, B),

(B, C) et (C, A) form a circuit.

1.2. 5 Connect ed nodes and net wor ks

Two nodes are said to be connected if the corresponding network contains at least

one chain linking them.

A network G is said to be connected (strongly connected) if there exists a chain (a

path) between every pair of nodes in it.

Figure 1.3: Example of a non connected network

1.2.6 Subnetw orks and part ia l net wor ks

A subnetwork of G = (N, A) is a network F = (N, A ) with the same set of nodes, but

with A ⊂ A. For example, If G is the network representing all the routes in Tunisia,

the network representing all the routes in the north of Tunisia is a subnetwork.

A

B

C

D

E

A

B

C

D

E

8/18/2019 CoursOPT-07-08


OPTIMIZATION 3A GI

7

1.3 Incidence and adjacency

To describe a network, a number of representations can be used. They are not all

equivalent with respect to algorithms’ efficiency.

There exist essentially two families of representations:

• Incidence matrices,

• Adjacency matrices.

1.3.1 Node-arc incidence mat r i x

The node-arc incidence matrix of a network G = (N, A) is the matrix E = (e ij), i = 1, …,

n et j = 1,…, m=|A|, that has a row associated with each node in G, and a column

associated with each arc in G, and which coefficients are equal to 0, 1 or –1.

If u = (i, j) ∈ A, the column u has all its coefficients equal to zero except: e iu = +1 and

eju = -1.

Example:

The node-arc incidence matrix of the following network is given in Figure 1.4.

+1 +1 0 0 0

-1 0 +1 +1 0

0 -1 -1 0 +1

0 0 0 -1 -1

Figure 1.4: Example of node-arc incidence matrix

1.3. 2 Node-edge incidence mat r ix

The node-edge incidence matrix of a network G = (N, A) is the matrix E = (eij), i = 1,

…, n et j = 1,…, m, that has a row associated with each node in G, and a column

associated with each edge in G, and which cofficients are equal to 0 or 1.

If u = (i, j) ∈ A, the column u has all its coefficients equal to zero except: e iu = +1 and

eju = +1.

u4 u2

u1 1

4

2

3

u3

u5

8/18/2019 CoursOPT-07-08


OPTIMIZATION 3A GI

8

Example:

The node-edge incidence matrix of the following network is:

1 1 0 0 0

1 0 1 1 0

0 1 1 0 1

0 0 0 1 1

Figure 1.5: Example of node-edge incidence matrix

1.3.3 Node-Node adj acency matr ix

The node-node incidence matrix of a network G = (N, A) is the matrix E = (e ij),

i = 1, …, n and j = 1,…, n, that has one row and one column associated with each node

in G, and which coefficients are equal to 0 or 1.

In the case of an undirected network, we can define two arcs: (i,j) and (j,i) for each

edge (i ;j). In this case, the adjacency matrix is symmetric.

Example:

The node-node incidence matrix of the following network is:

0 1 1 0

0 0 1 1

0 0 0 1

0 0 0 0

Figure 1.6: Example of node-node incidence matrix

u4 u2

u1 1

4

2

3

u3

u5

u4 u2

u1 1

4

2

3

u3

u5

8/18/2019 CoursOPT-07-08


OPTIMIZATION 3A GI

9

Chapter 2 The shortest path problem and its extensions

2.1 Introduction

Consider a directed network G = (N, A), with n nodes and m arcs. To each arc is

associated a real number cu called the length or weight of the arc u. If u = (i, j), the

arc length is denoted by cij. We define the length of a path as the sum of the lengths

of all its arcs.

The shortest path problem (SPP) consists of determining a path joining two given

nodes: an origin, or source S, and a destination, or sink T, with a minimal length.

The arc lengths may correspond to:

• distances,

• transportation costs,

• costs of constructing the arcs,

• travel times, …

The arc lengths can of arbitrary signs (+ or -). As a consequence, a path that contains

a circuit that has a strictly negative length is unbounded (as the cycle may be visited

an infinite number of times). A necessary condition for the existence of a shortest

path is the absence of circuits with negative lengths.

2.2 Applications

2.2.1 Equipment r eplacement pr oblems

A mine having an exploitation horizon of T years uses specific expensive equipment.

At the first year, i = 0, a new equipment has to be purchased. At the beginning of

each year, there are two options: either keep the equipment during the year [i, i+1],

or trade it by reselling it for its salvage value v(x) where x est is the age of the

equipment (number of year it has been used), and buying a new equipment with a

cost of p(i). At the last year, T, the equipment in service has to be sold for its salvage

value. The annual operational cost, r(x), depends on the age of the equipment. The

8/18/2019 CoursOPT-07-08


OPTIMIZATION 3A GI

10

values of p(i), v(x) et r(x) are supposed to be discounted to their present value at

year i = 0.

The problem consists of finding a minimum cost equipment replacement.

We can show that this problem is equivalent to the shortest path problem in a

particular network. Let us define the network G = (N, A) as follows (see figure 2.1):

A node is associated with each date, giving a total of T+1 nodes.

An arc joins the nodes i and j if and only if i < j. This arc corresponds to the decision

of bying an equipment at year i and replacing it at year j.

The length of each arc is the sum of purchasing and operational costs, from which theselling price is subtracted:

cij = p(i) + ∑s=1,j-i r(s) - v(j-i)

Figure 2.1: Network associated with the equipment replacement problem

An optimal policy is given by the shortest path from 0 to T, in the constructed

network.

2.2.2 Replenishment opt imi sat ion

A company has to ensure the supply of a certain material during a horizon of T periods

in order to be able to execute its production plan. For each period t (t = 1, …, T) the

purchase cost p(t) is given.

In order to take advantage of price variabilities, the company may constitute a stock

of unused items for a cost of h(t) per unit during each period t. The company may

also backorder customer demands in the case of a shortage. However, in this case, a

1 20 T

8/18/2019 CoursOPT-07-08


OPTIMIZATION 3A GI

11

penalty, µ per unit per period, has to be paid. We consider the problem of finding the

minimum cost supply (or replenishment) plan.

We can show that this problem is equivalent to the shortest path problem in a

particular network. Let us define the network G = (N, A) as follows (see figure 2.2):

N contains one particular node S and T nodes corresponding to each period; therfore,

a total of T+1 nodes.

The node S is linked to each node t: the arc (S, t) is associated with the decision of

supplying the material at period t. The length of such arc is p(t).

Each arc t (t = 1,…, T-1) is linked to the node t+1. An arc (t, t+1) corresponds to the

decision of storing during period t. The length of such arc is h(t).Each arc t (t = 2,…, T) is linked to the node t-1. An arc (t, t-1) corresponds to the

decision of backorder during period t. The length of such arc is µ.

Figure 2.2: Network associated with the replenishment optimisation problem

A path between S and t corresponds to a policy of supplying the material of period t.Hence, an optimal supply policy can be found by determining the the shortest paths

between S and all the nodes of the constructed network.

2.3 The principle of optimality

This principle is due to R. Bellman, and allows for characterizing an optimal path

(shortest or longest).

Theorem 2.1

Any optimal path is composed by optimal sub-paths.

p(1) p(2) p(3)

p(T)

h(1) h(2)

µ µ µ

2 31 T

S

8/18/2019 CoursOPT-07-08


OPTIMIZATION 3A GI

12

Proof:

Let C = (i1, i2, …, ip-1, ip) be a shortest path from node i1 to ip. We will show that the

the sub-path C’ = (i1, i2, …, ip-1) is an optimal path from i1 to ip-1.

If there exists another path C’’ linking the two nodes i1 and ip-1 and shorter than C’,

then the path obtained by adding the arc (ip-1, ip) to C’’ is shorter than C. This is a

contradiction with a fact that C is optimal. By induction over all the nodes in C, one

can show that all sub-paths in C are optimal.

Let N = {1, 2, …, n} and define for each node j a label πj equal to the length of the

shortest path from 1 to j. One immediate consequence of theorem 2.1 is the Bellman

equations:

π1 = 0

πj = mink≠j (πk + ckj) j = 2, …, n.

Example:

For the network in 2.3, π5 = min {π2 + c25 ; π3 + c35 ; π4 + c45}

Figure 2.3: Illustration of the principle of optimality of Bellman

Solving these equations for the case of a network without a negative circuit, gives theshortest path between the node 1 and all other nodes of the network. The algorithms

presented next allows for solving these equations.

2.4 Shortest path algorithms

There are different algorithms depending on the nature of the network:

• Nonnegative length (or costs),

•

Lengths of arbitrary signs,• Network without negative cycles

1

2

4

3

5

8/18/2019 CoursOPT-07-08


OPTIMIZATION 3A GI

13

and the problem at hand:

• Finding the shortest path from one node to another node,

• Finding the shortest paths from one node to all other nodes,

•

Finding the shortest paths between every couple of nodes.

All such algorithms are based on the principle of optimality of Bellman.

2.4.1 Netw orks wi t h nonnegat ive lengt hs: Dij kst ra ’ s algori t hm (1959)

Dijkstra’s algorithm allows for finding shortest paths from one node (node 1) to each

other node of any network having nonnegative arc lengths.

A label is assigned to each node. The nodes are portioned into two subsets: P and T:

• P is the set of nodes each having a permanent label,

• T is the set of nodes each having a temporary label.

A permanent label represents the length of the shortest path from node 1 to the

considered node and a temporary label represents an upper bound on that length.

For a network having n nodes, the algorithm executes n-1 iterations: at each

iteration, one node is ransferred from T to P, and the temporary labels are updated.

Dijkstra’s algorithm (1959)

Step 1 : Initialization :

Define γ (j) as the precedent of node j in the shortest path from 1 to j.

π1 = 0∀ j ≠ 1 πj = c1j and γ (j) = 1 if (1, j) ∈ N

πj = +∞ and γ (j) = - otherwise.

P = {1}, T = {2, …, n}

Step 2 : label fixing

Find i ∈ T, such that πi = minj∈T {πj}

Set: T = T\{i}P = P ∪ {i}

8/18/2019 CoursOPT-07-08


OPTIMIZATION 3A GI

14

If T = ∅ then terminate.

Step 3 : Update of temporary labels

Set πj = min {πj, πi + cij} ∀ j∈T such that (i,j) ∈ N;

if πj changes, update γ (j) = i

Go to Step 2

Example: A transportation company has to distribute its client’s merchandise located

at node 1 to a number of sales points numbered from 2 to 6. The travel time between

each couple of nodes and the direction of travel are given by the following network:

Figure 2.4: Example

It is required to find the shortest path between node 1 and each of the sales points.

Itération

1 2 3 4 5 6

1 (0,1) (7,1) (1,1) +∞ +∞ +∞

2 - (6,3) - +∞ (3,3) (5,3)

3 - (5,5) - (8,5) - (5,3)

4 - - - (8,5) - (5,3)

5 - - - (8,5) - -

Step 1 : π1 = 0 π2 = 7 π3 = 1 π4 = π5 = π6 = ∞, P = {1}, T = { 2, 3, 4, 5, 6}

γ (2) = γ (3)= 1

2

5

1

1

72

3

5

6

4

4

1

4

2

5

3

8/18/2019 CoursOPT-07-08


OPTIMIZATION 3A GI

15

Step 2 : i = 3, P = {1, 3}, T = {2, 4, 5, 6}

Step 3 : π2 = min {7, 1+5} = 6; γ (2) = 3 π5 = 3 π6 = 5; γ (5) = γ (6)= 3

Etc.

Dijkstra’s algorithm is based on label fixing principle: at each iteration, it selects one

node and permanently fixes its label πj.

2.4.2 Case of ar bit r ar y length signs: For d-Bel lman algori t hm (1956)

This algorithm is applicable for general networks, having arbitrary signs of arc

lengths. In the absence of circuits of negative lengths, it finds the shortest path

from one node to all others by successively adjusting the Bellman equations.Otherwise, it detects the presence of a negative circuit.

To each node j is associated a label πj(k) computed at iteration k :

Ford-Bellman algorithm (1956)

Step 1 : Initialization

π1

(1)

= 0∀ j ≠ 1 πj

(1) = c1j (and set γ (j) = 1) if (1, j) ∈ U

= +∞ otherwise.

k=1

Step 2 : Label update

For all j compute:

πj

(k+1)

= min {πj

(k)

, mini≠j {πi

(k)

+ cij}} ; update γ (j) if improvement occurs.

Step 3 : Test

If for all j πj(k+1) = πj

(k), terminate

if k≤n-1, Go to Step 2 with k=k+1

If k=n, There exists a negative circuit; terminate.

This algorithm converges in at most (n-1) iterations.

8/18/2019 CoursOPT-07-08


OPTIMIZATION 3A GI

16

Note: An improvement of this algorithm can be achieved by using the results of the

current iteration (i.e., using the updated labels of iteration k in the Bellman

equations).

Example: A salesman currently at city O decides to participate in an international fair

organized in city T. To get to his destination, he has to use the network presented in

figure 2.5. He estimates the ransportation cost as well as the benefits he can earn by

selling his products at different intermediate cities along his route. In order to

minimize his expenses, the salesman seeks the minimal cost route from city O to city

T. Costs are benefits (negative data) are shown on the arcs.

Figure 2.5 : Example

The shortest path from city O to city T has to be found. We have to use Ford-Bellman

algorithm because of the negative lengths:

Itération

O A B C D E T

1 (0,O) (2,O) (-5,O) (4,O) +∞ +∞ +∞ 2 (0,O) (2,O) (-6,A) (4,O) (-1,B) (-2,B) +∞

3 (0,O) (2,O) (-6,A) (4,O) (-3,E) (-3,B) (4,D)

4 (0,O) (2,O) (-6,A) (4,O) (-4,E) (-3,B) (2,D)

5 (0,O) (2,O) (-6,A) (4,O) (-4,E) (-3,B) (1,D)

The shortest path from O to T is: O → A → B → E → D → T with a total cost of 1 unit.The algorithm converges in 6 iterations.

7

2

4

4

-1

5

73

1

4

-8

O

A

B

C

D

E

T-5

8/18/2019 CoursOPT-07-08


OPTIMIZATION 3A GI

17

Ford algorithm is based on label corrections. It can change the label of any node until

the last iteration.

2.4.3 Case of a netw ork wi t h no circuit

For a network without any circuit, we can use a simple algorithm that finds the

shortest paths in just one iteration.

Note:

Any network without circuits always has a particular node called the root, with no

predecessor (otherwise a circuit would exist).

A network without circuits always has a terminal node (without any successor),

otherwise a circuit would exist.

Finding the shortest path from one node to all others

Step 1:

For a network of order n, without circuits, assign numbers 1 to n to the nodes such

that for each arc (i, j) of the network: i < j. To achieve this, one may proceed as

follows:

Assign the number 1 to the root node. Mark the node number 1 and delete it from the

graph and delete all arcs incident out of it.

If all nodes are numbered, terminate,

Otherwise, in the new network, find the nodes without any predecessors and assign to

them the following numbers, and delete them from the network. Go to 2.

Step 2:

The labels are recursively computed:

π1 = 0

πj = mini≠j (πi + cij) j = 2, …, n.

Finding the shortest path from all nodes to one given node

Step 1:

For a network without circuits, or order n, number the nodes such that for each arc

(i, j) of the network : i > j. To achieve this, one may proceed as follows:

8/18/2019 CoursOPT-07-08


OPTIMIZATION 3A GI

18

Assign the number 1 to the terminal node. Mark the node number 1 and delete it from

the graph and delete all arcs incident into it.

If all nodes are numbered, terminate,

Otherwise, in the new network, find the nodes without any successors and assign to

them the following numbers, and delete them from the network. Go to 2.

Step 2:

The labels are recursively computed:

π1 = 0

πj = mini≠j (πi + cji) j = 2, …, n.

Example : Find the shortest path from O to T in the following network :

Figure 2.6: Example

Step 1 : Numbering

O : 1, A : 2, C : 3, B : 4, E : 5, D : 6, T : 7.

Step 2 : Compute πi :

πO = 0

πA = min (0 + 6) = 6/O

πC = min (0 + 9) = 9/O

πB = min (0 + 3 ; 6 + 4 ; 9 + 2) = 3/O

πE = min (3 + 6 ; 9 + 4) = 9/B

πD = min (6 + 3 ; 9 + 1) = 9/A

πT = min (6 + 10 ; 9 + 6 ; 9 + 4) = 13/E

Particularly, the shortest path is: (O, B), (B, E) and (E, T) with a cost of 13.

2

4

4

O

A

B

C

D

E

T

10

6

3

9

3

1

6

4

6

8/18/2019 CoursOPT-07-08


OPTIMIZATION 3A GI

19

Step 1 : Numbering

T : 1, D : 2, E : 3, B : 4, A : 5, C : 6, O : 7.

Step 2: Compute πi :

πT = 0

πD = min (0 + 6) = 6

πE = min (0 + 4 ; 1 + 6) = 4

πB = min (4 + 6) = 10

πA = min (0 + 10 ; 6 + 3 ; 10 + 4) = 9

πC = min (4 + 4 ; 10 + 2) = 8

πO = min (10 + 3 ; 9 + 6 ; 8 + 9) = 13

Particularly, the shortest path is: (O, B), (B, E) and (E, T) with a cost of 13.

Note: The numbering is not unique; for example, we could have inverted the numbers

of nodes A and C.

8/18/2019 CoursOPT-07-08


OPTIMIZATION 3A GI

20

Chapter 3 Network Flow problems

3.1 IntroductionNetwork flow problems concern the circulation of material on/through the arcs of a

given network. Historically, these problems have been studied to represent electrical

current circulating on a network of dipoles. Nowadays, there exist a large number of

real situations that can be modeled using the concept of network flow:

- Transportation of merchandise from different distribution sites to different

customers through urbain, railway, marine, airway networks, or in some casesa mix of ransportation modes,

-

flow of liquids inside pipelines,- flow of information through communication networks,- Inventory management,- scheduling, …

3.2 Definitions

Given a network G = (N,A), we associate with each arc u of A a positive integer

number ku called capacity of arc u.

The set N contains two special nodes:

One node having a zero indegree, called the source S of G,

One node having a zero outdegree, called the sink T of G.

All other nodes are called transit nodes.

One property of such networks is their ability to transport flow on the arcs. A f low

vector is defined as ϕ = [ϕ1, ϕ2, …, ϕm]T ∈ ℜm with m components, where ϕu is the

amount of flow traversing arc u.

0 ≤ ϕu ≤ ku, ∀ u = 1, …, m

The flow (vector) ϕ is feasible if 0 ≤ ϕu ≤ ku, ∀ u = 1, …, m and it satisfies the

following flow conservation equations:

∑u∈Γ(i) ϕu - ∑ u∈Γ-1

(i) ϕu = 0 ∀ i ∈ X

where Γ(i) : the set of arcs incident out of i,

8/18/2019 CoursOPT-07-08


OPTIMIZATION 3A GI

21

Γ-1(i): the set of arcs incident into,

The arc capacity may be interpreted as the maximal flow that can circulate through

the arc.

3.3 The maximum value flow problem

3.3.1 Pr oblem def i n i t ion

Consider a network G = (N, A, K). Let S and T be the source and the sink of G,

respectively. In order for S and T to verify the flow conservation constraint, one may

construct a network G0 by adding to A and arc (T, S) having an infinite capacity,

called a reverse flow and having a variable flow denoted by 0.

∑u∈Γ(S) ϕu = ∑ u∈Γ-1(T) ϕu = ϕ0 = v(ϕ)

The maximum value flow problem consists in determining a feasible flow ϕ between S

and T in G0 of a maximum value (max v(ϕ)).

3.3.2 Appl icat ions

a. Finding the maximum capacity of a roadway network

Consider a roadway network composed of m links (arcs). Each link is characterized by

a maximum number of vehicles that can traverse it per a unit of time.

The problem consists of finding the maximum number of vehicles that can travel

between two given points of the network during a time unit.

In this case, the similarity with the maximum value flow problem is obvious.

b. Preemtive scheduling of tasks on parallel machines

A set of n tasks have to be executed on m identical machines. Each task j is reduced

to a single operation of a duration pj, having an early start time rj and a deadline dj.

Given that it is possible to interrupt the execution of a task and restart it later on

another machine (preemption), the problem consists of determining a schedule that

respects the temporal constraints or to concluding that such a schedule does not

exist.

8/18/2019 CoursOPT-07-08


OPTIMIZATION 3A GI

22

Let E = ∪n

j 1=

{rj, dj}. Ranking the elements of E results in at most 2n-1 time intervals

[eh, eh+1]. Construct a network by defining:

-

A node associated with each task,- A node associated with each time interval [eh, eh+1],- Two additional nodes S and T linked, respectively, to all tasks and all time

intervals,- An arc associated with each possibility of execution of one task during a given

time interval. Thus, an arc links task j to time interval [eh, eh+1] if and only if itis possible to execute the task j during that interval. The following conditionhas to be satisfied: rj ≤ eh and eh+1 ≤ dj.

The capacity of an arc (S, j) is p j, the capacity of linking one task to a time interval is

(eh+1 - eh), and the capacity of an arc linking a node associated with a time interval to

T is m(eh+1 - eh).

A feasible schedule exists iff the maximum value flow between S and T has a value of

P = ∑j=1,n pj.

c. Maximum number of disjoint paths in a network

Given a network G = (N, A) and two nodes S and T. Two paths joining S and T are said

to be disjoint if they do not have any arc in common. The network is said to be p-

connex if there exist at least p paths joining every pair of nodes.

Finding the maximum number of disjoint paths between two nodes S and T is

equivalent to solving a maximum value flow problem. This is achieved by assigning a

capacitiy of 1 unit to each arc and finding the maximum value flow between S and T.

3.3.3 Algori t hm of For d-Fulkerson (1959)

This is one of the simplest algorithms for solving the the maximum value flow

problem. It uses the concept of labeling. It is an iterative algorithm that consists of

determining, at each iteration, a chain between S and T that allows for augmenting

the flow value while respecting the capacity constraints. Such a chain is called a Flow

Augmenting Chain (FC).

To identify a chain between S and T, a labeling algorithm is applied. This algorithmconsists in constructing a tree rooted at S and which nodes are all labeled (a tree is a

8/18/2019 CoursOPT-07-08


OPTIMIZATION 3A GI

23

partial subnetwork that contains no cycles). At the beginning, only S is labeled, and at

each iteration, one labeled node is selected in order to label all its sucessors. Every

time a node j is selected, it is included in the tree while noting its predecssor γ (i),

which is the node that was responsible for its labeling. The algorithm stops when:

• the node T is labeled or

• all successors of labeled nodes are examined without being able to label the

node T.

In the latter case, we conclude that there is no chain from S to T in the network.

Here we notice that nodes may be in three possible states: (1) unlabeled, (2) labeled

and unscanned, and (3) labeled and scanned. At each iteration, a labeled and

unscanned node is selected, in order to examine its neighboring nodes. Once this is

done, node i becomes scanned but remains labeled until T is labeled or the algorithm

stops without labeling T.

Given a network G = (N, A, K), de Ford-Fulkerson algorithm, for determining the

maximum value flow between S and T, is described as follows:

Algorithm of Ford-Fulkerson (1959) :

Step 0 : Find a feasible flow vecteur ϕ between S and T. One can always start with

the zero flow vector.

Step 1 : Plant a tree with root at S

Let ϕ be the current flow vector. Label the node S with (*). S is now labeled and

unscanned. Go to step 2.

Step 2 : Tree growth

Find a labeled and unscanned node i and do the following:

If there exists an unlabeled node j such that (i,j) ∈ A and ϕ(i,j) < k(i,j), label the node j

with (+i) : unsaturated forward arc, thus the flow can increase

If there exists an unlabeled node j such that (j,i) ∈ A and ϕ(j,i) > 0, label he node j

with (-i) : backward arc and the flow can decrease

8/18/2019 CoursOPT-07-08


OPTIMIZATION 3A GI

24

If such a pair does not exist, there is a nonbreakthrough. The current flow vector is a

maximum value flow vector; Terminate. Otherwise, node i is the immediate

predecessor of j in a chain starting from S.

If j = T, there is a breakthrough, go to step 3. Otherwise repreat this step 2.

Step 3 : Flow augmentation

Since T is labeled, an FC has just been found (it can be deduced by tracing back the

predecessors starting from T). The current flow can be augmented by the amount δ,

which is the minimum value of the residual capacity (capacity-current flow) on

forward arcs and the flow values on backward arcs.

{ }u u uu FC,forward u FC,backward

min min (k ); minδ ϕ ϕ ∈ ∈

= −

Change the current flow vector, ϕ, as follows:

• Increase by δ the flow on each forward arc of FC,

• decrease by δ the flow on each backward arc of FC.

Erase the labels of all nodes and go back to step 1 with the new flow vector.

Example: Isopipe Company wants to send the maximum quantity of natural gas (per

hour) through its pipelines, from node O to node T of the following network (Figure

4.1).

Figure 4.1: Example of natural gas transportation

It is required that the natural gas passes through certain stations: 1, 2, 3, 4 or 5. The

different arcs of the network represent pipelines of different diameters. The

maximum quantity of gas that can be pumped (per hour) through each arc is show on

4

2

1

1

S

4

3

1

5

2

T

1

2

3

2

2

1

5

3

2

8/18/2019 CoursOPT-07-08


OPTIMIZATION 3A GI

25

Figure 4.1. Determine the maximum quantity of gas that can be sent from O to T

through this network.

Application of Ford-Fulkerson algorithm:

Step 1 : Consider the following initial solution :

ϕS1 = ϕ13 = 2, ϕS3 = 3, ϕ34 = 1, ϕ4T = 1, ϕ35 = ϕ5T = 4

Label node S with (*) and go to step 2.

Step 2 : beginning from node S, consider all arcs :

(S, 1) and (S, 3) are are forward and saturated; (S, 4) is not saturated and is not

labeled; label node 4 with (+S).

Figure 4.2: Application of Ford-Fulkerson algorithm

Node 4 : arc (4, 5) unsaturated and 5 unlabeled; label node 5 with (+4).

arc (4, T) saturated.

arc (3, 4) is backward and has a positive flow, label node 3 with (−4).

Node 3 : arc (3, 2) unsaturated and 2 unlabeled; label node 2 with (+3).arc (1, 3) has a positive flow , label node 1 with (−3).

Node 1 : arc (1, 2) with 2 labeled;

Node 2 : arc (2, T) unsaturated and T unlabeled; label node T with (+2).

T is labeled; therefore, there is a breakthrough: it is possible to increase the flow. Goto step 3.

+S

+2

(2)

-3

+3

-4

+4 (4)

(1)

(1)

(1)

(3)

2

(2)

4

2

1

1

S

4

3

1

5

2

T

1

2

3

2

1

5

3

2

*

8/18/2019 CoursOPT-07-08


OPTIMIZATION 3A GI

26

Step 3 : The FC can be identified by starting from T and going backward: (O, 4), (3,

4), (3, 2), (2, T). { }min 2,1,2,3 1δ = = . Subtract 1 unit from the flow on (3, 4) and

add it to the flow of the remaining forward arcs to obtain the new solution:

ϕS1 = 2, ϕ13 = 2, ϕS3 = 3, ϕS4 = 1, ϕ4T = 1, ϕ35 = ϕ5T = 4, ϕ32 = ϕ2T = 1

Erase all labels and go to step 1.

Step 1: Label node S with (*) and go to step 2.

Step 2 :

(S, 1) and (S, 3) are saturated; (S, 4) is an saturated and unlabeled; label node 4 with

(+S).

Node 4 : arc (4, 5) unsaturated and 5 unlabeled; indicate (+4) on node 5.

arc (4, T) saturated.

arc (3, 4) has a zero flow.

Node 5 : arc (5, T) unsaturated and T unlabeled; indicate (+5) on node T.

It is possible to increase the flow since node T is labeled. Go to step 3.

Step 3 : The FC is : (S, 4), (4, 5), (5, T) and δ=1; add 1 unit to flow on forward arcs to

obtain the following solution:

ϕS1 = 2, ϕ13 = 2, ϕS3 = 3, ϕS4 = 2, ϕ45 = 1, ϕ4T = 1, ϕ35 = 4, ϕ5T = 5, ϕ32 = ϕ2T = 1

Erase the labels and go to step 1.

Step 1 : Label node S with (*) and go to step 2.

Step 2 :

Starting from node S, no node can be labeled since all arcs are saturated. Hence,

node T cannot be labeled. The current solution is then optimal, with a flow value

equal to 7.

8/18/2019 CoursOPT-07-08


OPTIMIZATION 3A GI

27

3.4 Concept of cuts

A partition of the set of nodes into two subsets X and Xc (where Xc = N\X) defines a

cut, denoted by (X,Xc), which is the set of arcs with one node in X and the other in Xc.

(X,Xc) is a disconnecting set because deleting the arcs of (X,Xc) from the network G

leaves two disconnected components (no chain/path between any two nodes in X and

Xc, respectively). For directed networks, if only forward arcs are deleted (i.e., arcs

(i,j) such that i ∈ X and j ∈ Xc) there exists no path from any node in X to any node in

Xc (chains may still exist).

Special cuts are those that separate the source S from the sink T. By choosing a subset

X that contains the source S but not the sink T, the set of arcs (i, j) with i ∈ X and j ∉

X is a cut of the network. A cut separating S and T is a set of arcs such that if

eliminated, flow becomes impossible from S to T (S and T become disconnected).

Note: A network may contain many cuts.

The capacity of a cut, denoted by C(X,Xc) where S ∈ X and T ∉ X, is defined as the

sum of capacities of its forward arcs. Let F(X,Xc) be the value of the flow across the

the cut (X,Xc) and v(ϕ) be the value of a flow transiting from S to T.

Theorem 3.1:

If all nodes satisfiy the flow conservation equation then ϕ+(X) = ϕ-(X) (flow on forward

arcs equals flow on backward arcs).

Proof:

Let A1 = set of arcs (i,j) such that i and j ∈ X.

Flow leaving nodes of X = ϕ+ (X) +!

u

u A

ϕ ∈∑

Flow entering nodes of X = ϕ-(X) +!

u

u A

ϕ ∈∑

Since all nodes satisfy the flow conservation equation, then

ϕ+ (X) +!

u

u A

ϕ ∈∑ =ϕ-(X) +

!

u

u A

ϕ ∈∑ ⇒ ϕ+ (X) = ϕ-(X)

8/18/2019 CoursOPT-07-08


OPTIMIZATION 3A GI

28

Lemma 3.1:

The value of any flow is less than or equal to the capacity of any cut:

∀ ϕ and (X, Xc) separating S and T, we have v(ϕ) ≤ C(X, Xc)

Proof:

Taking into account the reverse arc: ϕ+(X) = ϕ-(X) =reverse arc

u

c

u

u ( X,X ),(T,S)

ϕ

∈≠

∑ + ϕ0

Then, ϕ0 = v(ϕ) = ϕ1(X) -reverse arc

u

c

u

u ( X,X ),(T,S)

ϕ

∈≠

∑ ≤ ϕ1(X) ≤ C(X, Xc)

As a consequence, the capacity of a cut represents a lower bound on the maximum

The maximum flow minimum cut theorem:

The maximum value among feasible flow vectors is equal to the minimum capacity of

cuts separating S and T : maxfeasible ϕ v(ϕ) = mincuts (X, Xc) C(X, Xc)

One consequence of this theorem is that the Ford-Fulkerson algorithm simultaneously

finds the maximum value flow and the mimum capacity cut.

3.5 Case of networks having lower bounds

In this case, a flow has to satisfy both lower and upper bounds (capacities):

lu ≤ ϕu ≤ ku, ∀ u = 1, …, m where lu is a lower bound on ϕu.

The network is then denoted by G = (N, A, L, K).

Here, the problem of finding a feasible flow is not as obvious as before. To find themaximum value flow, we apply two phases :

• Phase I : Find a compatible flow (if it exists)

• Phase II : Solve for the maximum value flow (for example, by applying an

adaptation of Ford-Fulkerson)

3. 5.1 Phase I

Let L+(i) =ij j (i )l

∈Γ∑ : Sum of lower bounds of arcs incident out of i;

8/18/2019 CoursOPT-07-08


OPTIMIZATION 3A GI

29

L-(i) =1 ji j (i )

l−∈Γ∑ : Sum of lower bounds of arcs incident into i;

Transform G into a network G' having zero lower bounds in order to find an initial

compatible flow for G. This is achieved by applying the following :

• Add the reverse arc (T , S) having a capacity = ∞

• Replace the capacity kij by k’ij = kij - lij

• Add a dummy source S’ and a dummy sink T’

• For each i such that L+(i) >0, add the arc (i, T’) of capacity L+(i)

• For each i such that L-(i), add the arc (S’,i) of capacity L-(i)

Theorem 3.2

There exists a compatible flow vector ϕ for G iff G the maximum value flow vector ϕ'

of G’ has a value L+(S’) = L-(T’). This means that ϕ' saturates the arcs incident out of

S’ and those incident into T’. In addition, for each arc (i, j) of G, we have:

ϕij = ϕ' ij + lij

3. 5. 2 Phase II

Once a compatible flow vector is found, Ford-Fulkerson algorithm can be applied with

a slight modification: backward arcs are labeled if ji > l ji and δ is computed based

on the difference ( ji - l ji)

The rest of the algorithm remains the same.

Example:

Figure 4.3: Example of network with lower bounds

S

3

1

2

T

(1,5)

(0,7)

(1,6)

(2,4)

(1,1)

(0,3)

(2,5)

8/18/2019 CoursOPT-07-08


OPTIMIZATION 3A GI

30

Figure 4.4: Example of transformed network G’

An initial compatible flow is then:

ϕS1 = 1, ϕS2 = 4, ϕ21 = 1, ϕ13 = 2, ϕ23 = 2, ϕ2T = 1, ϕ3T = 4

3.6 Minimum cost flow problems

3.6.1 Pr oblem def i n i t ion

Consider a connected network G = (N, A, L, K, C), where L is the vector of lower

bounds (l1, l2, …,lm)T, K is the vector of capacities (k1, k2, …,km)T, and C is the vector

of costs (c1, c2, …,cm)T. The cost of transporting a unit of flow on arc u is c u.

Therefore if ϕ is the flow vector, the total flow cost is: u uu Ac ϕ

∈∑ .

Hence, the minimum cost flow problem consists in:

• Finding, among maximum value flows, a feasible flow that minimizes total

cost.

• Or, Find the minimum cost way of transporting a flow of a fixed value v. In this

case, if v > maximum value flow, the problem is infeasible.

S

3

1

2

T

4

7

52

0

3

2

S’T’

1

4

2

32

3

+S’

+1

+∞

(1)

(1)

(3)

+S

+2

+T(4)

(4)

(2) (2)

+3

(2)(2)

(2)

8/18/2019 CoursOPT-07-08


OPTIMIZATION 3A GI

31

3.6.2 Appl icat ions

a. Optimal production planning

Consider the case of company that wants to plan the production of a given product

over a horizon of n periods. Knowing for each period i (i = 1… n):

• The demand di,

• the maximal production capacity ui,

• production cost/unit ci,

• storage capacity si,

• storage cost/unit hi, which applies to quantities left unused at th end of period i,

The problem is to determine the quantities to produce at each period, so that

demand of each period is satisfied with a minimum cost possible. Assume that initial

inventory is zero.

This problem can be reduced to finding a flow vector of a fixed value in a network G =

(N, A, C) constructed as follows:

• Associate a node to each period i (i = 1…n),

•

introduce two additional nodes S and T,

• link the node S to each node i with an arc (S, i) of capacity u i and a cost ci : a

flow on the arc (S, i) represents production during period i,

• link each node i with the node T with an arc (i, T) of capacity di and zero cost:

a flow on the arc (i, T) represents demand of period i,

• link each node i (i = 1…n-1) to node i+1 with un arc (i, i+1) of capacity si and a

cost hi : a flow on the arc (i, i+1) represents the quantity stored at the end of

period i.

It is easy to verify that to each feasible flow between S and T of a value v = ∑i di

corresponds a feasible production plan. The optimal plan is then obtained by solving a

minimum cost flow problem.

b. Assignment problem

Consider the situation where n operators have to be assigned to n jobs. Let cij be the

cost of assigning operator i to job j. The problem is to determine a minimum cost

8/18/2019 CoursOPT-07-08


OPTIMIZATION 3A GI

32

assignment of operators to jobs. This problem can be transformed into a minimum

cost flo problem in a bipartite network.

Let N1 be the set of operators and N² be the set of jobs:

For each i ∈ N1 and each j ∈ N2, create an arc (i, j) with a cost equal to cij and a

capacity equal to 1.

Create two additional nodes S and T: the node S is linked to each node i ∈ N1 and

each node j ∈ N2 is linked to node T. The arcs (S, i) and (i, T) have zero costs and

capacities equal to 1.

One can verify that to each flow between S and T of value n corresponds a feasible

assignment une affectation réalisable (and vice versa). The cost of such a flow is

equal to the assignment cost.

3.6.3 Residual net wor k

Let ϕ = [ϕ1, ϕ2, …, ϕm]T be a feasible flow vector. The residual network associated

with ϕ is the network G’(ϕ) having the same set of nodes as G, and which the set ofarcs is defined as follows:

To each arc u = (i, j) of G corresponds, at most, two arcs in G’(ϕ):

• u+ = (i, j) if lu≤ ϕu < ku ;

this arc has a residual capacity k’ u+ = ku - ϕu > 0 and a cost cu+=cu

• u- = (j, i) if ϕu > lu

this arc has a residual capacity: k’ u- = ϕu - lu > 0 and a cost cu-=-cu

Theorem 3.3

A necessary and sufficient condition for a feasible flow ϕ to be of minimum cost in G

is that no negative circuit exists in G’(ϕ).

This theorem shows that an optimal flow may be obtained by starting with a feasible

flow of value v and searching for a negative circuit in the residual network. If such a

8/18/2019 CoursOPT-07-08


OPTIMIZATION 3A GI

33

circuit does not exist, then we conclude that the current solution is optimal.

Otherwise, it is possible to increase the flow in the circuit by the minimum residual

capacity. The total value of the flow remains the same but its cost decreases.

3.6. 4 Solut ion met hods

We distinguish mainly two methods for solving tne minimum cost flow problem:

- Klein algorithm (modified by Bennington),- Algorithm of Busacker and Gowen (also called Roy algorithm).

a. Klein algorithm (1967)

It consists in finding, among maximum value flow vectors, the flow ϕ = [ϕu] that

minimizes the total cost C = ∑u∈U cu ϕu

Starting with a maximum value flow (obtained for example by the Ford-Fulkerson

algorithm), we seek to determine negative circuits in the corresponding residual

network. If such circuits do not exist, then the current flow is of minimum cost;

otherwise, a substitution along one of these circuits gives a flow with a smaller cost.

In this case, a new residual network is constructed and the process continues.

Algorithm of Klein (1967):

Step 0:

t = 0

Using a maximum value flow algorithm, find a maximum value flow for G.

If no feasible flow exists, Terminate, the problem is infeasible.

Otherwise, let ϕ0 be this flow vector.

Step 1:

Let ϕt be the flow at the tth iteration.

Construct the residual network G’(ϕt) associated with ϕt.

Step 2:

Serach for a negative circuit µ’ in G’(ϕt).

8/18/2019 CoursOPT-07-08


OPTIMIZATION 3A GI

34

If such a circuit does not exist, the curent flow vector is optimal, Terminate.

If such a circuit exists, Let µ be the cycle corresponding to µ’ in G.

let δt = min(i,j)∈µ’ (k’ij) the residual capacity of circuit µ’. Define the flow vector ϕt+1 as

follows:

ϕu t+1 = ϕu

t + δt if u+ ∈ µ’

ϕu t+1 = ϕu

t - δt if u- ∈ µ’

Set t = t + 1 and go back to step 1.

The main issue in this method is how to find a negative circuit. In fact, the search for

such circuits is achieved by planting a tree rooted at T (the destination) and covering

the considered residual network. The algorithm of Bennington solves this problem

through the enhancement of a first algorithm proposed by Klein.

Algorithm for finding a negative circuit in G’( )

Step 0:

Construct an arbitrary tree R1 rooted at T and spanning G’(ϕ) (having the same set of

node as G’(ϕ)), t = 1;

Step 1:

Associate to each node x of G’(ϕ) a value hx defined as:

hT = 0 and hy = hx + cxy’ if arc (x, y) ∈ Rt,

Step 2 :

Choose, if it exists, un arc (x, y) of G’(ϕ) that does not belong to Rt and such that:

hx + cxy’ < hy

If such arc does not exist, Terminate: in fact, the hx are the minimum costs of paths

from T to x, for all x. As a consequence, G’(ϕ) does not have any negative circuit.

Otherwise:

If the path from T to x in Rt traverses y, then the concatenation of the subpath

extracted from Rt: (y, …, x) and the the (x, y) yields a negative circuit,

Otherwise, modify the tree Rt by deleting the arc (z, y) of Rt (where z is the unique

predecessor of y in Rt) and adding the arc (x, y);

t = t + 1, go to step 1.

8/18/2019 CoursOPT-07-08


OPTIMIZATION 3A GI

35

Example: a flower vendor needs to transport flowers from its flower field (S) to the

city (T), using various vans of different capacities. The roadway network linking S to T

is given in Figure 4.5, where each arc is described by its capacity (in boxes/week) and

its unit travel cost (TND/box).

The vendor wants to study the problem of transporting the flowers per week so that

the maximum quantity is transported for a minimum cost.

Figure 4.5: Example of the flower vendor

Let us apply Klein algorithm:

Step 0 : Using Ford-Fulkerson algorithm, we search for a maximum value flow vectorfor G:

Let the initial solution be:

S → 1 → T : flow = 200

S → 2 → 3 → T : flow = 200

An augmenting path for this solution is:

S → 1 → 3 → T with an incremental flow δ = 100.

With this solution, we can no longer label the destination T: This is an optimal value

flow with a value of 500 boxes and a cost of 7300 TND.

Let ϕ0 be this flow.

(200, 13)

(200, 9)

(200, 16)

(300, 0)(200, 2)

(200, 0)

S

1

2

3

T(400, 2)

8/18/2019 CoursOPT-07-08


OPTIMIZATION 3A GI

36

Step 1:

The above is the residual network G’(ϕ0) relative to ϕ0.

Step 2 : Look in G’(ϕ0) for a circuit µ’ with a negative cost relatively to the cost

vector c’. To do this, construct a spanning tree for G’(ϕ0) :

Figure 4.6: A spanning tree for G’(ϕ0)

The arc (1, S) dos not blong to the spanning tree and h1 + c’1S < hS (-18 < -13).

The path from T to 1 in the tree does not go through S: Erase the arc (2, S) and add

arc (1, S). hS = -18

The arc (1, 3) does not belong to the new tree and h1 + c’13 < h3 (-7 < 0).The path from T to 1 in the tree does not go through node 3: Erase the arc (T, 3) and

add the arc (1, 3). h3 = -7 and h2 = -20.

(100, 2)

(300, -2)

(100, -9)

(200, -13)

(100, 9)

(200, -16)

(300, 0)(200, 2)

(200, 0)

S

1

2

3

T

[0]

[-13]

[-13]

[0]

(200, -13)

(200, -16)

(300, 0)

(200, 0)

S

1

2

3

T

[-16]

8/18/2019 CoursOPT-07-08


OPTIMIZATION 3A GI

37

Figure 4.7: New spanning tree for G’(ϕ0)

The arc (2, 1) does not belong to the newly chosen tree and h2 + c’21 < h1 (-18 < -16).

The path from T to 2 in the tree goes through node 1: we have a negative circuit with

a cost = -2 : (1, 3, 2, 1).

The smallest capacity on this circuit equals 100: in the network G, increase the flow

on the arcs (2, 1) and (1, 3) by 100 boxes and reduce the flow on the arc (2, 3) by 100

boxes.

Therefore, the cost is reduced by 100 * 2 = 200 TND

The new tree is obtained by eliminating the arcs (1, 3) and (3, 2) and by adding the

arcs (1, 2) and (2, 3).

No more negative circuits. Hence, the flow obtained is optimal. END.

The optimal solution is the following:

ϕ(S, 1) = 300, ϕ(S, 2) = 200, ϕ(2, 1) = 100, ϕ(2, 3) = 100, ϕ(1, 3) = 200, ϕ(1, T) = 200,

ϕ(3, T) = 300.

Its value is = 500 boxes, its cost is = 7100 TND.

b. Algorithm of Busacker and Gowen

Here, we want to find a flow vector ϕ = [ϕu] with a fixed value at minimum cost.

Starting with a zero flow vector (or any other flow with a value v0), construct a flow

of value v1 and having a minimum cost among all flow vectors of the same value. Then

construct a flow of value v2 > v1, and at minimum cost, …, etc until a flow of value v*

is obtained. This is achieved by searching, at each iteration, in the associated residualnetwork, a minimum cost path from S to T and by modifying the flow on its arcs.

[-18]

[-20]

[-7]

S

1

2

3

T

[-16]

[0]

8/18/2019 CoursOPT-07-08


OPTIMIZATION 3A GI

38

Algorithm of Busacker and Gowen

Step 0:

Choose as initial flow ϕ0 the null flow vector: v(ϕ0) = 0.

The residual network G’(ϕ0) is then identical to the initial network G.

Set t = 0.

Step 1:

Determine, using an appropriate algorithm, a minimum cost path from S to T in the

residual network G’(ϕt).

If such a path does not exist, then Terminate: the current flow is optimal.

Otherwise, go to step 2.

Step 2:

To the path just found corresponds to an augmenting chain in G. Increase the flow in

G by the allowed amount δt on this chain. This gives the new flow vector ϕt+1 of value

v(ϕt+1) = v(ϕt) + δt.

Step 3:

Construct the residual network G’(ϕt+1) associated with this new flow vector.

Set t = t + 1 and go to step 1.

At the end of this algorithm, we obtain a maximum value flow having a minimum

cost. In case a lower value is sought, the algorithm should be stopped once this value

is reached.

Example: Reconsider the previous example (see Figure 4.5).

Apply Busacker & Gowen algorithm:

Step 0: Let ϕ0 be the null flow vector.

The residual graph G’(ϕ0) is identical to G, t = 0.

Step 1: Apply Ford-Bellman to find a minimum cost (shortest) path from S to T in

G’(ϕ0).

8/18/2019 CoursOPT-07-08


OPTIMIZATION 3A GI

39

Iteration S 1 2 3 T

0 0 2 0 ∞ ∞

1 0 2 0 11 18

2 0 2 0 11 11

3 0 2 0 11 11

The optimal path is: S → 2 → 1 → 3 → T, with a cost of 11 DT/box.

Step 2:

Increase the flow in G by the maximum allowed amount on the associated chain: =

200 boxes, giving the flow of value 200.

Step 3: The residual network G’(ϕ1) associated with this new flow vector is the

following:

Figure 4.8: Residual network G’(ϕ1)

Step 1: Apply Ford-Bellman to find the shortest path, in terms of cost, from S to T in

G’(ϕ1).

Iteration S 1 2 3 T

0 0 2 ∞ ∞ ∞

1 0 2 0 ∞ 18

2 0 2 0 13 18

3 0 2 0 13 13

(200, 0)

(200, 13)

(200, -9)

(200, 16)

(100, 0)(200, -2)

(200, 0)

S

1

2

3

T(400, 2)

8/18/2019 CoursOPT-07-08


OPTIMIZATION 3A GI

40

The optimal is S → 1 → 2 → 3 → T, with a cots of 13 DT/box.

Step 2:

Increase the flow value in G by the maximum allowed amount =100 boxes. This gives

a flow ϕ2 having a value of 300.


following:


Step 1: Apply Ford-Bellman to find the shortest path, in terms of cost, from S to T in

G’(ϕ2).

Iteration S 1 2 3 T

0 0 2 ∞ ∞ ∞

1 0 2 0 ∞ 18

2 0 2 0 13 18

3 0 2 0 13 18

The optimal path is: S → 1 → T, with a cost of 18 DT/box.

Step 2:

The flow value can be increased by = 200 boxes, giving the flow ϕ3 of a value 500.

(100, -13)(100, 2)

(100, -2)

(100, 13)

(200, -9)

(200, 16)

(300, 0)(100, -2)

(200, 0)

F

1

2

3

T(300, 2)

8/18/2019 CoursOPT-07-08


OPTIMIZATION 3A GI

41


following:


There is no path from S to T in the residual network G’(ϕ3). END.

The optimal solution is the following:

ϕ(S, 1) = 300, ϕ(S, 2) = 200, ϕ(2, 1) = 100, ϕ(2, 3) = 100, ϕ(1, 3) = 200, ϕ(1, T) = 200,

ϕ(3, T) = 300.

The total flow = 500 boxes with a cost of 7100 DT.

(100, -13)(100, 2)

(300, -2)

(100, 13)

(200, -9)

(200, -16)

(300, 0)(100, -2)

(200, 0)

S

1

2

3

T(100, 2)

8/18/2019 CoursOPT-07-08


OPTIMIZATION 3A GI

42

Chapter 4 Dynamic Programming

4.1 Introduction

Dynamic Programming is a modeling framework that is generally used for optimization

problems involving sequential decision processes. In these dynamical problems,

decisions are made for a planning horizon, which is divided into periods or stages, and

a decison at one stage influences the outcome and the decisions of all the flollowing

stages.

This optimization technique was developed by R. Bellman in 1950 (USA). It consists of

decomposing a problem into subproblems or stages that can be easily solved.

Application examples:

1. Production and inventory control: It consists of planning the production in a

way that satisfies demand while minimizing total production and holding cost.

2. Resource allocation: involves allocating limited resources to a set of activities.

3.

Equipment replacement: deals with determining a minimum cost strategy that

specifies how long a machine or equipment should be kept before it is replaced

(or traded in).

Most of these problems can be solved using linear or integer programming. However,

dynamic programming is more efficient for problems with low dimensions (a finite set

of resources to assign to a number of activities) and, for some problems with

nonlinear cost or profit functions, it may be the only sensible method.

Dynamic programming can be used for problems having the following characteristics:

- Continuous or discrete decision variables

- Continuous or discrete horizons (or time)

- Finite of infinite horizons

- Deterministic or stochastic data (such as demand in production planning

problems)

8/18/2019 CoursOPT-07-08


OPTIMIZATION 3A GI

43

In this course we consider only: Discrete, finite horizon and deterministic dynamic

programming.

4.2 Dynamic Programming formulationsMany applications are reduced to finding the shortest (or the longest) path between

two nodes of a graph.

4.2.1 The short est r out e pr oblem in a net wor k

Consider the following example. Joe Cougar needs to travel from Nashville to Los

Angeles. In order to minimize his total travel cost, he decided to spend each night at

a friend's house living in each of the following cities: Kansas City, Omaha, Dallas, San

Antonio and Denver. Joe knows that after one day of driving he can reach Kansas City,

Omaha or Dallas and after 2 days he can arrive at San Antonio or Denver. Finally,

after 3 days he can be at Los Angeles. Joe has to decide at which cities he should stay

in order to minimize the total distance traveled. The corresponding network is shown

on the following figure.

Figure 4.1 A network example

Each stage i contains the set of cities Joe can reach at the beginning of day i.

4.2.2 General f orm of a dynamic pr ogr am

A dynamic program can be written as:

Max/Minx1, x2,… xT

T

t t t

t 1 g ( s ,x )

=∑

790

540760

940

270

790

1

2

3

4

5

6

7

610

580

660

1030

1390

Stage 1 Stage 2 Stage 3 Stage 4

8/18/2019 CoursOPT-07-08


OPTIMIZATION 3A GI

44

(DP) st = Φt (st-1, xt-1) t = 2, …, T

s1 given

t : period,

st : state variables,

xt : decision variables relative to period t.

Φt : a function that transforms the state of the system into another state if

decision xt is made.

s1 s2 s3 …………. sT sT+1

Figure 4.2 Graphical representation of a dynamic program

In general, a problem is decomposed into smaller subproblems. For example, a

problem having T variables can be decomposed into T subproblems of only one

variable.

4.2.3 Forwar d for mula t ion

We begin by finding the shortest route from the origin city to each of the cities of

stage 2. Then, we use this information to find the shortest path from the origin city to

each of the cities of stage 3. This information is then used to find the shortest path

from the origin city to the destination city. The idea is to solve, at each iteration, an

easy problem which helps to solve a more complex problem.

Forward recursion:

Let's define cij as the distance between city i and city j.

Then, define f t(j) as the length of the shortest route from city 1 to city j, given that

city j is at stage t.

Stage 1 : f 1(1) = 0 .

Stage 2 : f 2(2) = 580

gT(sT, xT)g2(s2, x2)g1(s1, x1)

xT

x2x1

8/18/2019 CoursOPT-07-08


OPTIMIZATION 3A GI

45

f 2(3) = 760

f 2(4) = 660

Stage 3 : the shortest route from city 1 to city 5 is the route having the minimal

disance among the different possible routes:

1 → 2 → 5 distance = c25 + f 2(2) = 610 + 580 = 1190*

or 1 → 3 → 5 distance = c35 + f 2(3) = 540 + 760 = 1300

or 1 → 4 → 5 distance = c45 + f 2(4) = 790 + 660 = 1450

The shortest route between cities 1 and 5 is 1 → 2 → 5 with f 3(5) = 1190 .

Similarly, the shortest route from city 1 to city 6 is the one having the minimal

distance among the different possible routes:

1 → 2 → 6 distance = c26 + f 2(2) = 790 + 580 = 1370

or 1 → 3 → 6 distance = c36 + f 2(3) = 940 + 760 = 1700

or 1 → 4 → 6 distance = c46 + f 2(4) = 270 + 660 = 930*


Stage 4 : the shortest route from city 1 to city 7 is the route having the minimal

disance among the different possible routes:

f 4(7) = Min {c57 + f 3(5) = 1030 + 1190 = 2220* ; c67 + f 3(6) = 1390 + 930 = 2320}

The shortest route between cities 1 and 7 is 1 → 2 → 5 → 7 with f 4(7) = 2220 .

What are we trying to find here? Answer: f 4(7).

f 4(7) uses f 3(i), f 3(i) uses f 2(i) and f 2(i) uses f 1(1) : this is a set of recursive equations,

also called functional equations. This is similar to finding the shortest path on an

acyclic network

General Forward formulation

Let f k(sk) be the value of an optimal sequence of decisions that takes the system from

state s1 till state sk.

The DP forward recursive formulation would be:

8/18/2019 CoursOPT-07-08


OPTIMIZATION 3A GI

46

f k(sk) = Max/Min xk {gk(sk, x k) + f k-1(sk-1)}

s1 given : boundary conditions

Objective: find f T+1(sT+1)

Applying this formulation to the shortest route example gives the following:

f t(j) = Mini {f t-1(i) + cij } where i is a state at stage t-1

f 1(1) = 0 : boundary conditions

Objective: find f 4(7)

In this case, f t(j) is the length of the shortest path from city 1 to city 7.

4.2.4 Backward formula t ion

For many other applications, it is easier to use a dynamic programming formulation

using a backward reasoning. This results into a backward formulation. Backward

reasoning may sometimes make a seemingly difficult problem easy to solve.

Example 4.1

Consider the following game. We have 30 matches. I begin by taking 1, 2 or 3

matches. Then, comes my opponent's turn to also take 1, 2 or 3 matches. We

continue this way until there is only one match left. The player who takes the last

match loses the game. As a first player, how can I garantee to win the game ?

If I make sure that it is my opponent's turn when there is only one match left, then I

would be sure of winning. Using the same reasoning by going backwards, if I make

sure it is my opponent's turn when there are 5 matches left, then I would be sure of

winning. The reason is that no matter how many matches he will take, I can always

make sure to get to the situation where it is his turn when there is only one match

left:

If he takes 1 I will take 3



Similarly, if if I make sure it is my opponent's turn when there are 9, 13, 17, 21, 25 or29 matches left, then I would be sure of winning. This way, I can garantee to win if I

8/18/2019 CoursOPT-07-08


OPTIMIZATION 3A GI

47

begin by taking 1 match. Then, at each one of my subsequent turns, I will make sure

to leave for my opponent either 25, 21, 17, 13, 9 or 5 matches.

With this backward reasoning, we are able to build up a full strategy starting from the

basic simple subproblem corresponding to “only one match”.

Let f k(sk) be the value of optimal sequence of decisions that takes the system from

state sk till state sT (final state)

The DP backward recursive formulation would be:

f k(sk) = Max/Min xk {gk(sk, x k) + f k+1(sk+1)}

sT+1 = Φ T (sT , x T ) : boundary conditions

Objective: Find f 1(s1)

Applying this formulation to the shortest route example give the following:

f t(i) = Min j {f t+1(j) + cij } where j is a state at stage t+1

f 4(7) = 0 : boundary conditions

Objective: find f 1(1)

Here, f t(i) is the length of the shortest route from city i to city 7 , given that city i is

at stage t. In the backward recursion, the computation begins with the last stage (or

states)

Stage 4 : f 4(7) = 0 .

Stage 3 : f 3(5) = 1030

f 3(6) = 1390

Stage 2 : The shortest route between cities 2 and 7 is the route having the minimaldistance among all possible routes:

2 → 5 → 7 distance = c25 + f 3(5) = 610 + 1030 = 1640*

or 2 → 6 → 7 distance = c26 + f 3(6) = 790 + 1390 = 2180


Similarly, shortest route between cities 3 and 7 is the route having the minimal

distance among all possible routes:

3 → 5 → 7 distance = c35 + f 3(5) = 540 + 1030 = 1570* or 3 → 6 → 7 distance = c36 + f 3(6) = 940 + 1390 = 2330

8/18/2019 CoursOPT-07-08


OPTIMIZATION 3A GI

48

The shortest route between cities 3and 7 is 3 → 5 → 7 with f 2(3) = 1570 .

Similarly, the shortest route between cities 4 and 7 is the route having the minimal


4 → 5 → 7 distance = c45 + f 3(5) = 790 + 1030 = 1820

or 4 → 6 → 7 distance = c46 + f 3(6) = 270 + 1390 = 1660*


Stage 1 : The shortest route between cities 1 and 7 is the route having the minimal


f 1(1) = Min {c12 + f 2(2) ; c13 + f 2(3) ; c14 + f 2(4)}

= Min {580 + 1640 ; 760 + 1570 ; 660 + 1660}

= Min {2220 ; 2330 ; 2320} = 2220

The shortest route between cities 1and 7 is 1 → 2 → 5 → 7 with f 1(1) = 2220 .

4.3 Dynamic programming complexity

Consider the shortest route problem. If we use complete enumeration, we would have

to compare between 3*2 = 6 possible routes. In this case DP, is not efficient.

However, for problems of larger dimensions, dynamic programming can be much more

efficient than enumeration. Consider a network with a total of 27 nodes, and having 7

stages, with 5 nodes at each of stages 2, …, 6. Suppose in addition, that we can go

from any node at stage k to any other node at stage k+1. The problem is to find the

shortest route from state 1 to state 27.

Using complete enumeration, one has to consider 55 = 3125 possible routes.

Number of elementary operations:

- (55) * 5 = 15.625 additions to determine the length of a route (5 additionsto determine the lenght of a route.

- (55) - 1 = 3.124 comparisons to determine the shoretst route.

Using dynamic programming: Let f t(i) be the length of the shoretst route from city i to

city 27 , given that city i is at stage t. We need to determine f 1(1).

8/18/2019 CoursOPT-07-08


OPTIMIZATION 3A GI

49

Starting at stage 6: f 6(22), f 6(23), f 6(24), f 6(25) and f 6(26) ⇒ No additions.

At stage 5 : f 5(17) = min j {c17,j + f 6(j)} ⇒ 5 additions.

f 5(17), f 5(18), f 5(19), f 5(20), f 5(21) ⇒ 5 * 5 = 25 additions.

For stages 4, 3 and 2, we have to make the same number of additions as at stage 5.

For stage 1 : f 1(1) = min j {c1,j + f 2(j)} ⇒ 5 additions.

This gives a total of : (4 * 25) + 5 = 105 additions.

In addition, for each of the nodes 1, 2, …, 21, we have to make comparisons to

determine the corresponding shortest route ⇒ (5 – 1) comparisons for each

node, resulting into : 21 * 4 = 84 comparisons.

Conclusion :

We can see that DP is much more efficient than explicit enumeration. Just by

considering the number of additions, DP is at least 148 times faster.

4.4 Characteristics of dynamic programming applications

Dynamic programming applications have certain commun characteristics such as:

1.

The problem can generally be decomposed into stages or periods eachrepresenting a decision. For the shortest route problem, a stage i correspondsto the set of cities where Joe can be at, at the beginning of day i. In certainscases, certain stages do no require a decision.

2. Each stage is associated with a number of states, where a state is the set ofinformation that is necessary in order to make a feasible decision. For theshortest route problem, the state at stage i is simply the city where Joe is at,at the beginning of day i.

3. We note that, to make an optimal decision at each stage, Joe does not need toknow how he reached a given city. All he needs to know is at which city he is ata given stage.

4. The decision taken at a given stage describes the transition of the system froma state at a given stage into a state of the next stage. For the shortest routeproblem, the decision made by Joe (here, Joe is both the system and thedecision maker) at a given stage is the city at which he will spend the followingnight.

5.

Given a state of a current stage, the optimal decision for each of the followingstages has to be independent of the previous decisions. This is the principle of

8/18/2019 CoursOPT-07-08


OPTIMIZATION 3A GI

50

optimality of Bellman. For the shortest route problem, this principle statesthat each optimal route is composed of optimal sub-routes.

If R is the shortest route between city 1 and city 5, with i as an intermediary

city, then the portion of R going from i to 5 has to be a shortest route from i to

5. At the stage where one is at city i, the decision to make is independent of

previously visited cities and thus, independent of the route selected to go from

1 to i.

6. If a problem is formulated using the concept of stages, there must be arecursive equation that determines the cost or the profit of a state at anystage 1, 2, …,t as a function of the cost of the profit of states belonging to

stages 1, 2, …, t-1 (forward recursion) where T is the total number of stages: f t(j) = Mini {f t-1(i) + cij } where i belongs to stage t-1

f 1(1) = 0 boundary conditions

Objective : find f 4(7)

Similarly, If a problem is formulated using the concept of stages, there must be

a recursive equation that determines the cost or the profit of a state at any

stage t, t+1, …, T as a function of the cost of the profit of states belonging to

stages t+1, …, T (backward recursion):

f t(i) = Min j {f t+1(j) + cij } where j belongs to stage t+1

f 4(7) = 0 boundary conditions


4.5 Production and inventory control

A company must decide how much to produce of a given product in order to satisfy

customers' demand over a horizon of T periods. The demand at a period t can besatisfied either from production during the current period or from inventory that is

available at the beginning of the current period.

Production and inventory holding costs vary from one period to another. the problem

is to determine how much to produce during each period so as to minimize total

production and holding cost.

The following are the data given for each period t = 1,… , T :- d t : demand of the period, know at the beginning of period 1,

8/18/2019 CoursOPT-07-08


OPTIMIZATION 3A GI

51

- ct : fixed production cost during period t,- pt : unit production cost during period t,- ht : unit holding cost per during period t; the quantity held in stock is the

quantity left unsold at the end the period t.

Let x t : be the quantity to produce during period t.

The production and inventory control can be formulated as a linear or integer

program (depending on whether the quantities are continuous or discrete) and thus

solved using appropriate algorithms. However, one major assumption has to be made:

- the cost functions have to be linear in terms of the quantities produced orheld in stock.

In some practical settings, this assumption is too restrictive if the cost functions have

to be nonlinear. In this case the problem can be solved using dynamic programming.

Problem formulation

State variables

it = Inventory at hand at the beginning of period t.

Decision variables:

x t : the quantity to produce during period t.

Let f t(it) : optimal cost corresponding to periods t,…, T if the inventory level at the

beginning of period t is st.

The recursive equation of the backward formulation is:

f t(it) = min feasible xt {gt(x t, it) + f t+1(it +x t - d t)}

it+1

where gt(x t, it) = ct + pt x t + ht(it + x t - d t) if x t>0

ht(it - d t) if x t=0

it ∈ [0, d t+1 + d t+2 + … + d T ]

Total demand left to satisfy

Boundary conditions:

f T (iT ) = min feasible xT {gT (x T , iT )}

8/18/2019 CoursOPT-07-08


OPTIMIZATION 3A GI

52

Example 4.2

A company knows the demands of the next 4 months:

Month t 1 2 3 4

Demand d t 1 3 2 4

At the beginning of each month t, the company has to determine the quantities to

produce x t, given that :

- Fixed production cost c = 3$,- Unit production cost p = 1$/unit,

-

Unit holding cost h = 0,5$/unit left at the end of the period.- Production capacity Lp= 5 units/month- Storage capacity Li = 4 units at the end of the month.

We suppose that there is no unit at hand at the beginning of period 1.

To formulate the problem of finding the minimum cost production and inventory plan,

one has to define the states and if possible the stages of the problem. The stages

have to be defined such that, if just one stage is left, the problem becomes easy to

solve. In this case, if we are at the beginning of the 4th month, the company cansatisfy the demand of the month at a minimal cost by producing just the necessary

quantity based on the inventory at the beginning of the month and the known

demand.

⇒ We can define the stage as the month since the problem corresponding to only one

month is easy to solve.

At the beginning of each month, the company must determine the quantity toproduce. Therefore, all is needed is to know the inventory level at the beginning of

the month.

⇒ The state variables correspond to the inventory level at the beginning of the

month, which is nothing but the inventory level at the end of the previous month.

i(t, i) for t = 2, 3, 4 and i = 0, 1, …Li= 4

i(1, 0) for t = 1, i = 0

8/18/2019 CoursOPT-07-08


OPTIMIZATION 3A GI

53

Recursive function of the backward formulation:

gt(x t, it) = 3 + 1 x t + 0.5 (it + x t – d t) if x t>0

0.5(it - d t) if x t=0

f t(it) : minimal cost of months t,…, 4 if the inventory level at the beginning of

month t is it.

f t(it) = minfeasible xt {gt(xt, it) + ft+1(it+xt - dt)}

In order to take into account the capacity constraints, we have to ensure:

0 ≤ (x t = it+1 - it + d t) ≤ 5 ⇔ it - d t ≤ it+1 ≤ 5 + it - d t

0 ≤ it ≤ 4

Boundary conditions :

f 4(i4) = 3 + (d 4 – s3) if s3 < d 4

f 4(i4) = 0 if s3 ≥ d 4


Month 4 : the company must produce just the necessary quantity to satisfy the month

4 demand. Since the inventory level is zero at the end of month 4, the holding cost

will be zero.

f 4(0) = 3 + 4 = 7$ x 4 = 4

f 4(1) = 3 + 3 = 6$ x 4 = 3

f 4(2) = 3 + 2 = 5$ x 4 = 2

f 4(3) = 3 + 1 = 4$ x 4 = 1

f 4(4) = 0 + 0 = 0$ x 4 = 0

Month 3 : we have to detremine the quantities to produce during month 3 so as to

minimize the total cost for months 3 and 4: F 3(s2)

f 3(i3) = min x3 {g3(x 3, i3) + f 4(i3 + x 3 - d 3)}

where

g3(x 3, i3) = 3 + x 3 + 0,5 (i3 + x 3 - 2) if x 3 >0

g3(x 3, i3) = 0,5 (i3 + x 3 - 2) if x 3 =0i4 = i3 + x 3 - 2

8/18/2019 CoursOPT-07-08


OPTIMIZATION 3A GI

54

0 ≤ x 3 ≤ 5 ; 0 ≤ i4 ≤ 4 ; 0 ≤ i3 ≤ 4

i3 x3 i4 g3 f 4(i3+x3-2) Total cost F3(i3) and x3(i3)

0 2 0 0 + 5 = 5 7 5 + 7 = 12*

0 3 1 0,5 + 6 = 6,5 6 6,5 + 6 = 12,5 f3(0) = 12

0 4 2 1 + 7 = 8 5 8 + 5 = 13 x3(0) = 2

0 5 3 1,5 + 8 = 9,5 4 9,5 + 4 =13,5

1 1 0 0 + 4 = 4 7 4 + 7 = 11

1 2 1 0,5 + 5 = 5,5 6 5,5 + 6 = 11,5 f3(1) = 10

1 3 2 1 + 6 = 7 5 5 + 7 = 12 x3(1) = 5

1 4 3 1,5 + 7 = 8,5 4 8,5 + 4 = 12,5

1 5 4 2 + 8 = 10 0 10 + 0 =10*

2 0 0 0 + 0 = 0 7 0 + 7 = 7*

2 1 1 0,5 + 4 = 4,5 6 4,5 + 6 = 10,5 f3(2) = 7

2 2 2 1 + 5 = 6 5 6 + 5 = 11 x3(2) = 0

2 3 3 1,5 + 6 = 7,5 4 7,5 + 4 = 11,5

2 4 4 2 + 7 = 9 0 9 + 0 = 9

3 0 1 0,5 + 0 = 0,5 6 0,5 + 6 = 6,5*

3 1 2 1 + 4 = 5 5 5 + 5 = 10 f3(3) = 6,5

3 2 3 1,5 + 5 = 6,5 4 6,5 + 4 = 10,5 x3(3) = 0

3 3 4 2 + 6 = 8 0 8 + 3 = 11

4 0 2 1 + 0 = 1 5 1 + 5 = 6* f3(4) = 6

4 1 3 1,5 + 4 = 5,5 4 5,5 + 4 = 9,5 x3(4) = 0

4 2 4 2 + 5 = 7 0 7 + 0 = 7

Month 2 : (Exercise)

We have to determine the quantities to produce during month 2 so as to minimize the

total cost over the months 2, 3 and 4: f 2(i2)

f 2(i2) = min x2 {g2(x 2, i2) + f 3(i2 + x 2 – d 2)}

where

g2(x 2, i2) = 3 + x 2 + 0,5 (i2 + x 2 - 2) if x 2>0

g2(x 2, i2) = 0,5 (i2 + x 2 - 2) if x 2 =0i3 = i2 + x 2 - 2

8/18/2019 CoursOPT-07-08


OPTIMIZATION 3A GI

55

0 ≤ x 3 ≤ 5 ; 0 ≤ i3 ≤ 4 ; 0 ≤ i2 ≤ 4

i2 x2 i3 g2 f 3(i2+x2-3) Total cost F2(i2) and x2(i2)

0 3 0 0 + 6 = 6 12 18 f2(0) = 16

0 4 1 0,5 + 7 = 7,5 10 17,5 x2(0) = 5

0 5 2 1 + 8 = 9 7 16*

1 2 0 0 + 5 = 5 12 17

1 3 1 0,5 + 6 = 6,5 10 16,5 f2(1) = 15

1 4 2 1 + 7 = 8 7 15* x2(1) = 4

1 5 3 1,5 + 8 = 9,5 6,5 16

2 1 0 0 + 4 = 4 12 16

2 2 1 0,5 + 5 = 5,5 10 15,5 f2(2) = 14

2 3 2 1 + 6 = 7 7 14* x2(2) = 3

2 4 3 1,5 + 7 = 8,5 6,5 15

2 5 4 2 + 8 = 10 6 16

3 0 0 0 + 0 = 0 12 12*

3 1 1 0,5 + 4 = 4,5 10 4,5 + 10 = 14,5 f2(3) = 12

3 2 2 1 + 5 = 6 7 6 + 7 = 13 x2(3) = 0

3 3 3 1,5 + 6 = 7,5 6,5 7,5 + 6,5 = 14

3 4 4 2 + 7 = 9 6 9 + 6 = 15

4 0 1 0,5 + 0 = 0,5 10 10,5*

4 1 2 1 + 4 = 5 7 12 f2(4) = 10,5

4 2 3 1,5 + 5 = 6,5 6,5 13 x2(4) = 0

4 3 4 2 + 6 = 8 6 14

Month 1 : We have to determine the quantities to produce during month 1 so as to

minimize the total cost over the months 1, 2, 3 and 4: f 1(0)

f 1(0) = min x1 {g1(x 1, i1) + f 2( x 1 – d 1)}

where

g1(x 1, 0) = 3 + x 1 + 0,5 (0 + x 1 - 2) and x 1>0

i2 = x 1 - 1

0 ≤ x 1 ≤ 5 ; 0 ≤ i2 ≤ 4

8/18/2019 CoursOPT-07-08


OPTIMIZATION 3A GI

56

i1 x1 i2 g1 f 2(x1-1) Coût total f 1(0) et x1(0)

0 1 0 0 + 4 = 4 16 20*

0 2 1 0,5 + 5 =

5,5

15 20,5 f1(0) = 20

0 3 2 1 + 6 = 7 14 21 x1(0) = 1

0 4 3 1,5 + 7 =

8,5

12 20,5

0 5 4 2 + 8 = 10 10,5 20,5

The cost of the optimal production and inventory plan is 20$. Optimal production plan:

Month t 1 2 3 4

Production xt 1 5 0 4

This problem can be addressed as a shortest route problem in a network, which

consists of finding the shortest route from node (1, 0) to node (5, 0) represented in

Figure 5.3.

Month inventory level at the beginning of the month

Figure 4.3 Network representation of the production and inventory control

problem

1, 0

2, 1

2, 2

2, 3

2, 0

2, 4

3, 1

3, 2

3, 3

3, 0

3, 4

4, 1

4, 2

4, 3

4, 0

4, 4

5, 0

x1

x2 x3

x4

8/18/2019 CoursOPT-07-08


OPTIMIZATION 3A GI

57

Theorem 4.1

The production and inventory control problem has a solution such that:

1. it * x t = 0 t = 1,…, T

2. If x t > 0 then x t = ∑ j=tn d j t ≤ n ≤ T

3. If it > 0 then it = ∑ j=tq d j t ≤ q ≤ T

Consequence

Using a backward procedure, if the inventory level at the beginning of a period is non

zero, then it is optimal not to produce at that period.

Example : (2, 3) (3, 1) and (3, 2) (4, 1)

4.6 Equipment replacement

The problem is to determine when an equipment or a machine should be replaced

over a given horizon.

Example 4.3

An auto repair shop uses an engine analyser. The purchase cost of a new analyser is

1000 $. The maintnance cost depends on the age of the analyser. We denote by mi the maintnance cost of the anlyser at its ith year of utilisation.

The analyser can be used for up to 3 years. After its ith year, it can be replaced by a

new one. It can also be sold for an amount that is equal to its salvage value, denoted

by si. The values of mi and si are summarized below:

Year i 1 2 3

mi ($) 60 80 120

si ($) 800 600 500

The shop manager needs to determine a replacement policy that minimizes total cost

over the next 5 years, assuming that at the end the 5-year period the analyser will be

sold for its salvage value.

C = maintnance cost + purchase cost – salvage value

8/18/2019 CoursOPT-07-08


OPTIMIZATION 3A GI

58

Year1 Year2 Year3 Year4

Year5

t=1 t=2 t=3 t=4 t=5 t=6

After the purchase of a new analyser, the manager has to know when to replace it

with a new one.

4.6. 1 Dynami c pr ogramming f ormul at ion (using st ages)

Stage: t : the beginning of year t.

State variable: x t : age of the equipment at t.

Decision variable: replace or keep the equipment.

Recursive function:

f t(x t) : minimal cost incurred for years t, t+1, …, 5, starting year t with an equipment

that has been used for x t years.

For t = 2, 3, 4, 5 we compute ft(1), ft(2) and ft(3)

ft(xt) = min {mx+1 + ft+1(xt+1) ; 1.000 - sxt + m1 + ft+1(xt+1)} for xt = 1 or 2

xt + 1 1

ft(3) = 1.000 – s3 + m1 + ft+1(1)

Equation linking the state variables and the decision variables:

If the decision is to replace the equipment: x t+1 = 1

If the decision is to keep the equipment: xt+1 = xt + 1

Boundary conditions: f6(1) = - 800 ; f6(2) = -600 ; f6(3) = -500

Objective: Find f1(0) = 1.000 + 60 + f2(1)

To solve this dynamic program, we start by computing f 5(x 5) then f 4(x 4) then f 3(x 3)

then f 2(x 2) then last f 1(0).

For t = 2, 3, 4, 5:

ft(1) = min {80 + ft+1(2) : keep the equipment

1.000 - 800 + 60 + ft+1(1) : replace the equipment }

ft(2) = min {120 + ft+1(3) : keep the equipment

8/18/2019 CoursOPT-07-08


OPTIMIZATION 3A GI

59

1.000 - 600 + 60 + ft+1(1) : replace the equipment }

ft(3) = 1.000 - 500 + 60 + ft+1(1) : replace the equipment

For t = 5 :

f5(1) = min {80 + f6(2) = -520, 1.000 - 800 + 60 + f6(1) = -540* }

⇒ f5(1) = -540 : replace the equipment

f5(2) = min {120 + f6(3) = -380*, 1.000 - 600 + 60 + f6(1) = -340 }

⇒ f5(2) = -380 : keep the equipment

f5(3) = 1.000 - 500 + 60 + f6(1) = -240 : replace the equipment

For t = 4 :

f4(1) = min {80 + f5(2) = -300*, 1.000 - 800 + 60 + f5(1) = -280 }


f4(2) = min {120 + f5(3) = -120*, 1.000 - 600 + 60 + f5(1) = -80 }


F4(3) = 1.000 - 500 + 60 + f5(1) = 20 : replace the equipment

For t = 3 :

f3(1) = min {80 + f4(2) = -40*, 1.000 - 800 + 60 + f4(1) = -40* }

⇒ f3(1) = -40: keep or replace the equipment

f3(2) = min {120 + f4(3) = 140*, 1.000 - 600 + 60 + f4(1) = 160 }

⇒ f3(2) = 140: keep the equipment

For t = 1 :

f2(1) = min {80 + f3(2) = 220*, 1.000 - 800 + 60 + f3(1) = 220* }

⇒ f2(1) = 220: keep or replace the equipment

Therefore, f1(0) = 1000 + 60 + f2(1) = 1280.

8/18/2019 CoursOPT-07-08


OPTIMIZATION 3A GI

60

The optimal solutions can be represented as follows:

2 3 4 5

1 3 4 5

4 5

keep the equipment replace the equipment

Note :We have assumed that costs are the same over the years. This is just to simplify the

computation of the costs ctx , which usually depend on t rather than the difference

x-t.

4.6.2 Dynamic pr ogr amming f ormulat ion (wit hout stages)

We define:

g(t) = Minimal cost incurred during periods t, t+1, …, 6, including purchase cost

and salvage, given a new equipment has been purchased at period t.

ctx = the net cost of purchasing a new equipment and using it until year x .

Backward formulation:

g(t) = minx{ctx + g(x)} t = 1, 2, 3, 4, 5

where t+1 ≤ x ≤ t+3

x ≤ 5

Boundary conditions: g(6) = 0

Objective : Find g(1)

ctx depends on the interval x-t

ctx = 1.000 + m1 + … + m x-t - s x-t

c12 = c23 = c34 = c45 = c56 = 1.000 + 60 - 800 = 260 MD

c13 = c24 = c35 = c46 = 1.000 + 60 + 80 - 600 = 540 MD

c14 = c25 = c36 = 1.000 + 60 + 80 + 120 - 500 = 760 MD

8/18/2019 CoursOPT-07-08


OPTIMIZATION 3A GI

61

g(6) = 0

g(5) = c56 + g(6) = 260 MD

g(4) = min{c45 + g(5) ; c46 + g(6)} = min{260+260=520 ; 540+0+540} = 520

g(3) = min{c34 + g(4) ; c35 + g(5) ; c36 + g(6)} = min{780 ; 800 ; 760} = 760

g(2) = min{c23 + g(3) ; c24 + g(4) ; c26 + g(6)} = min{1020 ; 1060 ; 1020} = 1020

g(1) = min{c12 + g(2) ; c13 + g(3) ; c14 + g(4)} = min{1280 ; 1300 ; 1280} = 1280

The optimal solution consists of purchasing a new equipment at t = 1, use it until year

2 or 4, then replace it with a new one.

If we replace it at year 2, we use it until year 3 or 5.If we replace the equipment at year 3 or 5, we keep the newly purchased until year

6.

If we replace the equipment at year 4, we use it until year 5, then use the newly

purchased equipment until year 6.

1 → 2→ 3 → 6 or 1 → 2 → 5 → 6 or 1 → 4 → 5 → 6

The optimal cost is 1280 $.

Exercise: network representation of the problem.

4.7 Equipment reliability

Reliability: it is the probability of good functionning under normal conditions during a

given period.

Consider a system comprising N equipments in series E1, E2, …, EN where each

equipment is characterized by its reliability Ri, and a purchase cost Ci.

The system reliability is R = ∏=

n

i

i R1

E1 E2 EN

E1 EN

EN

8/18/2019 CoursOPT-07-08


OPTIMIZATION 3A GI

62

Figure 4.4 Graphical representation of a reliability problem

In order to increase the reliability of the system, usually redundance is used. The

reliability R i and the cost Ci of each equipment i is dependent on the number of units

(copies) ki of the equipment, placed in parallel.

Given a limited budget, the reliability problem consists of determining the number of

elements to place in parallel, of each equipment so as to maximize the reliability of

the whole system.

Example 4.4

Consider the case of a system having 3 machines placed in series. Suppose that atmost 3 elements can be placed in parallel for each of the machines. The reliability Ri

and the cost Ci (in K$) of each machine i are given in the table below, as a function of

the number of elements palced in parallel:

M1 M2 M3

ki R1 C1 R2 C2 R3 C3

1 0.6 1 0.7 3 0.5 2

2 0.8 2 0.8 5 0.7 43 0.9 3 0.9 6 0.9 5

The available budget is: 10 K$.

How many of each of the machines M1, M2 and M3 should be purchased in order to

maximize the system's reliability, without exceeding the budget ?

4.7.1 Dynamic pr ogr amming f ormulat ion

Max ∏=

3

1

)(n

i

ii k R

S. to ∑=

3

1

)(i

ii k C ≤ b

ki = 1, 2 ou 3

Stage : i : machine

State variable: yi : available budget at the beginning of stage i, for stages i, i+1, …, 3Decision variable: ki: number of machines i

8/18/2019 CoursOPT-07-08


OPTIMIZATION 3A GI

63

Recursive function :

f i(y i) : optimal reliability for stages i, i+1, …, n, if the available budget is yi

fi(yi) = maxki∈{1,2,3} {Ri(ki) * fi+1(yi+1)}

Equation linking the state variables to the decision variables: y i+1 = y i - Ci(ki)

Boundary conditions: y3 ∈ [2, 6]

C3(1): min cost of purchasing one machine M3 10-C1(1)-C2(1)

Objective: f 1(10)

Resolution :

i = 3 : y3 ∈ [2, 6]

y3 f3(y3) = R3(k3) k3

2 0.5 1

3 0.5 1

4 0.7 2

5 0.9 3

6 0.9 4

i = 2 : y2 ∈ [5, 9] = [C2(1)+C3(1), 10-C1(1)]

y2 f2(y2) k2, k3

5 R2(1)*f3(2) = 0.7*0.5 = 0.35 1, 1

6 R2(1)*f3(3) = 0.7*0.5 = 0.35 1, 1

7 Max {R2(1)*f 3(4) ; R2(2)*f3(2)} =

Max {0.49 ; 0.4} = 0,49

1, 2

8 Max {R2(1)*f 3(5) ; R2(2)*f3(3) ; R2(3)*f3(2)} = Max {0.63 ; 0.4 ; 0.45} =

0,63

1, 3

9 Max {R2(1)*f 3(6) ; R2(2)*f3(4) ; R2(3)*f3(3)} = Max {0.63 ; 0.56 ; 0.45} =

0,63

1, 3

8/18/2019 CoursOPT-07-08


OPTIMIZATION 3A GI

64

i = 1 : y1 ∈ [6, 10] = [C1(1)+C2(1)+C3(1), 10]

y1 f1(y1) k1, k2, k3

6 R1(1)*f2(5) = 0.6*0.35 = 0.21 1, 1, 1

7 Max {R1(1)*f2(6) ; R1(2)*f 2(5)} =

Max {0.21 ; 0.28} = 0,28

2, 1, 1

8 Max {R1(1)*f2(7) ; R1(2)*f2(6) ; R1(3)*f 2(5)} =

Max {0.294 ; 0.28, 0.315} = 0,315

3, 1, 1

9 Max {R1(1)*f2(8) ; R1(2)*f 2(7) ; R1(3)*f3(6)} = Max {0.378 ; 0.392 ;

0.315} = 0,392

2, 1, 2

10 Max {R1(1)*f2(9) ; R1(2)*f 2(8) ; R1(3)*f3(7)} = Max {0.378 ; 0.504 ;

0.441} = 0,504

2, 1, 3

f1(10) = 0,504

The optimal solution consists of using 2 machines M1, 1 machine M2 and 3 machines

M3. The maximal reliability with a budget of 10 kD is equal = 0,504.

Note:

Knowing the initial state, we can also use a forward formulation. To do this, we

define yi = the amount of money spent during the stages 1,…, i.

The stage and the decision variables being the same, the recursive function becomes:

fi(yi) : the optimal reliability for stages 1,…, i, if the amount spent is yi .

fi(yi) = maxki∈{1,2,3} {Ri(ki) * fi-1(yi-1)}

The equation linking the state variables to the decision variables is: y i = y i-1 + Ci(ki)

Boundary conditions : y 1 ∈ [1, 3]

Objective: maxy3 f3(y3) with y3 ∈ [6, 10] = [C1(1)+C2(1)+C3(1), 10]

4.8 Summary of dynamic programming general aspects

In general, in dynamic programming, a stage is composed of all states possible of the

system being in that stage.

Typically, a backward formulation is such that:

8/18/2019 CoursOPT-07-08


OPTIMIZATION 3A GI

65

ft(i) = minimal cost from stage t until the last stage (T), given that at stage t,

the system is state i.

And, for a forward formulation:

ft(i) = minimal cost from stage 1 until the last stage t, given that at stage t, the

system is state i.

We can identify 3 different aspects of dynamic programming:

Aspect 1 : the set of possible decisions for a given state and a stage:

For the production problem:

If st-1 is the stock level at the beginning of month t, the set of possibledecisions at stage t is xt such that :

0 ≤ x t ≤ 5 and 0 ≤ st-1 + x t - d t ≤ 4 ⇒ - st-1 + d t ≤ x t ≤ 4 - st-1 + d t

For the equipment replacement problem:

If xt is the age of the equipment at stage t, the set of possible decisions at this

stage is:

Replace or keep the equipment if 1 ≤ x t < 3Replace the equipment if x t = 3

For the reliability problem:

If yt is the available budget at the beginning of stage t, the set of possible

decisions at stage kt such that:

1 ≤ kt ≤ 3 and ∑+=

T

1ti

)1(iC ≤ y t - Ct(kt) ≤ 10 - ∑=

t

1i

)1(iC

Aspect 2 : We have to specify how the cost during the current stage t depends on the

value of t, the current state and the decision taken at that stage: c(t) = f(st-1, x t)

For the production problem: gt(x t, y t, st-1) = ct y t + pt x t + ht(st-1 + x t - d t)

For the equipment replacement problem: c tx = 1.000 + m 1 + … + m x-t - s x-t

8/18/2019 CoursOPT-07-08


OPTIMIZATION 3A GI

66

For the reliability problem: R(i, ki) = Ri(ki)

Aspect 3 : We have to specify how the state of stage t depend on the value of t, the

state at the previous stage t-1 and the decision take at stage t.

For the production problem: st = st-1 + x t - d t

For the equipment replacement problem:

If the decision is to replace the equipment: x t+1 = 1

If the decision is to keep the equipment: x t+1 = x t + 1

For the reliability problem: y i+1 = y i - Ci(ki)

Notes:

Not all aspects are necessarily found in all recursions. very often, when the stage

contains sufficient information, it is used to define the state (ex: equipment

replacement problem).

For problems with a large number of states, dynamic programming becomes less

efficient than Integer programming techniques such as Branch-and-Bound.

dynamic programming is, in general, efficient for low dimensional problems.There are techniques that make solving high dimensional problems possible with

dynamic programming (e.g. turnpike theorem for the knapsack problem).

8/18/2019 CoursOPT-07-08


OPTIMIZATION 3A GI

67

References

[1] D. Bertsekas, 2001. Dynamic Programming and Optimal Control, Athena

Scientific, 2nd Edition.

[2] E. Denardo, 1982. Dynamic Programming : Theory and Applications, Prentice

Hall.

[3] F. S. Hillier and G. J. Lieberman, 2001. Introduction to Operations Research.

Holden-Day, Inc.

[4] K.G. Murty, 1995. Operations Research: Deterministic Optimization Models.

Prentice-Hall, Englewood Cliffs, New Jersey.

[5] K.G. Murty, 1992. Network Programming. Prentice-Hall.

[6] G. L. Nemhauser and L. A. Wolsey, 1988. Integer and Combinatorial

Optimization. Wiley Interscience.

[7] W. L. Winston, 1994. Operations Research : Applications and Algorithms,

Duxbury press.

8/18/2019 CoursOPT-07-08


OPTIMIZATION 3A GI

coursopt-07-08

Documents