algorithms ch3

8/6/2019 Algorithms Ch3

1/55

AlgorithmsAnalysis, Design and Implementation

Bahman Arasteh

Algorithm Design: Arasteh1

Department of Computer Engineering

Azad University of Tabriz

E-mail: [email protected]


2/55

Chapter 3: Dynamic Programming

The divide-and-conquer approach solves an

instance of a problem

by dividing it into smaller instances and then

blindly solving these smaller instances.

this is a top-town approach.


wor s n pro ems suc as ergesor , w ere esmaller instances are unrelated.

They are unrelated because consists of an array ofkeys that must be sorted independently.


3/55

Dynamic Programming

In problems where the smaller instances are

related, a divide-and-conquer algorithm often

ends up repeatedly solving common instances,

and the result is a very inefficient algorithm.

For example to compute the fifth Fibonacci term


we nee o compu e e our an rFibonacci terms.

However, the determinations of the fourth and

third Fibonacci terms are related in that they both

require the second Fibonacci term.


4/55

Dynamic Programming

Dynamic programming

Dynamic programming is similar to divide-and-

conquer in that an instance of a problem in

divided into smaller instances.

However, in this approach


we solve small instances first,

store the results, and later,

whenever we need a result, look it up instead ofrecomputing it.

Dynamic programming is therefore a bottom-upapproach.


5/55

Dynamic Programming

The steps in the development of a dynamic

programming algorithm are as follows:

1. Establisha recursive property that gives the solutionto an instance of the problem.


. o ve an ns ance o e pro em n a -fashion by solving smaller instances first


6/55

Dynamic Programming

The Binomial Coefficient:

Definition:

n

k

n

k n k for k n

!

!( )!0


Recursive Definition:

n

k

n

k

n

kk n

k or k n

1

1

10

1 0


7/55

Dynamic Programming

The Recursive Algorithm for Binomial Coefficient:



8/55

Dynamic Programming

Analysis of The Recursive Algorithm :



9/55

Dynamic Programming

Analysis of The Recursive Algorithm :



10/55

Dynamic Programming

Binomial Coefficient Using Divide-and-Conquer:



11/55

Dynamic Programming

The steps for constructing a dynamic programming

algorithm for this problem are as follows:1. Establisha recursive property. This has already been done in Equality

3.1. Written in terms of B, it is

B i j B i j B i j j i

[ ][ ][ ][ ] [ ][ ]

1 1 1 0


2. Solve an instance of the problem in a bottom-upfashion by computingthe rows in B in sequence starting with the first row


12/55

Dynamic Programming

The array Bused to compute the binomial coefficient:



13/55

Dynamic Programming

B[4][2] =?

B[0][0] = 1

B[1][0] = 1

B[1][1] = 1


B[2][0] = 1

B[2][1] = B[1][0] + B[1][1] = 1 + 1 = 2

B[2][2] =1

B[3][0] = 1

B[3][1] = B[2][0] + B[2][1] = 1 + 2 = 3

B[3][2] = B[2][1] + B[2][2] = 2 + 1 = 3

B[4][0] = 1

B[4][1] = B[3][0] + B[3][1] = 1 + 3 = 4

B[4][2] = B[3][1] + B[3][2] = 3 + 3 = 6


14/55

Dynamic Programming



15/55

Dynamic Programming



16/55

Dynamic Programming



17/55

Dynamic Programming

number of passes for each value of i:

i 0 1 2 3 k k+1 n

Number of

passes

1 2 3 4 k+1 k+1 k+1


)(

2

)1)(22()1)(1(

2

)1(

)1()1()1(4321

1

nkkkn

kknkk

kkkk

timeskn


18/55

Dynamic Programming

Floyd's Algorithm for Shortest Paths:

Suppose we have a labeled digraph that gives theflying time on certain routes connecting cities, and wewish to construct a table that gives the shortest timerequired to fly from any one city to any other.

We now have an instance of the allpairs shortest


paths (APSP) problem.

To state the problem precisely, we are given a directedgraph G = (V, E) in which each arc v w has a

non-negative cost C[v, w].


19/55

Dynamic Programming




20/55

Dynamic Programming




21/55

Dynamic Programming



D(k)[i][j] = minimum(D(k-1)[i][j], D(k-1)[i][k] + D(k-1)[k][j])


22/55

Dynamic Programming



D(k)[i][j] = minimum(D(k-1)[i][j], D(k-1)[i][k] + D(k-1)[k][j])


23/55

Dynamic Programming




24/55

Dynamic Programming




25/55

Dynamic Programming


D(2)[5][4]D(1)[2][4] = minimum(D(0)[2][4], D(0)[2][1] + D(0)[1][4] )

= minimum(2, 9 + 1) = 2

D(1)[5][2] = minimum(D(0)[5][2], D(0)[5][1] + D(0)[1][2] )


,

D(1)[5][4] = minimum(D(0)[5][4], D(0)[5][1] + D(0)[1][4] )= minimum(, 3 + 1) = 4

D(2)[5][4] = minimum(D(1)[5][4], D(1)[5][2] + D(1)[2][4] )

= minimum(4, 4 + 2) = 4


26/55

Dynamic Programming




27/55

Dynamic Programming

Analysis of Floyd's Algorithm for Shortest Paths:

T(n) = n n n = n3 (n3)



28/55

Dynamic Programming

Example of Previous Graph:



29/55

Dynamic Programming

Example :



30/55

Dynamic Programming

Example :



31/55

Dynamic Programming

Example :



32/55

Dynamic Programming

Print Shortest Path:



33/55

Dynamic Programming

Chained Matrix Multiplication :

Suppose we want to multiply a 2 2 matrix times a 3 4 matrix asfollows:


In general, to multiply an i jmatrix times a j kmatrix using thestandard method, it is necessary to do


34/55

Dynamic Programming

Chained Matrix Multiplication :

Consider the multiplication of the following four matrices:


we have the following number of elementary for each order


35/55

Dynamic Programming

Example: Suppose we have the following six matrices:

A1 A2 A3 A4 A5 A65 2 23 34 46 67 78

d0 d1 d1 d2 d2 d3 d3 d4 d4 d5 d5 d6M[4][6] = ?

To multiply A4, A5, and A6, we have the following two orders and


numbers of elementary multiplications:


36/55

Dynamic Programming

The optimal order for multiplying six matrices must have one of

these factorizations:1. A1 ( A2 A3 A4 A5 A6 )

2. ( A1 A2 ) ( A3 A4 A5 A6 )

3. ( A1 A2 A3 ) ( A4 A5 A6 )

4. ( A1 A2 A3 A4 ) ( A5 A6 )

5. A A A A A A


The number of multiplications for the kth factorization is

M[1][k] + M[k+1][6] + d0 dkd6 We have established that:

)]6][1[]][1[(min]6][1[ 6051 dddkMkMimumM kk


37/55

Dynamic Programming

0]][[

),]][1[]][[(min]][[ 11

iiM

jiifdddjkMkiMimumjiM jkijki



38/55

Dynamic Programming

M[1][2], M[2][3]=?



39/55

Dynamic Programming

M[1][2], M[2][3]=?



40/55

Dynamic Programming



41/55

Dynamic Programming

Analysis:

)()1)(1(

])[( 31

nnnn

diagonaldiagonalnn


1diagonal


42/55

Dynamic Programming

The Traveling Salesperson Problem (TSP):

Suppose a salesperson is planning a sales trip that

includes 20 cities.

Each city is connected to some of the other cities bya road.


,shortest route that starts at the salesperson's home

city, visits each of the cities once, and ends up at thehome city.

This problem of determining a shortest route is

called the Traveling Salesperson problem.


43/55

Dynamic Programming

The Traveling Salesperson Problem (TSP):

A tour(also called a Hamiltonian circuit) in adirected graph is a path from a vertex to itself thatpasses through each of the other vertices exactlyonce.

An o timal tourin a wei hted, directed ra h is


such a path of minimum length.

The Traveling Salesperson problem is to find anoptimal tour in a weighted, directed graph when atleast one tour exists.


44/55

Dynamic Programming

Example (TSP):



45/55

Dynamic Programming

TSP:

We solved this instance by simply considering allpossible tours.

In general, there can be an edge from every vertex toevery other vertex.


,the tour can be any of n 1 vertices, the third vertex on

the tour can be any of n 2 vertices, , the nth vertexon the tour can be only one vertex.

Therefore, the total number of tours is


46/55

Dynamic Programming

TSP:



47/55

Dynamic Programming

TSP:



48/55

Dynamic Programming

TSP:


A={v3}

D[v2][A] = length[v2, v3, v1] =

A={v3, v4}

D[v2][A] = minimum( length[v2, v3, v4, v1], length[v2, v4, v3, v1]

= minimum(20, ) = 20


49/55

Dynamic Programming

TSP:

Optimal Tour length:

}]),{][[]][1[(1

2min vvvimum jj

nj

VDjW


in general for i 1 and vi not in A:

]1][[]][[

}]){][[]][[(]][[ min:

iWD

AifADjiW

v

AD

v

vvimumv

i

jjAj

i

j


50/55

Dynamic Programming

TSP:



51/55

Dynamic Programming

TSP:



52/55

Dynamic Programming

TSP:



53/55

Dynamic Programming

Analysis of TSP:

21

1

n

n

k

nk

nk


1k

nk

k

2

1

1

11

1

111

nn

k

n

k

n

k nk

n

nk

n

nk

n

k


54/55

Dynamic Programming

Analysis of TSP:

Time Complexity:

T(n) = (n-1)(n-2)2n-3 (n22n)Memory Complexity:

M(n) = 2 n2n-1 = n2n (n2n)



55/55

End


algorithms ch3

Documents