level ii, term ii cse – 243 md. monjur-ul-hasan lecturer dept of cse, cuet email:...

LEVEL II, TERM IICSE – 243

MD. MONJUR-UL-HASANLECTURER

DEPT OF CSE, CUETEMAIL: [email protected]

Recursive & Dynamic Programming

Introduction to Recursion

Md. Monjur-ul-hasan. Lecturer, Dept. of CSE, CUET

2

"Normally", we have methods that call other methods. For example, the main() method calls the square()

method.

Recursive Method: A recursive method is a method that calls itself.

main()

square()

compute()

Why Recursive


3

In computer science, some problems are more easily solved by using recursive methods.

In this course, will see many of examples of this.

For example: Traversing through directories of a file system. Traversing through a tree of search results. Some sorting algorithms recursively sort data

For today, we will focus on the basic structure of using recursive methods.

Recursive Situation


4

LISTConsider the following list of numbers:

24, 88, 40, 37

Such a list can be defined as follows:

A LIST is a: number or a: number comma LIST

That is, a LIST is defined to be a single number, or a number followed by a comma followed by a LIST

The concept of a LIST is used to define itself

Recursive Situation


5

The recursive part of the LIST definition is used several times, terminating with the non-recursive part:

number comma LIST 24 , 88, 40, 37

number comma LIST 88 , 40, 37

number comma LIST 40 , 37

number 37

Recursive Situation


6

GCDgcd(a,b) = a ; when b

= gcd(b,a%b) ; otherwise

gcd(4032, 1272) = gcd(1272, 216)= gcd(216, 192)= gcd(192, 24)= gcd(24, 0)= 24.

Recursive Situation


7

Finding PowerThe power function, p(x,n)=xn, can be defined

recursively:

This leads to an power function that runs in O(n) time (for we make n recursive calls).

We can do better than this, however.

else)1,(

0 if1),(

nxpx

nnxp

Recursive Situation


8

Recursive SquaringWe can derive a more efficient linearly recursive

algorithm by using repeated squaring:

For example,24 = 2(4/2)2 = (24/2)2 = (22)2 = 42 = 1625 = 21+(4/2)2 = 2(24/2)2 = 2(22)2 = 2(42) = 3226 = 2(6/ 2)2 = (26/2)2 = (23)2 = 82 = 6427 = 21+(6/2)2 = 2(26/2)2 = 2(23)2 = 2(82) = 128.

Better then this solution is also available For Practice Try to solve (bn%P) in this same way for large

number of n

even is 0 if

odd is 0 if

0 if

)2/,(

)2/)1(,(

1

),(2

2

x

x

x

nxp

nxpxnxp

Recursive Situation


9

Reversing an ArrayAlgorithm ReverseArray(A, i, j): Input: An array A and nonnegative integer

indices i and j Output: The reversal of the elements in A

starting at index i and ending at j if i < j then

Swap A[i] and A[ j]ReverseArray(A, i + 1, j - 1)

return

Recursive Definition


10

Base case: You must always have some base case which can be solved without recursion

Making Progress: For cases that are to be solved recursively, the recursive call must always be a case that makes progress toward the base case.

Design Rule: Assume that the recursive calls work.

Compound Interest Rule: Never duplicate work by solving the same instance of a problem in separate recursive calls.

Visualizing Recursion


11

Recursion traceA box for each

recursive callAn arrow from

each caller to callee

An arrow from each callee to caller showing return value

Example recursion trace:

recursiveFactorial(3)




Return 1

call

call

call

call

return 1*1 = 1

return 2*1 = 2

return 3*2 = 6 final answer

elsenfn

nnf

)1(

0 if1)(

Visualizing Recursion


12

Time 2:Push: fact(3)

main()

fact(3)


Inside findFactorial(3):if (number <= 1) return 1;else return (3 *

factorial (2));

main()

fact(3)

fact(2)


Inside findFactorial(2):if (number <= 1) return 1;else return (2 *

factorial (1));

main()

fact(3)

fact(2)

fact(1)

Inside findFactorial(1):if (number <= 1) return 1;else return (1 * factorial

(0));

Time 5:Pop: fact(1)returns 1.

main()

fact(3)

fact(2)1


main()

fact(3)2


main()6

Several Types of Recursion


13

Liner Recursion May be more than one recursive method available but for

each recursion it calls only one among themTail Recursion

The recursive call will be the last statement of the methodChain Recursion

Function A will call Function B. function B will call Function C …… . … last function will call function A again.

Multiple Recursion f(x) = a when x = 0

= b when x = 1 = a*f(x-1) + b*f(x-2) otherwise

Defining Arguments for Recursion


14

In creating recursive methods, it is important to define the methods in ways that facilitate recursion.

This sometimes requires we define additional paramaters that are passed to the method.

For example, we defined the array reversal method as ReverseArray(A, i, j), not ReverseArray(A).

To verify that a recursive definition works: convince yourself that the base case(s) are handled correctly ASSUME RECURSIVE CALLS WORK ON SMALLER PROBLEMS, then

convince yourself that the results from the recursive calls are combined to solve the whole

When recursion?


15

when it is the most natural way of thinking about & implementing a solution can solve problem by breaking into smaller instances, solve,

combine solutions when it is roughly equivalent in efficiency to an iterative

solution OR

when the problems to be solved are so small that efficiency doesn't matter

think only one level deep make sure the recursion handles the base case(s) correctly assume recursive calls work correctly on smaller problems make sure solutions to the recursive problems are combined

correctly avoid infinite recursion

make sure there is at least one base case & each recursive call gets closer

Alternative of Recursive


16

any recursive method/class can be rewritten iteratively (i.e., using a loop) but sometimes, a recursive definition is MUCH clearer

e.g., PermutationGenerator would be very difficult to conceptualize & implement without recursion

Use stack to model the recursive into iterative method. [ use the data structure course concept]

When overhead is low it is always better to use recursive but in other case it is better to use iterative solution.

Recursion pros and cons


17

All recursive solutions can be implemented without recursion.

Recursion is "expensive". The expense of recursion lies in the fact that we have multiple activation frames and the fact that there is overhead involved with calling a method.

If both of the above statements are true, why would we ever use recursion?

In many cases, the extra "expense" of recursion is far outweighed by a simpler, clearer algorithm which leads to an implementation that is easier to code.

Ultimately, the recursion is eliminated when the compiler creates assembly language (it does this by implementing the stack).

Think Your Own Recursion


18

Try to solve the problem for base caseThink in the middle of the problem with

following criteria: Define that with same problem but with different

domain The change in domain should leads that to the base

case.

You need not to think all the problem together. Just think one step from middle and the base case.

Generating Permutation Systematically


19

numerous applications require systematically generating permutations (orderings) e.g., word games, debugging concurrent systems, tournament pairings, . . .

want to be able to take some sequence of items (say a String of characters) and generate every possible arrangement without duplicates

"123" "123", "132", "213", "231", "312", "321"

"tape" "tape", "taep", "tpae", "tpea", "teap", "tepa", "atpe", "atep", "apte", "apet", "aetp", "aept", "ptae", "ptea", "pate", "paet", "peta", "peat", "etap", "etpa", "eatp", "eapt", "epta", "epat“

how do we generate permutations systematically?• must maintain initial ordering• must get all permutations, with no duplicates

Generating Permutation Systematically


20

Generate all permutations that start with ‘t' , then 'a' then ‘p‘ then ‘e’

To generate permutations starting with ‘t', we need to find all permutations of "ape"

This is the same problem with simpler inputs. Use recursion

To get your permutation go: http://home.att.net/~srschmitt/script_permutations.html

All Permutation Algorithm


21

consider P is a global array contain the string

exch (int i, int j){ int t = p[i]; p[i] = p[j]; p[j] = t; }

generate(int N){

int c;if (N == 1) doit();for (c = 1; c <= N; c++){ exch(c, N); generate(N-1); exch(c, N); }

} Invoke by calling generate(N);

Challenge to Students (75 Marks)


22

1. Redesign the algorithm with no global variable2. Redesign the algorithm that u develop in 1 to

generate permutation which contain k element from the n length string

3. Redesign the algorithm of describe in 1 to generate all combination from a string by taking k element each time?

4. Design a correct algorithm to generate permutation when the string contain repetitive character in the given string

5. Design a correct Algorithm to generate Combination of k element among n element where the string contain repetitive character.

Marks


23

Fail to submit: -15 from your Lab work Duplicate submission : add absolute 0 with your lab

work Correct solution will add marks as follows

Q1: 5 Q2 & Q2: 10+10 = 20 Q4 20 Q5 30 Total 75This mark will add to you lab result.

Recursive to Iterative


24

What is the problems with this recursion?Is there any direct solution available of this

recursive function?How can we test and find if any such

available.

otherwisexfbxfa

xwhenb

xwhena

xf

)2(*)1(*

1

0

)(

Guess-and-Test Method


25

In the guess-and-test method, we guess a closed form solution and then try to prove it is true by induction:

Guess: T(n) < cn log n.

Wrong: we cannot make this last line be less than cnlogn

2iflog)2/(2

2if )(

nnbnnT

nbnT

nbncnncn

nbnncn

nbnnnc

nbnnTnT

loglog

log)2log(log

log))2/log()2/((2

log)2/(2)(

Guess-and-Test Method, Part 2


26

Recall the recurrence equation:

Guess #2: T(n) < cn log2 n.

if c > b. So, T(n) is O(n log2 n). In general, to use this method, you need to have a good guess

and you need to be good at induction proofs.

2iflog)2/(2

2if )(

nnbnnT

nbnT

ncn

nbncnncnncn

nbnncn

nbnnnc

nbnnTnT

2

2

2

2

log

loglog2log

log)2log(log

log))2/(log)2/((2

log)2/(2)(

Master Method


27

Many divide-and-conquer recurrence equations have the form:

The Master Theorem:

dnnfbnaT

dncnT

if)()/(

if )(

.1 somefor )()/( provided

)),((is)(then),(is)(if 3.

)log(is)(then),log(is)(if 2.

0),(is)(hen),(is)(if 1.

log

1loglog

loglog

nfbnaf

nfnTnnf

nnnTnnnf

wherennTtnOnf

a

kaka

aa

b

bb

bb

Master Method, Example 1


28

The form:

The Master Theorem:

Example:

Solution: logba=2, so case 1 says T(n) is O(n2).

dnnfbnaT

dncnT

if)()/(

if )(




)(is)(then),(is)(if 1.

log

1loglog

loglog

nfbnaf

nfnTnnf

nnnTnnnf

nnTnOnf

a

kaka

aa

b

bb

bb

nnTnT )2/(4)(



29

The form:

The Master Theorem:

Example:

Solution: logba=1, so case 2 says T(n) is O(n log2 n).

dnnfbnaT

dncnT

if)()/(

if )(





log

1loglog

loglog

nfbnaf

nfnTnnf

nnnTnnnf

nnTnOnf

a

kaka

aa

b

bb

bb

nnnTnT log)2/(2)(



30

The form:

The Master Theorem:

Example:

Solution: logba=0, so case 3 says T(n) is O(n log n). As f(n) = nlogn is asymptotically larger then nlog

ba =n

dnnfbnaT

dncnT

if)()/(

if )(





log

1loglog

loglog

nfbnaf

nfnTnnf

nnnTnnnf

nnTnOnf

a

kaka

aa

b

bb

bb

nnnTnT log)3/()(

Divide & Conquer Algorithm


31

Divide problem into sub-problemsConquer by solving sub-problems recursively. If

the sub-problems are small enough, solve them in brute force fashion

Combine the solutions of sub-problems into a solution of the original problem (tricky part)

Example

MergeSortFinding the middle point in the alignment matrix in linear spaceLinear space

sequence alignmentBlock AlignmentFour-Russians speedupConstructing LCS in sub-quadratic time

Dynamic Programming


32

fibonacci(5)

fibonacci(4) + fibonacci(3)fibonacci(3) + fibonacci(2) fibonacci(2) + fibonacci(1)

fibonacci(2) + fibonacci(1)

Dynamic Programming is an algorithm design technique for optimization problems: often minimizing or maximizing.

Like divide and conquer, DP solves problems by combining solutions to subproblems.

Unlike divide and conquer, subproblems are not independent.

Subproblems may share subsubproblems, However, solution to one subproblem may not affect the

solutions to other subproblems of the same problem. (More on this later.)

Dynamic Programming: How to think?


33

DP reduces computation by Solving subproblems in a bottom-up fashion. Storing solution to a subproblem the first time it is solved. Looking up the solution when subproblem is encountered

again.Key: determine structure of optimal solutions

Elements of Dynamic ProgrammingOptimal substructureOverlapping subproblemsMemorization

Optimizing Substructure


34

Show that a solution to a problem consists of making a choice, which leaves one or more subproblems to solve.

Suppose that you are given this last choice that leads to an optimal solution.

Given this choice, determine which subproblems arise and how to characterize the resulting space of subproblems.

Show that the solutions to the subproblems used within the optimal solution must themselves be optimal. Usually use cut-and-paste.

Need to ensure that a wide enough range of choices and subproblems are considered.

Optimal SubStructure


35

Optimal substructure varies across problem domains: 1. How many subproblems are used in an optimal solution. 2. How many choices in determining which subproblem(s) to

use.Informally, running time depends on (# of

subproblems overall) (# of choices).How many subproblems and choices do the examples

considered contain?Dynamic programming uses optimal substructure

bottom up. First find optimal solutions to subproblems. Then choose which to use in optimal solution to the problem.

Pieces of larger problem have a sequential dependency

Overlapping Subproblems


36

The space of subproblems must be “small”.The total number of distinct subproblems is

a polynomial in the input size. A recursive algorithm is exponential because it

solves the same problems repeatedly. If divide-and-conquer is applicable, then each

problem solved will be brand new.

DP Approach


37

Forward ApproachBackward Approach

Note that if the recurrence relations are formulated using the forward approach then the relations are solved backwards . i.e., beginning with the last decision

On the other hand if the relations are formulated using the backward approach, they are solved forwards.

You can also represent the problem by a multistage graph

A Simple DP Approach


38

Fibonacci NumbersInt fib(int n){

static int knownFib[MAXFIB];int x;if(knownFib[n]==0){

if(n==1 || n==2)knownFib[n] = 1;

else knownFib[n] = fib(n-1) + fib(n-2);}

}

Matrix-chain multiplication


39

What will be the number of scalar multiplication in a matrix multiplication of a 3X5 and 5X 10 matrix?

What will be new matrix’s direction?What is the number of scalar multiplication of the

following matrices A1 X A2 X A3 X A4

Where, the damnations are 3x5, 5x4, 4x2 and 2x5

We have the following choice:((A1 A2) A3) A4, # of scalar multiplications:

3 * 5 * 4 + 3 * 4 * 2 + 3 * 2 * 5 = 114(A1 (A2 A3)) A4, # of scalar multiplications:

3 * 5 * 2 + 5 * 4 * 2 + 3 * 2 * 5 = 100(A1 A2) (A3 A4), # of scalar multiplications:

3 * 5 * 4 + 3 * 4 * 5 + 4 * 2 * 5 = 160

Recursive Function


40

8 -40

Let m(i, j) denote the minimum cost for computing Ai Ai+1 … Aj

Computation sequence :

Time complexity : O(n3)

jiif

jiifpppj)1,m(kk)m(i,min

0j)m(i,

ik1ijki


41

Since subproblems overlap, we don’t use recursion. Instead, we construct optimal subproblems “bottom-up.” Ni,i’s are easy, so start with them. Then do length 2,3,… subproblems,and so on. Running time: O(n3)

Matrix Chan Multiplication


42

Algorithm matrixChain(S):Input: sequence S of n matrices to be multipliedOutput: number of operations in an optimal

paranethization of S1.for i ← 1 to n-1 do

1. Ni,i ← 02.for b ← 1 to n-1 do

1. for i ← 0 to n-b-1 do1. j ← i+b2. Ni,j ← +infinity3. for k ← i to j-1 do

1. Ni,j ← min{Ni,j , Ni,k +Nk+1,j +di dk+1 dj+1}2. Si,j ← k //for which Ni,j Minimum

Matrix Chan Multiplication


43

Print the Optimal Structure

OPTIMAL-MAT(s,i,j){

if(i==j)then print “A”I

else print “(“OPTIMAL-MAT(s,i,s[i,j)OPTIMAL-MAT(s,s[i,j]+1, j)print “)”;

}

Longest Common Subsequence (LCS)


44

Problem: Given 2 sequences, X = x1,...,xm and Y = y1,...,yn, find a common subsequence whose length is maximum.

springtime ncaa tournamentbasketball

printing north carolina krzyzewski

Subsequence need not be consecutive, but must be in order.

LCS (Native Algorithm)


45

For every subsequence of X, check whether it’s a subsequence of Y .

Time: Θ(n2m). 2m subsequences of X to check. Each subsequence takes Θ(n) time to check:

scan Y for first letter, for second, and so on.

LCS: Optimal Substructure


46

Notation:

prefix Xi = x1,...,xi is the first i letters of X.

This says what any longest common subsequence must look like; do you believe it?

Theorem

Let Z = z1, . . . , zk be any LCS of X and Y .

1. If xm = yn, then zk = xm = yn and Zk-1 is an LCS of Xm-1 and Yn-1.

2. If xm yn, then either zk xm and Z is an LCS of Xm-1 and Y .

3. or zk yn and Z is an LCS of X and Yn-1.

Theorem

Let Z = z1, . . . , zk be any LCS of X and Y .

1. If xm = yn, then zk = xm = yn and Zk-1 is an LCS of Xm-1 and Yn-1.

2. If xm yn, then either zk xm and Z is an LCS of Xm-1 and Y .

3. or zk yn and Z is an LCS of X and Yn-1.

Recursive Solution


47

.)end( )end( if]),[],,[max(

,)end( )end( if1],[

,empty or empty if0

],[

prefixcprefixc

prefixprefixcc


,)end( )end( if1],[

,empty or empty if0

],[

prefixcprefixc

prefixprefixcc

c[springtime, printing]

c[springtim, printing] c[springtime, printin]

[springti, printing] [springtim, printin] [springtim, printin] [springtime,

printi]

[springt, printing] [springti, printin] [springtim, printi] [springtime, print]

Memorization


48


,)end( )end( if1],[

,empty or empty if0

],[

prefixcprefixc

prefixprefixcc


,)end( )end( if1],[

,empty or empty if0

],[

prefixcprefixc

prefixprefixcc

p r i n t i n g

s

p

r

i

n

g

t

i

m

e

•Keep track of c[] in a table of nm entries:

•top/down

•bottom/up

LCS: Length Finding Algorithm


49

LCS-LENGTH (X, Y)1. m ← length[X]2. n ← length[Y]3. for i ← 1 to m4. do c[i, 0] ← 05. for j ← 0 to n6. do c[0, j ] ← 07. for i ← 1 to m8. do for j ← 1 to n9. do if xi = yj10. then c[i, j ] ← c[i1, j1] + 111. b[i, j ] ← “ ”12. else if c[i1, j ] ≥ c[i, j1]13. then c[i, j ] ← c[i 1, j ]14. b[i, j ] ← “↑”15. else c[i, j ] ← c[i, j1]16. b[i, j ] ← “←”17.return c and b


50

p r i n t i n g

s

P

r

i

n

g

t

i

m

e

p r i n t i n g

0 0 0 0 0 0 0 0 0

s 0 0 0 0 0 0 0 0 0

p 0 1 1 1 1 1 1 1 1

r 0 1 2 2 2 2 2 2 2

i 0 1 2 3 3 3 3 3 3

n 0 1 2 3 4 4 4 4 4

g 0 1 2 3 4 4 4 4 5

t 0 1 2 3 4 5 5 5 5

i 0 1 2 3 4 5 6 6 6

m 0 1 2 3 4 5 6 6 6

e 0 1 2 3 4 5 6 6 6

LCS: Constructing


51

PRINT-LCS (b, X, i, j)1. if i = 0 or j = 02. then return3. if b[i, j ] = “ ”4. then PRINT-LCS(b, X, i1, j1)5. print xi

6. elseif b[i, j ] = “↑”7. then PRINT-LCS(b, X, i1, j)8. else PRINT-LCS(b, X, i, j1)

level ii, term ii cse – 243 md. monjur-ul-hasan lecturer dept of cse, cuet email:...

Documents

recursive situation

hasan lecturer dept

recursive methods

recursive algorithm

recursive definition

recursive squaring

n recursive calls

master method