recursion in this chapter we study: recursion divide and conquer dynamic programming
TRANSCRIPT
![Page 1: Recursion In this Chapter we Study: Recursion Divide and Conquer Dynamic Programming](https://reader037.vdocuments.mx/reader037/viewer/2022102819/56649eb45503460f94bbcbff/html5/thumbnails/1.jpg)
Recursion
In this Chapter we Study:
• Recursion
• Divide and Conquer
• Dynamic Programming
![Page 2: Recursion In this Chapter we Study: Recursion Divide and Conquer Dynamic Programming](https://reader037.vdocuments.mx/reader037/viewer/2022102819/56649eb45503460f94bbcbff/html5/thumbnails/2.jpg)
Recursion involves:• Self-reference (as in a recursive definition, or in a recursive method)
• Each recursive call to a function should conduct to a problem of smaller size.
• The chain of self-reference is terminated by a base case, in which the solution to the problem is trivial.
General form of a recursion function:
if (Condition for which the problem is trivial) // fundamental case Trivial solution
else // general caseRecursive call to the function for a smaller case
![Page 3: Recursion In this Chapter we Study: Recursion Divide and Conquer Dynamic Programming](https://reader037.vdocuments.mx/reader037/viewer/2022102819/56649eb45503460f94bbcbff/html5/thumbnails/3.jpg)
)!1(
1!
nnn
if n = 0
if n > 0
Example 1:
The factorial of a non-negative integer, n, is defined as follows:
2/2/
2/2/
x
nn
nnn
xx
xxx
if n = 1
if n is odd
else
Example 2:
The power n of a number x is defined as follows:
Let us illustrate this with some examples.
x
![Page 4: Recursion In this Chapter we Study: Recursion Divide and Conquer Dynamic Programming](https://reader037.vdocuments.mx/reader037/viewer/2022102819/56649eb45503460f94bbcbff/html5/thumbnails/4.jpg)
Example 3:The GCD of two non-negative integers, neither of them 0,
is defined as follows:
m if n = 0gcd(m, n) gcd(n, m mod n) if n > 0
Example 4:The product of two positive integers m and n
is defined as follows:
m if n = 1P(m, n) m + P(m, n-1)if n > 1
![Page 5: Recursion In this Chapter we Study: Recursion Divide and Conquer Dynamic Programming](https://reader037.vdocuments.mx/reader037/viewer/2022102819/56649eb45503460f94bbcbff/html5/thumbnails/5.jpg)
Example 5: Towers of Hanoi
A classical example of a problem which is nicely solved by recursion is:
We have:n disks in order of decreasing size, which are stacked on a peg,
say A.
We want to:Move these disks to another peg, say B,
so that they are in the same decreasing order of size.
using a third peg, say C, as workspace, andunder the following rules:
• We can move only one disk at a time.
• We cannot place a larger disk on top of a smaller one in any move.
![Page 6: Recursion In this Chapter we Study: Recursion Divide and Conquer Dynamic Programming](https://reader037.vdocuments.mx/reader037/viewer/2022102819/56649eb45503460f94bbcbff/html5/thumbnails/6.jpg)
A B C
A B C
A B C
![Page 7: Recursion In this Chapter we Study: Recursion Divide and Conquer Dynamic Programming](https://reader037.vdocuments.mx/reader037/viewer/2022102819/56649eb45503460f94bbcbff/html5/thumbnails/7.jpg)
Recursive Solution
Move (n-1, A, C, B) +Move (1, A, B, C) +Move (n-1, C, B, A)
Move (n, A, B, C)
how many? from to workspace
![Page 8: Recursion In this Chapter we Study: Recursion Divide and Conquer Dynamic Programming](https://reader037.vdocuments.mx/reader037/viewer/2022102819/56649eb45503460f94bbcbff/html5/thumbnails/8.jpg)
Example 6:
Another nice example of recursion:already discussed in Stacks.
In fact there is a great similarity between stack and recursion!
Involves printing the digits of a number n, say n = 1234from right to left, say 4321
What the solution is?
We can extract the digits in reverse order in which we want to print them
![Page 9: Recursion In this Chapter we Study: Recursion Divide and Conquer Dynamic Programming](https://reader037.vdocuments.mx/reader037/viewer/2022102819/56649eb45503460f94bbcbff/html5/thumbnails/9.jpg)
Recursive Solution
Algorithm RecursiveReverseNumber
Input: A non-negative number, n
Output: The digits of n in reverse order.
procedure printDecimal(int n) Print n mod 10 if (n >= 10) printDecimal(n div 10)
What if we want to print digits in the same order?Idea: use a stack
![Page 10: Recursion In this Chapter we Study: Recursion Divide and Conquer Dynamic Programming](https://reader037.vdocuments.mx/reader037/viewer/2022102819/56649eb45503460f94bbcbff/html5/thumbnails/10.jpg)
Implementation of Recursion
Recursion is implemented by using a stack of activation records.
Each activation record stores all parametersrelated to the present call, so thatit can be completed when we return to this call.
Why a stack?Because calls are completed in the reverse order in which they are generated.
example: compute the sum of the integer from 1 to n
1 + 2 + . . . + n
That can be also write:
n + sum of the integer from 1 to (n - 1), ~ n + 1 + 2 + . . . + (n - 1) or n + Sum(n - 1)
![Page 11: Recursion In this Chapter we Study: Recursion Divide and Conquer Dynamic Programming](https://reader037.vdocuments.mx/reader037/viewer/2022102819/56649eb45503460f94bbcbff/html5/thumbnails/11.jpg)
Return 4 + Sum(3) = 4 + 6 = 10
call 1:Sum(4)
Return 3 + Sum(2) = 3 + 3 = 6
call 2:Sum(3)
Return 2 + Sum(1)
= 2 + 1 = 3 call 3: Sum(2)
n==1 Return 1
call 4: Sum(1)
4
34
234
1234
int Sum (int n) { if ( n == 1) // fondamental case
return 1 ; else // general case
return (n + Sum(n -1)) ;}
1234
![Page 12: Recursion In this Chapter we Study: Recursion Divide and Conquer Dynamic Programming](https://reader037.vdocuments.mx/reader037/viewer/2022102819/56649eb45503460f94bbcbff/html5/thumbnails/12.jpg)
Proof of correctness of recursive algorithms
The proof of the correctness of a recursive algorithm is often aproof by induction.
Formulating a proof by induction
1 State the proposition P(n) and the range of n for which you are trying to prove the proposition.
2 Verify the base case: that is, to verify that P(n) is true for the smallest value of n in the range.
3 Formulate the inductive hypothesis: that is, P(k) is true (for k < n).
4 Prove the induction step: that is, proving the induction hypothesis is true for the next value k + 1 of n.We prove the following:if P(k) is true, then P(k + 1) is also true.
5 Conclude that P(n) is true for all n in the stated range.
![Page 13: Recursion In this Chapter we Study: Recursion Divide and Conquer Dynamic Programming](https://reader037.vdocuments.mx/reader037/viewer/2022102819/56649eb45503460f94bbcbff/html5/thumbnails/13.jpg)
Example 1:
![Page 14: Recursion In this Chapter we Study: Recursion Divide and Conquer Dynamic Programming](https://reader037.vdocuments.mx/reader037/viewer/2022102819/56649eb45503460f94bbcbff/html5/thumbnails/14.jpg)
Example 2:
![Page 15: Recursion In this Chapter we Study: Recursion Divide and Conquer Dynamic Programming](https://reader037.vdocuments.mx/reader037/viewer/2022102819/56649eb45503460f94bbcbff/html5/thumbnails/15.jpg)
Correctness of Factorial algorithm
The factorial of an integer n is defined as follows: 0! = 1n! = 1 × 2 × 3 × 4 × . . . × n for all n 1
A recursive method to calculate the factorial of n:
public static int factorial(int n){ If (n==0)return 1; else return n*factorial(n-1);}
In order to prove the correctness of the factorial algorithm, we need to prove the following proposition:
n! = n × (n − 1)! is true for all n 1.
![Page 16: Recursion In this Chapter we Study: Recursion Divide and Conquer Dynamic Programming](https://reader037.vdocuments.mx/reader037/viewer/2022102819/56649eb45503460f94bbcbff/html5/thumbnails/16.jpg)
Proof by induction
1 Prove of the previous proposition.
2 Verify the base case: 1! = 1 × (1 − 1)! = 1 The above proposition is true for n = 1.
3 Induction hypothesis: k! = k × (k − 1)! true for n = k.
4 Prove induction step: prove that (k + 1)! = (k + 1) × k!.Proof:
5 Conclusion: the propositionn! = n × (n − 1)! is true for all n 1.
![Page 17: Recursion In this Chapter we Study: Recursion Divide and Conquer Dynamic Programming](https://reader037.vdocuments.mx/reader037/viewer/2022102819/56649eb45503460f94bbcbff/html5/thumbnails/17.jpg)
Considering the following problem:
Compute the value of the Fibonacci numbers : 1, 1, 2, 3, 5, 8, 13, 21, 34 …
Recursive definition:
Recursive implementationint fib(int n) { if (n==1 || n==2) return 1; else if (n>2) return fib(n-1)+ fib(n-2);}
)2()1(
1)(
nFibnFibnFib
if n > 1
if n=1 or n=2
int Iterativefib( int n){int prev1=1, prev2=1; current=1; for (int i=3; i<=n; i++){ current= prev1+prev2; prev2= prev1; prev1= current; } return current;}
![Page 18: Recursion In this Chapter we Study: Recursion Divide and Conquer Dynamic Programming](https://reader037.vdocuments.mx/reader037/viewer/2022102819/56649eb45503460f94bbcbff/html5/thumbnails/18.jpg)
The recursive solution for the Fibonacci number is simple but highly inefficient!
• The tree of function call exponentially grow.• Many call with the same parameters!• The iterative solution is much more efficient.
If n = 35 then:The iterative program performs 33 additions.The recursive program performs 18.5 million calls!!!
![Page 19: Recursion In this Chapter we Study: Recursion Divide and Conquer Dynamic Programming](https://reader037.vdocuments.mx/reader037/viewer/2022102819/56649eb45503460f94bbcbff/html5/thumbnails/19.jpg)
Complexity of the recursive algorithm
• The recursive algorithm fiboRec that calculates the nth Fibonacci number is exponential: it runs O(2n) time-complexity.
• The proof to the exponential complexity of requires proving the 2 following propositions:
• Proposition 1fiboRec(n) performs fn − 1 additions before it terminates (fn is the nth Fibonacci number).
• Proposition 2The number of additions fn − 1 performed by fiboRec(n) is greater than some exponential function of n.
![Page 20: Recursion In this Chapter we Study: Recursion Divide and Conquer Dynamic Programming](https://reader037.vdocuments.mx/reader037/viewer/2022102819/56649eb45503460f94bbcbff/html5/thumbnails/20.jpg)
Proof of proposition 1:
![Page 21: Recursion In this Chapter we Study: Recursion Divide and Conquer Dynamic Programming](https://reader037.vdocuments.mx/reader037/viewer/2022102819/56649eb45503460f94bbcbff/html5/thumbnails/21.jpg)
Proof of proposition 2:
![Page 22: Recursion In this Chapter we Study: Recursion Divide and Conquer Dynamic Programming](https://reader037.vdocuments.mx/reader037/viewer/2022102819/56649eb45503460f94bbcbff/html5/thumbnails/22.jpg)
Proof of proposition 2:
The complexity of the iterative algorithm for Fibonacci is O(n) !
![Page 23: Recursion In this Chapter we Study: Recursion Divide and Conquer Dynamic Programming](https://reader037.vdocuments.mx/reader037/viewer/2022102819/56649eb45503460f94bbcbff/html5/thumbnails/23.jpg)
Recursion or iteration?
• In general there exists an iterative and an recursive solution for each problem.
• A recursive approach is a good solution when the problem is intrinsically recursive.
In this case iterative solutions are much more complicate (ex. Towers of Hanoi).
• In general iterative solutions are more efficient than recursive ones.
• Redundancy of computation (ex. Fibonacci).
• Recursion imply number function calls.
• Each function call imply transfer of parameters, copy of the state of the program…
• Recursive call need a lot of memory.
![Page 24: Recursion In this Chapter we Study: Recursion Divide and Conquer Dynamic Programming](https://reader037.vdocuments.mx/reader037/viewer/2022102819/56649eb45503460f94bbcbff/html5/thumbnails/24.jpg)
Proof by induction that if n ≥ 5, n2 < 2n.
True for n = 5 : 25 < 32.
Set that k2 < 2k is true for a given k > 5.
Then 2k2 < 2 * 2k = 2k+1
When n > 5, n2 > 5n + 1 > 2n + 1
Then (k + 1)2 = k2 + 2k + 1 < k2 + k2 = 2 k2 < 2k+1
![Page 25: Recursion In this Chapter we Study: Recursion Divide and Conquer Dynamic Programming](https://reader037.vdocuments.mx/reader037/viewer/2022102819/56649eb45503460f94bbcbff/html5/thumbnails/25.jpg)
What is the big-Oh complexity of the following algorithm? What is the return value of the call func(15)?
Func(n) {If ( n ≤ 0 )
Return 10Else
Return Func(n-2) – 2
Func(15) = Func(13) – 2 = Func(11) – 2 – 2 = Func(9) – 2 – 2 - 2 = …Func(-1) – 2 – 2 – 2 – 2 – 2 – 2 – 2 – 2 = 10 – 2 – 2 – 2 – 2 – 2 – 2 – 2 – 2 = - 6
n/2 calls to subtractions then complexity of O(n)
![Page 26: Recursion In this Chapter we Study: Recursion Divide and Conquer Dynamic Programming](https://reader037.vdocuments.mx/reader037/viewer/2022102819/56649eb45503460f94bbcbff/html5/thumbnails/26.jpg)
Proof by induction that for n ≥ 0.
True for n = 0 : 0 = 0
Assume is true for a given n
1 + 4 + 9 + … + n2 + (n+1)2 =
=
=
We have also (n+2)(2(n+1) +1) = (n+2)(2n+3) = 2n2 + 7n + 6
Then
6
)12)(1(
0
2 nnn
in
6
)12)(1(
0
2 nnn
in
2)1(6
)12)(1(
n
nnn
6
))1(6)12()(1( nnnn
6
)672)(1( 2 nnn
6
)1)1(2)(2)(1(1
0
2
nnni
n
![Page 27: Recursion In this Chapter we Study: Recursion Divide and Conquer Dynamic Programming](https://reader037.vdocuments.mx/reader037/viewer/2022102819/56649eb45503460f94bbcbff/html5/thumbnails/27.jpg)
Divide-and-Conquer
An important paradigm…in algorithm design
Divide:It recursively breaks down a problem into two (or more) sub-problems of the same (or related) type, until these become simple enough to be solved directly (base case).
Conquer:The solutions to the sub-problems are then combined to
give a solution to the original problem.
![Page 28: Recursion In this Chapter we Study: Recursion Divide and Conquer Dynamic Programming](https://reader037.vdocuments.mx/reader037/viewer/2022102819/56649eb45503460f94bbcbff/html5/thumbnails/28.jpg)
Example: Linear and Binary search
Linear search
Given an unsorted array, find the index position of a value x in the array.
Algorithm:
Compare x to all values in the array until the value of x isfound or the end of the array is reached.
If x is found in the array return its index position, else return -1.
Complexity:
The algorithm is linear in the worst case.
![Page 29: Recursion In this Chapter we Study: Recursion Divide and Conquer Dynamic Programming](https://reader037.vdocuments.mx/reader037/viewer/2022102819/56649eb45503460f94bbcbff/html5/thumbnails/29.jpg)
low high key indices
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14
array 0 2 4 6 8 10 12 14 16 18 20 22 24 26 28
16 18 20 22 24 26 28
24 26 28
24 NOTE: refer to element of the array that are tested
x = BinarySearch(a, 0, 14, 25);
Binary search
![Page 30: Recursion In this Chapter we Study: Recursion Divide and Conquer Dynamic Programming](https://reader037.vdocuments.mx/reader037/viewer/2022102819/56649eb45503460f94bbcbff/html5/thumbnails/30.jpg)
Binary search
Given a sorted array, find the index position of a value x in the array.
int BinarySearch (const int a[ ] , int low , int high, int key ){ // a [ low . . high ] sorted in increasing order int mid ; if ( low > high ) // fundamental case: not found
return -1; else { mid = (low + high) / 2 ;
if ( a [ mid ] == key ) // fundamental case: found in the middlereturn mid ;
else if ( key < a [ mid ] ) // search in the lower half return BinarySearch ( a, low, mid - 1, key ); else // search in the higher half
return BinarySearch( a, mid + 1, high, key ) ; }} ;
The complexity in the worse case: O(log n)
![Page 31: Recursion In this Chapter we Study: Recursion Divide and Conquer Dynamic Programming](https://reader037.vdocuments.mx/reader037/viewer/2022102819/56649eb45503460f94bbcbff/html5/thumbnails/31.jpg)
An iterative version of the binary search
int BinarySearch (const int a[ ] , int low , int high, int key ) {
// a [ low . . high ] sorted in increasing orderint mid;while ( low <= high ) {
mid = (low + high) / 2 ;if ( a [ mid ] == key ) // found in the middle return mid ;else if ( key < a [ mid ] ) // search in lower half
high = mid - 1 ; else // search in higher half
low = mid + 1 ; } return -1 ; // key not found } ;
In fact, most of the recursive algorithm can be write in an iterative form. The complexity is the same but the program efficiency is greater.
![Page 32: Recursion In this Chapter we Study: Recursion Divide and Conquer Dynamic Programming](https://reader037.vdocuments.mx/reader037/viewer/2022102819/56649eb45503460f94bbcbff/html5/thumbnails/32.jpg)
How to chose the best algorithm for a given problem?
Look at the complexity of each part of the algorithm.
Define how many time each part of the algorithm will be apply.
Example with the linear/binary search:
linear search:
build the array in O(n)
search in the array in O(n)
binary search:
build the array in O(n)
sort the array in O(n log n)
search in the array in O(log n)
So: for a problem
with few searches -> linear search
with lot of searches -> binary search
![Page 33: Recursion In this Chapter we Study: Recursion Divide and Conquer Dynamic Programming](https://reader037.vdocuments.mx/reader037/viewer/2022102819/56649eb45503460f94bbcbff/html5/thumbnails/33.jpg)
Example: Maximum Contiguous Subsequence Problem
a1, a2, … , an/2 | an/2+1, an/2+2, …, an
• Find maximum contiguous subsequence of first half.
• Find maximum contiguous subsequence of second half.
MCS max. of these two?
first half second half
![Page 34: Recursion In this Chapter we Study: Recursion Divide and Conquer Dynamic Programming](https://reader037.vdocuments.mx/reader037/viewer/2022102819/56649eb45503460f94bbcbff/html5/thumbnails/34.jpg)
What if max. contiguous subsequence straddles both halves?
Example:-1, 2, 3, | 4, 5, -6
MCS in first half is2, 3
MCS in second half is4,5
However … the MCS of the given sequence is:2, 3, 4, 5
It straddles both halves!
Observation:For a straddling sequence,
an/2 is the last element in first half, andan/2+1 is the first element in the second half.
![Page 35: Recursion In this Chapter we Study: Recursion Divide and Conquer Dynamic Programming](https://reader037.vdocuments.mx/reader037/viewer/2022102819/56649eb45503460f94bbcbff/html5/thumbnails/35.jpg)
In the example:
Last element = 3First element = 4
Computing a max. straddling sequence:
a1, a2, a3, a4, |a5, a6, a7, a8
Find maximum of:
a4
a4 + a3
a4 + a3 + a2
a4 + a3+ a2+ a1
Let the maximum be max1
![Page 36: Recursion In this Chapter we Study: Recursion Divide and Conquer Dynamic Programming](https://reader037.vdocuments.mx/reader037/viewer/2022102819/56649eb45503460f94bbcbff/html5/thumbnails/36.jpg)
Find the maximum of:
a5
a5 + a6
a5 + a6 + a7
a5 + a6+ a7+ a8
Let the maximum be max2
Required maximum:
max1 + max2
Required subsequence:
Glue subsequence for max1and subsequence for max2
![Page 37: Recursion In this Chapter we Study: Recursion Divide and Conquer Dynamic Programming](https://reader037.vdocuments.mx/reader037/viewer/2022102819/56649eb45503460f94bbcbff/html5/thumbnails/37.jpg)
Complexity of for the straddling sequence
Theorem:For an n-element sequence, the complexity of finding a
maximum straddling sequence is O(n)
Proof:We look at each element exactly once!
Putting things together…
S = a1, a2, … , an/2, an/2+1, an/2+2, …, an
S1 = a1, a2, … , an/2
S2 = an/2+1, an/2+2, …, an
MCS ( S ) = Max ( MCS (S1), MCS (S2), Max (straddling sequence) )
MCS (S1) and MCS (S2) are found recursively.
![Page 38: Recursion In this Chapter we Study: Recursion Divide and Conquer Dynamic Programming](https://reader037.vdocuments.mx/reader037/viewer/2022102819/56649eb45503460f94bbcbff/html5/thumbnails/38.jpg)
Complexity of Recursive MCS
Recursive complexity:
How computing the global complexity?
T(n/2) = time to solve a problem of size n/2
O(n) = complexity of finding a Maximum straddling subsequence
1if,1
)1(1if,)()2(2)(
n
nnOnTnT
![Page 39: Recursion In this Chapter we Study: Recursion Divide and Conquer Dynamic Programming](https://reader037.vdocuments.mx/reader037/viewer/2022102819/56649eb45503460f94bbcbff/html5/thumbnails/39.jpg)
What is a solution to (1)?
Approach 1:
# of levels in the tree of recursive calls?1 + log2 n
O(n) work needed to go from one level of the tree of recursive call to the next higher level.
Therefore, we have a global complexity of:O (n log n)
![Page 40: Recursion In this Chapter we Study: Recursion Divide and Conquer Dynamic Programming](https://reader037.vdocuments.mx/reader037/viewer/2022102819/56649eb45503460f94bbcbff/html5/thumbnails/40.jpg)
n
n / 2 n / 2
n / 4n / 4 n / 4 n / 4
n/8n/8 n/8n/8 n/8n/8 n/8n/8
n
…
log n +1
The tree of recursive calls
22
nn
4444
nnnn
A total of… O (n log n)
![Page 41: Recursion In this Chapter we Study: Recursion Divide and Conquer Dynamic Programming](https://reader037.vdocuments.mx/reader037/viewer/2022102819/56649eb45503460f94bbcbff/html5/thumbnails/41.jpg)
Approach 2:20 T(n) 21 T(n/2) + c n,21 T(n/2) 22 T(n/4) + 2 c n/2,22 T(n/4) 23 T(n/8) + 22 c n/4,23 T(n/8) 24 T(n/16) + 23 c n/8,………
Adding up, we have:20 T(n) + 21 T(n/2) + 22 T(n/4) + 23 T(n/8) + … 21 T(n/2) + 22 T(n/4) + 23 T(n/8) + …+ c n + 2 c n/2 + 22 c n/4 + …
T(n) c n + c n + c n + … log2 n terms
T(n) c n log2 n
Recursive MCS takes…T(n) = O (n log n)
whereas non-recursive MCS takes…T(n) = O (n)
Divide-and-conquer Not always best solution !!
![Page 42: Recursion In this Chapter we Study: Recursion Divide and Conquer Dynamic Programming](https://reader037.vdocuments.mx/reader037/viewer/2022102819/56649eb45503460f94bbcbff/html5/thumbnails/42.jpg)
Another example:
Find both the maximum and minimum of a set of n elements, S
Naive solution:
Find maximum:n – 1 comparisons
Find minimum:n – 2 comparisons
Total: 2n - 3 comparisons
Can we do better??... Yes…
![Page 43: Recursion In this Chapter we Study: Recursion Divide and Conquer Dynamic Programming](https://reader037.vdocuments.mx/reader037/viewer/2022102819/56649eb45503460f94bbcbff/html5/thumbnails/43.jpg)
Algorithm Divide-and-Conquer
procedure maxMin(S)
if |S| = 2 // S = {a,b}return (max(a,b), min(a,b))
elsedivide S into two subsets, say S1 and S2, each with half of
elements.(max1, min1) maxMin(S1)(max2, min2) maxMin(S2)return(max(max1, max2), min(min1, min2))
![Page 44: Recursion In this Chapter we Study: Recursion Divide and Conquer Dynamic Programming](https://reader037.vdocuments.mx/reader037/viewer/2022102819/56649eb45503460f94bbcbff/html5/thumbnails/44.jpg)
Analysis of the divide and conquer algorithm for maxMin
Number of comparisons?
T(n) = # of comparisons on n elementsT(n) = 1 n = 2
= 2T(n/2) + 2, n > 2
Solution:T(4) = 2T(2) + 2 = 4T(8) = 2T(4) + 2 = 10T(16) = 2T(8) + 2 = 22
…T(2k) = 3 2k-1 – 2 (proof by induction)
= 3/2 2k - 2 vs. 2 2k - 3,
where n = 2k
![Page 45: Recursion In this Chapter we Study: Recursion Divide and Conquer Dynamic Programming](https://reader037.vdocuments.mx/reader037/viewer/2022102819/56649eb45503460f94bbcbff/html5/thumbnails/45.jpg)
However…T(n) = O(n) !!!
for both the naive andthe divide-and-conquer algorithm
For the curious!!
Approximately 3n/2 – 2 comparisons are both necessary and sufficient to find the maximum and minimum of a set of n elements.
![Page 46: Recursion In this Chapter we Study: Recursion Divide and Conquer Dynamic Programming](https://reader037.vdocuments.mx/reader037/viewer/2022102819/56649eb45503460f94bbcbff/html5/thumbnails/46.jpg)
Dynamic Programming
Reminder:
The complexity of the recursive Fibonacci algorithm is in O(2n) while the complexity of the iterative one is in O(n)!
Why?
` Because the recursive algorithm perform the same operations a huge number of time.
Idea: Memorizing the value in an array:
Dynamic programming algorithm for Fibonacci numbers:
For i=2 to n { t[i] = t[i-1] + t[i-2]}
4
. . .
0123n
11235
![Page 47: Recursion In this Chapter we Study: Recursion Divide and Conquer Dynamic Programming](https://reader037.vdocuments.mx/reader037/viewer/2022102819/56649eb45503460f94bbcbff/html5/thumbnails/47.jpg)
Essence of this method
• Solving problem method by combination of subproblems solutions.
• Apply to a recursive problem composed of dependent subproblems.
• In fact: a transformation of a recursive algorithm by the use of data structure to store intermediate solutions.
• Each subproblem is solved only one time and is store in an array for later use.
• Can lower the complexity from exponential to polynomial.
• Numerous application for optimization problems.
![Page 48: Recursion In this Chapter we Study: Recursion Divide and Conquer Dynamic Programming](https://reader037.vdocuments.mx/reader037/viewer/2022102819/56649eb45503460f94bbcbff/html5/thumbnails/48.jpg)
Another example of an optimization problem:
Given coins which worth c1, c2, …, cn cents
Make up change for k cents,using the minimum number of coins of the above denominations.
In order that this be always possible, we assume thatc1 = 1
Let min(k) denote the minimum number of coins needed to make k cents of change.
Then we set,min(k) = min { min(r) + min(k-r) }
for all 1 r k/2
if we know min(1), min(2), …, min(k-1)
![Page 49: Recursion In this Chapter we Study: Recursion Divide and Conquer Dynamic Programming](https://reader037.vdocuments.mx/reader037/viewer/2022102819/56649eb45503460f94bbcbff/html5/thumbnails/49.jpg)
Example:
c1 = 1 centc2 = 5 centsc3 = 10 centsc4 = 25 cents
min(1) = 1, and min(2) = min(1) + min(1) = 2We know also min(5) = min(10) = min(25) = 1
Already know the value of min(2), then
min(3) = min(1) + min(2) = 1 + 2 = 3
Again, already know values of min(2) and min(3), so:min(4) = min{min(1) + min(3), min(2) + min(2) } = min {4, 4} = 4
![Page 50: Recursion In this Chapter we Study: Recursion Divide and Conquer Dynamic Programming](https://reader037.vdocuments.mx/reader037/viewer/2022102819/56649eb45503460f94bbcbff/html5/thumbnails/50.jpg)
# of cents Min How solution is found?
1 1 min(1)
2 2 min{min(1)+min(1)}
3 3 min{min(1)+min(2)}
4 4 min{min(1)+min(3), min(2)+min(2)}
5 1 min{min(1)+min(4), min(2)+min(3)}
6 2 min{min(1)+min(5), min(2)+min(4), min(3)+min(3)}
7 3 min{min(1)+min(6), min(2)+min(5), min(3)+min(4)}
![Page 51: Recursion In this Chapter we Study: Recursion Divide and Conquer Dynamic Programming](https://reader037.vdocuments.mx/reader037/viewer/2022102819/56649eb45503460f94bbcbff/html5/thumbnails/51.jpg)
8 4 min{min(1)+min(7),min(2)+min(6), min(3)+min(5), min(4)+min(4)}
9 5 min{min(1)+min(8), min(2)+min(7), min(3)+min(6), min(4)+min(5)}
10 1 min{min(1)+min(9), min(2)+min(8), min(3)+min(7), min(4)+min(6)}
min(11) = min{ 1+10, 2+9, 3+8, 4+7, 5+6}= min{ 2, 7, 7, 7, 3}= 2
1+1=2 2+5=7 3+4=7 1+2=3
![Page 52: Recursion In this Chapter we Study: Recursion Divide and Conquer Dynamic Programming](https://reader037.vdocuments.mx/reader037/viewer/2022102819/56649eb45503460f94bbcbff/html5/thumbnails/52.jpg)
Why don’t we use recursion?Quite inefficient !!
What do we use instead?Dynamic Programming…
The algorithm uses one arrays:coinsUsed: Stores the minimum number of coins needed to
make change of k cents, k = 1,…, maxChange.
coinsUsed[0] 0; for cents 1 to maxChange do // maxChange = k
minCoins centsfor j 1 to diffCoins do // diffCoins = n
if (coins[j] > cents)continue // Cannot use coin j
if (coinsUsed[cents - coins[j]] + 1 < minCoins)minCoins coinsUsed[cents – coins[j]] + 1
coinsUsed[cents] minCoinsPrint “minimum number of coins:”, coinsUsed[maxChange]
![Page 53: Recursion In this Chapter we Study: Recursion Divide and Conquer Dynamic Programming](https://reader037.vdocuments.mx/reader037/viewer/2022102819/56649eb45503460f94bbcbff/html5/thumbnails/53.jpg)
Time Complexity:O ( n k )
where n = number of coins of different denominationsk = amount of change we want to make
![Page 54: Recursion In this Chapter we Study: Recursion Divide and Conquer Dynamic Programming](https://reader037.vdocuments.mx/reader037/viewer/2022102819/56649eb45503460f94bbcbff/html5/thumbnails/54.jpg)
Another example from bioinformatics : Sequence alignment by dynamic programming
• Important problem in bioinformatics: determining the level of similarity between two protein sequences.
• Very useful to discover the function of a new protein.
• A protein sequence is represented by a sequence of characters, each character corresponding to one amino acid.
• Align: to match the higher number of characters between two sequences in order to maximize a similarity score.
• Three possible configurations (match): Identity: A A Substitution (mismatch): A C Insertion or deletion (gap): A - or - A
Example: A T - G G - TA A C - G C T
![Page 55: Recursion In this Chapter we Study: Recursion Divide and Conquer Dynamic Programming](https://reader037.vdocuments.mx/reader037/viewer/2022102819/56649eb45503460f94bbcbff/html5/thumbnails/55.jpg)
• A score value is associated to each configuration.
• The global score is the sum of the score of all matches in the alignment.
Example:
Identity score: 4mismatch score: -1
gap score: -2
A T - G G - TA A C - G C T
Score: 4 - 1 - 2 - 2 + 4 - 2 + 4 = 5
Problem: to compute the best alignment, all the alignments have to be constructed.
![Page 56: Recursion In this Chapter we Study: Recursion Divide and Conquer Dynamic Programming](https://reader037.vdocuments.mx/reader037/viewer/2022102819/56649eb45503460f94bbcbff/html5/thumbnails/56.jpg)
• The solution is defined recursively.
• We define P = p1, p2, …, pn the first sequence and Q = q1, q2, …, qm the second sequence.
• We define F(i,j) the score of the best alignment between p1, p2, …, pi and q1, q2, …, qj.
![Page 57: Recursion In this Chapter we Study: Recursion Divide and Conquer Dynamic Programming](https://reader037.vdocuments.mx/reader037/viewer/2022102819/56649eb45503460f94bbcbff/html5/thumbnails/57.jpg)
Finding the score of i,j
i-
ij
-j
1…i1…j-1
1…i-11…j-1
1…i-11…j
+
+
+
Three ways to buildthe alignment 1…i
1…j
![Page 58: Recursion In this Chapter we Study: Recursion Divide and Conquer Dynamic Programming](https://reader037.vdocuments.mx/reader037/viewer/2022102819/56649eb45503460f94bbcbff/html5/thumbnails/58.jpg)
Cédric Notredame (21/04/23)
Finding the score of i,j
1…i-11…j-1
1…i1…j-1
1…i-11…j
In order to Compute the score of 1…i1…j
All we need are the scores of:
![Page 59: Recursion In this Chapter we Study: Recursion Divide and Conquer Dynamic Programming](https://reader037.vdocuments.mx/reader037/viewer/2022102819/56649eb45503460f94bbcbff/html5/thumbnails/59.jpg)
Cédric Notredame (21/04/23)
Formalizing the algorithm
F(i,j)= best
F(i-1,j) + Gap
F(i-1,j-1) + Mat[i,j]
F(i,j-1) + Gap X-
XX
-X
1…i1…j-1
1…i-11…j-1
1…i-11…j
+
+
+
• The direct application of the recursive formula is in O(cn).
• There is overlap between subproblems => dynamic programming.
![Page 60: Recursion In this Chapter we Study: Recursion Divide and Conquer Dynamic Programming](https://reader037.vdocuments.mx/reader037/viewer/2022102819/56649eb45503460f94bbcbff/html5/thumbnails/60.jpg)
Cédric Notredame (21/04/23)
Arranging Everything in a Table
- F A
-
F
A
S
T
T
1…I-11…J-1
1…I1…J-1
1…I-11…J
1…I 1…J
We use an bidimensional array D to store all the values of the partial alignments
![Page 61: Recursion In this Chapter we Study: Recursion Divide and Conquer Dynamic Programming](https://reader037.vdocuments.mx/reader037/viewer/2022102819/56649eb45503460f94bbcbff/html5/thumbnails/61.jpg)
Cédric Notredame (21/04/23)
Taking Care of the Limits
- F A-FAS
T
T -4Match=2MisMatch=-1Gap=-1
-3
FAT---
-1
F-
-2
FA--
-1F-
-2FA--
-3FAS---
0
![Page 62: Recursion In this Chapter we Study: Recursion Divide and Conquer Dynamic Programming](https://reader037.vdocuments.mx/reader037/viewer/2022102819/56649eb45503460f94bbcbff/html5/thumbnails/62.jpg)
Cédric Notredame (21/04/23)
Filing Up The Matrix
![Page 63: Recursion In this Chapter we Study: Recursion Divide and Conquer Dynamic Programming](https://reader037.vdocuments.mx/reader037/viewer/2022102819/56649eb45503460f94bbcbff/html5/thumbnails/63.jpg)
Cédric Notredame (21/04/23)
- F A
-
F
A
S -3
-2
-1
-1 -2
T
-3
T -4
-2+2
-2 +2
-3-2
+1 +1
-4-3
0 0
+1-2
-3 +10
+4
0 +4-1
0
+3 +30
-3
-4 0+3
0
-1 +3+2
+3
+2 +3-1
-4
-5 -1
+2-1
-2 +2+2
+5
+1 +5
0
![Page 64: Recursion In this Chapter we Study: Recursion Divide and Conquer Dynamic Programming](https://reader037.vdocuments.mx/reader037/viewer/2022102819/56649eb45503460f94bbcbff/html5/thumbnails/64.jpg)
Cédric Notredame (21/04/23)
Delivering the alignment: Trace-back
Score of 1…3 Vs 1…4
Optimal Aln Score
TT
S-
AA
FF
Dynamic programming algorithm complexity in O(n2)
![Page 65: Recursion In this Chapter we Study: Recursion Divide and Conquer Dynamic Programming](https://reader037.vdocuments.mx/reader037/viewer/2022102819/56649eb45503460f94bbcbff/html5/thumbnails/65.jpg)
- G C T C T G C G A A T A
- 0 -2 -4 -6 -8 -10 -12 -14 -16 -18 -20 -22 -24
C -2 -1 0 -2 -4 -6 -8 -10 -12 -14 -16 -18 -20
G -4 0 -2 -1 -3 -5 -4 -6 -8 -10 -12 -14 -16
T -6 -2 -1 0 -2 -1 -3 -5 -7 -9 -11 -10 -12
T -8 -4 -3 1 -1 0 -2 -4 -6 -8 -10 -9 -11
G -10 -6 -5 -1 0 -2 2 0 -2 -4 -6 -8 -10
A -12 -8 -7 -3 -2 -1 0 1 -1 0 -2 -4 -6
G -14 -10 -9 -5 -4 -3 1 -1 3 1 -1 -3 -5
A -16 -12 -11 -7 -6 -5 -1 0 1 5 3 1 -1
T -18 -14 -13 -9 -8 -4 -3 -2 -1 3 4 5 3
A -20 -16 -15 -11 -10 -6 -5 -4 -3 1 5 3 7
C -22 -18 -14 -13 -9 -8 -7 -3 -5 -1 3 4 5
T -24 -20 -16 -12 -11 -7 -9 -5 -4 -3 1 5 3
• Alignment between GCTCTGCGAATA and CGTTGAGATACT (match=2, mismatch = -1 and gap = -2)
G C T C T G C G A - A T A
- C G T T G A G A T A C T