![Page 1: Lecture 5 Today, how to solve recurrences We learned “guess and proved by induction” We also learned “substitution” method Today, we learn the “master](https://reader036.vdocuments.mx/reader036/viewer/2022070605/5a4d1ace7f8b9ab0599704b9/html5/thumbnails/1.jpg)
Lecture 5
Today, how to solve recurrences We learned “guess and proved by induction” We also learned “substitution” method Today, we learn the “master theorem”
More divide and conquer: closest pair problem matrix multiplication
![Page 2: Lecture 5 Today, how to solve recurrences We learned “guess and proved by induction” We also learned “substitution” method Today, we learn the “master](https://reader036.vdocuments.mx/reader036/viewer/2022070605/5a4d1ace7f8b9ab0599704b9/html5/thumbnails/2.jpg)
Master Theorem
Theorem 4.1 (CLRS, Theorem 4.1) Let a ≥ 1 and b > 1 be constants. Let f(n) be a function and let T(n) be defined on the nonnegative integers by
T(n) = aT(n/b) + f(n). Then
10
largefor )()/( AND )(
)(
)(
)(
log)(
log
log
log
log
log
c
nncfbnafnnf
nnf
nOnf
nf
nn
n
nT
a
a
a
a
a
b
b
b
b
b
![Page 3: Lecture 5 Today, how to solve recurrences We learned “guess and proved by induction” We also learned “substitution” method Today, we learn the “master](https://reader036.vdocuments.mx/reader036/viewer/2022070605/5a4d1ace7f8b9ab0599704b9/html5/thumbnails/3.jpg)
Note
Only apply to a particular family of recurrences.
f(n) is positive for large n. Key is to compare f(n) with nlog_b a Case 2, more general is f(n) = Θ( nlog_b a lgkn).
Then the result is T(n) = Θ( nlog_b a lgk+1n). Sometimes it does not apply. Ex. T(n) =
4T(n/2) + n2 /logn.
![Page 4: Lecture 5 Today, how to solve recurrences We learned “guess and proved by induction” We also learned “substitution” method Today, we learn the “master](https://reader036.vdocuments.mx/reader036/viewer/2022070605/5a4d1ace7f8b9ab0599704b9/html5/thumbnails/4.jpg)
Proof ideas of Master Theorem Consider a tree with T(n) at the root, and apply the recursion to
each node, until we get down to T(1) at the leaves. The first recursion is T(n) = aT(n/b) + f(n), so assign a cost of f(n) to the root. At the next level we have “a” nodes, each with a cost of T(n/b). When we apply the recursion again, we get a cost of af(n/b) for all of these. At the next level we have a2 nodes, each with a cost of T(n/b2). We get a cost of a2f(n/b2). We continue down to T(1) at the leaves. There are alog_b n leaves and each costs Θ(1), which gives Θ(alog_b n). The total cost associated with f is Σ 0 ≤ i ≤ log_b n - 1 ai f(n/bi).
Thus T(n) = Θ(n log_b a) + Σ 0 ≤ i ≤ (log_b n) - 1 ai f(n/bi). The three cases now come from deciding which term is
dominant. In case (1), the Θ term is dominant. In case (2), the terms are roughly equal (but the second term has an extra lg n factor). In case (3), the f(n) term is dominant. The details are somewhat painful, but can be found in CLRS, pp. 76-84.
![Page 5: Lecture 5 Today, how to solve recurrences We learned “guess and proved by induction” We also learned “substitution” method Today, we learn the “master](https://reader036.vdocuments.mx/reader036/viewer/2022070605/5a4d1ace7f8b9ab0599704b9/html5/thumbnails/5.jpg)
f (n/b)
Idea of master theorem
f (n/b) f (n/b)
(1)
…Recursion tree:
…
f (n) a
f (n/b2)f (n/b2) f (n/b2)…ah = logbn
f (n)
a f (n/b)
a2 f (n/b2)
…#leaves = ah
= alogbn
= nlogba
nlogba(1)
![Page 6: Lecture 5 Today, how to solve recurrences We learned “guess and proved by induction” We also learned “substitution” method Today, we learn the “master](https://reader036.vdocuments.mx/reader036/viewer/2022070605/5a4d1ace7f8b9ab0599704b9/html5/thumbnails/6.jpg)
Three common cases
Compare f (n) with nlogba:1. f (n) = O(nlogba – ) for some constant > 0.
• f (n) grows polynomially slower than nlogba (by an n factor).
Solution: T(n) = (nlogba) .
![Page 7: Lecture 5 Today, how to solve recurrences We learned “guess and proved by induction” We also learned “substitution” method Today, we learn the “master](https://reader036.vdocuments.mx/reader036/viewer/2022070605/5a4d1ace7f8b9ab0599704b9/html5/thumbnails/7.jpg)
f (n/b)
Idea of master theorem
f (n/b) f (n/b)
(1)
…Recursion tree:
…
f (n) a
f (n/b2)f (n/b2) f (n/b2)…ah = logbn
f (n)
a f (n/b)
a2 f (n/b2)
…
nlogba(1)CASE 1: The weight increases geometrically from the root to the leaves. The leaves hold a constant fraction of the total weight. (nlogba)
These functions increasefrom top to bottomgeometrically, hence weonly need to have thelast bottom term
![Page 8: Lecture 5 Today, how to solve recurrences We learned “guess and proved by induction” We also learned “substitution” method Today, we learn the “master](https://reader036.vdocuments.mx/reader036/viewer/2022070605/5a4d1ace7f8b9ab0599704b9/html5/thumbnails/8.jpg)
Case 2
Compare f (n) with nlogba:
2. f (n) = (nlogba lgkn) for some constant k 0.• f (n) and nlogba grow at similar rates.• This is clear for k=0. For k>0, the intuition is
that lgk n factor remain for constant fraction of levels, hence sum to the following
Solution: T(n) = (nlogba lgk+1n) .
![Page 9: Lecture 5 Today, how to solve recurrences We learned “guess and proved by induction” We also learned “substitution” method Today, we learn the “master](https://reader036.vdocuments.mx/reader036/viewer/2022070605/5a4d1ace7f8b9ab0599704b9/html5/thumbnails/9.jpg)
f (n/b)
Idea of master theorem
f (n/b) f (n/b)
(1)
…Recursion tree:
…
f (n) a
f (n/b2)f (n/b2) f (n/b2)…ah = logbn
f (n)
a f (n/b)
a2 f (n/b2)
…
nlogba(1)CASE 2: (k = 0) The weight is approximately the same on each of the logbn levels.
(nlogbalg n)
All levels same
![Page 10: Lecture 5 Today, how to solve recurrences We learned “guess and proved by induction” We also learned “substitution” method Today, we learn the “master](https://reader036.vdocuments.mx/reader036/viewer/2022070605/5a4d1ace7f8b9ab0599704b9/html5/thumbnails/10.jpg)
Case 3, c<1, akf(n/bk) geometrically decreases hence = Θ(f(n))
Compare f (n) with nlogba:3. f (n) = (nlogba + ) for some constant > 0.
• f (n) grows polynomially faster than nlogba (by an n factor),
and f (n) satisfies the regularity condition that a f (n/b) c f (n) for some constant c < 1.Solution: T(n) = ( f (n) ) .
![Page 11: Lecture 5 Today, how to solve recurrences We learned “guess and proved by induction” We also learned “substitution” method Today, we learn the “master](https://reader036.vdocuments.mx/reader036/viewer/2022070605/5a4d1ace7f8b9ab0599704b9/html5/thumbnails/11.jpg)
f (n/b)
Idea of master theorem
f (n/b) f (n/b)
(1)
…Recursion tree:
…
f (n) a
f (n/b2)f (n/b2) f (n/b2)…ah = logbn
f (n)
a f (n/b)
a2 f (n/b2)
…
nlogba(1)CASE 3: The weight decreases geometrically from the root to the leaves. The root holds a constant fraction of the total weight. ( f (n))
af(n/b)<(1-ε)f(n)
![Page 12: Lecture 5 Today, how to solve recurrences We learned “guess and proved by induction” We also learned “substitution” method Today, we learn the “master](https://reader036.vdocuments.mx/reader036/viewer/2022070605/5a4d1ace7f8b9ab0599704b9/html5/thumbnails/12.jpg)
Examples for the Master Theorem The Karatsuba recurrence has a = 3, b = 2, f(n) = cn. Then case
1 applies, and so T(n) = Θ(n l og_2 3 ), as we found. The mergesort recurrence has a = 2, b = 2, f(n) = n. Then case
2 applies, and so T(n) = Θ(n lg n). Finally, a recurrence like T(n) = 3T(n/2) + n2 gives rise to case 3.
In this case f(n) = n2, so 3f(n/2) = 3 (n/2)2 = (3/4) n2 ≤ c n2 for c = 3/4, and so T(n) = Θ(n2).
Note that the master theorem does not cover all cases. In particular, it does not cover the case
T(n) = 2 T(n/2) + n / lg n since then the only applicable case is case 3, but then the
inequality involving f does not hold.
![Page 13: Lecture 5 Today, how to solve recurrences We learned “guess and proved by induction” We also learned “substitution” method Today, we learn the “master](https://reader036.vdocuments.mx/reader036/viewer/2022070605/5a4d1ace7f8b9ab0599704b9/html5/thumbnails/13.jpg)
Closest pair problem
Input: A set of points P = {p1,…, pn} in two
dimensions Output:
The pair of points pi, pj that minimize the Euclidean distance between them.
![Page 14: Lecture 5 Today, how to solve recurrences We learned “guess and proved by induction” We also learned “substitution” method Today, we learn the “master](https://reader036.vdocuments.mx/reader036/viewer/2022070605/5a4d1ace7f8b9ab0599704b9/html5/thumbnails/14.jpg)
Distances Euclidean distance
1x 2x
2y
1y 11, yx
22 , yx
221
2212211 ,, yyxxyxyx
![Page 15: Lecture 5 Today, how to solve recurrences We learned “guess and proved by induction” We also learned “substitution” method Today, we learn the “master](https://reader036.vdocuments.mx/reader036/viewer/2022070605/5a4d1ace7f8b9ab0599704b9/html5/thumbnails/15.jpg)
Closest Pair Problem
![Page 16: Lecture 5 Today, how to solve recurrences We learned “guess and proved by induction” We also learned “substitution” method Today, we learn the “master](https://reader036.vdocuments.mx/reader036/viewer/2022070605/5a4d1ace7f8b9ab0599704b9/html5/thumbnails/16.jpg)
Closest Pair Problem
![Page 17: Lecture 5 Today, how to solve recurrences We learned “guess and proved by induction” We also learned “substitution” method Today, we learn the “master](https://reader036.vdocuments.mx/reader036/viewer/2022070605/5a4d1ace7f8b9ab0599704b9/html5/thumbnails/17.jpg)
Divide and Conquer
O(n2) time algorithm is easy Assumptions:
No two points have the same x-coordinates No two points have the same y-coordinates
How do we solve this problem in 1 dimension? Sort the number and walk from left to right to find
minimum gap.
![Page 18: Lecture 5 Today, how to solve recurrences We learned “guess and proved by induction” We also learned “substitution” method Today, we learn the “master](https://reader036.vdocuments.mx/reader036/viewer/2022070605/5a4d1ace7f8b9ab0599704b9/html5/thumbnails/18.jpg)
Divide and Conquer
Divide and conquer has a chance to do better than O(n2).
We can first sort the points by their x-coordinates and sort also by y-coordinates
![Page 19: Lecture 5 Today, how to solve recurrences We learned “guess and proved by induction” We also learned “substitution” method Today, we learn the “master](https://reader036.vdocuments.mx/reader036/viewer/2022070605/5a4d1ace7f8b9ab0599704b9/html5/thumbnails/19.jpg)
Closest Pair Problem
![Page 20: Lecture 5 Today, how to solve recurrences We learned “guess and proved by induction” We also learned “substitution” method Today, we learn the “master](https://reader036.vdocuments.mx/reader036/viewer/2022070605/5a4d1ace7f8b9ab0599704b9/html5/thumbnails/20.jpg)
Divide and Conquer for the Closest Pair Problem
Divide by x-median
![Page 21: Lecture 5 Today, how to solve recurrences We learned “guess and proved by induction” We also learned “substitution” method Today, we learn the “master](https://reader036.vdocuments.mx/reader036/viewer/2022070605/5a4d1ace7f8b9ab0599704b9/html5/thumbnails/21.jpg)
Divide
Divide by x-median
L R
![Page 22: Lecture 5 Today, how to solve recurrences We learned “guess and proved by induction” We also learned “substitution” method Today, we learn the “master](https://reader036.vdocuments.mx/reader036/viewer/2022070605/5a4d1ace7f8b9ab0599704b9/html5/thumbnails/22.jpg)
Conquer
Conquer: Recursively solve L and R
L R
1
2
![Page 23: Lecture 5 Today, how to solve recurrences We learned “guess and proved by induction” We also learned “substitution” method Today, we learn the “master](https://reader036.vdocuments.mx/reader036/viewer/2022070605/5a4d1ace7f8b9ab0599704b9/html5/thumbnails/23.jpg)
Combination I
Take the smaller one of 1 , 2 : = min(1 , 2 )
L R
2
![Page 24: Lecture 5 Today, how to solve recurrences We learned “guess and proved by induction” We also learned “substitution” method Today, we learn the “master](https://reader036.vdocuments.mx/reader036/viewer/2022070605/5a4d1ace7f8b9ab0599704b9/html5/thumbnails/24.jpg)
Combination IIIs there a point in L and a point in R whose distance is smaller than ?
= min(1 , 2 )
L R
![Page 25: Lecture 5 Today, how to solve recurrences We learned “guess and proved by induction” We also learned “substitution” method Today, we learn the “master](https://reader036.vdocuments.mx/reader036/viewer/2022070605/5a4d1ace7f8b9ab0599704b9/html5/thumbnails/25.jpg)
Combination II
If the answer is “no” then we are done!!! If the answer is “yes” then the closest such
pair forms the closest pair for the entire set
How do we determine this?
![Page 26: Lecture 5 Today, how to solve recurrences We learned “guess and proved by induction” We also learned “substitution” method Today, we learn the “master](https://reader036.vdocuments.mx/reader036/viewer/2022070605/5a4d1ace7f8b9ab0599704b9/html5/thumbnails/26.jpg)
Combination IIIs there a point in L and a point in R whose distance is smaller than ?
L R
![Page 27: Lecture 5 Today, how to solve recurrences We learned “guess and proved by induction” We also learned “substitution” method Today, we learn the “master](https://reader036.vdocuments.mx/reader036/viewer/2022070605/5a4d1ace7f8b9ab0599704b9/html5/thumbnails/27.jpg)
Combination IIIs there a point in L and a point in R whose distance is smaller than ?
Need only to consider the narrow bandO(n) time
L R
![Page 28: Lecture 5 Today, how to solve recurrences We learned “guess and proved by induction” We also learned “substitution” method Today, we learn the “master](https://reader036.vdocuments.mx/reader036/viewer/2022070605/5a4d1ace7f8b9ab0599704b9/html5/thumbnails/28.jpg)
Combination IIIs there a point in L and a point in R whose distance is smaller than ?
Denote this set by S, assume Sy is the sorted list of S by the y-coordinates.
L R
![Page 29: Lecture 5 Today, how to solve recurrences We learned “guess and proved by induction” We also learned “substitution” method Today, we learn the “master](https://reader036.vdocuments.mx/reader036/viewer/2022070605/5a4d1ace7f8b9ab0599704b9/html5/thumbnails/29.jpg)
Combination II
There exists a point in L and a point in R whose distance is less than if and only if there exist two points in S whose distance is less than .
If S is the whole thing, did we gain anything?
CLAIM: If s and t in S have the property that ||s-t|| < , then s and t are within 15 positions of each other in the sorted list Sy.
![Page 30: Lecture 5 Today, how to solve recurrences We learned “guess and proved by induction” We also learned “substitution” method Today, we learn the “master](https://reader036.vdocuments.mx/reader036/viewer/2022070605/5a4d1ace7f8b9ab0599704b9/html5/thumbnails/30.jpg)
Combination IIIs there a point in L and a point in R whose distance is smaller than ?
L R
There are at most one point in each box of size δ/2 by δ/2. Thus s and t cannot be too far apart.
![Page 31: Lecture 5 Today, how to solve recurrences We learned “guess and proved by induction” We also learned “substitution” method Today, we learn the “master](https://reader036.vdocuments.mx/reader036/viewer/2022070605/5a4d1ace7f8b9ab0599704b9/html5/thumbnails/31.jpg)
Closest-Pair Preprocessing:
Construct Px and Py as sorted-list by x- and y-coordinates Closest-pair(P, Px,Py)
Divide Construct L, Lx , Ly and R, Rx , Ry
Conquer Let 1= Closest-Pair(L, Lx , Ly ) Let 2= Closest-Pair(R, Rx , Ry )
Combination Let = min(1 , 2 ) Construct S and Sy For each point in Sy, check each of its next 15 points down
the list If the distance is less than , update the as this smaller
distance
![Page 32: Lecture 5 Today, how to solve recurrences We learned “guess and proved by induction” We also learned “substitution” method Today, we learn the “master](https://reader036.vdocuments.mx/reader036/viewer/2022070605/5a4d1ace7f8b9ab0599704b9/html5/thumbnails/32.jpg)
Complexity Analysis
Preprocessing takes O(n lg n) time Divide takes O(n) time Conquer takes 2 T(n/2) time Combination takes O(n) time T(n) = 2T(n/2) + cn
So totally takes O(n lg n) time.
![Page 33: Lecture 5 Today, how to solve recurrences We learned “guess and proved by induction” We also learned “substitution” method Today, we learn the “master](https://reader036.vdocuments.mx/reader036/viewer/2022070605/5a4d1ace7f8b9ab0599704b9/html5/thumbnails/33.jpg)
Matrix Multiplication
Suppose we multiply two NxN matrices together. Regular method is NxNxN = N3 multiplications O(N3)
![Page 34: Lecture 5 Today, how to solve recurrences We learned “guess and proved by induction” We also learned “substitution” method Today, we learn the “master](https://reader036.vdocuments.mx/reader036/viewer/2022070605/5a4d1ace7f8b9ab0599704b9/html5/thumbnails/34.jpg)
Can we Divide and Conquer?
C11 = A11*B11 + A12*B21 C12 = A11*B12 + A12*B22 C21 = A21*B11 + A22*B21 C22 = A21*B12 + A22*B22
Complexity : T(N) = 8T(N/2) + O(N2) = O(Nlog
28) = O(N3)
22211211
AAAA
A =
22211211
BBBB
B =
22211211
CCCC
C= A*B =
No improvement
![Page 35: Lecture 5 Today, how to solve recurrences We learned “guess and proved by induction” We also learned “substitution” method Today, we learn the “master](https://reader036.vdocuments.mx/reader036/viewer/2022070605/5a4d1ace7f8b9ab0599704b9/html5/thumbnails/35.jpg)
Strassen’s Matrix Multiplication
P1 = (A11+ A22)(B11+B22) P2 = (A21 + A22) * B11 P3 = A11 * (B12 - B22) P4 = A22 * (B21 - B11) P5 = (A11 + A12) * B22 P6 = (A21 - A11) * (B11 + B12) P7 = (A12 - A22) * (B21 + B22)
C11 = P1 + P4 - P5 + P7
C12 = P3 + P5 C21 = P2 + P4 C22 = P1 + P3 - P2 + P6
And do this recursively as usual.
VolkerStrassen
![Page 36: Lecture 5 Today, how to solve recurrences We learned “guess and proved by induction” We also learned “substitution” method Today, we learn the “master](https://reader036.vdocuments.mx/reader036/viewer/2022070605/5a4d1ace7f8b9ab0599704b9/html5/thumbnails/36.jpg)
Time analysis
T(n) = 7T(n/2) + O(n2 ) = 7logn by the Master Theorem =nlog7
=n2.81
Best bound: O(n2.376) by Coppersmith-Winograd.
Best known (trivial) lower bound: Ω(n2). Open: what is the true complexity of matrix
multiplication?