data structures and algorithmsrweba/algf09/solverecurrencessf.pdf · data structures and algorithms...
TRANSCRIPT
![Page 1: Data Structures and Algorithmsrweba/algf09/solverecurrencesSF.pdf · Data Structures and Algorithms Solving Recurrence Relations Chris Brooks Department of Computer Science University](https://reader031.vdocuments.mx/reader031/viewer/2022021423/5ad14e1c7f8b9a05208b8a07/html5/thumbnails/1.jpg)
Data Structures and Algorithms
Solving Recurrence RelationsChris Brooks
Department of Computer Science
University of San Francisco
Department of Computer Science — University of San Francisco – p.1/30
![Page 2: Data Structures and Algorithmsrweba/algf09/solverecurrencesSF.pdf · Data Structures and Algorithms Solving Recurrence Relations Chris Brooks Department of Computer Science University](https://reader031.vdocuments.mx/reader031/viewer/2022021423/5ad14e1c7f8b9a05208b8a07/html5/thumbnails/2.jpg)
4-0: Algorithm Analysis
for (i=1; i<=n*n; i++)
for (j=0; j<i; j++)
sum++;
Department of Computer Science — University of San Francisco – p.2/30
![Page 3: Data Structures and Algorithmsrweba/algf09/solverecurrencesSF.pdf · Data Structures and Algorithms Solving Recurrence Relations Chris Brooks Department of Computer Science University](https://reader031.vdocuments.mx/reader031/viewer/2022021423/5ad14e1c7f8b9a05208b8a07/html5/thumbnails/3.jpg)
4-1: Algorithm Analysis
for (i=1; i<=n*n; i++) Executed n*n times
for (j=0; j<i; j++) Executed <= n*n times
sum++; O(1)
Running Time: O(n4)
But can we get a tighter bound?
Department of Computer Science — University of San Francisco – p.3/30
![Page 4: Data Structures and Algorithmsrweba/algf09/solverecurrencesSF.pdf · Data Structures and Algorithms Solving Recurrence Relations Chris Brooks Department of Computer Science University](https://reader031.vdocuments.mx/reader031/viewer/2022021423/5ad14e1c7f8b9a05208b8a07/html5/thumbnails/4.jpg)
4-2: Algorithm Analysis
for (i=1; i<=n*n; i++)
for (j=0; j<i; j++)
sum++;
Exact # of times sum++ is executed:
n2∑
i=1
i =n2(n2 + 1)
2
=n4 + n2
2
∈ Θ(n4)
Department of Computer Science — University of San Francisco – p.4/30
![Page 5: Data Structures and Algorithmsrweba/algf09/solverecurrencesSF.pdf · Data Structures and Algorithms Solving Recurrence Relations Chris Brooks Department of Computer Science University](https://reader031.vdocuments.mx/reader031/viewer/2022021423/5ad14e1c7f8b9a05208b8a07/html5/thumbnails/5.jpg)
4-3: Recursive Functions
long power(long x, long n)
if (n == 0)
return 1;
else
return x * power(x, n-1);
How many times is this executed?
Department of Computer Science — University of San Francisco – p.5/30
![Page 6: Data Structures and Algorithmsrweba/algf09/solverecurrencesSF.pdf · Data Structures and Algorithms Solving Recurrence Relations Chris Brooks Department of Computer Science University](https://reader031.vdocuments.mx/reader031/viewer/2022021423/5ad14e1c7f8b9a05208b8a07/html5/thumbnails/6.jpg)
4-4: Recurrence Relations
T (n) = Time required to solve a problem of size n
Recurrence relations are used to determine the running timeof recursive programs – recurrence relations themselves arerecursive
T (0) = time to solve problem of size 0– Base Case
T (n) = time to solve problem of size n
– Recursive Case
Department of Computer Science — University of San Francisco – p.6/30
![Page 7: Data Structures and Algorithmsrweba/algf09/solverecurrencesSF.pdf · Data Structures and Algorithms Solving Recurrence Relations Chris Brooks Department of Computer Science University](https://reader031.vdocuments.mx/reader031/viewer/2022021423/5ad14e1c7f8b9a05208b8a07/html5/thumbnails/7.jpg)
4-5: Recurrence Relations
long power(long x, long n)
if (n == 0)
return 1;
else
return x * power(x, n-1);
T (0) = c1 for some constant c1
T (n) = c2 + T (n − 1) for some constant c2
Department of Computer Science — University of San Francisco – p.7/30
![Page 8: Data Structures and Algorithmsrweba/algf09/solverecurrencesSF.pdf · Data Structures and Algorithms Solving Recurrence Relations Chris Brooks Department of Computer Science University](https://reader031.vdocuments.mx/reader031/viewer/2022021423/5ad14e1c7f8b9a05208b8a07/html5/thumbnails/8.jpg)
4-6: Solving Recurrence Relations
T (0) = c1
T (n) = T (n − 1) + c2
If we knew T (n − 1), we could solve T (n).
T (n) = T (n − 1) + c2
Department of Computer Science — University of San Francisco – p.8/30
![Page 9: Data Structures and Algorithmsrweba/algf09/solverecurrencesSF.pdf · Data Structures and Algorithms Solving Recurrence Relations Chris Brooks Department of Computer Science University](https://reader031.vdocuments.mx/reader031/viewer/2022021423/5ad14e1c7f8b9a05208b8a07/html5/thumbnails/9.jpg)
4-7: Solving Recurrence Relations
T (0) = c1
T (n) = T (n − 1) + c2
If we knew T (n − 1), we could solve T (n).
T (n) = T (n − 1) + c2 T (n − 1) = T (n − 2) + c2
= T (n − 2) + c2 + c2
= T (n − 2) + 2c2
Department of Computer Science — University of San Francisco – p.9/30
![Page 10: Data Structures and Algorithmsrweba/algf09/solverecurrencesSF.pdf · Data Structures and Algorithms Solving Recurrence Relations Chris Brooks Department of Computer Science University](https://reader031.vdocuments.mx/reader031/viewer/2022021423/5ad14e1c7f8b9a05208b8a07/html5/thumbnails/10.jpg)
4-8: Solving Recurrence Relations
T (0) = c1
T (n) = T (n − 1) + c2
If we knew T (n − 1), we could solve T (n).
T (n) = T (n − 1) + c2 T (n − 1) = T (n − 2) + c2
= T (n − 2) + c2 + c2
= T (n − 2) + 2c2 T (n − 2) = T (n − 3) + c2
= T (n − 3) + c2 + 2c2
= T (n − 3) + 3c2
Department of Computer Science — University of San Francisco – p.10/30
![Page 11: Data Structures and Algorithmsrweba/algf09/solverecurrencesSF.pdf · Data Structures and Algorithms Solving Recurrence Relations Chris Brooks Department of Computer Science University](https://reader031.vdocuments.mx/reader031/viewer/2022021423/5ad14e1c7f8b9a05208b8a07/html5/thumbnails/11.jpg)
4-9: Solving Recurrence Relations
T (0) = c1
T (n) = T (n − 1) + c2
If we knew T (n − 1), we could solve T (n).
T (n) = T (n − 1) + c2 T (n − 1) = T (n − 2) + c2
= T (n − 2) + c2 + c2
= T (n − 2) + 2c2 T (n − 2) = T (n − 3) + c2
= T (n − 3) + c2 + 2c2
= T (n − 3) + 3c2 T (n − 3) = T (n − 4) + c2
= T (n − 4) + 4c2
Department of Computer Science — University of San Francisco – p.11/30
![Page 12: Data Structures and Algorithmsrweba/algf09/solverecurrencesSF.pdf · Data Structures and Algorithms Solving Recurrence Relations Chris Brooks Department of Computer Science University](https://reader031.vdocuments.mx/reader031/viewer/2022021423/5ad14e1c7f8b9a05208b8a07/html5/thumbnails/12.jpg)
4-10: Solving Recurrence Relations
T (0) = c1
T (n) = T (n − 1) + c2
If we knew T (n − 1), we could solve T (n).
T (n) = T (n − 1) + c2 T (n − 1) = T (n − 2) + c2
= T (n − 2) + c2 + c2
= T (n − 2) + 2c2 T (n − 2) = T (n − 3) + c2
= T (n − 3) + c2 + 2c2
= T (n − 3) + 3c2 T (n − 3) = T (n − 4) + c2
= T (n − 4) + 4c2
= . . .
= T (n − k) + kc2
Department of Computer Science — University of San Francisco – p.12/30
![Page 13: Data Structures and Algorithmsrweba/algf09/solverecurrencesSF.pdf · Data Structures and Algorithms Solving Recurrence Relations Chris Brooks Department of Computer Science University](https://reader031.vdocuments.mx/reader031/viewer/2022021423/5ad14e1c7f8b9a05208b8a07/html5/thumbnails/13.jpg)
4-11: Solving Recurrence Relations
T (0) = c1
T (n) = T (n − k) + k ∗ c2 for all k
If we set k = n, we have:
T (n) = T (n − n) + nc2
= T (0) + nc2
= c1 + nc2
∈ Θ(n)
Department of Computer Science — University of San Francisco – p.13/30
![Page 14: Data Structures and Algorithmsrweba/algf09/solverecurrencesSF.pdf · Data Structures and Algorithms Solving Recurrence Relations Chris Brooks Department of Computer Science University](https://reader031.vdocuments.mx/reader031/viewer/2022021423/5ad14e1c7f8b9a05208b8a07/html5/thumbnails/14.jpg)
4-12: Building a Better Power
Can we avoid making a linear number of function calls?
long power(long x, long n)
if (n==0) return 1;
if (n==1) return x;
if ((n % 2) == 0)
return power(x*x, n/2);
else
return power(x*x, n/2) * x;
Department of Computer Science — University of San Francisco – p.14/30
![Page 15: Data Structures and Algorithmsrweba/algf09/solverecurrencesSF.pdf · Data Structures and Algorithms Solving Recurrence Relations Chris Brooks Department of Computer Science University](https://reader031.vdocuments.mx/reader031/viewer/2022021423/5ad14e1c7f8b9a05208b8a07/html5/thumbnails/15.jpg)
4-13: Building a Better Power
long power(long x, long n)
if (n==0) return 1;
if (n==1) return x;
if ((n % 2) == 0)
return power(x*x, n/2);
else
return power(x*x, n/2) * x;
T (0) = c1
T (1) = c2
T (n) = T (n/2) + c3
(Assume n is a power of 2)
Department of Computer Science — University of San Francisco – p.15/30
![Page 16: Data Structures and Algorithmsrweba/algf09/solverecurrencesSF.pdf · Data Structures and Algorithms Solving Recurrence Relations Chris Brooks Department of Computer Science University](https://reader031.vdocuments.mx/reader031/viewer/2022021423/5ad14e1c7f8b9a05208b8a07/html5/thumbnails/16.jpg)
4-14: Solving Recurrence Relations
T (n) = T (n/2) + c3
Department of Computer Science — University of San Francisco – p.16/30
![Page 17: Data Structures and Algorithmsrweba/algf09/solverecurrencesSF.pdf · Data Structures and Algorithms Solving Recurrence Relations Chris Brooks Department of Computer Science University](https://reader031.vdocuments.mx/reader031/viewer/2022021423/5ad14e1c7f8b9a05208b8a07/html5/thumbnails/17.jpg)
4-15: Solving Recurrence Relations
T (n) = T (n/2) + c3 T (n/2) = T (n/4) + c3
= T (n/4) + c3 + c3
= T (n/4) + 2c3
Department of Computer Science — University of San Francisco – p.17/30
![Page 18: Data Structures and Algorithmsrweba/algf09/solverecurrencesSF.pdf · Data Structures and Algorithms Solving Recurrence Relations Chris Brooks Department of Computer Science University](https://reader031.vdocuments.mx/reader031/viewer/2022021423/5ad14e1c7f8b9a05208b8a07/html5/thumbnails/18.jpg)
4-16: Solving Recurrence Relations
T (n) = T (n/2) + c3 T (n/2) = T (n/4) + c3
= T (n/4) + c3 + c3
= T (n/4) + 2c3 T (n/4) = T (n/8) + c3
= T (n/8) + c3 + 2c3
= T (n/8) + 3c3
Department of Computer Science — University of San Francisco – p.18/30
![Page 19: Data Structures and Algorithmsrweba/algf09/solverecurrencesSF.pdf · Data Structures and Algorithms Solving Recurrence Relations Chris Brooks Department of Computer Science University](https://reader031.vdocuments.mx/reader031/viewer/2022021423/5ad14e1c7f8b9a05208b8a07/html5/thumbnails/19.jpg)
4-17: Solving Recurrence Relations
T (n) = T (n/2) + c3 T (n/2) = T (n/4) + c3
= T (n/4) + c3 + c3
= T (n/4) + 2c3 T (n/4) = T (n/8) + c3
= T (n/8) + c3 + 2c3
= T (n/8) + 3c3 T (n/8) = T (n/16) + c3
= T (n/16) + c3 + 3c3
= T (n/16) + 4c3
Department of Computer Science — University of San Francisco – p.19/30
![Page 20: Data Structures and Algorithmsrweba/algf09/solverecurrencesSF.pdf · Data Structures and Algorithms Solving Recurrence Relations Chris Brooks Department of Computer Science University](https://reader031.vdocuments.mx/reader031/viewer/2022021423/5ad14e1c7f8b9a05208b8a07/html5/thumbnails/20.jpg)
4-18: Solving Recurrence Relations
T (n) = T (n/2) + c3 T (n/2) = T (n/4) + c3
= T (n/4) + c3 + c3
= T (n/4)2c3 T (n/4) = T (n/8) + c3
= T (n/8) + c3 + 2c3
= T (n/8)3c3 T (n/8) = T (n/16) + c3
= T (n/16) + c3 + 3c3
= T (n/16) + 4c3 T (n/16) = T (n/32) + c3
= T (n/32) + c3 + 4c3
= T (n/32) + 5c3
Department of Computer Science — University of San Francisco – p.20/30
![Page 21: Data Structures and Algorithmsrweba/algf09/solverecurrencesSF.pdf · Data Structures and Algorithms Solving Recurrence Relations Chris Brooks Department of Computer Science University](https://reader031.vdocuments.mx/reader031/viewer/2022021423/5ad14e1c7f8b9a05208b8a07/html5/thumbnails/21.jpg)
4-19: Solving Recurrence Relations
T (n) = T (n/2) + c3 T (n/2) = T (n/4) + c3
= T (n/4) + c3 + c3
= T (n/4)2c3 T (n/4) = T (n/8) + c3
= T (n/8) + c3 + 2c3
= T (n/8)3c3 T (n/8) = T (n/16) + c3
= T (n/16) + c3 + 3c3
= T (n/16) + 4c3 T (n/16) = T (n/32) + c3
= T (n/32) + c3 + 4c3
= T (n/32) + 5c3
= . . .
= T (n/2k) + kc3
Department of Computer Science — University of San Francisco – p.21/30
![Page 22: Data Structures and Algorithmsrweba/algf09/solverecurrencesSF.pdf · Data Structures and Algorithms Solving Recurrence Relations Chris Brooks Department of Computer Science University](https://reader031.vdocuments.mx/reader031/viewer/2022021423/5ad14e1c7f8b9a05208b8a07/html5/thumbnails/22.jpg)
4-20: Solving Recurrence Relations
T (0) = c1
T (1) = c2
T (n) = T (n/2) + c3
T (n) = T (n/2k) + kc3
We want to get rid of T (n/2k). We get to a relation we can
solve directly when we reach T (1)
n/2k = 1
n = 2k
lg n = k
Department of Computer Science — University of San Francisco – p.22/30
![Page 23: Data Structures and Algorithmsrweba/algf09/solverecurrencesSF.pdf · Data Structures and Algorithms Solving Recurrence Relations Chris Brooks Department of Computer Science University](https://reader031.vdocuments.mx/reader031/viewer/2022021423/5ad14e1c7f8b9a05208b8a07/html5/thumbnails/23.jpg)
4-21: Solving Recurrence Relations
T (0) = c1
T (1) = c2
T (n) = T (n/2) + c3
T (n) = T (n/2k) + kc3
We want to get rid of T (n/2k). We get to a relation we can
solve directly when we reach T (1)lg n = k
T (n) = T (n/2lg n) + lg nc3
= T (1) + c3 lg n
= c2 + c3 lg n
∈ Θ(lg n)Department of Computer Science — University of San Francisco – p.23/30
![Page 24: Data Structures and Algorithmsrweba/algf09/solverecurrencesSF.pdf · Data Structures and Algorithms Solving Recurrence Relations Chris Brooks Department of Computer Science University](https://reader031.vdocuments.mx/reader031/viewer/2022021423/5ad14e1c7f8b9a05208b8a07/html5/thumbnails/24.jpg)
4-22: Power Modifications
long power(long x, long n)
if (n==0) return 1;
if (n==1) return x;
if ((n % 2) == 0)
return power(x*x, n/2);
else
return power(x*x, n/2) * x;
Department of Computer Science — University of San Francisco – p.24/30
![Page 25: Data Structures and Algorithmsrweba/algf09/solverecurrencesSF.pdf · Data Structures and Algorithms Solving Recurrence Relations Chris Brooks Department of Computer Science University](https://reader031.vdocuments.mx/reader031/viewer/2022021423/5ad14e1c7f8b9a05208b8a07/html5/thumbnails/25.jpg)
4-23: Power Modifications
long power(long x, long n)
if (n==0) return 1;
if (n==1) return x;
if ((n % 2) == 0)
return power(power(x,2), n/2);
else
return power(power(x,2), n/2) * x;
This version of power will not work. Why?
Department of Computer Science — University of San Francisco – p.25/30
![Page 26: Data Structures and Algorithmsrweba/algf09/solverecurrencesSF.pdf · Data Structures and Algorithms Solving Recurrence Relations Chris Brooks Department of Computer Science University](https://reader031.vdocuments.mx/reader031/viewer/2022021423/5ad14e1c7f8b9a05208b8a07/html5/thumbnails/26.jpg)
4-24: Power Modifications
long power(long x, long n)
if (n==0) return 1;
if (n==1) return x;
if ((n % 2) == 0)
return power(power(x,n/2), 2);
else
return power(power(x,n/2), 2) * x;
This version of power also will not work. Why?
Department of Computer Science — University of San Francisco – p.26/30
![Page 27: Data Structures and Algorithmsrweba/algf09/solverecurrencesSF.pdf · Data Structures and Algorithms Solving Recurrence Relations Chris Brooks Department of Computer Science University](https://reader031.vdocuments.mx/reader031/viewer/2022021423/5ad14e1c7f8b9a05208b8a07/html5/thumbnails/27.jpg)
4-25: Power Modifications
long power(long x, long n)
if (n==0) return 1;
if (n==1) return x;
if ((n % 2) == 0)
return power(x,n/2) * power(x,n/2);
else
return power(x,n/2) * power(x,n/2) * x;
This version of power does work.
What is the recurrence relation that describes its runningtime?
Department of Computer Science — University of San Francisco – p.27/30
![Page 28: Data Structures and Algorithmsrweba/algf09/solverecurrencesSF.pdf · Data Structures and Algorithms Solving Recurrence Relations Chris Brooks Department of Computer Science University](https://reader031.vdocuments.mx/reader031/viewer/2022021423/5ad14e1c7f8b9a05208b8a07/html5/thumbnails/28.jpg)
4-26: Power Modifications
long power(long x, long n)
if (n==0) return 1;
if (n==1) return x;
if ((n % 2) == 0)
return power(x,n/2) * power(x,n/2);
else
return power(x,n.2) * power(x,n/2) * x;
T (0) = c1
T (1) = c2
T (n) = T (n/2) + T (n/2) + c3
= 2T (n/2) + c3
(Again, assume n is a power of 2)
Department of Computer Science — University of San Francisco – p.28/30
![Page 29: Data Structures and Algorithmsrweba/algf09/solverecurrencesSF.pdf · Data Structures and Algorithms Solving Recurrence Relations Chris Brooks Department of Computer Science University](https://reader031.vdocuments.mx/reader031/viewer/2022021423/5ad14e1c7f8b9a05208b8a07/html5/thumbnails/29.jpg)
4-27: Solving Recurrence Relations
T (n) = 2T (n/2) + c3 T (n/2) = 2T (n/4) + c3
= 2[2T (n/4) + c3]c3
= 4T (n/4) + 3c3 T (n/4) = 2T (n/8) + c3
= 4[2T (n/8) + c3] + 3c3
= 8T (n/8) + 7c3
= 8[2T (n/16) + c3] + 7c3
= 16T (n/16) + 15c3
= 32T (n/32) + 31c3
. . .
= 2kT (n/2k) + (2k− 1)c3
Department of Computer Science — University of San Francisco – p.29/30
![Page 30: Data Structures and Algorithmsrweba/algf09/solverecurrencesSF.pdf · Data Structures and Algorithms Solving Recurrence Relations Chris Brooks Department of Computer Science University](https://reader031.vdocuments.mx/reader031/viewer/2022021423/5ad14e1c7f8b9a05208b8a07/html5/thumbnails/30.jpg)
4-28: Solving Recurrence Relations
T (0) = c1
T (1) = c2
T (n) = 2kT (n/2k) + (2k− 1)c3
Pick a value for k such that n/2k = 1:n/2k = 1
n = 2k
lg n = k
T (n) = 2lg nT (n/2lg n) + (2lg n− 1)c3
= nT (n/n) + (n − 1)c3
= nT (1) + (n − 1)c3
= nc2 + (n − 1)c3
∈ Θ(n)
Department of Computer Science — University of San Francisco – p.30/30