algorithm analysis algorithm analysis lectures 3 & 4 resources data structures & algorithms...
TRANSCRIPT
![Page 1: Algorithm Analysis Algorithm Analysis Lectures 3 & 4 Resources Data Structures & Algorithms Analysis in C++ (MAW): Chap. 2 Introduction to Algorithms (Cormen,](https://reader036.vdocuments.mx/reader036/viewer/2022082821/5697c00d1a28abf838cc93d9/html5/thumbnails/1.jpg)
Algorithm AnalysisAlgorithm AnalysisLectures 3 & 4
ResourcesData Structures & Algorithms Analysis in C++ (MAW): Chap. 2Introduction to Algorithms (Cormen, Leiserson, & Rivest): Chap.1Algorithms Theory & Practice (Brassard & Bratley): Chap. 1
![Page 2: Algorithm Analysis Algorithm Analysis Lectures 3 & 4 Resources Data Structures & Algorithms Analysis in C++ (MAW): Chap. 2 Introduction to Algorithms (Cormen,](https://reader036.vdocuments.mx/reader036/viewer/2022082821/5697c00d1a28abf838cc93d9/html5/thumbnails/2.jpg)
Algorithms
• An algorithm is a well-defined computational procedure that takes some value or a set of values, as input and produces some value, or a set of values as output.
• Or, an algorithm is a well-specified set of instructions to be solve a problem.
![Page 3: Algorithm Analysis Algorithm Analysis Lectures 3 & 4 Resources Data Structures & Algorithms Analysis in C++ (MAW): Chap. 2 Introduction to Algorithms (Cormen,](https://reader036.vdocuments.mx/reader036/viewer/2022082821/5697c00d1a28abf838cc93d9/html5/thumbnails/3.jpg)
Efficiency of Algorithms
• Empirical– Programming competing algorithms and
trying them on different instances
• Theoretical– Determining mathematically the quantity of
resources (execution time, memory space, etc) needed by each algorithm
![Page 4: Algorithm Analysis Algorithm Analysis Lectures 3 & 4 Resources Data Structures & Algorithms Analysis in C++ (MAW): Chap. 2 Introduction to Algorithms (Cormen,](https://reader036.vdocuments.mx/reader036/viewer/2022082821/5697c00d1a28abf838cc93d9/html5/thumbnails/4.jpg)
Analyzing Algorithms• Predicting the resources that the
algorithm requires:• Computational running time• Memory usage• Communication bandwidth
• The running time of an algorithm • Number of primitive operations on a particular
input size• Depends on
– Input size (e.g. 60 elements vs. 70000)– The input itself ( partially sorted input for a sorting
algorithm)
![Page 5: Algorithm Analysis Algorithm Analysis Lectures 3 & 4 Resources Data Structures & Algorithms Analysis in C++ (MAW): Chap. 2 Introduction to Algorithms (Cormen,](https://reader036.vdocuments.mx/reader036/viewer/2022082821/5697c00d1a28abf838cc93d9/html5/thumbnails/5.jpg)
Order of Growth
• The order (rate) of growth of a running time– Ignore machine dependant constants
– Look at growth of T(n) as n notation
• Drop low-order terms• Ignore leading constants• E.g.
– 3n3 + 90n2 – 2n +5 = (n3)
![Page 6: Algorithm Analysis Algorithm Analysis Lectures 3 & 4 Resources Data Structures & Algorithms Analysis in C++ (MAW): Chap. 2 Introduction to Algorithms (Cormen,](https://reader036.vdocuments.mx/reader036/viewer/2022082821/5697c00d1a28abf838cc93d9/html5/thumbnails/6.jpg)
Mathematical Background
![Page 7: Algorithm Analysis Algorithm Analysis Lectures 3 & 4 Resources Data Structures & Algorithms Analysis in C++ (MAW): Chap. 2 Introduction to Algorithms (Cormen,](https://reader036.vdocuments.mx/reader036/viewer/2022082821/5697c00d1a28abf838cc93d9/html5/thumbnails/7.jpg)
Mathematical Background
• Definitions:
– T(N) = O(f(N)) iff c and n0 T(N) c.f(N) when N n0
– T(N) = (g(N)) iff c and n0 T(N) c.g(N) when N n0
– T(N) = (h(N)) iff T(N) = O(h(N)) and
T(N) = (h(N))
![Page 8: Algorithm Analysis Algorithm Analysis Lectures 3 & 4 Resources Data Structures & Algorithms Analysis in C++ (MAW): Chap. 2 Introduction to Algorithms (Cormen,](https://reader036.vdocuments.mx/reader036/viewer/2022082821/5697c00d1a28abf838cc93d9/html5/thumbnails/8.jpg)
Mathematical Background
• Definitions:
– T(N) = o(f(N)) iff c and n0 T(N) c.f(N) when N n0
– T(N) = (g(N)) iff c and n0 T(N) c.g(N) when N n0
![Page 9: Algorithm Analysis Algorithm Analysis Lectures 3 & 4 Resources Data Structures & Algorithms Analysis in C++ (MAW): Chap. 2 Introduction to Algorithms (Cormen,](https://reader036.vdocuments.mx/reader036/viewer/2022082821/5697c00d1a28abf838cc93d9/html5/thumbnails/9.jpg)
Mathematical Background
• Rules:
– If T1(N) = O(f(N)) and T2(N) = O(g(N)) thena) T1(N) + T2(N) = max( O(f(N)),O(g(N))b) T1(N) * T2(N) = O(f(N) * g(N))
– If T(N) is a polynomial of degree k, then T(N) = (Nk)
– Logk N = O(N) for any constant k.
![Page 10: Algorithm Analysis Algorithm Analysis Lectures 3 & 4 Resources Data Structures & Algorithms Analysis in C++ (MAW): Chap. 2 Introduction to Algorithms (Cormen,](https://reader036.vdocuments.mx/reader036/viewer/2022082821/5697c00d1a28abf838cc93d9/html5/thumbnails/10.jpg)
More …1. 3n3 + 90n2 – 2n +5 = O(n3 )2. 2n2 + 3n +1000000 = (n2) 3. 2n = o(n2) ( set membership)
4. 3n2 = O(n2) tighter (n2) 5. n log n = O(n2) 6. True or false:
– n2 = O(n3 )– n3 = O(n2) – 2n+1= O(2n)– (n+1)! = O(n!)
![Page 11: Algorithm Analysis Algorithm Analysis Lectures 3 & 4 Resources Data Structures & Algorithms Analysis in C++ (MAW): Chap. 2 Introduction to Algorithms (Cormen,](https://reader036.vdocuments.mx/reader036/viewer/2022082821/5697c00d1a28abf838cc93d9/html5/thumbnails/11.jpg)
Ranking by Order of Growth
1 n n log n n2 nk
(3/2)n 2n (n)! (n+1)!
![Page 12: Algorithm Analysis Algorithm Analysis Lectures 3 & 4 Resources Data Structures & Algorithms Analysis in C++ (MAW): Chap. 2 Introduction to Algorithms (Cormen,](https://reader036.vdocuments.mx/reader036/viewer/2022082821/5697c00d1a28abf838cc93d9/html5/thumbnails/12.jpg)
Running time calculations• Rule 1 – For Loops
The running time of a for loop is at most the running time of the statement inside the for loop (including tests) times the number of iterations
• Rule 2 – Nested LoopsAnalyze these inside out. The total running time
of a statement inside a group of nested loops is the running time of the statement multiplied by the product of the sizes of all the loops
![Page 13: Algorithm Analysis Algorithm Analysis Lectures 3 & 4 Resources Data Structures & Algorithms Analysis in C++ (MAW): Chap. 2 Introduction to Algorithms (Cormen,](https://reader036.vdocuments.mx/reader036/viewer/2022082821/5697c00d1a28abf838cc93d9/html5/thumbnails/13.jpg)
Running time calculations: Examples
Example 1:sum = 0;for (i=1; i <=n; i++)
sum += n;
Example 2:sum = 0;for (j=1; j<=n; j++)for (i=1; i<=j; i++)sum++;for (k=0; k<n; k++)A[k] = k;
![Page 14: Algorithm Analysis Algorithm Analysis Lectures 3 & 4 Resources Data Structures & Algorithms Analysis in C++ (MAW): Chap. 2 Introduction to Algorithms (Cormen,](https://reader036.vdocuments.mx/reader036/viewer/2022082821/5697c00d1a28abf838cc93d9/html5/thumbnails/14.jpg)
How to rank?