Download - Analysis of Algorithms
![Page 1: Analysis of Algorithms](https://reader036.vdocuments.mx/reader036/viewer/2022062321/56813291550346895d99263b/html5/thumbnails/1.jpg)
Analysis of AlgorithmsAnalysis of Algorithms
![Page 2: Analysis of Algorithms](https://reader036.vdocuments.mx/reader036/viewer/2022062321/56813291550346895d99263b/html5/thumbnails/2.jpg)
Two Approaches for measuring Two Approaches for measuring running time of a programrunning time of a program BenchmarkingBenchmarking
A small collection of typical inputs that can A small collection of typical inputs that can serve as performance standardsserve as performance standards
AnalysisAnalysis Determining the general time a program Determining the general time a program
takes as a function of input sizetakes as a function of input size
![Page 3: Analysis of Algorithms](https://reader036.vdocuments.mx/reader036/viewer/2022062321/56813291550346895d99263b/html5/thumbnails/3.jpg)
Running TimeRunning Time
T(n)T(n) a function to represent the units of time a function to represent the units of time
taken by a program with input of size ntaken by a program with input of size n
T(n) T(n) # statements executed# statements executed
![Page 4: Analysis of Algorithms](https://reader036.vdocuments.mx/reader036/viewer/2022062321/56813291550346895d99263b/html5/thumbnails/4.jpg)
Running TimeRunning Time
TTww(n)(n) worst case running timeworst case running time maximum running time among all inputs of size maximum running time among all inputs of size
nn
TTavgavg(n)(n) average running timeaverage running time avg running time over all inputs of size navg running time over all inputs of size n more realistic; harder to computemore realistic; harder to compute
![Page 5: Analysis of Algorithms](https://reader036.vdocuments.mx/reader036/viewer/2022062321/56813291550346895d99263b/html5/thumbnails/5.jpg)
Calculate T(n)Calculate T(n)
small = i;small = i;
for (j=i+1; j<=n; j++) {for (j=i+1; j<=n; j++) {
if (A[j] < A[small])if (A[j] < A[small])
small = j;small = j;
Count 1 unit of time for each assign, compareCount 1 unit of time for each assign, compare
![Page 6: Analysis of Algorithms](https://reader036.vdocuments.mx/reader036/viewer/2022062321/56813291550346895d99263b/html5/thumbnails/6.jpg)
Big-O notationBig-O notation
A tool to analyze a program's efficiency.A tool to analyze a program's efficiency.
An approximation ofAn approximation of work an algorithm performs work an algorithm performs as a function of size of inputas a function of size of input
![Page 7: Analysis of Algorithms](https://reader036.vdocuments.mx/reader036/viewer/2022062321/56813291550346895d99263b/html5/thumbnails/7.jpg)
WorkWork
A measure of A measure of effort expended effort expended by computerby computer in performing a computationin performing a computation
![Page 8: Analysis of Algorithms](https://reader036.vdocuments.mx/reader036/viewer/2022062321/56813291550346895d99263b/html5/thumbnails/8.jpg)
Big-Big-OO
f(x)f(x) is O( is O(g(x)g(x))) if there are constantsif there are constants C C and and kk such that such that
| | f(x)f(x) | | C C | | g(x)g(x) | |
whenever whenever xx > > kk
![Page 9: Analysis of Algorithms](https://reader036.vdocuments.mx/reader036/viewer/2022062321/56813291550346895d99263b/html5/thumbnails/9.jpg)
Show f(x) = xShow f(x) = x22+2x+1 is O(x+2x+1 is O(x22))
Prove:Prove: |x|x22+2x+1|+2x+1| C C | | xx22 | |
Choose k > 0, and f(x) is always positiveChoose k > 0, and f(x) is always positive
Let C be the sum of f(x) coefficientsLet C be the sum of f(x) coefficients
xx22+2x+1+2x+1 1x1x22+2x+2x22+1x+1x2 = 2 = 4x4x22
This inequality hold beginning at x=1This inequality hold beginning at x=1
Choosing k=1, C=4, xChoosing k=1, C=4, x22+2x+1 is O(x+2x+1 is O(x22))
![Page 10: Analysis of Algorithms](https://reader036.vdocuments.mx/reader036/viewer/2022062321/56813291550346895d99263b/html5/thumbnails/10.jpg)
Show f(x) = 7xShow f(x) = 7x33 is O(x is O(x22))
Prove:Prove: |7x|7x33|| C C | | xx2 2 ||
Assuming k > 0, divide both sides by xAssuming k > 0, divide both sides by x22
7x7x C C
No such C exists, since x is arbitrarily largeNo such C exists, since x is arbitrarily large (Choose any C, and there is an x > C)(Choose any C, and there is an x > C)
7x7x33 is is notnot O(x O(x22))
![Page 11: Analysis of Algorithms](https://reader036.vdocuments.mx/reader036/viewer/2022062321/56813291550346895d99263b/html5/thumbnails/11.jpg)
Commonly used Big-O values Commonly used Big-O values (orders of magnitude)(orders of magnitude)
Big-O Big-O Name Name O(1)O(1) Constant time Constant time LowerLower
O(log n)O(log n) Logarithmic time Logarithmic time
O(n)O(n) Linear time Linear time
O(n log n) O(n log n)
O(nO(n22)) Quadratic time Quadratic time
O(nO(n33)) Cubic time Cubic time
O(nO(nkk) ) Polynomial time Polynomial time
O(2O(2nn)) Exponential time Exponential time HigherHigher
O(n!) O(n!) Factorial time Factorial time Lower is always O(higher) for nLower is always O(higher) for n
![Page 12: Analysis of Algorithms](https://reader036.vdocuments.mx/reader036/viewer/2022062321/56813291550346895d99263b/html5/thumbnails/12.jpg)
Big-O Big-O Example algorithmExample algorithm
O(1)O(1) Assigning value to ith array element; Assigning value to ith array element;
always the same number of stepsalways the same number of steps
not necessarily shortnot necessarily short
a program which takes 320 steps, a program which takes 320 steps,
reguardless of input valuesreguardless of input values
O(log n)O(log n) Binary search, Binary search,
What power of 2 is greater than a input number? What power of 2 is greater than a input number?
A loop whose terminal value is being successively A loop whose terminal value is being successively halved or doubledhalved or doubled
![Page 13: Analysis of Algorithms](https://reader036.vdocuments.mx/reader036/viewer/2022062321/56813291550346895d99263b/html5/thumbnails/13.jpg)
Big-O Big-O Example algorithmExample algorithm
O(n)O(n) Printing all elements of an array; Printing all elements of an array; searching an unordered array; searching an unordered array; A loop which executes 1 to N times A loop which executes 1 to N times (where N is the input size, (where N is the input size, the number of data values being the number of data values being
processed)processed)
O(n log n) O(n log n) Faster sorts; Faster sorts; A loop whose index is being halved/doubled A loop whose index is being halved/doubled inside a loop executing from 1 to Ninside a loop executing from 1 to N
![Page 14: Analysis of Algorithms](https://reader036.vdocuments.mx/reader036/viewer/2022062321/56813291550346895d99263b/html5/thumbnails/14.jpg)
Big-O Big-O Example algorithmExample algorithm
O(nO(n22) ) Slower sorts; Slower sorts; A loop which executes from 1 to N times A loop which executes from 1 to N times inside a loop executing from 1 to N inside a loop executing from 1 to N
timestimes
O(nO(n33) ) Incrementing all the elements in a NxNxN array; Incrementing all the elements in a NxNxN array; A 1..N loop inside a 1..N loop inside a 1..N loopA 1..N loop inside a 1..N loop inside a 1..N loop
O(nO(nkk) ) kk levels of loops inside loops levels of loops inside loops
![Page 15: Analysis of Algorithms](https://reader036.vdocuments.mx/reader036/viewer/2022062321/56813291550346895d99263b/html5/thumbnails/15.jpg)
Big-O Big-O Example algorithmExample algorithm
O(2O(2nn)) List all the subsets of a set with n elements; List all the subsets of a set with n elements; practical only with small values of n; practical only with small values of n; n=64 takes 5 years on a supercomputern=64 takes 5 years on a supercomputer
O(n!) O(n!) List all the possible arrangements of a set with List all the possible arrangements of a set with n n elementselements
![Page 16: Analysis of Algorithms](https://reader036.vdocuments.mx/reader036/viewer/2022062321/56813291550346895d99263b/html5/thumbnails/16.jpg)
Table of Common Running TimesTable of Common Running Times
N Linear N Log N Quadratic Cubic
10 10 10.0 100 100020 20 26.0 400 800030 30 44.3 900 2700040 40 64.0 1600 6400050 50 84.9 2500 12500060 60 106.7 3600 21600070 70 129.2 4900 34300080 80 152.2 6100 48800090 90 175.9 8100 729000
100 100 200.0 10000 1000000
![Page 17: Analysis of Algorithms](https://reader036.vdocuments.mx/reader036/viewer/2022062321/56813291550346895d99263b/html5/thumbnails/17.jpg)
log n
n
n lo
g n
n2
2n
n!
![Page 18: Analysis of Algorithms](https://reader036.vdocuments.mx/reader036/viewer/2022062321/56813291550346895d99263b/html5/thumbnails/18.jpg)
Growth Rate of Some Growth Rate of Some FunctionsFunctions
0100200300400500600700800900
1000
0 10 20 30 40 50 60 70 80 90 100
Linear
nLogn
Quadratic
Cubic
![Page 19: Analysis of Algorithms](https://reader036.vdocuments.mx/reader036/viewer/2022062321/56813291550346895d99263b/html5/thumbnails/19.jpg)
Computing order of complexity of Computing order of complexity of an algorithman algorithm a.) Multiplicative constants do not matter.a.) Multiplicative constants do not matter.
O(3n) = O(n)O(3n) = O(n)
b.) Addition is done by taking the max. b.) Addition is done by taking the max. O(nO(n22) + O(n) = O(max(n) + O(n) = O(max(n22, n)) = O(n, n)) = O(n22)) O(nO(n33) + O(log n) = O(max(n) + O(log n) = O(max(n33, log n)) = O(n, log n)) = O(n33))
c.) Multiplication remains multiplication.c.) Multiplication remains multiplication. O(n) * O(log n) = O(n log n)O(n) * O(log n) = O(n log n) O(n) * O(nO(n) * O(n22) = O(n) = O(n33))
![Page 20: Analysis of Algorithms](https://reader036.vdocuments.mx/reader036/viewer/2022062321/56813291550346895d99263b/html5/thumbnails/20.jpg)
Example ComplexitiesExample Complexities
4n4n3 3 + 20n + 30 = O(n+ 20n + 30 = O(n33)) n + 10000 = O(n)n + 10000 = O(n) 4n4n4 4 + 20n + 30 = O(n+ 20n + 30 = O(n44)) 22nn + n + n33 = O(2 = O(2nn)) 200 = O(1)200 = O(1)
![Page 21: Analysis of Algorithms](https://reader036.vdocuments.mx/reader036/viewer/2022062321/56813291550346895d99263b/html5/thumbnails/21.jpg)
What is the Big-O ofWhat is the Big-O of
outputoutput inputinput assignmentassignment ifif if-elseif-else
a blocka block while loopwhile loop for loopsfor loops nested for loopsnested for loops
![Page 22: Analysis of Algorithms](https://reader036.vdocuments.mx/reader036/viewer/2022062321/56813291550346895d99263b/html5/thumbnails/22.jpg)
Running Time of StatementsRunning Time of Statements
Simple statements Simple statements (i.e. initialization of variables) are O(1)(i.e. initialization of variables) are O(1)
Loops are O(g(n) f(n))Loops are O(g(n) f(n)) g(n) is upper bound on number of loop g(n) is upper bound on number of loop
iterations iterations f(n) is upper bound on the body of the loop. f(n) is upper bound on the body of the loop.
((If g(n) and f(n) are constant, then this is If g(n) and f(n) are constant, then this is constant timeconstant time).).
![Page 23: Analysis of Algorithms](https://reader036.vdocuments.mx/reader036/viewer/2022062321/56813291550346895d99263b/html5/thumbnails/23.jpg)
Running Time of Statements Running Time of Statements (continued)(continued) Conditional statements are Conditional statements are
O(max(f(n),g(n))) O(max(f(n),g(n))) where f(n) is upper bound on then part and where f(n) is upper bound on then part and g(n) is upper bound on else partg(n) is upper bound on else part
Blocks of statements with complexities Blocks of statements with complexities ff11(n), f(n), f22(n), ..,f(n), ..,fkk(n) (n)
have complexity have complexity O(fO(f11(n) + f(n) + f22(n)+ ...+ f(n)+ ...+ fkk(n))(n))
![Page 24: Analysis of Algorithms](https://reader036.vdocuments.mx/reader036/viewer/2022062321/56813291550346895d99263b/html5/thumbnails/24.jpg)
Simple AnalysisSimple Analysis
cin >> n; // 1if (n > 20) // 1
cout << “Number is > 20” << endl; // 1else
cout << “Number is <= 20” << endl; // 1
T(n) = 2 + max (1,1) = 3 = O(1)
![Page 25: Analysis of Algorithms](https://reader036.vdocuments.mx/reader036/viewer/2022062321/56813291550346895d99263b/html5/thumbnails/25.jpg)
Example AnalysisExample Analysis
cin >> n; // 1factorial = 1; // 1
for (i = 2; i <=n; i++) // 1 init + 1 test + //(1 test + 1 inc) per iteraton
factorial *= i; // 1
cout << factorial; // 1
T(n) = 5 + 3*(n-1) = O(n)
![Page 26: Analysis of Algorithms](https://reader036.vdocuments.mx/reader036/viewer/2022062321/56813291550346895d99263b/html5/thumbnails/26.jpg)
Another exampleAnother example
cin >> n; // 1if (n > 0) { // 1 factorial = 1; // 1 for (i = 2; i <=n; i++) // 1 initialization +1 test +
//(1 test + 1 inc) per iterationfactorial *= i; // 1
cout << factorial; // 1}else cout << “Can’t do factorial of” << n; // 1 T(n) = 2 + max(4+3*(n-1), 1) = O(n)
![Page 27: Analysis of Algorithms](https://reader036.vdocuments.mx/reader036/viewer/2022062321/56813291550346895d99263b/html5/thumbnails/27.jpg)
Analysis of simple function Analysis of simple function callscalls
int factorial(int n){ int fact=1; for (i = 2; i <=n; i++)
fact *= i; return fact;} void main(){ int n; // 1
cin >> n; // 1 cout << factorial(n) << endl; //O(n)}
Main is O(n)
![Page 28: Analysis of Algorithms](https://reader036.vdocuments.mx/reader036/viewer/2022062321/56813291550346895d99263b/html5/thumbnails/28.jpg)
Analysis of Nested LoopsAnalysis of Nested LoopsEXAMPLE 1:for (int i = 0; i < n; i++) // n*O(n) = O(n2) for (int j=0; j<n;j++) // n*O(1) = O(n) x++; // O(1)
EXAMPLE 2:for (int i = 0; i < n; i++) for (int j=i; j<n;j++) x++; // O(1)
T(n) = n+(n-1) + (n-2)+…+ 1 = n(n+1)/2 = O(n2)
![Page 29: Analysis of Algorithms](https://reader036.vdocuments.mx/reader036/viewer/2022062321/56813291550346895d99263b/html5/thumbnails/29.jpg)
Summing blocks of Summing blocks of statementsstatements
for (int i = 0; i < n; i++) // O(n2) for (int j=0; j<n;j++) x++;
// Next block follows in same programfor (int i = 0; i < n; i++) // O(n) x++;
T(n) = O(n2) + O(n) = O(n2 + n) = O(n2)
![Page 30: Analysis of Algorithms](https://reader036.vdocuments.mx/reader036/viewer/2022062321/56813291550346895d99263b/html5/thumbnails/30.jpg)
Loops with different limitsLoops with different limits
for (int i = 0; i < n; i++) // n*O(m) = O(mn) for (int j=0; j<m;j++) // m*O(1) = O(m) x++; // O(1)
![Page 31: Analysis of Algorithms](https://reader036.vdocuments.mx/reader036/viewer/2022062321/56813291550346895d99263b/html5/thumbnails/31.jpg)
Recursive AlgorithmsRecursive Algorithms
Complexity of a recursive algorithm is the Complexity of a recursive algorithm is the product ofproduct of
Amount of work done on any one levelAmount of work done on any one level Number of recursive callsNumber of recursive calls
![Page 32: Analysis of Algorithms](https://reader036.vdocuments.mx/reader036/viewer/2022062321/56813291550346895d99263b/html5/thumbnails/32.jpg)
Recursive FactorialRecursive Factorialint factorial(int n){ if ( n <= 1)
return 1; else return n * factorial(n - 1);} void main(){ int n; // 1
cin >> n; // 1 cout << factorial(n) << endl; //O(n)}
Main is O(n)
![Page 33: Analysis of Algorithms](https://reader036.vdocuments.mx/reader036/viewer/2022062321/56813291550346895d99263b/html5/thumbnails/33.jpg)
More complex loopsMore complex loops
for (i = 1;i < n; i *=2) x++;
for (i=n; i > 0; i/=2) x++;
Repetitive halving or doubling results in logarithmic complexity. Both loops are O(log n)
![Page 34: Analysis of Algorithms](https://reader036.vdocuments.mx/reader036/viewer/2022062321/56813291550346895d99263b/html5/thumbnails/34.jpg)
Worst-Case and Average-Worst-Case and Average-Case AnalysisCase Analysis Worst-case is a guarantee over Worst-case is a guarantee over all all
inputs of a given size inputs of a given size
Average-case is the running time as an Average-case is the running time as an average over all the possible inputs of a average over all the possible inputs of a given sizegiven size
![Page 35: Analysis of Algorithms](https://reader036.vdocuments.mx/reader036/viewer/2022062321/56813291550346895d99263b/html5/thumbnails/35.jpg)
What is the Big-O ofWhat is the Big-O of
outputoutput inputinput assignmentassignment ifif if-elseif-else
a blocka block while loopwhile loop for loopsfor loops nested for loopsnested for loops
![Page 36: Analysis of Algorithms](https://reader036.vdocuments.mx/reader036/viewer/2022062321/56813291550346895d99263b/html5/thumbnails/36.jpg)
![Page 37: Analysis of Algorithms](https://reader036.vdocuments.mx/reader036/viewer/2022062321/56813291550346895d99263b/html5/thumbnails/37.jpg)