meljun cortes algorithm analysis of algorithm efficiency

8/21/2019 MELJUN CORTES ALGORITHM Analysis of Algorithm Efficiency

1/37

Analysis of Algorithm efficiency

Design and Analysis of Algorithm

* Property of STI

Page 1 of 37

Anal ysi s o f A lgo r i t hm

Eff ic iency

c Definition of Analysis of Algorithm

c Time and Space Efficiency

c Asymptotic Order of Growth

c Algorithm Case Efficiencies

c Asymptotic Notations

c Basic Efficiency Classes

c Mathematical Analysis of RecursiveAlgorithms

c Mathematical Analysis of Non-Recursive

Algorithms

c Empirical Analysis of Algorithm


2/37



* Property of STI

Page 2 of 37

An alysi s Fr am ew o r k

c Analysis of algorithm

q used in a narrower technical sense to mean an

investigation of algorithms efficiency withrespect to the two resources running time andmemory space

c main considerations:

q simplicity, generality and efficiency of algorithmq speed and memory of computer nowadays


3/37



* Property of STI

Page 3 of 37


c Space efficiency

q amount of memory required by an algorithm torun the application

c Time efficiency

q indicates how fast an algorithm is processedin the machine

c Measuring an Input’s Sizeq examine an algorithm’s efficiency as a

function of some parametern indicating thealgorithm’s input size

c Techniques in Measuring Running Time

q Using standard unit of time measurement

q Counting the number of times eachalgorithm’s operations is executed


4/37



* Property of STI

Page 4 of 37


cDisadvantages of using techniques in analysisof algorithm:

q Dependence of the computer’s speed

q Dependence on the quality of the programimplementing the computer

q Dependence on the compiler used ingenerating the machine code

q Difficulty of clocking the actual running time ofthe program


5/37



* Property of STI

Page 5 of 37


cOrder of Growth

q how do time or space requirements grow as

the problem size increases

Approximate value of different important functionsfor algorithm analysis


6/37



* Property of STI

Page 6 of 37


c Worst-case efficiency

c Best-case efficiency

c Average-case efficiency

c Amortized efficiency


7/37



* Property of STI

Page 7 of 37

Asym p t o t i c No tat i o n

c Asymptotic complexity

q way of expressing the main component of thecost of an algorithm using idealized units of

computational work

c Notation used in comparing and ranking theorder of growth

q

Big-0h Notation (0

)

q Big-0mega Notation (? )

q Big-Theta Notation (T)


8/37



* Property of STI

Page 8 of 37


c Big-0 Notation (0)

q introduced in 1854 by Paul Bachmann

q most commonly used notation for specifying

asymptotic complexity

c Big-O means “on the order of”

q formal method of expressing the upper boundof an algorithm's running time.

c Writef(n) = O(g(n)) if there exist constantsc > 0 , n0 > 0 such that0 = f(n) = cg(n) for alln = n 0


9/37



* Property of STI

Page 9 of 37


c Practical Example:

q O (n)

• printing a list of n items to the screen, lookingat each item once

q O (log n )

• taking a list of items, cutting it in half

repeatedly until there's only one item left

q O (n2 )

• taking a list of n items, and comparing everyitem to every other item


10/37



* Property of STI

Page 10 of 37


c Theory Example:

q Given:

f(n) = 2 n + 8 , and g(n) = n2

q Find c , so that 2n + 8


11/37



* Property of STI

Page 11 of 37


c The Big-O notation can be derived from f(n)using the following steps:

q set the coefficient of the term to 1

q keep the largest term in the function anddiscard the others

q terms are ranked from lowest to highest

log 2 n, n, nlog 2 n, n 2,n 3 . . . .n k 2 n n!


12/37



* Property of STI

Page 12 of 37


c Example#1:

q remove all coefficients

• n2 + n

q remove the smaller factors• n2

q Big-O notation:

• o(f(n) = o(n2 )


13/37



* Property of STI

Page 13 of 37


c Example#2:

f(n) = a j n k + a j-1 n

k-1 + …+a 2 n 2 +a 1 n+a 0

q remove all coefficients

• f(n) = n k +n k -1 + …+n 2 +n+1

q the order of a polynomial expression is:

• o(f(n) = o(n k )


14/37



* Property of STI

Page 14 of 37


c Big-0 Measures of Efficiency


15/37



* Property of STI

Page 15 of 37


c Big-0mega Notation (? )

q t(n) ? (g(n))

q Sample formal proof wheren 3 ? (n 2 ) • B 3 = n 2 for all n = 0

q selectc =1 and n 0 = 0


16/37



* Property of STI

Page 16 of 37


c Big-Theta Notation (T)

q applied in tight bound

q function t(n) is said to be in T(g(n)), denotedt(n) T(g(n))

• if t(n) is bounded both above and below by somepositive constant multiples of g(n) for all large n

• If there exist some positive constant c1 and c2 and

some nonnegative integer n0 such that:

c 2 g (n) = t (n) = c 1 g(n) for all n = n 0


17/37



* Property of STI

Page 17 of 37


c For example, let us prove that right inequality(the upper bound):

q First, we prove the right inequality (the upperbounds)


18/37



* Property of STI

Page 18 of 37


c Continuation:

q Second, we prove the left inequality (the lowerbounds).

q Hence:


19/37



* Property of STI

Page 19 of 37

Basi c Eff i c i en cy Cl asses

c Algorithm’s efficiency

q a function of the number of elements to beprocessed

q general format:

f (n) = efficiency


20/37



* Property of STI

Page 20 of 37


c Linear Loops

q Example:• identify how many times the body of the loop is

repeated:

q the number of iterations is directlyproportional to the loop factor

q the higher the factor, the higher the number of

loopsq efficiency is directly proportional to the

number of iterations

1 x = 02 loop (x >= 1000)

1 application code2 x = x plus 1

3 end loop

f (n) = n


21/37



* Property of STI

Page 21 of 37


c Linear Loops

q Example:

q the number of iterations is half the loop factor

q Efficiency is proportional to half the factor

1 x = 02 loop (x >= 1000)

1 application code2 x = x plus 2

3 end loop

f (n) = n/2


22/37



* Property of STI

Page 22 of 37


c Logarithmic Loops

q a loop in which the controlling variable ismultiplied or divided in each loop

q determine how many times the body of theloops is repeated:

1 x = 1

2 loop (x >= 1000)1 application code2 x = x multiply 2

3 end loop

1 x = 10002 loop (x >= 1)

1 application code2 x = x divide 2

3 end loop


23/37



* Property of STI

Page 23 of 37


c Logarithmic Loops

f (n) = [log2n]

Analysis of Multiplication and

Division Loops


24/37



* Property of STI

Page 24 of 37


c Nested Loop

q loop within a loop

q includes Linear Logarithmic, Dependent

Quadratic, and Quadratic

Iterations = outer loop iterations x

inner loop iterations


25/37



* Property of STI

Page 25 of 37


c Linear logarithmic:

c number of iteration in the inner loop is [log210]

1 x = 1

2 loop (x


26/37



* Property of STI

Page 26 of 37


c Dependent quadratic :

c inner loop depends on the outer loop for one of itsfactors

c the number of iteration is 55

1 x = 10

2 loop (x


27/37



* Property of STI

Page 27 of 37


c average is 5.5


28/37



* Property of STI

Page 28 of 37


c Quadratic:

1 x = 10

2 loop ( x


29/37



* Property of STI

Page 29 of 37

SEATWORK

c Algorithm efficiency application:Statewhether the statement below isTrueorFalse.

Defend your answer.

1. 20n2+ 2n + 5 = O(n2)

2. 100n + 5 = O(n2)


30/37



* Property of STI

Page 30 of 37

M at h em at i cal An alysi s o f

Recu r si ve A lgo r i t hm s

c General plan of analyzing Time Efficiency ofrecursive algorithm

1. Decide on a parameter (or parameters)indicating an input’s size.

2. Identify the algorithm’s basic operation.

3. Check whether number of times the basicoperation is executed can vary on differentinputs of the same size

4. Set up recurrence relation, with appropriateinitial condition, for the number of times thebasic operation is executed.

5. Solve the recurrence or at least ascertain the

order of growth of its solution.


31/37



* Property of STI

Page 31 of 37

M at h em at i cal An alysi s o f

Recu r si ve A lgo r i t hm s

c Compute the factorial function f(n) = n! for anarbitrary nonnegative integer n. Since:

n! = 1 , ….(n-1) . n = (n-1)! . n for n =1

c With 0 != 1 by definition, we can compute F(n) =F(n-1) . n with the following recursive algorithm.

c F(n) = F(n-1) . n for everyn > 0

c M(n) = M (n-1 ) + 1 for everyn > 0

1 if n = 0 return 12 else

3 return F(n-1) * n


32/37



* Property of STI

Page 32 of 37

M ath em at i cal An alysi s o f

N o n -Recu r si ve A l go r i t h m s

c General Plan for Analyze Time Efficiency

1.

Decide on a parameter (or parameters)indicating an input’s size.

2. Identify the algorithms basic operation.

3. Check whether the number of times the basicoperation is executed depends only on thesize of an input.

4. Set up a sum expressing the number of timesthe algorithm’s basic operation is executed.

5. Using standard formulas and rules of summanipulation, either find a close-form formulafor the count, or the very least, establish itsorder of growth.


33/37



* Property of STI

Page 33 of 37

M ath em at i cal An alysi s o f

N o n -Recu r si ve A l go r i t h m s

c Example:

c find a formula in expressingC as a functionof size n

maxValue = A[0]loop index = n

if A[index ] > maxValue

return maxValue


34/37



* Property of STI

Page 34 of 37

Em p i r i cal An al ysi s o f

A l g o r i t hm c Empiricalanalysis

q alternative of the mathematical analysis ofalgorithms efficiency

c General Plan for Empirical Analysis ofAlgorithm Time Efficiency

1. Understand the experiments purpose.

2. Decide on efficiency metric M to be measure

and the measurement unit (an operation’scount vs. a time unit).

3. Decide on characteristics of the input sample(its range, size and so on).

4. Prepare a program implementing thealgorithm ( or algorithms) for the

experimentation.

5. Generate a sample of inputs.

6. Run algorithms on the sample inputs andrecord the data observed.

7. Analyzed the data obtained.


35/37



* Property of STI

Page 35 of 37

Em p i r i cal An al ysi s o f

A l g o r i t hm

c Limitations of Empirical Analysis

q Quality of implementation

q Extraneous factors

q Requires a working program


36/37



* Property of STI

Page 36 of 37

SEATWORK

c Calculate the run-time efficiency of thefollowing pseudocode for program

segments.(10 points each)

x = 0

loop( x


37/37


ASSIGNM ENT

c Research for the following topics:

1. Definition of brute force algorithm techniquedesign technique

2. Applications of brute force algorithm designtechnique

c Note: The research work should be in aprinted copy and compiled in a folder with atitle page.

meljun cortes algorithm analysis of algorithm efficiency

Documents