Algorithm Analysis

Algorithm

An algorithm is a clearly specified set of instructions which, when followed, solves a problem. recipesdirections for putting together a bicycleprograms

Algorithm Analysis

The process by which we determine the amount of resources used by the algorithm

Resources usually measured timememory space

We will concentrate on run time of algorithms

Common functions encountered

Some commonly encountered functions are: ,linear ,quadratic ,cubic

The next slide shows the graphs of these functions

)log(NN2N

3N

N

Growth Rates

Growth rates

-20

0

20

40

60

80

100

120

0 20 40 60 80 100 120

N

Series1

Series2

Series3

Series4

Running Time T(N)Example 1

Problem: Given an array of N elements, find the smallest.

Solution:1. key = Integer.Mininum_Value

2. for( int i=0 ; i< inarray.length ; i++ )

3. if ( inarray[i] < key ) key = inarray[i];

Running Time T(N)Example 2

Suppose we wish to download a file 2 seconds for setting up the connectiondownloading proceeds at 2.8 K/sec

If the downloaded file is N kilobytes, the time to download is T(N) = N/2.8 + 280K file takes about 31 seconds160K file takes about 59 seconds

This is a linear algorithm

Functions in Increasing Order

Function Name

c

log N

log2 N

N

N log N

N2

N3

2N

Constant

Logarithmic

Log-squared

Linear

N log N

Quadratic

Cubic

Exponential

Measurement of T(N)

Place your timer before/after the algorithm Run the algorithm multiple times Take the average of the total time

1. time = System.currentTimeMillis();

2. for (int i = 0; i<num_iteration ; i++)

3. // Your algorithm comes here

4. newtime = System.currentTimeMillis();

5. output (newTime-time)/num_iteration

Big-O notation

We use big-O to specify growth rates of algorithms linear functions are specified as O(N)quadratic functions as O(N2)

Formal Definition of Big-O

Definition: T(N) is O(F(N)) if there are positive constants c and N0 such that T(N)cF(N) whenever N N0 .

Usually T(N) is a complicated function involving N.

We choose F(N) as simple as possible.

Examples of Big-O

Find equivalent Big-Oh functions for the following runtime T(N).

T1(N)=N2

T2(N)=20*N

T3(N)=N+1000*N2

T4(N)=N+60 N3 -1000*N2

T5(N)=N*log(N)+N2

T6(N)=50*N*log(N) + N

T7(N)=N + 60 N3 - N*log(N)

T8(N)= 20*N +100*N2

T9(N)=90*N2 + 60 N3

Examples of Big-O

Algorithm Analysis

A1: Minimum element in an arrayGiven an array of N elements, find the

smallest.

A2: Closest points in the planeGiven N points in the plane, find the pair of

points that are closest together.

A3: Collinear points in the planeGiven N points in the plane, determine if

any three are on the same line.

A1: Minimal Element in an Array

Algorithm Initialize min as the first element Scan thru the other N-1 elements, updating min as

appropriate

There is a fixed amount of work for each element of the array

The running time is O(N)

A2: Closest Point in the Plane

AlgorithmCalculate the distance between each pair of

points [there are N(N-1)/2 pairs of points] find the minimum of these distances

Finding the minimum of N(N-1)/2 distances by this algorithm is O(N2)

There is a O(N log N) algorithm to solve this problem, and one thought to be O(N).

A3: Collinear Points in the Plane

AlgorithmEnumerate all triples of the N points. There

are N(N-1)(N-2)/6 of these triples.check to see if the three points are on the

same lineThis is a O(N3) algorithmThere is a clever quadratic algorithm for

solving this problem

Maximum Contiguous Subsequence Sum Problem

Given (possibly negative) integers A1, A2,…, AN , find (and identify the subsequence corresponding to) the maximum value Ai + Ai+1 +…+ Ak .

The maximum contiguous subsequence sum is zero if all the integers are negative

Examples

With input of [-2, 11, -4, 13, -5, 2], then the maximum sum is 20 using items at positions 2 through 4.

With input of [ 1, -3, 4, -2, -1, 6], then the maximum sum is 7 using the last four items.

Obvious O(N3) Algorithm

maxSum = 0;for (i=0; i < N; i++)

for (j=i; j < N; j++){thisSum = 0;for (k=i;k <= j; k++)

thisSum += a[k];if (thisSum > maxSum){

maxSum = thisSum;seqStart = i;seqStop = j;

}}

return maxSum;

Improving the Algorithm

If we can eliminate one of the loops, we will be able to reduce this algorithm to a O(N2) algorithm.

Observe that Ai + Ai+1 +…+ Ak = (Ai + Ai+1 +…+ Ak-1 ) + Ak . The inner loop is used to calculate Ai + Ai+1 +…+ Ak-1 , then this result is thrown away and then Ai + Ai+1 +…+ Ak is calculated (duplicating work)

Improved O(N2) Algorithm

maxSum = 0;for (i=0; i < N; i++){

thisSum = 0;for (j=i; j < N; j++){

thisSum += a[j];if (thisSum > maxSum){

maxSum = thisSum;seqStart = i;seqStop = j;

}}

}return maxSum;

Observations

no maximum contiguous subsequence sum begins with a subsequence which has a negative sum

all subsequences which border on the maximum contiguous subsequence must have negative or zero sums

Improving again

Using these observations, any time the current subsequence sum (thisSum) becomes negative, we can begin a new subsequence sum at the next position since the subsequence we are searching for cannot begin with a subsequence with negative sum.

Linear Algorithm

maxSum = thisSum = 0;for (i=0, j=0; j < N; j++){

thisSum += a[j];if (thisSum > maxSum){

maxSum = thisSum;seqStart = i;seqStop = j;

}else if (thisSum < 0){

i = j+1;thisSum = 0;

}}return maxSum;

Big-O

Definition: T(N) is O(F(N)) if there are positive constants c and N0 such that T(N)cF(N) whenever N N0 .

This means, that for sufficiently large N, T(N) is bounded by some multiple of F(N).

This means that the growth rate of T is no worse than the growth rate of F.

Big- (omega)

Definition: T(N) is (F(N)) if there are positive constants c and N0 such that T(N) cF(N) whenever N N0 .

This means, that for sufficiently large N, T(N) is bounded below by some multiple of F(N).

This means that the growth rate of T is no better than the growth rate of F.

Big- (theta)

Definition: T(N) is (F(N)) if and only if T(N) is O(F(N)) and T(N) is (F(N)).

This means that the growth rate of T is equal to the growth rate of F.

Little-o

Definition: T(N) is o(F(N)) if and only if T(N) is O(F(N)) and T(N) is not (F(N)).

This means that the growth rate of T is strictly better than the growth rate of F

Summary

Mathematical Expression Relative Rates of Growth

T(N) = O(F(N)) Growth of T(N) growth of F(N)

T(N) = (F(N)) Growth of T(N) growth of F(N)

T(N) = (F(N)) Growth of T(N) = growth of F(N)

T(N) = o(F(N)) Growth of T(N) < growth of F(N)

The Logarithm

Definition: For any B, N > 0, logBN = K if and only if BK = N

In Computer Science, base 2 logarithms are used almost exclusively.

The use of the symbol log indicates log2 unless otherwise indicated

Use of logarithms

Bits in a binary numberThe number of bits needed to represent N

consecutive integers is log NRepeated Doubling

Starting with X=1, the number of times needed to double X to reach at least N is log N

Searching

Sequential Search the average number of comparisons made

to find an item in an unordered list of N items is N/2 = O(N)

An unsuccessful search requires testing every item in the list, so is also O(N)

Binary Search

If searching a sorted list of N items, we can do better

low = 1; high = N; while (low < high)

{mid = (low+high)/2;if (A[mid]< x)

low = mid +1;else high = mid;

} if (X == A[low])

return low; return -1;

Limitations of Big-O

In the following definition of O:

T(N) is O(F(N)) if there are positive constants c and N0 such that T(N)cF(N) whenever N N0 .

N0 can be a very large number.

The positive constants c can be a large number.

Example : Compare the running time two algorithms

Algorithm1 = 10000N

Algorithm2 = 2N log(N)

Other Issues of Algorithm Analysis

Memory accessDisk access Average-case v.s. worst-case running

time.Implementation Difficulties