Algorithm Analysis

Chapter 5

Algorithm

• An algorithm is a clearly specified set of instructions which, when followed, solves a problem.– recipes– directions for putting together a bicycle– programs

Algorithm Analysis

• The process by which we determine the amount of resources used by the algorithm

• Resources usually measured– time– space

• We will concentrate on run time of algorithms

Common functions encountered

• Some commonly encountered functions are:– linear– N log N– quadratic– cubic

• The next slide shows the graphs of these functions

Growth RatesGrowth rates

-20

0

20

40

60

80

100

120

0 20 40 60 80 100 120

N

Series1

Series2

Series3

Series4

Example - Downloading

• Suppose we wish to download a file – 2 seconds for setting up the connection– downloading proceeds at 2.8 Kb/sec

• If the downloaded file is N kilobytes, the time to download is T(N) = N/2.8 + 2– 80K file takes about 31 seconds– 160K file takes about 59 seconds

• This is a linear algorithm

Big-O notation

• We use big-O to specify growth rates of algorithms– linear functions are specified as O(N)– quadratic functions as O(N2)

Functions in Increasing OrderFunction Name

c

log N

log2 N

N

N log N

N2

N3

2N

Constant

Logarithmic

Log-squared

Linear

N log N

Quadratic

Cubic

Exponential

Examples

• Minimum element in an array– Given an array of N elements, find the smallest.

• Closest points in the plane– Given N points in the plane, find the pair of

points that are closest together.

• Collinear points in the plane– Given N points in the plane, determine if any

three are on the same line.

Minimal Element in an Array

• Algorithm– Initialize min as the first element– Scan thru the other N-1 elements, updating min

as appropriate

• There is a fixed amount of work for each element of the array

• The running time is O(N)

Closest Point in the Plane

Algorithm

– Calculate the distance between each pair of points [there are N(N-1)/2 pairs of points]

– find the minimum of these distances

Finding the minimum of N(N-1)/2 distances by this algorithm is O(N2)

Collinear Points in the Plane

• Algorithm– Enumerate all triples of the N points. There are

N(N-1)(N-2)/6 of these triples.– check to see if the three points are on the same

line

• This is a O(N3) algorithm

Maximum Contiguous Subsequence Sum Problem

• Given (possibly negative) integers A1, A2,…, AN , find (and identify the subsequence corresponding to) the maximum value Ai + Ai+1 +…+ Ak .

• The maximum contiguous subsequence sum is zero if all the integers are negative

Examples

• With input of [-2, 11, -4, 13, -5, 2], then the maximum sum is 20 using items at positions 2 through 4.

• With input of [ 1, -3, 4, -2, -1, 6], then the maximum sum is 7 using the last four items.

Obvious O(N3) AlgorithmmaxSum = 0;for (i=0; i < N; i++)

for (j=i; j < N; j++) {

thisSum = 0;for (k=i;k <= j; k++)

thisSum += a[k];if (thisSum > maxSum)

{maxSum = thisSum;seqStart = i;seqStop = j;

}}

return maxSum;

Improving the Algorithm

• If we can eliminate one of the loops, we will be able to reduce this algorithm to a O(N2) algorithm.

• Observe that Ai + Ai+1 +…+ Ak = (Ai + Ai+1 +…+ Ak-1 ) + Ak . The inner loop is used to calculate Ai + Ai+1 +…+ Ak-1 , then this result is thrown away and then Ai + Ai+1 +…+ Ak is calculated (duplicating work)

Improved O(N2) Algorithm maxSum = 0; for (i=0; i < N; i++) {

thisSum = 0;for (j=i; j < N; j++)

{thisSum += a[j];if (thisSum > maxSum)

{maxSum = thisSum;seqStart = i;seqStop = j;

}}

}return maxSum;

Observations

1.No maximum contiguous subsequence begins with a subsequence which has a negative sum.

2. All subsequences which border on the maximum contiguous subsequence must have negative or zero sums

Observations

3.Let Ai,j be the first sequence with Si,j < 0. Then, for any i <= p <= j and p <=q, Ap,q is not a maximum contiguous subsequence or is equal to (in sum) an already seen maximum contiguous subsequence. (Theorem 5.3 on page 159)

Improving again

• Using these observations, any time the current subsequence sum (thisSum) becomes negative, we can begin a new subsequence sum at the next position. Thus we can rewrite the algorithm using only a single loop.

Linear Algorithm maxSum = thisSum = 0;for (i=0, j=0; j < N; j++)

{thisSum += a[j];if (thisSum > maxSum)

{maxSum = thisSum;seqStart = i;seqStop = j;

}else if (thisSum < 0)

{i = j+1;thisSum = 0;

}}return maxSum;

Big-O

• Definition: T(N) is O(F(N)) if there are positive constants c and N0 such that T(N)cF(N) whenever N N0 .

• This means that T is eventually bounded above by some constant times F, so F could be used an a crude upper-bound estimate of the function T.

• This means that the growth rate of T is no worse than the growth rate of F.

Big- (omega)

• Definition: T(N) is (F(N)) if there are positive constants c and N0 such that T(N) cF(N) whenever N N0 .

• This means that T is eventually bounded below by some multiple of F so, F could be used as a crude lower-bound estimate for T.

• This means that the growth rate of T is no better than the growth rate of F.

Big- (theta)

• Definition: T(N) is (F(N)) if and only if T(N) is O(F(N)) and T(N) is (F(N)).

• That means eventually T is bounded below by some constant times F and bounded above by some constant time F.

• This means that the growth rate of T is equal to the growth rate of F.

Little-o

• Definition: T(N) is o(F(N)) if and only if T(N) is O(F(N)) and T(N) is not (F(N)).

• This means that the growth rate of T is strictly better than the growth rate of F.

Summary

Mathematical Expression Relative Rates of Growth

T(N) = O(F(N)) Growth of T(N) growth of F(N)

T(N) = (F(N)) Growth of T(N) growth of F(N)

T(N) = (F(N)) Growth of T(N) = growth of F(N)

T(N) = o(F(N)) Growth of T(N) < growth of F(N)

The Logarithm

• Definition: For any B, N > 0, logBN = K if and only if BK = N

• In Computer Science, base 2 logarithms are used almost exclusively.

• The use of the symbol log indicates log2 unless otherwise indicated

Use of logarithms

• Bits in a binary number– The number of bits needed to represent N

consecutive integers is log N

• Repeated Doubling– Starting with X=1, the number of times needed

to double X to reach at least N is log N

Searching

• Sequential Search– the average number of comparisons made to

find an item in an unordered list of N items is N/2 = O(N)

– An unsuccessful search requires testing every item in the list, so is also O(N)

Binary Search

• If searching a sorted list of N items, we can do betterlow = 1; high = N; while (low < high)

{mid = (low+high)/2;if (A[mid]< x)

low = mid +1;else high = mid;

} if (X == A[low])

return low; return -1;

Homework

Pages 176-181: # 3, 4, 5(a,b), 6, 8, 11, 12, 14,

15, 17, and 28.