1

Bounding Facts

Prof. Dan Barrish-Flood

Algorithms (su04)

2

More Bounding Facts

If f(n) = o(g(n)), then f(n) (g(n))

Proof:

Assume

0)(

)(

n

)o"" of definition(by But

,)(

)(

:)(by Divide

for ),()(

lim

01

01

ng

nf

nnng

nfc

ng

nnnfngc

3

More Bounding Facts (continued)

So eventually ( for n large enough)

ion!Contradict)(

)( .1c

ng

nf

4

Summations

ii

ii

ii

ii

i ii

ii

ii

n

n

ii

bacbica

baibia

acca

aaaa

)(

)(

)(

:Linearity

100

Example:

(5i + 2i)2

i

5 2

5 2

2

2

i i

i i

i i

i i

5

Common Summations

)(2)(

2)1( =

21

22

1

nnnnn

nin

i

122

22Then .2Let :Proof

1224212

1

1

10

1

0

n

n

i

in

i

i

nnn

i

i

SSS

SS

6

In general:

1

0

2

0

12

0

0

1

0

1 =)1(

1 substact

=

:Proof

1

1 then ,1 if that noting and

1

1

nn

i

i

nn

i

i

nnn

i

i

i

inn

i

i

rrr

rrrr

rrrrrr

rrr

r

rr

So...

rr

ri

i

n n

0

1 11

divide by (r – 1)

7

Running Times for Some Simple Iterative Algorithms

CLEAR-MATRIX(A[1n,1n])

1 for i 1 to n

2 do for j 1 to n

3 do A[i,j]=0

T n n n n( ) ( ) ( ) ( ) 2 2

0 0 0

0 0 0

0 0 0

8

Running Times for Some Simple Iterative Algorithms (continued)

IDENTITY-MATRIX(A[1n,1n])

1 CLEAR-MATRIX(A)

2 for i 1 to n do

3 do A[i,i]=1

T n n n n( ) ( ) ( ) ( ) 2 2

1 0 0

0 1 0

0 0 1

same asymptotic running time as CLEAR-MATRIX

9

CLEAR-LOWER-TRIANGLE(A[1n,1n])

(including diagonal)

1 for i 1 to n

2 do for j 1 to i

3 do A[i,j]=0

Line 3 happens i times.

i

i=1

n

i=1

n

n nn

( )( )

12

2

10

CLEAR-UPPER-TRIANGLE(A[1n,1n])

(including diagonal)

1 for i 1 to n

2 do for j i to n

3 do A[i,j]=0

Line 3 happens (n - i + 1) times.

Let Then we get

k

i=1

n

k=1

n

k n i

n nn

1

12

2

.

( )( )

11

AVERAGE-NEIGHBORS(A[1n])

1 for i 2 to n-1 do

2 sum 0

3 for j i1 to i+1 do

4 sum sum+A[j]

5 A[i] sum

1

3

8 3 2 5 1 6

4 4 3 2 413

13

2

T(n)=(n) LINEAR!

{

12

Mergesort

– To sort a list of n items:• sort the first half of the list• sort the second half• merge the sorted halves

– Sorting 1 or 0 items is trivial.– Merging two sorted lists is easily done in

linear ((n)) time. (Just take the smallest item from each list and put it on the merged list.)

13

MERGESORT(A[1nif n then

return A

3 else

4 B MERGESORT(A[1 n/25 C MERGESORT(A[n/2n])

6 return MERGE(B,C)

14

Mergesort Example

1 2 2 3 4 5 6 6

2 4 5 6 1 2 3 6

2 5

5 2 4 6 1 3 2 6

4 6 1 3 2 6

merge

merge merge

merge merge mergemerge

sorted sequence

The operation of mergesort on the array A= The lengthsof the sorted sequences being merged increases as the algorithm progressesfrom bottom to top.

15

The Call Tree for Mergesort

MS(n/4)

MS(n)

MS(n/2) MS(n/2)

MS(n/4) MS(n/4) MS(n/4)

...MS(1) ...................................... MS(0)...........

16

What is the running time of Mergesort?

Let’s call it T(n).

time taken by

recursive calls

T(n) = 2T(n/2) + n

T(1) = 1

This is a recurrence equation, or recurrence for short.

T(n) = + time to merge

17

Solving Recurrences

Method 1: Know the answer.– Not recommended as a general technique.

Method 2: Recursion Trees– Good for intuition

Method 3: Telescoping + Transformations– Fairly general

18

Recursion Trees

1. Draw the call tree for the algorithm.

2. Label each node with the work done at that invocation only.

3. Determine the total work done at each level of the tree.

4. Sum the values at each level to get the total running time.

19

Recursive Factorial T(1) = 1

FACT(n) T(n) = T(n-1) + 1, n ≥ 2

1 if n then Call tree:

2 return 1 FACT(n)

3 else |

4 return n*FACT(n-1) FACT(n-1)

|

FACT(n-2)

FACT(1)

20

Recursion Tree for FACT

1 1

1 1

1 1

1 1

n

n levels

total workat each level

21

Recursion Tree for Mergesort

n n

n/2 n

n/4 n

n n(lgn+1)

n/2

n/4 n/4 n/4

total work at each level

lgn + 1levels

n(lgn+1) = nlgn + n = Θ(nlgn)

22

TelescopingConsider

T(n) = T(n-1) + 1, n≥1

T(0) = 1

(The recurrence for

FACT(n).)

“world’s simplest recurrence”

We can solve it like this:

T(n) - T(n-1) = 1

T(n-1) - T(n-2) = 1

T(n-2) - T(n-3) = 1

T(1) - T(0) = 1

T(n) - T(0) = n

T(n) = n+1

23

Generalizing ...The RHS’s can be different from each other - as long as they don’t contain T.

1)0(

1 ,2)1()( E.g.

T

nnTnT n

T n T n

T n T n

T T

n

n

( ) ( )

( ) ( )

( ) ( )

1 2

1 2 2

1 0 2

1

1

T n T i

i

n( ) ( )

0 2

1

T n i

i

nn( )

2 2 10

1

Check

24

Also...The LHS doesn’t have to be T(n). All we need is that

expr(n)-expr(n-1) = ...

nT(n) - (n-1)T(n-1) = 1 (n-1)T(n-1) - (n-2)T(n-2) = 1

2T(2) - T(1) = 1

nT(n) - T(1) = n-1

nT(n) = n, T(n) = 1

E.g. nT(n) = (n-1)T(n-1) + 1, T(1) = 1

25

Finally...

It doesn’t matter whether the “base case” is T(0), T(1), T(2), etc. - as long as it’s some constant.

Our goal is to get all recurrences into telescoping form.

26

Domain TransformationsConsider: T(n) = T(n/2) + 1 , n 2 T(1) = 1 (The recurrence for binary search)

Not in the form we want, which is T(n) = T(n-1) + ...

But let’s do the following:

n k n

S k T

T n T S k

T n T T S k

T T S

k

k

k

k k

2

2

2

2 2 2 2 1

1 2 0

1

0

(

Now:

( ) = ( ) = ( )

( ) = (

lg )

( ) ( )

) ( ) ( )

( ) ( ) ( )

27

Our recurrence

T(n) = T(n/2) + 1, n ≥ 2

T(1) = 1

has become

S(k) = S(k-1) + 1, k ≥ 1

S(0) = 1

which telescopes, giving S(k) = k+ 1.

Back-substituting:

T(n) = S(k) = k+1 = lgn+ 1

(since we defined k = lgn.)

28

Another recurrence:

T(n) = T(n/2) + n, T(1) = 1

12)()(

)222 (notingget we,2 Since

.12 sit' - #23 slideon thissolved ve' We

2)1()(

)2()( ,2 Letting

1

1

nnTkS

n

kSkS

TkSn

kkk

k

k

kk

29

Range TransformationsT(n) = 2T(n-1) + 1, T(1) = 1.

The “2” multiplying T(n-1) is the problem.

So let’s multiply through by

(Trust me.) We call this a summing factor.

2 n.

2 2 2 1 2

2 1 2

2 1 2

1

1

n n n

n n

n n

T n T n

T n

T n

( ) ( )

( )

( )( )

It’s in telescoping form!

(recurrence for “Towers of Hanoi “ puzzle.)

30

Let

Then

becomes

and becomes

S n T n

T n T n

S n S n

T S

n

n n n

n

( ) ( ).

( ) ( )

( ) ( )

( ) ( ) .

( )

2

2 2 1 2

1 2

1 1 1 1 2

1

Telescoping, we get

=

S n n n

i

ni

( )

( )

12

12

12

12

1

1

31

( ) ( )

)

)

( )

12

1

12

0

11

12 1

1

1 1

2 1 1 12

1

i

ni

i

ni

n

nn

n

= =(

-

= -( 12 =

121

2

12

12

Since then

=

=

S n T n

T n S n

n

n

n

n

n

( ) ( ),

( ) ( )

( )

.

2

2

2 1

2 1

12

32

Now we are ready to tackle MERGESORT’s recurrence.

T(n) = 2T(n/2) + n, n > 1

T(1) = 1

First, a domain transformation:

1)0(

1,1)1()(

gives

)(2)(

: transrange a Now

1)0(

1,2)1(2)(

yields )2()( ,2

R

kkRkR

kSkR

S

kkSkS

TkSn

k

k

kk

world’s simplest recurrence!

33

We can solve that: R(k) = k + 1Back-substituting:

S k R k

S k k

k n

S k T n n

n n

T n n n

k

k

n

( ) ( ),

( ) ( )

lg ,

( ) ( ) (lg )

(lg )

( ) ( lg ).

lg

2

2 1

2 1

1

so

so

So

34

The Telescope/Transformation Method: A Summary

1. Do domain transformations to deal with the argument to T.

E.g. T(n/2) S(k-1).

2. Do range transformations to handle multipliers of T.

E.g. 2T(n-1) 2 T(n-1).

3. Telescope.

4. Check.

-(n-1)