1 advanced induction supplementary notes prepared by raymond wong presented by raymond wong

1

Advanced Induction

Supplementary Notes

Prepared by Raymond WongPresented by Raymond Wong

2

e.g.1 (Page 5)

Let n be a non-negative power of 2. Suppose that

T(n) 2T(n/2) + cn for some constant c. T(n) > 0

Prove that T(n) = O(n log n)

Let n be a non-negative power of 2. T(n) 2T(n/2) + cn T(n) > 0Prove that T(n) = O(n log n)

3

e.g.1


We want to prove that T(n) = O(n log n)

By the definition of big O, we need to show the following.

n0 Z and k R such that n > n0, T(n) k.n log n

nn0

T(n)

k.n log nHow can we find the value for n0?

How can we find the value for k?The above equation can be proved by our

traditional induction if we know the valuefor n0 and k.

How can we prove it?

4

e.g.1





Step 1: Prove that P( ? ) (i.e., the base case) is true.

Suppose that n > n0.Let P(n) be “T(n) k.n log n……………..(*)”

We don’t know the base case. Thus, we need to test the “smallest” value of n for the base case.

Consider P(1)

RHS of (*) = k.1 log 1

We want to see whether P(1) is true. (i.e., “T(1) k.1 log 1……………..(*)”)

= k.1.0

= 0

In other words, according to (*), we need to prove T(1) 0

Since T(n) > 0, we have T(1) > 0

We conclude that P(1) is false

P(1) is false.

Thus, it is not possible to prove that T(1) 0

Since we don’t want “n = 1” to happen,

I want that n is larger (e.g., n = 2, 22, …)

n0 1

Since n > n0, we have “n0 1”

5

e.g.1








Consider P(2)

RHS of (*) = k.2 log 2

We want to see whether P(2) is true. (i.e., “T(2) k.2 log 2……………..(*)”)

= k.2.1

= 2k

In other words, according to (*), we need to prove T(2) 2k

Currently, we don’t know the value of k.

P(1) is false.

P(2) is true.

In order to prove T(2) 2k, we should set a value of k such that k T(2)/2

k T(2)/2

Thus, P(2) is true.

n0 1

6

e.g.1







P(1) is false.

P(2) is true.k T(2)/2

We guess that P(2), P(22), P(23)…are also true.

Thus, the base case is when n = 2.

P(2)

n0 1

7

e.g.1






P(1) is false.


Step 2: Prove that “P(n/2) P(n)” is true for all n > 2.

Step 2(a): Assume that P(n/2) is true for n > 2.

Step 2(b): According to P(n/2), we deduce that P(n) is true.

That is, T(n/2) k.(n/2) log (n/2)

Consider T(n)

We want to show that

T(n) k.n log n

n0 1

2T(n/2) + cn

2.(k.(n/2) log (n/2)) + cn

= k.n log (n/2) + cn

= k.n log n – k.n log 2 + cn

= k.n log n – kn + cn


In order to prove T(n) k.n log n, we should set a value of k such that – kn + cn 0

(i.e., k c)

Thus, P(n) is true.

P(n) is true.k c

8

e.g.1






P(1) is false.





That is, T(n/2) k.(n/2) log (n/2) We want to show that

T(n) k.n log n

n0 1

P(n) is true.k c

We prove that “P(n/2) P(n)” is true for all n > 2 By Mathematical Induction, n 2, T(n) k.n log n

We should set some values for n0 and k.

Since n0 1, we set n0 = 1.

Since k T(2)/2 and k c, we set k = max{c, T(2)/2}

9

e.g.2 (Page 11)

Consider a traditional induction proof. Let n be a non-negative power of 2. Suppose that

T(n) 2T(n/2) + cn for some constant c. T(n) > 0

Prove that T(n) = O(n log n)

Let n be a non-negative power of 2. T(n) 2T(n/2) + cn T(n) > 0Prove that n > 1, T(n) k.n log n where k = max{c, T(2)/2}

n > 1, T(n) k.n log n where k = max{c, T(2)/2}

10

e.g.2


Step 1: Prove that P(2) (i.e., the base case) is true.

We want to show that “T(2) k.2 log 2 where k = max{c, T(2)/2} ”

Consider T(2)= T(2)/2 . 2(i.e., “T(2) k.2 where k = max{c, T(2)/2} ”)

k . 2

Thus, P(2) is true.

(Since k = max{c, T(2)/2}, k T(2)/2)

11

e.g.2





That is, T(n/2) k.(n/2) log (n/2)

Consider T(n)


T(n) k.n log n

2T(n/2) + cn

2.(k.(n/2) log (n/2)) + cn

= k.n log (n/2) + cn

= k.n log n – k.n log 2 + cn

= k.n log n – kn + cn

k.n log n (Since k = max{c, T(2)/2}, k c and thus –kn + cn 0)Thus, P(n) is true.

We prove that “P(n/2) P(n)” is true for all n > 2 By Mathematical Induction, n 2, T(n) k.n log n

12

e.g.3 () Traditional Induction Proof

The values of n0 and k are given

Advanced Induction Proof The values of n0 and k are to be found

when we are proving.

Advantage: We can use the original induction for the proof.Disadvantage: The values of n0 and k look “strange”. We don’t know the reason why we have these “strange” values.

Advantage: The values of n0 and k are found during the proof. Thus, we know why the values of n0 and k are generated. Disadvantage: It looks more complicated during the proof.

13

e.g.4 ()


T(n) T(n/3) + cn for some constant c. T(n) > 0

Prove that T(n) = O(n)

Let n be a non-negative power of 3. T(n) T(n/3) + cn T(n) > 0Prove that T(n) = O(n)

14

e.g.4We want to prove that T(n) = O(n)


n0 Z and k R such that n > n0, T(n) k.n

nn0

T(n)

k.nHow can we find the value for n0?





15





Suppose that n > n0.Let P(n) be “T(n) k.n ……………..(*)”


Consider P(1)

RHS of (*) = k.1

We want to see whether P(1) is true. (i.e., “T(1) k.1 ……………..(*)”)

In other words, according to (*), we need to prove T(1) k


P(1) is true.

In order to prove T(1) k, we should set a value of k such that k T(1)

k T(1)

Thus, P(1) is true.

I guess n can be 1, 3, 32, …


n0 0


= k

16








P(1)

P(1) is true.k T(1)

n0 0


17








That is, T(n/3) k.(n/3)

Consider T(n)


T(n) k.n

T(n/3) + cn

k.(n/3) + cn

= k.n + (c – 2k/3)n


In order to prove T(n) k.n, we should set a value of k such that c – 2k/3 0

(i.e., k 3c/2)

Thus, P(n) is true.

P(n) is true.k 3c/2

P(1) is true.k T(1)

n0 0


= k.n – 2kn/3 + cn

18








That is, T(n/3) k.(n/3) We want to show that

T(n) k.n

We prove that “P(n/3) P(n)” is true for all n > 1 By Mathematical Induction, n 1, T(n) k.n

We should set some values for n0 and k.


Since k T(1) and k 3c/2, we set k = max{3c/2, T(1)}

P(n) is true.k 3c/2

P(1) is true.k T(1)

n0 0


19

e.g.5 (Page 15)

Consider a traditional induction proof. Let n be a non-negative power of 3. Suppose that

T(n) T(n/3) + cn for some constant c. T(n) > 0

Prove that T(n) = O(n)

Let n be a non-negative power of 3. T(n) T(n/3) + cn T(n) > 0Prove that n > 0, T(n) k.n where k = max{3c/2, T(1)}

n > 0, T(n) k.n where k = max{3c/2, T(1)}

20

e.g.5Step 1: Prove that P(1) (i.e., the base case) is true.

We want to show that “T(1) k.1 where k = max{3c/2, T(1)} ”

Consider T(1)= T(1).1

k . 1

Thus, P(1) is true.


(since k = max{3c/2, T(1)}, k T(1))

21

e.g.5Step 2: Prove that “P(n/3) P(n)” is true for all n > 1.



That is, T(n/3) k.(n/3)

Consider T(n)


T(n) k.n

T(n/3) + cn

k.(n/3) + cn

= kn/3 + cn

= kn – 2kn/3 + cn

= kn + (c – 2k/3)n

kn (Since k = max{3c/2, T(1)}, k 3c/2 and thus c – 2k/3 0)Thus, P(n) is true.

We prove that “P(n/3) P(n)” is true for all n > 1 By Mathematical Induction, n 1, T(n) k.n


22

e.g.6 (Page 16)


T(n) 4T(n/2) + cn for some positive constant c.

T(n) > 0 Prove that T(n) = O(n2)

Let n be a non-negative power of 2. T(n) 4T(n/2) + cn where c:+ve T(n) > 0Prove that T(n) = O(n2)

23

e.g.6

We first show an “inappropriate” proof After that, we show an “appropriate”

proof


24

e.g.6We want to prove that T(n) = O(n2)


n0 Z and k R such that n > n0, T(n) k.n2

nn0

T(n)

k.n2How can we find the value for n0?





25





Suppose that n > n0.Let P(n) be “T(n) k.n2 ……………..(*)”


Consider P(1)

RHS of (*) = k.12

We want to see whether P(1) is true. (i.e., “T(1) k.12 ……………..(*)”)

In other words, according to (*), we need to prove T(1) k


P(1) is true.

In order to prove T(1) k, we should set a value of k such that k T(1)

k T(1)

Thus, P(1) is true.

I guess n can be 1, 2, 22…


n0 0


= k

26








P(1)

P(1) is true.k T(1)

n0 0


27








That is, T(n/2) k.(n/2)2

Consider T(n)


T(n) k.n2

4T(n/2) + cn

4.k.(n/2)2 + cn

= 4.k.n2/4 + cn

In order to prove T(n) k.n2, we should have the following cn 0

The inequality “cn 0” is impossible.

P(1) is true.k T(1)

n0 0


= k.n2 + cnNote that c is positive and n is positive

28





The inequality “cn 0” is impossible.

P(1) is true.k T(1)

n0 0


Note that c is positive and n is positive

What went wrong?

Statement P(n) is too weak to be proved by induction.

To fix this, let’s see if we can prove something that is actually strongerthat we were originally trying to prove – namely T(n) k1

.n2 – k2n for some positive constants k1 and k2

29

e.g.6

Now, we show an “appropriate” proof.

30



n0 Z and k1, k2 R such that n > n0, T(n) k1n2 – k2 n

nn0

T(n)

k1n2 – k2nHow can we find the value for n0?How can we find the value for k1 and

k2?The above equation can be proved by ourtraditional induction if we know the valuefor n0, k1 and k2.



31



n0 Z and k1, k2 R such that n > n0, T(n) k1n2 – k2n


Suppose that n > n0.Let P(n) be “T(n) k1n2 – k2n ……………..(*)”


Consider P(1)

RHS of (*) = k1.12 – k2

.1

We want to see whether P(1) is true. (i.e., “T(1) k1

.12 – k2.1 ……………..(*)”)

In other words, according to (*), we need to prove T(1) k1 – k2

Currently, we don’t know the values of k1 and k2.

P(1) is true.

In order to prove T(1) k1 – k2, we should set values of k1 and k2 such that k1 – k2 T(1)

k1 – k2 T(1)

Thus, P(1) is true.

I guess n can be 1, 2, 22,…


n0 0


= k1 – k2

32








P(1)

P(1) is true.Let n be a non-negative power of 2. T(n) 4T(n/2) + cn where c:+ve T(n) > 0Prove that T(n) = O(n2)

k1 – k2 T(1)

n0 0

33








That is, T(n/2) k1.(n/2)2–k2(n/2)

Consider T(n)


T(n) k1n2 – k2n

4T(n/2) + cn

4.(k1(n/2)2–k2(n/2)) + cn

= 4.(k1.n2/4–k2n/2) + cn


In order to prove T(n) k1n2 – k2n, we should set a value of k such that c – k2 0

P(1) is true.Let n be a non-negative power of 2. T(n) 4T(n/2) + cn where c:+ve T(n) > 0Prove that T(n) = O(n2)

= k1n2 – 2k2n + cn

= k1n2 – k2n – k2n+ cn

(i.e., k2 c)

k2 c P(n) is true.

k1 – k2 T(1)

n0 0

= k1n2 – k2n + (c– k2)nThus, P(n) is true.

34








That is, T(n/2) k1.(n/2)2–k2(n/2) We want to show

that T(n) k1n2 – k2n

P(1) is true.Let n be a non-negative power of 2. T(n) 4T(n/2) + cn where c:+ve T(n) > 0Prove that T(n) = O(n2) k2 c P(n) is true.

k1 – k2 T(1)

n0 0

We prove that “P(n/2) P(n)” is true for all n > 1 By Mathematical Induction, n 1, T(n) k1n2 – k2n

We should set some values for n0, k1, k2.


Since k2 c, we set k2 = c.

Since k1 – k2 T(1) (i.e., k1 T(1) + k2) we set k1 = T(1) + c.

35

e.g.7 (Page 19) Why was it easier to prove

stronger statement T(n) k1n2 – k2n

than to prove weaker statement T(n) kn2?

36

e.g.7 Why was it easier to prove

stronger statement T(n) k1n2 – k2n

than to prove weaker statement T(n) kn2?

Proving something about p(n) uses p(1) … p(n-1)

The stronger that p(1) … p(n-1) are, the greater help they provide in proving p(n).

Our first proof was that p(n-1) were too weak, and thus we were not able to use them to prove p(n).

When we give an induction proof with a stronger statement, weare using a stronger inductive hypothesis.

1 advanced induction supplementary notes prepared by raymond wong presented by raymond wong

Documents