CMU 15-251

Time complexity

Teachers:

Anil Ada

Ariel Procaccia (this time)

Course overview

2

The big O

3

Adding two 𝑛-bit numbers

4

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*+

Adding two 𝑛-bit numbers

5

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

+

Adding two 𝑛-bit numbers

6

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

+

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

Adding two 𝑛-bit numbers

7

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

+

Time complexity

• 𝑇 𝑛 =𝑛

•

•

8

A great idea

•

• 𝑀 𝑐𝑀

• 𝑛𝑀 𝑐 ⋅ 𝑛

• 𝑀′ 𝑐′

• 𝑀′ 𝑐′𝑛

9

A great idea

•

•

10

𝑛

A great idea

•

11

𝑛

Multiplying two 𝑛-bit numbers

12

×

𝑛2

* * * * * * * ** * * * * * * *

* * * * * * * ** * * * * * * *

* * * * * * * ** * * * * * * *

* * * * * * * ** * * * * * * *

* * * * * * * ** * * * * * * *

* * * * * * * ** * * * * * * *

Linear vs. quadratic

• 𝑐n2

•

•

13

1

10𝑛2

10𝑛

0 2000

4000

100

1000

2000

3000

Nursery school addition

• 𝑛 𝑎 𝑏 𝑎𝑏

• 𝑇 𝑛

• 𝑏 = 00⋯0

• 𝑏 = 11⋯1

1. 𝑐 log 𝑛 2

2. 𝑐𝑛 log 𝑛

3. 𝑐𝑛2

4. 𝑐𝑛2𝑛

14

Worst case time

15

𝑇(𝑛) 𝐴 =

𝑥𝑛

𝐴 𝑥

Kindergarten multiplication

• 𝑛 𝑎 𝑏𝑎 𝑎 𝑏 − 1

• 𝑇 𝑛 = 𝑐𝑛2𝑛

•

•

16

More formally: big 𝑂• 𝑓: ℕ → ℕ𝑓(𝑛) = 𝑂 𝑛

𝑐

𝑛 𝑓 𝑛 ≤ 𝑐𝑛

•

𝑓

17

𝑛

More formally: Ω

• 𝑓: ℕ → ℕ𝑓(𝑛) = Ω 𝑛

𝑐

𝑛 𝑓 𝑛 ≥ 𝑐𝑛

•

𝑓

18

𝑛

More formally: Θ

• 𝑓: ℕ →ℕ 𝑓 𝑛 = Θ 𝑛𝑓 𝑛 = 𝑂(𝑛)𝑓 𝑛 = Ω(𝑛)

• 𝑓

19

𝑛

More formally and generally

• 𝑓 𝑛 = 𝑂 𝑔 𝑛 𝑐

𝑛𝑓 𝑛 ≤ 𝑐 ⋅ 𝑔(𝑛)

• 𝑓 𝑛 = Ω 𝑔 𝑛 𝑐

𝑛𝑓 𝑛 ≥ 𝑐 ⋅ 𝑔(𝑛)

• 𝑓 𝑛 = Θ(𝑛) 𝑓 𝑛 = 𝑂 𝑔 𝑛 𝑓 𝑛 =

Ω 𝑔 𝑛

20

Exercises

• 𝑛4 + 3𝑛 + 22 = O(𝑛4)

• 𝑛4 + 3𝑛 + 22 = Ω(𝑛4 log 𝑛)

•

1. ln 𝑛 = 𝑂 log 𝑛

2. ln 𝑛 = Ω log 𝑛

21

Exercises

• log 𝑛! = ?

1. Θ 𝑛

2. Θ(𝑛 log 𝑛)

3. Θ 𝑛2

4. Θ 2n

•

1. 𝑓 = 𝑂(𝑔) 𝑔 = 𝑂 ℎ ⇒ 𝑓 = 𝑂(ℎ)

2. 𝑓 = 𝑂(ℎ) 𝑔 = 𝑂 ℎ ⇒ 𝑓 = 𝑂(𝑔)

22

Names for growth rates

• 𝑇 𝑛 = 𝑂 𝑛

• 𝑇 𝑛 = 𝑂 𝑛2

• 𝑘 ∈ ℕ

𝑇 𝑛 = 𝑂 𝑛𝑘

o 13𝑛28 + 11𝑛17 + 2

23

Names of growth rates

• 𝑘 ∈ ℕ𝑇 𝑛 = 𝑂 𝑘𝑛

o 𝑇 𝑛 = 𝑛2𝑛 = 𝑂(3𝑛)

• 𝑇 𝑛 = 𝑂(log 𝑛)

o

o

24

Limits of the possible

25

104 108

108

1016

𝑛

𝑛2

𝑛32𝑛

Two similar problems

•

o

o

o

•

o 𝑛!

o 2𝑛

o 1.657𝑛

26

Two similar problems

•

o

o

o

•

𝑂 𝑛2

•

27

Polynomial time

28

29

Dragon Age: Inquisition

Representation

•

•

o 𝑛 1,… , 𝑛 𝑣1, … , 𝑣𝑛𝑤1, … , 𝑤𝑛 𝐵 𝑉

o

o 𝑆 𝑖∈𝑆𝑤𝑖 ≤ 𝐵 𝑖∈𝑆 𝑣𝑖 ≥ 𝑉

30

Representation*

•

o 𝑛 × 𝐵 𝐴

o 𝐴(𝑖, 𝑗) = max{𝐴 𝑖 − 1, 𝑗 , 𝐴 𝑖 − 1, 𝑗 − 𝑤𝑖 + 𝑣𝑖}

31

1 4 52 3 23 5 2

𝐵 = 5

1 2 3 4 5123

0 0 0 0 40 3 3 3 40 5 5 8 8

Representation

•

𝑂(𝑛𝐵)

•

o 2𝑛 ⋅ max{log 𝐵, log 𝑉}

o

•

o 2𝑛 ⋅ max{𝐵, 𝑉}

o

32

Cool growth rates: 2stack

• 2STACK 0 = 1

• 2STACK 𝑛 = 2

•

o 2STACK 1 = 2

o 2STACK 2 = 4

o 2STACK 3 = 16

o 2STACK 4 = 65536

o 2STACK 5 = yikes!

33

2222222

2

2STACK(𝑛 − 1)2

Cool growth rates: log∗

• log∗ 𝑛 =𝑛 ≤ 1

•

o 2STACK 1 = 2 log∗ 2 = 1

o 2STACK 2 = 4 log∗ 4 = 2

o 2STACK 3 = 16 log∗ 16 = 3

o 2STACK 4 = 65536 log∗ 65536 = 4

o 2STACK 5 = yikes! log∗ yikes! = 5

34

Cool growth rates: log∗

• log∗

• 𝑂(𝑛 log 𝑛 2log∗ 𝑛)

35

What we have learned

•

o

o

•

o

o

36