• Professor Jeanne Ferrante

1

http://www.jflap.org/jflaptmp/

CSE 105 Theory of

Computation

Today’s Agenda • More Undecidability • Time Complexity

– Big-O Asymptotic Analysis – Time Complexity Classes – The Complexity Class P

Reminders and announcements: • HW 7 due Friday May 27 by 11:59 pm • Reading Quiz 9 due Mon May 30 by 11:59 pm • Final Exam Study Guide is out!! Note: Computable Functions (HW 7) • REVIEW SESSION (Nathan Speidel):

– Thurs Jun 2 7 pm – 8:50 pm in Peterson 108 • Final Exam: If you need a LEFT handed seat, make a private post to

instructors on Piazza with your section number and PID ASAP • Final: Sat. Jun 4, 11:30 am – 2:29 pm Room WLH 2001 • CAPE AND TA Evals: OPEN!

2

Review: Reduction • For 2 problems P1 and P2, P1 reduces to P2 if

any solution for P2 can be used to solve P1

3

P1 P2 Yes No

Yes No (May be)

Harder

(May be) Easier

In other words: Using a solution for P2 as a subroutine gives a solution for P1.

ETM is undecidable Proof by contradiction: Assume that ETM is decidable, with TM R. We show that ATM reduces to ETM. But we know that ATM is undecidable, so ETM must be undecidable. • Construct TM MATM and show that it decides ATM: MATM = “ // Use R as a subroutine to get decider for ATM Correctness: MATM is a decider since R is, and accepts <M,w> iff L(X) is nonempty iff M accepts w.

What does R(<M>) do on TM M when L(M) = φ ? A. Accept D. Don’t Know B. Reject C. Fails to halt

ETM is undecidable Proof by contradiction: Assume that ETM is decidable, with TM R. We show that ATM reduces to ETM. But we know that ATM is undecidable, so ETM must be undecidable. • Construct TM MATM and show that it decides ATM: MATM = “ // Use R as a subroutine to get decider for ATM Correctness: MATM is a decider since R is, and accepts <M,w> iff L(X) is nonempty iff M accepts w.

What does R(<M>) do on TM M when L(M) ≠ φ ? A. Accept D. Don’t Know B. Reject C. Fails to halt

ETM is undecidable

Proof: Assume that ETM is decidable, with TM R. We show that ATM reduces to ETM. But we know that ATM is undecidable, so ETM must be undecidable. • Using R, we construct a TM MATM and show that it decides ATM: MATM =

Correctness: MATM is a decider since R is, and accepts <M,w> iff L(X) is nonempty iff M accepts w.

“ On input <M,w>: 1. Construct TM X, X = “On input y

a) If y ≠ w, reject b) If y = w, runs M on w. If M accepts, accept “

// Note that L(X) is nonempty iff M accepts w. 2. Run R on <X>. If R accepts, accept. If R rejects, reject. “

What is L(X) ? A. ATM B. {w} C. w or φ D. {w} or φ E. None of the

above

Only one of these can be recognizable: ETM or ETM . WHY?

Qu: Which one is recognizable? A. ETM

B. ETM

C. Neither of them

We define TM C = “On input <M>: For positive integers i = 1,2, 3.. 1. Run M for i steps on the first i input strings (in

lexicographical order). 2. If M every accepts a string, then ACCEPT. “

Review Theorem 4.22: A language L is decidable iff L is both TR and co-TR

• L Decidable -> TR and co-TR – Easy proof! Just use the decider to recognize L,

so it is TR. Then use the decider again to build a recognizer for L, by flipping the accept/reject result.

• L TR and co-TR -> Decidable – We can use the same run-in-parallel method we

did for ATM and ATM to build a decider for L: • Run recognizer TM’s for L and L in parallel • If the TM for L accepts, accept. If the TM for L

accepts, reject. • One of these must accept, so we can conclude this

is a decider, and accepts ATM . 8

Status: Language Hierarchy

Regular

Context-Free

Decidable

Turing-Recognizable

CF but not Regular: {0 n1n| n > 0} By pumping lemma (Regular) Decidable but not CF: { anbncn | n > 0} By pumping lemma (CF) Turing-Recognizable but Not Decidable: ATM By Diagonalization ETM By Reduction Not Turing Recognizable: ATM TH 4.22 ETM

ATM ATM

ETM ETM

MEASURING TIME COMPLEXITY OF DECIDABLE PROBLEMS BIG-O

10

What is “Theory of Computation”?

11

What is a Computer? A Computation? • We’ve looked at a number of computational

models: • Finite Automata (DFA, NFA), PDA, Turing Machines

Computability: What problems are computers capable of solving?

• By Church-Turing thesis, Computations on TM’s • There are many problems that can be shown

unsolvable (undecidable) Complexity: How easy or hard is a given problem

to solve? Are some problems inherently harder than others?

• What resources are needed to solve a problem?

Decidability vs. Complexity • Decidability gives the answer (eventually)

but gives no idea about how long it takes to compute the answer

• Complexity measures the resources (time, space, power…) needed to get the answer

• We will concentrate on TIME(n) as a function of length of input string, n

Decidability What can conceptually be solved, given unlimited resources?

Complexity What can practically be solved, given limited resources?

12

Example: Travelling Salesperson Problem Given: Collection {C1,…Cn} of cities, Distance k, Distance Matrix D(i,j): distance between Ci and Cj Problem: Is there a tour of all cities with total distance < k? • Good news: Its solvable! How?

Complexity of enumerating all tours to check total distance < k:

With n cities, there are (n-1)! choices of tours E. g., n = 50, 49! ≈ 1045

Not practically computable to enumerate all tours! • But complexity of checking if a given tour has

distance less than k is easy!

13

Time Complexity Def. Let M be a (deterministic) TM that always halts. The running time or time complexity of M is the function f: N R+ where f(n) is the maximum number of steps of the TM M on any input of length n. • We say that “M runs in time f(n)”

This is worst-case analysis •A different approach: average-case

Because it may be difficult to figure out exact number of steps, we will estimate it

14

Big-O Notation • To estimate the asymptotic running time, ignore:

– Constant coefficients – Lower order terms

Def. Let f and g be functions from N to R+. f(n) = O(g(n)) if there are positive integers c and n0 such that, for every n ≥ n0,

f(n) ≤ c g(n). • Example: f(n) = 2 n3 + 17 n2 + 3n + 10 Claim f(n) = O(n3 ): Let c = 2 + 17 + 3 + 10= 32, and n0 = 2. Then f(n) = 2 n3 + 17 n2 + 3n + 10 ≤ 32 n3 for n ≥ 2

, ,

IGNORE IGNORE

15

Big-O Questions Recall f(n) = O(g(n)) if there are positive integers c and n0 such that for every n ≥ n0,

f(n) ≤ c g(n).

Is f(n) = 2 n3 + 17 n2 + 30n + 1000 = O(n2 )? A. No B. Yes C. Maybe

Is f(n) = 2 n3 + 17 n2 + 30n + 1000 = O(n4 )? A. No B. Yes C. Maybe

16

More on Big-O • When f(n) = O(g(n)) we say g(n) is an

(asymptotic) upper bound for f(n). • If f(n) = logb (n)

– changing the value of base b corresponds to multiplication by a constant

– We can ignore the base b and write f(n) = log(n).

• We can add up Big-O terms: – f(n) = O(n2 ) + O(n) + O(log(n)) = O(n2 ) WHY?

• O(nc ) for constant c is a polynomial bound • O(2nc ) for c > 0 is an exponential bound

17

Example: TM M accepts {0k1k | k ≥ 0} M = “On input w:

1. Scan across the tape and reject if w not of form 0*1*

2. Repeat the following while both 0’s and 1’s: a) Scan across tape, crossing off single 0 and single 1

3. If 0’s remain after all 1’s checked off, reject. If 1’s remain after all 0’s checked off, reject. Otherwise, accept.”

18

Example: M accepts {0k1k | k ≥ 0} M = “On input w:

1. Scan across the tape and reject if w not of form 0*1*

2. Repeat the following while both 0’s and 1’s:

a) Scan across tape, crossing off single 0 and single 1

3. If 0’s remain after all 1’s checked off, reject. If 1’s remain after all 0’s checked off, reject. Otherwise, accept.”

Total is O(n) + O(n2 ) + O(n) = O(n2 )

2n steps

O(n) steps

O(n) steps

n/2 scans, each with

19

TIME COMPLEXITY CLASSES OF DECIDABLE PROBLEMS

Have we got the time?

20

Time Complexity Classes Def. Let t: N R+ be a function.

The time complexity class TIME(t(n)) is the collection of languages L that are decidable by a O(t(n)) TM. {0k1k | k ≥ 0} is in TIME(n2 ) because there is a TM M that decides it in O(n2 ) Can we do better? Would have to find a TM algorithm that is O(n) or O(n log n)…

21

A Different TM M’ decides {0k1k | k ≥ 0} M’ = “On input w:

1. Scan across the tape and reject if w not of form 0*1* O(n) steps

2. Repeat the following while both 0’s and 1’s: a) Scan across tape, checking whether total number

of 0’s and 1’s remaining is even or odd. If odd, reject //Must be even if same number of 0’s and 1’s O(n) steps

b) Scan across tape, crossing off every other 0 starting with first 0, and then crossing off every other 1, starting with first 1 O(n) steps

3. If no 0’s and no 1’s remain on the tape, accept. Otherwise, reject.” O(n) steps

How often is the loop at 2 repeated? A. O(n) B. O(log(n)) C. Neither of these

22

TM M’’ with 2 tapes decides {0k1k | k ≥ 0} M’’ = “On input w:

1. Scan across the tape 1 and reject if w not of form 0*1*

2. Scan across the 0’s on tape 1 until first 1. At same time, copy the 0’s on tape 2.

3. Scan across the 1’s on tape 1 until end of input. For each 1 read on tape 1, cross of 0 on tape 2. If all 0’s crossed off before all 1’s read, reject.

4. If all 0’s now crossed off, accept. If any 0’s remain, reject.”

What is the running time of M’’? A. O(n) B. O(n log(n)) C. O(n2 )

23

Decidability vs. Time Complexity Decidability • M always halts but

don’t know how long to halt

• Does not depend on variant of TM model: – Number of tapes – Nondeterminism

Time Complexity T(n) • M will halt within time

T(n) on all inputs of length n

• Depends on variant of TM model: – t(n) multitape TM

equivalent to O(t 2(n) ) single tape – t(n) nondeterministic

single tape equivalent to

2O(t(n)) deterministic single tape

24

Running times of decider TM’s

Deterministic q0

Nondeterministic q0

f(n)= MAX number of steps with input length n

f(n) = MAX number of steps on any branch with input length n

qrej

qacc

qrej

qrej qacc

25

CLASS P (DETERMINISTIC POLYNOMIAL TIME)

Most useful!

26

P = U DTIME(nk ) • P is the class of languages that can be

decided in polynomial time on a deterministic, single-tape TM

• Why do we study this particular class? – There is a dramatic difference in growth rate

between polynomial and exponential functions • n = 1000, n3 = 1 billion, 2n > no. of atoms in universe

– It contains the “feasibly” solvable problems • “Cobham’s Thesis”

– P the same for all models that are polynomially equivalent to deterministic, 1-tape TM’s

• t(n) multitape equivalent to O(t 2(n) ) single tape TM

27

k

Encodings Matter in Class P • We’ll assume polynomial time encodings

– Ex. Multiplication of two numbers • Matters how we encode the input and answer:

– Unary encoding of n exponentially larger than binary encoding of n

• Graph encoding choices – Encode nodes and edges, n nodes, ≤ n2

edges – Use n by n adjacency matrix to determine if

edge from node i to node j – Analyze graph problems in number n of

nodes; still polynomial in size of graph

28

Analyzing Problems in Class P • We’ll use high-level algorithm descriptions • We’ll describe the algorithm in stages

– We’ll need to show that there are only a polynomial number of stages for input of size n

– We’ll need to show that each stage has only polynomial number of steps for input of size n

• We can’t use nondeterminism or guessing • The algorithm needs to be implementable

on a deterministic TM in polynomial time

29

Example: PATH is in P PATH = {<G,s,t> | G is a directed graph with n nodes, and there is a directed path from node s to node t} • Enumerating all paths in G is exponential in n • Better?

Constant time

Constant time

O(n2 )

O(n)

M = “On input <G,s,t>: 1. Mark node s 2. Repeat until there are no unmarked nodes:

a) Scan all edges of G, and if edge (a,b) is found with a marked and b unmarked, mark b.

3. If t is marked, accept; otherwise, reject. “

30

More Examples in Class P • RELPRIME = { <x,y> | x and y are relatively prime}

– Enumerating through all possible common divisors would yield exponential time!

– Uses Euclidean Algorithm to see if GCD(x,y) = 1 • GCD(x,y) is the largest integer that divides both x and y • Ex. GCD (10, 6) = 2

• Every CFL in P: {w | w is generated by CFG G} in P – Every CFL is decidable, but

• Enumerating all derivations of w where |w|=n, is exponential at best

– Use Dynamic Programming to show in P • To be in P, must usually find way to avoid brute-

force searches!

Qu: Is sorting n numbers in P?

A. Yes B. No C. Depends on the method used