the acceptance problem is not decidable. reductions and

86
Computational Models - Lecture 8 The Acceptance Problem is not decidable. Diagonalization. Reductions and undecidability Computable functions Mapping reductions More undecidable languages Undecidability by Rice Theorem Sipser’s book, Chapter 5, Sections 5.1, 5.3 Slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 1

Upload: others

Post on 04-Apr-2022

5 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: The Acceptance Problem is not decidable. Reductions and

Computational Models - Lecture 8

The Acceptance Problem is not decidable.Diagonalization.

Reductions and undecidability

Computable functions

Mapping reductions

More undecidable languages

Undecidability by Rice Theorem

Sipser’s book, Chapter 5, Sections 5.1, 5.3

Slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 1

Page 2: The Acceptance Problem is not decidable. Reductions and

More Undecidable Problems

We have already

Established Turing Machines as the gold standard ofcomputers and computability . . .

seen examples of solvable problems . . .

and saw one problem, ATM, that is computationallyunsolvable.

Today, we look at other computationally unsolvableproblems. We introduce the techniques of mappingreductions, of Rice Theorem, and of reductions bycomputational histories for proving that languages areundecidable/non-enumerable.

Slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 2

Page 3: The Acceptance Problem is not decidable. Reductions and

Acceptance, Again

We are now able to prove the undecidability of

ATM = {〈M,w〉|M is a TM that accepts w} .

Proof: By contradiction. Suppose a TM, H, is a decider forATM.

On input 〈M,w〉, where M is a TM and w is a string, H haltsand accepts if and only if M accepts w. Furthermore, Hhalts and rejects if M fails to accept w.

Slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 3

Page 4: The Acceptance Problem is not decidable. Reductions and

Acceptance (2)

On input 〈M,w〉, where M is a TM and w is a string, H haltsand accepts if and only if M accepts w. Furthermore, Hhalts and rejects if M fails to accept w.

H(〈M,w〉) =

{

accept if M accepts w

reject if M does not accept w

Slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 4

Page 5: The Acceptance Problem is not decidable. Reductions and

Acceptance (3)

Now we construct a new TM, D, with H as a subroutine.

D does the following

Calls H to determine what TM, M , does when the inputto M is its own description, 〈M〉.When D determines this, it does the opposite.

So D rejects if M accepts 〈M〉, and accepts if M doesnot accept 〈M〉.

Slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 5

Page 6: The Acceptance Problem is not decidable. Reductions and

Acceptance (4)

More precisely, D does the following:

Run H on input 〈M, 〈M〉〉.Output the opposite of what H outputs:

If H accepts, reject, andIf H rejects, accept.

Slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 6

Page 7: The Acceptance Problem is not decidable. Reductions and

Self Reference (4)

Don’t be confused by the notion of running a machine on itsown description!

Actually, you should get used to it.

Notion of self-reference comes up again and again indiverse areas.

Read “Gödel, Escher, Bach, an Eternal Golden Braid”,by Douglas Hofstadter.

This notion of self-reference is the basic idea behindGödel’s revolutionary result.

Compilers do this all the time . . . .

Slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 7

Page 8: The Acceptance Problem is not decidable. Reductions and

The Punch Line

So far we have,

D(〈M〉) =

{

reject if M accepts 〈M〉accept if M does not accept 〈M〉

What happens if we run D on its own description?

D(〈D〉) =

{

reject if D accepts 〈D〉accept if D does not accept 〈D〉

Oh, oh...Or, more accurately, a contradiction (to what?) ♣

Slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 8

Page 9: The Acceptance Problem is not decidable. Reductions and

Once Again

Assume that TM H decides ATM.

Then use H to build a TM, D, that when given 〈M〉,accepts exactly when M does not accept.

Run D on its own description.

D does:H accepts 〈M,w〉 when M accepts w.D rejects 〈M〉 exactly when M accepts 〈M〉.D rejects 〈D〉 exactly when D accepts 〈D〉.

Last step leads to contradiction.

Therefore neither TM D nor H can exist.

So ATM is undecidable!

Slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 9

Page 10: The Acceptance Problem is not decidable. Reductions and

A Non-enumerable Language

We already saw a non-decidable language: ATM.

Can we do better (i.e., worse)?

We now display a language that isn’t even recursivelyenumerable . . . .

Slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 10

Page 11: The Acceptance Problem is not decidable. Reductions and

A Non-enumerable Language

Earlier we saw

Theorem: If L and L are both enumerable, then L isdecidable.

Corollary: If L is not decidable, then either L or L is notenumerable.

Definition: A language is co-enumerable if it is thecomplement of an enumerable language.

Reformulating Theorem: A language is decidable if andonly if it is both enumerable and co-enumerable.

Slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 11

Page 12: The Acceptance Problem is not decidable. Reductions and

ATM is not Enumerable

Theorem: If L and L are both enumerable, then L isdecidable.

We proved that ATM is undecidable.

On the other hand, we saw that the universal TM, U ,accepts ATM.

Therefore ATM is enumerable.

If ATM were also enumerable, then by theorem ATMwas decidable.

Therefore ATM is not enumerable. ♣

Slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 12

Page 13: The Acceptance Problem is not decidable. Reductions and

Languages

enumerableco-enumerable decidable

A TMA DFAA TM

???

Question: Are there any languages in the area marked ????

Answer: Yes, heaps (why?)Slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 13

Page 14: The Acceptance Problem is not decidable. Reductions and

The Acceptance Problem (again)

We saw

ATM = {〈M,w〉|M is a TM that accepts w}

Our proof that ATM is undecidable was actually adiagonalization proof.

To see this, we start with a short “diagonalization reminder”.

Slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 14

Page 15: The Acceptance Problem is not decidable. Reductions and

Comparing Sizes of Sets

Suppose A and B are two sets, and we wish to comparetheir sizes.

If both A and B are finite, we can count how many elementseach of them has, and compare the numbers.

This method does not generalize to infinite sets.

Slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 15

Page 16: The Acceptance Problem is not decidable. Reductions and

Comparing Sizes of Sets (2)

Alternatively, we can pair the elements of A and B. If theypair perfectly, they have equal sizes.

Slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 16

Page 17: The Acceptance Problem is not decidable. Reductions and

Correspondence

Question: What does it mean to say that two infinite sets arethe same size?

Answered by Georg Cantor in 1873: Pair them off.

A map f : A → B is a correspondence if f satisfies

f one-to-one: if a1 6= a2 then f(a1) 6= f(a2).

f onto: for every b ∈ B, there is an a ∈ A such thatf(a) = b.

Question: What does it mean to say that sets A and B arethe same size?

Answer: A and B are the same size if there is acorrespondence from A to B.

Slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 17

Page 18: The Acceptance Problem is not decidable. Reductions and

Correspondence (2)

Question: In a crowded room, how can we tell if there aremore people than chairs, or more chairs than people?

Answer Establish a correspondence: ask everyone to sitdown.

Slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 18

Page 19: The Acceptance Problem is not decidable. Reductions and

Correspondence

Claim: The set N of natural numbers has thesame size as the set E of even numbers

Proof: Let f(i) = 2i.

Remark: The set E is a proper subset of the set N ,yet they are the same size!

Slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 19

Page 20: The Acceptance Problem is not decidable. Reductions and

Countable Sets

Definition A set A is countable if

either A is finite, or

A has the same size as N , the natural numbers.

We have just seen that E is countable.

A countable set is sometimes said to have size ℵ0.

Claim: The set Z of integers is countable.

Proof: Define f : N → Z by

f(i) =

{

i/2 if i is even−(⌊i/2⌋ + 1) if i is odd

Slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 20

Page 21: The Acceptance Problem is not decidable. Reductions and

Pop Quiz

In Heaven, there is a hotel with a countable number ofrooms.

One day, the society of Prophets, Oracles, and AIResearchers holds a 3-day convention that books everyroom in the hotel.

Then one more guest arrives, claiming he invented Lisp,and angrily demanding a room.

You are the manager. What do you do?

Answer: Ask the guest in room i to move to room i + 1, andput the newcomer in room 1.

Slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 21

Page 22: The Acceptance Problem is not decidable. Reductions and

Pop Quiz #2

Then a countable number of guests arrive, all angrilydemanding rooms. (What a noise!)

Now what do you do?

Answer: Ask the guest in room i to move to room 2i, and putthe newcomers in the odd-numbered rooms.

Slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 22

Page 23: The Acceptance Problem is not decidable. Reductions and

Rational Numbers

LetQ =

{m

n| m,n ∈ N

}

Theorem: Q is countable.

This claim may seem counterintuitive.

Idea

list Q as 2-dim array

begin counting with the first row . . .

Why doesn’t this work?

Slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 23

Page 24: The Acceptance Problem is not decidable. Reductions and

Rational Numbers (2)

1/2 1/3 1/4 1/51/1

2/1

3/1

4/1

5/1

2/2 2/3 2/4 2/5

1/2 3/3 3/4 3/5

4/2 4/3 4/4

5/35/2

Enumerate numbers along northeast and diagonals,skipping duplicates.

Does this mean that every infinite set is countable?

Slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 24

Page 25: The Acceptance Problem is not decidable. Reductions and

The Real Numbers

Every real number has a decimal representation.

For example, π = 3.1415926 . . .,√

2 = 1.4142136 . . ., and0 = 0.0000000 . . ..

Let R be the set of real numbers.

Theorem: R is uncountable.

R is sometimes said to have size ℵ1.

This is Cantor’s historic proof, which

introduced the diagonalization method.

Slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 25

Page 26: The Acceptance Problem is not decidable. Reductions and

The Real Numbers

Assume there is a correspondence between N and R.Write it down:

n f(n)

1 3.14159. . .2 55.55555. . .3 40.18642. . .4 15.20601. . .

We now show that there is a number x not in this list.

Slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 26

Page 27: The Acceptance Problem is not decidable. Reductions and

Diagonalization

Pick 0 ≤ x ≤ 1, so its significant digits follow decimal point.Will ensure x 6= f(n) for all n.

n f(n)

1 3.14159. . .2 55.55555. . .3 40.18643. . .4 15.20607. . .

Slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 27

Page 28: The Acceptance Problem is not decidable. Reductions and

Diagonalization n f(n)

1 3.14159. . .

2 55.55555. . .

3 40.18643. . .

4 15.20607. . .

First fractional digit of f(1) is 1, so pick first fractionaldigit of x to be something else (say, 2).

Second fractional digit of f(2) is 5, so pick secondfractional digit of x to be something else (say, 6).

and so on . . .

x = 0.2691 . . .

Slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 28

Page 29: The Acceptance Problem is not decidable. Reductions and

Diagonalization

A similar proof shows there are languages that are notenumerable.

the set of Turing machines is countable, butthe set of languages is uncountable!

Ergo,there exist languages that are not enumerable(why?)indeed, “most” languages are not enumerable(explain)

Slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 29

Page 30: The Acceptance Problem is not decidable. Reductions and

∃ Countably Many Turing Machines

Claim: The set of strings, Σ∗, is countable.

Proof: List strings of length 0, then length 1, then 2, and soon. This exhausts all of Σ∗.The union of countably many finite sets is countable.

Slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 30

Page 31: The Acceptance Problem is not decidable. Reductions and

∃ Countably Many Turing Machines (2)

Claim: The set of all Turing machines is countable.

Proof: Each TM M has an encoding as a string 〈M〉.Therefore there is a one-to-one mapping from the set of allTMs into (but not onto) Σ∗.

Since Σ∗ is countable, so is the set of all TMs.

Slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 31

Page 32: The Acceptance Problem is not decidable. Reductions and

The Set of All Languages is Uncountable

Let B be the set of of infinite binary sequences.

Claim B is uncountable.

Proof Diagonalization argument, essentially identical to theproof that R is uncountable.

(additional helpful clue: think of binary sequence as binaryexpansion!)

Slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 32

Page 33: The Acceptance Problem is not decidable. Reductions and

The Set of Languages is Uncountable (2)

Let L be the set of all languages over alphabet Σ.Recall B is the set of of infinite binary sequences.We give a correspondence

χ : L → B

called the language’s characteristic sequence.

Let Σ∗ = {s1, s2, s3, . . .} (in lexicographic order).

Each language L ∈ L is associated with a uniquesequence χ(L) ∈ B:

the i-th bit of χ(L) is 1 if and only if si ∈ L.

Slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 33

Page 34: The Acceptance Problem is not decidable. Reductions and

The Set of Languages is Uncountable (3)

Each language L ∈ L has a unique sequence χ(L) ∈ B:the i-th bit of χ(L) is 1 if and only if si ∈ L.

Example:

Σ∗ {ε, 0, 1, 00, 01, 10, 11, 000 . . .}A { 0, 00, 01, 000 . . .}χ(A) {0, 1, 0 1, 1, 0, 0, 1 . . .}

The map χ : L → Bis one-to-one and onto (why?),

and is hence a correspondence.

It follows that L is uncountable.

Slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 34

Page 35: The Acceptance Problem is not decidable. Reductions and

TMs vs. Languages

We saw that the set of all Turing machines is countable.

We saw that the set L of all languages over alphabet Σ isuncountable.

Therefore there are languages that are not accepted by anyTM.

This is an existential proof – it does not explicitly show anysuch language.

Slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 35

Page 36: The Acceptance Problem is not decidable. Reductions and

Reflections on Diagonalization

This proof that the acceptance problem is undecidable isactually diagonalization in transparent disguise.To unveil this, let’s start by making a table.

〈M1〉 〈M2〉 〈M3〉 〈M4〉 . . .M1 accept acceptM2 accept accept accept acceptM3

M4 accept accept...

Entry (i, j) is accept if Mi accepts 〈Mj〉, andblank if Mi rejects or loops on 〈Mj〉.

Slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 36

Page 37: The Acceptance Problem is not decidable. Reductions and

Diagonalization (2)

〈M1〉 〈M2〉 〈M3〉 〈M4〉 . . .

M1 accept acceptM2 accept accept accept acceptM3

M4 accept accept...

Run H on on corresponding inputs. In new table,entry (i, j) states whether H accepts 〈Mi, 〈Mj〉〉.

〈M1〉 〈M2〉 〈M3〉 〈M4〉 . . .M1 accept reject accept rejectM2 accept accept accept acceptM3 reject reject reject rejectM4 accept accept reject reject...

......

...

Slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 37

Page 38: The Acceptance Problem is not decidable. Reductions and

Diagonalization (3)

Now we add D to the table.

By assumption, H is a TM, and therefore so is D.D occurs on the list M1,M2, . . . of all TMs.D computes the opposite of the diagonal entries.At diagonal entry, D computes its own opposite!

〈M1〉 〈M2〉 〈M3〉 . . . 〈D〉M1 accept reject acceptM2 accept accept acceptM3 reject reject rejectM4 accept accept reject...

... . . .D reject reject accept ???...

... . . .♣

Slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 38

Page 39: The Acceptance Problem is not decidable. Reductions and

Halting vs Acceptance Problem

We have already established that ATM is undecidable.

Here is a closely related problem, which is alsoundecidable:

HTM = {〈M,w〉|M is a TM and M halts on input w}

We will now show the decidability of the Halting Problemusing a new technique: reducibility.

Slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 39

Page 40: The Acceptance Problem is not decidable. Reductions and

Reducibility

Always involves two problems, A and B.

Desired Property: If A reduces to B, then any solution of Bcan be used to find a solution of A.

Remark: This property says nothing about solving A byitself or B by itself.

Slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 40

Page 41: The Acceptance Problem is not decidable. Reductions and

Examples

Reductions:

Traveling from Boston to Paris . . .

reduces to buying plane ticket . . .

which reduces to earning the money for that ticket . . .

which reduces to finding a job(or getting the $s from mom and dad. . . )

Slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 41

Page 42: The Acceptance Problem is not decidable. Reductions and

Examples

Reductions:

Measuring area of rectangle . . .

reduces to measuring lengths of height and width.

Also:

Solving a system of linear equations . . .

reduces to inverting a matrix.

Slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 42

Page 43: The Acceptance Problem is not decidable. Reductions and

Reducibility

If A is reducible to B, then

A cannot be harder than B

if B is decidable, so is A.

if A is undecidable and reducible to B,then B is undecidable.

Slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 43

Page 44: The Acceptance Problem is not decidable. Reductions and

Additional Undecidable Problems

We have already established that ATM is undecidable.

Now - to the original halting problem:

HTM = {〈M,w〉|M is a TM and M halts on input w}

Clarification: How does HTM differ from ATM?

Slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 44

Page 45: The Acceptance Problem is not decidable. Reductions and

Undecidable Problems

HTM = {〈M,w〉|M is a TM and M halts on input w}

Theorem: HTM is undecidable.

Proof idea:

By contradiction.

Assume HTM is decidable.

Let R be a TM that decides HTM.

Use R to construct S, a TM that decides ATM.

So ATM is reduced to HTM.

Since ATM is undecidable, so is HTM.

Slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 45

Page 46: The Acceptance Problem is not decidable. Reductions and

Undecidable Problems

Theorem: HTM is undecidable.

Proof: Assume, by way of contradiction, that TM R decidesHTM. Define a new TM, S, as follows:

On input 〈M,w〉,run R on 〈M,w〉.If R rejects, reject.

If R accepts (meaning M halts on w), simulate M on wuntil it halts (namely run U on 〈M,w〉).If M accepted, accept; otherwise reject.

TM S decides ATM, a contradiction ♣Slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 46

Page 47: The Acceptance Problem is not decidable. Reductions and

Undecidable Problems

HTM = {〈M,w〉|M is a TM and M halts on input w}

Theorem: HTM is undecidable.

What we actually did was a reductionfrom ATM to HTM.

This will be formalized later.

Slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 47

Page 48: The Acceptance Problem is not decidable. Reductions and

Undecidable Problems (2)

Does a TM accept any string at all?

EMPTYTM = {〈M〉|M is a TM and L(M) = ∅}

Theorem: EMPTYTM is enumerable.

Proof - in the tutorial this week.Theorem: EMPTYTM is undecidable.

Proof structure: by contradiction

Assume EMPTYTM is decidable.

Let R be a TM that decides EMPTYTM.

Use R to construct S, a TM that decides ATM.Slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 48

Page 49: The Acceptance Problem is not decidable. Reductions and

Undecidable Problems (2)

EMPTYTM = {〈M〉|M is a TM and L(M) = ∅}

First attempt: When S receives input 〈M,w〉, it calls R withinput 〈M〉.

If R accepts, then reject, because M does not acceptany string, let alone w.

But what if R rejects?

Second attempt: Let’s modify M .

Slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 49

Page 50: The Acceptance Problem is not decidable. Reductions and

Undecidable Problems (2)

EMPTYTM = {〈M〉|M is a TM and L(M) = ∅}

Define Mw: on input x,

1. if x 6= w, reject.

2. if x = w, run M on w and accept if M does.

Mw either

accepts just w, or

accepts nothing.

Slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 50

Page 51: The Acceptance Problem is not decidable. Reductions and

Undecidable Problems (2)

Machine Mw: on input x,

1. if x 6= w, reject.

2. if x = w, run M on w and accept if M does.

Question:Can a TM construct Mw from M?

Answer:Easily, because we need only hardwire w, and add a fewextra states to perform the “x = w?′′ test.

Slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 51

Page 52: The Acceptance Problem is not decidable. Reductions and

Undecidable Problems (2)

EMPTYTM = {〈M〉|M is a TM and L(M) = ∅} .

Theorem: EMPTYTM is undecidable.

Define S as follows:

On input 〈M,w〉, where M is a TM and w a string,

Construct Mw from M and w.

Run R on input 〈Mw〉,if R accepts, reject; if R rejects, accept.

TM S decides ATM, a contradiction ♣

Slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 52

Page 53: The Acceptance Problem is not decidable. Reductions and

Undecidable Problems (3)

Does a TM accept a regular language?

REGTM = {〈M〉|M is a TM and L(M) is regular}

Theorem:REGTM is undecidable.

Skeleton of Proof:

By contradiction.

Assume REGTM is decidable.

Let R be a TM that decides REGTM.

Use R to construct S, a TM that decides ATM.

But how?Slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 53

Page 54: The Acceptance Problem is not decidable. Reductions and

Undecidable Problems (3)

REGTM = {〈M〉|M is a TM and L(M) is regular}

Intuition: Modify M so that the resulting TM accepts aregular language if and only if M accepts w.

Design M2 so that

if M does not accept w, then M2 accepts {0n1n|n ≥ 0}(non-regular)

if M accepts w, then M2 accepts Σ∗ (regular).

Slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 54

Page 55: The Acceptance Problem is not decidable. Reductions and

Undecidable Problems (3)

Given M and w, construct M2:

On input x,

1. If x has the form 0n1n, accept it.

2. Otherwise, run M on input w and accept x if M acceptsw.

Claim:

If M does not accept w, then M2 accepts {0n1n|n ≥ 0}.

If M accepts w, then M2 accepts Σ∗.

Given 〈M,w〉 there is a TM which constructs 〈M2〉.

Slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 55

Page 56: The Acceptance Problem is not decidable. Reductions and

Undecidable Problems (3)

REGTM = {〈M〉|M is a TM and L(M) is regular}

Theorem: REGTM is undecidable.

Define S:

On input 〈M,w〉,1. Construct M2 from M and w.

2. Run R on input 〈M2〉.3. If R accepts, accept; if R rejects, reject.

TM S decides ATM, a contradiction ♣

Slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 56

Page 57: The Acceptance Problem is not decidable. Reductions and

Undecidable Problems (4)

Are two TMs equivalent?

EQTM = {〈M1,M2〉 | M1,M2 are TMs and

L(M1) = L(M2)}

Theorem: EQTM is undecidable.

We are getting tired of reducing ATM to everything.

Let’s try instead a reduction from EMPTYTM to EQTM.

Slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 57

Page 58: The Acceptance Problem is not decidable. Reductions and

Undecidable Problems (4)

EQTM = {〈M1,M2〉 | M1,M2 are TMs and

L(M1) = L(M2)}

Theorem: EQTM is undecidable.Idea:

EMPTYTM is the problem of testing whether a TMlanguage is empty.EQTM is the problem of testing whether two TMlanguages are the same.If one of these two TM languages happens to be empty,then we are back to EMPTYTM.So EMPTYTM is a special case of EQTM.

The rest is easy.Slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 58

Page 59: The Acceptance Problem is not decidable. Reductions and

Undecidable Problems (4)

EQTM = {〈M1,M2〉 | M1,M2 are TMs and

L(M1) = L(M2)}

Theorem: EQTM is undecidable.

Let MNO be this TM: On input x, reject.

Let R decide EQTM.

Let S be: On input 〈M〉:1. Run R on input 〈M,MNO〉.2. If R accepts, accept; if R rejects, reject.

If R decides EQTM, then S decides EMPTYTM. ♣

Slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 59

Page 60: The Acceptance Problem is not decidable. Reductions and

Bucket of Undecidable Problems

Same techniques prove undecidability of

Does a TM accept a decidable language?

Does a TM accept a context-free language?

Does a TM accept a finite language?

Does a TM halt on all inputs?

Is there an input string that causes a TM to traverse allits states?

Slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 60

Page 61: The Acceptance Problem is not decidable. Reductions and

Reducibility

So far, we have seen many examples of reductions fromone language to another, but the notion was neither definednor treated formally.

Reductions play an important role in

decidability theory (here and now)

complexity theory (to come)

Time to get formal.

Slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 61

Page 62: The Acceptance Problem is not decidable. Reductions and

Computable Functions

A TM computes a total function

f : Σ∗ −→ Σ∗

if the TM

when starting with an input w,

it halts with only f(w) written on tape.

The definition can be extended to functions of more thanone variable, where some special separator symbolindicates end of one variable and beginning of next.

Slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 62

Page 63: The Acceptance Problem is not decidable. Reductions and

Computable Functions

A TM computes a partial function

f : Σ∗ −→ (Σ∗ ∪ ⊥)

if the TM

when starting with an input w,

if f(w) is defined, TM halts with only f(w) on tape,

if f(w) is undefined, TM does not halt.

Computable functions are also called (total or partial)recursive functions.

Slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 63

Page 64: The Acceptance Problem is not decidable. Reductions and

Computable Functions

Claim: All the “usual” arithmetic functions on integers arecomputable.

These include addition, subtraction, multiplication, division(quotient and remainder), exponentiation, roots (to aspecified precision), modular exponentiation, greatestcommon divisor.

Even non-arithmetic functions, like logarithms andtrigonometric functions, can be computed (to a specifiedprecision), using Taylor expansion or other numericmathematic techniques.

Exercise: Design a TM that on input 〈m,n〉, halts with〈m + n〉 on tape.

Slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 64

Page 65: The Acceptance Problem is not decidable. Reductions and

Mapping Reductions

Definition: Let A and B be two languages. We say thatthere is a mapping reduction from A to B, and denote

A ≤m B

if there is a computable function

f : Σ∗ −→ Σ∗

such that, for every w,

w ∈ A⇐⇒f(w) ∈ B.

The function f is called the reduction from A to B.

Slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 65

Page 66: The Acceptance Problem is not decidable. Reductions and

Mapping Reductions

A mapping reduction converts questions aboutmembership in A to membership in B

Slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 66

Page 67: The Acceptance Problem is not decidable. Reductions and

Mapping Reductions: Properties

Theorem 1:If A ≤mB and B is decidable, then A is decidable.

Theorem2 :If A ≤mB and B is recursively enumerable, then A isrecursively enumerable.

Slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 67

Page 68: The Acceptance Problem is not decidable. Reductions and

Mapping Reductions: Corollaries

Corollary 1: If A ≤m B and A is undecidable, then B isundecidable.

Corollary 2: If A ≤m B and A is not in RE , then B is not inRE .

Corollary 3: If A ≤m B and A is not in coRE , then B is not incoRE .

Slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 68

Page 69: The Acceptance Problem is not decidable. Reductions and

Example: Halting

Recall that

ATM = {〈M,w〉|TM M accepts input w}HTM = {〈M,w〉|TM M halts on input w}

Earlier (last week) we proved that

HTM undecidable

by (de facto) reduction from ATM.

Let’s reformulate this.

Slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 69

Page 70: The Acceptance Problem is not decidable. Reductions and

Example: Halting

Define a computable function, f :

input of form 〈M,w〉output of form 〈M ′, w′〉where we want to have〈M,w〉 ∈ ATM ⇐⇒ 〈M ′, w′〉 ∈ HTM.

Slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 70

Page 71: The Acceptance Problem is not decidable. Reductions and

Example: From ATM to HTM

The following machine, F , computes this function f .F = on input 〈M,w〉:

Construct the following machine M ′.M ′: on input x

run M on x

If M accepts, accept.if M rejects, enter a loop.

output 〈M ′, w〉The function f is clearly computable. It is also a mappingreduction from ATM to HTM. ♣

Slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 71

Page 72: The Acceptance Problem is not decidable. Reductions and

Example: From EMPTYTM to EQTM

A mapping reduction from EMPTYTM to EQTM:

f(M) = 〈M,M1〉

where M1 is the TM that rejects all inputs.

Slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 72

Page 73: The Acceptance Problem is not decidable. Reductions and

Example: From ATM to EMPTYTM

On input 〈M,w〉 we constructed Mw, such thatMw = {w} if M accepts w, and Mw = ∅ otherwise.

So f(〈M,w〉) = Mw is a mapping reduction from ATM to

EMPTYTM!

Our proof is still ok, as R is closed undercomplementation.

But we still have no mapping reduction from ATM toEMPTYTM. In fact, it can be shown that there is nosuch reduction!

Slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 73

Page 74: The Acceptance Problem is not decidable. Reductions and

TM Equality

Theorem: Both EQTM and its complement, EQTM, are notenumerable. Stated differently, EQTM is neitherenumerable nor co-enumerable.

We show that ATM is reducible to EQTM. The samefunction is also a mapping reduction from ATM to

EQTM, and thus EQTM is not enumerable.

We then show that ATM is reducible to EQTM. Thenew function is also a mapping reduction from ATM toEQTM, and thus EQTM is not enumerable.

Slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 74

Page 75: The Acceptance Problem is not decidable. Reductions and

TM Equality

Claim: ATM is reducible to EQTM.

f1 : ATM −→ EQTM works as follows:

F1: On input 〈M,w〉Construct machine M1: on any input, reject.

Construct machine M2: on input x, run M on w. If itaccepts, accept.

Output 〈M1,M2〉.

Slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 75

Page 76: The Acceptance Problem is not decidable. Reductions and

TM Equality

F1: On input 〈M,w〉Construct machine M1: on any input, reject.Construct machine M2: on any input x, run M on w.

If it accepts, accept x.Output 〈M1,M2〉.

Note

M1 accepts nothing

if M accepts w then M2 accepts everything,and otherwise nothing.

so 〈M,w〉 ∈ ATM ⇐⇒ 〈M1,M2〉 ∈ EQTMf1 is clearly computable. Thus it is a reduction fromATM to EQTM. ♠

Slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 76

Page 77: The Acceptance Problem is not decidable. Reductions and

TM Equality

Claim: ATM is reducible to EQTM.

f2 : ATM −→ EQTM works as follows:

F : On input 〈M,w〉Construct machine M1: on any input, accept.

Construct machine M2: on any input x, run M on w.If it accepts, accept.

Output 〈M1,M2〉.

Slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 77

Page 78: The Acceptance Problem is not decidable. Reductions and

TM Equality

F2: On input 〈M,w〉Construct machine M1: on any input, accept.Construct machine M2: on any input x, run M on w.

If it accepts, accept.Output 〈M1,M2〉.

Note

M1 accepts everything

if M accepts w, then M2 accepts everything,and otherwise nothing.

〈M,w〉 ∈ ATM ⇐⇒ 〈M1,M2〉 ∈ EQTM.

f2 is clearly computable. Thus it is a reduction fromATM to EQTM. ♣

Slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 78

Page 79: The Acceptance Problem is not decidable. Reductions and

Non Trivial Properties of RE Languages

A few examples

L is finite.

L is infinite.

L contains the empty string.

L contains no prime number.

L is co-finite.

. . .All these are non-trivial properties of enumerablelanguages, since for each of them there is L1, L2 ∈ RE suchthat L1 satisfies the property but L2 does not.

Q.: Are there any trivial properties of RE languages?

Slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 79

Page 80: The Acceptance Problem is not decidable. Reductions and

Rice’s Theorem

Theorem Let C be a proper non-empty subset of the set ofenumerable languages. Denote by LC the set of all TMsencodings, 〈M〉, such that L(M) is in C. Then LC isundecidable.

(See problem 5.22 in Sipser’s book)

Slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 80

Page 81: The Acceptance Problem is not decidable. Reductions and

Rice’s Theorem

Theorem Let C be a proper non-empty subset of the set ofenumerable languages. Denote by LC the set of all TMsencodings, 〈M〉, such that L(M) is in C. Then LC isundecidable.

Proof by reduction from ATM.

Given M and w, we will construct M0 such that:

If M accepts w, then 〈M0〉 ∈ LC.

If M does not accept w, then 〈M0〉 6∈ LC .

Slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 81

Page 82: The Acceptance Problem is not decidable. Reductions and

Rice’s Theorem

Without loss of generality, ∅ 6∈ C.

(Otherwise, look at C, also proper and non-empty.)

Since C is not empty, there exists some language L ∈ C.Let ML be a TM accepting this language (recall Ccontains only recuresively enumerable languages).

continued . . .

Slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 82

Page 83: The Acceptance Problem is not decidable. Reductions and

Rice’s Theorem

Given 〈M,w〉, construct M0 such that:

If M accepts w, then L(M0) = L ∈ C.

If M does not accept w, then L(M0) = ∅ 6∈ C.

M0 on input y:

1. Run M on w.

2. If M rejects w, reject y.

3. If M accepts w, run ML on y.a. if ML accepts, accept y, andb. if ML rejects, reject y.

Claim: The transformation 〈M,w〉 → 〈M0〉 is a mappingreduction from ATM to LC.

Slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 83

Page 84: The Acceptance Problem is not decidable. Reductions and

Rice’s Proof (Concluded)

If 〈M,w〉 ∈ ATM then M0 gets to step 2, and runs ML ony.

In this case, L(M0) = L, so L(M0) ∈ C.

On the other hand, if 〈M,w〉 6∈ ATM then M0 never getsto step 3.

In this case, L(M0) = ∅, so L(M0) 6∈ C.

This establishes the fact that 〈M,w〉 ∈ ATM iff〈M0〉 ∈ LC. So we have ATM ≤m LC, thus LC isundecidable. ♣

Slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 84

Page 85: The Acceptance Problem is not decidable. Reductions and

Rice’s Theorem (Reflections)

Rice’s theorem can be used to show undecidability ofproperties like

“does L(M) contain infinitely many primes”, “doesL(M) contain an arithmetic progression of length 15”,“is L(M) empty”

Decidability of properties related to the encoding itselfcannot be inferred from Rice. For example “does 〈M〉 hasan even number of states” is decidable.

Properties like “does M reaches state q6 on the emptyinput string” are undecidable, but this does not follow fromRice’s theorem.

Slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 85

Page 86: The Acceptance Problem is not decidable. Reductions and

Rice’s Theorem (Few More Reflections)

The way we stated it, Rice’s theorem does not sayanything on membership in RE of problems like “isL(M) finite”. It only says that such languages are not inR.

However, looking at the proof, we see that if ∅ 6∈ C thenATM ≤m LC, while if ∅ ∈ C then ATM ≤m LC.

So if ∅ ∈ C, we can actually conclude that LC /∈ RE.

In particular, since the empty set is finite, we canconclude that the set of coding of TMs, whose languageis finite, is not recursively enumerable.

Slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 86