computational models: class...

Computational Models: Class 8†

Benny Chor

School of Computer ScienceTel Aviv University

December 7, 2015

†Based on slides by Maurice Herlihy, Brown University, and modifications by IftachHaitner and Yishay Mansour.

Benny Chor (Tel Aviv University) Computational Models, Class 8 December 7, 2015 1 / 65

Class Outline

Universal TMs, HTM /∈ R, an L3 /∈ RE ∪ co−RE (reminder)Hilbert’s 10th problemMapping reductionsProving undecidability by mapping reductionsThe Busy Beaver problem

Sipser’s book, 5.1–5.3


Part I

Previous Lecture – Reminders


Encoding of Turing Machines

Just like any piece of code in any programming language, Turingmachines can be encoded as strings.Such encoding will enable us

To check (by an algorithm) that a given string is a legal encodingof a TM.(Similar to a compiler checking for syntax errors.)To build a universal machine that can read such encoding andemulates the encoded TM on any input string.(Similar to running an interpreter.)


Universal Turing Machines

Algorithm 1 (Universal TM U)On input 〈M,w〉, where 〈M〉 and 〈w〉 are binary strings separated by111.

Checks that 〈M,w〉 is a proper encoding of a TM.Emulate M(w) (how?)

I Accept, if M enters its accept stateI Reject, if M enters its reject state

Notice that as a consequence, if M on input w enters an infinite loop,so does U on input 〈M,w〉.


Universal Turing Machines (2)

The universal machine U obviously has a fixed number of states(100 should do).Despite this, it can simulate machines M with many more states.Universal machines inspired the development of stored-programcomputers in the 40s and 50s.Most of you have seen a universal machine, and have even usedone!


Alan Mathison Turing: The Halting Problem is Undecidable

(figure courtesy of Sivan Toledo)

Turing has shown that no computer program can check if a computercode (say written in Python) will halt or enter an infinite loop.

This is a fundamental limitation of computations. Not just a limitation ofa specific programming language or a manifestation of incompetenceof today’s programmers.


The Acceptance Problem (a close sister of the Halting Problem)

ATM = {〈M,w〉|M is a TM that accepts w}

HTM = {〈M,w〉|M is a TM that halts on w}

Theorem 2ATM is recursively enumerable (namely in ATM ∈ RE).

Proof: The universal machine accepts ATM. ♣

Theorem 3ATM is not recursive (namely ATM /∈ R).

Proof: A diagonalization argument. ♣

Small modifications yield proofs of HTM ∈ RE and HTM /∈ R as well.


A Non-Enumerable Language

We already saw two non-decidable language: ATM and HTM.Can we do better (i.e., worse)?We now display a language that isn’t even recursively enumerable. . . .

Corollary 4

If L is not decidable, then either L or L is not recursively enumerable.

Proof: Assume otherwise, by R = RE ∩ co−RE , L would bedecidable. ♣

Corollary 5

ATM is not recursively enumerable


A Language Outside RE ∪ co−RE

Consider the languageL3 = {〈M1〉,w1, 〈M2〉,w2|w1 ∈ L(M1),w2 6∈ L(M2)}.

We will show that L3 6∈ RE and also L3 6∈ co−RE .

The idea of the proof is that if L3 ∈ co−RE , we can design a Turingmachine, M ′, which accepts ATM.

And if L3 ∈ RE , we can design another Turing machine, M ′′, which alsoaccepts ATM.

In both cases, this leads to a contradiction, as ATM /∈ RE .


Claim: L3 6∈ RE

Recall L3 = {〈M1〉,w1, 〈M2〉,w2|w1 ∈ L(M1),w2 6∈ L(M2)}

Proof:Assume that L3 ∈ RE , i.e., ∃M∗ s.t. L3 = L(M∗).

Algorithm 6 (TM M ′ for ATM)On input 〈M,w〉

1 Let M1 be a TM s.t. L(M1) = 1∗.

2 Give 〈M,w〉, generate < M1,1,M,w >.

3 Run M∗ on < M1,1,M,w >.

4 If M∗ accepts < M1,1,M,w >, then M ′ will accept 〈M,w〉.

M ′ accepts ATM, contradiction. ♣


Claim: L3 6∈ co−RE

Recall L3 = {〈M1〉,w1, 〈M2〉,w2|w1 ∈ L(M1),w2 6∈ L(M2)}.

Proof:Assume that L3 ∈ co−RE , i.e. ∃M∗∗ s.t. L̄3 = L(M∗∗).

Algorithm 7 (TM M ′′ for ATM)On input 〈M,w〉

1 Let M2 be a TM s.t. L(M2) = 0∗.

2 Given 〈M,w〉, generate < M,w ,M2,1 >.

3 Run M∗∗ on < M,w ,M2,1 >.

4 If M∗∗ accepts < M,w ,M2,1 >, then M ′′ will accept 〈M,w〉.

M ′′ accepts ATM, contradiction. ♣


And Now to Something Completely Different:Hilbert’s 10th Problem

Har Karbolet, south east boundary of The Big CraterBenny Chor (Tel Aviv University) Computational Models, Class 8 December 7, 2015 13 / 65

Part II

Hilbert’s Tenth Problem


Hilbert’s 10th Problem

In 1900, David Hilbert delivered a now-famous address at theInternational Congress of Mathematicians in Paris, France.

Presented 23 central mathematical problems, (additional ref)challenge for the next (20th) centurythe 10th problem directly concerned algorithms


http://en.wikipedia.org/wiki/Hilbert

http://mathworld.wolfram.com/HilbertsProblems.html


The Problem: Devise an algorithm that tests whether a multivariatepolynomial with integer coefficients† has an integral root.

Actually, what he said (translated from German) was

“to devise a process according to which it can be determinedby a finite number of operations”.

Note thatHilbert explicitly asks that algorithm be “devised”apparently Hilbert assumes that such an algorithm must exist, andsomeone “only” need find it.

†e.g. 3x2y2 + z2 − 2 or x3 + y3 + t4 − z3 − 3xytzBenny Chor (Tel Aviv University) Computational Models, Class 8 December 7, 2015 16 / 65


We now know no algorithm exists for this task.Mathematicians in 1900 could not have proved this, because theydidn’t have a formal notion of an algorithm.Intuitive notions suffice for constructing algorithms, like Euclid gcdor Newton–Raphson root finding (we know one when we see it).

Formal notions are required to show that no algorithm exists.



In 1970, 23 years old Yuri Matijasevic̆, building on work of MartinDavis, Hilary Putnam, and Julia Robinson, proved that no algorithmexists for testing whether a multivariate polynomial has integral roots(a survey of the proof)


http://www.goldenmuseum.com/1612Hilbert_engl.html

Reformulating Hilbert’s Tenth Problem

Consider the language:

D = {p | p is a multivariate polynomial with integer coefficients,which has an integral root}

Hilbert’s tenth problem asks whether this language is decidable.

It is easy to see that this language is recursively enumerable(why?).

By Matijasevic̆ theorem, it is not decidable, so it is in RE \ R.


Univariate Polynomials

Consider the simpler language:

D1 = {p | p is a polynomial over x with an integral root}

Here is an algorithm that accepts D1.On input p,

evaluate p with x set successively to 0,1,−1,2,−2, . . ..if p evaluates to zero, accept.


Univariate Polynomials (2)

D1 = {p | p is a polynomial over x with an integral root}

Note that

If p has an integral root, the machine accepts.If not, M1 loops.M1 is an acceptor, but not a decider.



In fact, D1 is decidable.

Can show that all real roots of p[x ] lie inside interval

( −|kcmax/c1|, |kcmax/c1| ) ,

where k is number of terms, cmax is max coefficient, and c1 ishigh-order coefficient.

By Matijasevic̆ theorem, such effective bounds on range of real rootscannot be computed for multivariable polynomials.

Furthermore, it was shown that the problem is undecidable even whenrestricted to polynomials of degree ≤ 4 in just nine variables.


And Now to Something Completely Different:Reductions

Har Karbolet, south east boundary of The Big CraterBenny Chor (Tel Aviv University) Computational Models, Class 8 December 7, 2015 24 / 65

Part III

Reductions


Reminder and Goal

We have alreadyEstablished Turing Machines as the gold standard of computersand computability.Seen examples of solvable problems.And saw two related problem, ATM and HTM that arecomputationally unsolvable (i.e. undecidable)

ATM = {〈M,w〉 |M is a TM that accepts w}

HTM = {〈M,w〉 |M is a TM that halts on input w}

Today, we look at other computationally unsolvable problems, andprove undecidability via the technique of mapping reductions.


Computable Functions

Definition 8 (total computable functions)A TM M computes a total function f : Σ∗ −→ Σ∗, if when starting withan input w , M halts with (only) f (w) written on tape.a

aThe definition naturally extends to functions of more than one variable,where some special separator symbol indicates end of one variable andbeginning of next.


Computable Functions (take 2)

Definition 9 (partially computable functions)A TM M computes a partial function f : Σ∗ −→ (Σ∗∪ ⊥), if whenstarting with an input w :

if f (w) is defined (i.e., 6=⊥), M halts with only f (w) on tape,if f (w) is undefined, M does not halt.

Computable functions are also called (total or partial) recursivefunctions.


Examples

Claim 10All the “usual” arithmetic functions on integers are computable.

These include addition, subtraction, multiplication, division (quotientand remainder), exponentiation, roots (to a specified precision),modular exponentiation, greatest common divisor.

Even non-arithmetic functions, like logarithms and trigonometricfunctions, can be computed (to a specified precision), using Taylorexpansion or other numeric mathematic techniques.

Exercise 11Design a TM that on input 〈m,n〉, halts with 〈m + n〉 on tape.

Remark: Note that unary, binary, octal, decimal, or hexadecimalencoding of the numbers will influence the details of such TM.


Examples, take 2

A useful class of functions modifies TM descriptions. For example†:

Algorithm 12On input w : If w 6= 〈M〉 for some TM, return w .Else if w = 〈M〉 for some TM, return 〈M ′〉, where

L(M ′) = L(M), andM ′ does not halt for w 6∈ L(M ′)

Question 13Is the function defined above total? is it computable?

†We saw TM encodings in lecture 7. Checking if a string is a legal TM encoding isa simple syntactic issue, similar to IDLE checking that a given Python program issyntactically OK - parenthesis are balanced, newline and tab after a colon, etc.


Examples, take 3

Implementing the M to M ′ transformation:

Given M = (Q,Σ, Γ, δ,q0,qa,qr ), build M ′ = (Q′,Σ, Γ, δ′,q0,qa,qr )

Let Q′ = Q ∪ {q∗} (a new state).Define δ′ as follows:

δ′(q, σ) :=

(q∗, σ,R), if δ(q, σ) = (qr , ·, ·) M’ enters q∗

(q∗, σ,R) q = q∗ in q∗, M’ moves right(q′, σ′,D), otherwise if δ(q, σ) = (q′, σ′,D)

Question 14Suppose your input is a Python function that returns either True orFalse. Viewing it as a string, how would you code (in Python) amodification similar to the above?


Reducibility

Finding your way around a new city, reduces to . . .obtaining a city map (more realistically, it probably reduces tohaving google maps on your smartphone :-).Finding the median in an array, reduces to . . . sorting an array.

This is the core idea behind programming languages’ functionsand procedures.


Reducibility, in our Context

Involves two problems, or languages, A and B.

Desired property: If A reduces to B, then any solution of B can be usedto find a solution of A.

Remark 15“A reduces to B” says nothing about solving A by itself or B by itself.


Reducibility, in our Context (cont.)

Fact 16If A reduces to B, then A cannot be harder than B

if B is decidable, so is A.if A is undecidable, then B is undecidable.

We next use reductions and the undecidability of ATM, to show theundecidability of several additional problems.


HTM is undecidable

ATM = {〈M,w〉|M is a TM that accepts w}HTM = {〈M,w〉|M is a TM and M halts on input w}

Theorem 17HTM is undecidable.


HTM is undecidable (proof by reduction)

Proof: Assume, by way of contradiction, that TM R decides HTM.

Algorithm 18 (TM S)On input 〈M,w〉,

1 Emulate R on 〈M,w〉.2 If R rejects, reject.3 If R accepts (meaning M halts on w), emulate M on w until it halts

(namely run U on 〈M,w〉).4 If M accepted, accept; otherwise reject.

TM S decides ATM, a contradiction ♣

What we actually did is a reduction from ATM to HTM.


REGTM is Undecidable (does a TM Accepts a Regular Language)

Consider the languageREGTM = {〈M〉 |M is a TM and L(M) is regular}

Theorem 19REGTM is undecidable.

Proof’s idea: By contradiction.Assume REGTM is decidable and let R be a TM that decidesREGTM.Use R to construct S – a TM that decides ATM.

Question 20But how we construct S?

Intuition: On input 〈M,w〉, build Mw that accepts a regular language iffM accepts w .


The Turing Machine Mw : Design Goals and ImplementationDesign Goals:

if M does not accept w , then Mw accepts the (non-regular)language {0n1n|n ≥ 0}if M accepts w , then Mw accepts the (regular) language Σ∗ .

Implementation:

Algorithm 21 (Mw )On input x ,

1 If x has the form 0n1n, accept it.2 Otherwise, emulate M on input w and accept x if M accepts w .

Claim 221 If M does not accept w , then Mw accepts (the language){0n1n|n ≥ 0}.

2 If M accepts w , then Mw accepts (the language) Σ∗.3 The function: on input 〈M,w〉 output 〈Mw 〉, is computable.


REGTM is Undecidable: The Turing Machine S

Algorithm 23 (S)On input 〈M,w〉,

1 Construct Mw from M and w .2 Emulate R on input 〈Mw 〉, where R be a TM that decides REGTM.3 If R accepts, accept; if R rejects, reject.

Claim 24S decides ATM.

A contradiction to the undecidability of ATM. ♣


Bucket of Undecidable Problems

Same techniques can be used to prove undecidability of

Does a TM accept the empty language?Does a TM accept a decidable language?Does a TM accept a context-free language?Does a TM accept a finite language?Does a TM halt on all inputs?Is there an input string that causes a TM to traverse all its states?


And Now to Something Completely Different:Mapping Reductions

The lower part of Nahal Ofran, south east boundary of The Big CraterBenny Chor (Tel Aviv University) Computational Models, Class 8 December 7, 2015 41 / 65

Part IV

Mapping Reductions


Reducibility

So far, we have seen a few examples of reductions from one languageto another, but the notion was neither defined nor treated very formally.

Reductions play an important role indecidability theory (here and now)complexity theory (to come)

Time to get formal.


Mapping Reductions

Definition 25A total, computable function f : Σ∗ −→ Σ∗ is a reduction from languageA to language B, if w ∈ A⇐⇒ f (w) ∈ B, for every w ∈ Σ∗.


Mapping Reductions, take 2

If a reduction from A to B exists, we say that A is mapping reducible toB, denoted by A ≤m B.

A mapping reduction converts questions about membership in A toquestions about membership in B†

Remark 26

Note that A ≤m B ⇐⇒ A ≤m B

†but not the other way around, as we will see soon.Benny Chor (Tel Aviv University) Computational Models, Class 8 December 7, 2015 45 / 65

Applications

Theorem 27If A ≤m B and B is decidable, then A is decidable.

Proof: Let M be the decider for B, and f a (mapping) reduction from Ato B.

Algorithm 28 (N)On input w

1 Compute f (w)

2 Emulate M on input f (w) and output exactly what M outputs. ♠

Corollary 29If A ≤m B and A is undecidable, then B is undecidable.

In fact, this corollary has already been implicitly used when we provedundecidability of languages other than ATM.


HTM is Undecidable – Revisited

Recall thatATM = {〈M,w〉|M is a TM that accepts w}HTM = {〈M,w〉|M is a TM and M halts on input w}

Earlier we proved that HTM undecidable by (de facto) reduction fromATM. Let us reformulate this by an explicit mapping reduction.

Claim 30ATM ≤m HTM


HTM is Undecidable – Revisited (cont.)

The following computable, program modification function, f ,establishes 〈M,w〉 ∈ ATM ⇐⇒ f (〈M,w〉) ∈ HTM.

Definition 31 (f )

On input 〈M,w〉, return 〈M`oop,w〉.TM M`oop is defined as follows:On input x , emulate M(x) and

accepts if M accepts; enters a loop if M rejects.


Building Mòop

Yes, you are right. We have already seen this construction:

Given M = (Q,Σ, Γ, δ,q0,qa,qr ), buildMòop = (Q′,Σ, Γ, δ′,q0,qa,qr )

Let Q′ = Q ∪ {q∗}Define δ′ as follows:

δ′(q, σ) :=

(q∗, σ,R), if δ(q, σ) = (qr , ·, ·)(q∗, σ,R) q = q∗

(q′, σ′,D), otherwise if δ(q, σ) = (q′, σ′,D)

Note that L(Mòop) = L(M) and 〈M,w〉 ∈ ATM⇐⇒ f (〈M,w〉) ∈ HTM


HTM,ε is Undecidable

We have defined HTM = {〈M,w〉 |M is a TM and M halts on input w},and define now HTM,ε = {〈M〉 |M is a TM and M halts on input ε}.

Claim 32HTM ≤m HTM,ε

The following computable function f establishes〈M,w〉 ∈ HTM ⇐⇒ f (〈M,w〉) ∈ HTM,ε.

Definition 33 (f )On input 〈M,w〉, return 〈Me

w 〉.TM Me

w is defined as follows:On input x , erase x , write (the now fixed) w , and emulate M on input w .

If the input to f is not of the form 〈M,w〉, f produces 〈Mloop〉, anencoding of a TM that never halts.


Building Mew (for those rough souls)

Given M = (Q,Σ, Γ, δ,q0,qa,qr ) and w = w1, . . .wn, buildMe

w = (Q′,Σ, Γ, δ′,−→q 1,qa,qr ), assume n ≥ 1.Let Q′ = Q ∪ {−→q 1, . . .

−→q n+1,←−q ,←−q 1}

Define δ′ as follows:

δ′(q, σ) :=

(−→q 2, $,R) q =

−→q 1(−→q i+1,wi ,R) q =

−→q i ,2 ≤ i ≤ n(−→q n+1, ,R) q =

−→q n+1, σ 6=(←−q , σ,L) q =

−→q n+1, σ =

(←−q , σ,L) q =

←−q , σ 6= $

(←−q 1,w1,R) q =

←−q , σ = $

(q0, σ,L) q =←−q 1

(q′, σ′,D), otherwise if δ(q, σ) = (q′, σ′,D)


HTM,ε is Undecidable, concluding

If M halt on w , then Mew halts on every input x , including ε.

If M does not halt on w , then Mew does not halt on any input x ,

including ε.

Thus, Mew halts on ε⇐⇒ M halts on w .

In other words, 〈M,w〉 ∈ HTM ⇐⇒ f (〈M,w〉) = 〈Mew 〉 ∈ HTM,ε,

establishing that indeed HTM ≤m HTM,ε. ♠


The Mapping Reducible Relation is Not Symmetric

HTM,ε = {〈M〉|M is a TM and M halts on ε}

Claim 34Let L = {0n : n ∈ N}. Then L ≤m HTM,ε, but HTM,ε �m L

Proof: It is easy to show that HTM,ε �m L (ahhhm..., why?).For proving L ≤m HTM,ε, define f as follows:

Definition 35 (f )On input w . Return 〈MHalt〉 if w ∈ L, and return 〈MLoop〉 otherwise.Where MHalt halts on ε, and MLoop loops on ε.

Note that deciding membership in L is really easy, and that both 〈MHalt〉and 〈MLoop〉 are just fixed strings, which can be hard wired into themachine that computes f .


Enumerability

Theorem 36If A ≤m B and B is enumerable, then A is enumerable.

Proof is same as before, using accepters instead of deciders.

Corollary 37If A ≤m B and A is not enumerable, then B is not enumerable.


TM Equality

EQTM = {〈M1,M2〉 : M1,M2 are TMs and L(M1) = L(M2)}

Theorem 38

Both EQTM and, its complement, EQTM, are not enumerable.

Stated differently, EQTM is neither enumerable nor co-enumerable, orEQTM /∈ RE ∪ co−RE .This is the second time we see a language /∈ RE ∪ co−RE .

We will show that

ATM ≤m EQTM, and ATM ≤m EQTM.It follows that ATM ≤m EQTM and ATM ≤m EQTM (why?)Hence, neither EQTM nor EQTM are enumerable.


ATM ≤m EQTM

Claim 39ATM ≤m EQTM.

Definition 40 (f )On input 〈M,w〉

1 Construct a machine M1 that accepts Σ∗

2 Construct a machine M2 (= Mew ) accepting all x , iff M accepts w .

3 Return 〈M1,M2〉.

Noteif M accepts w , then M2 accepts everything. Otherwise, M2accepts nothing.Hence, 〈M,w〉 ∈ ATM ⇐⇒ 〈M1,M2〉 ∈ EQTM. ♣


ATM ≤m EQTM

Claim 41

ATM ≤m EQTM.

Definition 42 (f )On input 〈M,w〉:

1 Construct a machine M1 that accepts ∅.2 Construct a machine M2 (= Me

w ) accepting all x , iff M accepts w .3 Return 〈M1,M2〉.

NoteIf M accepts w then M2 accepts everything. Otherwise, M2accepts nothing.Hence, 〈M,w〉 ∈ ATM ⇐⇒ 〈M1,M2〉 ∈ EQTM ♣


TM Equality, Summary

Since ATM ≤m EQTM, then ATM ≤m EQTM

Since ATM ≤m EQTM then ATM ≤m EQTM

Hence, neither EQTM nor EQTM are enumerable.Stated differently, EQTM is neither enumerable nor co-enumerable,or EQTM /∈ RE ∪ co−RE .


And Now to Something Completely Different:The Busy Beaver Problem

Nahal Ofran, south east boundary of The Big CraterBenny Chor (Tel Aviv University) Computational Models, Class 8 December 7, 2015 59 / 65

Part V

The Busy Beaver Problem


The Busy Beaver Function, Definition

We focus on one tape TMs, with Σ = {0,1} and Γ = {0,1, }.

Definition 43 (Sn and BB(n))

For n ∈ N, let Sn = {all n-state TM’s that halt on ε}.Let BB(n) be maximum # of steps taken by some M ∈ Sn on input ε.

The set Sn is finite (under standard encoding).Every M ∈ Sn runs for finitely many steps on ε.BB(n) is a total function from N to N (in particular, BB(n) ∈ N forevery n ∈ N).

Known bounds on values of BB (size not including qA,qR states):

size (n) 2 3 4 5 6BB(n) 6 21 107 ≥ 47,176,870 ≥ 7.4× 1036534


The Busy Beaver Function is Not Computable

The BB(n) function grows way too fast. No TM can cope with suchrapid growth. This leads to

Theorem 44The busy beaver function is not computable.

Proof: Consider the undecidable languageHTM,ε = {〈M〉|M is a TM and M halts on ε}.

Suppose, towards a contradiction, that BB is computable, using TM R.


The Busy Beaver Function is Not Computable

Suppose, towards a contradiction, that BB is computable, using TM R.

Algorithm 45 (S)On input 〈M〉

1 Determine the number of states in M, n, and computem = R(n) = BB(n).

2 Emulate M on ε for m + 1 steps.3 If M halts then accept, otherwise reject.

Note that if M did not halt in m + 1 steps, then it will never halt!

Claim 46S decides HTM,ε (a contradiction to BB(·) being computable). ♣.


The Busy Beaver Problem

(taken from http://www.saltine.org/joebeaver1.jpg)


The Bounded Busy Beaver Function Is Computable

Definition 47For d ∈ N, define the function BBd : N 7→ N as

BBd (n) :=

{BB(n), n ≤ d ,0, otherwise.

Theorem 48The function BBd is computable for every d ∈ N.

Proof’s idea: “Hardwire" the values BB(1) . . . ,BB(d) into a TM tocompute BBd .Note: Existential proof.


computational models: class...

Documents