CS 3240 Chapter 11

They may not halt on every possible input!And not just because the creator of a specific TM was a doofus This is related to the major mathematical/computational discovery of the 20th century!There are propositions that cannot be decided (proven)

A question is decidable if there is a TM that always halts and answers yes or no for each possible inputThe TM therefore constitutes an algorithmA language is decidable if there is a TM that always halts and answers accept or reject whenever an input string is in the language or notaka Turing acceptable

A function is computable if there is a TM that always halts with the appropriate output for each possible input in the function domainaka Total function

Let g(x,y) be some computable functionLet f(x) = the smallest p where g(x, p) = 1, or 0, if such a p does not existPseudo-algorithm for f(x): m = 0; while (g(x, m) != 1) ++m; cout

3 possibilities in general when a TM processes an input string:Accepts (goes to an accepting halt state)Rejects (e.g., crashes, or gives a no answer)Hangs (infinite loop)

A language for which there is a TM that always halts for and accepts strings in the language is recognizableIt knows one when it sees one :-)But it may hang on strings not in the language

A function that is defined for only some of its domain elements is a partial functionIt may hang (or blow up) on some inputse.g., divide by zeroThis is the computational analogue to a recognizable languageyoull always get an answer with a valid inputit may hang on invalid input

*CS 3240 - Introduction

LanguageMachineGrammarRegularFinite AutomatonRegular Expression,Regular GrammarContext-FreePushdown AutomatonContext-Free GrammarRecursively EnumerableTuring MachineUnrestricted Phrase-Structure Grammar

CS 3240 - Introduction

A language L over the alphabet is called recursively enumerable (aka recognizable) if there is a TM T that accepts every word in L and either rejects (crashes) or loops forever for every word in the language L', the complement of L: accept(T) = L reject(T) + loop(T) = L'recursively enumerable (r.e.) = recognizable

We just saw one (slide 5)Let g(x,y) be some computable functionLet f(x) = the smallest p where g(x, p) = 1, or 0, if such a p does not existLet Lg be the set of functions, f, corresponding to all computable functions g, as explained aboveSuch functions can be encoded as strings, and are therefore countableLg can then be seen as a language (a set of strings)

A language L over the alphabet is called recursive (aka decidable) if there is a TM T that accepts every word in L and rejects every word in L'; that is, accept(T) = L reject(T) = L'A TM that accepts a recursive set is a model of an algorithm. It always halts.recursive = decidable

There exist languages that are r.e. but not recursiveWe just saw oneThere exist languages that aren't even r.e.!(You'll see one soon)All are contrivedLanguages generated by grammars are r.e. or better

All r.e. languages are closed under union, intersection, concatenation, and Kleene*Everything but complement!Recursive languages are also closed under complementAlso: If L and L' are r.e., then L is recursiveHere come the Proofs

Let M be a machine that decides a recursive language, LForm the machine M' by inverting the acceptability output of MGoes to a reject state insteadThen M' decides L'So L' is recursive

Suppose L and L' are both r.e.Let M recognize L, and M' recognize L'M may hang on elements of L', but M' doesn'tForm a new machine, M* that calls M and M' in parallel (non-deterministically)If M accepts w, so does M*If M' accepts w, reject wThere are no other possibilities! (No hanging)Therefore, L is decidable/recursive, by definition

TMs can recognize/accept strings from certain languages and/or compute functions If there is a TM, M, that accepts a language, L, and M always halts, then L is recursive If there is no such M for L, but there is instead a machine M that accepts every string in L, but M may hang on strings not in L, then L is recursively enumerable

The complement of a recursive language is recursiveIn fact, recursive languages are closed under all operations, like regular languages arer.e. languages are closed under intersectionThe complement of a r.e. language may not be r.e.But if it is, then both languages are actually recursive!

A Timely Interlude

The real numbers in (0,1) are uncountableThey cannot be mapped in a 1-to-1 fashion to the counting numbersProof:Assume they can be: r0, r1, r2, Arrange their digits in a table a[ ][ ]each row, a[i], contains the digits or riThe diagonal sequence (a[n][n]+1) mod 10, representing a valid real number, is not in the table! Contradiction!

The power set of the natural numbers, Nthe set of all subsets (2N)Suppose it is countableThe we can enumerate the sets: p0, p1, p2, Now consider the set T = {i | i f(i) = pi }Certainly T is a set of integers, so T P(N)Call it pkQuestion: Is k in T = pk?

The power set of any countably infinite set is uncountableMuch bigger than a countable set!The number of countable sets is negligible compared to the number of uncountable sets

Well?

A language over an alphabet is a subset of *the latter being an infinite setThe set of all languages over is therefore the power set of * (2*)which we just showed is uncountableSo the number of languages over any finite alphabet is uncountable

The # of TMs are countable, but the # of languages is notTherefore, some languages cannot be recognized by a TMThere aren't enough TMs to go around!Just like there are more reals than integersSo, non-r.e. languages must exist!

Just take the complement of any r.e. language that is not recursiveExample (page 279):Consider all TMs, Mi with alphabet ={a}.Let X = {ai: ai L(Mi)}This is r.e., but not recursiveBecause we can construct a TM that carries out the computation, but it may not halt when ai L(Mi)Then X must be non-r.e.!

Suppose X is r.e.Then there is a TM, Mk, that recognizes itNow ask the question, is ak X ?If it is, this means ak L(Mk) = X, by definitionContradiction!If it isnt, then this means that ak L(Mk) = XAgain, by definitionAgain, a contradiction! X is not r.e.

What we have just described is the membership problem:Given a r.e. language, L, and a string, w, is w in L?We have just shown that the membership problem is, in general, undecidable

*CS 3240 - Introduction

LanguageMachineGrammarRegularFinite AutomatonRegular Expression,Regular GrammarContext-FreePushdown AutomatonContext-Free GrammarRecursively EnumerableTuring MachineUnrestricted Grammar

CS 3240 - Introduction

Left-hand side of the rule is a concatenation of one or more symbolsThere must be at least one variable on the left is not allowed on the leftAny string is allowed on the right, including aka Type 0 Grammar

S aAbc | abc | Ab bAAc BbccbB BbaB aa | aaAS aAbc abAc abBbcc aBbbcc aabbcc

An A is created as the left-most variable. It travels to the right until it finds a 'c', then creates a new b and c, and becomes a B. The B moves back to create the extra needed a. The last rule allows the option to do it all over again (by introducing another A). This is similar to what the Turing machine for the same language does, except the TM marks instead of generates the letters.

We have already seen CFGs for thisThis unrestricted grammar:Introduces Xs and Ys togetherAllows moving Xs and Ys arbitrarily by swapping themS XYS | XY YX YX XY X a Y b

Cannot use the decidability algorithms for CFGse.g., CYK algorithm does not applyNo Normal FormNon-null productions may create shorter stringsTerminals can disappear!

For every r.e. language, there is an unrestricted grammar that generates itFor every unrestricted grammar, there is a TM that recognizes its languageIt may or may not decide itWe will not prove thisbut the book does

Like Unrestricted Grammars except:Right-hand side must be no shorter than left-hand sideSo strings never shrinkHence the name non-contracting or monotonicCannot contain anywhereContext-sensitive languages don't need unlimited memorySince intermediate sentential forms never shrink, the largest memory requirement is proportional to |w|Accepting machine: Linear Bounded Automaton

S aAbc | abcAb bAAc BbccbB BbaB aa | aaA

Every context-sensitive language is recursivethere is a TM that accepts it (i.e., always halts)But not all recursive languages are context-sensitiveSo Context Sensitive languages form a proper subset of Recursive LanguagesJust like Recursive languages are a proper subset of Recursively enumerable languages

Type 3 grammarType 2 grammarType 1 grammarType 0 grammar

LanguageGrammarMachineRecursively Enumerable(Turing Recognizable)Unrestricted Phrase Structure (Type 0)TM (that may not halt)Recursive(Turing Decidable) "TM (that always halts)Context-SensitiveContext Sensitive (Type 1)(monotonic/non-contracting)Linear-bounded automata(bounded TM)Context FreeContext Free (Type 2)PDADeterministic Context Free "DPDARegularRegular (Type 3)(Right-linear or Left-linear)FA

*******Draw a picture?*Draw a picture.***Do on board*If it is, it isnt, and vice-versa! Contradiction!*********