jeff edmonds york university cosc 4111 lecture 1 church's thesischurch's...
TRANSCRIPT
Jeff EdmondsYork University
COSC 4111Lecture 1
• Church's Thesis• Primitive Recursive• TM to Primitive Recursive?• Primitive Recursive << TM• TM to Recursive• Register Machine• And/Or/Not Circuits• Computer Circuits• Circuits Depth• Arithmetic Circuits
Models of Computability• Neural Nets
• Poly-Time• Non-Deterministic Machine• Quantum Machine• Context Free Grammars• JAVA to Machine Code
(Compiling/Parsing)• Context Sensitive Grammars• TM to Grammar• Decide vs Accept• Humans• Complexity Classes
Please feel free to ask questions!
Please give me feedbackso that I can better serve
you.
Thanks for the feedback that you have given me already.
Church’s ThesisA computational problem is computable• by a Java Program
• by a Turing Machine
• by a (simple) Recursive Program
• by a (simple) Register Machine
• by a Circuit: And/Or/Not, Computer, Arithmetic, & Neural Nets.
• by a Non-Deterministic Machine
• by a Quantum Machine
• by a Context Sensitive Grammar
NonUniform unreasonable Uniform reasonableComputes the same,
but faster.
Decide vs Accept
Computes the same,but faster.
Church’s Thesis
A Java program can easily simulate each of them.
A computational problem is computable• by a Java Program
• by a Turing Machine
• by a (simple) Recursive Program
• by a (simple) Register Machine
• by a Circuit: And/Or/Not, Computer, Arithmetic, & Neural Nets.
• by a Non-Deterministic Machine
• by a Quantum Machine
• by a Context Sensitive Grammar
Proof
Church’s Thesis Proof
A Java program can be simulated by
a TM. A TM can be simulated by
a Recursive Program. A Recursive Program can be simulated by a
Register Machine. A TMcan be simulated by a
Circuit.
A TMcan be simulated by a
Context Sensitive Grammar.
Non-Deterministic Machine
can be simulated by a Deterministic one.
A computational problem is computable• by a Java Program
• by a Turing Machine
• by a (simple) Recursive Program
• by a (simple) Register Machine
• by a Circuit: And/Or/Not, Computer, Arithmetic, & Neural Nets.
• by a Non-Deterministic Machine
• by a Quantum Machine
• by a Context Sensitive Grammar
A Quantum Machine can be simulated by a
Deterministic one.
Church’s Thesis Proof
A computational problem is computable• by a Java Program
• by a Turing Machine
• by a (simple) Recursive Program
• by a (simple) Register Machine
• by a Circuit: And/Or/Not, Computer, Arithmetic, & Neural Nets.
• by a Non-Deterministic Machine
• by a Quantum Machine
• by a Context Sensitive Grammar
Note you can get anywhere from
anywhere.
Church’s ThesisA computational problem is computable• by a Java Program
• by a Turing Machine
• by a (simple) Recursive Program Define
X = 33Y = 12ac = 3bd = 6
(a+b)(c+d) = 18XY = 396
RecursionFriends & Strong Induction
• Consider your input instance• Allocate work
• Construct one or more sub-instances• It must be smaller
• and meet the precondition• Assume by magic your friends give you the
answer for these.• Don’t micro-manage them by tracing
out what they and their friends friends do.
• Use this help to solve your own instance.• Do not worry about anything else.
• Who your boss is…• How your friends solve their instance…X = 3
Y = 1XY = 3
X = 3Y = 2
XY = 6
X = 6Y = 3
XY = 18
MULT(X,Y):
If |X| = |Y| = 1 then RETURN XY
Break X into a,b and Y into c,d
e = MULT(a,c) and f =MULT(b,d)
RETURN
e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f
We will now define two models of computation:• Primitive Recursion Functions and • μ-Recursive Functions• They were developed about the same time as
Turing Machines• Like TMs, the allowed set of allowed operations
is very restricted.• It was a big historical deal when it was prove that
The Class of μ-Recursive Problems = The Class of Turing Computable Problems
Primitive Recursive
Primitive Recursivef is defined from g & h by primitive recursion if• f(x,y) = h(x,y-1,f(x,y-1))• f(x,0) = g(x)
Does not change. Could be x1,x2,..,xn.
Smaller instanceHelp from friend
(very specific)
Base caseMy solution
Primitive Recursivef is defined from g & h by primitive recursion if• f(x,y) = h(x,y-1,f(x,y-1))• f(x,0) = g(x)
f(x,y) could be computed iteratively:yˈ = 0f = g(x)loop Loop Invariant: exit when yˈ=y
yˈ = yˈ+1fˈ = h(x,yˈ-1,f’)end loopreturn(fˈ)
fˈ= f(x,yˈ)
Primitive Recursivef is defined from g & h by primitive recursion if• f(x,y) = h(x,y-1,f(x,y-1))• f(x,0) = g(x)
f is defined from g & h by composition if• f(x,y) = g(h1(x,y),…,hn(x,y))
Initial functions:• Zero(x) = 0• Successor(x) = x+1• In,i(x1,x2,..,xn ) = xi
Ex• If you have defined g(y,x)
and want f(x,y) = g(y,x)• f(x,y) = g(I2,2(x,y),I2,1(x,y))
Primitive Recursivef is defined from g & h by primitive recursion if• f(x,y) = h(x,y-1,f(x,y-1))• f(x,0) = g(x)
f is defined from g & h by composition if• f(x,y) = g(h1(x,y),…,hn(x,y))
No really dude.If it is not on this page,
you can’t do it.
And you want me to be able to compute everything?
Initial functions:• Zero(x) = 0• Successor(x) = x+1• In,i(x1,x2,..,xn ) = xi
Primitive Recursivef is defined from g & h by primitive recursion if• f(x,y) = h(x,y-1,f(x,y-1))• f(x,0) = g(x)
f is defined from g & h by composition if• f(x,y) = g(h1(x,y),…,hn(x,y))
There is a beauty to the simplicity of these programs.Each routine f() is defined
by specifyingInitial functions:• Zero(x) = 0• Successor(x) = x+1• In,i(x1,x2,..,xn ) = xi
• not lots of fancy code, but only• the number of arguments
f(x1,x2,..,xn,y) • and which functions
g & h to call.
Primitive Recursivef is defined from g & h by primitive recursion if• f(x,y) = h(x,y-1,f(x,y-1))• f(x,0) = g(x)
Examples:
x+y = [x+(y-1)] Get help from your friend.My solution from his.
+1x+0 = x
This is not formal enough.
Ok.
Primitive Recursivef is defined from g & h by primitive recursion if• f(x,y) = h(x,y-1,f(x,y-1))• f(x,0) = g(x)
Examples:
Sum(x,y) = h( x,y-1,Sum(x,y-1) )
Sum(x,0) = g(x)• We define a new primitive recursive function Sum• from a previously define h & g.• h has arguments:
• All the other variables x left unchanged.• y one smaller.• Recursive solution on input with y one smaller.
Primitive Recursivef is defined from g & h by primitive recursion if• f(x,y) = h(x,y-1,f(x,y-1))• f(x,0) = g(x)
Examples:
Sum(x,y) = h( x,y-1,Sum(x,y-1) )
Sum(x,0) = g(x)• We define a new primitive recursive function Sum• from a previously define ones h & g.• When y is zero, Sum is defined from
• g on all the other variables x left unchanged.
Primitive Recursivef is defined from g & h by primitive recursion if• f(x,y) = h(x,y-1,f(x,y-1))• f(x,0) = g(x)
Examples:
Sum(x,y) = h( x,y-1,Sum(x,y-1) ) = Successor( I3,3( x,y-1,Sum(x,y-1)) ) = Sum(x,y-1) + 1
• We want to use the recursive solution• But it must be extracted from the tuple• using the I function.• Then we add one to this recursive solution.
Primitive Recursivef is defined from g & h by primitive recursion if• f(x,y) = h(x,y-1,f(x,y-1))• f(x,0) = g(x)
Examples:
Sum(x,y) = h( x,y-1,Sum(x,y-1) ) = Successor( I3,3( x,y-1,Sum(x,y-1)) ) = Sum(x,y-1) + 1Sum(x,0) = g(x) = I1,1(x)
Primitive Recursivef is defined from g & h by primitive recursion if• f(x,y) = h(x,y-1,f(x,y-1))• f(x,0) = g(x)
Examples:
x×y = [x×(y-1)] Get help from your friend.My solution from his.
+xx×0 = 0
Primitive Recursivef is defined from g & h by primitive recursion if• f(x,y) = h(x,y-1,f(x,y-1))• f(x,0) = g(x)
Examples:
Prod(x,y) = h( x,y-1,Prod(x,y-1) ) = Sum( I3,3( x,y-1,Prod(x,y-1)), I3,1( x,y-1,Prod(x,y-1)) ) = Sum( Prod(x,y-1), x )
• Now we define Prod from previously defined Sum.• Sum has two arguments
• Prod(x,y-1) • x
Extracted from tupleExtracted from tuple
Primitive Recursivef is defined from g & h by primitive recursion if• f(x,y) = h(x,y-1,f(x,y-1))• f(x,0) = g(x)
Examples:
xy = [xy-1] Get help from your friend.My solution from his.
×xx0 = 1
Primitive Recursivef is defined from g & h by primitive recursion if• f(x,y) = h(x,y-1,f(x,y-1))• f(x,0) = g(x)
Examples:
log(y/2) =
log(16) = 24 = 164 because
log(20) = first power of 2 below it is 16.4 because
20/2=10, 10/2=5, 5/2=2, 2/2=1
log(20)=4
log(y) = # of time to divide by 2 until you get 1
one less
Primitive Recursivef is defined from g & h by primitive recursion if• f(x,y) = h(x,y-1,f(x,y-1))• f(x,0) = g(x)
Examples:
log(y) = log(y/2) Get help from your friend.My solution from his.
+1
This is true. But is it formally Primitive
Recursive?
Sure!We showed y/2 is prim. rec.So we use it to show log is.
= log( div(y,2) ) +1
Primitive Recursivef is defined from g & h by primitive recursion if• f(x,y) = h(x,y-1,f(x,y-1))• f(x,0) = g(x)
Examples:
log(y) = log(y/2) Get help from your friend.My solution from his.
+1
Oops
= log( div(y,2) ) +1
You must usef(y) = h(y-1,f(y-1))You must give your
friend <y-1>.
Primitive Recursivef is defined from g & h by primitive recursion if• f(x,y) = h(x,y-1,f(x,y-1))• f(x,0) = g(x)
Is this legal?f(x,y) = h(x/2,y+5,f(x,y-1))
Not officially.But you can legally defineh’(x,y,c) = h(x/2,y+5,c)
and then definef(x,y) = h’(x,y,f(x,y-1))
Primitive Recursivef is defined from g & h by primitive recursion if• f(x,y) = h(x,y-1,f(x,y-1))• f(x,0) = g(x)
Examples:
Predecessor(y) = Predecessor(y-1) Get help from your friend.
My solution from his.+1
Now a hard one.How do you make something smaller?
This looks fine.
Primitive Recursivef is defined from g & h by primitive recursion if• f(x,y) = h(x,y-1,f(x,y-1))• f(x,0) = g(x)
Examples:
Predecessor(y) Get help from your friend.My solution from his.
+1
This does not make anything smaller, but defers to its friend to makes
something else smaller.What is the first thing to get smaller?
= Predecessor(y-1)
Primitive Recursivef is defined from g & h by primitive recursion if• f(x,y) = h(x,y-1,f(x,y-1))• f(x,0) = g(x)
Examples:
Predecessor(y) Get help from your friend.My solution from his.
+1
But what is the base case?
Predecessor(0) = -1
Bug: Negative values are not defined!
= Predecessor(y-1)
Primitive Recursivef is defined from g & h by primitive recursion if• f(x,y) = h(x,y-1,f(x,y-1))• f(x,0) = g(x)
Examples:
Predecessor(y) Get help from your friend.My solution from his.
+1
Ok, as a special case, we will set it to zero.
Predecessor(0) = -10
Predecessor(2) = 1,Predecessor(1) = 0, Predecessor(0) =
= Predecessor(y-1)
0.
But what is the base case?
Primitive Recursivef is defined from g & h by primitive recursion if• f(x,y) = h(x,y-1,f(x,y-1))• f(x,0) = g(x)
Examples:
Predecessor(y) Get help from your friend.My solution from his.
+1
What about y=1?
We need more cases.
Predecessor(1) = Predecessor(0)+1 = 0 +1 0
= Predecessor(y-1)
Predecessor(0) = -10
Primitive Recursivef is defined from g & h by primitive recursion if• f(x,y) = h(x,y-1,f(x,y-1))• f(x,0) = g(x)
Examples:
Predecessor(y)
Oh dear! I don’t know.It seems impossible.
Predecessor(0) = 0
Pred+1 if y > 1= h(y-1,Pred(y-1)) =
0 if y = 1
Bug: “if “ and “equality” are not defined.
Primitive Recursivef is defined from g & h by primitive recursion if• f(x,y) = h(x,y-1,f(x,y-1))• f(x,0) = g(x)
Examples:
h(x,y-1,f(x,y-1))
That is what we want.
h(x,y-1,f) = I3,2(x,y-1,f)
Lets start back at the formal definition
Predecessor(y) =
Predecessor(0) = g(x) = 0
Yay! That does it.
Primitive Recursivef is defined from g & h by primitive recursion if• f(x,y) = h(x,y-1,f(x,y-1))• f(x,0) = g(x)
Examples:
h(x,y-1,f(x,y-1))
That is what we want.
h(x,y-1,f) = I3,2(x,y-1,f)
Lets start back at the formal definition
Predecessor(y) =
Predecessor(0) = g(x) = 0Actually, to avoid negative values,the actual definition is• f(x, Successor(y)) = h(x,y,f(x,y))I changed it because I thought it looked better.
Primitive Recursivef is defined from g & h by primitive recursion if• f(x,y) = h(x,y-1,f(x,y-1))• f(x,0) = g(x)
Examples:
x-y = [x-(y-1)] Get help from your friend.My solution from his.
-1
x-0 = x
3-5 = Pred(3-4) = Pred(Pred(Pred(Pred(Pred(3-0)))))= Pred(Pred(Pred(Pred(Pred(3)))))= Pred(Pred(0))= 0 Fair enough.
Primitive Recursivef is defined from g & h by primitive recursion if• f(x,y) = h(x,y-1,f(x,y-1))• f(x,0) = g(x)
max(x,y) = (x-y)+y
Examples:
(8-5)+5 = (3)+5 = 8
(5-8)+8 = (0)+8 = 8
min(x,y) = x+y-max(x,y)
Primitive Recursivef is defined from g & h by primitive recursion if• f(x,y) = h(x,y-1,f(x,y-1))• f(x,0) = g(x)
Examples:
f(x,0) = g(x) = 0 f(x,y) = h(x,y-1,f(x,y-1)) = 1
Truth(y) = 0 if y=01 else
Here true≥1 and false=0.
Primitive Recursivef is defined from g & h by primitive recursion if• f(x,y) = h(x,y-1,f(x,y-1))• f(x,0) = g(x)
Examples:
Truth(y) = 0 if y=01 else
Here true≥1 and false=0.
Neg(R) = 1-Truth(R)
(R and S) = min(R,S)
(R or S) = max(R,S)
(x>y) = x-y(x=y) = Neg(x>y) and Neg(y>x)
Primitive Recursivef is defined from g & h by primitive recursion if• f(x,y) = h(x,y-1,f(x,y-1))• f(x,0) = g(x)
Examples:
Select(x) = 5 if x = 03 if x = 17 if x = 2 …4 if x = r
= 5×(x=0) + 3×(x=1) + 7×(x=2) + … + 4×(x=r)
3 = Successor(Successor(Successor(Zero)))
mod(y,x) = modx(y) = Remainder(y/x)eg mod(12,5) = 2
This is a hugely important in math & computer science
In field mod 5, the universe has 5 “numbers” {0,1,2,3,4}Think of them as equivalence classes … -8 =mod 5 -3 =mod 5 2 =mod 5 7 …Two operations + and ×
3+4 = 7 =mod 5 2 3×4 = 12 =mod 5 2Additive inverse
-3 = 2 because -3+2 =mod 5 0Multiplicative Inverse
½ = 3 because 2×3=mod 5 1
Primitive Recursive
mod(y,x) = modx(y) = Remainder(y/x)eg mod(8,3) = 2
Counting Mod 3
1 2 0 1 2 0 1 2
SuccModx(r) = 1 if r = 02 if r = 23 if r = 3 …x-1 if r= x-20 if r= x-1
= (r+1)×(r=x)
Primitive Recursive
Primitive Recursivef is defined from g & h by primitive recursion if• f(x,y) = h(x,y-1,f(x,y-1))• f(x,0) = g(x)
Examples:
modx(y) =
Get help from your friend.
My solution from his.modx(y-1)SuccModx( )
SuccModx(r) = 1 if r = 02 if r = 23 if r = 3 …x-1 if r= x-20 if r= x-1
modx(0) = 0
Primitive Recursivef is defined from g & h by primitive recursion if• f(x,y) = h(x,y-1,f(x,y-1))• f(x,0) = g(x)
Examples:
y/x = [(y-1)/x] Get help from your friend.My solution from his.
13/5 = 2 12/5 = 210/5 = 2 9/5 = 1
[(y-1)/x] +1
When does each occur?
Primitive Recursivef is defined from g & h by primitive recursion if• f(x,y) = h(x,y-1,f(x,y-1))• f(x,0) = g(x)
Examples:
y/x = [(y-1)/x]
13/5 = 2 12/5 = 210/5 = 2 9/5 = 1
[(y-1)/x] +1
When does each occur?
0/x = 0
elseif modx(y)=0
mod(y,x) = modx(y) = Remainder(y/x)eg mod(8,3) = 2 8/3 = 2
Counting Mod 3
# people/3 = The number of time go
back to zero.
1 2 0 1 2 0 1 2
1 2
Primitive Recursive
How do I check if 11 is prime?
Make sure r = 2,3,4,…,10 don’t divide into it,
Primitive Recursive
2 3 4 5 6 7 8 9 10
I can do each withmod(11,r) ≠ 0
Lets call such an r good.
How do I know all these n guys are good?
The first n-1 are good.
Primitive Recursive
Get help from your friend.
My solution from his.
2 3 4 5 6 7 8 9 10
I’m good
Ok they must all be good.
Primitive Recursivef is defined from g & h by primitive recursion if• f(x,y) = h(x,y-1,f(x,y-1))• f(x,0) = g(x)
Examples:
AllGood(x,y) = r ≤ y Good(x,r)
= AllGood(x,y-1)
Get help from your friend.My solution from his.
and Good(x,y)
AllGood(x,0) = Good(x,0)
Prime(x) = r ≤ x-1 mod(x,r) ≠0 = AllGood(x,x-1) where Good(x,r) = [mod(x,r) ≠0 or r{0,1}]
or r{0,1}
Primitive Recursivef is defined from g & h by primitive recursion if• f(x,y) = h(x,y-1,f(x,y-1))• f(x,0) = g(x)
Examples:
ExistsGood(x,y) = r ≤ y Good(x,r)
= ExitsGood(x,y-1)
Get help from your friend.My solution from his.
or Good(x,y)
ExistsGood(x,0) = Good(x,0)
PowerOf2(x) = yes for 1,2,4,8,16,32,… = r ≤ x 2r=x = ExistsGood(x,x) where Good(x,r) = [2r=x]
Primitive Recursivef is defined from g & h by primitive recursion if• f(x,y) = h(x,y-1,f(x,y-1))• f(x,0) = g(x)
Examples:
NumGood(x,y) = |{ r≤ y | Good(x,r) }|
= NumGood(x,y-1)
Get help from your friend.My solution from his.
+ Good(x,y)
NumGood(x,0) = Good(x,0)
log(16) = 24 = 164 because
log(20) = first power of 2 below it is 16.4 because
How many powers of 2 are below? |{1,2,4,8,16}=5one more than log(20)
=log(8)+1or becauserecursively
log(2) =1 log(1) =0
Primitive Recursivef is defined from g & h by primitive recursion if• f(x,y) = h(x,y-1,f(x,y-1))• f(x,0) = g(x)
Examples:
NumGood(x,y) = |{ r≤ y | Good(x,r) }|
= NumGood(x,y-1)
Get help from your friend.My solution from his.
+ Good(x,y)
NumGood(x,0) = Good(x,0)
log(x) = = NumGood(x,x) where Good(x,r) = PowerOf2(r) and r≠1
|{r ≤ x | PowerOf2(r) and r≠1 }|
Church’s Thesis Proof
A computational problem is computable
• by a Turing Machine
• by a (simple) Recursive Program
Primitive Recursive Programs seem to be able to compute a lot.
But can they compute everything a TM can?
The input to a TM M is a binary string Istring.The input to a PR P is an integer Iinteger.Istring is the binary representation of Iinteger.
Church’s Thesis Proof
A computational problem is computable
• by a Turing Machine
• by a (simple) Recursive Program
To prove this, we must do a simulation of a TM computation
by a primitive recursive program.2)
TM to Primitive Recursive
• We want to encode this as the configuration of• A recursive program at time t,• i.e. as a tuple <q,tape,head> of whole numbers.
• Consider some computational problem P(I)that is computable by some TM M.• We must construct a recursive program computing P(I).• Consider some input I.• Consider some time step t.• Consider the configuration of M on I at time t.
101 0 0 1 0q
• Let tape =
TM to Primitive Recursive
10100102
be the binary number with the bits on the tape.
Grows this way.
Grows this way.
• Flip around the tape.
Consider some configuration of a TM
101 0 0 1 0q
Consider some configuration of a TM
• Let tape =
• Let head =
• <q,tape,head> specifies the current configuration of the TM.
1 0 10010q
1 0 00000
= tape
= head
TM to Primitive Recursive
01001012
be the binary number with the bits on the tape. 00001002
be the binary number with a one only where the head is.
Consider some configuration <q,tape,head> of a TM
• Let NextQ(q,tape,head) = q’ NextTape(q,tape,head) = tape’ NextHead(q,tape,head) = head’ giving that <q’,tape’,head’> is the next configuration that the TM will be in.
• We need to show that these functions are primitive recursive.
1 0 10010q
1 0 00000
= tape
= head
TM to Primitive Recursive
Consider some configuration <q,tape,head> of a TM
• rightTape = Contents 01001 of tape starting at head =
• Char c under head =
1 0 10010q
1 0 00000
tape/head
= tape
= head
Remainder( rightTape/2 )
c
TM to Primitive Recursive
Consider some configuration <q,tape,head> of a TM
• TM specifies the transition function δ(q,c) = <q’,c’,direction>
• Note that q and c both are from a finite range.• Hence δ can be computed using a primitive recursive select.
1 0 10010q
1 0 00000
= tape
= head
c
Select(x) = 5 if x = 03 if x = 17 if x = 2 …4 if x = r
= 5×(x=0) + 3×(x=1) + 7×(x=2) + … + 4×(x=r)
TM to Primitive Recursive
Consider some configuration <q,tape,head> of a TM
• TM specifies the transition function δ(q,c) = <q’,c’,direction>
• Note that q and c both are from a finite range.• Hence δ can be computed using a primitive recursive select.
1 0 10010q
1 0 00000
= tape
= head
c
• Hence NextQ(q,tape,head) = q’ NextC(q,tape,head) = c’ NextDirection(q,tape,head) = direction are primitive recursive.
TM to Primitive Recursive
Consider some configuration <q,tape,head> of a TM
1 0 10010q
1 0 00000
= tape
= head
TM to Primitive Recursive
• NextTape(q,tape,head) =
tape + headtape - headtape
if c=0 & c’=1if c=1 & c’=0if c=c’
• NextHead(q,tape,head) =head×2
head/2if direction = rightif direction = left
are also primitive recursive
Consider some configuration <q,tape,head> of a TM
• Let Config(x,y) = <q,tape,head> be the configuration that the TM will be in on input x after y time steps.
1 0 10010q
1 0 00000
= tape
= head
TM to Primitive Recursive
Config(x,y) = Get help from your friend.My solution from his.
Config(x,y-1)Next( )
Config(x,0) = <qstart,x,1> is the configuration at time zero.
Note a TM is designed to stay in the same configuration once it reaches a halting state qhalt.
Consider some configuration <q,tape,head> of a TM
• Let Output(x,t) be the output of the TM on input x if it halts within t time steps.
1 0 10010q
1 0 00000
= tape
= head
TM to Primitive Recursive
Note a TM is designed to have the output on the tape when it halts.
Output(x,t) = Tape(Config(x,t)) if TM has halted by time t“Has not halted” else
• We are done because everything is primitive recursive!Careful!
Consider some configuration <q,tape,head> of a TM
1 0 10010q
1 0 00000
= tape
= head
TM to Primitive Recursive
• We need this to be primitive recursive!Is it?
• Let Output(x) = the output of the TM if it halts.∞ else
Consider some configuration <q,tape,head> of a TM
1 0 10010q
1 0 00000
= tape
= head
TM to Primitive Recursive
• Suppose the TM is known to halt in time at most 2n
t ≤ 2x = 2 where n = log2 x = size(x)
• Then Output(x) = Output(x,2x)which is primitive recursive.
Church’s Thesis Proof
A computational problem is computable
• by a Turing Machine
• by a (simple) Recursive Program Primitive
in double exponential time
• Comparing these models:• Is there a danger a primitive recursive program
will run forever on an input?• Is there a danger a TM
will run forever on an input?• Is it possible that they
have the same computing power?
No!
Definitely
No!
Church’s Thesis Proof
A computational problem is computable
• by a Turing Machine
• by a (simple) Recursive Program Primitive
• Comparing these models:• Having a TM run forever is a bad thing.
But being able to compute as long as it needs to is a good thing.
• There are computational problems computable
by TMs but not by primitive recursive programs.
Church’s Thesis Proof
A computational problem is computable
• by a Turing Machine
• by a (simple) Recursive Program Primitive
• Let us bound the running time of a given primitive recursive program.
For this we need Ackermann’s function.
• Comparing these models:
Ackermann’s Function
) )(nA( A(n) A
)( A) (A
kk-k
kk
1
:Step Inductive
00
:Case Base
1
) )))))(A(A(A(A(A( A(n) A
k
kk-k-k-k-k-k 0
that,on induction by Proof
11111
n applications
.each for function different a i.e.
Define
kA
A(k,n)(n) A
k
k
How big is A(5,5)?
n (n) A 2 0
Ackermann’s Function
) )))))(A(A(A(A(A( A(n) A
k
kk-k-k-k-k-k 0
that,on induction by Proof
11111
n applicationsn (n) A 2 0
(n) A 1 )(T 02222 1
n applications
n 2
Ackermann’s Function
) )))))(A(A(A(A(A( A(n) A
k
kk-k-k-k-k-k 0
that,on induction by Proof
11111
n applicationsn (n) A 2 0
n (n) A 2 1
(n) A 2 )(A 02222 1
n applications
n2
Ackermann’s Function
n (n) A 2 2
(n) A 3
(n) A 4
) )))))(A(A(A(A(A( A(n) A
k
kk-k-k-k-k-k 0
that,on induction by Proof
11111
n applicationsn (n) A 2 0
n (n) A 2 1
Ackermann’s Function
y (y) A 2 0
y (y) A 2 1
R0(y) = succ(succ(y))
= 2+2×(n-1)
For which k can Ackermann’s Ak(y) be computed?
R1(y) = R1(y-1)R0( )
Get help from your friend.My solution from his.
Ackermann’s Function
y (y) A 2 0
y (y) A 2 1
y (y) A 2 2
R0(y) = succ(succ(y))
= 2+2×(n-1)
For which k can Ackermann’s Ak(y) be computed?
= 2×2n-1 R1(y) = R1(y-1)R0( )
Get help from your friend.My solution from his.
R2(y) = R2(y-1)R1( )
Ackermann’s Function
y (y) A 2 0
y (y) A 2 1
y (y) A 2 2
(y) A 3
R0(y) = succ(succ(y))
= 2+2×(n-1)
For which k can Ackermann’s Ak(y) be computed?
= 2×2n-1
= 2 -1
R1(y) = R1(y-1)R0( )
Get help from your friend.My solution from his.
R2(y) = R2(y-1)R1( )
R3(y) = R3(y-1)R2( )
Ackermann’s Function
) )))))(A(A(A(A(A( A(y) A
k
kk-k-k-k-k-k 0
that,on induction by Proof
11111
y applications
For which k can Ackermann’s Ak(y) be computed?
][ 011111 ) ))))(A(A(A(A(A A kk-k-k-k-k-
y-1 applications
Get help from your friend.My solution from his.
Rk(y) = Rk(y-1)Rk-1( )
R0(y) = succ(succ(y))
R1(y) = R1(y-1)R0( )R2(y) = R2(y-1)R1( )
R3(y) = R3(y-1)R2( )
Ackermann’s FunctionFor which k can Ackermann’s Ak(y) be computed?
R0(y) = succ(succ(y))
R1(y) = R1(y-1)R0( )R2(y) = R2(y-1)R1( )
Hence (by induction) For each k, Ackermann’s Ak(y) is computed by the primitive recursive program Rk(y)
k, PR Rk, y Rk (y) = Ak(y)
This is a different program for each k!Is there one PR program R(k,y) that works for all k?
PR R, y,k R (k,y) = A(k,y)
Rk(y) = Rk(y-1)Rk-1( )
R3(y) = R3(y-1)R2( )
Sorry no. How does the complexity of Rk increase with k in a waythat cant grow with input size?Applications of Prim Recur
Constant vs FiniteConstant Resources
Fixed at Compile TimeResources
Grow with Input
Java:
TM:
PR:
• # Lines of code• # & Range of Variables • Instruction Set
(Computed in One Step)
• Allocated memory• Running time (Might not Halt)
• # of States (# Bits on Black Board)
• Tape Alphabet• State Transitions
(Computed in One Step)
• # Cells used• Running time (Might not Halt)
• # Lines of code• # of Variables • Function Names
Mult(x,y), Sum(x,y), Succ(y)• Applications of Prim Recur
• Range of Variables• Running time
x+yx+(y-1)
x+(y-2)….
Depth of Rec Tree
x+0
Will HaltTime is
Bounded
Model of Computation: Turing Machine First Order Logic
q q’
k, TM M, x,yk M multiplies xy in one time step.
Yes. We showed you can.The number of states needed grows as a
function of k.
Model of Computation: Turing Machine First Order Logic
q q’
No, it cannot remember arbitraryintegers in its states.
The number of states needed grows as a function of x & y.
TM M, x,y M multiplies xy in one time step.
k, PR Rk, n Rk (n) = Ackk(n)
Yes. We showed you can.It needs k applications of primitive recursion.
First Order Logicf is defined from g & h by primitive recursion if• f(x,y) = h(x,y-1,f(x,y-1))• f(x,0) = g(x)
No, it cannot have a growing number of applications of primitive recursion.
PR R, k,n R(k,n) = Ack(k,n)
First Order Logicf is defined from g & h by primitive recursion if• f(x,y) = h(x,y-1,f(x,y-1))• f(x,0) = g(x)
Ackermann’s Function• Claim: For each k, Ak(y) can be computed
with primitive recursive Rk(y) with k applications of primitive recursion.
• Proof: • True for k = 0, because R0(y) = y+2• Assume by way of induction that it is true for k-1.
Get help from your friend.My solution from his.
Rk(y) = Rk(y-1)Rk-1( ).
Ackermann’s Function• Claim: No primitive recursive
with k applications of primitive recursion grows as fast as than Ak+1(y).
• Proof: By seeing that this is tight.
To design the fastest growing Fastk(y) with k applications of primitive recursion:• First let Fastk-1(y) be the fastest growing
with k-1 applications of primitive recursion:• By induction assume Fastk-1(y)≈Ak-1(y).
• Make Fastk(y) as big as you can.
Get help from your friend.My solution from his.
Fastk(y) = Fastk(y-1)Fastk-1( Fastk-1( Fastk-1( Gives Ak(y).Gives Ak(3y) << Ak+1(y)
Ackermann’s Function
• Claim: A(k,n) is not primitive recursive.• Proof: Suppose it was by program R(k,n).• Let kR denote the number of applications of
primitive recursion in R(k,n).• For k>kR, A(k,n) grows much faster than R(k,n).• Contradiction.
k, PR Rk, n Rk (n) = Ackk(n)
Yes. We showed you can.It needs k applications of primitive recursion.
First Order Logicf is defined from g & h by primitive recursion if• f(x,y) = h(x,y-1,f(x,y-1))• f(x,0) = g(x)
No, it cannot have a growing number of applications of primitive recursion.
PR R, k,n R(k,n) = Ack(k,n)
First Order Logicf is defined from g & h by primitive recursion if• f(x,y) = h(x,y-1,f(x,y-1))• f(x,0) = g(x)
Church’s Thesis Proof
A computational problem is computable
• by a Turing Machine
• by a (simple) Recursive Program Primitive
• There are computational problems computable by TMs but not by primitive recursive programs.
• We need to give primitive recursive programs more power. • For one, it needs to be given the possibility of
running as long as it wants to. (at the danger of running forever.)
RecursiveLet μ denote the least number operator. If g is recursive then so is f(x) = μy [g(x,y)] computed as follows.
procedure f(x) for y = 0..∞
if g(x,y) = 0 return(y)
Note if • "y g(x,y) ≠ 0 • or first a y is tried for which g(x,y) does not halt.
(denoted g(x,y) = ∞)Then this code does not halt (denoted f(x) = ∞)
RecursiveLet μ denote the least number operator. If g is recursive then so is f(x) = μy [g(x,y)] computed as follows.
procedure f(x) for y = 0..∞
if g(x,y) = 0 return(y)
=
the least number y such that • g(x,y) = 0• "y’<y g(x,y’) ≠ ∞
∞ if no such y exists
Consider some configuration <q,tape,head> of a TM
• Recall Config(x,t) = <q,tape,head> is the configuration that the TM will be in on input x after t time steps.
• Let HaltTime(x) = t be the time at which the TM will halt.
1 0 10010q
1 0 00000
= tape
= head
TM to Recursive
HaltTime(x) = HasHalted( )
0 iff this is a halting configuration.HasHalted(<q,tape,head>) =
μt Config(x,t)
Consider some configuration <q,tape,head> of a TM
• Recall Config(x,t) = <q,tape,head> is the configuration that the TM will be in on input x after t time steps.
• Let HaltTime(x) = t be the time at which the TM will halt.• Recall Output(x,t) is the output of the TM on input x
if it halts after t time steps.and Output(x) is the output of the TM on input x.
1 0 10010q
1 0 00000
= tape
= head
TM to Recursive
Output(x) = Output(x,HaltTime(x))
Proving that this is Recursive!
Church’s Thesis Proof
A computational problem is computable
• by a Turing Machine
• by a (simple) Recursive Program
• by a (simple) Register Machine
Done
Define and prove this now.
Hilbert’s 10th problem was is there an algorithm to find an integer solution to a set (or even single) polynomial equation.Register Machines were designed to perhaps be better than Turing Machines at this task.
Register Machine Model
Contents of Memory Cell
Number of Memory Cells
Turing Machines constant size(0/1)
arbitrarily many
Register Machines arbitrarilylarge integer
constant number(10)
Fixed size as input grows
$ c, " I,
Size grows with input" I, $ c,
Register Machine Model
Model of Computation: Register Machine
• The model has a constant number of registers that can take on any integer ≥ 0.
• A program is specified by code made from the following commands:• Ri = 0• Ri = Ri+1• if Ri=Rj goto line k
Surely this is not enough!How do you decrement?
Copy: Ri = Rj
Ri
1. Ri = 02. if Ri=Rj goto line 53. Ri = Ri+14. if Ri=Ri goto line 25. done
Rj
Register Machine Model
Add: Ri = Rj + Rk
Ri
1. Ri = Rj
2. Rk’ = 03. if Rk’=Rk goto line 74. Ri = Ri+15. Rk’ = Rk’+16. goto line 37. done
RkRj Rk’
Register Machine Model
Subtract: Ri = Rj - Rk
Ri
1. Ri = 02. Rj’ = Rk
3. if Rj’=Rj goto line 74. Ri = Ri+15. Rj’ = Rj’+16. goto line 37. done
RjRkRj’
Register Machine Model
Subroutine Call: Ri = subtract(x,y)
1. Ri = 02. Rj’ = Rk
3. if Rj’=Rj goto line 74. Ri = Ri+15. Rj’ = Rj’+16. goto line 37. done
Register Machine Modelsubtract(Rj,Rk)
Solutions:1. Copy the code in every
where you use it.2. Ri = 0• Rj = 0• if Ri=Rj goto line k
3. But how do you where to return to?•Before calling save a pointer where to return to 10. Rreturn = 5 11. goto line 1
8. Rreturn = 5 goto line 12
Recursive to Register Machine
• Zero(x) = 0• Successor(x) = x+1• In,i(x1,x2,..,xn ) = xi
• f(x,y) = g(h1(x,y),…,hn(x,y))
Recursive Register Machine
Ri = 0Ri = Ri+1Rk = Ri
Use results as inputs
Primitive recursion• f(x,y) = h(x,y-1,f(x,y-1))• f(x,0) = g(x)
f = g(x)for y’=1..y
f = h(x,y’-1,f)return(f)
Least Number Operator f(x) = μy [g(x,y)]
for y = 0..∞if g(x,y) = 0 return(y)
Church’s Thesis Proof
A computational problem is computable
• by a Turing Machine
• by a (simple) Recursive Program
• by a (simple) Register Machine Done
Church’s Thesis Proof
A computational problem is computable• by a Java Program
• by a Turing Machine
• by an And/Or/Not Circuit
x3x2x1
OR
ORANDAND
OR
NOT
• A circuit is a directed acyclic graph of and/or/not gates
0 1 0
• An input X assigns a bit to each incoming wire.
0 0
0
0 0
1
And/Or/Not Circuits
The bits percolate down to the output wires.
• Key differences in this model.A given TM • can take an input with
an arbitrarily large number of bits.
And/Or/Not Circuits
A given circuit• has a fixed number n of
input wires an hence can only take inputs with this number of bits.
• Key differences in this model.A given TM • has a fixed finite
description
And/Or/Not Circuits
A given circuit• For each n, the circuit Cn
has a finite description, but in order handle inputs of all size, we need a different one for each n, C1, C2, C3, ….
• Key differences in this model.A given TM • has a fixed finite
description
And/Or/Not Circuits
A given circuit• Allowing it to have
an infinite description C1, C2, C3, …. is not fair.
• To make it fair, each Cn
should be constructed in a Uniform way.
• Key differences in this model.A given TM • the contents of the tape
and location of the head depend on the input (and on the time step) and hence are not known at “compile time”.
And/Or/Not Circuits
A given circuit• the 0/1 value on
each wire (input & internal) depend on the input (but not on time)
• Key differences in this model.A given TM • the contents of the tape
and location of the head depend on the input (and on the time step) and hence are not known at “compile time”.
And/Or/Not Circuits
A given circuit• the gates and
their connections must be fixed at “compile time”
• Clearly circuits compute.• Any function f(X) of n bits can be computed
with a nonuniform circuit of size O(2n).
And/Or/Not Circuits
X f(X)
000000 0
000001 1
000010 0
000011 0
000100 1
2n
f(x)
n
= Cn
And/Or/Not Circuits
X f(X)
000000 0
000001 1
000010 0
000011 0
000100 1
2n
n
Outputs 1iff
X = 000001
¬x1¬x2 ¬x3 …xn
AND
¬x1 ¬x2 ¬x3 x4 … ¬xn
AND
Outputs 1iff
X = 000100
x1 x2 ¬x3 … ¬xn
AND
Outputs 1iff
X = 110..0
…
Repeat this for every value of X
for which f(X)=1.
And/Or/Not Circuits¬x1¬x2 ¬x3 …xn
AND
¬x1 ¬x2 ¬x3 x4 … ¬xn
AND
x1 x2 ¬x3 … ¬xn
AND…
Outputs 1 iff f(X)=1.
OR
Computable
History of Computability
• Which computational problemsare computable by such circuits?
• Unreasonable because for each n,a separate circuit C1, C2, C3, …. of size 2n needs to be defined.
• This is big enough to list all the answers. This “algorithm” does not have finite size.
• An algorithm that needs something different specified for each n is called “nonuniform”.
Church’s Thesis Proof
A computational problem is computable• by a Java Program
• by a Turing Machine
• by an And/Or/Not Circuit
NonUniform & arbitrarily large input unreasonableNonUniform& fixed input size reasonableUniform & arbitrarily large input reasonable
If a TM can compute it in time T(n), then it can be computed by
a uniform circuit of size O(T2(n)).
TM to CircuitsGiven a TM M,
Need a separate circuit C1, C2, C3, ….
for each input size n.
f(x)
whose running time on inputs of size n is at most T(n)..
T(n)
T(n)
n
A given circuit• has a fixed number n of
input wires an hence can only take inputs with this number of bits.
TM to CircuitsGiven a TM M,
All dependent on the input.f(x)
whose running time on inputs of size n is at most T(n)..
T(n)
T(n)
n
• the 0/1 value on each wire at the tth level encodes the tape and head position of the TM at time t.
t
TM to CircuitsASCII-celled TM at time t
OLLEH
Encoded by wires at layer t+1 of the circuits.O
00110100H
01001000E
01000011A
01000000L
01001101q’
Encoded by wires at layer t of the circuits.
O00110100
H01001000
E01000011
L01001101
L01001101
q
ASCII-celled TM at time t+1
OLAEH
TM to Circuits
Put gates between these layers to compute t+1 from t.
O00110100
H01001000
E01000011
A01000000
L01001101
q’
O00110100
H01001000
E01000011
L01001101
L01001101
q
TM to Circuits
O00110100
H01001000
E01000011
A01000000
L01001101
q’
O00110100
H01001000
E01000011
L01001101
L01001101
q
These bits(i.e. if head is here and contents of this cell)
influence which bits?
TM to Circuits
O00110100
H01001000
E01000011
A01000000
L01001101
q’
O00110100
H01001000
E01000011
L01001101
L01001101
q
These bits
Move head rightand change state
Write Character
or move leftand change state
leave putand change state
TM to Circuits
O00110100
H01001000
E01000011
A01000000
L01001101
q’
O00110100
H01001000
E01000011
L01001101
L01001101
q
Conversely, these output bits depend on which input bits?
TM to Circuits
O00110100
H01001000
E01000011
A01000000
L01001101
q’
O00110100
H01001000
E01000011
L01001101
L01001101
q
Some smallcircuit
Computes the entiretransition function
δ(q,c) = <q’,c’,direction>
TM to Circuits
O00110100
H01001000
E01000011
A01000000
L01001101
q’
O00110100
H01001000
E01000011
L01001101
L01001101
q
Some smallcircuit
Hence, the same small circuitgets copied all along.
A given TM • the contents of the tape
and location of the head depend on the input (and on the time step) and hence are not known at “compile time”.
A given circuit• the gates and
their connections must be fixed at “compile time”
TM to Circuits
O00110100
H01001000
E01000011
A01000000
L01001101
q’
O00110100
H01001000
E01000011
L01001101
L01001101
q
Some smallcircuit
Some smallcircuit
Some smallcircuit
Some smallcircuit
Some smallcircuit
TM to Circuits
O00110100
H01001000
E01000011
L01001101
L01001101
qstart
Some smallcircuit
Some smallcircuit
Some smallcircuit
Some smallcircuit
Some smallcircuit
qstart x1 x2 x3 x4 x5 …. Top layer is specified by input X.
TM to Circuits
O00110100
H01001000
E01000011
A01000000
L01001101
qhalt
Some smallcircuit
Some smallcircuit
Some smallcircuit
Some smallcircuit
Some smallcircuit
y1 y2 y3 y4 y5 ….
The bottom layer specifies the output.
O00110100
H01001000
E01000011
L01001101
L01001101
qstart
Some smallcircuit
Some smallcircuit
Some smallcircuit
Some smallcircuit
Some smallcircuit
qstart x1 x2 x3 x4 x5 ….
TM to Circuits• If a TM can compute it in time T(n), • then it uses at most T(n) cells of tape,• then the size of the circuit is O(T(n)×T(n)).• The circuit is constructed in a uniform way
from many copies of the same small circuit.
Church’s Thesis Proof
A computational problem is computable• by a Java Program
• by a Turing Machine
• by an And/Or/Not Circuit
If a TM can compute it in time T(n), then it can be computed by
a uniform circuit of size O(T2(n)).
Done
A Java program• on an input of size n• can produce this
uniform T(n)×T(n) circuit• and evaluate it on the input.
Uniform & arbitrarily large input reasonable
Church’s ThesisA computational problem is computable• by a Java Program
• by a Turing Machine
• by an And/Or/Not CircuitComputer Circuit
• A computer circuit differs from a circuit in two ways:• Memory• Cycles in the graphs of gates
Computer Circuits• Time is kept by a clock.• Value xt may change through time.• But when clock is high, current value
is stored in memory.• When clock is low, it remembers
previous value.• Output of memory is this value stored.
Memory
xt
xt
Clockxtxt-1
Computer Circuits• Time is kept by a clock.• Value xt may change through time.• But when clock is high, current value
is stored in memory.• When clock is low, it remembers
previous value.• Output of memory is this value stored.
Memory
xt
zt
Clockyt
zt = yt
xt if clock=1if clock=0
¬c
AND
OR
c
AND
xt+1
yt
Computer CircuitsASCII-celled TM at time t
OLLEHq
ASCII-celled TM at time t+1
OLAEHq’
Encoded by wires at layer t+1 of the circuits.O
00110100H
01001000E
01000011A
01000000L
01001101q’
Encoded by wires at layer t of the circuits.
O00110100
H01001000
E01000011
L01001101
L01001101
q
Computer Circuits
O00110100
H01001000
E01000011
A01000000
L01001101
q’
O00110100
H01001000
E01000011
L01001101
L01001101
qmemory
Some smallcircuit
Some smallcircuit
Some smallcircuit
Some smallcircuit
Some smallcircuit
Computer Circuits
O00110100
H01001000
E01000011
L01001101
L01001101
qmemory
Some smallcircuit
Some smallcircuit
Some smallcircuit
Some smallcircuit
Some smallcircuit
Church’s ThesisA computational problem is computable• by a Java Program
• by a Turing Machine
• by an And/Or/Not CircuitComputer Circuit Done
Circuit Depth
Circuits Depth• The depth of a circuit:
• is the length of the longest path from an input to an output.
• It indicates evaluation time.• It relates to parallel
computation time.
x3x2x1
OR
ORANDAND
OR
NOT
0 1 0
0
0 0
0 0 1• The Size of a circuit:• is the number of gates.• It relates to sequential
computation time.
x1 ¬x2 x3 …¬xn
AND
Any function f(X) of n bits can be computed with a circuit of size 2n.
¬x1 x2 ¬x3 … xn
AND
x1 x2 ¬x3 … ¬xn
AND
Outputs 1 iff f(X)=1.
…
OR
1
If you allow arbitrary fan.But we assume 2 inputs.
Circuits Depth
x1 ¬x2 x3 …¬xn
AND
Any function f(X) of n bits can be computed with a circuit of size 2n.
¬x1 x2 ¬x3 … xn
AND
x1 x2 ¬x3 … ¬xn
AND
Outputs 1 iff f(X)=1.
…
OR
O(log(n))
O(n)
Circuits Depth
O00110100
H01001000
E01000011
L01001101
L01001101
qstart
Some smallcircuit
Some smallcircuit
Some smallcircuit
Some smallcircuit
Some smallcircuit
qstart x1 x2 x3 x4 x5 ….
• If a TM can compute it in time T(n), • then it uses at most T(n) cells of tape,• then the size of the circuit is O(T(n)×T(n)).• The circuit is constructed in a uniform way
from many copies of the same small circuit. O(T(n))
Circuits Depth
**
*
* **
*
* **
*
* **
*
* **
*
* **
*
* **
*
* **
*
* **
*
* **
*
***
*
+*
*
How to add 2 n-bit numbers.
**
*
*
**
*
*
**
*
*
**
*
*
**
*
*
**
*
*
**
*
*
**
*
*
**
*
*
**
*
+*
*
*
**
*
Takes O(n) time.Even with parallel help.
O(n) circuit depth.
How to add 2 n-bit numbers.
**
*
*
**
*
*
**
*
*
**
*
*
**
*
*
**
*
*
**
*
*
**
*
*
**
*
*
**
*
+*
*
*
**
*
xi = ith input bit of X. yi = ith input bit of Y.
zi = ith output bit of Z.
ci = ith carry bit.
zi = lowBit( ci+xi+yi ) +
xi yi ci
ci+1 zi ci+1 = highBit( ci+xi+yi )
How to add 2 n-bit numbers.
c1 x1 y1
+z1
x0 y0 c0
+z0
c2 x2 y2
+z2
c3 x3 y3
+z3
c4 x4 y4
+z4
c5 x5 y5
+z5
c5 x5 y5
+z6 z7
O(n) circuit depth.
We need to compute the carries ci sooner.
Is O(n) depth intrinsic to adding or can it be done with less depth?
How to add 2 n-bit numbers.
**
**
**
**
**
**
**
**
**
**
**
+ **
Previous alg I was taught Little extra thought.
Size (# of gates)
Depth (path leng)
O(n)
O(n)
O(n2)
O(logn)
O(n)
O(logn)
How to add 2 n-bit numbers.
Trade off between size and depth! Win-Win
**
**
**
**
**
**
**
**
**
**
**
+ **
At level l= 1,2,3,…,logn from the circuit inputs.Break the input bits into n/d blocks of size d = 2l.
n/d = 3 blocks d = 4 bits per blocks
How to add 2 n-bit numbers.
Xd0
Yd0
Xd1
Yd1
Xd2
Yd2
n/d = 3 blocks d = 4 bits per blocks
Note 0 ≤ Xli ≤ 2d-1
Value of 2d gives a carry out of these d bits.Xd
i + Ydi ≥ 2d, then there is definitely a carry
to the next block. We say the block “Generates” a carry. Wire gd
i = true.Xd
i + Ydi ≥ 2d-1, then there is a carry out
only if there is a carry in. We say the block “Propagates” a carry. Wire pd
i = true.
1
How to add 2 n-bit numbers.
**
**
**
**
**
**
**
**
**
**
+
AND
yixi
g1i
1n/1 = 12 blocks d = 1 bits per blocks
Wire g1i = block “Generates” a carry (ie. one out for sure)
xi+yi ≥ 21 = 2
g1i = And( xi, yi )
Wire pdi = block “Propagates” a carry (ie. one out if one in)
xi+yi ≥ 21-1 = 1 pd
i = Or( xi, yi )
1
*1
OR
yixi
p1i
10
How to add 2 n-bit numbers.
n/d = 3 blocks d = 4 bits per blocks
Wire gdi = block “Generates” a carry (ie. one out for sure)
iff first half generates a carry and second half propagates it or second half generates it = (gd/2
2i and pd/22i+1) or (gd/2
2i+1)
Wire pdi = block “Propogates” a carry (ie. one out if one in)
iff first half propagates a carry and second half propagates a carry = (pd/2
2i and pd/22i+1)
1
How to add 2 n-bit numbers. 1
Xd/22i+1
Yd/22i+1
Xd/22i
Yd/22i
1 1
n/d = 3 blocks d = 4 bits per blocks How to add 2 n-bit numbers.
Xd/22i+1
Yd/22i+1
Xd/22i
Yd/22i
gdi pd
i
gd/22i+1 pd/2
2i+1 gd/22i pd/2
2i
gdi+1 p
di+1
gd/22i+3 pd/2
2i+3 gd/22i+2 pd/2
2i+2
Xd/22i+1
Yd/22i+1
Xd/22i
Yd/22i
g2di/2 p
2di/2
Little extra thought.
Size
Depth
n+n/2+n/4+…1 = O(n)
O(logn)
n/d = 3 blocks d = 4 bits per blocks
Suppose you know the carries between blocks of size d.Then there is a carry between these block of size d/2, iff carry at beginning and propagate or generate.
How to add 2 n-bit numbers. *
Xd/22i+1
Yd/22i+1
Xd/22i
Yd/22i
* 1 1
Compute ALL carries between bits.
How to add 2 n-bit numbers.
*
Little extra thought.
Size
Depth
n+n/2+n/4+…1 = O(n)
O(logn)
* * * * * * * * * * * * * * * *
Compute ALL carries between bits.
**
*
*
**
*
*
**
*
*
**
*
*
**
*
*
**
*
*
**
*
*
**
*
*
**
*
*
**
*
+*
*
*
**
*
xi = ith input bit of X. yi = ith input bit of Y.
zi = ith output bit of Z.
ci = ith carry bit.
zi = lowBit( ci+xi+yi )
+
xi yi ci
zi
How to add 2 n-bit numbers.
Little extra thought.
Size
Depth
O(n)O(1)
**
*
*
**
*
*
**
*
*
**
*
*
**
*
*
**
*
*
**
*
*
**
*
*
**
*
*
**
*
+*
*
*
**
*
How to add 2 n-bit numbers.
Previous alg I was taught Little extra thought.
Size (# of gates)
Depth (path leng)
O(n)
O(n)
O(n2)
O(logn)
O(n)
O(logn)
How to multiply 2 n-bit numbers.
* * * * * * * * * * * * * * * *
* * * * * * * * * * * * * * * *
* * * * * * * * * * * * * * * *
* * * * * * * * * * * * * * * *
* * * * * * * * * * * * * * * *
* * * * * * * * * * * * * * * *
x1 x2 x3
xn
Add n 2n-bit
numbers
O(n logn) circuit depth.Can we multiply in O(log2n) depth?
How to multiply 2 n-bit numbers.
x3
+
x1 x2
+
x4
+x5
+x6
+x7
+
Can we multiply in O(logn) depth?
O(logn) circuit depth.
Church’s ThesisA computational problem is computable• by a Java Program
• by a Turing Machine
• by an And/Or/Not CircuitComputer Circuit
DoneCircuit Depth
Arithmetic Circuits
x3x2x1
×
+ × +
/
-
• An arithmetic circuit has +, -, ×, & / gates.
3 5 7
• An input X assigns a real number to each incoming wire.
8 21
-14
168 -12
12
Arithmetic Circuits
The real numbers percolate down to the output wires.
Church’s ThesisA computational problem is computable• by a Java Program
• by a Turing Machine
• by an And/Or/Not CircuitComputer Circuit
Done
Circuit Depth
Arithmetic CircuitsNeural Nets
• Inputs x1, x2 , x3 , …, xn and output y are binary.• Weights w1 , w2 , w3 , …, wn are real numbers
(possibly negative).• y = 1 iff Σi wi×xi ≥ T• The neural net learns by adjusting weights wi.
y
Threshold
T
x1 x2 x3 … xn w1 w2 w3 … wn
Neural Nets
• You can build an AND, OR, and NOT gate from these.• Hence, they are as powerful as circuits.
y
Threshold
T
x1 x2 x3 … xn w1 w2 w3 … wn
Neural Nets
Church’s ThesisA computational problem is computable• by a Java Program
• by a Turing Machine
• by an And/Or/Not CircuitComputer Circuit
Done
Circuit Depth
Arithmetic CircuitsNeural Nets
Church’s ThesisA computational problem is computable• by a Java Program
• by a Turing Machine
• by a (simple) Recursive Program
• by a (simple) Register Machine
• by a Circuit: And/Or/Not, Computer, Arithmetic, & Neural Nets.
Done
Turing 1936
Java ComputableHistory of Computability
TM ComputableAre these definitionsequivalent?
Church says “Yes”All reasonable models
of computation are equivalentas far as what they can compute.
What about time?
Computable
Exp
Poly
Halting
Jack Edmonds 1965
Time: input size running timeFor large n, n100 << 2n
History of Computability
Computable
Exp
Poly
Jack Edmonds 1965
History of Computability
Revised Church says:All reasonable models
of computation are equivalentwithin a polynomial in time
Halting
History of Computability
Revised Church says:All reasonable models
of computation are equivalentwithin a polynomial in time• For any two models of computation M1 and M2,
• there is a constant c,• for any computation problem P,• if P can be solved in time T(n) in M1,• then it can be solved in at most time (nT(n))c in M2.
Note c depends on the models M1 and M2, but not on the problem P.
Church’s ThesisA computational problem is computable• by a Java Program
• by a Turing Machine
• by a (simple) Recursive Program
• by a (simple) Register Machine
• by a Circuit: And/Or/Not, Computer, Arithmetic, & Neural Nets.
• by a Non-Deterministic Machine?
Proof
Reasonable?
Reasonable?
Reasonable?
Reasonable
Yes, polynomially slower
See assignment
See assignmentReasonable
Non-Deterministic MachinesA Non-Deterministic TM/Java/… is • the same as a deterministic one,
except that its moves are not deterministic. • It has a “choice” as to which step to take next.• Deterministic: (qi,c) = <qj,c',right>
• Non-Deterministic: <qi,c; qj,c',right> ϵ • A non-deterministic machine M on input I
is said to accept its input • if there exists an accepting computation.
• Note problems computed need yes/no answers.
Jack Edmonds Steve Cook
Non-Deterministic Machines• If I is a “yes” instance:• A non-deterministic (fairy god mother) could prove
it to you by telling you which moves to make. • This information is said to be a witness.• Given an instance I and a witness/solution S,
there is a poly-time deterministic alg Valid(I,S) to test whether or not S is a valid.
• With this you can convince your non-believing boss that I is a “yes” instance:
• If I is a “no” instance:• There is no witness she could tell you. • You cannot convince your boss.
• An equivalent definition:• Problem P can be computed non-deterministically
if P(I) = S Valid(I,S)
• Example: Satisfiablity:• Instance: A circuit I.• Solution: A satisfying assignment S. • I is a yes instance if there is such an assignment . • Given an instance I and a solution S,
there is a poly-time alg Valid(I,S) to test whether or not S is a valid solution of I.
Non-Deterministic Machines
Circuit I:
0 1 1 0 1 0 1 Assignment S:
1
P(I) = S Valid(I,S)
• Suppose you and your boss only have polynomial time i.e. Time(Valid(I,S)) n1000, where n = |I|. Note |S| n1000. Then the problem P is said to be in Non-Deterministic Polynomial Time (NP).
Non-Deterministic Machines
P(I) = S Valid(I,S)
NPIf P NP Deterministic Time is? Given an input I, try all witnesses S.n1000
2
Exp
Poly
SAT
• Suppose you and your boss only have polynomial time i.e. Time(Valid(I,S)) n1000, where n = |I|. Note |S| n1000. Then the problem P is said to be in Non-Deterministic Polynomial Time (NP).
Non-Deterministic Machines
P(I) = S Valid(I,S)
NP
Exp
Poly
GCD
SAT
If P Polynomial Time, You don’t even need the fairy good mother. P NP
• Suppose you and your boss have TM computable time i.e. Valid(I,S)) is computed by a TM. Note |S| can be arbitrarily long. Then the problem P is said to be in Acceptable
Non-Deterministic Machines
NP
If I don’t have Fairy God mother,why don’t I just try every possible witness S?
Exp
Poly
Computable
Acceptable
GCD
SAT• If I is a “yes” instance
there is a witness alg halts and answers “yes”
• If I is a “no” instance there is no witness alg runs forever
A Problem P is said to be computable/decidable/recursive
• if on every input the machine halts and give the correct answer.
Acceptable
Computable
Non-Deterministic Machines
Acceptable
Computable
Three equivalent definitions for a problem PAcceptable: TM M• On every yes input, the machine halts
and gives the correct answer.• On every no input, the machine
could halt and gives the correct answer or could run for ever.
• Witnesses: computable Valid such that P(I) = S Valid(I,S)• Enumerable: TM M• Every yes input I, is eventually printed.• No no input is ever printed.
Non-Deterministic Machines
Acceptable
ComputableIs there a languagethat can be acceptedbut not computed?
Turing proves uncomputable.Alg: Run M on I. If it halts halt and say “yes” If it does not halt run forever.
Halting(M,I) = Does TM M halt on input I.
Halting
Non-Deterministic Machines
The Post Correspondence Problem (PCP)• The input is a finite collection of dominoes
• A solution is a finite sequence of the dominoes bca
aab
caa
abcc
aab
• So that the combined string on the top is the same as that on the bottom
a b c a a a b ca b c a a a b c
bca ,
aab ,
caa ,
abccI =
Non-Deterministic Machines
I can give a solution as a
witness.
bca
aab
caa
abcc
aab
Non-Deterministic Machines
I can give a solution as a
witness.
But how big is this witness?
NP
Exp
Poly
Computable
Acceptable
• If dominos can’t be repeated, |S| |I|
• If dominos can be repeated, |S| can be arbitrarily long.
PCP?
PCP?
Without Repeats
WithRepeats
CoNPnot SAT
Computable
Exp
Poly
NP
GCD
SAT
HaltingAcceptable CoAcceptable
Not Halting
Non-DeterministicYes instances I
• have accepting computations
• have witness/solutions.No instances I
• Do not.
Co-Non-DeterministicNo instances I
• have accepting computations
• have witness/solutions.Yes instances I
• Do not.
Non-Deterministic Machines
Computable
Exp
Poly
Known
GCD
NP
complete
NP-Complete Problems
Problem Pnew is NP-Complete
• Pnew not too hard.
• Pnew NP
Pnew
Test in poly-time if a given solution
is valid
Computable
Exp
Poly
Known
GCD
NP
complete
NP-Complete Problems
• Pnew sufficiently hard.
Pnew
Sat
Problem Pnew is NP-Complete
• Pnew not too hard.
• Pnew NP
If being able to solve this problem fast means that you can solve every problem in the class fast.
Church’s ThesisA computational problem is computable• by a Java Program
• by a Turing Machine
• by a (simple) Recursive Program
• by a (simple) Register Machine
• by a Circuit: And/Or/Not, Computer, Arithmetic, & Neural Nets.
• by a Non-Deterministic Machine
• by a Quantum Machine?
Proof
Reasonable?
Reasonable?
Reasonable?
Reasonable
Yes, polynomially slower
See assignment
See assignmentReasonable
Reasonable wrt computing power
Unreasonably fast.
Quantum MachinesWhat about Quantum Machines?• Based on quantum mechanics,
at each point in time a quantum TM is in the super-position of any number of configurations of normal TMs. • In time T, it can only flip T quantum coins so can only be
in the super-position of 2T configurations.• Hence, can be simulated by a normal TM in time 2T×T
time.• Hence, Quantum Machines can’t compute more,
just sometimes faster.• Factoring can be done in poly-time, ie 6=2×3.• It is believed that NP-complete problems still take
exponential time.
Church’s ThesisA computational problem is computable• by a Java Program
• by a Turing Machine
• by a (simple) Recursive Program
• by a (simple) Register Machine
• by a Circuit: And/Or/Not, Computer, Arithmetic, & Neural Nets.
• by a Non-Deterministic Machine
• by a Quantum Machine
Proof
Reasonable?
Reasonable?
Reasonable?
Reasonable
Yes, polynomially slower
See assignment
See assignmentReasonable
Reasonable wrt computing powerUnreasonablely fast.
Reasonable wrt computing powerUnreasonablely fast.
Church’s ThesisA computational problem is computable• by a Java Program
• by a Turing Machine
• by a (simple) Recursive Program
• by a (simple) Register Machine
• by a Circuit: And/Or/Not, Computer, Arithmetic, & Neural Nets.
• by a Non-Deterministic Machine
• by a Quantum Machine
Proof
• by a Context Sensitive Grammar
These are all Automata Machines
computing to accept or reject inputs.
What about Grammars that
generate strings?
Generates Strings:
GrammarsContext Free Grammar:
Generates Language/Computational Problem:Grammar generates string
iff string is in the language iff Computational Problem says Yes.
GrammarsContext Free Grammar:• Terminals: a,b,c,…
characters in the final string being generated.• NonTerminals: A,B,C,…
characters that generate more substrings.• Rules: A aAb
cBccBcc At any time, you can replace A with aAb• Start Symbol: S• Stop when no more nonterminals in your string.• Examples:• English, JAVA, …
GrammarsGrammar for spew & plop an # bn:• Spew: SaSb• Plop: S # S
aaSbbaaaSbbb
aaaaSbbbbaaaaaSbbbbb
aSb
aaaaa#bbbbb
Grammars
Grammar for S {0,1}* an bm c2m+3 dn {0,1}*
Linked because must be the same size n.
Linked because must be
the double the size.
• Step 1: Look for links and concatenations
Not Linked because *
means anything.
Grammars
Extra stuck in
Extra concatenated on
Extra concatenated on
• Step 1: Look for links and concatenations
Grammar for S {0,1}* an bm c2m+3 dn {0,1}*
Grammars
Grammar for S {0,1}* an bm c2m+3 dn {0,1}*
A Q A
• Step 1: Look for links and concatenations• Step 2: Give names to the parts.• Step 3: Build Grammar rules.• Step 4: Recurse.
• Concatenate: S AQA
Grammars
Grammar for A {0,1}i
• Step 1: Look for links and concatenations• Step 2: Give names to the parts.• Step 3: Build Grammar rules.• Step 4: Recurse.
Spew
(empty string)
C
• Spew: A CA• Plop: A ε
A
CCACCCA
CCCCACCCCCA
CA
CCCCC
Grammars
Grammar for C {0,1}
• Step 1: Look for links and concatenations• Step 2: Give names to the parts.• Step 3: Build Grammar rules.
Union
• Union: C 0 | 1
CCCCC1 0 0 1 0
Grammars
Grammar for Q an bm c2m+3 dn
• Which to spew first a&d or b&c
• Step 1: Look for links and concatenations.
Linked because must be the same size n.
Linked because must be
the double the size.
Grammars
Grammar for Q an bm c2m+3 dn
• Step 1: Look for links and concatenations• Step 2: Give names to the parts.• Step 3: Build Grammar rules.• Step 4: Recurse.
T
• Spew: Q aQd• Plop: Q Tccc
Q
aaQddaaaQddd
aaaaQddddaaaaaQddddd
aQd
aaaaaTcccddddd
Grammars
Grammar for T bm c2m
• Step 1: Look for links and concatenations• Step 2: Give names to the parts.• Step 3: Build Grammar rules.
Q
aaQddaaaQddd
aaaaQddddaaaaaQddddd
aQd
aaaaaTcccdddddaaaaabTcccccddddd
aaaaabbTcccccccdddddaaaaabbbTcccccccccdddddaaaaabbbcccccccccddddd
• Spew: T bTcc• Plop: T ε
Grammars
Grammar for S {0,1}* an bm c2m+3 dn {0,1}*
(concatenate)S AQA
AQ
AT
• Step 1: Look for links and concatenations• Step 2: Give names to the parts.• Step 3: Build Grammar rules.
(spew & plop)A CA | ε(union)C 0 | 1(spew & plop)Q aQd | Tccc(spew & plop)T bTcc | ε
Q an bm c2m+3 dn
A {0,1}*
T bm c2m
C {0,1}
S
Grammars
Grammar for S an bm cn dm
• Step 1: Look for links and concatenations
Linked because must be
the same size.
Grammars
Grammar for S an bm cn dm
• Step 1: Look for links and concatenations
Spew
Plop
Not linked
Links over lap and hencecan’t be done by a CFG!
Pumping Lemma (ugly)
Grammars
Grammar for S an bn cn
• Step 1: Look for links and concatenations
Linked because must be
the same size.
Links over lap and hencecan’t be done by a CFG!
Grammars
Grammar for S α1 α2 … αn # α1 α2 … αn
• Step 1: Look for links and concatenations
Linked because must be the same string.
…
Links over lap and hencecan’t be done by a CFG!
Grammars
Grammar for S α1 α2 … αn # αn αn-1 … α1
• Step 1: Look for links and concatenations
Linked because must be the same string.
…
Links do not over lap and hencecan be done by a CFG!
S 0S0 | 1S1 | ε
Grammars
S • begin Alg• x = 5• begin loop• x = ( [ { 3 } ] )
• exit loop• return x• end Alg
Linked
This is what CFG is really good at!
• Step 1: Look for links and concatenations
Machine Code vs JavaMachine Model:• one line of code
Java Model:• fancy data structures• loops• recursions• object oriented
Machine Code vs JavaMachine Model:• one line of code• one line of memory cells.
Java Model:• fancy data structures• loops• recursions• object oriented
Machine Code vs JavaMachine Model:• one line of code• one line of memory cells.
Java Model:• fancy data structures• loops• recursions• object oriented
Proof:• This is what a compiler does.
by Java Programby a Machine Code
If a computational problem is computable
Input: Java Code
Parsing/Compiling
Output: MARIE Machine Codesimulating the Java code.
Challenge: Keep track of three algorithms simultaneously • The compiler• The Java code being compiled• The MARIE code being produced.
Friends - Strong Induction View of Recursion:• The sub-instance given must be
• smaller and • must be an instance to the same problem.
• Combine solution given by friend to construct your own solution for your instance.
• Focus on one step. • Do not talk of their friends friends friends.
• Solve small instances on your own.
Parsing/Compiling
Parsing/Compiling
Algorithm: GetExp( s, i )Input: s is a string of tokens i is a start indexOutput: p is a parsing of the longest valid expression j is the end index
s=6*8+((2+42)*(5+12)+987*7*123+15*54)
Parsing/Compiling
Algorithm: GetTerm( s, i )Input: s is a string of tokens i is a start indexOutput: p is a parsing of the longest valid term j is the end index
s=6*8+((2+42)*(5+12)+987*7*123+15*54)
Parsing/Compiling
Algorithm: GetFact( s, i )Input: s is a string of tokens i is a start indexOutput: p is a parsing of the longest valid factor j is the end index
s=6*8+((2+42)*(5+12)+987*7*123+15*54)
Algorithm: GetExp( m )
MARIE Machine Code thatevaluates an expression and stores its value in memory cell indexed by m.
Output:
Algorithm: GetTerm( m )
MARIE Machine Code thatevaluates a term and stores its value in memory cell indexed by m.
Output:
Algorithm: GetFact( m )MARIE Machine Code thatevaluates a factor and stores its value in memory cell indexed by m.
Next token determines which case the factor is.
Output:
Algorithm: GetFactArray( m )MARIE Machine Code thatevaluates a factor and stores its value in memory cell indexed by m.
Output:
Grammars
A language (a set if strings)• is accepted by a TM
if the TM accepts every string in the language and no other. • is generated by a Grammar
if the grammar generates every string in the language and no other.
The language anbn is generated by the grammar• SaSb• S
The language anbncn is generated by no context free grammars
GrammarsS
B
• Start with the start symbol, S.• You can think of choosing which rule to apply
Non-Deterministically. • If all characters in the string become terminal symbols,
then this string is generated • If the process ends because no rule can be applied
then this process fails to generate a string.• or if it continues forever.
D A
Which strings the does the following grammar generate? • S A
D B
• A a• B bB a bbB
bbbBbbbbB
bbbbbB
bB
This grammar only generates the one string a.
GrammarsContext Free Grammar:• C aBc• Context does not matter.
Context Sensitive Grammar:• dCc aBcc• Context does matter.
acBadCcdaBAbCaBc acBad aBc cdaBAb
acBadCcdaBAbdCa
acBa aBcc daBAb
GrammarsGrammar for anbncn :• Spew equal numbers: SABCS• Allow sorting: BAAB
CAAC CBBC• Turn Cc: S Tc
CTc Tcc
• Turn Bb: Tc Tb
BTb Tbb
• Turn Aa: Tb Ta
ATa Taa
• End: Ta ε• Nothing else can be generated
because, no other way to get rid of all the nonterminals.
SABCABCABCS
AAABBBCCCTc
ABCABCACBS
AAABBBCCCS
AAABBBCCTcc
AAABBBTcccc
AAATbbbbccc
AAABBBTbccc
Taaaabbbccc
aaabbbccc
Church’s Thesis Proof
A computational problem is computable• by a Java Program
• by a Turing Machine
• by a Context Sensitive Grammar
TM to Grammar
• Encode configuration of the TM as a string of terminals and nontermnials.
• Encode tape as string of terminals: 10110100• Mark the beginning and the end of the tape• Encode state as • Encode head location as
Consider some configuration of a TM
101 1 0 1 0q
10Qi11010a nonterminal Qi for each state qi.
nonterms 1011010
TM to Grammar
Qstart1011010Beginning
TM:
TM starting configuration on input 10110110:
101 1 0 1 0qstart
Qstart1011010Grammar:
TM to Grammar
Qstart1011010Beginning
TM:
QacceptGrammar:
We assume that if the input is in the language, then the TM halts in the state qaccept
with the tape empty.
TM accepts with configuration:
qaccept
QacceptEnding
TM to Grammar
Qstart1011010 QacceptBeginning Computation Ending
TM: Once input is given,computation
is deterministic.
TM to Grammar
Qstart1011010
Grammar:
Rule: A 0AA 1A
QacceptBeginning Computation Ending
TM: Deterministic.
S 1011010The grammar mustbe able to generate
each string in the language.
Hence, some of the steps of the derivation must be nondeterministic.
TM to Grammar
Qstart1011010
Grammar:
QacceptBeginning Computation Ending
TM: Deterministic.
S 1011010Nondeterministic
The TM and the Grammar seem to go in opposite directions.
The standard solution is to have the Grammar simulate
the TM backwards.
TM to Grammar
Qstart1011010 QacceptBeginning Computation Ending
TM: Deterministic.
The TM deterministically overwrites the 0/1 with c.
The backwards TMnondeterministically guesses
whether the overwritten value was 0 or 1.
Backwards TM:
Transitions: (qi,0) = <qj,c,right> (qi,1) = <qj,c,right>
We, however, will stick with forward moving TMs.
TM to Grammar
Qstart1011010
Grammar:
QacceptBeginning Computation Ending
TM: Deterministic.
S 1011010Nondeterministic
Remembers the input at the end, so that it can be “outputted”.
Our Grammar:
1011010 Qaccept
Needs to have two copies of the input at the beginning.
1011010 Qstart1011010
TM to Grammar
Qstart1011010
Grammar:
QacceptBeginning Computation Ending
TM: Deterministic.
S 1011010Nondeterministic
Nondeterministically guesses the inputand then simulates the TM
deterministically.
Our Grammar:
1011010 Qaccept1011010 Qstart1011010
TM to Grammar
S• Start:Generates:Our Grammar:
1011010 Qstart1011010Start config
of TMCopy of
input
Nondeterministically
guesses the input.
TM to Grammar
S S’ S’ 0S’0 1S’1 Q’start
S• Start:
Our Grammar:
1011010 Qstart1011010Two copies of the input
1011010 Q’start0101101Reverse input
Generates:
Q’start See assignment.
TM to Grammar
S• Start:
Our Grammar:
1011010 Qstart1011010
Generates:See assignment.
b
• Blanks: Add trailing blanks.
1011010 Qstart1011010bbbb
• Accepting TM Computation:
1011010 Qacceptbbbb
1011010 Qaccept
• Blanks: Remove trailing blanks.b
1011010
• Accepting: Qaccept ε
TM to Grammar
S• Start:
Our Grammar:
1011010 Qstart1011010
Generates:See assignment.
b
• Blanks: Add trailing blanks.
1011010 Qstart1011010bbbb
1011010 ???Q’????
• Rejecting or not halting TM computation:
1011010
The derivation never removes all nonterminals and
hence no string is generated!
TM to Grammar
Transition (qi,1) = <qj,c,right>
TM
101 1 0 1 0qi
10Qi11010Grammar:
TM is in state qi and sees a 1.
c01 1 0 1 0
qj
Transformations
10Qj1101010Qjc101010cQj1010
Rule: Qi1 cQj
TM to Grammar
Transition (qi,1) = <qj,c,left>
TM
101 1 0 1 0qi
10Qi11010Grammar:
TM is in state qi and sees a 1.
c01 1 0 1 0
qj
Transformations
10Qj1101010Qjc10101Qj0c1010
Rule: 0Qi1 Qj0c1Qi1 Qj1c
TM to Grammar
S• Start:
Our Grammar:
1011010 Qstart1011010
Generates:See assignment.
b
• Blanks: Add trailing blanks.
1011010 Qstart1011010bbbb
1011010 Qacceptbbbb
1011010 Qaccept
• Blanks: Remove trailing blanks.b
1011010
• Accepting: Qaccept ε
• Transitions: Qi1 cQj0Qi1 Qj0c1Qi1 Qj1c
Church’s Thesis Proof
A computational problem is computable• by a Java Program
• by a Turing Machine
• by a Context Sensitive Grammardone
A Java program can easily determine if a
string can be generated by a
grammar.
Or can it?
Grammar to Java• Given a string, a JAVA program would have to determine
if it can be parsed by the context sensitive grammar.• It wont know which rules to choose.• But it can try all combinations of rules, ie. all derivations.
Grammar to Java• But how many possible derivations are there
and how long is each?• With a context free grammar, • each rule makes the string longer.• Hence, no derivation can be longer then the length n of
the string.• Hence, there can’t be more than (#rules)n derivations.
• With a context sensitive grammar, • some rules make the string longer and others make it
shorter.• Hence, derivation can be arbitrarily long.• Hence, there are an infinite number of derivations.
Grammar to Java• Given that there an infinite number of derivations,
how can a Java program try them all?• If it follows one path of rules too long it will never get to
try other paths.• It should try all paths of length one, then two, then three, …• Each derivation will eventually be reached.• If there is a derivation of the string, then
the Java program will eventually halt and accept the string.• If there is a not derivation of the string, then
the Java program willrun for ever!
Computable
AcceptableThe set of languages generated by
context sensitive grammarsis equal to the class of languages
accepted by a TM.
Acceptable• if on every yes input • the TM halts and says yes.• the grammar generates the input
• on every no input• the TM runs for ever or says no.• the grammar fails to generates the input
• gets to a nonterminal Qno with no rule• or never stops derivation.
Grammar to Java
Computable
Acceptable
HaltingIs there a
context sensitive grammarthat generates all <P,I>for which P halts on I?
Yes!• Let MU be a “universal” TM
which halts on input <P,I> iff P halts on I.• Let GHalt be the grammar that generates <P,I>
iff MU halts on <P,I>.
Grammar to Java
Church’s Thesis Proof
A computational problem is computable• by a Java Program
• by a Turing Machine
• by a Context Sensitive Grammardone
acceptable
Church’s ThesisA computational problem is computable• by a Java Program
• by a Turing Machine
• by a (simple) Recursive Program
• by a (simple) Register Machine
• by a Circuit: And/Or/Not, Computer, Arithmetic, & Neural Nets.
• by a Quantum Machine
• by a Context Sensitive Grammar
NonUniform unreasonable Uniform reasonableComputes the same,
but faster.
Decide vs Accept
Church’s ThesisA computational problem is computable• by a Java Program
• by a Turing Machine
• by a (simple) Recursive Program
• by a (simple) Register Machine
• by a Circuit: And/Or/Not, Computer, Arithmetic, & Neural Nets.
• by a Quantum Machine
• by a Context Sensitive Grammar • by a Human?
Human
What about the human brain? Can it compute more than a TM?• Science: • The brain is just an elaborate machine (neural net).• Hence can’t do any more than a TM.
• New Age:• Quantum mechanics is magical.• The brain is based on quantum mechanics.• Hence, the brain can do much more than a machine.• But we already showed QM can’t compute more.
• Religion:• The human has a soul.• Hence, the brain can do much more than a machine.
Computable
Exp
Poly
A complexity class is a set of computational problems that have a similar difficulty in computing them.
Design a new class:1. Choose some model
Java or Circuits2. Deterministic or
Nondeterministic3. Limit some resource
Time or Spaceto Log, Poly, Exp, ….
Complexity Classes
NP Co-NP
Computable
Exp
Poly
A complexity class is a set of computational problems that have a similar difficulty in computing them.
Proving C1 C2 is not too hard.Prove C2 can simulate C1.
Proving C1 C2 is hard.Prove P ϵ C2 and P ϵ C1.
Complexity Classes
NP Co-NP
Computable
Exp
Poly
A complexity class is a set of computational problems that have a similar difficulty in computing them.
A problem is complete for a class if being able to solve this problem fast means that you can solve every problem in the class fast.
There are complete problems for every class.
Complexity Classes
complete
NP Co-NP
Computable
Exp
Poly
NP
complete
GCD
SATCoNP
completenot SAT
Halting
Games
CoAcceptableAcceptableNon-Deterministic Machines
Jeff EdmondsYork University COSC 4111
Lecture 2
ComputableHalting
Acceptable, Witness, EnumerableComputableAckermann's TimeDouble Exponential TimeE: Exponential TimeExp TimePSpacePH: Poly-Time HierarchyNP & Co-NP
Complexity ClassesPoly-TimeNC: Poly-log depth CircuitsNC2NL: Non-Det Log SpaceL: Log-SpaceAC0, Thres0, Arith0
: Constant DepthNC0: Constant Fan-out