jeff edmonds york university building a computer from sand layers of abstractionlayers of...
TRANSCRIPT
Jeff EdmondsYork University
Building a ComputerFrom Sand
• Layers of Abstraction• Building an AND Gate• AND/OR/NOT Circuits• Building a CPU• Circuit Depth• Parsing/Compiling• Neural Nets• Quantum Machines• Humans
Please feel free to ask questions!
Please give me feedbackso that I can better serve
you.
Thanks for the feedback that you have given me already.
Layers of Abstraction
Roumani-CSE 4
Roumani-CSE 5
SEMICONDUCTOR
Roumani-CSE 6
Roumani-CSE
Select between two alternatives A and B
7
Roumani-CSE 8
Roumani-CSE
CPU
DRAM
I/O
9
Roumani-CSE
Loader
Linker
Memory Manager
I/O Controller
Process Manager
0x3c0110010x342400280x201002260x000040200x000048200x3c0110010x002908210x8c2a00000x515000060x212900040x292a00280x1540fffa0x3c0110010x342400310x200200040x0000000c0x03e00008
boolean found = false;for (int i = 0; i < 10 && !found; i++){ found = (target == list[i]);}
la $a0, yes addi $s0, $0, 550 add $t0, $0, $0 add $t1, $0, $0lbl: lw $t2, list($t1) beq $t2, $s0, ok addi $t1, $t1, 4 slti $t2, $t1, 40 bne $t2, $0, lbl la $a0, nook: addi $v0, $0, 4 syscall jr $ra
boolean found = list.contains(target);
Select * from list where key = target;
boolean found = service.invoke(target);
10
Roumani-CSE 11
The Computing Universe
Vision | Robotics | AI | HCI | DB | Sim | Bio | DC | QC
Vision | Robotics | AI | HCI | DB | Sim | Bio | DC | QC
Roumani-CSE 12
Areas of Study
Lets do it again more slowly.
AND
yx
z
Building an AND Gate
Electricity can’t jump large gap from Cathode to Plate.It can jump from Cathode to Grid.
And if it starts flying, then it keeps going to the Plate.Large jump happens iff power
to Grid AND to Plate
AND
yx
z
Building an AND Gate
AND
yx
z
Building an AND Gate
Building an AND Gate
Building an AND Gate
Four electrons
in the outer ring.
Fourholes
in the outer ring.
Same as Carbon
Building an AND GateSame as Carbon
Building an AND GateSame as Carbon
Building an AND GateSame as Carbon
Each Carbon/Siliconbonds with four others.
Building an AND GateSame as Carbon
Building an AND GateSame as Carbon
Building an AND Gate
Electricity does not flow through it because all the electrons are happy.
Building an AND Gate
Electricity flows through it because this extra electron
moves.
Dope it with what is one to the right in the periotic table,
i.e. one extra electron.Called N
Building an AND Gate
Electricity flows through it because this extra hole
moves.
Dope it with what is one to the left in the periotic table,
i.e. one extra hole.Called P
Building an AND Gate
Electricity flows from N to Pextra electron to extra hole.
N P
Electricity does not flow from P to N extra hole to extra electron
Building an AND Gate
Once the electricity startsthe electrons keep tunneling through!!!
N P
Building an AND Gate
Tunneling happens iff power across PN
ANDacross PNP
N P
Building an AND Gate
Tunneling happens iff power across PN
ANDacross PNP
AND
yx
z
Building an AND Gate
x3x2x1
OR
ORANDAND
OR
NOT
• A circuit is a directed acyclic graph of and/or/not gates
0 1 0
• An input X assigns a bit to each incoming wire.
0 0
0
0 0
1
The bits percolate down to the output wires.
And/Or/Not Circuits
• Clearly circuits compute.• Any function f(X) of n bits can be computed
with a nonuniform circuit of size O(2n).
And/Or/Not Circuits
X f(X)
000000 0
000001 1
000010 0
000011 0
000100 1
2n
f(x)
n
= Cn
And/Or/Not Circuits
X f(X)
000000 0
000001 1
000010 0
000011 0
000100 1
2n
n
Outputs 1iff
X = 000001
¬x1¬x2 ¬x3 …xn
AND
¬x1 ¬x2 ¬x3 x4 … ¬xn
AND
Outputs 1iff
X = 000100
x1 x2 ¬x3 … ¬xn
AND
Outputs 1iff
X = 110..0
…
Repeat this for every value of X
for which f(X)=1.
And/Or/Not Circuits¬x1¬x2 ¬x3 …xn
AND
¬x1 ¬x2 ¬x3 x4 … ¬xn
AND
x1 x2 ¬x3 … ¬xn
AND…
Outputs 1 iff f(X)=1.
OR
Multiplexor
r bits to be addressed
y1
yr
x1 xlogrlog r bitsforms address x
yx The addressed bit
Multiplexor
And/Or/Not Circuits
¬x1¬x2 ¬x3 …xlogr
AND
¬x1 ¬x2 ¬x3 x4 … ¬xlogr
AND
x1 x2 ¬x3 … ¬xlogr
AND…
Multiplexor outputs addressed bit yx.
OR
Multiplexor
All r value that might get addressed.
AND
y1
AND
yi
AND
yr
Get the bit addressed.
Circuit Size ≈ 2logr = r = size of table.
And/Or/Not Circuits
• Time is kept by a clock.• Value xt may change through time.• But when clock is high, current value
is stored in memory.• When clock is low, it remembers
previous value.• Output of memory is this value stored.
Memory
xt
xt
Clockxtxt-1
And/Or/Not Circuits
• Time is kept by a clock.• Value xt may change through time.• But when clock is high, current value
is stored in memory.• When clock is low, it remembers
previous value.• Output of memory is this value stored.
Memory
xt
zt
Clockyt
zt = yt
xt if clock=1if clock=0
¬c
AND
OR
c
AND
xt+1
yt
And/Or/Not Circuits
Building a CPURAM
Add *B to A
38
10
accumulator
A
State of CPU & Computer at time t.register
B program counter
PC
Building a CPU
10
accumulator
A
State of CPU & Computer at time t.register
B program counter
PCM
ultiplexor
RAM
Add *B to A
38
Add *B to A
Command
Building a CPU
10
accumulator
A
State of CPU & Computer at time t.register
B program counter
PCM
ultiplexor
RAM
Add *B to A
38
Add *B to A
Command
38
*B
Building a CPU
10
accumulator
A
State of CPU & Computer at time t.register
B program counter
PC
Multiplexor
RAM
Add *B to A
38
Add *B to A
Command
38
*B
38
Building a CPU
10
accumulator
A
State of CPU & Computer at time t.register
B program counter
PC
Multiplexor
RAM
Add *B to A
38
Add *B to A
Command
38
*B
3810
Building a CPU
10
accumulator
A
State of CPU & Computer at time t.register
B program counter
PC
RAM
Add *B to A
38
Add *B to A
Command
3810
Add
48New A
Building a CPU
10
accumulator
A
State of CPU & Computer at time t.register
B program counter
PC
RAM
Add *B to A
38
IncNew PC
48New A New B
Save A to *B
Building a CPU
10
accumulator
A
State of CPU & Computer at time t.register
B program counter
PC
RAM
Add *B to A
38
New PC48
New A New B
Save A to *B
Clock48
register
B
Building a CPU
48
accumulator
A
State of CPU & Computer at time t.register
B program counter
PC
RAM
Add *B to A
38
Save A to *B
register
B
Save A to *B
Command
Reverse
Multiplexor
48
Building a CPU
48
accumulator
A
State of CPU & Computer at time t.register
B program counter
PC
RAM
Add *B to A
38
Save A to *B
48
Much slowerClock
Circuits Depth• The depth of a circuit:
• is the length of the longest path from an input to an output.
• It indicates evaluation time.• It relates to parallel
computation time.
x3x2x1
OR
ORANDAND
OR
NOT
0 1 0
0
0 0
0 0 1• The Size of a circuit:• is the number of gates.• It relates to sequential
computation time.
x1 ¬x2 x3 …¬xn
AND
Any function f(X) of n bits can be computed with a circuit of size 2n.
¬x1 x2 ¬x3 … xn
AND
x1 x2 ¬x3 … ¬xn
AND
Outputs 1 iff f(X)=1.
…
OR
1
If you allow arbitrary fan.But we assume 2 inputs.
Circuits Depth
x1 ¬x2 x3 …¬xn
AND
Any function f(X) of n bits can be computed with a circuit of size 2n.
¬x1 x2 ¬x3 … xn
AND
x1 x2 ¬x3 … ¬xn
AND
Outputs 1 iff f(X)=1.
…
OR
O(log(n))
O(n)
Circuits Depth
O00110100
H01001000
E01000011
L01001101
L01001101
qstart
Some smallcircuit
Some smallcircuit
Some smallcircuit
Some smallcircuit
Some smallcircuit
qstart x1 x2 x3 x4 x5 ….
• If a TM can compute it in time T(n), • then it uses at most T(n) cells of tape,• then the size of the circuit is O(T(n)×T(n)).• The circuit is constructed in a uniform way
from many copies of the same small circuit. O(T(n))
Circuits Depth
How to add 2 n-bit numbers.
**
**
**
**
**
**
**
**
**
**
**
+
How to add 2 n-bit numbers.
**
*
**
**
**
**
**
* **
**
**
**
**
+
How to add 2 n-bit numbers.
**
*
**
**
**
**
* **
* **
*
**
**
**
**
+
How to add 2 n-bit numbers.
**
*
**
**
**
* **
* **
*
* **
*
**
**
**
**
+
How to add 2 n-bit numbers.
**
*
**
**
* **
* **
*
* **
*
* **
*
**
**
**
**
+
How to add 2 n-bit numbers.
**
*
* **
*
* **
*
* **
*
* **
*
* **
*
* **
*
* **
*
* **
*
* **
*
***
*
+*
*
**
*
*
**
*
*
**
*
*
**
*
*
**
*
*
**
*
*
**
*
*
**
*
*
**
*
*
**
*
+*
*
*
**
*
Takes O(n) time.Even with parallel help.
O(n) circuit depth.
How to add 2 n-bit numbers.
How to add 2 n-bit numbers.
**
*
*
**
*
*
**
*
*
**
*
*
**
*
*
**
*
*
**
*
*
**
*
*
**
*
*
**
*
+*
*
*
**
*
xi = ith input bit of X. yi = ith input bit of Y.
zi = ith output bit of Z.
ci = ith carry bit.
zi = lowBit( ci+xi+yi ) +
xi yi ci
ci+1 zi ci+1 = highBit( ci+xi+yi )
How to add 2 n-bit numbers.
c1 x1 y1
+z1
x0 y0 c0
+z0
c2 x2 y2
+z2
c3 x3 y3
+z3
c4 x4 y4
+z4
c5 x5 y5
+z5
c5 x5 y5
+z6 z7
O(n) circuit depth.
We need to compute the carries ci sooner.
Is O(n) depth intrinsic to adding or can it be done with less depth?
**
**
**
**
**
**
**
**
**
**
**
+ **
Previous alg I was taught Little extra thought.
Size (# of gates)
Depth (path leng)
O(n)
O(n)
O(n2)
O(logn)
O(n)
O(logn)
How to add 2 n-bit numbers.
Trade off between size and depth! Win-Win
**
**
**
**
**
**
**
**
**
**
**
+ **
At level l= 1,2,3,…,logn from the circuit inputs.Break the input bits into n/d blocks of size d = 2l.
n/d = 3 blocks d = 4 bits per blocks
How to add 2 n-bit numbers.
Xd0
Yd0
Xd1
Yd1
Xd2
Yd2
n/d = 3 blocks d = 4 bits per blocks
Note 0 ≤ Xli ≤ 2d-1
Value of 2d gives a carry out of these d bits.Xd
i + Ydi ≥ 2d, then there is definitely a carry
to the next block. We say the block “Generates” a carry. Wire gd
i = true.Xd
i + Ydi ≥ 2d-1, then there is a carry out
only if there is a carry in. We say the block “Propagates” a carry. Wire pd
i = true.
1
How to add 2 n-bit numbers.
**
**
**
**
**
**
**
**
**
**
+
AND
yixi
g1i
1n/1 = 12 blocks d = 1 bits per blocks
Wire g1i = block “Generates” a carry (ie. one out for sure)
xi+yi ≥ 21 = 2
g1i = And( xi, yi )
Wire pdi = block “Propagates” a carry (ie. one out if one in)
xi+yi ≥ 21-1 = 1 pd
i = Or( xi, yi )
1
*1
OR
yixi
p1i
10
How to add 2 n-bit numbers.
n/d = 3 blocks d = 4 bits per blocks
Wire gdi = block “Generates” a carry (ie. one out for sure)
iff first half generates a carry and second half propagates it or second half generates it = (gd/2
2i and pd/22i+1) or (gd/2
2i+1)
Wire pdi = block “Propogates” a carry (ie. one out if one in)
iff first half propagates a carry and second half propagates a carry = (pd/2
2i and pd/22i+1)
1
How to add 2 n-bit numbers. 1
Xd/22i+1
Yd/22i+1
Xd/22i
Yd/22i
1 1
n/d = 3 blocks d = 4 bits per blocks How to add 2 n-bit numbers.
Xd/22i+1
Yd/22i+1
Xd/22i
Yd/22i
gdi pd
i
gd/22i+1 pd/2
2i+1 gd/22i pd/2
2i
gdi+1 p
di+1
gd/22i+3 pd/2
2i+3 gd/22i+2 pd/2
2i+2
Xd/22i+1
Yd/22i+1
Xd/22i
Yd/22i
g2di/2 p
2di/2
Little extra thought.
Size
Depth
n+n/2+n/4+…1 = O(n)
O(logn)
n/d = 3 blocks d = 4 bits per blocks
Suppose you know the carries between blocks of size d.Then there is a carry between these block of size d/2, iff carry at beginning and propagate or generate.
How to add 2 n-bit numbers. *
Xd/22i+1
Yd/22i+1
Xd/22i
Yd/22i
* 1 1
Compute ALL carries between bits.
How to add 2 n-bit numbers.
*
Little extra thought.
Size
Depth
n+n/2+n/4+…1 = O(n)
O(logn)
* * * * * * * * * * * * * * * *
Compute ALL carries between bits.
**
*
*
**
*
*
**
*
*
**
*
*
**
*
*
**
*
*
**
*
*
**
*
*
**
*
*
**
*
+*
*
*
**
*
xi = ith input bit of X. yi = ith input bit of Y.
zi = ith output bit of Z.
ci = ith carry bit.
zi = lowBit( ci+xi+yi )
+
xi yi ci
zi
How to add 2 n-bit numbers.
Little extra thought.
Size
Depth
O(n)O(1)
**
*
*
**
*
*
**
*
*
**
*
*
**
*
*
**
*
*
**
*
*
**
*
*
**
*
*
**
*
+*
*
*
**
*
How to add 2 n-bit numbers.
Previous alg I was taught Little extra thought.
Size (# of gates)
Depth (path leng)
O(n)
O(n)
O(n2)
O(logn)
O(n)
O(logn)
How to multiply 2 n-bit numbers.
* * * * * * * * * * * * * * * *
* * * * * * * * * * * * * * * *
* * * * * * * * * * * * * * * *
* * * * * * * * * * * * * * * *
* * * * * * * * * * * * * * * *
* * * * * * * * * * * * * * * *
x1 x2 x3
xn
Add n 2n-bit
numbers
O(n logn) circuit depth.
How to multiply 2 n-bit numbers.
x3
+
x1 x2
+
x4
+x5
+x6
+x7
+
O(logn) circuit depth.
Can we multiply in O(log2n) depth?
Can we multiply in O(log2n) depth?
How to multiply 2 n-bit numbers.
Can we multiply in O(logn) depth?
x11 x12 x9 x10 x7 x8 x5 x6 x3 x4 x1 x2
++++++
+++
+
6.
How to multiply 2 n-bit numbers.
Output:
Input: s=6*8+((2+42)*(5+12)+987*7*123+15*54)
Parsing/Compiling
Context Free Grammar:
Input: Java Code
Parsing/Compiling
Output: MARIE Machine Codesimulating the Java code.
x3x2x1
×
+ × +
/
-
• An arithmetic circuit has +, -, ×, & / gates.
3 5 7
• An input X assigns a real number to each incoming wire.
8 21
-14
168 -12
12
Arithmetic Circuits
The real numbers percolate down to the output wires.
Neural Nets
• Inputs x1, x2 , x3 , …, xn and output y are binary.• Weights w1 , w2 , w3 , …, wn are real numbers
(possibly negative).• y = 1 iff Σi wi×xi ≥ T• The neural net learns by adjusting weights wi.
y
Threshold
T
x1 x2 x3 … xn w1 w2 w3 … wn
Neural Nets
Quantum MachinesWhat about Quantum Machines?• Based on quantum mechanics,
at each point in time a quantum TM is in the super-position of any number of configurations of normal TMs. • In time T, it can only flip T quantum coins so can only be
in the super-position of 2T configurations.• Hence, can be simulated by a normal TM in time 2T×T
time.• Hence, Quantum Machines can’t compute more,
just sometimes faster.• Factoring can be done in poly-time, ie 6=2×3.• It is believed that NP-complete problems still take
exponential time.
Human
What about the human brain? Can it compute more than a TM?• Science: • The brain is just an elaborate machine (neural net).• Hence can’t do any more than a TM.
• New Age:• Quantum mechanics is magical.• The brain is based on quantum mechanics.• Hence, the brain can do much more than a machine.• But we already showed QM can’t compute more.
• Religion:• The human has a soul.• Hence, the brain can do much more than a machine.
End