ist 4 information and logic - brain.caltech.edubrain.caltech.edu/ist4/lectures/lect 1715 p.pdf ·...
TRANSCRIPT
![Page 1: IST 4 Information and Logic - brain.caltech.edubrain.caltech.edu/ist4/lectures/lect 1715 p.pdf · AON: AND, OR, Not LT: ... A Logical Calculus of Ideas Immanent in Nervous Activity](https://reader034.vdocuments.mx/reader034/viewer/2022051722/5a9dd63d7f8b9ae0108c4872/html5/thumbnails/1.jpg)
IST 4Information and Logic
![Page 2: IST 4 Information and Logic - brain.caltech.edubrain.caltech.edu/ist4/lectures/lect 1715 p.pdf · AON: AND, OR, Not LT: ... A Logical Calculus of Ideas Immanent in Nervous Activity](https://reader034.vdocuments.mx/reader034/viewer/2022051722/5a9dd63d7f8b9ae0108c4872/html5/thumbnails/2.jpg)
mon tue wed thr fri30 M1 1= todayT
6 M1
13 1 2M2x= hw#x out
oh oh
oh oh
= todayT
M
20 227 M2
x= hw#x due
idt
oh oh
h h Students’ MQ27 M2
4 311 3 4Mx= MQx out
midtermsoh
oh oh
oh = office hours oh Students MQpresentations
11 3 418 4 525
Mx= MQx out
Mx= MQx dueoh oh
ohoh
25
1 5oh
oh
oh
oh
T
![Page 3: IST 4 Information and Logic - brain.caltech.edubrain.caltech.edu/ist4/lectures/lect 1715 p.pdf · AON: AND, OR, Not LT: ... A Logical Calculus of Ideas Immanent in Nervous Activity](https://reader034.vdocuments.mx/reader034/viewer/2022051722/5a9dd63d7f8b9ae0108c4872/html5/thumbnails/3.jpg)
MQsMQs1. Everyone has a gift! (Tuesday)
2. Memory (Thursday)
![Page 4: IST 4 Information and Logic - brain.caltech.edubrain.caltech.edu/ist4/lectures/lect 1715 p.pdf · AON: AND, OR, Not LT: ... A Logical Calculus of Ideas Immanent in Nervous Activity](https://reader034.vdocuments.mx/reader034/viewer/2022051722/5a9dd63d7f8b9ae0108c4872/html5/thumbnails/4.jpg)
Tuesday 6/2 2:30pm –
1. Christopher Haack: The gift of resilience
Tuesday, 6/2, 2:30pm
p g
2. Joon Lee: Settling is not an option
3 Spencer Strumwasser: The gift of dyslexia 3. Spencer Strumwasser: The gift of dyslexia
4. Richard Zhu: The gift of memory
5 A h i H i Th ift f i l iti5. Ashwin Hari: The gift of musical composition
6. Jessica Nassimi: Evolution—a gift in disguise
7. Serena Delgadillo: The gift of self-expression
8. Megan Keehan: Gift of motherliness
9. Zane Murphy: Grandmother and the piano
![Page 5: IST 4 Information and Logic - brain.caltech.edubrain.caltech.edu/ist4/lectures/lect 1715 p.pdf · AON: AND, OR, Not LT: ... A Logical Calculus of Ideas Immanent in Nervous Activity](https://reader034.vdocuments.mx/reader034/viewer/2022051722/5a9dd63d7f8b9ae0108c4872/html5/thumbnails/5.jpg)
Thursday 6/4 2:30pm –Thursday, 6/4, 2:30pm
1. Connor Lee: Memory is a fickle thing – blessing or curse
2. Pallavi Aggarwal: The wonders of human memory
3. Peter Kundzicz and Anshul Ramachandran: Muscle memories
4. Siva Gangavarapu: A cultural retrospection. g p p
5. Philip Liu: The light of other days
6 Jason Simon: Math and Broadway6. Jason Simon: Math and Broadway
7. Yujie Xu: Memory v.s. ESL
8 C l Zh Wh 8. Celia Zhang: When memory sours
![Page 6: IST 4 Information and Logic - brain.caltech.edubrain.caltech.edu/ist4/lectures/lect 1715 p.pdf · AON: AND, OR, Not LT: ... A Logical Calculus of Ideas Immanent in Nervous Activity](https://reader034.vdocuments.mx/reader034/viewer/2022051722/5a9dd63d7f8b9ae0108c4872/html5/thumbnails/6.jpg)
Last LectureGates and circuits AON: AND, OR, Not
LT: Linear ThresholdLT: Linear Threshold
>>
abc
ab >>
>>
abc
ab >
>
>
>
bc
abc
abc
>>>>
>>
>>
bc
abc
abc
![Page 7: IST 4 Information and Logic - brain.caltech.edubrain.caltech.edu/ist4/lectures/lect 1715 p.pdf · AON: AND, OR, Not LT: ... A Logical Calculus of Ideas Immanent in Nervous Activity](https://reader034.vdocuments.mx/reader034/viewer/2022051722/5a9dd63d7f8b9ae0108c4872/html5/thumbnails/7.jpg)
Last LectureAON: AND, OR, Not LT: Linear Threshold
General construction for symmetric functions
AON 54
AONLT-l
*
*
2LT-nl *
Exponential gap in sizeWhat are the symmetric functions that can be computed functions that can be computed by a single LT gate? * = it is optimal
![Page 8: IST 4 Information and Logic - brain.caltech.edubrain.caltech.edu/ist4/lectures/lect 1715 p.pdf · AON: AND, OR, Not LT: ... A Logical Calculus of Ideas Immanent in Nervous Activity](https://reader034.vdocuments.mx/reader034/viewer/2022051722/5a9dd63d7f8b9ae0108c4872/html5/thumbnails/8.jpg)
Linear Threshold and SYM
![Page 9: IST 4 Information and Logic - brain.caltech.edubrain.caltech.edu/ist4/lectures/lect 1715 p.pdf · AON: AND, OR, Not LT: ... A Logical Calculus of Ideas Immanent in Nervous Activity](https://reader034.vdocuments.mx/reader034/viewer/2022051722/5a9dd63d7f8b9ae0108c4872/html5/thumbnails/9.jpg)
LT: Linear Threshold
![Page 10: IST 4 Information and Logic - brain.caltech.edubrain.caltech.edu/ist4/lectures/lect 1715 p.pdf · AON: AND, OR, Not LT: ... A Logical Calculus of Ideas Immanent in Nervous Activity](https://reader034.vdocuments.mx/reader034/viewer/2022051722/5a9dd63d7f8b9ae0108c4872/html5/thumbnails/10.jpg)
Symmetric Functions and LT Circuits
Q: How is SYM related to LT1 ??
Q: Which class has more functions?
Definitions:
Q
Definitions:
(1) SYM = the class of Boolean symmetric functions
(2) LT1 = the class of Boolean functions that can be (2) LT1 the class of Boolean functions that can be realized by a single LT gate.
![Page 11: IST 4 Information and Logic - brain.caltech.edubrain.caltech.edu/ist4/lectures/lect 1715 p.pdf · AON: AND, OR, Not LT: ... A Logical Calculus of Ideas Immanent in Nervous Activity](https://reader034.vdocuments.mx/reader034/viewer/2022051722/5a9dd63d7f8b9ae0108c4872/html5/thumbnails/11.jpg)
AND, OR, XOR and MAJ are symmetric functions
Q Whi h t i f ti i LT ?
|X| AND OR XOR MAJ
Q: Which symmetric functions are in LT1?
0 0 0 0 0
1 0 1 1 01 0 1 1 0
2 0 1 0 1
3 1 1 1 1
LT1 LT1 LT1not LT1
LT1 = the class of Boolean functions that can be realized by a single LT gate.
LT1 LT1 LT1not LT1
realized by a single LT gate.
![Page 12: IST 4 Information and Logic - brain.caltech.edubrain.caltech.edu/ist4/lectures/lect 1715 p.pdf · AON: AND, OR, Not LT: ... A Logical Calculus of Ideas Immanent in Nervous Activity](https://reader034.vdocuments.mx/reader034/viewer/2022051722/5a9dd63d7f8b9ae0108c4872/html5/thumbnails/12.jpg)
Definition: A t i B l f ti i i TH if it h t t A symmetric Boolean function is in TH if it has at most a single transition in the symmetric function table
= a transition
|X| AND OR XOR MAJ
0 0 0 0 0
1 0 1 1 0
2 0 1 0 1
3 1 1 1 1
Not in THIn TH
![Page 13: IST 4 Information and Logic - brain.caltech.edubrain.caltech.edu/ist4/lectures/lect 1715 p.pdf · AON: AND, OR, Not LT: ... A Logical Calculus of Ideas Immanent in Nervous Activity](https://reader034.vdocuments.mx/reader034/viewer/2022051722/5a9dd63d7f8b9ae0108c4872/html5/thumbnails/13.jpg)
The Class TH is in LT1
|X| TH0 TH1 TH2 TH3 TH0 TH1 TH2 TH3
0 1 0 0 0 0 1 1 10 1 0 0 0 0 1 1 11 1 1 0 0 0 0 1 1
1 1 1 0 0 0 0 12 1 1 1 0 0 0 0 1
3 1 1 1 1 0 0 0 0
![Page 14: IST 4 Information and Logic - brain.caltech.edubrain.caltech.edu/ist4/lectures/lect 1715 p.pdf · AON: AND, OR, Not LT: ... A Logical Calculus of Ideas Immanent in Nervous Activity](https://reader034.vdocuments.mx/reader034/viewer/2022051722/5a9dd63d7f8b9ae0108c4872/html5/thumbnails/14.jpg)
Q: How is TH related to SYM and LT1 ??
We know that:
Q
We know that:
We Proved that:
SYM LT1TH
![Page 15: IST 4 Information and Logic - brain.caltech.edubrain.caltech.edu/ist4/lectures/lect 1715 p.pdf · AON: AND, OR, Not LT: ... A Logical Calculus of Ideas Immanent in Nervous Activity](https://reader034.vdocuments.mx/reader034/viewer/2022051722/5a9dd63d7f8b9ae0108c4872/html5/thumbnails/15.jpg)
TH is exactly in the intersection of SYM and LT1
Theorem:
Q: What are the 4 functions?
Proof: Not today... you might want to try and prove it...
Q: What are the 4 functions?
SYM LT1TH
???
![Page 16: IST 4 Information and Logic - brain.caltech.edubrain.caltech.edu/ist4/lectures/lect 1715 p.pdf · AON: AND, OR, Not LT: ... A Logical Calculus of Ideas Immanent in Nervous Activity](https://reader034.vdocuments.mx/reader034/viewer/2022051722/5a9dd63d7f8b9ae0108c4872/html5/thumbnails/16.jpg)
LT1 Function that is not SymmetricLT1 Function that is not Symmetric
0 00 1
-1-2
00
11
1 01 1
0-1
10
-1-1
![Page 17: IST 4 Information and Logic - brain.caltech.edubrain.caltech.edu/ist4/lectures/lect 1715 p.pdf · AON: AND, OR, Not LT: ... A Logical Calculus of Ideas Immanent in Nervous Activity](https://reader034.vdocuments.mx/reader034/viewer/2022051722/5a9dd63d7f8b9ae0108c4872/html5/thumbnails/17.jpg)
lLinear Threshold Circuits
for symmetric functionsfor symmetric functions
![Page 18: IST 4 Information and Logic - brain.caltech.edubrain.caltech.edu/ist4/lectures/lect 1715 p.pdf · AON: AND, OR, Not LT: ... A Logical Calculus of Ideas Immanent in Nervous Activity](https://reader034.vdocuments.mx/reader034/viewer/2022051722/5a9dd63d7f8b9ae0108c4872/html5/thumbnails/18.jpg)
5AON 54
AONLT-l
L l 2LT-nl
General construction for symmetric functions
![Page 19: IST 4 Information and Logic - brain.caltech.edubrain.caltech.edu/ist4/lectures/lect 1715 p.pdf · AON: AND, OR, Not LT: ... A Logical Calculus of Ideas Immanent in Nervous Activity](https://reader034.vdocuments.mx/reader034/viewer/2022051722/5a9dd63d7f8b9ae0108c4872/html5/thumbnails/19.jpg)
|X| XOR Q: compute XOR with TH gates?
0 01 1
with TH gates?
2 0
1|X| TH1 TH2 TH1+TH2-1
0 0 1 01 1 1 12 1 0 02 1 0 0
![Page 20: IST 4 Information and Logic - brain.caltech.edubrain.caltech.edu/ist4/lectures/lect 1715 p.pdf · AON: AND, OR, Not LT: ... A Logical Calculus of Ideas Immanent in Nervous Activity](https://reader034.vdocuments.mx/reader034/viewer/2022051722/5a9dd63d7f8b9ae0108c4872/html5/thumbnails/20.jpg)
LT Depth-2 Circuits
TH1
+-1
TH2 |X| TH1 TH2 TH1+TH2-1 |X| TH1 TH2
0 0 1 01 1 1 11 1 1 12 1 0 0
![Page 21: IST 4 Information and Logic - brain.caltech.edubrain.caltech.edu/ist4/lectures/lect 1715 p.pdf · AON: AND, OR, Not LT: ... A Logical Calculus of Ideas Immanent in Nervous Activity](https://reader034.vdocuments.mx/reader034/viewer/2022051722/5a9dd63d7f8b9ae0108c4872/html5/thumbnails/21.jpg)
Generalization
|X| f( )
z
|X| f(x)0 0
1 1
2 1
3 0
4 04 0
![Page 22: IST 4 Information and Logic - brain.caltech.edubrain.caltech.edu/ist4/lectures/lect 1715 p.pdf · AON: AND, OR, Not LT: ... A Logical Calculus of Ideas Immanent in Nervous Activity](https://reader034.vdocuments.mx/reader034/viewer/2022051722/5a9dd63d7f8b9ae0108c4872/html5/thumbnails/22.jpg)
Generalization
|X| f( )
z
|X| f(x)0 0
1 1
2 1
3 0
4 04 0
![Page 23: IST 4 Information and Logic - brain.caltech.edubrain.caltech.edu/ist4/lectures/lect 1715 p.pdf · AON: AND, OR, Not LT: ... A Logical Calculus of Ideas Immanent in Nervous Activity](https://reader034.vdocuments.mx/reader034/viewer/2022051722/5a9dd63d7f8b9ae0108c4872/html5/thumbnails/23.jpg)
Generalization
|X| f( ) TH1
z
|X| f(x) TH1
0 0 0
1 1 1
2 1 1
3 0 1
4 0 14 0 1
![Page 24: IST 4 Information and Logic - brain.caltech.edubrain.caltech.edu/ist4/lectures/lect 1715 p.pdf · AON: AND, OR, Not LT: ... A Logical Calculus of Ideas Immanent in Nervous Activity](https://reader034.vdocuments.mx/reader034/viewer/2022051722/5a9dd63d7f8b9ae0108c4872/html5/thumbnails/24.jpg)
Generalization
|X| f( ) TH1 TH3
z
|X| f(x) TH1 TH3
0 0 0 1
1 1 1 1
2 1 1 1
3 0 1 0
4 0 1 04 0 1 0
![Page 25: IST 4 Information and Logic - brain.caltech.edubrain.caltech.edu/ist4/lectures/lect 1715 p.pdf · AON: AND, OR, Not LT: ... A Logical Calculus of Ideas Immanent in Nervous Activity](https://reader034.vdocuments.mx/reader034/viewer/2022051722/5a9dd63d7f8b9ae0108c4872/html5/thumbnails/25.jpg)
Generalization
|X| f( ) TH1 TH3 Σ 1
z
|X| f(x) TH1 TH3 Σ -10 0 0 1 0
1 1 1 1 1
2 1 1 1 1
3 0 1 0 0
4 0 1 0 04 0 1 0 0
![Page 26: IST 4 Information and Logic - brain.caltech.edubrain.caltech.edu/ist4/lectures/lect 1715 p.pdf · AON: AND, OR, Not LT: ... A Logical Calculus of Ideas Immanent in Nervous Activity](https://reader034.vdocuments.mx/reader034/viewer/2022051722/5a9dd63d7f8b9ae0108c4872/html5/thumbnails/26.jpg)
|X| f( ) TH1 TH3 Σ 1|X| f(x) TH1 TH3 Σ -10 0 0 1 0
1 1 1 1 1
2 1 1 1 1
3 0 1 0 0
4 0 1 0 04 0 1 0 0
![Page 27: IST 4 Information and Logic - brain.caltech.edubrain.caltech.edu/ist4/lectures/lect 1715 p.pdf · AON: AND, OR, Not LT: ... A Logical Calculus of Ideas Immanent in Nervous Activity](https://reader034.vdocuments.mx/reader034/viewer/2022051722/5a9dd63d7f8b9ae0108c4872/html5/thumbnails/27.jpg)
Generalization to EQz EQ
00
00 0
1
1
1
n
1
00
![Page 28: IST 4 Information and Logic - brain.caltech.edubrain.caltech.edu/ist4/lectures/lect 1715 p.pdf · AON: AND, OR, Not LT: ... A Logical Calculus of Ideas Immanent in Nervous Activity](https://reader034.vdocuments.mx/reader034/viewer/2022051722/5a9dd63d7f8b9ae0108c4872/html5/thumbnails/28.jpg)
Generalization to EQz EQ
00 1
1
1
12
0
1
0 1
![Page 29: IST 4 Information and Logic - brain.caltech.edubrain.caltech.edu/ist4/lectures/lect 1715 p.pdf · AON: AND, OR, Not LT: ... A Logical Calculus of Ideas Immanent in Nervous Activity](https://reader034.vdocuments.mx/reader034/viewer/2022051722/5a9dd63d7f8b9ae0108c4872/html5/thumbnails/29.jpg)
Generalization to SYMz M
+-1
Q:h h l fWhat is the generalization to arbitrary symmetric functions?
![Page 30: IST 4 Information and Logic - brain.caltech.edubrain.caltech.edu/ist4/lectures/lect 1715 p.pdf · AON: AND, OR, Not LT: ... A Logical Calculus of Ideas Immanent in Nervous Activity](https://reader034.vdocuments.mx/reader034/viewer/2022051722/5a9dd63d7f8b9ae0108c4872/html5/thumbnails/30.jpg)
Generalization to SYMz M
Q:Q:What is the generalization to arbitrary symmetric functions?
A:A:Consider the symmetric function table, it is a sum of non-overlapping 1-intervals
0
0
1
1 Sum of two TH functions
![Page 31: IST 4 Information and Logic - brain.caltech.edubrain.caltech.edu/ist4/lectures/lect 1715 p.pdf · AON: AND, OR, Not LT: ... A Logical Calculus of Ideas Immanent in Nervous Activity](https://reader034.vdocuments.mx/reader034/viewer/2022051722/5a9dd63d7f8b9ae0108c4872/html5/thumbnails/31.jpg)
Back to XORX
n TH gates for XOR of n variables
0 0
1
2
1
0
3
4
1
0
5 1
![Page 32: IST 4 Information and Logic - brain.caltech.edubrain.caltech.edu/ist4/lectures/lect 1715 p.pdf · AON: AND, OR, Not LT: ... A Logical Calculus of Ideas Immanent in Nervous Activity](https://reader034.vdocuments.mx/reader034/viewer/2022051722/5a9dd63d7f8b9ae0108c4872/html5/thumbnails/32.jpg)
LT l Circuit Design Algorithm for SYMLT-l Circuit Design Algorithm for SYM
f(X)
0 1
f(X)
0
1
2
1
1
02
3
0
1Subtract 1 for every
4
5
1
0
Subtract 1 for everyisolated 1-block
67
11
![Page 33: IST 4 Information and Logic - brain.caltech.edubrain.caltech.edu/ist4/lectures/lect 1715 p.pdf · AON: AND, OR, Not LT: ... A Logical Calculus of Ideas Immanent in Nervous Activity](https://reader034.vdocuments.mx/reader034/viewer/2022051722/5a9dd63d7f8b9ae0108c4872/html5/thumbnails/33.jpg)
The Layered Construction for SYM -Some History
Saburo Muroga1925- 2009
1959
Was born in Japan Majority Decision
PhD in 1958 from Tokyo U, Japan
1960-1964: Researcher at IBM Research, NY,
1964-2002: professor at the University of Illinois, Urbana-Champaign
![Page 34: IST 4 Information and Logic - brain.caltech.edubrain.caltech.edu/ist4/lectures/lect 1715 p.pdf · AON: AND, OR, Not LT: ... A Logical Calculus of Ideas Immanent in Nervous Activity](https://reader034.vdocuments.mx/reader034/viewer/2022051722/5a9dd63d7f8b9ae0108c4872/html5/thumbnails/34.jpg)
Saburo Muroga1925- 2009
HW#5 problem 2a
![Page 35: IST 4 Information and Logic - brain.caltech.edubrain.caltech.edu/ist4/lectures/lect 1715 p.pdf · AON: AND, OR, Not LT: ... A Logical Calculus of Ideas Immanent in Nervous Activity](https://reader034.vdocuments.mx/reader034/viewer/2022051722/5a9dd63d7f8b9ae0108c4872/html5/thumbnails/35.jpg)
neural circuits and logicg
some more historysome more history...
![Page 36: IST 4 Information and Logic - brain.caltech.edubrain.caltech.edu/ist4/lectures/lect 1715 p.pdf · AON: AND, OR, Not LT: ... A Logical Calculus of Ideas Immanent in Nervous Activity](https://reader034.vdocuments.mx/reader034/viewer/2022051722/5a9dd63d7f8b9ae0108c4872/html5/thumbnails/36.jpg)
Being Homeless and Interdisciplinary Research
W M C ll hWarren McCulloch1899 - 1969
Walter Pitts1923 - 1969
Neurophysiologist, MD Logician, Autodidactg ,
Warren McCulloch arrived in early 1942 to the University of Chicago, invited Pitts, who was homeless, to live with his family
h ll h d P ll b d In the evenings McCulloch and Pitts collaborated. Pitts was familiar with the work of Leibniz on computing.They considered the question of whether the nervous system is a kind of universal computing device as described by Leibnizuniversal computing device as described by Leibniz
This led to their 1943 seminal neural networks paper:A Logical Calculus of Ideas Immanent in Nervous Activity
![Page 37: IST 4 Information and Logic - brain.caltech.edubrain.caltech.edu/ist4/lectures/lect 1715 p.pdf · AON: AND, OR, Not LT: ... A Logical Calculus of Ideas Immanent in Nervous Activity](https://reader034.vdocuments.mx/reader034/viewer/2022051722/5a9dd63d7f8b9ae0108c4872/html5/thumbnails/37.jpg)
Warren McCulloch W l Pi
ImpactWarren McCulloch
1899 - 1969 Walter Pitts1923 - 1969
Neurophysiologist, MD Logician, Autodidactg ,
This led to their 1943 seminal neural networks paper:p pA Logical Calculus of Ideas Immanent in Nervous Activity
Neural networks d Logic Ti MNeural networks and Logic Time Memory
Threshold Logic and Learning
State Machines
![Page 38: IST 4 Information and Logic - brain.caltech.edubrain.caltech.edu/ist4/lectures/lect 1715 p.pdf · AON: AND, OR, Not LT: ... A Logical Calculus of Ideas Immanent in Nervous Activity](https://reader034.vdocuments.mx/reader034/viewer/2022051722/5a9dd63d7f8b9ae0108c4872/html5/thumbnails/38.jpg)
neural circuits and memorym m y
computing with dynamicscomputing with dynamics
![Page 39: IST 4 Information and Logic - brain.caltech.edubrain.caltech.edu/ist4/lectures/lect 1715 p.pdf · AON: AND, OR, Not LT: ... A Logical Calculus of Ideas Immanent in Nervous Activity](https://reader034.vdocuments.mx/reader034/viewer/2022051722/5a9dd63d7f8b9ae0108c4872/html5/thumbnails/39.jpg)
Linear ThresholdSome AdjustmentsSome Adjustments
Linear Threshold (LT) gateLinear Threshold (LT) gate
-tthreshold
t
-t
-1
t
1
-1
![Page 40: IST 4 Information and Logic - brain.caltech.edubrain.caltech.edu/ist4/lectures/lect 1715 p.pdf · AON: AND, OR, Not LT: ... A Logical Calculus of Ideas Immanent in Nervous Activity](https://reader034.vdocuments.mx/reader034/viewer/2022051722/5a9dd63d7f8b9ae0108c4872/html5/thumbnails/40.jpg)
AND Function with {0,1}
-21
-21
0 00 1
1 0
-2-11
0001 0
1 1-10
01
![Page 41: IST 4 Information and Logic - brain.caltech.edubrain.caltech.edu/ist4/lectures/lect 1715 p.pdf · AON: AND, OR, Not LT: ... A Logical Calculus of Ideas Immanent in Nervous Activity](https://reader034.vdocuments.mx/reader034/viewer/2022051722/5a9dd63d7f8b9ae0108c4872/html5/thumbnails/41.jpg)
AND Function with {-1,1}
Th AND f ti f t i bl ith { 1 1} ???The AND function of two variables with {-1, 1}: ???
-1-1-11
-3-1--
1-111
11
1-11-+
![Page 42: IST 4 Information and Logic - brain.caltech.edubrain.caltech.edu/ist4/lectures/lect 1715 p.pdf · AON: AND, OR, Not LT: ... A Logical Calculus of Ideas Immanent in Nervous Activity](https://reader034.vdocuments.mx/reader034/viewer/2022051722/5a9dd63d7f8b9ae0108c4872/html5/thumbnails/42.jpg)
AND Function with {-1,1}
Th AND f ti f t i bl ith { 1 1} ???The AND function of two variables with {-1, 1}: ???
-1-1-11
-3-1
1-111
11
1-11
![Page 43: IST 4 Information and Logic - brain.caltech.edubrain.caltech.edu/ist4/lectures/lect 1715 p.pdf · AON: AND, OR, Not LT: ... A Logical Calculus of Ideas Immanent in Nervous Activity](https://reader034.vdocuments.mx/reader034/viewer/2022051722/5a9dd63d7f8b9ae0108c4872/html5/thumbnails/43.jpg)
Linear Threshold with MemoryEl h t b l f i d i A i ltElephants are symbols of wisdom in Asian culturesand are famed for their exceptional memory
A memory nosey
Remembers the last f(X)
![Page 44: IST 4 Information and Logic - brain.caltech.edubrain.caltech.edu/ist4/lectures/lect 1715 p.pdf · AON: AND, OR, Not LT: ... A Logical Calculus of Ideas Immanent in Nervous Activity](https://reader034.vdocuments.mx/reader034/viewer/2022051722/5a9dd63d7f8b9ae0108c4872/html5/thumbnails/44.jpg)
Feedback NetworksE lExample
-1 -1 00
weights thresholds
Th t t f th t k th t th t d The state of the network: the vector that corresponds to the states (noses…) of the gates
![Page 45: IST 4 Information and Logic - brain.caltech.edubrain.caltech.edu/ist4/lectures/lect 1715 p.pdf · AON: AND, OR, Not LT: ... A Logical Calculus of Ideas Immanent in Nervous Activity](https://reader034.vdocuments.mx/reader034/viewer/2022051722/5a9dd63d7f8b9ae0108c4872/html5/thumbnails/45.jpg)
Feedback NetworksExampleExample
1 2Label the gates
-1 -1 00 1 1
1 2Label the gates
-1 -1 00
![Page 46: IST 4 Information and Logic - brain.caltech.edubrain.caltech.edu/ist4/lectures/lect 1715 p.pdf · AON: AND, OR, Not LT: ... A Logical Calculus of Ideas Immanent in Nervous Activity](https://reader034.vdocuments.mx/reader034/viewer/2022051722/5a9dd63d7f8b9ae0108c4872/html5/thumbnails/46.jpg)
Feedback NetworksExampleExample
1 2
-1 -1 00 1 1
1 2
1-1 -1 00
![Page 47: IST 4 Information and Logic - brain.caltech.edubrain.caltech.edu/ist4/lectures/lect 1715 p.pdf · AON: AND, OR, Not LT: ... A Logical Calculus of Ideas Immanent in Nervous Activity](https://reader034.vdocuments.mx/reader034/viewer/2022051722/5a9dd63d7f8b9ae0108c4872/html5/thumbnails/47.jpg)
Feedback NetworksExampleExample
1 2
-1 -1 00 -1 1
1 2
1-1 -1 00
![Page 48: IST 4 Information and Logic - brain.caltech.edubrain.caltech.edu/ist4/lectures/lect 1715 p.pdf · AON: AND, OR, Not LT: ... A Logical Calculus of Ideas Immanent in Nervous Activity](https://reader034.vdocuments.mx/reader034/viewer/2022051722/5a9dd63d7f8b9ae0108c4872/html5/thumbnails/48.jpg)
Feedback NetworksExampleExample
1 2
-1 -1 00 -1 1
1 2-1
-1 -1 00
11 i bl -11 is a stable state
![Page 49: IST 4 Information and Logic - brain.caltech.edubrain.caltech.edu/ist4/lectures/lect 1715 p.pdf · AON: AND, OR, Not LT: ... A Logical Calculus of Ideas Immanent in Nervous Activity](https://reader034.vdocuments.mx/reader034/viewer/2022051722/5a9dd63d7f8b9ae0108c4872/html5/thumbnails/49.jpg)
Feedback NetworksExampleExample
1 2
-1 -1 00 1 1
1 2
-1 -1 00
![Page 50: IST 4 Information and Logic - brain.caltech.edubrain.caltech.edu/ist4/lectures/lect 1715 p.pdf · AON: AND, OR, Not LT: ... A Logical Calculus of Ideas Immanent in Nervous Activity](https://reader034.vdocuments.mx/reader034/viewer/2022051722/5a9dd63d7f8b9ae0108c4872/html5/thumbnails/50.jpg)
Feedback NetworksExampleExample
1 2
-1 -1 00 1 1
1 21
-1 -1 00
![Page 51: IST 4 Information and Logic - brain.caltech.edubrain.caltech.edu/ist4/lectures/lect 1715 p.pdf · AON: AND, OR, Not LT: ... A Logical Calculus of Ideas Immanent in Nervous Activity](https://reader034.vdocuments.mx/reader034/viewer/2022051722/5a9dd63d7f8b9ae0108c4872/html5/thumbnails/51.jpg)
Feedback NetworksExampleExample
1 2
-1 -1 00 1 -1
1 21
-1 -1 00
![Page 52: IST 4 Information and Logic - brain.caltech.edubrain.caltech.edu/ist4/lectures/lect 1715 p.pdf · AON: AND, OR, Not LT: ... A Logical Calculus of Ideas Immanent in Nervous Activity](https://reader034.vdocuments.mx/reader034/viewer/2022051722/5a9dd63d7f8b9ae0108c4872/html5/thumbnails/52.jpg)
Feedback NetworksExampleExample
1 2
-1 -1 00 1 -1
1 2-1
-1 -1 00
1 1 i bl 1-1 is a stable state
![Page 53: IST 4 Information and Logic - brain.caltech.edubrain.caltech.edu/ist4/lectures/lect 1715 p.pdf · AON: AND, OR, Not LT: ... A Logical Calculus of Ideas Immanent in Nervous Activity](https://reader034.vdocuments.mx/reader034/viewer/2022051722/5a9dd63d7f8b9ae0108c4872/html5/thumbnails/53.jpg)
Feedback NetworksExampleExample
1 2
-1 -1 00 1 1
1 2
-1 -1 00
State transition diagram (state space)1
The node
11 -11
2
1state
The node that computes -1-11-1
2Q:Is -1-1 a stable state?
![Page 54: IST 4 Information and Logic - brain.caltech.edubrain.caltech.edu/ist4/lectures/lect 1715 p.pdf · AON: AND, OR, Not LT: ... A Logical Calculus of Ideas Immanent in Nervous Activity](https://reader034.vdocuments.mx/reader034/viewer/2022051722/5a9dd63d7f8b9ae0108c4872/html5/thumbnails/54.jpg)
Feedback NetworksExampleExample
1 2
-1 -1 00 -1 -1
1 2-1
-1 -1 00
Answer: NoAnswer: No
Q:Is -1-1 a stable state?
![Page 55: IST 4 Information and Logic - brain.caltech.edubrain.caltech.edu/ist4/lectures/lect 1715 p.pdf · AON: AND, OR, Not LT: ... A Logical Calculus of Ideas Immanent in Nervous Activity](https://reader034.vdocuments.mx/reader034/viewer/2022051722/5a9dd63d7f8b9ae0108c4872/html5/thumbnails/55.jpg)
Feedback NetworksExampleExample
1 2
-1 -1 00 1 -1
1 2-1
-1 -1 00
11 111
11 -11
2-1-11-1
1
![Page 56: IST 4 Information and Logic - brain.caltech.edubrain.caltech.edu/ist4/lectures/lect 1715 p.pdf · AON: AND, OR, Not LT: ... A Logical Calculus of Ideas Immanent in Nervous Activity](https://reader034.vdocuments.mx/reader034/viewer/2022051722/5a9dd63d7f8b9ae0108c4872/html5/thumbnails/56.jpg)
Feedback NetworksExampleExample
1 2
-1 -1 00 -1 -1
1 2-1
-1 -1 00
11 111
11 -11
2-1-11-1
1
![Page 57: IST 4 Information and Logic - brain.caltech.edubrain.caltech.edu/ist4/lectures/lect 1715 p.pdf · AON: AND, OR, Not LT: ... A Logical Calculus of Ideas Immanent in Nervous Activity](https://reader034.vdocuments.mx/reader034/viewer/2022051722/5a9dd63d7f8b9ae0108c4872/html5/thumbnails/57.jpg)
Feedback NetworksExampleExample
1 2
-1 -1 00 -1 1
1 2-1
-1 -1 00
11 111
11 -11
2 2-1-11-1
1
![Page 58: IST 4 Information and Logic - brain.caltech.edubrain.caltech.edu/ist4/lectures/lect 1715 p.pdf · AON: AND, OR, Not LT: ... A Logical Calculus of Ideas Immanent in Nervous Activity](https://reader034.vdocuments.mx/reader034/viewer/2022051722/5a9dd63d7f8b9ae0108c4872/html5/thumbnails/58.jpg)
Feedback NetworksExampleExample
1 2
-1 -1 00 -1 1
1 2
-1 -1 00
11 111stable states
11 -11
2 2-1-11-1
1
![Page 59: IST 4 Information and Logic - brain.caltech.edubrain.caltech.edu/ist4/lectures/lect 1715 p.pdf · AON: AND, OR, Not LT: ... A Logical Calculus of Ideas Immanent in Nervous Activity](https://reader034.vdocuments.mx/reader034/viewer/2022051722/5a9dd63d7f8b9ae0108c4872/html5/thumbnails/59.jpg)
neural circuits and memorym m y
associative memoryassociative memory
![Page 60: IST 4 Information and Logic - brain.caltech.edubrain.caltech.edu/ist4/lectures/lect 1715 p.pdf · AON: AND, OR, Not LT: ... A Logical Calculus of Ideas Immanent in Nervous Activity](https://reader034.vdocuments.mx/reader034/viewer/2022051722/5a9dd63d7f8b9ae0108c4872/html5/thumbnails/60.jpg)
Feedback NetworksComputing with DynamicsComputing with Dynamics
1stable states11 -11
2
1
2
stable states
-1-11-1
2
1
2
Input:initial state
Output:stable state
FeedbackNetwork
n t al state
11 -1111 11
![Page 61: IST 4 Information and Logic - brain.caltech.edubrain.caltech.edu/ist4/lectures/lect 1715 p.pdf · AON: AND, OR, Not LT: ... A Logical Calculus of Ideas Immanent in Nervous Activity](https://reader034.vdocuments.mx/reader034/viewer/2022051722/5a9dd63d7f8b9ae0108c4872/html5/thumbnails/61.jpg)
Feedback NetworksComputing with DynamicsComputing with Dynamics
1stable states11 -11
2
1
2
stable states
-1-11-1
2
1
2
Input:initial state
Output:stable state
FeedbackNetwork
n t al state
11 1-111 1 1
![Page 62: IST 4 Information and Logic - brain.caltech.edubrain.caltech.edu/ist4/lectures/lect 1715 p.pdf · AON: AND, OR, Not LT: ... A Logical Calculus of Ideas Immanent in Nervous Activity](https://reader034.vdocuments.mx/reader034/viewer/2022051722/5a9dd63d7f8b9ae0108c4872/html5/thumbnails/62.jpg)
Feedback NetworksComputing with DynamicsComputing with Dynamics
Input: Output:Associative Memory“The Leibniz-Boole Machine”p
initial statep
stable stateThe Leibniz Boole Machine
FeedbackNetwork
![Page 63: IST 4 Information and Logic - brain.caltech.edubrain.caltech.edu/ist4/lectures/lect 1715 p.pdf · AON: AND, OR, Not LT: ... A Logical Calculus of Ideas Immanent in Nervous Activity](https://reader034.vdocuments.mx/reader034/viewer/2022051722/5a9dd63d7f8b9ae0108c4872/html5/thumbnails/63.jpg)
Feedback NetworksComputing with DynamicsComputing with Dynamics
Input: Output:Associative Memory“The Leibniz-Boole Machine”p
initial statep
stable stateThe Leibniz Boole Machine
FeedbackNetwork
![Page 64: IST 4 Information and Logic - brain.caltech.edubrain.caltech.edu/ist4/lectures/lect 1715 p.pdf · AON: AND, OR, Not LT: ... A Logical Calculus of Ideas Immanent in Nervous Activity](https://reader034.vdocuments.mx/reader034/viewer/2022051722/5a9dd63d7f8b9ae0108c4872/html5/thumbnails/64.jpg)
Feedback NetworksComputing with DynamicsComputing with Dynamics
Input: Output:Associative Memory“The Leibniz-Boole Machine”p
initial statep
stable stateThe Leibniz Boole Machine
FeedbackNetwork
![Page 65: IST 4 Information and Logic - brain.caltech.edubrain.caltech.edu/ist4/lectures/lect 1715 p.pdf · AON: AND, OR, Not LT: ... A Logical Calculus of Ideas Immanent in Nervous Activity](https://reader034.vdocuments.mx/reader034/viewer/2022051722/5a9dd63d7f8b9ae0108c4872/html5/thumbnails/65.jpg)
Feedback NetworksComputing with DynamicsComputing with Dynamics
Input: Output:Associative Memory“The Leibniz-Boole Machine”p
initial statep
stable stateThe Leibniz Boole Machine
FeedbackNetwork
![Page 66: IST 4 Information and Logic - brain.caltech.edubrain.caltech.edu/ist4/lectures/lect 1715 p.pdf · AON: AND, OR, Not LT: ... A Logical Calculus of Ideas Immanent in Nervous Activity](https://reader034.vdocuments.mx/reader034/viewer/2022051722/5a9dd63d7f8b9ae0108c4872/html5/thumbnails/66.jpg)
Feedback NetworksComputing with DynamicsComputing with Dynamics
Input: Output:Associative Memory“The Leibniz-Boole Machine”p
initial statep
stable stateThe Leibniz Boole Machine
FeedbackNetwork
![Page 67: IST 4 Information and Logic - brain.caltech.edubrain.caltech.edu/ist4/lectures/lect 1715 p.pdf · AON: AND, OR, Not LT: ... A Logical Calculus of Ideas Immanent in Nervous Activity](https://reader034.vdocuments.mx/reader034/viewer/2022051722/5a9dd63d7f8b9ae0108c4872/html5/thumbnails/67.jpg)
Feedback NetworksComputing with DynamicsComputing with Dynamics
Input: Output:Associative Memory“The Leibniz-Boole Machine”p
initial statep
stable stateThe Leibniz Boole Machine
FeedbackNetwork
![Page 68: IST 4 Information and Logic - brain.caltech.edubrain.caltech.edu/ist4/lectures/lect 1715 p.pdf · AON: AND, OR, Not LT: ... A Logical Calculus of Ideas Immanent in Nervous Activity](https://reader034.vdocuments.mx/reader034/viewer/2022051722/5a9dd63d7f8b9ae0108c4872/html5/thumbnails/68.jpg)
Feedback NetworksComputing with DynamicsComputing with Dynamics
Input: Output:Associative Memory“The Leibniz-Boole Machine”p
initial statep
stable stateThe Leibniz Boole Machine
FeedbackNetwork
![Page 69: IST 4 Information and Logic - brain.caltech.edubrain.caltech.edu/ist4/lectures/lect 1715 p.pdf · AON: AND, OR, Not LT: ... A Logical Calculus of Ideas Immanent in Nervous Activity](https://reader034.vdocuments.mx/reader034/viewer/2022051722/5a9dd63d7f8b9ae0108c4872/html5/thumbnails/69.jpg)
Feedback NetworksComputing with DynamicsComputing with Dynamics
Input: Output:Associative Memory“The Leibniz-Boole Machine”p
initial statep
stable stateThe Leibniz Boole Machine
FeedbackNetwork
![Page 70: IST 4 Information and Logic - brain.caltech.edubrain.caltech.edu/ist4/lectures/lect 1715 p.pdf · AON: AND, OR, Not LT: ... A Logical Calculus of Ideas Immanent in Nervous Activity](https://reader034.vdocuments.mx/reader034/viewer/2022051722/5a9dd63d7f8b9ae0108c4872/html5/thumbnails/70.jpg)
????
Who is this person?????
![Page 71: IST 4 Information and Logic - brain.caltech.edubrain.caltech.edu/ist4/lectures/lect 1715 p.pdf · AON: AND, OR, Not LT: ... A Logical Calculus of Ideas Immanent in Nervous Activity](https://reader034.vdocuments.mx/reader034/viewer/2022051722/5a9dd63d7f8b9ae0108c4872/html5/thumbnails/71.jpg)
John Hopfield
f ld d l ( l h )Feedback Networks
Hopfield Model (Caltech 1982)
![Page 72: IST 4 Information and Logic - brain.caltech.edubrain.caltech.edu/ist4/lectures/lect 1715 p.pdf · AON: AND, OR, Not LT: ... A Logical Calculus of Ideas Immanent in Nervous Activity](https://reader034.vdocuments.mx/reader034/viewer/2022051722/5a9dd63d7f8b9ae0108c4872/html5/thumbnails/72.jpg)
Feedback NetworksJohn Hopfield
f ld d l ( l h )
1 2
Hopfield Model (Caltech 1982)
-1 -1 00 1 1
-11 20 0
1 1
1
i = node i 0 = threshold ti
-1
1 = state vi -1 = weight of edge (i,j)
![Page 73: IST 4 Information and Logic - brain.caltech.edubrain.caltech.edu/ist4/lectures/lect 1715 p.pdf · AON: AND, OR, Not LT: ... A Logical Calculus of Ideas Immanent in Nervous Activity](https://reader034.vdocuments.mx/reader034/viewer/2022051722/5a9dd63d7f8b9ae0108c4872/html5/thumbnails/73.jpg)
The matrix descriptionm p
![Page 74: IST 4 Information and Logic - brain.caltech.edubrain.caltech.edu/ist4/lectures/lect 1715 p.pdf · AON: AND, OR, Not LT: ... A Logical Calculus of Ideas Immanent in Nervous Activity](https://reader034.vdocuments.mx/reader034/viewer/2022051722/5a9dd63d7f8b9ae0108c4872/html5/thumbnails/74.jpg)
Feedback Networks/The Vector/Matrix Description
An n node feedback network can be specified by:An n node feedback network can be specified by:• W an nxn matrix of weights• T an n vector of thresholds f• V an n vector of states
1
4
2
34
5
![Page 75: IST 4 Information and Logic - brain.caltech.edubrain.caltech.edu/ist4/lectures/lect 1715 p.pdf · AON: AND, OR, Not LT: ... A Logical Calculus of Ideas Immanent in Nervous Activity](https://reader034.vdocuments.mx/reader034/viewer/2022051722/5a9dd63d7f8b9ae0108c4872/html5/thumbnails/75.jpg)
The Matrix DescriptionE lExample
An n node feedback network can be specified by:An n node feedback network can be specified by:• W an nxn matrix of weights• T an n vector of thresholds f• V an n vector of states
1 20 0
-121
-1
11 1
-1
-1
![Page 76: IST 4 Information and Logic - brain.caltech.edubrain.caltech.edu/ist4/lectures/lect 1715 p.pdf · AON: AND, OR, Not LT: ... A Logical Calculus of Ideas Immanent in Nervous Activity](https://reader034.vdocuments.mx/reader034/viewer/2022051722/5a9dd63d7f8b9ae0108c4872/html5/thumbnails/76.jpg)
The Matrix DescriptionComputationComputation
C t ti i N (W T)Computation in N= (W,T)1
4
2
34
5by column
![Page 77: IST 4 Information and Logic - brain.caltech.edubrain.caltech.edu/ist4/lectures/lect 1715 p.pdf · AON: AND, OR, Not LT: ... A Logical Calculus of Ideas Immanent in Nervous Activity](https://reader034.vdocuments.mx/reader034/viewer/2022051722/5a9dd63d7f8b9ae0108c4872/html5/thumbnails/77.jpg)
Order of computationf mp
serial and parallel
![Page 78: IST 4 Information and Logic - brain.caltech.edubrain.caltech.edu/ist4/lectures/lect 1715 p.pdf · AON: AND, OR, Not LT: ... A Logical Calculus of Ideas Immanent in Nervous Activity](https://reader034.vdocuments.mx/reader034/viewer/2022051722/5a9dd63d7f8b9ae0108c4872/html5/thumbnails/78.jpg)
Modes of OperationModes of Operation
Q: when do the nodes compute?Q: when do the nodes compute?
Serial mode: one node at a time (arbitrary order)
-1
( y )
21-1
![Page 79: IST 4 Information and Logic - brain.caltech.edubrain.caltech.edu/ist4/lectures/lect 1715 p.pdf · AON: AND, OR, Not LT: ... A Logical Calculus of Ideas Immanent in Nervous Activity](https://reader034.vdocuments.mx/reader034/viewer/2022051722/5a9dd63d7f8b9ae0108c4872/html5/thumbnails/79.jpg)
Modes of OperationModes of Operation
Q: when do the nodes compute?Q: when do the nodes compute?
Serial mode: one node at a time (arbitrary order)
-1
( y )
21-1
![Page 80: IST 4 Information and Logic - brain.caltech.edubrain.caltech.edu/ist4/lectures/lect 1715 p.pdf · AON: AND, OR, Not LT: ... A Logical Calculus of Ideas Immanent in Nervous Activity](https://reader034.vdocuments.mx/reader034/viewer/2022051722/5a9dd63d7f8b9ae0108c4872/html5/thumbnails/80.jpg)
Modes of OperationModes of Operation
Q: when do the nodes compute?Q: when do the nodes compute?
Serial mode: one node at a time (arbitrary order)
-1
( y )
21-1
Fully-Parallel mode: all nodes at the same time-1
21-1
![Page 81: IST 4 Information and Logic - brain.caltech.edubrain.caltech.edu/ist4/lectures/lect 1715 p.pdf · AON: AND, OR, Not LT: ... A Logical Calculus of Ideas Immanent in Nervous Activity](https://reader034.vdocuments.mx/reader034/viewer/2022051722/5a9dd63d7f8b9ae0108c4872/html5/thumbnails/81.jpg)
Three examplesmp
![Page 82: IST 4 Information and Logic - brain.caltech.edubrain.caltech.edu/ist4/lectures/lect 1715 p.pdf · AON: AND, OR, Not LT: ... A Logical Calculus of Ideas Immanent in Nervous Activity](https://reader034.vdocuments.mx/reader034/viewer/2022051722/5a9dd63d7f8b9ae0108c4872/html5/thumbnails/82.jpg)
Example 1Serial Mode – Symmetric Weight MatrixSerial Mode – Symmetric Weight Matrix
21-1
-11
The state space:
1stable states
The state space:
11 -11
2
1
2-1-11-1
1
![Page 83: IST 4 Information and Logic - brain.caltech.edubrain.caltech.edu/ist4/lectures/lect 1715 p.pdf · AON: AND, OR, Not LT: ... A Logical Calculus of Ideas Immanent in Nervous Activity](https://reader034.vdocuments.mx/reader034/viewer/2022051722/5a9dd63d7f8b9ae0108c4872/html5/thumbnails/83.jpg)
Example 2Fully Parallel (FP) Mode Symmetric Weight MatrixFully-Parallel (FP) Mode – Symmetric Weight Matrix
Q: how does the state space look?
21-1
Q p
21-1
start with 11start with 11
It’s acycle!cycle!
![Page 84: IST 4 Information and Logic - brain.caltech.edubrain.caltech.edu/ist4/lectures/lect 1715 p.pdf · AON: AND, OR, Not LT: ... A Logical Calculus of Ideas Immanent in Nervous Activity](https://reader034.vdocuments.mx/reader034/viewer/2022051722/5a9dd63d7f8b9ae0108c4872/html5/thumbnails/84.jpg)
Example 2Fully Parallel (FP) Mode Symmetric Weight MatrixFully-Parallel (FP) Mode – Symmetric Weight Matrix
21-1
21-1
h The state space:
stable states11 -11
l f l th 2-1-11-1
cycle of length 2
![Page 85: IST 4 Information and Logic - brain.caltech.edubrain.caltech.edu/ist4/lectures/lect 1715 p.pdf · AON: AND, OR, Not LT: ... A Logical Calculus of Ideas Immanent in Nervous Activity](https://reader034.vdocuments.mx/reader034/viewer/2022051722/5a9dd63d7f8b9ae0108c4872/html5/thumbnails/85.jpg)
Example 2Fully Parallel Mode Symmetric Weight Matrix
3AntisymmetricFully-Parallel Mode – Symmetric Weight MatrixAntisymmetric
W T = −W21
-1
1
11W W
-1
Q: how does the state space look?p
![Page 86: IST 4 Information and Logic - brain.caltech.edubrain.caltech.edu/ist4/lectures/lect 1715 p.pdf · AON: AND, OR, Not LT: ... A Logical Calculus of Ideas Immanent in Nervous Activity](https://reader034.vdocuments.mx/reader034/viewer/2022051722/5a9dd63d7f8b9ae0108c4872/html5/thumbnails/86.jpg)
Example 3Fully Parallel Mode Antisymmetric Weight Matrix
1
Fully-Parallel Mode – Antisymmetric Weight Matrix
21-1
Q: how does the state space look?
cycle of length 4
![Page 87: IST 4 Information and Logic - brain.caltech.edubrain.caltech.edu/ist4/lectures/lect 1715 p.pdf · AON: AND, OR, Not LT: ... A Logical Calculus of Ideas Immanent in Nervous Activity](https://reader034.vdocuments.mx/reader034/viewer/2022051722/5a9dd63d7f8b9ae0108c4872/html5/thumbnails/87.jpg)
Example 3Fully Parallel Mode Antisymmetric Weight Matrix
1
Fully-Parallel Mode – Antisymmetric Weight Matrix
21-1
The state space:
11 -11
-1-11-1
cycle of length 4
![Page 88: IST 4 Information and Logic - brain.caltech.edubrain.caltech.edu/ist4/lectures/lect 1715 p.pdf · AON: AND, OR, Not LT: ... A Logical Calculus of Ideas Immanent in Nervous Activity](https://reader034.vdocuments.mx/reader034/viewer/2022051722/5a9dd63d7f8b9ae0108c4872/html5/thumbnails/88.jpg)
The Three Cases
11 -1111 -11 11 -11
-1-11-1-1-11-1 -1-11-1
1 2 3
Wmode symmetric antisymmetricCycle lengths
serial 1 ?1Example #
fully-parallel 1,2 42 3
![Page 89: IST 4 Information and Logic - brain.caltech.edubrain.caltech.edu/ist4/lectures/lect 1715 p.pdf · AON: AND, OR, Not LT: ... A Logical Calculus of Ideas Immanent in Nervous Activity](https://reader034.vdocuments.mx/reader034/viewer/2022051722/5a9dd63d7f8b9ae0108c4872/html5/thumbnails/89.jpg)
The Three Cases
Wmode symmetric antisymmetricCycle lengths
mode ymm ymm
serial 1 ?1
Example #fully-parallel 1,2 4
2 3
Example #
1 Hopfield 1982
2 G l 19852 Goles 1985
3 Goles 1986
![Page 90: IST 4 Information and Logic - brain.caltech.edubrain.caltech.edu/ist4/lectures/lect 1715 p.pdf · AON: AND, OR, Not LT: ... A Logical Calculus of Ideas Immanent in Nervous Activity](https://reader034.vdocuments.mx/reader034/viewer/2022051722/5a9dd63d7f8b9ae0108c4872/html5/thumbnails/90.jpg)
Proof Ideas
C l l th W symmetric antisymmetricCycle lengths Wmode symmetric antisymmetric
serial 1 ?1Example #
fully-parallel 1,2 4
1
2 3The proofs of these three results use the conceptof an energy function
For the serial mode:
Sh th tShow that:
Namely, stable states are local max of the energy E
![Page 91: IST 4 Information and Logic - brain.caltech.edubrain.caltech.edu/ist4/lectures/lect 1715 p.pdf · AON: AND, OR, Not LT: ... A Logical Calculus of Ideas Immanent in Nervous Activity](https://reader034.vdocuments.mx/reader034/viewer/2022051722/5a9dd63d7f8b9ae0108c4872/html5/thumbnails/91.jpg)
Questions on Convergence
Posted on the class web site
Cycle lengths Wd symmetric antisymmetric
Posted on the class web site
Cycle lengths Wmode symmetric antisymmetric
serial 1 ?1Example #
fully-parallel 1,2 42 3
1 Hopfield 1982
2 Goles 1985 Q1: Are the three cases “distinct”?2 Goles 1985
3 Goles 1986
Q
Q2: Elementary proof? (wo/energy)
![Page 92: IST 4 Information and Logic - brain.caltech.edubrain.caltech.edu/ist4/lectures/lect 1715 p.pdf · AON: AND, OR, Not LT: ... A Logical Calculus of Ideas Immanent in Nervous Activity](https://reader034.vdocuments.mx/reader034/viewer/2022051722/5a9dd63d7f8b9ae0108c4872/html5/thumbnails/92.jpg)