the binding problem -...
TRANSCRIPT
![Page 1: The Binding Problem - inst.eecs.berkeley.eduinst.eecs.berkeley.edu/~cs182/sp07/notes/lecture22-binding.pdfLokendra Shastri ICSI, Berkeley The Binding Problem Massively Parallel](https://reader034.vdocuments.mx/reader034/viewer/2022051802/5af105207f8b9a8c308df826/html5/thumbnails/1.jpg)
Lokendra Shastri Lokendra Shastri ICSI, Berkeley ICSI, Berkeley
The Binding Problem
Massively Parallel Brain
Unitary Conscious Experience
Many Variations and Proposals
Our focus: The Variable Binding Problem
![Page 2: The Binding Problem - inst.eecs.berkeley.eduinst.eecs.berkeley.edu/~cs182/sp07/notes/lecture22-binding.pdfLokendra Shastri ICSI, Berkeley The Binding Problem Massively Parallel](https://reader034.vdocuments.mx/reader034/viewer/2022051802/5af105207f8b9a8c308df826/html5/thumbnails/2.jpg)
Lokendra Shastri Lokendra Shastri ICSI, Berkeley ICSI, Berkeley
![Page 3: The Binding Problem - inst.eecs.berkeley.eduinst.eecs.berkeley.edu/~cs182/sp07/notes/lecture22-binding.pdfLokendra Shastri ICSI, Berkeley The Binding Problem Massively Parallel](https://reader034.vdocuments.mx/reader034/viewer/2022051802/5af105207f8b9a8c308df826/html5/thumbnails/3.jpg)
Lokendra Shastri Lokendra Shastri ICSI, Berkeley ICSI, Berkeley
Problem• Binding problem
– In vision• You do not exchange the colors of the shapes below
– In behavior• Grasp motion depends on object to grasp
– In inference• Human(x) -> Mortal(x)• Must bind a variable to x
![Page 4: The Binding Problem - inst.eecs.berkeley.eduinst.eecs.berkeley.edu/~cs182/sp07/notes/lecture22-binding.pdfLokendra Shastri ICSI, Berkeley The Binding Problem Massively Parallel](https://reader034.vdocuments.mx/reader034/viewer/2022051802/5af105207f8b9a8c308df826/html5/thumbnails/4.jpg)
Lokendra Shastri Lokendra Shastri ICSI, Berkeley ICSI, Berkeley
Automatic Inference
• Inference needed for many tasks– Reference resolution– General language understanding– Planning
• Humans do this quickly and without conscious thought– Automatically– No real intuition of how we do it
![Page 5: The Binding Problem - inst.eecs.berkeley.eduinst.eecs.berkeley.edu/~cs182/sp07/notes/lecture22-binding.pdfLokendra Shastri ICSI, Berkeley The Binding Problem Massively Parallel](https://reader034.vdocuments.mx/reader034/viewer/2022051802/5af105207f8b9a8c308df826/html5/thumbnails/5.jpg)
Lokendra Shastri Lokendra Shastri ICSI, Berkeley ICSI, Berkeley
Other Solutions in Inference
• Brute-force enumeration– Does not scale to depth of human knowledge
• Signature propagation (direct reference)– Difficult to pass enough information to directly
reference each object– Unifying two bindings (e.g. reference resolution) is
difficult
• Temporal synchrony example (SHRUTI)– Little biological evidence
![Page 6: The Binding Problem - inst.eecs.berkeley.eduinst.eecs.berkeley.edu/~cs182/sp07/notes/lecture22-binding.pdfLokendra Shastri ICSI, Berkeley The Binding Problem Massively Parallel](https://reader034.vdocuments.mx/reader034/viewer/2022051802/5af105207f8b9a8c308df826/html5/thumbnails/6.jpg)
Lokendra Shastri Lokendra Shastri ICSI, Berkeley ICSI, Berkeley
SHRUTI• SHRUTI does
inference by connections between simple computation nodes
• Nodes are small groups of neurons
• Nodes firing in sync reference the same object
![Page 7: The Binding Problem - inst.eecs.berkeley.eduinst.eecs.berkeley.edu/~cs182/sp07/notes/lecture22-binding.pdfLokendra Shastri ICSI, Berkeley The Binding Problem Massively Parallel](https://reader034.vdocuments.mx/reader034/viewer/2022051802/5af105207f8b9a8c308df826/html5/thumbnails/7.jpg)
Lokendra Shastri Lokendra Shastri ICSI, Berkeley ICSI, Berkeley
A Neurally Plausible model of Reasoning
Lokendra Shastri Lokendra Shastri ICSI, Berkeley ICSI, Berkeley
Lokendra ShastriInternational Computer Science Institute
Berkeley, CA 94704
![Page 8: The Binding Problem - inst.eecs.berkeley.eduinst.eecs.berkeley.edu/~cs182/sp07/notes/lecture22-binding.pdfLokendra Shastri ICSI, Berkeley The Binding Problem Massively Parallel](https://reader034.vdocuments.mx/reader034/viewer/2022051802/5af105207f8b9a8c308df826/html5/thumbnails/8.jpg)
Lokendra Shastri Lokendra Shastri ICSI, Berkeley ICSI, Berkeley
Five levels of Neural Theory of Language
Cognition and Language
Computation
S tructured Connectionism
Computational Neurobiology
Biology
SHRUTISHRUTI
abstraction
![Page 9: The Binding Problem - inst.eecs.berkeley.eduinst.eecs.berkeley.edu/~cs182/sp07/notes/lecture22-binding.pdfLokendra Shastri ICSI, Berkeley The Binding Problem Massively Parallel](https://reader034.vdocuments.mx/reader034/viewer/2022051802/5af105207f8b9a8c308df826/html5/thumbnails/9.jpg)
Lokendra Shastri Lokendra Shastri ICSI, Berkeley ICSI, Berkeley
“John fell in the hallway. Tom had cleaned it. He got hurt.”
⇒ Tom had cleaned the hallwaythe hallway.
⇒ The hallway floor was wetThe hallway floor was wet..
⇒ John slipped and fell on the wet floorJohn slipped and fell on the wet floor.
⇒ JohnJohn got hurt as a result of the fallas a result of the fall.
such inferences establish referential and causal coherence.
![Page 10: The Binding Problem - inst.eecs.berkeley.eduinst.eecs.berkeley.edu/~cs182/sp07/notes/lecture22-binding.pdfLokendra Shastri ICSI, Berkeley The Binding Problem Massively Parallel](https://reader034.vdocuments.mx/reader034/viewer/2022051802/5af105207f8b9a8c308df826/html5/thumbnails/10.jpg)
Lokendra Shastri Lokendra Shastri ICSI, Berkeley ICSI, Berkeley
Reflexive Reasoning
Ubiquitous
Automatic, effortless
Extremely fast --- almost a reflex response
of our cognitive apparatus
![Page 11: The Binding Problem - inst.eecs.berkeley.eduinst.eecs.berkeley.edu/~cs182/sp07/notes/lecture22-binding.pdfLokendra Shastri ICSI, Berkeley The Binding Problem Massively Parallel](https://reader034.vdocuments.mx/reader034/viewer/2022051802/5af105207f8b9a8c308df826/html5/thumbnails/11.jpg)
Lokendra Shastri Lokendra Shastri ICSI, Berkeley ICSI, Berkeley
Reflexive Reasoning
Not all reasoning is reflexive
Contrast with reflective reasoning
deliberate
involves explicit consideration of alternatives
require props (paper and pencil)
e.g., solving logic puzzles … differential equations
![Page 12: The Binding Problem - inst.eecs.berkeley.eduinst.eecs.berkeley.edu/~cs182/sp07/notes/lecture22-binding.pdfLokendra Shastri ICSI, Berkeley The Binding Problem Massively Parallel](https://reader034.vdocuments.mx/reader034/viewer/2022051802/5af105207f8b9a8c308df826/html5/thumbnails/12.jpg)
How fast is reflexive reasoning?
• We understand language at the rate of 150-400 words per minute
⇒ Reflexive inferences required for establishing inferential and causal coherence are drawn within a few hundred milliseconds
Lokendra Shastri Lokendra Shastri ICSI, Berkeley ICSI, Berkeley
![Page 13: The Binding Problem - inst.eecs.berkeley.eduinst.eecs.berkeley.edu/~cs182/sp07/notes/lecture22-binding.pdfLokendra Shastri ICSI, Berkeley The Binding Problem Massively Parallel](https://reader034.vdocuments.mx/reader034/viewer/2022051802/5af105207f8b9a8c308df826/html5/thumbnails/13.jpg)
How can a system of slow and simple neuron-like elements
• encode a large body of semantic and episodic knowledge and yet
• perform a wide range of inferences within a few hundred milliseconds?
Lokendra Shastri Lokendra Shastri ICSI, Berkeley ICSI, Berkeley
![Page 14: The Binding Problem - inst.eecs.berkeley.eduinst.eecs.berkeley.edu/~cs182/sp07/notes/lecture22-binding.pdfLokendra Shastri ICSI, Berkeley The Binding Problem Massively Parallel](https://reader034.vdocuments.mx/reader034/viewer/2022051802/5af105207f8b9a8c308df826/html5/thumbnails/14.jpg)
Characterization of reflexive reasoning?
• What can and cannot be inferred via reflexive processes?
Lokendra Shastri Lokendra Shastri ICSI, Berkeley ICSI, Berkeley
![Page 15: The Binding Problem - inst.eecs.berkeley.eduinst.eecs.berkeley.edu/~cs182/sp07/notes/lecture22-binding.pdfLokendra Shastri ICSI, Berkeley The Binding Problem Massively Parallel](https://reader034.vdocuments.mx/reader034/viewer/2022051802/5af105207f8b9a8c308df826/html5/thumbnails/15.jpg)
Shruti
http://www.icsi.berkeley.edu/~shastri/shruti
• Lokendra Shastri• V. Ajjanagadde (Penn, ex-graduate student)
• Carter Wendelken (UCB, ex-graduate student) • D. Mani (Penn, ex-graduate student)
• D.J. Grannes (UCB, ex-graduate student)
• Jerry Hobbs, USC/ISI (abductive reasoning)
• Marvin Cohen, CTI (metacognition; belief and utility)
• Bryan Thompson, CTI (metacognition; belief and utility)
Lokendra Shastri Lokendra Shastri ICSI, Berkeley ICSI, Berkeley
![Page 16: The Binding Problem - inst.eecs.berkeley.eduinst.eecs.berkeley.edu/~cs182/sp07/notes/lecture22-binding.pdfLokendra Shastri ICSI, Berkeley The Binding Problem Massively Parallel](https://reader034.vdocuments.mx/reader034/viewer/2022051802/5af105207f8b9a8c308df826/html5/thumbnails/16.jpg)
Reflexive Reasoningrepresentational and processing issues
• Activation-based (dynamic) representation of
events and situations (relational instances)
![Page 17: The Binding Problem - inst.eecs.berkeley.eduinst.eecs.berkeley.edu/~cs182/sp07/notes/lecture22-binding.pdfLokendra Shastri ICSI, Berkeley The Binding Problem Massively Parallel](https://reader034.vdocuments.mx/reader034/viewer/2022051802/5af105207f8b9a8c308df826/html5/thumbnails/17.jpg)
Lokendra Shastri Lokendra Shastri ICSI, Berkeley ICSI, Berkeley
Dynamic representation of relational instances
“John gave Mary a book”giver: John
recipient: Mary
given-object: a-book
giver
a-book
Mary
recipient
John
given-object
*
![Page 18: The Binding Problem - inst.eecs.berkeley.eduinst.eecs.berkeley.edu/~cs182/sp07/notes/lecture22-binding.pdfLokendra Shastri ICSI, Berkeley The Binding Problem Massively Parallel](https://reader034.vdocuments.mx/reader034/viewer/2022051802/5af105207f8b9a8c308df826/html5/thumbnails/18.jpg)
Lokendra Shastri Lokendra Shastri ICSI, Berkeley ICSI, Berkeley
Reflexive Reasoning
• Expressing dynamic bindings
• Systematically propagating dynamic bindings
• Computing coherent explanations and predictions
– evidence combination– instantiation and unification of entities
Requires compatible neural mechanisms for:
All of the above must happen rapidly
![Page 19: The Binding Problem - inst.eecs.berkeley.eduinst.eecs.berkeley.edu/~cs182/sp07/notes/lecture22-binding.pdfLokendra Shastri ICSI, Berkeley The Binding Problem Massively Parallel](https://reader034.vdocuments.mx/reader034/viewer/2022051802/5af105207f8b9a8c308df826/html5/thumbnails/19.jpg)
Lokendra Shastri Lokendra Shastri ICSI, Berkeley ICSI, Berkeley
Learning
• one-shot learning of events and situations (episodic memory)
• gradual/incremental learning of concepts, relations, schemas, and causal structures
![Page 20: The Binding Problem - inst.eecs.berkeley.eduinst.eecs.berkeley.edu/~cs182/sp07/notes/lecture22-binding.pdfLokendra Shastri ICSI, Berkeley The Binding Problem Massively Parallel](https://reader034.vdocuments.mx/reader034/viewer/2022051802/5af105207f8b9a8c308df826/html5/thumbnails/20.jpg)
Lokendra Shastri Lokendra Shastri ICSI, Berkeley ICSI, Berkeley
Relation focal-cluster
+ - ? fall-pat fall-loc
FALLFALL
![Page 21: The Binding Problem - inst.eecs.berkeley.eduinst.eecs.berkeley.edu/~cs182/sp07/notes/lecture22-binding.pdfLokendra Shastri ICSI, Berkeley The Binding Problem Massively Parallel](https://reader034.vdocuments.mx/reader034/viewer/2022051802/5af105207f8b9a8c308df826/html5/thumbnails/21.jpg)
Lokendra Shastri Lokendra Shastri ICSI, Berkeley ICSI, Berkeley
Entity, category and relation focal-clusters
+ - ? fall-pat fall-loc
FALLFALL
![Page 22: The Binding Problem - inst.eecs.berkeley.eduinst.eecs.berkeley.edu/~cs182/sp07/notes/lecture22-binding.pdfLokendra Shastri ICSI, Berkeley The Binding Problem Massively Parallel](https://reader034.vdocuments.mx/reader034/viewer/2022051802/5af105207f8b9a8c308df826/html5/thumbnails/22.jpg)
Lokendra Shastri Lokendra Shastri ICSI, Berkeley ICSI, Berkeley
Entity, category and relation focal-clusters
+ - ? fall-pat fall-loc
FALLFALL
Functional nodes in a focal-cluster [collector (+/-), enabler (?), and role nodes] may be situated in different brain regions
![Page 23: The Binding Problem - inst.eecs.berkeley.eduinst.eecs.berkeley.edu/~cs182/sp07/notes/lecture22-binding.pdfLokendra Shastri ICSI, Berkeley The Binding Problem Massively Parallel](https://reader034.vdocuments.mx/reader034/viewer/2022051802/5af105207f8b9a8c308df826/html5/thumbnails/23.jpg)
Lokendra Shastri Lokendra Shastri ICSI, Berkeley ICSI, Berkeley
Focal-cluster of a relational schema
FALLFALL + - ? fall-pat fall-loc
focal-clusters of motor schemasassociated with fall
focal-clusters of lexical know-ledge associated with fall
focal-clusters of perceptual schemas and sensory representations associated with fall
focal-clusters of otherrelational schemascausally related to fall
episodicmemories offall events
![Page 24: The Binding Problem - inst.eecs.berkeley.eduinst.eecs.berkeley.edu/~cs182/sp07/notes/lecture22-binding.pdfLokendra Shastri ICSI, Berkeley The Binding Problem Massively Parallel](https://reader034.vdocuments.mx/reader034/viewer/2022051802/5af105207f8b9a8c308df826/html5/thumbnails/24.jpg)
Lokendra Shastri Lokendra Shastri ICSI, Berkeley ICSI, Berkeley
Focal-clusters
Nodes in the fall focal-cluster become active when
• perceiving a fall event
• remembering a fall event
• understanding a sentence about a fall event
• experiencing a fall event
A focal-cluster is like a “supra-mirror” cluster
![Page 25: The Binding Problem - inst.eecs.berkeley.eduinst.eecs.berkeley.edu/~cs182/sp07/notes/lecture22-binding.pdfLokendra Shastri ICSI, Berkeley The Binding Problem Massively Parallel](https://reader034.vdocuments.mx/reader034/viewer/2022051802/5af105207f8b9a8c308df826/html5/thumbnails/25.jpg)
Lokendra Shastri Lokendra Shastri ICSI, Berkeley ICSI, Berkeley
Focal-cluster of an entity
JohnJohn + ?
focal-clusters of motor schemasassociated with John
focal-clusters of lexical know-ledge associated with John
focal-clusters of perceptualschemas and sensoryrepresentations associatedwith John
focal-clusters of otherentities and categoriessemantically related to John
episodic memorieswhere John is oneof the role-fillers
![Page 26: The Binding Problem - inst.eecs.berkeley.eduinst.eecs.berkeley.edu/~cs182/sp07/notes/lecture22-binding.pdfLokendra Shastri ICSI, Berkeley The Binding Problem Massively Parallel](https://reader034.vdocuments.mx/reader034/viewer/2022051802/5af105207f8b9a8c308df826/html5/thumbnails/26.jpg)
Lokendra Shastri Lokendra Shastri ICSI, Berkeley ICSI, Berkeley
+ - ? fall-pat fall-loc
Fall
+ ?
+ ?
Hallway
John
“John fell in the hallway”
![Page 27: The Binding Problem - inst.eecs.berkeley.eduinst.eecs.berkeley.edu/~cs182/sp07/notes/lecture22-binding.pdfLokendra Shastri ICSI, Berkeley The Binding Problem Massively Parallel](https://reader034.vdocuments.mx/reader034/viewer/2022051802/5af105207f8b9a8c308df826/html5/thumbnails/27.jpg)
Lokendra Shastri Lokendra Shastri ICSI, Berkeley ICSI, Berkeley
+ - ? fall-pat fall-loc
Fall
+ ?
+ ?
Hallway
John
“John fell in the hallway”
![Page 28: The Binding Problem - inst.eecs.berkeley.eduinst.eecs.berkeley.edu/~cs182/sp07/notes/lecture22-binding.pdfLokendra Shastri ICSI, Berkeley The Binding Problem Massively Parallel](https://reader034.vdocuments.mx/reader034/viewer/2022051802/5af105207f8b9a8c308df826/html5/thumbnails/28.jpg)
Lokendra Shastri Lokendra Shastri ICSI, Berkeley ICSI, Berkeley
+ -- ? fall-pat fall-loc
Fall
+ ?
Hallway
John + ?
+:Fall
+:John
fall-pat
fall-loc
+:Hallway
“John fell in the hallway”
![Page 29: The Binding Problem - inst.eecs.berkeley.eduinst.eecs.berkeley.edu/~cs182/sp07/notes/lecture22-binding.pdfLokendra Shastri ICSI, Berkeley The Binding Problem Massively Parallel](https://reader034.vdocuments.mx/reader034/viewer/2022051802/5af105207f8b9a8c308df826/html5/thumbnails/29.jpg)
Lokendra Shastri Lokendra Shastri ICSI, Berkeley ICSI, Berkeley
Encoding “slip => fall” in Shruti
SLIPSLIP + - ? slip-pat slip-loc
FALLFALL + - ? fall-pat fall-loc
+ ? r1 r2
mediatormediator
Such rules are learned gradually via observations, by being told …
![Page 30: The Binding Problem - inst.eecs.berkeley.eduinst.eecs.berkeley.edu/~cs182/sp07/notes/lecture22-binding.pdfLokendra Shastri ICSI, Berkeley The Binding Problem Massively Parallel](https://reader034.vdocuments.mx/reader034/viewer/2022051802/5af105207f8b9a8c308df826/html5/thumbnails/30.jpg)
Lokendra Shastri Lokendra Shastri ICSI, Berkeley ICSI, Berkeley
“John slipped in the hallway”
SlipSlip + - ? slip-pat slip-loc
FallFall + - ? fall-pat fall-loc
mediatormediatorr2 r1 ?+
+ ?
HallwayJohn
+ ?
“→ John fell in the hallway”
+:slip
+:John
slip-pat
+:Hallway
slip-loc
+:Fall
fall-pat
fall-loc
![Page 31: The Binding Problem - inst.eecs.berkeley.eduinst.eecs.berkeley.edu/~cs182/sp07/notes/lecture22-binding.pdfLokendra Shastri ICSI, Berkeley The Binding Problem Massively Parallel](https://reader034.vdocuments.mx/reader034/viewer/2022051802/5af105207f8b9a8c308df826/html5/thumbnails/31.jpg)
Lokendra Shastri Lokendra Shastri ICSI, Berkeley ICSI, Berkeley
A Metaphor for Reasoning
• An episode of reflexive reasoning is a transient propagation of rhythmic activity
• Each entity involved in this reasoning episode is a phase in this rhythmic activity
• Bindings are synchronous firings of cell clusters
• Rules are interconnections between cell-clusters that support propagation of synchronous activity
![Page 32: The Binding Problem - inst.eecs.berkeley.eduinst.eecs.berkeley.edu/~cs182/sp07/notes/lecture22-binding.pdfLokendra Shastri ICSI, Berkeley The Binding Problem Massively Parallel](https://reader034.vdocuments.mx/reader034/viewer/2022051802/5af105207f8b9a8c308df826/html5/thumbnails/32.jpg)
Lokendra Shastri Lokendra Shastri ICSI, Berkeley ICSI, Berkeley
![Page 33: The Binding Problem - inst.eecs.berkeley.eduinst.eecs.berkeley.edu/~cs182/sp07/notes/lecture22-binding.pdfLokendra Shastri ICSI, Berkeley The Binding Problem Massively Parallel](https://reader034.vdocuments.mx/reader034/viewer/2022051802/5af105207f8b9a8c308df826/html5/thumbnails/33.jpg)
Lokendra Shastri Lokendra Shastri ICSI, Berkeley ICSI, Berkeley
Focal-clusters with intra-cluster links
JohnJohn
+ ?
+ - ? fall-pat fall-loc
FALLFALL
+e +v ?v ?e
PersonPerson
Shruti always seeks explanations
![Page 34: The Binding Problem - inst.eecs.berkeley.eduinst.eecs.berkeley.edu/~cs182/sp07/notes/lecture22-binding.pdfLokendra Shastri ICSI, Berkeley The Binding Problem Massively Parallel](https://reader034.vdocuments.mx/reader034/viewer/2022051802/5af105207f8b9a8c308df826/html5/thumbnails/34.jpg)
Lokendra Shastri Lokendra Shastri ICSI, Berkeley ICSI, Berkeley
Encoding “slip => fall” in Shruti
SLIPSLIP + - ? slip-pat slip-loc
FALLFALL + - ? fall-pat fall-loc
+ ? r1 r2
mediatormediator
![Page 35: The Binding Problem - inst.eecs.berkeley.eduinst.eecs.berkeley.edu/~cs182/sp07/notes/lecture22-binding.pdfLokendra Shastri ICSI, Berkeley The Binding Problem Massively Parallel](https://reader034.vdocuments.mx/reader034/viewer/2022051802/5af105207f8b9a8c308df826/html5/thumbnails/35.jpg)
Lokendra Shastri Lokendra Shastri ICSI, Berkeley ICSI, Berkeley
Linking focal-clusters of types and entities
JohnJohn + ?
+e +v ?v ?e
AgentAgent+e +v ?v ?e
+e +v ?v ?e
+ ?
+e +v ?v ?e
HallwayHallway
LocationLocation
ManMan
PersonPerson
![Page 36: The Binding Problem - inst.eecs.berkeley.eduinst.eecs.berkeley.edu/~cs182/sp07/notes/lecture22-binding.pdfLokendra Shastri ICSI, Berkeley The Binding Problem Massively Parallel](https://reader034.vdocuments.mx/reader034/viewer/2022051802/5af105207f8b9a8c308df826/html5/thumbnails/36.jpg)
Lokendra Shastri Lokendra Shastri ICSI, Berkeley ICSI, Berkeley
Focal-clusters and context-sensitive priors (T-facts)
+ ?
+ - ? fall-pat fall-loc
+e +v ?v ?e
cs-priorscs-priors**
cs-priorscs-priors**
context-sensitive priorscontext-sensitive priors**
* * cortical circuitscortical circuits
entitiesentitiesandandtypestypes
entities and typesentities and types
JohnJohn
FALLFALL
PersonPerson
![Page 37: The Binding Problem - inst.eecs.berkeley.eduinst.eecs.berkeley.edu/~cs182/sp07/notes/lecture22-binding.pdfLokendra Shastri ICSI, Berkeley The Binding Problem Massively Parallel](https://reader034.vdocuments.mx/reader034/viewer/2022051802/5af105207f8b9a8c308df826/html5/thumbnails/37.jpg)
Lokendra Shastri Lokendra Shastri ICSI, Berkeley ICSI, Berkeley
Focal-clusters and episodic memories (E-facts)
+ ?
+ - ? fall-pat fall-loc
+e +v ?v ?e
episodic memoriesepisodic memoriese-memoriese-memories
e-memoriese-memories
fromfromrole-fillersrole-fillers
to role-fillersto role-fillers
** **
**
** hippocampalhippocampal circuitscircuits
JohnJohn
FALLFALL
PersonPerson
![Page 38: The Binding Problem - inst.eecs.berkeley.eduinst.eecs.berkeley.edu/~cs182/sp07/notes/lecture22-binding.pdfLokendra Shastri ICSI, Berkeley The Binding Problem Massively Parallel](https://reader034.vdocuments.mx/reader034/viewer/2022051802/5af105207f8b9a8c308df826/html5/thumbnails/38.jpg)
Lokendra Shastri Lokendra Shastri ICSI, Berkeley ICSI, Berkeley
Explaining away in Shruti
SLIPSLIP
+ ?
FALLFALL + ?
+ ?
TRIPTRIP
+ ? mediatormediator
+ ? mediatormediator
![Page 39: The Binding Problem - inst.eecs.berkeley.eduinst.eecs.berkeley.edu/~cs182/sp07/notes/lecture22-binding.pdfLokendra Shastri ICSI, Berkeley The Binding Problem Massively Parallel](https://reader034.vdocuments.mx/reader034/viewer/2022051802/5af105207f8b9a8c308df826/html5/thumbnails/39.jpg)
Lokendra Shastri Lokendra Shastri ICSI, Berkeley ICSI, Berkeley
Other features of Shruti
• Mutual inhibition between collectors of incompatible entities
• Merging of phases -- unification
• Instantiation of new entities
• Structured priming
![Page 40: The Binding Problem - inst.eecs.berkeley.eduinst.eecs.berkeley.edu/~cs182/sp07/notes/lecture22-binding.pdfLokendra Shastri ICSI, Berkeley The Binding Problem Massively Parallel](https://reader034.vdocuments.mx/reader034/viewer/2022051802/5af105207f8b9a8c308df826/html5/thumbnails/40.jpg)
Lokendra Shastri Lokendra Shastri ICSI, Berkeley ICSI, Berkeley
Unification in Shruti : merging of phases
The activity in focal-clusters of two entity or relational The activity in focal-clusters of two entity or relational
instances will synchronize if there is evidence that instances will synchronize if there is evidence that
the two instances are the samethe two instances are the same
R1: Is there an entity A of type T filling role r in situation PIs there an entity A of type T filling role r in situation P? (Did a man fall in the hallway?)(Did a man fall in the hallway?)
R2: Entity B of type T is filling role r in situation P.Entity B of type T is filling role r in situation P.
(John fell in the hallway.)(John fell in the hallway.)
In such a situation, the firing of A and B will synchronize.
Consequently, A and B will unify, and so will the relational instances involving A and B.
![Page 41: The Binding Problem - inst.eecs.berkeley.eduinst.eecs.berkeley.edu/~cs182/sp07/notes/lecture22-binding.pdfLokendra Shastri ICSI, Berkeley The Binding Problem Massively Parallel](https://reader034.vdocuments.mx/reader034/viewer/2022051802/5af105207f8b9a8c308df826/html5/thumbnails/41.jpg)
Lokendra Shastri Lokendra Shastri ICSI, Berkeley ICSI, Berkeley
Entity instantiation in Shruti
If Shruti encodes the rule-like knowledge:
x:Agent y:Location fall(x,y) => hurt(x)
it automatically posits the existence of a location where John fell in response to the dynamic instantiation of hurt(x)
![Page 42: The Binding Problem - inst.eecs.berkeley.eduinst.eecs.berkeley.edu/~cs182/sp07/notes/lecture22-binding.pdfLokendra Shastri ICSI, Berkeley The Binding Problem Massively Parallel](https://reader034.vdocuments.mx/reader034/viewer/2022051802/5af105207f8b9a8c308df826/html5/thumbnails/42.jpg)
Lokendra Shastri Lokendra Shastri ICSI, Berkeley ICSI, Berkeley
Encoding “fall => hurt” in Shruti
FALLFALL + - ? fall-pat fall-loc
HURTHURT
+ ? r1 r2
mediatormediatortype (semantic)restrictions
role-fillerinstantiation
+ - ? hurt-pat
![Page 43: The Binding Problem - inst.eecs.berkeley.eduinst.eecs.berkeley.edu/~cs182/sp07/notes/lecture22-binding.pdfLokendra Shastri ICSI, Berkeley The Binding Problem Massively Parallel](https://reader034.vdocuments.mx/reader034/viewer/2022051802/5af105207f8b9a8c308df826/html5/thumbnails/43.jpg)
Lokendra Shastri Lokendra Shastri ICSI, Berkeley ICSI, Berkeley
![Page 44: The Binding Problem - inst.eecs.berkeley.eduinst.eecs.berkeley.edu/~cs182/sp07/notes/lecture22-binding.pdfLokendra Shastri ICSI, Berkeley The Binding Problem Massively Parallel](https://reader034.vdocuments.mx/reader034/viewer/2022051802/5af105207f8b9a8c308df826/html5/thumbnails/44.jpg)
Lokendra Shastri Lokendra Shastri ICSI, Berkeley ICSI, Berkeley
The activation trace of +:slip and +:trip
“John fell in the hallway. Tom had cleaned it.He got hurt.”
s2s1 s3
![Page 45: The Binding Problem - inst.eecs.berkeley.eduinst.eecs.berkeley.edu/~cs182/sp07/notes/lecture22-binding.pdfLokendra Shastri ICSI, Berkeley The Binding Problem Massively Parallel](https://reader034.vdocuments.mx/reader034/viewer/2022051802/5af105207f8b9a8c308df826/html5/thumbnails/45.jpg)
Lokendra Shastri Lokendra Shastri ICSI, Berkeley ICSI, Berkeley
![Page 46: The Binding Problem - inst.eecs.berkeley.eduinst.eecs.berkeley.edu/~cs182/sp07/notes/lecture22-binding.pdfLokendra Shastri ICSI, Berkeley The Binding Problem Massively Parallel](https://reader034.vdocuments.mx/reader034/viewer/2022051802/5af105207f8b9a8c308df826/html5/thumbnails/46.jpg)
Lokendra Shastri Lokendra Shastri ICSI, Berkeley ICSI, Berkeley
A Metaphor for Reasoning
• An episode of reflexive reasoning is a transient propagation of rhythmic activity
• Each entity involved in this reasoning episode is a phase in this rhythmic activity
• Bindings are synchronous firings of cell clusters
• Rules are interconnections between cell-clusters that support context-sensitive propagation of activity
• Unification corresponds to merging of phases
• A stable inference (explanation/answer) corresponds to reverberatory activity around closed loops
![Page 47: The Binding Problem - inst.eecs.berkeley.eduinst.eecs.berkeley.edu/~cs182/sp07/notes/lecture22-binding.pdfLokendra Shastri ICSI, Berkeley The Binding Problem Massively Parallel](https://reader034.vdocuments.mx/reader034/viewer/2022051802/5af105207f8b9a8c308df826/html5/thumbnails/47.jpg)
Lokendra Shastri Lokendra Shastri ICSI, Berkeley ICSI, Berkeley
Support for Shruti
• Neurophysiological evidence: transient synchro-nization of cell firing might encode dynamic bindings
• Makes plausible predictions about working memory limitations
• Speed of inference satisfies performance requirements of language understanding
• Representational assumptions are compatible with a biologically realistic model of episodic memory
![Page 48: The Binding Problem - inst.eecs.berkeley.eduinst.eecs.berkeley.edu/~cs182/sp07/notes/lecture22-binding.pdfLokendra Shastri ICSI, Berkeley The Binding Problem Massively Parallel](https://reader034.vdocuments.mx/reader034/viewer/2022051802/5af105207f8b9a8c308df826/html5/thumbnails/48.jpg)
Lokendra Shastri Lokendra Shastri ICSI, Berkeley ICSI, Berkeley
Neurophysiological evidence for synchrony
• Synchronous activity found in anesthetized cat as well as in anesthetized and awake monkey.
• Spatially distributed cells exhibit synchronous activity if they represent information about the same object.
• Synchronous activity occurs in the gamma band (25--60Hz) (maximum period of about 40 msec.)
• frequency drifts by 5-10Hz, but synchronization stays stable for 100-300 msec
• In humans EEG and MEG signals exhibit power spectrum shifts consistent with synchronization of cell ensembles– orienting or investigatory behavior; delayed-match-to- sample task;
visuo-spatial working memory task
![Page 49: The Binding Problem - inst.eecs.berkeley.eduinst.eecs.berkeley.edu/~cs182/sp07/notes/lecture22-binding.pdfLokendra Shastri ICSI, Berkeley The Binding Problem Massively Parallel](https://reader034.vdocuments.mx/reader034/viewer/2022051802/5af105207f8b9a8c308df826/html5/thumbnails/49.jpg)
Lokendra Shastri Lokendra Shastri ICSI, Berkeley ICSI, Berkeley
Predictions: constraints on reflexive inference
• gamma band activity (25-60Hz) underlies dynamic bindings (the maximum period ~40 msec.)
• allowable jitter in synchronous firing 3 msec. lead/lag.
⇒ only a small number of distinct conceptual only a small number of distinct conceptual entities can participate in an episode of entities can participate in an episode of reasoningreasoning
7 +/- 27 +/- 2 (40 divided by 6)
as the number of entities increases beyond five, their activity starts overlapping, leading to cross-talk
Note: Not a limit on the number of co-active bindings!Note: Not a limit on the number of co-active bindings!
![Page 50: The Binding Problem - inst.eecs.berkeley.eduinst.eecs.berkeley.edu/~cs182/sp07/notes/lecture22-binding.pdfLokendra Shastri ICSI, Berkeley The Binding Problem Massively Parallel](https://reader034.vdocuments.mx/reader034/viewer/2022051802/5af105207f8b9a8c308df826/html5/thumbnails/50.jpg)
Lokendra Shastri Lokendra Shastri ICSI, Berkeley ICSI, Berkeley
Predictions: Constraints on reflexive reasoning
1.1. A large number of relational instances (facts) can be A large number of relational instances (facts) can be co-active, and numerous rules can fire in parallel, but co-active, and numerous rules can fire in parallel, but
2.2. only a small number of distinct entities can serve as only a small number of distinct entities can serve as role-fillers in this activityrole-fillers in this activity
3.3. only a small number of instances of the same only a small number of instances of the same predicate can be co-active at the same timepredicate can be co-active at the same time
4.4. the depth of inference is bounded – systematic the depth of inference is bounded – systematic reasoning via binding propagation degrades to a mere reasoning via binding propagation degrades to a mere spreading of activation beyond a certain depth.spreading of activation beyond a certain depth.
2 and 3 specify limits on Shruti’s working memory2 and 3 specify limits on Shruti’s working memory
![Page 51: The Binding Problem - inst.eecs.berkeley.eduinst.eecs.berkeley.edu/~cs182/sp07/notes/lecture22-binding.pdfLokendra Shastri ICSI, Berkeley The Binding Problem Massively Parallel](https://reader034.vdocuments.mx/reader034/viewer/2022051802/5af105207f8b9a8c308df826/html5/thumbnails/51.jpg)
Lokendra Shastri Lokendra Shastri ICSI, Berkeley ICSI, Berkeley
Massively Parallel Inference
• if gamma band activity underlies propagation of bindings
• each binding propagation step takes ca. 25 msec.
• inferring “John may be hurt” and “John may have slipped” from “John fell” would take only ca. 200 msec.
• time required to perform inference is independent of the size of the causal model
![Page 52: The Binding Problem - inst.eecs.berkeley.eduinst.eecs.berkeley.edu/~cs182/sp07/notes/lecture22-binding.pdfLokendra Shastri ICSI, Berkeley The Binding Problem Massively Parallel](https://reader034.vdocuments.mx/reader034/viewer/2022051802/5af105207f8b9a8c308df826/html5/thumbnails/52.jpg)
Lokendra Shastri Lokendra Shastri ICSI, Berkeley ICSI, Berkeley
Probabilistic interpretation of link weights
P(C/E) = P(E/C) P(C)/P(E)
![Page 53: The Binding Problem - inst.eecs.berkeley.eduinst.eecs.berkeley.edu/~cs182/sp07/notes/lecture22-binding.pdfLokendra Shastri ICSI, Berkeley The Binding Problem Massively Parallel](https://reader034.vdocuments.mx/reader034/viewer/2022051802/5af105207f8b9a8c308df826/html5/thumbnails/53.jpg)
Lokendra Shastri Lokendra Shastri ICSI, Berkeley ICSI, Berkeley
![Page 54: The Binding Problem - inst.eecs.berkeley.eduinst.eecs.berkeley.edu/~cs182/sp07/notes/lecture22-binding.pdfLokendra Shastri ICSI, Berkeley The Binding Problem Massively Parallel](https://reader034.vdocuments.mx/reader034/viewer/2022051802/5af105207f8b9a8c308df826/html5/thumbnails/54.jpg)
Lokendra Shastri Lokendra Shastri ICSI, Berkeley ICSI, Berkeley
Encoding X-schema
![Page 55: The Binding Problem - inst.eecs.berkeley.eduinst.eecs.berkeley.edu/~cs182/sp07/notes/lecture22-binding.pdfLokendra Shastri ICSI, Berkeley The Binding Problem Massively Parallel](https://reader034.vdocuments.mx/reader034/viewer/2022051802/5af105207f8b9a8c308df826/html5/thumbnails/55.jpg)
Lokendra Shastri Lokendra Shastri ICSI, Berkeley ICSI, Berkeley
![Page 56: The Binding Problem - inst.eecs.berkeley.eduinst.eecs.berkeley.edu/~cs182/sp07/notes/lecture22-binding.pdfLokendra Shastri ICSI, Berkeley The Binding Problem Massively Parallel](https://reader034.vdocuments.mx/reader034/viewer/2022051802/5af105207f8b9a8c308df826/html5/thumbnails/56.jpg)
Lokendra Shastri Lokendra Shastri ICSI, Berkeley ICSI, Berkeley
![Page 57: The Binding Problem - inst.eecs.berkeley.eduinst.eecs.berkeley.edu/~cs182/sp07/notes/lecture22-binding.pdfLokendra Shastri ICSI, Berkeley The Binding Problem Massively Parallel](https://reader034.vdocuments.mx/reader034/viewer/2022051802/5af105207f8b9a8c308df826/html5/thumbnails/57.jpg)
Lokendra Shastri Lokendra Shastri ICSI, Berkeley ICSI, Berkeley
![Page 58: The Binding Problem - inst.eecs.berkeley.eduinst.eecs.berkeley.edu/~cs182/sp07/notes/lecture22-binding.pdfLokendra Shastri ICSI, Berkeley The Binding Problem Massively Parallel](https://reader034.vdocuments.mx/reader034/viewer/2022051802/5af105207f8b9a8c308df826/html5/thumbnails/58.jpg)
Lokendra Shastri Lokendra Shastri ICSI, Berkeley ICSI, Berkeley
Proposed Alternative Solution
• Indirect references– Pass short signatures, “fluents”
• Functionally similar to SHRUTI's time slices
– Central “binder” maps fluents to objects• In SHRUTI, the objects fired in that time slice
– Connections need to be more complicated than in SHRUTI
• Fluents are passed through at least 3 bits• But temporal synchrony is not required
![Page 59: The Binding Problem - inst.eecs.berkeley.eduinst.eecs.berkeley.edu/~cs182/sp07/notes/lecture22-binding.pdfLokendra Shastri ICSI, Berkeley The Binding Problem Massively Parallel](https://reader034.vdocuments.mx/reader034/viewer/2022051802/5af105207f8b9a8c308df826/html5/thumbnails/59.jpg)
Lokendra Shastri Lokendra Shastri ICSI, Berkeley ICSI, Berkeley
Components of the System
• Object references– Fluents– Binder
• Short term storage– Predicate state
• Long term storage– Facts, mediators, what predicates exist
• Inference– Mediators
• Types– Ontology
![Page 60: The Binding Problem - inst.eecs.berkeley.eduinst.eecs.berkeley.edu/~cs182/sp07/notes/lecture22-binding.pdfLokendra Shastri ICSI, Berkeley The Binding Problem Massively Parallel](https://reader034.vdocuments.mx/reader034/viewer/2022051802/5af105207f8b9a8c308df826/html5/thumbnails/60.jpg)
Lokendra Shastri Lokendra Shastri ICSI, Berkeley ICSI, Berkeley
Fluents:
• Roles are just patterns of activation 3-4 bits
![Page 61: The Binding Problem - inst.eecs.berkeley.eduinst.eecs.berkeley.edu/~cs182/sp07/notes/lecture22-binding.pdfLokendra Shastri ICSI, Berkeley The Binding Problem Massively Parallel](https://reader034.vdocuments.mx/reader034/viewer/2022051802/5af105207f8b9a8c308df826/html5/thumbnails/61.jpg)
Lokendra Shastri Lokendra Shastri ICSI, Berkeley ICSI, Berkeley
Binder:
• What does the pattern mean?– The binder gives fluent patterns meaning
![Page 62: The Binding Problem - inst.eecs.berkeley.eduinst.eecs.berkeley.edu/~cs182/sp07/notes/lecture22-binding.pdfLokendra Shastri ICSI, Berkeley The Binding Problem Massively Parallel](https://reader034.vdocuments.mx/reader034/viewer/2022051802/5af105207f8b9a8c308df826/html5/thumbnails/62.jpg)
Lokendra Shastri Lokendra Shastri ICSI, Berkeley ICSI, Berkeley
Predicates:
• Represent short term beliefs about the world
• Basic unit of inference
Negative Collector
PositiveCollector
QuestionNode
Role Nodes (hold fluents)
![Page 63: The Binding Problem - inst.eecs.berkeley.eduinst.eecs.berkeley.edu/~cs182/sp07/notes/lecture22-binding.pdfLokendra Shastri ICSI, Berkeley The Binding Problem Massively Parallel](https://reader034.vdocuments.mx/reader034/viewer/2022051802/5af105207f8b9a8c308df826/html5/thumbnails/63.jpg)
Lokendra Shastri Lokendra Shastri ICSI, Berkeley ICSI, Berkeley
Facts:
• Support or refute belief in a specific set of bindings of a given predicate
![Page 64: The Binding Problem - inst.eecs.berkeley.eduinst.eecs.berkeley.edu/~cs182/sp07/notes/lecture22-binding.pdfLokendra Shastri ICSI, Berkeley The Binding Problem Massively Parallel](https://reader034.vdocuments.mx/reader034/viewer/2022051802/5af105207f8b9a8c308df826/html5/thumbnails/64.jpg)
Lokendra Shastri Lokendra Shastri ICSI, Berkeley ICSI, Berkeley
Inference:• Connections between predicates form evidential
links– Big(x) & CanBite(x) => Scary(x)– Poisonous(x) & CanBite(x) => Scary(x)– Strength of connections and shape of neuron response
curve determines exactly what “evidence” means
• Direct connections won't work– Consider Big(f1) & Poisonous(f1)– We want to “Or” over a number of “And”s
![Page 65: The Binding Problem - inst.eecs.berkeley.eduinst.eecs.berkeley.edu/~cs182/sp07/notes/lecture22-binding.pdfLokendra Shastri ICSI, Berkeley The Binding Problem Massively Parallel](https://reader034.vdocuments.mx/reader034/viewer/2022051802/5af105207f8b9a8c308df826/html5/thumbnails/65.jpg)
Lokendra Shastri Lokendra Shastri ICSI, Berkeley ICSI, Berkeley
Solution: Mediators
• Multiple antecedents
• Role consistency
![Page 66: The Binding Problem - inst.eecs.berkeley.eduinst.eecs.berkeley.edu/~cs182/sp07/notes/lecture22-binding.pdfLokendra Shastri ICSI, Berkeley The Binding Problem Massively Parallel](https://reader034.vdocuments.mx/reader034/viewer/2022051802/5af105207f8b9a8c308df826/html5/thumbnails/66.jpg)
Lokendra Shastri Lokendra Shastri ICSI, Berkeley ICSI, Berkeley
Mediators (continued)
![Page 67: The Binding Problem - inst.eecs.berkeley.eduinst.eecs.berkeley.edu/~cs182/sp07/notes/lecture22-binding.pdfLokendra Shastri ICSI, Berkeley The Binding Problem Massively Parallel](https://reader034.vdocuments.mx/reader034/viewer/2022051802/5af105207f8b9a8c308df826/html5/thumbnails/67.jpg)
Lokendra Shastri Lokendra Shastri ICSI, Berkeley ICSI, Berkeley
Fluents:
• Roles are just patterns of activation 3-4 bits
![Page 68: The Binding Problem - inst.eecs.berkeley.eduinst.eecs.berkeley.edu/~cs182/sp07/notes/lecture22-binding.pdfLokendra Shastri ICSI, Berkeley The Binding Problem Massively Parallel](https://reader034.vdocuments.mx/reader034/viewer/2022051802/5af105207f8b9a8c308df826/html5/thumbnails/68.jpg)
Lokendra Shastri Lokendra Shastri ICSI, Berkeley ICSI, Berkeley
Binder:
• What does the pattern mean?– The binder gives fluent patterns meaning
![Page 69: The Binding Problem - inst.eecs.berkeley.eduinst.eecs.berkeley.edu/~cs182/sp07/notes/lecture22-binding.pdfLokendra Shastri ICSI, Berkeley The Binding Problem Massively Parallel](https://reader034.vdocuments.mx/reader034/viewer/2022051802/5af105207f8b9a8c308df826/html5/thumbnails/69.jpg)
Lokendra Shastri Lokendra Shastri ICSI, Berkeley ICSI, Berkeley
Multiple Assertions
• As described so far, the system cannot simultaneously represent Big(f1) and Big(f2)
• Solution– Multiple instances of predicates– Requires more complex connections
• Signals must pass only between clusters with matching fluents
• Questions must requisition an appropriate number of clusters
![Page 70: The Binding Problem - inst.eecs.berkeley.eduinst.eecs.berkeley.edu/~cs182/sp07/notes/lecture22-binding.pdfLokendra Shastri ICSI, Berkeley The Binding Problem Massively Parallel](https://reader034.vdocuments.mx/reader034/viewer/2022051802/5af105207f8b9a8c308df826/html5/thumbnails/70.jpg)
Lokendra Shastri Lokendra Shastri ICSI, Berkeley ICSI, Berkeley
Multiple Assertions (detail)
• Connections between Predicates and their evidence Mediators are easy 1-1
![Page 71: The Binding Problem - inst.eecs.berkeley.eduinst.eecs.berkeley.edu/~cs182/sp07/notes/lecture22-binding.pdfLokendra Shastri ICSI, Berkeley The Binding Problem Massively Parallel](https://reader034.vdocuments.mx/reader034/viewer/2022051802/5af105207f8b9a8c308df826/html5/thumbnails/71.jpg)
Lokendra Shastri Lokendra Shastri ICSI, Berkeley ICSI, Berkeley
Multiple Assertions (detail)
• Connections between Predicates and their evidence Mediators are easy 1-1
• Evidential connections of Mediators and their evidence Predicates are easy– Just connect + and - nodes dependent on
matching fluents
• Questions going between Mediators and evidence Predicates are hard– Add a selection network to deal with one
question at a time
![Page 72: The Binding Problem - inst.eecs.berkeley.eduinst.eecs.berkeley.edu/~cs182/sp07/notes/lecture22-binding.pdfLokendra Shastri ICSI, Berkeley The Binding Problem Massively Parallel](https://reader034.vdocuments.mx/reader034/viewer/2022051802/5af105207f8b9a8c308df826/html5/thumbnails/72.jpg)
Lokendra Shastri Lokendra Shastri ICSI, Berkeley ICSI, Berkeley
Components of the System
• Object references– Fluents– Binder
• Short term storage– Predicate state
• Long term storage– Facts, mediators, what predicates exist
• Inference– Mediators
• Types– Ontology
![Page 73: The Binding Problem - inst.eecs.berkeley.eduinst.eecs.berkeley.edu/~cs182/sp07/notes/lecture22-binding.pdfLokendra Shastri ICSI, Berkeley The Binding Problem Massively Parallel](https://reader034.vdocuments.mx/reader034/viewer/2022051802/5af105207f8b9a8c308df826/html5/thumbnails/73.jpg)
Lokendra Shastri Lokendra Shastri ICSI, Berkeley ICSI, Berkeley
Limitations
• Size of network is linear with knowledge base
• Short-term knowledge limited by number of fluents
• Depth of inference limited in time
• Number of same assertions limited
• Inference only goes entirely correctly with ground instances (e.g. “Fido” and not “dog”)
![Page 74: The Binding Problem - inst.eecs.berkeley.eduinst.eecs.berkeley.edu/~cs182/sp07/notes/lecture22-binding.pdfLokendra Shastri ICSI, Berkeley The Binding Problem Massively Parallel](https://reader034.vdocuments.mx/reader034/viewer/2022051802/5af105207f8b9a8c308df826/html5/thumbnails/74.jpg)
Lokendra Shastri Lokendra Shastri ICSI, Berkeley ICSI, Berkeley
Questions
![Page 75: The Binding Problem - inst.eecs.berkeley.eduinst.eecs.berkeley.edu/~cs182/sp07/notes/lecture22-binding.pdfLokendra Shastri ICSI, Berkeley The Binding Problem Massively Parallel](https://reader034.vdocuments.mx/reader034/viewer/2022051802/5af105207f8b9a8c308df826/html5/thumbnails/75.jpg)
Lokendra Shastri Lokendra Shastri ICSI, Berkeley ICSI, Berkeley
Representing belief and utility in Shruti
• associate utilities with states of affairs (relational instances)
• encode utility facts:
– context sensitive memories of utilities associated with certain events or event-types
• propagate utility along causal structures
• encode actions and their consequences
![Page 76: The Binding Problem - inst.eecs.berkeley.eduinst.eecs.berkeley.edu/~cs182/sp07/notes/lecture22-binding.pdfLokendra Shastri ICSI, Berkeley The Binding Problem Massively Parallel](https://reader034.vdocuments.mx/reader034/viewer/2022051802/5af105207f8b9a8c308df826/html5/thumbnails/76.jpg)
Lokendra Shastri Lokendra Shastri ICSI, Berkeley ICSI, Berkeley
Encoding “Fall => Hurt”
FallFall+ $ + $ -- $ $ ?? patient location
HurtHurt
+ $ + $ -- $ $ ?? patient location
mediatormediator ++ $$ ?? r1 r2r1 r2
![Page 77: The Binding Problem - inst.eecs.berkeley.eduinst.eecs.berkeley.edu/~cs182/sp07/notes/lecture22-binding.pdfLokendra Shastri ICSI, Berkeley The Binding Problem Massively Parallel](https://reader034.vdocuments.mx/reader034/viewer/2022051802/5af105207f8b9a8c308df826/html5/thumbnails/77.jpg)
Lokendra Shastri Lokendra Shastri ICSI, Berkeley ICSI, Berkeley
Focal-clusters augmented to encode belief and utility
AttackAttack
+ p n - p n ? target location
UF
to roles and role-fillers
from roles and role-fillers
*
*UF: utility fact; either a specific reward fact (R-fact) or a generic value fact (V-fact)
![Page 78: The Binding Problem - inst.eecs.berkeley.eduinst.eecs.berkeley.edu/~cs182/sp07/notes/lecture22-binding.pdfLokendra Shastri ICSI, Berkeley The Binding Problem Massively Parallel](https://reader034.vdocuments.mx/reader034/viewer/2022051802/5af105207f8b9a8c308df826/html5/thumbnails/78.jpg)
Lokendra Shastri Lokendra Shastri ICSI, Berkeley ICSI, Berkeley
Behavior of augmented Shruti
Shruti reflexively
• Makes observations
• Seeks explanations
• Makes predictions
• Instantiates goals
• Seeks plans that enhance expected future utility– identify actions that are likely to lead to
desirable situations and prevent undesirable ones
![Page 79: The Binding Problem - inst.eecs.berkeley.eduinst.eecs.berkeley.edu/~cs182/sp07/notes/lecture22-binding.pdfLokendra Shastri ICSI, Berkeley The Binding Problem Massively Parallel](https://reader034.vdocuments.mx/reader034/viewer/2022051802/5af105207f8b9a8c308df826/html5/thumbnails/79.jpg)
Lokendra Shastri Lokendra Shastri ICSI, Berkeley ICSI, Berkeley
Shruti suggests how different sorts of knowledge may be encoded within neurally plausible networks
• Entities, types and their relationships (John is a Man)
• Relational schemas/frames corresponding to action and event types (Falling, giving, …)
• Causal relations between relational schemas (If you fall you can get hurt)
• Taxon/Semantic facts (Children often fall)
• Episodic facts (John fell in the hallway on Monday)
• Utility facts (It is bad to be hurt) Lokendra Shastri Lokendra Shastri ICSI, Berkeley ICSI, Berkeley
![Page 80: The Binding Problem - inst.eecs.berkeley.eduinst.eecs.berkeley.edu/~cs182/sp07/notes/lecture22-binding.pdfLokendra Shastri ICSI, Berkeley The Binding Problem Massively Parallel](https://reader034.vdocuments.mx/reader034/viewer/2022051802/5af105207f8b9a8c308df826/html5/thumbnails/80.jpg)
Lokendra Shastri Lokendra Shastri ICSI, Berkeley ICSI, Berkeley
Current status of learning in Shruti
Episodic facts: A biologically grounded model of “one-shot” episodic memory formation• Shastri, 1997; Proceedings of CogSci 1997
• _2001; Neurocomputing
• _2002; Trends in Cognitive Science
• _In Revision; Behavioral and Brain Science
(available as a Technical Report)
Lokendra Shastri Lokendra Shastri ICSI, Berkeley ICSI, Berkeley
![Page 81: The Binding Problem - inst.eecs.berkeley.eduinst.eecs.berkeley.edu/~cs182/sp07/notes/lecture22-binding.pdfLokendra Shastri ICSI, Berkeley The Binding Problem Massively Parallel](https://reader034.vdocuments.mx/reader034/viewer/2022051802/5af105207f8b9a8c308df826/html5/thumbnails/81.jpg)
Lokendra Shastri Lokendra Shastri ICSI, Berkeley ICSI, Berkeley
…current status of learning in Shruti
Work in Progress
• Causal rules
• Categories
• Relational schemasShastri and Wendelken 2003; Neurocomputing
Lokendra Shastri Lokendra Shastri ICSI, Berkeley ICSI, Berkeley
![Page 82: The Binding Problem - inst.eecs.berkeley.eduinst.eecs.berkeley.edu/~cs182/sp07/notes/lecture22-binding.pdfLokendra Shastri ICSI, Berkeley The Binding Problem Massively Parallel](https://reader034.vdocuments.mx/reader034/viewer/2022051802/5af105207f8b9a8c308df826/html5/thumbnails/82.jpg)
Lokendra Shastri Lokendra Shastri ICSI, Berkeley ICSI, Berkeley
![Page 83: The Binding Problem - inst.eecs.berkeley.eduinst.eecs.berkeley.edu/~cs182/sp07/notes/lecture22-binding.pdfLokendra Shastri ICSI, Berkeley The Binding Problem Massively Parallel](https://reader034.vdocuments.mx/reader034/viewer/2022051802/5af105207f8b9a8c308df826/html5/thumbnails/83.jpg)
Lokendra Shastri Lokendra Shastri ICSI, Berkeley ICSI, Berkeley
Questions