![Page 1: Deictic Pronoun Learning and Mirror Self- Identification](https://reader035.vdocuments.mx/reader035/viewer/2022062315/56816493550346895dd66661/html5/thumbnails/1.jpg)
A paper by Kevin Gold and Brian Scassellati
Presented by Paul Dilley
Deictic Pronoun Learning and Mirror Self-Identification
![Page 2: Deictic Pronoun Learning and Mirror Self- Identification](https://reader035.vdocuments.mx/reader035/viewer/2022062315/56816493550346895dd66661/html5/thumbnails/2.jpg)
Paper’s focusTwo milestones in human reasoning about the
self:
1. Mirror self-identification
2. Using deictic pronouns such as “I” and “You”
Building a robot to do these – in order to answer questions about the two
milestones
![Page 3: Deictic Pronoun Learning and Mirror Self- Identification](https://reader035.vdocuments.mx/reader035/viewer/2022062315/56816493550346895dd66661/html5/thumbnails/3.jpg)
• Few animals recognise themselves in the mirror
• Social intelligence needed to do so – assumption
Bonbo
s Chimpan
zee
sOran
gutan
s
Gorilla
s Human
s
Dolphin
s
Orca
sElep
han
tsMag
pie
sBarn
owls
PigsMirror self-recognition
![Page 4: Deictic Pronoun Learning and Mirror Self- Identification](https://reader035.vdocuments.mx/reader035/viewer/2022062315/56816493550346895dd66661/html5/thumbnails/4.jpg)
Mirror Test (Mirror Test.mp4)
Mark placed on being secretly
Mirror placed in front of being
It moves limbs to the mark?
Requires “Mirror Test.mp4” to be in the same directory:
Or watch the embedded movie in a media player:
Click here if the movie does not play
Fin!
How is the mirror test done?
![Page 5: Deictic Pronoun Learning and Mirror Self- Identification](https://reader035.vdocuments.mx/reader035/viewer/2022062315/56816493550346895dd66661/html5/thumbnails/5.jpg)
The mirror test and “I”Problems:• Does the mirror test = Social intelligence?
• How different is the intelligence of using “I” vs. touching a mark, during a mirror test?
“I” and “You” are known as deictic pronouns
“Dik-tik”
![Page 6: Deictic Pronoun Learning and Mirror Self- Identification](https://reader035.vdocuments.mx/reader035/viewer/2022062315/56816493550346895dd66661/html5/thumbnails/6.jpg)
Build a robot?The paper author’s built a robot that can
perform self mirror recognition and apply deictic pronouns to the reflection
Help answer questions on previous slide about human intelligence
Provide an insight for intelligent robot building
![Page 7: Deictic Pronoun Learning and Mirror Self- Identification](https://reader035.vdocuments.mx/reader035/viewer/2022062315/56816493550346895dd66661/html5/thumbnails/7.jpg)
How it works
Audio is captured using adual-channel microphone Robot uses cameras to capture images
Face recognition
Motion Detection
Colour Detection
Self Recognition ModuleFinds motion that coincides with motor
control commands
Speech recognition
Sound localisation
Word learning
![Page 8: Deictic Pronoun Learning and Mirror Self- Identification](https://reader035.vdocuments.mx/reader035/viewer/2022062315/56816493550346895dd66661/html5/thumbnails/8.jpg)
How it worksWord learning
Identify already understood words
Match agents with the properties of understood words
Eventual learning of “I” and “you”
Property TypesActions – “speaking”
Being the target of actions – “addressee”
1. No grammatical parsing, sentences are treated as simple collections of words
2. Each word learnt with confidence adds an associated property to a list of properties to look for in the
environment
3. Each agent is then matched
The process is done using chi-squares
![Page 9: Deictic Pronoun Learning and Mirror Self- Identification](https://reader035.vdocuments.mx/reader035/viewer/2022062315/56816493550346895dd66661/html5/thumbnails/9.jpg)
Self IdentificationExploratory arm movements + studying the
time delay for motion detection
These experimental values are put onto a distribution with 95% falling between 0.5 to 1 seconds
Future movements in this time frame had the moving object labelled as “self”
![Page 10: Deictic Pronoun Learning and Mirror Self- Identification](https://reader035.vdocuments.mx/reader035/viewer/2022062315/56816493550346895dd66661/html5/thumbnails/10.jpg)
Who has the ball?The robot goes into speaker mode and
prepares to address the questioner
Nico searches for utterances of learnt words, e.g. “got”
This is linked to an associated action “hasBall”
Nico then searches for a property that is unassociated with “hasBall” but true for the associated agent
![Page 11: Deictic Pronoun Learning and Mirror Self- Identification](https://reader035.vdocuments.mx/reader035/viewer/2022062315/56816493550346895dd66661/html5/thumbnails/11.jpg)
Implementation
The robot learnt words-property pairs observing
two people play a game of catch with a dataset of 50
utterances
“You” became associated with the addressee and
“I” with the speaker
![Page 12: Deictic Pronoun Learning and Mirror Self- Identification](https://reader035.vdocuments.mx/reader035/viewer/2022062315/56816493550346895dd66661/html5/thumbnails/12.jpg)
ImplementationNico was placed in front of a mirror, with the ball
either at his base or being held by the experimenter
Nico correctly answered “I got the ball” 16 out f 20
times.
The incorrect answer “You got...” was given twice,
and the robot misunderstood twice
![Page 13: Deictic Pronoun Learning and Mirror Self- Identification](https://reader035.vdocuments.mx/reader035/viewer/2022062315/56816493550346895dd66661/html5/thumbnails/13.jpg)
ImplementationNico got confused when the experimenter
was holding the ball
He moved within Nico’s arm test confusing the bot
“You got the ball” was only correct 11/20 times
When the bot was told to stop fidgeting the result was much better: 18/20 times correct
![Page 14: Deictic Pronoun Learning and Mirror Self- Identification](https://reader035.vdocuments.mx/reader035/viewer/2022062315/56816493550346895dd66661/html5/thumbnails/14.jpg)
Conclusions
Bot learns “I” and “you” from observation and applies them correctly
It’s use of “I” for a mirror image is an exciting step for
robotics
It does not represent “self-awareness” from a
scientific point of view – but is useful for
understanding behaviours
Importantly, the bot can associate words with the
actions in the environment – rather than just visual
properties
The authors believe: Applying words with functions and roles is more useful than with
superficial appearance
The authors believe: Allowing robots to
map human actions and properties onto
robotic states in general would be the next step for research
![Page 15: Deictic Pronoun Learning and Mirror Self- Identification](https://reader035.vdocuments.mx/reader035/viewer/2022062315/56816493550346895dd66661/html5/thumbnails/15.jpg)
Recognising in the mirrorMost crucially: Nico could recognise its own feedback as self-generated – but a pre-programmed kinetic model, when the robot knows where its arm is, would probably suffice. It would require less training
The authors conclude:• The mirror test might be a better test of recognising feedback
• It’s not that useful: “why waste neurons on it” – but it has benefits to humans, but may not be as significant to intelligence as previously thought
• But asking a robot “Who is that in the mirror?” and answers such as “I” do touch on several different aspects of human intelligence
What do I think about the paper?
![Page 16: Deictic Pronoun Learning and Mirror Self- Identification](https://reader035.vdocuments.mx/reader035/viewer/2022062315/56816493550346895dd66661/html5/thumbnails/16.jpg)
Most great apes can pass the mirror test by
adult hood
Thanks for listening!