![Page 1: Sensory-motor integration Bijan Pesaran May 1 st, 2008](https://reader036.vdocuments.mx/reader036/viewer/2022081514/56649d6c5503460f94a4c84e/html5/thumbnails/1.jpg)
Sensory-motor integration
Bijan Pesaran
May 1st, 2008
![Page 2: Sensory-motor integration Bijan Pesaran May 1 st, 2008](https://reader036.vdocuments.mx/reader036/viewer/2022081514/56649d6c5503460f94a4c84e/html5/thumbnails/2.jpg)
![Page 3: Sensory-motor integration Bijan Pesaran May 1 st, 2008](https://reader036.vdocuments.mx/reader036/viewer/2022081514/56649d6c5503460f94a4c84e/html5/thumbnails/3.jpg)
Visual response in PMv centered on hand
Graziano and Gross (1994)
![Page 4: Sensory-motor integration Bijan Pesaran May 1 st, 2008](https://reader036.vdocuments.mx/reader036/viewer/2022081514/56649d6c5503460f94a4c84e/html5/thumbnails/4.jpg)
Visual response in PMv centered on hand
Graziano and Gross (1994)
![Page 5: Sensory-motor integration Bijan Pesaran May 1 st, 2008](https://reader036.vdocuments.mx/reader036/viewer/2022081514/56649d6c5503460f94a4c84e/html5/thumbnails/5.jpg)
Duhamel et al (1997)
Invariant visual response in VIP
![Page 6: Sensory-motor integration Bijan Pesaran May 1 st, 2008](https://reader036.vdocuments.mx/reader036/viewer/2022081514/56649d6c5503460f94a4c84e/html5/thumbnails/6.jpg)
Duhamel et al (1997)
![Page 7: Sensory-motor integration Bijan Pesaran May 1 st, 2008](https://reader036.vdocuments.mx/reader036/viewer/2022081514/56649d6c5503460f94a4c84e/html5/thumbnails/7.jpg)
PRR neurons are eye-centered
Batista et al (1999)
![Page 8: Sensory-motor integration Bijan Pesaran May 1 st, 2008](https://reader036.vdocuments.mx/reader036/viewer/2022081514/56649d6c5503460f94a4c84e/html5/thumbnails/8.jpg)
Batista et al (1999)
Updating of response in PRR
![Page 9: Sensory-motor integration Bijan Pesaran May 1 st, 2008](https://reader036.vdocuments.mx/reader036/viewer/2022081514/56649d6c5503460f94a4c84e/html5/thumbnails/9.jpg)
Mechanisms for encoding multiple variables
• Centered tuning curve)(),( HTfTHf
![Page 10: Sensory-motor integration Bijan Pesaran May 1 st, 2008](https://reader036.vdocuments.mx/reader036/viewer/2022081514/56649d6c5503460f94a4c84e/html5/thumbnails/10.jpg)
Eye position influences visual response of 7a neurons
Andersen et al (1985)
![Page 11: Sensory-motor integration Bijan Pesaran May 1 st, 2008](https://reader036.vdocuments.mx/reader036/viewer/2022081514/56649d6c5503460f94a4c84e/html5/thumbnails/11.jpg)
Mechanisms for encoding multiple variables
• Centered tuning curve
• Multiplicative encoding - gain field)()(),( EThEgTEf
Tuning curveGain field
)(),( HTfTHf
![Page 12: Sensory-motor integration Bijan Pesaran May 1 st, 2008](https://reader036.vdocuments.mx/reader036/viewer/2022081514/56649d6c5503460f94a4c84e/html5/thumbnails/12.jpg)
Visual receptive fields in VIP partially shift with eye position
Avillac et al (2005)
Eye positions
![Page 13: Sensory-motor integration Bijan Pesaran May 1 st, 2008](https://reader036.vdocuments.mx/reader036/viewer/2022081514/56649d6c5503460f94a4c84e/html5/thumbnails/13.jpg)
Mechanisms for encoding multiple variables
• Centered tuning curve
• Multiplicative encoding - gain field
• Partial shift encoding – intermediate reference frame
)()(),( EThEgTEf
)()(),( EThEgTEf
Tuning curveGain field
)(),( ETfTEf
1
![Page 14: Sensory-motor integration Bijan Pesaran May 1 st, 2008](https://reader036.vdocuments.mx/reader036/viewer/2022081514/56649d6c5503460f94a4c84e/html5/thumbnails/14.jpg)
Eye position encoding
• Widespread phenomenon throughout visual and motor cortices
Peaked response
Eye leftEye middleEye right
Response
Target position
![Page 15: Sensory-motor integration Bijan Pesaran May 1 st, 2008](https://reader036.vdocuments.mx/reader036/viewer/2022081514/56649d6c5503460f94a4c84e/html5/thumbnails/15.jpg)
Eye position encoding
• Difficulty addressing flanking responses
Peaked response Flanking response
Eye leftEye middleEye right
Response
Target position
![Page 16: Sensory-motor integration Bijan Pesaran May 1 st, 2008](https://reader036.vdocuments.mx/reader036/viewer/2022081514/56649d6c5503460f94a4c84e/html5/thumbnails/16.jpg)
n
nnn ThEgTEf ),(
![Page 17: Sensory-motor integration Bijan Pesaran May 1 st, 2008](https://reader036.vdocuments.mx/reader036/viewer/2022081514/56649d6c5503460f94a4c84e/html5/thumbnails/17.jpg)
ThEgTEf )(),(
Gain model
n
nnn ThEgTEf ),(
![Page 18: Sensory-motor integration Bijan Pesaran May 1 st, 2008](https://reader036.vdocuments.mx/reader036/viewer/2022081514/56649d6c5503460f94a4c84e/html5/thumbnails/18.jpg)
ThEgTEf )(),(
Gain model
n
nnn ThEgTEf ),(
)(),( ETfTEf
Shifting model
![Page 19: Sensory-motor integration Bijan Pesaran May 1 st, 2008](https://reader036.vdocuments.mx/reader036/viewer/2022081514/56649d6c5503460f94a4c84e/html5/thumbnails/19.jpg)
Geometry of visually-guided reaching
• Eye-centered• Hand-centered• Relative position vectors
Nonlinear3D geometry and muscle dynamics
HH
TT
EE
T-ET-E
E-HE-H
H-EH-E
T-HT-H
![Page 20: Sensory-motor integration Bijan Pesaran May 1 st, 2008](https://reader036.vdocuments.mx/reader036/viewer/2022081514/56649d6c5503460f94a4c84e/html5/thumbnails/20.jpg)
Geometry of visually-guided reaching
Nonlinear3D geometry and muscle dynamics
HH
TT
EE
T-ET-E
E-HE-H
H-EH-E
T-HT-H
MIP PMdBatista et al (1999)Eye-centered
Graziano et al (1997)Hand-centered
PMv
What is code?
T-ET-E
T-HT-H
Pesaran et al(2006)
![Page 21: Sensory-motor integration Bijan Pesaran May 1 st, 2008](https://reader036.vdocuments.mx/reader036/viewer/2022081514/56649d6c5503460f94a4c84e/html5/thumbnails/21.jpg)
Decompose the 3-D response matrix
500
exp
1000exp1
1),,(
2ETEH
THEf
![Page 22: Sensory-motor integration Bijan Pesaran May 1 st, 2008](https://reader036.vdocuments.mx/reader036/viewer/2022081514/56649d6c5503460f94a4c84e/html5/thumbnails/22.jpg)
Distinguish between different reference frames
500
exp
1000exp1
1),,(
2HTEH
THEf
![Page 23: Sensory-motor integration Bijan Pesaran May 1 st, 2008](https://reader036.vdocuments.mx/reader036/viewer/2022081514/56649d6c5503460f94a4c84e/html5/thumbnails/23.jpg)
Define multiple effector reference frames
500exp
500exp
500exp),,(
222 EHHTETTHEf
![Page 24: Sensory-motor integration Bijan Pesaran May 1 st, 2008](https://reader036.vdocuments.mx/reader036/viewer/2022081514/56649d6c5503460f94a4c84e/html5/thumbnails/24.jpg)
Multiple reference frames
• Direct visuomotor transformation– Supports Area 5 data (Buneo et al 2002)
Direct Transformation
Serial TransformationHead-on-body
positionEye-in-head
position
Eye-centered
target location
Head-centered
target location
Body-centered
target location
Hand-centered
target location
Body-centeredhand position
+-
Motor output
Eye-centered
target location
Hand-centered
target location
Eye-centeredhand location
+- Nonlinear
3D geometry and muscle dynamics
Motor output
Nonlinear 3D geometry and muscle dynamics
![Page 25: Sensory-motor integration Bijan Pesaran May 1 st, 2008](https://reader036.vdocuments.mx/reader036/viewer/2022081514/56649d6c5503460f94a4c84e/html5/thumbnails/25.jpg)
MIP PMd
Nonlinear muscle
dynamics
Encoding relative position
Nonlinear 3D geometry and muscle dynamics
Batista et al 1999
Graziano et al 1997
H
T
E
T-E
E-H
H-E
T-H
PMv
• Coordinating the hand and the eye
• Eye-centered• Hand-centered• Relative hand-
eye position• Vectors in
abstract space
![Page 26: Sensory-motor integration Bijan Pesaran May 1 st, 2008](https://reader036.vdocuments.mx/reader036/viewer/2022081514/56649d6c5503460f94a4c84e/html5/thumbnails/26.jpg)
Design reference frame task
0 1 2
Start Cue DelayGo
Hold
Time (sec)
![Page 27: Sensory-motor integration Bijan Pesaran May 1 st, 2008](https://reader036.vdocuments.mx/reader036/viewer/2022081514/56649d6c5503460f94a4c84e/html5/thumbnails/27.jpg)
Design reference frame task
0 1 2
Start Cue DelayGo
Hold
Time (sec)
![Page 28: Sensory-motor integration Bijan Pesaran May 1 st, 2008](https://reader036.vdocuments.mx/reader036/viewer/2022081514/56649d6c5503460f94a4c84e/html5/thumbnails/28.jpg)
Design reference frame task
• Four initial eye positions
![Page 29: Sensory-motor integration Bijan Pesaran May 1 st, 2008](https://reader036.vdocuments.mx/reader036/viewer/2022081514/56649d6c5503460f94a4c84e/html5/thumbnails/29.jpg)
Design reference frame task
• Four initial eye and hand positions
![Page 30: Sensory-motor integration Bijan Pesaran May 1 st, 2008](https://reader036.vdocuments.mx/reader036/viewer/2022081514/56649d6c5503460f94a4c84e/html5/thumbnails/30.jpg)
Design reference frame task
• Four initial eye, hand and target positions• 111 PMd neurons and 48 MIP neurons
![Page 31: Sensory-motor integration Bijan Pesaran May 1 st, 2008](https://reader036.vdocuments.mx/reader036/viewer/2022081514/56649d6c5503460f94a4c84e/html5/thumbnails/31.jpg)
![Page 32: Sensory-motor integration Bijan Pesaran May 1 st, 2008](https://reader036.vdocuments.mx/reader036/viewer/2022081514/56649d6c5503460f94a4c84e/html5/thumbnails/32.jpg)
Example PMd neuron
![Page 33: Sensory-motor integration Bijan Pesaran May 1 st, 2008](https://reader036.vdocuments.mx/reader036/viewer/2022081514/56649d6c5503460f94a4c84e/html5/thumbnails/33.jpg)
![Page 34: Sensory-motor integration Bijan Pesaran May 1 st, 2008](https://reader036.vdocuments.mx/reader036/viewer/2022081514/56649d6c5503460f94a4c84e/html5/thumbnails/34.jpg)
Example MIP neuron
![Page 35: Sensory-motor integration Bijan Pesaran May 1 st, 2008](https://reader036.vdocuments.mx/reader036/viewer/2022081514/56649d6c5503460f94a4c84e/html5/thumbnails/35.jpg)
![Page 36: Sensory-motor integration Bijan Pesaran May 1 st, 2008](https://reader036.vdocuments.mx/reader036/viewer/2022081514/56649d6c5503460f94a4c84e/html5/thumbnails/36.jpg)
![Page 37: Sensory-motor integration Bijan Pesaran May 1 st, 2008](https://reader036.vdocuments.mx/reader036/viewer/2022081514/56649d6c5503460f94a4c84e/html5/thumbnails/37.jpg)
Sensory-motor transformations
Head-on-bodyposition
Eye-in-headposition
Eye-centered
target location
Head-centered
target location
Body-centered
target location
Hand-centered
target location
Body-centeredhand position
+-
Serial Transformation
+-
+-
![Page 38: Sensory-motor integration Bijan Pesaran May 1 st, 2008](https://reader036.vdocuments.mx/reader036/viewer/2022081514/56649d6c5503460f94a4c84e/html5/thumbnails/38.jpg)
Sensory-motor transformations
Eye-centeredhand location
Eye-centeredtarget location
Hand-centeredtarget location
-+
Direct Transformation
Buneo et al.(2002)
![Page 39: Sensory-motor integration Bijan Pesaran May 1 st, 2008](https://reader036.vdocuments.mx/reader036/viewer/2022081514/56649d6c5503460f94a4c84e/html5/thumbnails/39.jpg)
Sensory-motor transformations
Eye-centeredtarget location
Hand-centeredtarget location-
+
Relative position Transformations
Eye-centeredtarget location
Hand-centeredtarget location
-
+
Relative location of hand and eye
Eye-centeredhand location
Hand-centeredeye location
![Page 40: Sensory-motor integration Bijan Pesaran May 1 st, 2008](https://reader036.vdocuments.mx/reader036/viewer/2022081514/56649d6c5503460f94a4c84e/html5/thumbnails/40.jpg)
Relative position particularly useful for coordination
• Relative hand-eye position could be important for hand-eye coordination– Can map hand and eye onto each other– Computationally efficient
• No need to code for all combinations of locations
H
T
E
T-E
E-H
H-E
T-H
![Page 41: Sensory-motor integration Bijan Pesaran May 1 st, 2008](https://reader036.vdocuments.mx/reader036/viewer/2022081514/56649d6c5503460f94a4c84e/html5/thumbnails/41.jpg)
Relative position particularly useful for coordination
• Relative hand-eye position could be important for hand-eye coordination– Can map hand and eye onto each other– Computationally efficient
• No need to code for all combinations of locations
H
T
E
T-E
E-H
H-E
T-H
![Page 42: Sensory-motor integration Bijan Pesaran May 1 st, 2008](https://reader036.vdocuments.mx/reader036/viewer/2022081514/56649d6c5503460f94a4c84e/html5/thumbnails/42.jpg)
Relative position particularly useful for coordination
• Relative hand-eye position could be important for hand-eye coordination– Can map hand and eye onto each other– Computationally efficient
• No need to code for all combinations of locations
H
T
E
T-E
E-H
H-E
T-H