decoding information for grasping from the macaque … et al 2017.pdf · 2017. 4. 18. · matteo...

53
Accepted manuscripts are peer-reviewed but have not been through the copyediting, formatting, or proofreading process. Copyright © 2017 the authors This Accepted Manuscript has not been copyedited and formatted. The final version may differ from this version. Research Articles: Systems/Circuits Decoding information for grasping from the macaque dorsomedial visual stream Matteo Filippini 1,* , Rossella Breveglieri 1,* , M. Ali Akhras 1,# , Annalisa Bosco 1 , Eris Chinellato 2 and Patrizia Fattori 1 1 Dept. Pharmacy and Biotechnology, Univ. of Bologna, 40126 Bologna, Italy 2 School of Science and Engineering, Middlesex University, London NW4 4BT, United Kingdom DOI: 10.1523/JNEUROSCI.3077-16.2017 Received: 3 October 2016 Revised: 19 February 2017 Accepted: 22 February 2017 Published: 20 March 2017 Author contributions: M.F., M.A.A., and E.C. analyzed data; M.F., R.B., E.C., and P.F. wrote the paper; R.B. and A.B. performed research; P.F. designed research. *These authors equally contributed to this work This work was supported, by Ministero dell'Universita` e della Ricerca (Italy) Firb 2013 N. RBFR132BKP, and by Fondazione del Monte di Bologna e Ravenna (Italy). We thank N. Marzocchi and G. Placenti for setting up the experimental apparatus.; The authors declare no competing financial interests. Correspondence should be addressed to Prof. Patrizia Fattori, Dept. of Pharmacy and Biotechnology, Universita` di Bologna, Piazza di Porta S. Donato, 2, 40126 Bologna, Italy. E-mail: [email protected] Cite as: J. Neurosci ; 10.1523/JNEUROSCI.3077-16.2017 Alerts: Sign up at www.jneurosci.org/cgi/alerts to receive customized email alerts when the fully formatted version of this article is published.

Upload: others

Post on 19-Mar-2021

1 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Decoding information for grasping from the macaque … et al 2017.pdf · 2017. 4. 18. · Matteo Filippini 1,*, Rossella Breveglieri 1,*, M. Ali Akhras 1,#, Annalisa Bosco 1, Eris

Accepted manuscripts are peer-reviewed but have not been through the copyediting, formatting, or proofreadingprocess.

Copyright © 2017 the authors

This Accepted Manuscript has not been copyedited and formatted. The final version may differ from this version.

Research Articles: Systems/Circuits

Decoding information for grasping from the macaque dorsomedial visual stream

Matteo Filippini1,*, Rossella Breveglieri1,*, M. Ali Akhras1,#, Annalisa Bosco1, Eris Chinellato2 and Patrizia Fattori1

1Dept. Pharmacy and Biotechnology, Univ. of Bologna, 40126 Bologna, Italy2School of Science and Engineering, Middlesex University, London NW4 4BT, United Kingdom

DOI: 10.1523/JNEUROSCI.3077-16.2017

Received: 3 October 2016

Revised: 19 February 2017

Accepted: 22 February 2017

Published: 20 March 2017

Author contributions: M.F., M.A.A., and E.C. analyzed data; M.F., R.B., E.C., and P.F. wrote the paper; R.B.and A.B. performed research; P.F. designed research.

*These authors equally contributed to this work

This work was supported, by Ministero dell'Universita` e della Ricerca (Italy) Firb 2013 N. RBFR132BKP, and byFondazione del Monte di Bologna e Ravenna (Italy). We thank N. Marzocchi and G. Placenti for setting up theexperimental apparatus.; The authors declare no competing financial interests.

Correspondence should be addressed to Prof. Patrizia Fattori, Dept. of Pharmacy and Biotechnology,Universita` di Bologna, Piazza di Porta S. Donato, 2, 40126 Bologna, Italy. E-mail: [email protected]

Cite as: J. Neurosci ; 10.1523/JNEUROSCI.3077-16.2017

Alerts: Sign up at www.jneurosci.org/cgi/alerts to receive customized email alerts when the fully formattedversion of this article is published.

Page 2: Decoding information for grasping from the macaque … et al 2017.pdf · 2017. 4. 18. · Matteo Filippini 1,*, Rossella Breveglieri 1,*, M. Ali Akhras 1,#, Annalisa Bosco 1, Eris
Page 3: Decoding information for grasping from the macaque … et al 2017.pdf · 2017. 4. 18. · Matteo Filippini 1,*, Rossella Breveglieri 1,*, M. Ali Akhras 1,#, Annalisa Bosco 1, Eris
Page 4: Decoding information for grasping from the macaque … et al 2017.pdf · 2017. 4. 18. · Matteo Filippini 1,*, Rossella Breveglieri 1,*, M. Ali Akhras 1,#, Annalisa Bosco 1, Eris
Page 5: Decoding information for grasping from the macaque … et al 2017.pdf · 2017. 4. 18. · Matteo Filippini 1,*, Rossella Breveglieri 1,*, M. Ali Akhras 1,#, Annalisa Bosco 1, Eris
Page 6: Decoding information for grasping from the macaque … et al 2017.pdf · 2017. 4. 18. · Matteo Filippini 1,*, Rossella Breveglieri 1,*, M. Ali Akhras 1,#, Annalisa Bosco 1, Eris

et al. et

al. et al. et al. et al.

et al. et al. et al. et

al. et al. et al.

et al.

et al. et al. et al. et

al. et al. et al. et al.

et al.

et al.

et al.

et al. et

al. et al. et al. et al.

et al. et al.

Page 7: Decoding information for grasping from the macaque … et al 2017.pdf · 2017. 4. 18. · Matteo Filippini 1,*, Rossella Breveglieri 1,*, M. Ali Akhras 1,#, Annalisa Bosco 1, Eris

et al. et al. et al.

et al.

et al.

et

al. et al.

Page 8: Decoding information for grasping from the macaque … et al 2017.pdf · 2017. 4. 18. · Matteo Filippini 1,*, Rossella Breveglieri 1,*, M. Ali Akhras 1,#, Annalisa Bosco 1, Eris
Page 9: Decoding information for grasping from the macaque … et al 2017.pdf · 2017. 4. 18. · Matteo Filippini 1,*, Rossella Breveglieri 1,*, M. Ali Akhras 1,#, Annalisa Bosco 1, Eris

Experimental procedure

Macaca fascicularis

i.v

et al.

i.m. i.m.

et al.

Page 10: Decoding information for grasping from the macaque … et al 2017.pdf · 2017. 4. 18. · Matteo Filippini 1,*, Rossella Breveglieri 1,*, M. Ali Akhras 1,#, Annalisa Bosco 1, Eris

et al. et al.

Behavioral task

et al.

Page 11: Decoding information for grasping from the macaque … et al 2017.pdf · 2017. 4. 18. · Matteo Filippini 1,*, Rossella Breveglieri 1,*, M. Ali Akhras 1,#, Annalisa Bosco 1, Eris

FREE

OBJ-VIS

DELAY

GRASP

Page 12: Decoding information for grasping from the macaque … et al 2017.pdf · 2017. 4. 18. · Matteo Filippini 1,*, Rossella Breveglieri 1,*, M. Ali Akhras 1,#, Annalisa Bosco 1, Eris

FREE

×

et al.

Tested objects

Handle:

finger prehension

Page 13: Decoding information for grasping from the macaque … et al 2017.pdf · 2017. 4. 18. · Matteo Filippini 1,*, Rossella Breveglieri 1,*, M. Ali Akhras 1,#, Annalisa Bosco 1, Eris

Stick-in-groove

advanced

precision grip

Ring

hook grip

Plate

primitive precision grip,

Ball: whole-hand prehension

Data analysis

scikit-learn,

Page 14: Decoding information for grasping from the macaque … et al 2017.pdf · 2017. 4. 18. · Matteo Filippini 1,*, Rossella Breveglieri 1,*, M. Ali Akhras 1,#, Annalisa Bosco 1, Eris

population

response neural decoding

Population response

Page 15: Decoding information for grasping from the macaque … et al 2017.pdf · 2017. 4. 18. · Matteo Filippini 1,*, Rossella Breveglieri 1,*, M. Ali Akhras 1,#, Annalisa Bosco 1, Eris

et al.

Neural Decoding

Page 16: Decoding information for grasping from the macaque … et al 2017.pdf · 2017. 4. 18. · Matteo Filippini 1,*, Rossella Breveglieri 1,*, M. Ali Akhras 1,#, Annalisa Bosco 1, Eris

et al.

et al. et al.

Page 17: Decoding information for grasping from the macaque … et al 2017.pdf · 2017. 4. 18. · Matteo Filippini 1,*, Rossella Breveglieri 1,*, M. Ali Akhras 1,#, Annalisa Bosco 1, Eris

scikit-learn

et al.

time

et al.

Page 18: Decoding information for grasping from the macaque … et al 2017.pdf · 2017. 4. 18. · Matteo Filippini 1,*, Rossella Breveglieri 1,*, M. Ali Akhras 1,#, Annalisa Bosco 1, Eris
Page 19: Decoding information for grasping from the macaque … et al 2017.pdf · 2017. 4. 18. · Matteo Filippini 1,*, Rossella Breveglieri 1,*, M. Ali Akhras 1,#, Annalisa Bosco 1, Eris

et al.

et al.

Page 20: Decoding information for grasping from the macaque … et al 2017.pdf · 2017. 4. 18. · Matteo Filippini 1,*, Rossella Breveglieri 1,*, M. Ali Akhras 1,#, Annalisa Bosco 1, Eris

Decoding results

Page 21: Decoding information for grasping from the macaque … et al 2017.pdf · 2017. 4. 18. · Matteo Filippini 1,*, Rossella Breveglieri 1,*, M. Ali Akhras 1,#, Annalisa Bosco 1, Eris

Object recognition within the object presentation epoch

Page 22: Decoding information for grasping from the macaque … et al 2017.pdf · 2017. 4. 18. · Matteo Filippini 1,*, Rossella Breveglieri 1,*, M. Ali Akhras 1,#, Annalisa Bosco 1, Eris

Decoding the neural information within the planning epoch

Page 23: Decoding information for grasping from the macaque … et al 2017.pdf · 2017. 4. 18. · Matteo Filippini 1,*, Rossella Breveglieri 1,*, M. Ali Akhras 1,#, Annalisa Bosco 1, Eris

Decoding the neural information within the grasping execution epoch

Time-course of the decoding performance

Page 24: Decoding information for grasping from the macaque … et al 2017.pdf · 2017. 4. 18. · Matteo Filippini 1,*, Rossella Breveglieri 1,*, M. Ali Akhras 1,#, Annalisa Bosco 1, Eris
Page 25: Decoding information for grasping from the macaque … et al 2017.pdf · 2017. 4. 18. · Matteo Filippini 1,*, Rossella Breveglieri 1,*, M. Ali Akhras 1,#, Annalisa Bosco 1, Eris

Generalization analysis

Page 26: Decoding information for grasping from the macaque … et al 2017.pdf · 2017. 4. 18. · Matteo Filippini 1,*, Rossella Breveglieri 1,*, M. Ali Akhras 1,#, Annalisa Bosco 1, Eris
Page 27: Decoding information for grasping from the macaque … et al 2017.pdf · 2017. 4. 18. · Matteo Filippini 1,*, Rossella Breveglieri 1,*, M. Ali Akhras 1,#, Annalisa Bosco 1, Eris
Page 28: Decoding information for grasping from the macaque … et al 2017.pdf · 2017. 4. 18. · Matteo Filippini 1,*, Rossella Breveglieri 1,*, M. Ali Akhras 1,#, Annalisa Bosco 1, Eris

et al.

et al.

et al.

Page 29: Decoding information for grasping from the macaque … et al 2017.pdf · 2017. 4. 18. · Matteo Filippini 1,*, Rossella Breveglieri 1,*, M. Ali Akhras 1,#, Annalisa Bosco 1, Eris

et al.

et al.

visual/object

motor/grip

et al. et al.

Page 30: Decoding information for grasping from the macaque … et al 2017.pdf · 2017. 4. 18. · Matteo Filippini 1,*, Rossella Breveglieri 1,*, M. Ali Akhras 1,#, Annalisa Bosco 1, Eris

et al.

et al.

et al.

Offline decoding from single cells in dorso-medial fronto-parietal areas: perspectives

on BCIs

et al. et al

Page 31: Decoding information for grasping from the macaque … et al 2017.pdf · 2017. 4. 18. · Matteo Filippini 1,*, Rossella Breveglieri 1,*, M. Ali Akhras 1,#, Annalisa Bosco 1, Eris

et al.

et al. et al. et al.

et al. et al.

et al.

et al. et al.

et al. et al. et al.

et al.

et al.

et al.

et al.

et al. et al.

Page 32: Decoding information for grasping from the macaque … et al 2017.pdf · 2017. 4. 18. · Matteo Filippini 1,*, Rossella Breveglieri 1,*, M. Ali Akhras 1,#, Annalisa Bosco 1, Eris

et al.

et al.

et al. et al. et

al.

et al.

et al.

Future directions

et al.

et al. et al.

et

al. et al.

et al.

Page 33: Decoding information for grasping from the macaque … et al 2017.pdf · 2017. 4. 18. · Matteo Filippini 1,*, Rossella Breveglieri 1,*, M. Ali Akhras 1,#, Annalisa Bosco 1, Eris

et al. medial

Science

Proceedings of the 31st Annual International Conference of

the IEEE Engineering in Medicine and Biology Society: Engineering the Future

of Biomedicine, EMBC 2009

Page 34: Decoding information for grasping from the macaque … et al 2017.pdf · 2017. 4. 18. · Matteo Filippini 1,*, Rossella Breveglieri 1,*, M. Ali Akhras 1,#, Annalisa Bosco 1, Eris

Annu. Rev. Psychol.

J Neurophysiol

J. Neurosci.

Sci. Rep.

J. Cogn. Neurosci.

Neuroreport

Cereb.

Cortex

Page 35: Decoding information for grasping from the macaque … et al 2017.pdf · 2017. 4. 18. · Matteo Filippini 1,*, Rossella Breveglieri 1,*, M. Ali Akhras 1,#, Annalisa Bosco 1, Eris

PLoS Biol.

J. Neuroeng. Rehabil.

Neuroscience

The Visual Neuroscience of Robotic

Grasping

Neurocomputing

Lancet

Page 36: Decoding information for grasping from the macaque … et al 2017.pdf · 2017. 4. 18. · Matteo Filippini 1,*, Rossella Breveglieri 1,*, M. Ali Akhras 1,#, Annalisa Bosco 1, Eris

Eur. J. Neurosci.

Cereb. Cortex

J. Neurosci.

J. Neurosci.

Eur. J. Neurosci.

J. Neurosci.

J. Physiol.

Eur. J. Neurosci.

Page 37: Decoding information for grasping from the macaque … et al 2017.pdf · 2017. 4. 18. · Matteo Filippini 1,*, Rossella Breveglieri 1,*, M. Ali Akhras 1,#, Annalisa Bosco 1, Eris

Eur. J. Neurosci.

J. Neurosci.

J. Neurophysiol.

Annu. Rev. Neurosci.

Nature

Page 38: Decoding information for grasping from the macaque … et al 2017.pdf · 2017. 4. 18. · Matteo Filippini 1,*, Rossella Breveglieri 1,*, M. Ali Akhras 1,#, Annalisa Bosco 1, Eris

Nature

J. Neural Eng.

Trends Cogn. Sci.

IEEE Trans. Biomed. Eng.

J Neurosci

J. Neurophysiol.

J. Neurosci.

Page 39: Decoding information for grasping from the macaque … et al 2017.pdf · 2017. 4. 18. · Matteo Filippini 1,*, Rossella Breveglieri 1,*, M. Ali Akhras 1,#, Annalisa Bosco 1, Eris

Eur. J. Neurosci.

Nat. Neurosci.

Eur. J. Neurosci.

Rob. Auton. Syst.

Neuroimage

J. Neurosci.

Science

Nat. Rev. Neurosci.

Page 40: Decoding information for grasping from the macaque … et al 2017.pdf · 2017. 4. 18. · Matteo Filippini 1,*, Rossella Breveglieri 1,*, M. Ali Akhras 1,#, Annalisa Bosco 1, Eris

J. Neurophysiol.

Front. Psychol.

Nature

J. Neurosci.

Elife

Curr Opin Neurobiol

Neuron

Page 41: Decoding information for grasping from the macaque … et al 2017.pdf · 2017. 4. 18. · Matteo Filippini 1,*, Rossella Breveglieri 1,*, M. Ali Akhras 1,#, Annalisa Bosco 1, Eris

J. Neurophysiol.

Cell

Neuron

Nature

Prog. Brain Res.

Nature

J. Neurophysiol.

Science

Nature

Page 42: Decoding information for grasping from the macaque … et al 2017.pdf · 2017. 4. 18. · Matteo Filippini 1,*, Rossella Breveglieri 1,*, M. Ali Akhras 1,#, Annalisa Bosco 1, Eris

J. Neurosci.

J. Neurosci.

Nature

Nature

Annu. Rev. Neurosci.

Proc. Seventeenth Int. Florida

Artif. Intell. Res. Soc. Conf. FLAIRS 2004

Page 43: Decoding information for grasping from the macaque … et al 2017.pdf · 2017. 4. 18. · Matteo Filippini 1,*, Rossella Breveglieri 1,*, M. Ali Akhras 1,#, Annalisa Bosco 1, Eris
Page 44: Decoding information for grasping from the macaque … et al 2017.pdf · 2017. 4. 18. · Matteo Filippini 1,*, Rossella Breveglieri 1,*, M. Ali Akhras 1,#, Annalisa Bosco 1, Eris
Page 45: Decoding information for grasping from the macaque … et al 2017.pdf · 2017. 4. 18. · Matteo Filippini 1,*, Rossella Breveglieri 1,*, M. Ali Akhras 1,#, Annalisa Bosco 1, Eris
Page 46: Decoding information for grasping from the macaque … et al 2017.pdf · 2017. 4. 18. · Matteo Filippini 1,*, Rossella Breveglieri 1,*, M. Ali Akhras 1,#, Annalisa Bosco 1, Eris
Page 47: Decoding information for grasping from the macaque … et al 2017.pdf · 2017. 4. 18. · Matteo Filippini 1,*, Rossella Breveglieri 1,*, M. Ali Akhras 1,#, Annalisa Bosco 1, Eris
Page 48: Decoding information for grasping from the macaque … et al 2017.pdf · 2017. 4. 18. · Matteo Filippini 1,*, Rossella Breveglieri 1,*, M. Ali Akhras 1,#, Annalisa Bosco 1, Eris
Page 49: Decoding information for grasping from the macaque … et al 2017.pdf · 2017. 4. 18. · Matteo Filippini 1,*, Rossella Breveglieri 1,*, M. Ali Akhras 1,#, Annalisa Bosco 1, Eris
Page 50: Decoding information for grasping from the macaque … et al 2017.pdf · 2017. 4. 18. · Matteo Filippini 1,*, Rossella Breveglieri 1,*, M. Ali Akhras 1,#, Annalisa Bosco 1, Eris
Page 51: Decoding information for grasping from the macaque … et al 2017.pdf · 2017. 4. 18. · Matteo Filippini 1,*, Rossella Breveglieri 1,*, M. Ali Akhras 1,#, Annalisa Bosco 1, Eris
Page 52: Decoding information for grasping from the macaque … et al 2017.pdf · 2017. 4. 18. · Matteo Filippini 1,*, Rossella Breveglieri 1,*, M. Ali Akhras 1,#, Annalisa Bosco 1, Eris
Page 53: Decoding information for grasping from the macaque … et al 2017.pdf · 2017. 4. 18. · Matteo Filippini 1,*, Rossella Breveglieri 1,*, M. Ali Akhras 1,#, Annalisa Bosco 1, Eris