artificial vision & neuroprosthetics

J e a n n a N i k o l o v - R a m i r e z . N e u r o s c i e n c e 2 0 1 4 S u p e r v i s o r : M . E r n s t . 2 3 . D e c . 2 0 1 4

A r t i f i c i a l V i s i o n & N e u r o p r o s t h e t i c s

Motivational statement §  Interest in aesthe+cs and visual

percep+on §  Contribu+on of aesthe+cs and form

to insight §  Interest in robo+cs and informa+on

processing §  Sensory subs+tu+on §  Inves+ga+ng advancements and

challenges in the field of human-‐brain interfaces

§  More specifically nascent field of Visual Neuroprosthe+cs

30/12/14 2

Aesthe+cs and Visual

Percep+on

Neuroscience and Informa+on

processing

Robo+cs and AI

NeuroScience, MEi:CogSci Nikolov 2014.12.23

Outline

§  History of Ar+ficial Vision §  Visual Apparatus and Re+na §  Current Approaches in Prosthe+c

Rehabilita+on §  Epire+nal implants §  Subre+nal implants §  Transchoroidal prostheses §  Op+c nerve prostheses §  Cor+cal and LGN implants

§  Advantages and Drawbacks Comparison

§  Conclusions and Further Work §  References §  Extra: Bach-‐y-‐Rita and Neuroplas+city

30/12/14 3 NeuroScience, MEi:CogSci Nikolov 2014.12.23

connection with neuroscience

30/12/14 NeuroScience, MEi:CogSci Nikolov 2014.12.23 4

Neural Processing of Visual Informa+on

Dana Founda+on Report

Neuroscien+fic Advances

Interfacing

History of artificial Vision §  1755

Charles LeRoy, first +me electrical device restored a flicker of visual percep+on

§  1929 Foerster: Electrical s+mula+on of the visual cortex (occipital lobe) resulted in a blind pa+ent seeing a spot of light (phosphene).

§  1960ies Giles Brindley’s implanta+on of an 80-‐electrode device onto the visual cortex of a blind pa+ent renewed the possibili+es of ar+ficial vision restora+on.

§  1970ies Dobelle brain implants

§  2013 FDA approval of first Re+nal Prosthesis System.

Lorach, H., et al. Neural stimulation for visual rehabilitation: Advances and challenges. J. Physiol. (2012), http://

dx.doi.org/10.1016/j.jphysparis.2012.10.003

Human Visual Apparatus

§  Photoreceptor §  Rods (low light) and cones (color)

§  Bipolar cells §  Ganglion cells

§  Axons form the op+c nerve to lateral geniculate nucleus of thalamus


Approaches in Prosthetic rehabilitation

§  Epire+nal implants §  Subre+nal implants §  Transchoroidal prostheses

§  Op+c nerve prostheses §  Cor+cal and LGN implants


Electrodes resolution

2010, hap://images.dailytech.com/nimage/14077_large_vision_resolu+on.png

hap://www.cs.noa.ac.uk/~abb/papers/a47.pdf


ARGUS I ARGUS II

IRIS, Intelligent Medical Implant

EPIRET-3 STS

OPTIC NERVE IMPLANT

DOBELLE Cortical implant

BOSTON RETINA

IMPLANT Subret.

RETINA IMPLANT AG

STANFORD SUBRET.

MIcrophotodiode

ASR Vision

Institute Paris

UTAH ELECTRODE

ARRAY

strategies advantages and drawbacks

•  Re+nal and op+c nerve implanta+ons are safer than brain s+mula+on approaches. •  Implant stability has been demonstrated in all techniques. •  The electrode-‐+ssue contact is improved in subre+nal approaches. •  The processing complexity increases in higher visual streams. •  The poten+al acuity restora+on is highly dependent on the ability to s+mulate a

limited corresponding visual field. •  Re+nal and op+c nerve strategies are only suited for pa+ents with intact ganglion

cells and op+c nerve (re+ni+s pigmentosa and AMD). •  Brain s+mula+on in contrast can be used in any visual impairment.


Conclusions and Further Questions §  Research is nascent. §  Challenges include:

§  Accoun+ng for eye movements, (use eye tracking system to select in real +me the part of the image corresponding to gaze direc+on)

§  research into signal processing from photoreceptors, §  target specific cell types independently (e.g. ON and OFF), §  electrode miniaturiza+on, §  material op+miza+on, §  mul+plexing of s+mula+on channels , §  developing of real +me processing algorithms (adequate filtering) to provide

relevant physiological s+muli, encoding of visual informa+on into electrical s+muli.

§  Other promising strategies are emerging: §  Optogene+cs §  Cell therapy

30/12/14 MEi:CogSci MoPE Nikolov 2014.12.17 12

References

1.  Lorach, H., Marre, O., Sahel, J. A., Benosman, R., & Picaud, S. (2013). Neural s+mula+on for visual rehabilita+on: Advances and challenges. Journal of Physiology-‐Paris, 107(5), 421-‐431. Chicago

2.  Kien, T. T., Maul, T., & Bargiela, A. (2012). A review of re+nal prosthesis approaches. In Interna+onal Journal of Modern Physics: Conference Series (Vol. 9, pp. 209-‐231). World Scien+fic Publishing Company.

3.  Weiland, J. D., Liu, W., & Humayun, M. S. (2005). Re+nal prosthesis. Annu. Rev. Biomed. Eng., 7, 361-‐401.

4.  hap://www.the-‐scien+st.com/?ar+cles.view/ar+cleNo/41324/+tle/Neuroprosthe+cs/ 5.  hap://www.the-‐scien+st.com/?ar+cles.view/ar+cleNo/41052/+tle/The-‐Bionic-‐Eye/ 6.  hap://isites.harvard.edu/fs/docs/icb.topic793620.files/Re+nal_ar+ficial.pdf 7.  hap://archive.wired.com/wired/archive/10.09/vision_pr.html 8.  hap://www.bostonre+nalimplant.org/assets/Uploads/KellyTBME2011.pdf 9.  hap://biomed.brown.edu/Courses/BI108/BI108_1999_Groups/Vision_Team/Cor+cal.htm 10.  hap://www.lems.brown.edu/~jgr/cor+cal_prosthesis_proposal.htm 11.  hap://www.technologyreview.com/news/407739/brain-‐implants-‐to-‐restore-‐vision/ 12.  hap://www.dana.org/Publica+ons/ReportOnProgress/

Ar+ficial_Sight_Restora+on_of_Sight_through_Use_of_Argus/ 13.  hap://www.natureasia.com/en/research/highlight/8524

30/12/14 MEi:CogSci MoPE Nikolov 2014.12.17 13

MEi:CogSci MoPE Nikolov 2014.12.17 30/12/14 14

THANK YOU!

Epiretinal: Argus I and Argus II

Humayun et al. hap://www.expertsmind.com/topic/neuroscience/re+nal-‐processing-‐93034.aspx

Pros: •  S+mula+ng close to

photoreceptors takes advantage of na+ve processing power in the thalamus and cortex

•  Surgical complica+ons not necessarily as significant as cor+cal approach

Cons: •  Requires func+onal

op+c nerve pathway •  May s+mulate op+c

nerve fibers rather than cell bodies

•  Difficult to adhere electrode array to re+na

First in 2002


SUBRETINAL

hap://www.bostonre+nalimplant.org/assets/Uploads/KellyTBME2011.pdf

Pros: •  Inserted below re+na •  Maintained between the choroid and the re+na itself •  No addi+onal tack for fixa+on •  Posi+on increases implant stability but risk of re+nal

detachments Cons: •  Subre+nal s+mula+on threshold were found to be lower than

for epire+nal s+mula+on

hap://optobionics.com/asrdevice.shtml


Transchoroidal prostheses

Bionic Vision Australia Media Release, haps://app.box.com/s/bq9jt8g1uvs014dex84s/1/2495873247/21429693717/1

Pros: •  S+mulate re+na from the outer part •  Easier implanta+on, low +ssue damage •  No risk of re+nal detachment Cons: •  Requires higher current intensi+es to elicit visual

percepts because of the increased distance between electrodes and inner re+nal neurons


OPTIC Nerve Prostheses

C-‐Sight project hap://contest.techbriefs.com/2012/entries/medical/2933

Pros: •  Surgical complica+ons not

necessarily as significant as cor+cal approach

Cons: •  Requires func+onal op+c nerve

pathway •  Will require complex electrode

array to provide any useful paaerned vision

•  Very difficult surgical access


Cortical and LGN IMplants Pros: •  Only approach for individual with non-‐func+onal

re+nas and/or op+c nerves •  Implant site robust and protected by skull •  Phosphene thresholds are low (1 -‐ 10 uA range) •  Ac+vates electrodes on surface of visual cortex

Cons: •  S+mula+on site far from photoreceptors (no re+nal or thalamic processing), thus

some visual processing is missing •  Problems of mul+ple feature representa+ons in V1 (color, lines, mo+on, ocular

dominance) •  Requires permanent skull interface •  Highly invasive with major risks of infec+on and inflamma+on •  Cellular death around the electrodes occurring amer electrical s+mula+on

hap://biomed.brown.edu/Courses/BI108/2006-‐108websites/group03re+nalimplants/dobell.htm


Bach-y-Rita and Neuroplasticity

•  haps://www.youtube.com/watch?v=7s1VAVcM8s8

ARGUS II

Bionic contact lenses

EYE and Retina