brain reading with fmri - national institutes of health · 2013. 8. 21. · detecting and...
TRANSCRIPT
![Page 1: Brain Reading with fMRI - National Institutes of Health · 2013. 8. 21. · detecting and classifying distributed patterns of fMRI activity in human visual cortex. Neuroimage, 19,](https://reader033.vdocuments.mx/reader033/viewer/2022060915/60a8b76c2423127e53343d29/html5/thumbnails/1.jpg)
Brain Reading with fMRI
07.31.13
Chris Baker Laboratory of Brain and Cognition, NIMH
![Page 2: Brain Reading with fMRI - National Institutes of Health · 2013. 8. 21. · detecting and classifying distributed patterns of fMRI activity in human visual cortex. Neuroimage, 19,](https://reader033.vdocuments.mx/reader033/viewer/2022060915/60a8b76c2423127e53343d29/html5/thumbnails/2.jpg)
I Saw That on TV…..
CBS – 60 Minutes, 2009
![Page 3: Brain Reading with fMRI - National Institutes of Health · 2013. 8. 21. · detecting and classifying distributed patterns of fMRI activity in human visual cortex. Neuroimage, 19,](https://reader033.vdocuments.mx/reader033/viewer/2022060915/60a8b76c2423127e53343d29/html5/thumbnails/3.jpg)
“mind reading”
“decoding” “prediction”
“thought identification”
“what your spouse is really thinking, or your boss or the guy sitting across from you on the bus” What can we really do?
![Page 4: Brain Reading with fMRI - National Institutes of Health · 2013. 8. 21. · detecting and classifying distributed patterns of fMRI activity in human visual cortex. Neuroimage, 19,](https://reader033.vdocuments.mx/reader033/viewer/2022060915/60a8b76c2423127e53343d29/html5/thumbnails/4.jpg)
Let’s Read Some Brains
1) Training 2) Test
![Page 5: Brain Reading with fMRI - National Institutes of Health · 2013. 8. 21. · detecting and classifying distributed patterns of fMRI activity in human visual cortex. Neuroimage, 19,](https://reader033.vdocuments.mx/reader033/viewer/2022060915/60a8b76c2423127e53343d29/html5/thumbnails/5.jpg)
Face-selective cortex "(Fusiform Face Area, FFA)"
>"
Scene-selective cortex "(Parahippocampal Place Area, PPA)"
>
Epstein and Kanwisher (1998). Nature, 392, 598-601 Kanwisher et al. (1997). J. Neurosci., 17, 4302-4311
1) Training
![Page 6: Brain Reading with fMRI - National Institutes of Health · 2013. 8. 21. · detecting and classifying distributed patterns of fMRI activity in human visual cortex. Neuroimage, 19,](https://reader033.vdocuments.mx/reader033/viewer/2022060915/60a8b76c2423127e53343d29/html5/thumbnails/6.jpg)
PPA (scenes)
FFA (faces)
1) Training
![Page 7: Brain Reading with fMRI - National Institutes of Health · 2013. 8. 21. · detecting and classifying distributed patterns of fMRI activity in human visual cortex. Neuroimage, 19,](https://reader033.vdocuments.mx/reader033/viewer/2022060915/60a8b76c2423127e53343d29/html5/thumbnails/7.jpg)
S1
S2
Faces Scenes
Faces Scenes
2) Test
![Page 8: Brain Reading with fMRI - National Institutes of Health · 2013. 8. 21. · detecting and classifying distributed patterns of fMRI activity in human visual cortex. Neuroimage, 19,](https://reader033.vdocuments.mx/reader033/viewer/2022060915/60a8b76c2423127e53343d29/html5/thumbnails/8.jpg)
S1
S2
Faces Scenes
Faces Scenes
PPA (scenes)
FFA (faces)
![Page 9: Brain Reading with fMRI - National Institutes of Health · 2013. 8. 21. · detecting and classifying distributed patterns of fMRI activity in human visual cortex. Neuroimage, 19,](https://reader033.vdocuments.mx/reader033/viewer/2022060915/60a8b76c2423127e53343d29/html5/thumbnails/9.jpg)
Brain activity Stimulus
• Decoding models – Uses voxel activity to predict stimulus information
• Encoding models – Explicit description of how information is represented in activity of
single voxels
Naselaris et al (2011). Neuroimage.
![Page 10: Brain Reading with fMRI - National Institutes of Health · 2013. 8. 21. · detecting and classifying distributed patterns of fMRI activity in human visual cortex. Neuroimage, 19,](https://reader033.vdocuments.mx/reader033/viewer/2022060915/60a8b76c2423127e53343d29/html5/thumbnails/10.jpg)
Brain activity Stimulus
• Decoding models – Uses voxel activity to predict stimulus information
• Encoding models – Explicit description of how information is represented in activity of
single voxels
Naselaris et al (2011). Neuroimage.
![Page 11: Brain Reading with fMRI - National Institutes of Health · 2013. 8. 21. · detecting and classifying distributed patterns of fMRI activity in human visual cortex. Neuroimage, 19,](https://reader033.vdocuments.mx/reader033/viewer/2022060915/60a8b76c2423127e53343d29/html5/thumbnails/11.jpg)
“brain reading”
“decoding” “prediction”
“classification”
Multi Voxel Pattern Analysis (MVPA)
![Page 12: Brain Reading with fMRI - National Institutes of Health · 2013. 8. 21. · detecting and classifying distributed patterns of fMRI activity in human visual cortex. Neuroimage, 19,](https://reader033.vdocuments.mx/reader033/viewer/2022060915/60a8b76c2423127e53343d29/html5/thumbnails/12.jpg)
Univariate vs. Multivariate
• Classic fMRI analyses = univariate – Each voxel considered independently
• Multivariate – Responses of voxels considered jointly
– Pattern of response
![Page 13: Brain Reading with fMRI - National Institutes of Health · 2013. 8. 21. · detecting and classifying distributed patterns of fMRI activity in human visual cortex. Neuroimage, 19,](https://reader033.vdocuments.mx/reader033/viewer/2022060915/60a8b76c2423127e53343d29/html5/thumbnails/13.jpg)
Condition 1 Condition 2
Condition 3 Condition n
![Page 14: Brain Reading with fMRI - National Institutes of Health · 2013. 8. 21. · detecting and classifying distributed patterns of fMRI activity in human visual cortex. Neuroimage, 19,](https://reader033.vdocuments.mx/reader033/viewer/2022060915/60a8b76c2423127e53343d29/html5/thumbnails/14.jpg)
Voxel 2
Voxel 1
Condition A
Condition B
?
![Page 15: Brain Reading with fMRI - National Institutes of Health · 2013. 8. 21. · detecting and classifying distributed patterns of fMRI activity in human visual cortex. Neuroimage, 19,](https://reader033.vdocuments.mx/reader033/viewer/2022060915/60a8b76c2423127e53343d29/html5/thumbnails/15.jpg)
Voxel 2
Voxel 1
Condition A
Condition B
Euclidean Distance
?
![Page 16: Brain Reading with fMRI - National Institutes of Health · 2013. 8. 21. · detecting and classifying distributed patterns of fMRI activity in human visual cortex. Neuroimage, 19,](https://reader033.vdocuments.mx/reader033/viewer/2022060915/60a8b76c2423127e53343d29/html5/thumbnails/16.jpg)
Voxel 2
Voxel 1
Condition A
Condition B
Correlation
?
![Page 17: Brain Reading with fMRI - National Institutes of Health · 2013. 8. 21. · detecting and classifying distributed patterns of fMRI activity in human visual cortex. Neuroimage, 19,](https://reader033.vdocuments.mx/reader033/viewer/2022060915/60a8b76c2423127e53343d29/html5/thumbnails/17.jpg)
Voxel 2
Voxel 1
Condition A
Condition B
Support Vector Machine (SVM)
?
![Page 18: Brain Reading with fMRI - National Institutes of Health · 2013. 8. 21. · detecting and classifying distributed patterns of fMRI activity in human visual cortex. Neuroimage, 19,](https://reader033.vdocuments.mx/reader033/viewer/2022060915/60a8b76c2423127e53343d29/html5/thumbnails/18.jpg)
Linear Classifiers
• Euclidean distance • Correlation • Linear SVM • Fisher Least Discriminant Analysis • Neural networks (without hidden layer) • Gaussian Naïve Bayes Classifiers
Non-linear classifiers increase risk of overfitting
![Page 19: Brain Reading with fMRI - National Institutes of Health · 2013. 8. 21. · detecting and classifying distributed patterns of fMRI activity in human visual cortex. Neuroimage, 19,](https://reader033.vdocuments.mx/reader033/viewer/2022060915/60a8b76c2423127e53343d29/html5/thumbnails/19.jpg)
Object representations in ventral temporal cortex (Haxby et al, 2001)
• Participants viewed blocks of images from 8 categories
• 1-back task
• Split-half correlation analysis
![Page 20: Brain Reading with fMRI - National Institutes of Health · 2013. 8. 21. · detecting and classifying distributed patterns of fMRI activity in human visual cortex. Neuroimage, 19,](https://reader033.vdocuments.mx/reader033/viewer/2022060915/60a8b76c2423127e53343d29/html5/thumbnails/20.jpg)
Voxel 2
Voxel 1
Condition A
Condition B
Correlation
Odd runs
Odd runs
? Even runs
![Page 21: Brain Reading with fMRI - National Institutes of Health · 2013. 8. 21. · detecting and classifying distributed patterns of fMRI activity in human visual cortex. Neuroimage, 19,](https://reader033.vdocuments.mx/reader033/viewer/2022060915/60a8b76c2423127e53343d29/html5/thumbnails/21.jpg)
![Page 22: Brain Reading with fMRI - National Institutes of Health · 2013. 8. 21. · detecting and classifying distributed patterns of fMRI activity in human visual cortex. Neuroimage, 19,](https://reader033.vdocuments.mx/reader033/viewer/2022060915/60a8b76c2423127e53343d29/html5/thumbnails/22.jpg)
![Page 23: Brain Reading with fMRI - National Institutes of Health · 2013. 8. 21. · detecting and classifying distributed patterns of fMRI activity in human visual cortex. Neuroimage, 19,](https://reader033.vdocuments.mx/reader033/viewer/2022060915/60a8b76c2423127e53343d29/html5/thumbnails/23.jpg)
Higher within- than between-category
correlations
![Page 24: Brain Reading with fMRI - National Institutes of Health · 2013. 8. 21. · detecting and classifying distributed patterns of fMRI activity in human visual cortex. Neuroimage, 19,](https://reader033.vdocuments.mx/reader033/viewer/2022060915/60a8b76c2423127e53343d29/html5/thumbnails/24.jpg)
Decoding Accuracy
![Page 25: Brain Reading with fMRI - National Institutes of Health · 2013. 8. 21. · detecting and classifying distributed patterns of fMRI activity in human visual cortex. Neuroimage, 19,](https://reader033.vdocuments.mx/reader033/viewer/2022060915/60a8b76c2423127e53343d29/html5/thumbnails/25.jpg)
Decoding Orientation in Early Visual Cortex (Kamitani and Tong, 2005)
• Participants viewed blocks of oriented lines (8 possible orientations)
• Linear SVM
![Page 26: Brain Reading with fMRI - National Institutes of Health · 2013. 8. 21. · detecting and classifying distributed patterns of fMRI activity in human visual cortex. Neuroimage, 19,](https://reader033.vdocuments.mx/reader033/viewer/2022060915/60a8b76c2423127e53343d29/html5/thumbnails/26.jpg)
Highly accurate decoding of orientation
![Page 27: Brain Reading with fMRI - National Institutes of Health · 2013. 8. 21. · detecting and classifying distributed patterns of fMRI activity in human visual cortex. Neuroimage, 19,](https://reader033.vdocuments.mx/reader033/viewer/2022060915/60a8b76c2423127e53343d29/html5/thumbnails/27.jpg)
Decoding Attended Orientation
![Page 28: Brain Reading with fMRI - National Institutes of Health · 2013. 8. 21. · detecting and classifying distributed patterns of fMRI activity in human visual cortex. Neuroimage, 19,](https://reader033.vdocuments.mx/reader033/viewer/2022060915/60a8b76c2423127e53343d29/html5/thumbnails/28.jpg)
Perception, working memory and imagery (Albers et al, 2013)
![Page 29: Brain Reading with fMRI - National Institutes of Health · 2013. 8. 21. · detecting and classifying distributed patterns of fMRI activity in human visual cortex. Neuroimage, 19,](https://reader033.vdocuments.mx/reader033/viewer/2022060915/60a8b76c2423127e53343d29/html5/thumbnails/29.jpg)
Working memory Working Memory
Decoding of mental images
Imagery
![Page 30: Brain Reading with fMRI - National Institutes of Health · 2013. 8. 21. · detecting and classifying distributed patterns of fMRI activity in human visual cortex. Neuroimage, 19,](https://reader033.vdocuments.mx/reader033/viewer/2022060915/60a8b76c2423127e53343d29/html5/thumbnails/30.jpg)
Decoding across working memory, imagery and perception
![Page 31: Brain Reading with fMRI - National Institutes of Health · 2013. 8. 21. · detecting and classifying distributed patterns of fMRI activity in human visual cortex. Neuroimage, 19,](https://reader033.vdocuments.mx/reader033/viewer/2022060915/60a8b76c2423127e53343d29/html5/thumbnails/31.jpg)
Limitations of Early Decoding Studies
• Restricted stimulus domains – Oriented lines
– Small number of selected categories
• No decoding of novel stimuli or categories [but see Spiridon and Kanwisher(2002)]
Kamitani and Tong (2005)
Haxby et al (2001)
![Page 32: Brain Reading with fMRI - National Institutes of Health · 2013. 8. 21. · detecting and classifying distributed patterns of fMRI activity in human visual cortex. Neuroimage, 19,](https://reader033.vdocuments.mx/reader033/viewer/2022060915/60a8b76c2423127e53343d29/html5/thumbnails/32.jpg)
Decoding Dreams (Horikawa et al, 2013)
![Page 33: Brain Reading with fMRI - National Institutes of Health · 2013. 8. 21. · detecting and classifying distributed patterns of fMRI activity in human visual cortex. Neuroimage, 19,](https://reader033.vdocuments.mx/reader033/viewer/2022060915/60a8b76c2423127e53343d29/html5/thumbnails/33.jpg)
Brain activity Stimulus
• Decoding models – Uses voxel activity to predict stimulus information
• Encoding models – Explicit description of how information is represented in activity of
single voxels
Naselaris et al (2011). Neuroimage.
![Page 34: Brain Reading with fMRI - National Institutes of Health · 2013. 8. 21. · detecting and classifying distributed patterns of fMRI activity in human visual cortex. Neuroimage, 19,](https://reader033.vdocuments.mx/reader033/viewer/2022060915/60a8b76c2423127e53343d29/html5/thumbnails/34.jpg)
Model-based approach to decoding���(Kay et al, 2008)
1) Characterize relationship between visual stimuli and fMRI activity (i.e. build a model) – Complex, natural visual images – Early retinotopic visual cortex
2) Measure fMRI activity to one of many possible novel images
3) Compare actual activity to predicted activity for full set of novel images to determine which image was viewed
![Page 35: Brain Reading with fMRI - National Institutes of Health · 2013. 8. 21. · detecting and classifying distributed patterns of fMRI activity in human visual cortex. Neuroimage, 19,](https://reader033.vdocuments.mx/reader033/viewer/2022060915/60a8b76c2423127e53343d29/html5/thumbnails/35.jpg)
Large gray-scale images
![Page 36: Brain Reading with fMRI - National Institutes of Health · 2013. 8. 21. · detecting and classifying distributed patterns of fMRI activity in human visual cortex. Neuroimage, 19,](https://reader033.vdocuments.mx/reader033/viewer/2022060915/60a8b76c2423127e53343d29/html5/thumbnails/36.jpg)
1) Build a Model
![Page 37: Brain Reading with fMRI - National Institutes of Health · 2013. 8. 21. · detecting and classifying distributed patterns of fMRI activity in human visual cortex. Neuroimage, 19,](https://reader033.vdocuments.mx/reader033/viewer/2022060915/60a8b76c2423127e53343d29/html5/thumbnails/37.jpg)
RF model for one voxel
![Page 38: Brain Reading with fMRI - National Institutes of Health · 2013. 8. 21. · detecting and classifying distributed patterns of fMRI activity in human visual cortex. Neuroimage, 19,](https://reader033.vdocuments.mx/reader033/viewer/2022060915/60a8b76c2423127e53343d29/html5/thumbnails/38.jpg)
Novel Image to be Identified
![Page 39: Brain Reading with fMRI - National Institutes of Health · 2013. 8. 21. · detecting and classifying distributed patterns of fMRI activity in human visual cortex. Neuroimage, 19,](https://reader033.vdocuments.mx/reader033/viewer/2022060915/60a8b76c2423127e53343d29/html5/thumbnails/39.jpg)
Compare observed to predicted activity
![Page 40: Brain Reading with fMRI - National Institutes of Health · 2013. 8. 21. · detecting and classifying distributed patterns of fMRI activity in human visual cortex. Neuroimage, 19,](https://reader033.vdocuments.mx/reader033/viewer/2022060915/60a8b76c2423127e53343d29/html5/thumbnails/40.jpg)
Performance
![Page 41: Brain Reading with fMRI - National Institutes of Health · 2013. 8. 21. · detecting and classifying distributed patterns of fMRI activity in human visual cortex. Neuroimage, 19,](https://reader033.vdocuments.mx/reader033/viewer/2022060915/60a8b76c2423127e53343d29/html5/thumbnails/41.jpg)
Additional results
• Works on single trials • Not just retinotopy • Accurate even with long delay between
model fitting and testing
![Page 42: Brain Reading with fMRI - National Institutes of Health · 2013. 8. 21. · detecting and classifying distributed patterns of fMRI activity in human visual cortex. Neuroimage, 19,](https://reader033.vdocuments.mx/reader033/viewer/2022060915/60a8b76c2423127e53343d29/html5/thumbnails/42.jpg)
Limitations of Kay et al.
• Still requires comparison with set of candidate images • Will likely fail with more homogeneous images (e.g. two
faces)
• Whole image comparison – What about same central object on different
backgrounds?
• How sensitive to fixation differences? • Novel subjects?
• Visual perception is dynamic
![Page 43: Brain Reading with fMRI - National Institutes of Health · 2013. 8. 21. · detecting and classifying distributed patterns of fMRI activity in human visual cortex. Neuroimage, 19,](https://reader033.vdocuments.mx/reader033/viewer/2022060915/60a8b76c2423127e53343d29/html5/thumbnails/43.jpg)
Semantic space
• Huth et al (2012). A continuous semantic space describes the representation of thousands of object and action categories across the human brain. Neuron.
• http://gallantlab.org/brainviewer/
![Page 44: Brain Reading with fMRI - National Institutes of Health · 2013. 8. 21. · detecting and classifying distributed patterns of fMRI activity in human visual cortex. Neuroimage, 19,](https://reader033.vdocuments.mx/reader033/viewer/2022060915/60a8b76c2423127e53343d29/html5/thumbnails/44.jpg)
Visual Image Reconstruction���[Miyawaki et al (2008)]
• Model based decoding • Characterize relationship between activity
and contrast of local image patches • Use activity to predict contrast within
image
![Page 45: Brain Reading with fMRI - National Institutes of Health · 2013. 8. 21. · detecting and classifying distributed patterns of fMRI activity in human visual cortex. Neuroimage, 19,](https://reader033.vdocuments.mx/reader033/viewer/2022060915/60a8b76c2423127e53343d29/html5/thumbnails/45.jpg)
![Page 46: Brain Reading with fMRI - National Institutes of Health · 2013. 8. 21. · detecting and classifying distributed patterns of fMRI activity in human visual cortex. Neuroimage, 19,](https://reader033.vdocuments.mx/reader033/viewer/2022060915/60a8b76c2423127e53343d29/html5/thumbnails/46.jpg)
Image presentation
![Page 47: Brain Reading with fMRI - National Institutes of Health · 2013. 8. 21. · detecting and classifying distributed patterns of fMRI activity in human visual cortex. Neuroimage, 19,](https://reader033.vdocuments.mx/reader033/viewer/2022060915/60a8b76c2423127e53343d29/html5/thumbnails/47.jpg)
Reconstructions
![Page 48: Brain Reading with fMRI - National Institutes of Health · 2013. 8. 21. · detecting and classifying distributed patterns of fMRI activity in human visual cortex. Neuroimage, 19,](https://reader033.vdocuments.mx/reader033/viewer/2022060915/60a8b76c2423127e53343d29/html5/thumbnails/48.jpg)
Limitations of Miyawaki et al
• Similar limitations to Kay et al. • Simple, non-natural stimuli • Small image size
For extension of Kay et al. into reconstruction, see Naselaris et al (2009)
![Page 49: Brain Reading with fMRI - National Institutes of Health · 2013. 8. 21. · detecting and classifying distributed patterns of fMRI activity in human visual cortex. Neuroimage, 19,](https://reader033.vdocuments.mx/reader033/viewer/2022060915/60a8b76c2423127e53343d29/html5/thumbnails/49.jpg)
Applications for fMRI brain reading
1) Understanding how information is represented in the brain
2) Lie detection? 3) Prosthetic devices? 4) Disorders of consciousness
![Page 50: Brain Reading with fMRI - National Institutes of Health · 2013. 8. 21. · detecting and classifying distributed patterns of fMRI activity in human visual cortex. Neuroimage, 19,](https://reader033.vdocuments.mx/reader033/viewer/2022060915/60a8b76c2423127e53343d29/html5/thumbnails/50.jpg)
Disorders of consciousness
• Vegetative state • Locked-in syndrome • Enabling communication in the
absence of overt motor behavior
![Page 51: Brain Reading with fMRI - National Institutes of Health · 2013. 8. 21. · detecting and classifying distributed patterns of fMRI activity in human visual cortex. Neuroimage, 19,](https://reader033.vdocuments.mx/reader033/viewer/2022060915/60a8b76c2423127e53343d29/html5/thumbnails/51.jpg)
Decoding Tasks
Motor Imagery (playing tennis)
Spatial Imagery (walking house)
Owen et al (2006); Monti et al (2010)
![Page 52: Brain Reading with fMRI - National Institutes of Health · 2013. 8. 21. · detecting and classifying distributed patterns of fMRI activity in human visual cortex. Neuroimage, 19,](https://reader033.vdocuments.mx/reader033/viewer/2022060915/60a8b76c2423127e53343d29/html5/thumbnails/52.jpg)
54 patients with severe brain injury
![Page 53: Brain Reading with fMRI - National Institutes of Health · 2013. 8. 21. · detecting and classifying distributed patterns of fMRI activity in human visual cortex. Neuroimage, 19,](https://reader033.vdocuments.mx/reader033/viewer/2022060915/60a8b76c2423127e53343d29/html5/thumbnails/53.jpg)
![Page 54: Brain Reading with fMRI - National Institutes of Health · 2013. 8. 21. · detecting and classifying distributed patterns of fMRI activity in human visual cortex. Neuroimage, 19,](https://reader033.vdocuments.mx/reader033/viewer/2022060915/60a8b76c2423127e53343d29/html5/thumbnails/54.jpg)
Real time fMRI spelling
Sorger et al (2012)
![Page 55: Brain Reading with fMRI - National Institutes of Health · 2013. 8. 21. · detecting and classifying distributed patterns of fMRI activity in human visual cortex. Neuroimage, 19,](https://reader033.vdocuments.mx/reader033/viewer/2022060915/60a8b76c2423127e53343d29/html5/thumbnails/55.jpg)
Real time fMRI spelling
![Page 56: Brain Reading with fMRI - National Institutes of Health · 2013. 8. 21. · detecting and classifying distributed patterns of fMRI activity in human visual cortex. Neuroimage, 19,](https://reader033.vdocuments.mx/reader033/viewer/2022060915/60a8b76c2423127e53343d29/html5/thumbnails/56.jpg)
“mind reading”
“decoding” “prediction”
“thought identification”
“what your spouse is really thinking, or your boss or the guy sitting across from you on the bus” What can we really do?
![Page 57: Brain Reading with fMRI - National Institutes of Health · 2013. 8. 21. · detecting and classifying distributed patterns of fMRI activity in human visual cortex. Neuroimage, 19,](https://reader033.vdocuments.mx/reader033/viewer/2022060915/60a8b76c2423127e53343d29/html5/thumbnails/57.jpg)
Key Readings
![Page 58: Brain Reading with fMRI - National Institutes of Health · 2013. 8. 21. · detecting and classifying distributed patterns of fMRI activity in human visual cortex. Neuroimage, 19,](https://reader033.vdocuments.mx/reader033/viewer/2022060915/60a8b76c2423127e53343d29/html5/thumbnails/58.jpg)
Overviews and Methods
• Cox and Savoy (2003). Functional magnetic resonance imaging (fMRI) “brain reading”: detecting and classifying distributed patterns of fMRI activity in human visual cortex. Neuroimage, 19, 261-270.
• Mur et al. (2008). Revealing representational content with pattern information fMRI - an introductory guide. Social Cognitive and Affective Neuroscience, 4, 101-109.
• Norman et al. (2006). Beyond mind-reading: multi-voxel pattern analysis of fMRI data. Trends in Cognitive Sciences, 10, 424-430.
• Sorger et al. (2012). A real-time fMRI-based spelling device immediately enabling robust motor-independent communication. Current Biology, 22, 1333-1338.
• Naselaris et al (2011). Encoding and decoding in fMRI. Neuroimage, 56, 400-410.
![Page 59: Brain Reading with fMRI - National Institutes of Health · 2013. 8. 21. · detecting and classifying distributed patterns of fMRI activity in human visual cortex. Neuroimage, 19,](https://reader033.vdocuments.mx/reader033/viewer/2022060915/60a8b76c2423127e53343d29/html5/thumbnails/59.jpg)
Specific Studies
• Albers et al (2013). Shared representations for working memory and mental imagery in early visual cortex. Current Biology.
• Haxby et al. (2001). Distributed and overlapping representations of faces and objects in ventral temporal cortex. Science, 293, 2425-2430.
• Kamitani and Tong (2005). Decoding the visual and subjective contents of the human brain. Nature Neuroscience, 8, 679-685.
• Kay et al (2008). Identifying natural images from human brain activity. Nature, 452, 352-355. • Kay and Gallant (2009). I can see what you see. Nature Neuroscience, 12, 245-246. • Miyawaki et al (2008). Visual image reconstruction from human brain activity using a
combination of multiscale local image decoders. Neuron, 60, 915-929.
• Nishimoto et al. (2011). Reconstructing visual experiences from brain activity evoked by natural movies. Current Biology, 21, 1641-1646.
![Page 60: Brain Reading with fMRI - National Institutes of Health · 2013. 8. 21. · detecting and classifying distributed patterns of fMRI activity in human visual cortex. Neuroimage, 19,](https://reader033.vdocuments.mx/reader033/viewer/2022060915/60a8b76c2423127e53343d29/html5/thumbnails/60.jpg)
Disorders of Consciousness
• Monti et al. (2010). Willful modulation of brain activity in disorders of consciousness. New England Journal of Medicine, 362, 579-589.
• Owen et al. (2006). Detecting awareness in the vegetative state. Science, 313, 1402
![Page 61: Brain Reading with fMRI - National Institutes of Health · 2013. 8. 21. · detecting and classifying distributed patterns of fMRI activity in human visual cortex. Neuroimage, 19,](https://reader033.vdocuments.mx/reader033/viewer/2022060915/60a8b76c2423127e53343d29/html5/thumbnails/61.jpg)
Resources
• SVM toolbox – http://www.csie.ntu.edu.tw/~cjlin/libsvm/
• Python MVPA toolbox – http://www.pymvpa.org/
• Princeton MVPA toolbox – http://code.google.com/p/princeton-mvpa-
toolbox/
![Page 62: Brain Reading with fMRI - National Institutes of Health · 2013. 8. 21. · detecting and classifying distributed patterns of fMRI activity in human visual cortex. Neuroimage, 19,](https://reader033.vdocuments.mx/reader033/viewer/2022060915/60a8b76c2423127e53343d29/html5/thumbnails/62.jpg)