workshop on medical vr and ar€¦ · 0 2 4 6 8 10-1-0.5 0 0.5 1 time (s) 0 2 4 6 8 10-1-0.5 0 0.5...
TRANSCRIPT
Stanford University
April 5, 2018
Workshop on Medical VR and AR
To register go to:
https://scien.stanford.edu/index.php/medicalvrar/
Panel discussions with
Stanford physicians who are using VR and AR applications
Talks by researchers who are
developing VR and AR technologies to advance healthcare
Interactive demos featuring
research projects, clinical applications and startup ventures.
Human neuroimaging using MRI: Methods and stories
Quantitative Measurements
∞Computational Models
∞Check and Share
Brian A. WandellStanford Neurosciences Institute
Stanford Center for Cognitive and Neurobiological ImagingStanford Center for Image Systems Engineering
Presenting the work of many people from my lab at Stanford
Outline
• A very brief introduction to human brain structures
• The MRI instrument: basic signals and image formation
• Interesting case studies using MRI
• MRI diagnostic tools for psychiatry: Reading impairment
Even simple judgments –
such as lightness - depend
on substantial interpretation
of the image data carried out
by brain circuits
Cognitive neuroscience is the
study of how the brain
performs these actions
(Anderson and Winawer,
Nature, 2005)
Cortical computational elements
5
Brain computations depend
on a variety of cells; an
important class of these cells,
the neurons, have their cell
bodies located in the cerebral
cortex (gray matter).
The cortex is a sheet (2-4
mm thick) of tissue that
covers the surface of the
brain; other subcortical
regions and types of cells
matter too!
image from Graham Johnson
Neurons/mm3: 104-106
Cortical Neurons: 1011
Synapses/neuron: 103
Cortical Synapses: 1014
Surface area of each
hemisphere: 20 x 30 cm2
Neuron: impulse-conducting cell; bodies are in the cerebral cortexAxon: a thin fiber that carries the output impulses from a neuronDendrite: a branching process of a neuron that receives impulses from other neuronsSynapse: The point of connection between neurons
Long-range communication architecture (tracts)
Courtesy Professor Ugur Ture
• There are many long-range connections
• These connections are not passive – they change their properties in response to use
• A system with active wires
The human brain
macaque
human
mouse
0.5 g 1500 g100 g
Image adapted from J. Horton
1: 15: 3000 (volume ratios)
• Brains differ
• Check which system was measured
Gross brain anatomy
Brain volume - 1000 – 1500 cm3
Cerebral cortex area - 1200 cm2 (20 x 30 cm) x 2Cerebellum area - 500 cm2
Vision –Occipital and temporal
Hearing –Temporal
Reading –Temporal, parietal, Frontal
The brain is studied at an enormous range of spatial scales
• There is no reliable
theory or model that
relates measurements at
different scales
• This doesn’t stop
scientists from
speculating or speaking
hopefully about
relationships
MRI instrumentation
MRI instrumentation can be
used in many ways to learn
about brain tissue, structure,
and activity
Schematic of MR scanner and magnetic field lines
Liquid helium
Magnetic field lines
http://www.magnet.fsu.edu/education/tutorials/magnetacademy/mri/fullarticle.html
Magnetic resonance signal: The free induction decay experiment
Beaker of water in a perfectly uniform magnetic
field
RF signal excites spins
CoilBloch Purcell
This is the voltage signal we measure and interpret
RF signal is detected
Using a gradient to resolve the spatial locations of two beakers of water
• In a uniform magnetic
field, the free induction
decay produces a single
frequency response from
the two beakers
Magnetic gradients are used to resolve spatial signals
0 2 4 6 8 10-1
-0.5
0
0.5
1
Time (s)
0 2 4 6 8 10-1
-0.5
0
0.5
1
Time (s)
Gradient
• If we introduce a magnetic
field gradient across the two
beakers, the emissions are at
different frequencies;
frequency encodes position
• This principle is developed
into a much more complex
methodology to form images
rapidly as gradients are
switched
The emission frequency varies with local field
MRI instrumentation occupies a couple of rooms
• To measure RF signals in the
living human requires a
substantial set of calibrated
and carefully controlled
components
• The main magnet is super-
cooled (helium) so that the
coils are injected with current
once and no further energy is
needed to sustain the magnet
(Machine room)
Modern MRI scanners human (A-C) and small bore animal (D)
• MRI scanners are
noninvasive
• The RF coils and gradients
are programmable, which
produces many different
types of measurements
• We are still learning ways to
create and interpret MRI data
GE Siemens
Philips
(D)
Bruker
MRI indirectly measures activity
• Blood oxygen level dependent (BOLD) signal
Neural and vascular activity are coupled
Source of
noise
Walter K.
“On turning down a left occipital bone
flap, a large angry-looking angioma
arteriale racemosum of the left occ.
Lobe was disclosed which extensively
involved the visual cortex.
The haemorrhage occasioned by the
bone flap was so excessive that the
operation had to be abandoned
without touching the tumour.
A decompression, however, was made.
The patient was discharged … with
greatly improved vision.”
Observations Upon the Vascularity of the Human
Occipital Lobe During Visual Activity
J.F. Fulton, M.D. (1928)
Neural and vascular activity are coupled
• Subject noted that ‘the noise in the
back of his head increased in intensity
when he was using his eyes.’
• No increase for hearing, touch or
smell
• Increased more when he tried
harder
Source of
noise
Walter K.
Observations Upon the Vascularity of the Human
Occipital Lobe During Visual Activity
J.F. Fulton, M.D. (1928)
Conclusion: Blood flows to the active parts
of the brain when we use a sense (vision).
The blood is an indicator of neural activity.
MRI scanners can measure the blood oxygen level
B0 RF
• There is more oxygenated
blood in regions of the brain
that are active compared to
control regions
• If we alternately present a
stimulus and take it away, the
scanner signal measures the
small, spatially localized,
modulations of the blood oxygen
level over time
Macular stimulation
Perimacular stimulation
Remarkable progress from PET to advanced MRI in 25 years m
m
1986
Voxel size
mm
Fox et al., 1986Nature
(Wandell and Winawer, 2011)
4020
1052.5
1 cm
2.5 5 10 20 40
Eccentricity (deg)
V1
V2
V3
2009
mm
mm
Voxel size
Human eccentricity mapping with fMRI
(Engel et al., 1994,1997; Sereno; Tootell, DeYoe; Others)
• Inflated brain• Gray/white
are sulci/gyri
Pseudo-color representation of visual field map
Angular measurements delineate visual field map boundaries
Combining eccentricity and angle data yields maps
Visual field map reviews366 Neuron 56, October 25, 2007
Vision Research 51 (2011) 718-737
• Maps tile the occipital lobe
• Extend into IPS and VOT
• Response properties differ
• Identification from gross anatomy
Quantitative modeling: the population receptive field (pRF)
‘Responses can be
obtained in a given optic
nerve fiber only upon
illumination of a certain
restricted region of the
retina, termed the
receptive field of the fiber
(Hartline, 1936)’.
+ On0 Off
• Functional description
• Stimulus-referred
Sherrington, 1910Kuffler, 1953
Population receptive field idea
x
y
time
Stimulus
Time (sec)s1
x
y
(x1,y1,s1)
Parameters
% B
OL
D
Predicted BOLD (including HRF)
Observed
• For each voxel, find a
spatial receptive field that
explains the fMRI
measurement.
• The spatial RF model is
the object of interest.
• Minimally, the model is
linear in contrast and has
an (x,y) location in the
visual field and a spread
• More complex models are
also being studied (e.g.,
CSS)
Population receptive field idea
x
y
time
Time (sec)s1
x
y
(x1,y1,s1)
Parameters
% B
OL
D
Predicted BOLD (including HRF)
Observed
Stimulus
• For each voxel, find a
spatial receptive field that
explains the fMRI
measurement.
• The spatial RF model is
the object of interest.
• Minimally, the model is
linear in contrast and has
an (x,y) location in the
visual field and a spread
• More complex models are
also being studied (e.g.,
CSS)
Population receptive field idea
x
y
time
Stimulus
Time (sec)s1
x
y
(x1,y1,s1)
Parameters
% B
OL
D
Predicted BOLD (including HRF)
Observed
Stimulus
• For each voxel, find a
spatial receptive field that
explains the fMRI
measurement.
• The spatial RF model is
the object of interest.
• Minimally, the model is
linear in contrast and has
an (x,y) location in the
visual field and a spread
• More complex models are
also being studied (e.g.,
CSS)
PRF size varies substantially and regularly across visual cortex
pR
F s
ize
(deg
)
pRF eccentricity (deg)
• At common eccentricities, different maps have different pRF sizes• PRF size increases with eccentricity for all maps• Bands are bootstrap estimates of the standard error
• Attention
• Stability and Plasticity
• Prosopagnosia
• Development and aging
• Autism
• Alzheimer’s disease
Trends in Cognitive Sciences, June 2015, Vol. 19, No. 6 349
Image-computable cortical models – Winawer lab
Neurological surprises
The case of a missing optic chiasm (Hoffman et al., 2012)
LH
Right visual field Left visual field
The axonal pathways from the eye to brain
• The optic nerve from each eye meets in the optic chiasm
• Half the fibers cross to the other side of the brain
(seen from bottom)
• The chiasm is visible in a standard anatomical image
The optic chiasm
Diffusion MRI (dMRI)
• We can measure the axonal pathways using another MRI modality, diffusion MRI (dMRI) coupled with computational algorithms (tractography)
• Eye movement (nystagmus)
• Discovered during routine testing for nystagmus
• Resolved after a few weeks
A subject without a chiasm (Achiasmic)
Right visual field Left visual field
Achiasmic axonal pathways from the eye to brain
• For this subject, the optic nerves do not cross!
• The right eye sends signals only to the right brain, and the left eye only to the left brain
(seen from bottom)
Left eye Right eye• Slight decrease in visual
acuity
• Slightly reduced
peripheral visual fields
• No stereopsis
• Prominent infantile and
see-saw nystagmus
(resolved)
Subject AC2 characteristics
FMRI confirms that the right and
left visual fields are overlaid in cortex
Right hemisphere Left hemisphere
L hemisphere
Left visual field
FMRI confirms that the right and
left visual fields are overlaid in cortex
Right hemisphere Left hemisphere
L hemisphere
Right visual field
Modeling the time course (1 Gaussian)
Modeling the time course (2 Gaussians)
Achiasmic - Folded representation
V1
ControlRight retina
Temporal
LH
Right hemisphere
Achiasmic - Folded representation
V1
AchiasmicRight retina
Temporal
LH
Right hemisphere
Summary – the system view
1. A genetic defect that
disrupts crossing at the
chiasm signaling causes a
developmental
reorganization in visual
cortex.
2. Despite the profoundly
disrupted V1 maps, the
rest of the brain figures
out what to do.
Brain plasticity
Specializations of brain function
Wandell et al. 2007, New Encyclopedia of Neurosciences
Reading-
specific loss
Damage to small regions of
gray matter can produce very
specific cognitive problems,
such as face-blindness, loss
of color vision, loss of motion
perception, or loss of
reading ability
Brain plasticity and stability
In development, timing Matters: The search for miracle cures
Recovery from early blindness(Gregory and Wallace, 1963)
Gregory’s patient SB
• Born in 1900, lost site in both eyes
because of corneal infections
• Prior to 2 years of age; kept
bandaged to reduce puss
• Went to a school for the blind to
learn a trade; married
• Received a corneal graft in London
at the age of 52
Gregory’s patient SB
Quite recently he had been
struck by how objects changed
their shape when he walked
round them. He would look at a
lamp post, walk round it, stand
studying it from a different
aspect, and wonder why it looked
different and yet the same.
(Gregory, 1974, p. 111)
Michael May(images courtesy Michael May)
Images from Michael May, Sendero Group
• Chemical explosion (3 yrs old)
• One eye lost; other cornea (and limbic stem cells) destroyed
• Blind (no contrast or form) from age 3 through 46
Recovered sight?(images courtesy Michael May)
Limbic stem cells and corneal replacement
84 m
• Similar to controls at
low spatial frequency
• Substantially worse
above 0.25 cpd
• Constant for the 7
years following surgery
Controls
MM
Perceptual contrast sensitivity functions
Specializations of brain function
Wandell et al. 2007, New Encyclopedia of Neurosciences
Reading-
specific loss
Damage to small regions of
gray matter can produce very
specific cognitive problems,
such as face-blindness, loss
of color vision, loss of motion
perception, or loss of
reading ability
MOTIONMOTION
14 deg
Ca
PO
MM
Ca
PO
Control
MM has an unusual cortical map
MM
AAB
Motion selective cortex
• Responds powerfully
• Is organized as a map
• Has the same size as in
controls
14 deg
LiGFuG
Control
Object and face-related responses
LiG
FuG
Posterior
Medial
Diagnosing the reading circuitry
VWFAV1
Retina
LGN
Optics
PMK
K
Extrastriate
Pulvinar
VWFA
Wernicke
BrocaSTG
IPS
SLF
Arcuate
VOF
ILF
Locating reading circuits and maps
VWFA - essential for reading, but not unique to reading
Measuring the activity while reading (fMRI)
We can see the locations of the cortical activations during reading
Through the maps and on to the VWFA
Field of view in reading circuitry of a single subject
0 1.00.5
5°10°
Sub 20
10020 60
% variance explained by
pRF model (word stimuli)
Using pRF methods,
we have learned that
the portion of cortex
engaged in reading
only sees a small part
of the visual field
This may be why it is
very hard to read in
the peripheral field
Small field of view for the reading circuitry
15 deg
5 deg
Le et al. 2017 Journal of Vision
Left and right hemispheres
Using pRF methods,
we have learned that
the portion of cortex
engaged in reading
only sees a small part
of the visual field
This may be why it is
very hard to read in
the peripheral field
Field of view of the VOT reading reading circuitry
• There are significant differences between participants
• We are correlating these differences with measures of word recognition
• With colleagues we are studying how the FOV in Israeli readers
Sub01 Sub02 Sub03 Sub04 Sub05
Sub06 Sub07 Sub08 Sub09 Sub10
Sub11 Sub12 Sub13 Sub14 Sub15
Sub16 Sub17 Sub18 Sub19 Sub20
5°10°
Left hemisphere
only
Diagnosing the reading circuitry
Long-range neural connections for reading
Inferior fronto-occipital fasciculum
150 Directions, 2 mm3, B=2000 projected on a 1 mm3 T1 anatomical image
Franco Pestilli 2014 - Stanford University
White matter fascicles are generatedby step-wise sampling of local diffusion information
Left IFOF
VWFAV1
Retina
LGN
Optics
PMK
K
Extrastriate
Pulvinar
VWFA
WernickeBroca
STG
IPSSLF
ArcuateVOF
ILF
Major components of the reading pathway
Wandell and Le (2017)
Learning to See WordsB.A. Wandell, A. Rauschecker and J. Yeatman (2012).Annual Review of Psychology Vol. 63, pp.31-53.
The goal: Diagnosis
Identifying the locations
and responses in a poor
reader that differ
significantly from
measurements in good
readers
Diffusion (FA) changes differs between good and poor readers
• Measured brain and behavior at 4 time points (data management!)
• The first measurements predict reading over the next few years
• The rate and direction of FA development differs between good and poor readers in both the Arcuate and the ILF
Blue: Good readersRed: Poor readers
More linear
(FA)
More circular
Fractional anisotropy (displaced)
Diffusion (FA) changes differs between good and poor readers
Mean FA
development
slopes
Left ILF
Fractional anisotropy
Age• Measured brain and behavior at 4 time points (data management!)
• The first measurements predict reading over the next few years
• The rate and direction of FA development differs between good and poor readers in both the Arcuate and the ILF
Correlations between tract diffusion change and seeing words
• Development measured
by dMRI in the ILF and
Arcuate, but not others
tracts, correlates with
the ability to rapidly see
words
• This is one reason we
think that the wires are
active, changing in
response to learning and
memory
r = 0.51
(Yeatman et al., 2012, PNAS)
Mea
sure
d r
ead
ing
sco
re
Measured FA development rate
Neuroprognosis
• Simple models that combine tissue
properties from two tracts (ILF and
AF) predict measured reading skill
• The predictions are not yet useful;
they are statistically reliable
r = 0.66 (43%)7-11 yrs
Predicted reading score
Mea
sure
d r
ead
ing
sco
re
Predicting reading scores from rate of white matter development
(Yeatman et al., 2012)
Connectionism: Mismatch hypothesis
VOT Specialized processing for faces, words, other things
General visual inputs
Connectionism: Mismatch hypothesis
General visual inputs
Computational neuroimaging: Reading circuitry
• We have made progress in computational
neuroimaging methods, so that we have the maps
and some computational methods for key
properties (pRF)
• We can follow responses to words up to VOT cortex
in living human subjects at mm resolution; using
dMRI we can identify the major tracts that carry
these signals and that the cortex learns to recognize
words using these circuits
• We hope to build computational models based on
these MR measurements of the reading circuits,
relating the neuroimaging measures to behavior,
and to understand the biological reasons for
success and failure of the reading circuitry in each
child
Summary
• A very brief introduction to human brain
structures
• The MRI instrument: basic signals and
image formation
• Interesting case studies using MRI
• MRI diagnostic tools for
psychiatry: Reading impairment
Thanks to NIH, NSF, Simons, Weston-Havens, Wallenberg Foundation
HiromasaTakemura
Franco Pestilli
Jason Yeatman
Ariel Rokem
Kendrick Kay
Jon Winawer
Alyssa Brewer
Michal Ben-Shachar
Serge Dumoulin
Rosemary Le
Andreas Rauschecker
BobDougherty
Gunnar Schaefer
Michael Perry
YoichiroMasuda
Hiroshi Horiguchi
Heidi Baseler
Alex Wade
Anthony Morland
Stephen Engel
Stanford University
April 5, 2018
Workshop on Medical VR and AR
To register go to:
https://scien.stanford.edu/index.php/medicalvrar/
Panel discussions with
Stanford physicians who are using VR and AR applications
Talks by researchers who are
developing VR and AR technologies to advance healthcare
Interactive demos featuring
research projects, clinical applications and startup ventures.