neural decoding - the university of edinburgh · causal decoding organism faces causal (on-line)...
TRANSCRIPT
![Page 1: Neural Decoding - The University of Edinburgh · Causal decoding Organism faces causal (on-line) decoding problem. Prediction of the current/future stimulus requires temporal correlation](https://reader033.vdocuments.mx/reader033/viewer/2022042606/5f7c9a11102319160d11141f/html5/thumbnails/1.jpg)
Neural Decoding
Matthias Hennig
School of Informatics, University of Edinburgh
January 2019
0Acknowledgements: Mark van Rossum, Chris Williams and slides from GatsbyLiam Paninski.
1 / 54
![Page 2: Neural Decoding - The University of Edinburgh · Causal decoding Organism faces causal (on-line) decoding problem. Prediction of the current/future stimulus requires temporal correlation](https://reader033.vdocuments.mx/reader033/viewer/2022042606/5f7c9a11102319160d11141f/html5/thumbnails/2.jpg)
Decoding brain activity
Classi�cationWhich one?
Reconstruction:Homunculus
2 / 54
![Page 3: Neural Decoding - The University of Edinburgh · Causal decoding Organism faces causal (on-line) decoding problem. Prediction of the current/future stimulus requires temporal correlation](https://reader033.vdocuments.mx/reader033/viewer/2022042606/5f7c9a11102319160d11141f/html5/thumbnails/3.jpg)
The Homunculus
Flynculus [Rieke et al., 1996]
3 / 54
![Page 4: Neural Decoding - The University of Edinburgh · Causal decoding Organism faces causal (on-line) decoding problem. Prediction of the current/future stimulus requires temporal correlation](https://reader033.vdocuments.mx/reader033/viewer/2022042606/5f7c9a11102319160d11141f/html5/thumbnails/4.jpg)
Overview
1 Stimulus discrimination, signal detection theory2 Maximum likelihood and MAP decoding3 Bounds and Fisher information4 Spike train decoding and GLMs
4 / 54
![Page 5: Neural Decoding - The University of Edinburgh · Causal decoding Organism faces causal (on-line) decoding problem. Prediction of the current/future stimulus requires temporal correlation](https://reader033.vdocuments.mx/reader033/viewer/2022042606/5f7c9a11102319160d11141f/html5/thumbnails/5.jpg)
Why decoding?
Understanding the neural code.Given spikes, what was the stimulus?What aspects of the stimulus does the system encode? (capacityis limited)What information can be extracted from spike trains:
By “downstream” areas? Homunculus.By the experimenter? Ideal observer analysis.
What is the coding quality?Design of neural prosthetic devices
Related to encoding, but encoding does not answer above questionsexplicitly.
5 / 54
![Page 6: Neural Decoding - The University of Edinburgh · Causal decoding Organism faces causal (on-line) decoding problem. Prediction of the current/future stimulus requires temporal correlation](https://reader033.vdocuments.mx/reader033/viewer/2022042606/5f7c9a11102319160d11141f/html5/thumbnails/6.jpg)
Decoding examples
Hippocampal place cells: how is location encoded?Retinal ganglion cells: what information is sent to the brain? Whatis discarded?Motor cortex: how can we extract as much information as possiblefrom a collection of M1 cells?
6 / 54
![Page 7: Neural Decoding - The University of Edinburgh · Causal decoding Organism faces causal (on-line) decoding problem. Prediction of the current/future stimulus requires temporal correlation](https://reader033.vdocuments.mx/reader033/viewer/2022042606/5f7c9a11102319160d11141f/html5/thumbnails/7.jpg)
Decoding theory
Probability of the stimulus, prior: P(s)Probability of a measured neural response: P(r)Joint probability of stimulus and response: P(r, r)Conditional probabilities: P(r|s), P(s|r)Marginal: P(r) =
∑s P(r|s)P(s)
Note: P(r, s) = P(r|s)P(s) = P(s|r)P(r)
Bayes theorem:
P(s|r) =P(r|s)P(s)
P(r)
Note that we need to know the stimulus statistics.
7 / 54
![Page 8: Neural Decoding - The University of Edinburgh · Causal decoding Organism faces causal (on-line) decoding problem. Prediction of the current/future stimulus requires temporal correlation](https://reader033.vdocuments.mx/reader033/viewer/2022042606/5f7c9a11102319160d11141f/html5/thumbnails/8.jpg)
Example: Discrimination between two stimuli
Test subject report left or right motion (2 alternative forced choice,2AFC). See [Dayan and Abbott, 2002], chapter 3.2.
8 / 54
![Page 9: Neural Decoding - The University of Edinburgh · Causal decoding Organism faces causal (on-line) decoding problem. Prediction of the current/future stimulus requires temporal correlation](https://reader033.vdocuments.mx/reader033/viewer/2022042606/5f7c9a11102319160d11141f/html5/thumbnails/9.jpg)
MT neurons in this task
[Britten et al., 1992]
Some single neurons do as well as animal!Possibility for averaging might be limited due to correlation?Population might still be better/faster? [Cohen and Newsome, 2009]
9 / 54
![Page 10: Neural Decoding - The University of Edinburgh · Causal decoding Organism faces causal (on-line) decoding problem. Prediction of the current/future stimulus requires temporal correlation](https://reader033.vdocuments.mx/reader033/viewer/2022042606/5f7c9a11102319160d11141f/html5/thumbnails/10.jpg)
[Britten et al., 1992] Assuming rate histograms are Gaussian with equalvariance σ2, the discriminability is
d ′ =< r >+ − < r >−
σ
10 / 54
![Page 11: Neural Decoding - The University of Edinburgh · Causal decoding Organism faces causal (on-line) decoding problem. Prediction of the current/future stimulus requires temporal correlation](https://reader033.vdocuments.mx/reader033/viewer/2022042606/5f7c9a11102319160d11141f/html5/thumbnails/11.jpg)
Discriminate between response distributions P(r−) and P(r+).(directions + and −), and discrimination threshold z on firing rate:
Hit rate: β(z) = P(r ≥ z|+)
False alarm rate: α(z) = P(r ≥ z|−)
stimulus correct False+ β 1− β- 1− α α
Probability of correct answer: (β(z) + 1− α(z))/2Can be used to find the optimal z.
11 / 54
![Page 12: Neural Decoding - The University of Edinburgh · Causal decoding Organism faces causal (on-line) decoding problem. Prediction of the current/future stimulus requires temporal correlation](https://reader033.vdocuments.mx/reader033/viewer/2022042606/5f7c9a11102319160d11141f/html5/thumbnails/12.jpg)
ROC curves
Discriminate between response distributions P(r−) and P(r+).The Receiver Operating Characteristic (ROC) gives graphical intuition:
Vary decision threshold and measure error rates.Larger area under curve means better discriminability.Shape relates to underlying distributions.
12 / 54
![Page 13: Neural Decoding - The University of Edinburgh · Causal decoding Organism faces causal (on-line) decoding problem. Prediction of the current/future stimulus requires temporal correlation](https://reader033.vdocuments.mx/reader033/viewer/2022042606/5f7c9a11102319160d11141f/html5/thumbnails/13.jpg)
[Britten et al., 1992]
P(correct) =
∫ 1
0βdα
When responses are Gaussian:
P(correct) =12
erfc(< r >− − < r >+
2σ
)13 / 54
![Page 14: Neural Decoding - The University of Edinburgh · Causal decoding Organism faces causal (on-line) decoding problem. Prediction of the current/future stimulus requires temporal correlation](https://reader033.vdocuments.mx/reader033/viewer/2022042606/5f7c9a11102319160d11141f/html5/thumbnails/14.jpg)
Population decoding
[Dayan and Abbott (2001) after Theunissen and Miller (1991)]
Cricket Cercal System: Information about wind direction is encoded byfour types of neurons (
f (s)
rmax
)= [cos(s − sa)]+
14 / 54
![Page 15: Neural Decoding - The University of Edinburgh · Causal decoding Organism faces causal (on-line) decoding problem. Prediction of the current/future stimulus requires temporal correlation](https://reader033.vdocuments.mx/reader033/viewer/2022042606/5f7c9a11102319160d11141f/html5/thumbnails/15.jpg)
Let ca denote a unit vector in the direction of sa, and v be a unitvector parallel to the wind velocity(
f (s)
rmax
)= [v · ca]+
Crickets are Cartesian, 4 directions 45◦, 135◦, −135◦, −45◦
Population vector is defined as
vpop =4∑
a=1
(r
rmax
)a
ca
15 / 54
![Page 16: Neural Decoding - The University of Edinburgh · Causal decoding Organism faces causal (on-line) decoding problem. Prediction of the current/future stimulus requires temporal correlation](https://reader033.vdocuments.mx/reader033/viewer/2022042606/5f7c9a11102319160d11141f/html5/thumbnails/16.jpg)
Vector method of decoding
[Dayan and Abbott (2001) after Salinas and Abbott (1994)]
16 / 54
![Page 17: Neural Decoding - The University of Edinburgh · Causal decoding Organism faces causal (on-line) decoding problem. Prediction of the current/future stimulus requires temporal correlation](https://reader033.vdocuments.mx/reader033/viewer/2022042606/5f7c9a11102319160d11141f/html5/thumbnails/17.jpg)
Primary Motor Cortex (M1)
Certain neurons in M1 of the monkey can be described by cosinefunctions of arm movement direction (Georgopoulos et al, 1982)Similar to cricket cercal system, but note:
Non-zero offset rates r0(f (s)− r0
rmax
)= v · ca
Non-orthogonal: there are many thousands of M1 neurons thathave arm-movement-related tuning curves
17 / 54
![Page 18: Neural Decoding - The University of Edinburgh · Causal decoding Organism faces causal (on-line) decoding problem. Prediction of the current/future stimulus requires temporal correlation](https://reader033.vdocuments.mx/reader033/viewer/2022042606/5f7c9a11102319160d11141f/html5/thumbnails/18.jpg)
[Dayan and Abbott (2001) after Kandel et al (1991)]
18 / 54
![Page 19: Neural Decoding - The University of Edinburgh · Causal decoding Organism faces causal (on-line) decoding problem. Prediction of the current/future stimulus requires temporal correlation](https://reader033.vdocuments.mx/reader033/viewer/2022042606/5f7c9a11102319160d11141f/html5/thumbnails/19.jpg)
Optimal Decoding
p(s|r) =p(r|s)p(s)
p(r)
Maximum likelihood decoding (ML): s = argmaxs p(r|s)
Maximum a posteriori (MAP): s = argmaxs p(s)p(r|s)
Note these two are equivalent if p(s) is flat.Bayes: mimimize loss
sB = argmins∗
∫s
L(s, s∗)p(s|r)ds
For squared loss L(s, s∗) = (s − s∗)2, optimal s∗ is posteriormean, sB =
∫s p(s|r)s.
19 / 54
![Page 20: Neural Decoding - The University of Edinburgh · Causal decoding Organism faces causal (on-line) decoding problem. Prediction of the current/future stimulus requires temporal correlation](https://reader033.vdocuments.mx/reader033/viewer/2022042606/5f7c9a11102319160d11141f/html5/thumbnails/20.jpg)
Optimal Decoding for the cricket
For the cercal system, assuming indep. noise
p(r|s) =∏
a
p(ra|s)
where each p(ra|s) is modelled as a Gaussian with means andvariancesp(s) is uniform (hence MAP=ML)ML decoding finds a peak of the likelihoodBayesian method finds posterior meanThese methods improve performance over the vector method (butnot that much, due to orthogonality...)
20 / 54
![Page 21: Neural Decoding - The University of Edinburgh · Causal decoding Organism faces causal (on-line) decoding problem. Prediction of the current/future stimulus requires temporal correlation](https://reader033.vdocuments.mx/reader033/viewer/2022042606/5f7c9a11102319160d11141f/html5/thumbnails/21.jpg)
Cricket Cercal System
[Dayan and Abbott (2001) after Salinas and Abbott (1994)]
21 / 54
![Page 22: Neural Decoding - The University of Edinburgh · Causal decoding Organism faces causal (on-line) decoding problem. Prediction of the current/future stimulus requires temporal correlation](https://reader033.vdocuments.mx/reader033/viewer/2022042606/5f7c9a11102319160d11141f/html5/thumbnails/22.jpg)
General Consideration of Population Decoding
[Dayan and Abbott (2001)]
Gaussian tuning curves.
22 / 54
![Page 23: Neural Decoding - The University of Edinburgh · Causal decoding Organism faces causal (on-line) decoding problem. Prediction of the current/future stimulus requires temporal correlation](https://reader033.vdocuments.mx/reader033/viewer/2022042606/5f7c9a11102319160d11141f/html5/thumbnails/23.jpg)
Poisson firing model over time T , count na = raT spikes.
p(r|s) =N∏
a=1
(fa(s)T )na
na!exp(−fa(s)T )
log p(r|s) =N∑
a=1
na log fa(s) + . . .
The terms in . . . are independent of s, and we assume∑
a fa(s) isindependent of s (all neurons sum to the same average firing rate).
23 / 54
![Page 24: Neural Decoding - The University of Edinburgh · Causal decoding Organism faces causal (on-line) decoding problem. Prediction of the current/future stimulus requires temporal correlation](https://reader033.vdocuments.mx/reader033/viewer/2022042606/5f7c9a11102319160d11141f/html5/thumbnails/24.jpg)
ML decoding
sML is stimulus that maximizes log p(r|s), determined by
N∑a=1
raf ′a(sML)
fa(sML)= 0
If all tuning curves are Gaussian fa = A exp[−(s − sa)2/2σ2w ] then
sML =
∑a rasa∑
a ra
which is simple and intuitive, known as Center of Mass (cfpopulation vector)
24 / 54
![Page 25: Neural Decoding - The University of Edinburgh · Causal decoding Organism faces causal (on-line) decoding problem. Prediction of the current/future stimulus requires temporal correlation](https://reader033.vdocuments.mx/reader033/viewer/2022042606/5f7c9a11102319160d11141f/html5/thumbnails/25.jpg)
Accuracy of the estimator
Bias and variance of an estimator sest
best (s) = 〈sest〉 − sσ2
est (s) = 〈(sest − 〈sest〉)2〉〈(s − sest )
2〉 = b2est (s) + σ2
est
Thus for an unbiased estimator, MSE 〈(s − sest )2〉 is given by σ2
est ,the variance of the estimator
25 / 54
![Page 26: Neural Decoding - The University of Edinburgh · Causal decoding Organism faces causal (on-line) decoding problem. Prediction of the current/future stimulus requires temporal correlation](https://reader033.vdocuments.mx/reader033/viewer/2022042606/5f7c9a11102319160d11141f/html5/thumbnails/26.jpg)
Fisher information
Fisher information is a measure of the curvature of the loglikelihood near its peak
IF (s) =
⟨−∂
2 log p(r|s)
∂s2
⟩s
= −∫
drp(r|s)∂2 log p(r|s)
∂s2
(the average is over trials measuring r while s is fixed)Cramér-Rao bound says that for any estimator[Cover and Thomas, 1991]
σ2est ≥
(1 + b′est (s))2
IF (s)
efficient estimator if σ2est =
(1+b′est (s))2
IF (s).
In the bias-free case an efficient estimator σ2est = 1/IF (s).
ML decoder is typically efficient when N →∞.26 / 54
![Page 27: Neural Decoding - The University of Edinburgh · Causal decoding Organism faces causal (on-line) decoding problem. Prediction of the current/future stimulus requires temporal correlation](https://reader033.vdocuments.mx/reader033/viewer/2022042606/5f7c9a11102319160d11141f/html5/thumbnails/27.jpg)
Fisher information
In homogeneous systems IF indep. of s.
More generally Fisher matrix (IF )ij(s) =⟨−∂2 log p(r|s)
∂si∂sj
⟩s.
Taylor expansion of Kullback-LeiblerDKL(P(s),P(s + δs)) ≈
∑ij δsiδsj(IF )ij
Not a Shannon information measure (not in bits), but related toShannon information in special cases,e.g.[Brunel and Nadal, 1998, Yarrow et al., 2012].
27 / 54
![Page 28: Neural Decoding - The University of Edinburgh · Causal decoding Organism faces causal (on-line) decoding problem. Prediction of the current/future stimulus requires temporal correlation](https://reader033.vdocuments.mx/reader033/viewer/2022042606/5f7c9a11102319160d11141f/html5/thumbnails/28.jpg)
Fisher information for a population
For independent Poisson spikers
IF (s) =
⟨−∂
2 log p(r|s)
∂s2
⟩= T
∑a
〈ra〉
((f ′a(s)
fa(s)
)2
− f ′′a (s)
fa(s)
)
For dense, symmetric tuning curves, the second term sums to zero.Using fa(s) = 〈ra〉 we obtain
IF (s) = T∑
a
(f ′a(s))2
fa(s)
For dense fa(s) = Ae−(s−s0+a.ds)2/2σ2w with density ρ = 1/ds, sum
becomes integralIF =
√2πTAρ/σw
28 / 54
![Page 29: Neural Decoding - The University of Edinburgh · Causal decoding Organism faces causal (on-line) decoding problem. Prediction of the current/future stimulus requires temporal correlation](https://reader033.vdocuments.mx/reader033/viewer/2022042606/5f7c9a11102319160d11141f/html5/thumbnails/29.jpg)
For Gaussian tuning curves
[Dayan and Abbott (2001)]
Note that Fisher information vanishes at peak as f ′a(s) = 0 there.Can be used to create optimal tuning curves,[Dayan and Abbott, 2002] chapter 3.3.Discriminability d ′ = ∆s
√IF (s) for a small ∆F .
29 / 54
![Page 30: Neural Decoding - The University of Edinburgh · Causal decoding Organism faces causal (on-line) decoding problem. Prediction of the current/future stimulus requires temporal correlation](https://reader033.vdocuments.mx/reader033/viewer/2022042606/5f7c9a11102319160d11141f/html5/thumbnails/30.jpg)
FI predicts human performance
[Dayan and Abbott (2001)]
Orientation discrimination for stimuli of different size (different N)Solid line: estimated minimum standard deviation at the CramerRao boundTriangles: human standard deviation as function of stimulus size(expressed in N)
30 / 54
![Page 31: Neural Decoding - The University of Edinburgh · Causal decoding Organism faces causal (on-line) decoding problem. Prediction of the current/future stimulus requires temporal correlation](https://reader033.vdocuments.mx/reader033/viewer/2022042606/5f7c9a11102319160d11141f/html5/thumbnails/31.jpg)
Slope as strategy
From paper on bat echo location [Yovel et al., 2010] )31 / 54
![Page 32: Neural Decoding - The University of Edinburgh · Causal decoding Organism faces causal (on-line) decoding problem. Prediction of the current/future stimulus requires temporal correlation](https://reader033.vdocuments.mx/reader033/viewer/2022042606/5f7c9a11102319160d11141f/html5/thumbnails/32.jpg)
Spike train decoding
Dayan and Abbott §3.4Estimate the stimulus from spike times ti to minimize e.g.〈s(t)− sest (t)〉2
First order reconstruction:
sest (t − τ0) =∑
ti
K (t − ti)− 〈r〉∫
dτK (τ)
The second term ensures that 〈sest (t)〉 = 0Delay τ0 can be included to make decoding easier: predictstimulus at time t − τ0 based on spikes up to time t
32 / 54
![Page 33: Neural Decoding - The University of Edinburgh · Causal decoding Organism faces causal (on-line) decoding problem. Prediction of the current/future stimulus requires temporal correlation](https://reader033.vdocuments.mx/reader033/viewer/2022042606/5f7c9a11102319160d11141f/html5/thumbnails/33.jpg)
Causal decoding
Organism faces causal (on-line) decoding problem.Prediction of the current/future stimulus requires temporalcorrelation of the stimulus. Example: in head-direction systemneural code correlates best with future direction.Requires K (t − ti) = 0 for t ≤ ti .
sest (t − τ0) =∑
ti
K (t − ti)− 〈r〉∫
dτK (τ)
Delay τ0 buys extra time
33 / 54
![Page 34: Neural Decoding - The University of Edinburgh · Causal decoding Organism faces causal (on-line) decoding problem. Prediction of the current/future stimulus requires temporal correlation](https://reader033.vdocuments.mx/reader033/viewer/2022042606/5f7c9a11102319160d11141f/html5/thumbnails/34.jpg)
Causal decoding
Delay τ0 = 160 ms. (B: shifted/causal kernel, C: non-causal kernel)At time t estimate s(t − τ0):Spikes 1..4: contribute because stimulus is correlated (right tail of K)Spikes 5..7: contribute because of τ0Spikes 8, 9,... : have not occurred yet. Stimulus uncorrelated: Kernelfrom STA [Dayan and Abbott (2001)]
34 / 54
![Page 35: Neural Decoding - The University of Edinburgh · Causal decoding Organism faces causal (on-line) decoding problem. Prediction of the current/future stimulus requires temporal correlation](https://reader033.vdocuments.mx/reader033/viewer/2022042606/5f7c9a11102319160d11141f/html5/thumbnails/35.jpg)
Decoding from a GLM
0 25 50 75 100 125 150 175 2002
0
2
4stimulusSTAGLM
0 25 50 75 100 125 150 175 200
0.0
2.5
5.0
7.5
0 25 50 75 100 125 150 175 200Time bin
5
10
s
35 / 54
![Page 36: Neural Decoding - The University of Edinburgh · Causal decoding Organism faces causal (on-line) decoding problem. Prediction of the current/future stimulus requires temporal correlation](https://reader033.vdocuments.mx/reader033/viewer/2022042606/5f7c9a11102319160d11141f/html5/thumbnails/36.jpg)
H1 neuron ofthe flySolid line isreconstructionusing acausalfilterNote,reconstructionquality willdepend onstimulus
[Dayan and Abbott (2001) after Rieke et al (1997)]
36 / 54
![Page 37: Neural Decoding - The University of Edinburgh · Causal decoding Organism faces causal (on-line) decoding problem. Prediction of the current/future stimulus requires temporal correlation](https://reader033.vdocuments.mx/reader033/viewer/2022042606/5f7c9a11102319160d11141f/html5/thumbnails/37.jpg)
MAP estimation
According to Bayes theorem
p(s|r) = p(r |s)p(s)/p(r)
log p(s|r) = log p(r |s) + log p(s) + c
Tractable as long as rhs. is concave, which excludes heavy tailed p(s).Requires to compute:
s = argmaxs log p(s|r)
This is numerically hard, see [Pillow et al., 2011] for methods.
37 / 54
![Page 38: Neural Decoding - The University of Edinburgh · Causal decoding Organism faces causal (on-line) decoding problem. Prediction of the current/future stimulus requires temporal correlation](https://reader033.vdocuments.mx/reader033/viewer/2022042606/5f7c9a11102319160d11141f/html5/thumbnails/38.jpg)
MAP decoding simulation
[Pillow et al., 2011]
38 / 54
![Page 39: Neural Decoding - The University of Edinburgh · Causal decoding Organism faces causal (on-line) decoding problem. Prediction of the current/future stimulus requires temporal correlation](https://reader033.vdocuments.mx/reader033/viewer/2022042606/5f7c9a11102319160d11141f/html5/thumbnails/39.jpg)
Conclusion stimulus reconstruction
Stimulus reconstruction similar to encoding problem. ButResponse is given, can not be choosen to be whiteImposing causality adds realism but reduces quality
The reconstruction problem can be ill-posed. It is not alwayspossible to reconstruct stimulus (cf dictionary). For instance:complex cell.Still, the cell provides information about the stimulus. Could try toread the code, rather than reconstruct the stimulus (e.g. idealobserver)
39 / 54
![Page 40: Neural Decoding - The University of Edinburgh · Causal decoding Organism faces causal (on-line) decoding problem. Prediction of the current/future stimulus requires temporal correlation](https://reader033.vdocuments.mx/reader033/viewer/2022042606/5f7c9a11102319160d11141f/html5/thumbnails/40.jpg)
Readout of Object Identity from Macaque IT Cortex
[Hung et al., 2005]Recording from ∼ 300 sites in the Inferior Temporal (IT) cortexPresent images of 77 stimuli (of different objects) at variouslocations and scales in the visual field.Task is to categorize objects into 8 classes, or identify all 77objectsPredictions based on one-vs-rest linear SVM classifiers, usingdata in 50 ms bins from 100 ms to 300 ms after stimulus onset
40 / 54
![Page 41: Neural Decoding - The University of Edinburgh · Causal decoding Organism faces causal (on-line) decoding problem. Prediction of the current/future stimulus requires temporal correlation](https://reader033.vdocuments.mx/reader033/viewer/2022042606/5f7c9a11102319160d11141f/html5/thumbnails/41.jpg)
[Hung et al., 2005]
41 / 54
![Page 42: Neural Decoding - The University of Edinburgh · Causal decoding Organism faces causal (on-line) decoding problem. Prediction of the current/future stimulus requires temporal correlation](https://reader033.vdocuments.mx/reader033/viewer/2022042606/5f7c9a11102319160d11141f/html5/thumbnails/42.jpg)
What does this tell us?
Performance of classifiers can provide a lower bound on theinformation available in the population activity.Assuming independence, correlations are ignored. Correlationcould limit or enhance information.Distributed representationLinear classifier can plausibly be implemented in neural hardware
42 / 54
![Page 43: Neural Decoding - The University of Edinburgh · Causal decoding Organism faces causal (on-line) decoding problem. Prediction of the current/future stimulus requires temporal correlation](https://reader033.vdocuments.mx/reader033/viewer/2022042606/5f7c9a11102319160d11141f/html5/thumbnails/43.jpg)
4. Population Encoding
Dayan and Abbott §3.3Population encoding uses a large number of neurons to representinformationAdvantage 1: reduction of uncertainty due to neuronal variability(Improves reaction time).Advantage 2: Ability to represent a number of different stimulusattributes simultaneously (e.g. in V1 location and orientation).
43 / 54
![Page 44: Neural Decoding - The University of Edinburgh · Causal decoding Organism faces causal (on-line) decoding problem. Prediction of the current/future stimulus requires temporal correlation](https://reader033.vdocuments.mx/reader033/viewer/2022042606/5f7c9a11102319160d11141f/html5/thumbnails/44.jpg)
Population codes and correlations: Retina
Fit coupled GLM-model (see encoding) to retina data
[Pillow et al., 2008]
44 / 54
![Page 45: Neural Decoding - The University of Edinburgh · Causal decoding Organism faces causal (on-line) decoding problem. Prediction of the current/future stimulus requires temporal correlation](https://reader033.vdocuments.mx/reader033/viewer/2022042606/5f7c9a11102319160d11141f/html5/thumbnails/45.jpg)
Population codes and correlations: Retina
[Pillow et al., 2008]
45 / 54
![Page 46: Neural Decoding - The University of Edinburgh · Causal decoding Organism faces causal (on-line) decoding problem. Prediction of the current/future stimulus requires temporal correlation](https://reader033.vdocuments.mx/reader033/viewer/2022042606/5f7c9a11102319160d11141f/html5/thumbnails/46.jpg)
Hippocampal Place Cell Decoding
[Brown et al., 1998]Encoding in place cells: modelled as inhomogeneous Poissonprocesses with Gaussian receptive fields (incl. theta oscillations),assuming independenceEncoding as path: random walkBayesian filter decoding: compute posterior and then updateestimate in next time step using information from spikes (combineprior and new information).Non-Bayesian decoding: compute estimate in time steps givenspikes in time window (only current information).
46 / 54
![Page 47: Neural Decoding - The University of Edinburgh · Causal decoding Organism faces causal (on-line) decoding problem. Prediction of the current/future stimulus requires temporal correlation](https://reader033.vdocuments.mx/reader033/viewer/2022042606/5f7c9a11102319160d11141f/html5/thumbnails/47.jpg)
[Brown et al., 1998]
Bayes, maximum likelihood, linear kernel and correlation based.47 / 54
![Page 48: Neural Decoding - The University of Edinburgh · Causal decoding Organism faces causal (on-line) decoding problem. Prediction of the current/future stimulus requires temporal correlation](https://reader033.vdocuments.mx/reader033/viewer/2022042606/5f7c9a11102319160d11141f/html5/thumbnails/48.jpg)
Example: Motor decoding
[Shpigelman et al., 2005]
Rhesus monkey, 43 electrodes in M1Monkey controls cursors on a screen using two manipulanda toperform a centre-out reaching taskPredict hand velocity based on 10 time bins, each of length 100ms in all 43 neurons.Can use linear regression, polynomial regression, Gaussiankernel (support vector regression), spikernel (allows time warping)More sophisticated methods outperform linear regression, butlinear is already decent
State-of-the-art w. Kalman filters [Gilja et al., 2012]
48 / 54
![Page 49: Neural Decoding - The University of Edinburgh · Causal decoding Organism faces causal (on-line) decoding problem. Prediction of the current/future stimulus requires temporal correlation](https://reader033.vdocuments.mx/reader033/viewer/2022042606/5f7c9a11102319160d11141f/html5/thumbnails/49.jpg)
[Shpigelman et al., 2005]
49 / 54
![Page 50: Neural Decoding - The University of Edinburgh · Causal decoding Organism faces causal (on-line) decoding problem. Prediction of the current/future stimulus requires temporal correlation](https://reader033.vdocuments.mx/reader033/viewer/2022042606/5f7c9a11102319160d11141f/html5/thumbnails/50.jpg)
Decoding from your brain
[Miyawaki et al., 2008]
fMRI, linear decoder from voxel activations.50 / 54
![Page 51: Neural Decoding - The University of Edinburgh · Causal decoding Organism faces causal (on-line) decoding problem. Prediction of the current/future stimulus requires temporal correlation](https://reader033.vdocuments.mx/reader033/viewer/2022042606/5f7c9a11102319160d11141f/html5/thumbnails/51.jpg)
Summary
Discrimination between stimuli (or actions) using just recordedspikes is possible and tractable.Full reconstruction of a stimulus is hard, especially when stimulusdimensionality is high.But also unclear to what extent this is this even possible.Decoding can tell us how much information neural activity carriesabout the outside world, it provides an upper bound.BMI applications are on the horizon.
51 / 54
![Page 52: Neural Decoding - The University of Edinburgh · Causal decoding Organism faces causal (on-line) decoding problem. Prediction of the current/future stimulus requires temporal correlation](https://reader033.vdocuments.mx/reader033/viewer/2022042606/5f7c9a11102319160d11141f/html5/thumbnails/52.jpg)
References I
Britten, K. H., Shadlen, M. N., Newsome, W. T., and Movshon, J. A. (1992).The analysis of visual motion: a comparison of neuronal and psychophysical performance.J Neurosci, 12:4745–4765.
Brown, E. N., Frank, L. M., Tang, D., Quirk, M. C., and Wilson, M. A. (1998).A statistical paradigm for neural spike train decoding applied to position prediction fromensemble firing patterns of rat hippocampal place cells.Journal of Neuroscience, 18(18):7411–7425.
Brunel, N. and Nadal, J.-P. (1998).Mutual information, Fisher information, and population coding.Neural Comp., 10:1731–1757.
Cohen, M. R. and Newsome, W. T. (2009).Estimates of the contribution of single neurons to perception depend on timescale andnoise correlation.J Neurosci, 29(20):6635–6648.
Cover, T. M. and Thomas, J. A. (1991).Elements of information theory.Wiley, New York.
Dayan, P. and Abbott, L. F. (2002).Theoretical Neuroscience.MIT press, Cambridge, MA.
52 / 54
![Page 53: Neural Decoding - The University of Edinburgh · Causal decoding Organism faces causal (on-line) decoding problem. Prediction of the current/future stimulus requires temporal correlation](https://reader033.vdocuments.mx/reader033/viewer/2022042606/5f7c9a11102319160d11141f/html5/thumbnails/53.jpg)
References II
Gilja, V., Nuyujukian, P., Chestek, C. A., Cunningham, J. P., Yu, B. M., Fan, J. M.,Churchland, M. M., Kaufman, M. T., Kao, J. C., Ryu, S. I., and Shenoy, K. V. (2012).A high-performance neural prosthesis enabled by control algorithm design.Nat Neurosci, 15(12):1752–1757.
Hung, C. P., Kreiman, G., Poggio, T., and DiCarlo, J. J. (2005).Fast Readout of Object Identity from Macaque Inferior Temporal Cortex.Science, 310:863–866.
Miyawaki, Y., Uchida, H., Yamashita, O., Sato, M.-a., Morito, Y., Tanabe, H. C., Sadato, N.,and Kamitani, Y. (2008).Visual image reconstruction from human brain activity using a combination of multiscalelocal image decoders.Neuron, 60(5):915–929.
Pillow, J. W., Ahmadian, Y., and Paninski, L. (2011).Model-based decoding, information estimation, and change-point detection techniques formultineuron spike trains.Neural computation, 23(1):1–45.
Pillow, J. W., Shlens, J., Paninski, L., Sher, A., Litke, A. M., Chichilnisky, E. J., andSimoncelli, E. P. (2008).Spatio-temporal correlations and visual signalling in a complete neuronal population.Nature, 454(7207):995–999.
53 / 54
![Page 54: Neural Decoding - The University of Edinburgh · Causal decoding Organism faces causal (on-line) decoding problem. Prediction of the current/future stimulus requires temporal correlation](https://reader033.vdocuments.mx/reader033/viewer/2022042606/5f7c9a11102319160d11141f/html5/thumbnails/54.jpg)
References III
Rieke, F., Warland, D., de Ruyter van Steveninck, R., and Bialek, W. (1996).Spikes: Exploring the neural code.MIT Press, Cambridge.
Shpigelman, L., Singer, Y., Paz, R., and Vaadia, E. (2005).Spikernels: Predicting Arm Movements by Embedding Population Spike Rate Patterns inInner-Product Spaces.Neural Comput, 17:671–690.
Yarrow, S., Challis, E., and Seri?s, P. (2012).Fisher and Shannon information in finite neural populations.Neural Comput, 24(7):1740–1780.
Yovel, Y., Falk, B., Moss, C. F., and Ulanovsky, N. (2010).Optimal localization by pointing off axis.Science, 327(5966):701–704.
54 / 54