spike-triggered neural characterization - princeton...

Spike-triggered neural characterizationHoward Hughes Medical Institute and The Salk Intitute

La Jolla CA USAOdelia Schwartz

Gatsby Computational Neuroscience UnitUniversity College London London UKJonathan W Pillow

Department of Brain and Cognitive Sciences andMcGovern Institute for Brain Research MIT

Cambridge MA USANicole C Rust

Howard Hughes Medical Institute andCenter for Neural Science New York University

New York NY USAEero P Simoncelli

Response properties of sensory neurons are commonly described using receptive fields This description may be formalizedin a model that operates with a small set of linear filters whose outputs are nonlinearly combined to determine theinstantaneous firing rate Spike-triggered average and covariance analyses can be used to estimate the filters and nonlinearcombination rule from extracellular experimental data We describe this methodology demonstrating it with simulated modelneuron examples that emphasize practical issues that arise in experimental situations

Keywords spike-triggered analysis characterization neural response receptive field reverse correlation nonlinear

Introduction

A fundamental goal of sensory systems neuroscience isthe characterization of the functional relationship betweenstimuli and neural responses The purpose of such acharacterization is to elucidate the computation beingperformed by the system Many electrophysiologicalstudies in sensory areas describe neural firing rates inresponse to highly restricted sets of stimuli that areparameterized by one or perhaps two stimulus parametersAlthough such Btuning curve[ measurements have led toconsiderable understanding of neural coding they provideonly a partial glimpse of the full neural response functionOn the other hand it is not feasible to measure neuralresponses to all stimuli One way to make progress is torestrict the response function to a particular model (orclass of models) In this modeling approach the problemis reduced to developing a set of stimuli along with amethodology for fitting the model to measurements ofneural responses to those stimuli One wants a model thatis flexible enough to provide a good description of neuralresponses but simple enough that the fitting is bothtractable and well constrained under realistic experimentaldata conditionsOne class of solutions which we refer to as Bspike-

triggered analysis[ has received considerable attention inrecent years due to a variety of new methodologiesimprovements in stimulus generation technology and

demonstration of physiological results In these methodsone generally assumes that the probability of a neuroneliciting a spike (ie the instantaneous firing rate) isgoverned only by recent sensory stimuli More specificallythe response model is assumed to be an inhomogeneousPoisson process whose rate is a function of the stimulipresented during a recent temporal window of fixedduration In the forward neural response model the stimuliare mapped to a scalar value that determines the instanta-neous firing rate of a Poisson spike generator Our job in theanalysis is to work backward From the stimuli that elicitedspikes we aim to estimate this firing rate function Theanalysis of experimental data is thus reduced to examiningthe properties of the stimuli within temporal windowspreceding each recorded spike known as the spike-triggered stimulus ensemble (Figure 1A)Understanding how the spike-triggered distribution

differs from the raw stimuli is key to determining thefiring rate function It is often useful to visualize theanalysis problem geometrically (Figure 1B) Considerinput stimuli which at each time step consist of an arrayof randomly chosen pixel values (8 pixels in thisexample) The neural response at any particular momentin time is assumed to be completely determined by thestimulus segment that occurred during a prespecifiedinterval in the past (6 time steps in this example) Theoverall stimulus dimensionality is high (48 dimensionshere) but we can depict a projection of the stimuli ontotwo vectors The raw stimulus ensemble and the spike-

Journal of Vision (2006) 6 484ndash507 httpjournalofvisionorg6413 484

doi 10 1167 6 4 13 ISSN 1534-7362 ARVOReceived September 4 2005 published July 17 2006

triggered ensemble are then two clouds of points in thisspace Intuitively the task of estimating the neuralresponse function corresponds to describing the ways inwhich these two clouds differ In practice when the inputstimulus space is of high dimensionality one cannotestimate the neural response function without furtherassumptionsSpike-triggered analysis has been employed to estimate

the terms of a WienerVolterra expansion (KorenbergSakai amp Naka 1989 Marmarelis amp Marmarelis 1978Volterra 1959 Wiener 1958) in which the mapping fromstimuli to firing rate is described using a low-orderpolynomial (see Dayan amp Abbott 2001 Rieke Warlandde Ruyter van Steveninck amp Bialek 1997 for a review)Although any reasonable function can be approximated asa polynomial the firing rate nonlinearities found in theresponses of sensory neurons (eg half-wave rectifiedrapidly accelerating and saturating) tend to require apolynomial with many terms (see eg Rieke et al 1997)However the amount of data needed for accurateestimation grows rapidly with the number of termsTherefore in an experimental setting where one canestimate only the first few terms of the expansion thepolynomial places a strong restriction on the nonlinearityAs an alternative to the polynomial approximation one

can assume that the response function operates on a low-dimensional linear subspace of the full stimulus space(Bialek amp de Ruyter van Steveninck 2005 de Ruyter van

Steveninck amp Bialek 1988) That is the response of aneuron is modeled with a small set of linear filters whoseoutputs are combined nonlinearly to generate the instanta-neous firing rate This is in contrast to the WienerVolterraapproach which in general does not restrict the subspacebut places a restriction on the nonlinearity1 By concen-trating the data into a space of reduced dimensionality theneural response can be fit with less restriction on the formof the nonlinearityA number of techniques have been developed to estimate

the linear subspace and subsequently the nonlinearity Inthe most widely used form of this analysis the linear frontend is limited to a single filter that serves as an explicitrepresentation of the Breceptive field[ of the neuron but thenonlinearity is essentially unrestricted With the rightchoice of stimuli this linear filter may be estimated bycomputing the spike-triggered average (STA) stimulus (iethe mean stimulus that elicited a spike) The STA has beenwidely used in studying auditory neurons (eg EggermontJohannesma amp Aertsen 1983) In the visual system STAhas been used to characterize retinal ganglion cells(eg Meister Pine amp Baylor 1994 Sakai amp Naka1987) lateral geniculate neurons (eg Reid amp Alonzo1995) and simple cells in primary visual cortex (V1eg DeAngelis Ohzawa amp Freeman 1993 Jones ampPalmer 1987 McLean amp Palmer 1989) Given the STAfilter one typically has enough experimental data toconstruct a nonparametric estimate of the nonlinearity

Figure 1 The spike-triggered stimulus ensemble (A) Discretized stimulus sequence and observed neural response (spike train) On eachtime step the stimulus consists of an array of randomly chosen values (eight for this example) These could represent for example theintensities of a fixed set of individual pixels on the screen or the contrast of each of a set of fixed sinusoidal gratings that are additivelysuperimposed The neural response at any particular moment in time is assumed to be completely determined by the stimulus segmentthat occurred during a prespecified interval in the past In this figure the segment covers six time steps and lags three time steps behindthe current time (to account for response latency) The spike-triggered ensemble consists of the set of segments associated with spikes(B) Geometric (vector space) view of the spike-triggered ensemble Stimuli (here 48-dimensional) are projected onto two spacendashtimevectors In this example each of the two vectors contained 1 stixel (spacendashtime pixel) set to a value of 1 and the other 47 stixels were setto 0 For these given vectors the projection is equivalent to the intensity of the corresponding stixel in the stimulus More generally onecan project the stimuli onto any two 48-dimensional vectors The spike-triggered stimulus segments (white points) constitute a subset ofall stimulus segments presented (black points)

Journal of Vision (2006) 6 484ndash507 Schwartz et al 485

(ie a lookup table Anzai Ohzawa amp Freeman 1999Chichilnisky 2001 deBoer amp Kuyper 1968 Eggermontet al 1983) For some classes of nonlinearity it has alsobeen shown that one can write down a closed-formsolution for the estimates of the linear filter and non-linearity in a single step (Nykamp amp Ringach 2002)This methodology may be extended to the recovery of

multiple filters (ie a low-dimensional subspace) and thenonlinear combination rule One approach to finding a low-dimensional subspace is the spike-triggered covariance(STC Bialek amp de Ruyter van Steveninck 2005 de Ruytervan Steveninck amp Bialek 1988) STC has been used tocharacterize multidimensional models and a nonlinearcombination rule in systems ranging from the invertebratemotion system (Bialek amp de Ruyter van Steveninck 2005Brenner Bialek amp de Ruyter van Steveninck 2000 deRuyter van Steveninck amp Bialek 1988) to songbirdforebrain auditory neurons (Sen Wright Doupe amp Bialek2000) to vertabrate retina cells (Pillow Simoncelli ampChichilnisky 2003 Schwartz Chichilnisky amp Simoncelli2002) and mammalian cortex (Horwitz Chichilnisky ampAlbright 2005 Rust Schwartz Movshon amp Simoncelli2004 2005 Touryan Lau amp Dan 2002) In additionseveral authors have recently developed subspace estima-tion methods that use higher order statistical measures(Paninski 2003 Sharpee Rust amp Bialek 2003 2004) Areview of spike-triggered subspace approaches may also befound in Ringach (2004) and Simoncelli Pillow Paninskiamp Schwartz (2004)Despite the theoretical elegance and experimental

applicability of the subspace methods there are a host ofissues that an experimentalist is likely to confront whenattempting to use them How should one choose thestimulus space Howmany spikes does one need to collectHow does one know if the recovered filters are significantHow should one interpret the filters How do the filter

responses relate to the nonlinear firing rate function and soon In this article we describe the family of spike-triggeredsubspace methods in some detail placing emphasis onpractical experimental issues and demonstrating these(where possible) with simulations We focus our discussionon the STA and STC analyses which have become quitewidely used experimentally A software implementation ofthe methods described is available on the Internet at http

The linearndashnonlinear Poisson(LNP) model

Experimental approaches to characterizing neurons aregenerally based on an underlying response model Here weassume a model constructed from a cascade of threeoperations

1 a set of linear filters fkY1IkY

mg2 a nonlinear transformation that maps the instanta-

neous responses of these filters to a scalar firing rateand

3 a Poisson spike generation process whose instanta-neous firing rate is determined by the output of thenonlinear stage

This LNP cascade is illustrated in Figure 2 The thirdstage which essentially amounts to an assumption that thegeneration of spikes depends only on the recent stimulus(and not on the history of previous spike times) is oftennot stated explicitly but is critical to the analysisIf we assume a discretized stimulus space we can

express the instantaneous firing rate of the model as

rethtTHORN frac14 NethkY1 sYethtTHORN kY2 s

YethtTHORNIkY

m sYethtTHORNTHORN eth1THORN

where sYethtTHORN is a vector containing the stimuli over an

appropriate temporal window preceding the time t Herethe linear response of filter i (ie the projection or dotproduct of the filter k

Y

i with the stimuli sYethtTHORN) is given by

kY

i I sYethtTHORN The nonlinear transformation N(I) operates over

the linear filter responses

Spike-triggered analysis

We aim to characterize the LNP model by analyzing thespike-triggered stimulus ensemble The spike-triggeredanalysis techniques proceed as follows

1 Estimate the low-dimensional linear subspace (set offilters) This effectively projects the high-dimensionstimulus into a low-dimensional subspace that theneuron cares about

Figure 2 Block diagram of the LNP model On each time step thecomponents of the stimulus vector are linearly filtered by k

Y

0IkY

mThe responses of the linear filters are then passed through anonlinear function N(I) whose output determines the instanta-neous firing rate of a Poisson spike generator


wwwcnsnyuedu~lcvneuralChar

2 Compute the filter responses for the stimulus andestimate the nonlinear firing rate function based onthese responses As noted earlier typical physio-logical data sets allow nonparametric estimates ofthe nonlinearity for one or two filters but requiremore model restrictions as the number of filtersincreases

In the following subsections we describe these steps indetail In the Experimental issues section we also stressthe importance of an additional step validating theresulting model by comparing it to neural responses fromother stimuli

Subspace (filter) estimation

In general one can search for any deviation between theraw and spike-triggered stimulus ensembles This can bedone for instance using measures of information theory(Paninski 2003 Sharpee et al 2003 2004) Anothernatural approach is to consider only changes in low-ordermoments between the raw and spike-triggered stimulusHere we focus on changes in the first and secondmoments which may be computed efficiently and manip-ulated using a set of standard linear algebraic techniquesWe also briefly discuss how the analysis relates to theWienerVolterra approach

Spike-triggered average

The simplest deviation between the spike-triggered andraw stimulus distributions is a change in the meanAssuming that the raw stimuli have zero mean this can be

estimated by computing the average of the spike-triggeredensemble (STA)

A frac14 1

N~N

nfrac141

sYtneth THORN eth2THORN

where tn is the time of the nth spike sYethtnTHORN is a vector

representing the stimuli presented during the temporalwindow preceding that time and N is the total number ofspikes In practice the times tn are binned If there is morethan one spike in a bin then the stimulus vector for thattime bin is multiplied by the number of spikes thatoccurred The STA is illustrated in Figure 3AFor an LNP model with a single linear filter the STA

provides an unbiased estimate of this filter2 provided thatthe input stimuli are spherically symmetric (Bussgang1952 Chichilnisky 2001 Paninski 2003) and the non-linearity of the model is such that it leads to a shift in themean of the spike-triggered ensemble relative to the rawensemble (see Limitations and potential failures sectionand Experimental issues section) This last requirementrules out for example a model with a symmetricnonlinearity such as full-wave rectification or squaringFor an LNPmodel with multiple filters the STA provides

an estimate of a particular linear combination of the modelfilters subject to the same restrictions on input stimuli andthe form of the nonlinearity given above (Paninski 2003Schwartz et al 2002) That is the STA lies in thesubspace spanned by the filters but one cannot assumethat it will exactly represent any particular filter in themodel

Figure 3 Two alternative illustrations of STA (A) The STA is constructed by averaging the spike-triggered stimulus segments (red boxes)and subtracting off the average over the full set of stimulus segments (B) Geometric (vector space) depiction of spike-triggered averagingin two dimensions Black points indicate raw stimuli White points indicate stimuli eliciting a spike The STA indicated by the line in thediagram corresponds to the difference between the mean (center of mass) of the spike-triggered ensemble and the mean of the rawstimulus ensemble


Spike-triggered covariance

The STA only recovers a single filter Additional filtersmay be recovered seeking directions in the stimulus spacein which the variance of the spike-triggered ensemblediffers from that of the raw ensemble Assuming that theraw stimuli have spherical covariance this is achieved bycomputing the STC matrix

C frac14 1

Nj1~N

nfrac141

sYtneth THORNj A

sYtneth THORNj A

T eth3THORN

where the [I]T indicates the transpose of the vector Againthe tn are binned in practice and this means that each termshould be multiplied by the number of spikes occurring inthe associated time binThe STCmatrix embodies the multidimensional variance

structure of the spike-triggered ensemble Specifically thevariance of the ensemble in any direction specified by a unitvector u is simply uTCu The surface swept out by allsuch unit vectors scaled by the square root of theirassociated variance is a multidimensional ellipsoid Theprinciple axes of this ellipsoid along with the associatedvariances may be recovered as the eigenvectors andassociated eigenvalues of the STC matrix This is

illustrated in Figure 4 The consistency of the STCestimate is guaranteed provided that the input stimuli areGaussian (Paninski 2003) and the nonlinearity of themodel is such that it leads to a change in the variance ofthe spike-triggered ensemble relative to the raw ensem-ble Note that the Gaussianity is a more severe require-ment than the spherical symmetry required for STAanalysis (see Limitations and potential failures sectionand Experimental issues section)The STA and STC filters together form a low-

dimensional linear subspace in which neural responsesare generated A number of groups have presenteddifferent approaches for combining the STA and STCanalyses in practice these variants all converge to thesame estimated subspace3 Usually the STA is sub-tracted prior to computing the STC filters (BrennerBialek amp de Ruyter van Steveninck 2000 de Ruytervan Steveninck amp Bialek 1988) It is often (but notalways) the case that the STA will lie within thesubspace spanned by the significant STC axes Depend-ing on the nonlinear properties of the response it couldcoincide with either high- or low-variance STC axesTo simplify visualization and interpretation of the axeswe have chosen for all of our examples to perform theSTC analysis in a subspace orthogonal to the STA

Figure 4 Two alternative illustrations of STC (A) The STC is determined by constructing the covariance of the spike-triggered stimuli(relative to the raw stimuli) followed by an eigenvector analysis of the covariance matrix This can result in multiple filters that representdirections in stimulus space for which the spike-triggered stimuli have lower or higher variance than the raw stimuli (B) Geometricdepiction of STC Black points indicate raw stimuli White points indicate stimuli eliciting a spike Ellipses represent the covariance of eachensemble Specifically the distance from the origin to the ellipse along any particular direction is the standard deviation of the ensemble inthat direction Raw stimuli are distributed in a circular (Gaussian) fashion Spike-triggered stimuli are elliptically distributed with a reducedvariance (relative to the raw stimuli) along the minor axis The minor axis of the ellipse corresponds to a suppressive direction Stimuli thathave a large component along this direction (either positive or negative) are less likely to elicit a spike The variance of the major axis ofthe ellipse matches that of the raw stimuli and thus corresponds to a direction in stimulus space that does not affect the neuronrsquos firingrate


Specifically we compute STC on a set of stimuli fromwhich the STA has been projected

sY frac14 s

Yj frac12sYT AA=kAk2 eth4THORN

Comparison to WienerVolterra analysis

The STA provides an estimate of the first (linear) termin a polynomial series expansion of the system responsefunction and thus is the first term of the WienerVolterraseries Whereas the WienerVolterra approach assumesthat the nonlinearity is literally a polynomial in the STAsubspace approach the nonlinearity is essentially unre-stricted For nonlinearities such as a sigmoid the WienerVolterra expansion would require many terms to capturethe neural response function An example of STA analysisfor characterizing a model with a single filter andsigmoidal nonlinearity is presented in the model simu-lations belowThe second-order term in the Wiener series expansion

describes the response as a weighted sum over all pairwise

products of components in the stimulus vector Theweights of this sum (the second-order Wiener kernel)may be estimated from the STCmatrix However the STCmethod is not just a specific implementation of a second-order WienerVolterra model The STC approach usesthe STC matrix as a means to obtain a linear subspacewithin which the nonlinearity is much less restricted Incontrast the second-order WienerVolterra approachassumes a quadratic nonlinearity This is suitable forcharacterizing nonlinearities such as the Benergy model[(Adelson amp Bergen 1985) of complex cells in primaryvisual cortex (eg Emerson Bergen amp Adelson 1992Emerson Citron Vaughn amp Klein 1987 Szulborski ampPalmer 1990) however it cannot describe responsefunctions with nonlinearities such as divisive gaincontrol (Albrecht amp Geisler 1991 Heeger 1992)because these cannot be formulated as sums (or differ-ences) of squared terms An STASTC approach is moreflexible in capturing such nonlinearities (Rust Schwartzet al 2005 Schwartz et al 2002) as we demonstrate inthe next section

Simulations of example model neurons

We simulate an example ideal simple cell model forwhich there is only a single filter followed by half-waverectification and then squaring Specifically the instanta-neous firing rate is determined by

gethsYTHORN frac14 r )kY s

Y

22

eth5THORN

The spike-triggered analysis results are shown inFigure 5 The spike-triggered ensemble exhibits a changein the mean relative to the raw stimulus ensemble due tothe asymmetric nonlinearity We recover the STA filter bycomputing the change in the mean (Equation 2) Next weconsider changes in the variance between the raw andspike-triggered stimulus ensemble For this model neuronthere is no further relationship between the stimulus spaceand spikes In the limit of infinite data the spike-triggeredensemble would be a randomly selected subset of the rawstimulus ensemble and the variance in any direction wouldbe identical to that of the raw stimulus set In anexperimental setting the finiteness of the spike-triggeredensemble produces random fluctuation of the variance indifferent directions As a result there are small randomincreases or decreases in variance of the spike-triggeredensemble relative to the raw stimulus set This is reflectedin the eigenvalue analysis of Figure 5 Due to the randomfluctuations the sorted eigenvalues cover a range around aconstant value of 1 (ie the variance of the raw stimulusensemble) but are not exactly equal to this constant value

Figure 5 Eigenvalues and eigenvectors for an LNP model with asingle linear filter followed by a point nonlinearity The simulationis based on a sequence of 50000 stimuli with a responsecontaining 1891 spikes Top Model filter and nonlinearity As inFigure 1 filters are 6 8 and thus live in a 48-dimensionalspace The nonlinearity cartoon represents half squaring Positivefilter responses are squared and negative filter responses are setto zero Bottom STA filter and sorted eigenvalues of covariancematrix of stimuli eliciting spikes (STC) We plot the first 47eigenvalues and omit the last eigenvalue which is zero due toprojecting out the STA (see Equation 4) The eigenvalues aregradually descending and corresponding eigenvectors appearunstructured


Now consider an example model neuron for which thereis more than a single filter We simulate an ideal V1complex cell model (see also simulations in Sakai ampTanaka 2000) The model is constructed from two spacendashtime-oriented linear receptive fields one symmetric andthe other antisymmetric (Adelson amp Bergen 1985) Thelinear responses of these two filters are squared and

summed and the resulting signal then determines theinstantaneous firing rate

gethsYTHORN frac14 r ethkY1 sYTHORN2 thorn ethkY2 s

YTHORN2h i

eth6THORN

Spike-triggered analysis on the model neuron is shown inFigure 6 The STA is close to zero This occurs becausefor every stimulus there is a stimulus of opposite polarity(corresponding to a vector on opposite sides of the origin)that is equally likely to elicit a spike and thus the averagestimulus eliciting a spike will be zero The recoveredeigenvalues indicate that two directions within this spacehave substantially higher variance than the others Theeigenvectors associated with these two eigenvalues corre-spond to the two filters in the model (formally they spanthe same subspace) In contrast eigenvectors correspond-ing to eigenvalues in the gradually descending regionappear arbitrary in their structureFinally we consider a version of a divisive gain control

model (eg Geisler 1992 Heeger 1992)

g sYeth THORN frac14 r

1thorn )kY

1 sY

22

1thorn ethkY2 sYTHORN2 thorn 4ethkY3 s

YTHORN2 eth7THORN

The analysis results are shown in Figure 7 First werecover the STA filter which is nonzero due to thehalf squaring in the numerator A nonsymmetrical non-linearity of this sort is captured by changes in the meanNext we examine the sorted eigenvalues obtained fromthe STC analysis Most of the eigenvalues descendgradually but the last two eigenvalues lie significantlybelow the rest and their associated eigenvalues spanapproximately the same subspace as the actual simulationfilters

Significance testing

How do we know if the recovered STA and STC filtersare significant In some cases such as a prototypicalcomplex cell in primary visual cortex there is essentiallyno difference between the mean of the raw and spike-triggered stimuli (Rust Schwartz et al 2005 Touryanet al 2002) which leads to a weak STA To quantify thiswe test the hypothesis that the difference between themean of the raw and spike-triggered stimulus is nodifferent than what one would expect by chance Wespecifically test whether the magnitude of the true spike-triggered stimulus STA is smaller or equal to what wouldbe expected by chance More specifically we generate adistribution of random STA filters by bootstrapping Werandomly time-shift the spike train relative to the rawstimulus sequence gather the resulting spike-triggeredstimulus ensemble and perform the STA analysis The

Figure 6 Eigenvalues and eigenvectors for an LNP ideal complexcell model In this model the Poisson spike generator is driven bythe sum of squares of two oriented linear filter responses As inFigure 1 filters are 6 8 and thus live in a 48-dimensionalspace The simulation is based on a sequence of 50000 rawstimuli with a response containing 4298 spikes Top Modelincluding two input filters nonlinearities and Poisson spikingBottom STA filter is unstructured for the ideal complex cell Theplot also shows the eigenvalues sorted in descending order Weplot the first 47 eigenvalues and omit the last eigenvalue which iszero due to projecting out the STA (see Equation 4) Two of theeigenvalues are substantially larger than the others and indicatethe presence of two directions in the stimulus space along whichthe model responds The others correspond to stimulus directionsthat the model ignores Also shown are three example eigenvec-tors (6 8 linear filters) two of which are structured while one isunstructured


randomly time-shifted spike train retains all temporalstructure that is present in the original spike train Werepeat this 1000 times each time computing the averageof the stimulus subset We can then set a significance

criterion (eg the 95 confidence interval) within whichwe deem the magnitude of the true STA to beinsignificantThe issue of significance is also of importance for the

STC filters Although the low-variance eigenvalues areclearly below the gradually descending region in theillustrated example of Figure 7 the distinction is not soobvious in some experimental situations An example inwhich the significance cutoff is not clear-cut is shown inFigure 8 A significance test should allow us to determinethe number of eigenvector axes (filters) corresponding tosignificant increases or decreases in the variance That iswe would like to find changes in variances in the spike-triggered ensemble that are not just due to chance(because of the finiteness of the number of spikes) butthat relate to actual neural response characteristicsThe significance testing must be done in a nested fashion

because the distribution of the lowest and highest eigen-values under the null hypothesis depends on the dimension-ality of the space We begin by assuming that none of theeigenvalues are significant We compare the true eigen-values to the eigenvalues of randomly selected stimuli withthe same interspike interval If the largest and smallest trueeigenvalues lie within the range of largest and smallesteigenvalues of the randomly selected stimuli we canconclude that none of our axes are significant and acceptthe hypothesis More specifically to compute the randomlyselected eigenvalues we generate distributions of minimalmaximal eigenvalues by bootstrappingWe randomly time-shift the spike train relative to the raw stimulus sequencegather the resulting spike-triggered stimulus ensembleperform the STA and STC analysis on the spike-triggeredensemble and extract the minimum and maximum eigen-values After repeating 1000 times we estimate the 95confidence interval for both the largest and smallesteigenvalues We then ask whether the maximal andminimal eigenvalues obtained from the true spike-triggeredensemble lie within this interval If so we accept thehypothesis

Figure 8 Nested hypothesis testing Gray solid line corresponds to 95 confidence interval assuming no suppressive axes (A) twosuppressive axes (B) and four suppressive axes (C) If the eigenvalues lie within the confidence interval the hypothesis is accepted Forthe assumption of no or two suppressive axes some eigenvalues lie below the confidence interval indicating that the hypothesis isincorrect In contrast for the assumption of four suppressive axes eigenvalues lie roughly within the confidence interval

Figure 7 Eigenvalues and eigenvectors for an LNP divisivenormalization model The simulation is based on a sequence of250000 stimuli with a response containing 30444 spikes TopModel Bottom STA filter sorted eigenvalues of covariancematrix of stimuli eliciting spikes (STC) and eigenvectors Two ofthe eigenvalues are substantially lower than the others andindicate the presence of two suppressive directions in thestimulus space


Figure 8A shows that the hypothesis of no significanteigenvalues is unlikely to be correct for this example Thesmallest eigenvalue lies far beyond the confidenceinterval We therefore assume that the largest outlier(here the smallest eigenvalue) has a corresponding axisthat significantly affects the variance of the neuralresponse We proceed to test the hypothesis that allremaining axes are insignificant To do so we first projectout the axis deemed significant and repeat the boot-strapping in the remaining subspace Note that the

distribution of eigenvalues (gray region in Figures 8A8B and 8C) changes as the dimensionality of theremaining space decreases We continue this process in anested fashion until the largest and smallest eigenvaluesfrom the true spike-triggered ensemble lie within theirestimated confidence intervals Figure 8B shows that wecannot accept the hypothesis of two significant axesFinally the hypothesis of four significant axes (Figure 8C)is accepted and results in eigenvalues that lie within theconfidence interval

Figure 9 Accuracy in filter estimation Simulations are shown for the divisive normalization example of Figure 7 Bottom The erroris computed as a function of the ratio of number of spikes to stimulus dimensionality Stimulus dimensionality is held fixed for allsimulations but a number of input stimuli (and thus spikes) are varied Black line and points are the bootstrap-estimated error (meanangular error obtained from bootstrapping see main text) of estimation of the lowest eigenvector The gray line is the theoreticalprediction of the mean angular error computed as the square root of the ratio of the stimulus dimensionality (here 48) to number ofspikes (see Equation 8 and Paninski 2003) We multiply the theoretical prediction by a constant parameter that yields the least squareerror with the bootstrap-estimated error above for the last five points (because the theoretical prediction only holds for the small errorregime)


Filter estimation accuracy

Assuming that the recovered STA and STC filters aresignificant we would also like to understand how accuratethey are The accuracy of our estimated filters depends onthree quantities (1) the dimensionality of the stimulusspace d (2) the number of spikes collected N and (3) thestrength of the response signal relative to the standarddeviation of the raw stimulus ensemble AAsymptotically the errors decrease as

MAE kY

frac14 A

BethkYTHORN

ffiffiffiffid

N

r eth8THORN

where MAE indicates the mean of the angular error (thearccosine of the normalized dot product) between theestimated filter and the true filter and BethkYTHORN is aproportionality factor that depends inversely on thestrength of the response signal (Paninski 2003) Thestrength of response signal is the length of the STA vectorin the limit of infinite data and is not available in anexperimental situation However the number of spikesand number of stimulus dimensions are known and thusthe function of Equation 8 may be used to extrapolate theerror behavior based on bootstrap estimates To demon-strate this we simulate an experiment on the modeldivisive normalization neuronWe describe a bootstrapping methodology for estimating

the MAE and show that it is reasonably matched to thetheoretical prediction of the error in Equation 8 when theratio of number of spikes to number of stimulusdimensions is sufficiently high We run a pilot experimenton the model divisive normalization neuron and collect409600 input samples We consider how the ratio ofstimulus dimensionality to number of spikes affectsaccuracy Specifically we hold the stimulus dimension-ality fixed and vary the number of input samples (and thusspikes) For a given number of input samples we boot-strap drawing (with replacement) random subsets ofstimuli equal to the number of input samples We consider

the spike-triggered stimuli from this subset and computethe STA and STC We repeat this many times (here1000) and derive an estimate of the mean angular errorfor a given STC filter This is achieved by computing themean of the 1000 estimated filters from the boot-strappingVwe will denote this the mean estimated filterand then for each of the 1000 estimated filters bycomputing its mean angular error with the mean estimatedfilter and taking an average over these computations Thisanalysis assumes that there is no systematic bias in theestimates (such as those shown in Figure 15)In Figure 9 we plot the error estimates for the filter

corresponding to the lowest eigenvalue As the number ofspikes to number of stimulus dimensions increases theerror is reduced We also show for three example ratiosthe eigenvalues and the filter estimate corresponding to thelowest eigenvalue For a low ratio of spike counts tostimulus dimensions the eigenvalues descend graduallyand the smallest one is not separated from the rest for ahigh ratio of spike counts to stimulus dimensions theeigenvalues take on a pattern similar to Figure 7 Finallywe return to Equation 8 We fit this equation (andcorresponding proportionality factor) to the errors derivedfrom bootstrapping and obtain a rather good match for thelow error regime Such an analysis could be used in anexperimental situation to determine data requirements fora given error level by extrapolating the curve from valuesestimated from a pilot experiment In the Experimentalissues section we elaborate on running a pilot experimentto choose a reasonable tradeoff between number of spikesand stimulus dimensionality

Characterizing the nonlinearity

According to the LNPmodel the firing rate of a neuron isgiven by a nonlinear transformation of the linear filterresponses (Figure 2) Using the same set of stimuli andspike data as for the linear filter estimation we seek toestimate the nonlinearity and thus characterize a neuralmodel that specifies the full transformation from stimulusto neural firing rate We therefore need to estimate the

Figure 10 Nonlinearity for an LNP model with a single linear filter followed by a point nonlinearity Left Raw (black) and spike-triggered(white) histograms of the linear (STA) responses Right The quotient of the spike-triggered and raw histograms gives an estimate of thenonlinearity that generates the firing rate


firing rate of the neuron as a function of the linear filterresponses This can be seen using Bayes rule

P spikeksYeth THORN frac14 PethspikeTHORNPethsYkspikeTHORN

PethsYTHORN eth9THORN

and therefore

P spikeksYeth THORNograve PethsYkspikeTHORN


where PethspikeksYTHORN is the instantaneous firing ratePethsYkspikeTHORN is the frequency of occurrence of spike-triggered stimuli and PethsYTHORN is the frequency of occurrenceof raw stimuliThe problem of estimating the nonlinearity can thus be

described as one of estimating the ratio of two probabilitydensities of Equation 10 The accuracy of the estimation isdependent on the dimensionality (number of filters) in thelinear subspace For one or two filters we can use simplehistograms to estimate the numerator and denominator ofEquation 10 For more filters this becomes impractical

due to the so-called Bcurse of dimensionality[ Theamount of data needed to sufficiently fill the histogrambins in a d-dimensional space grows exponentially with dIn this case we typically need to incorporate additionalassumptions about the form of the nonlinearityConsider a model LNP neuron with only a single filter

followed by a point nonlinearity First we estimate thelinear filter by computing the STA Then we compute thelinear filter response for each stimulus by taking a dotproduct of the filter with the stimulus We do this for allinstantiations of the spike-triggered stimuli and compute ahistogram estimating the numerator density PethsYkspikeTHORN wedo this for all instantiations of the raw stimuli andcompute a histogram estimating the denominator densityPethsYTHORN The nonlinearity that determines the firing rate isthen the ratio of these two densities or the ratio of thehistogram values in each bin An example is shown inFigure 10 (see also Chichilnisky 2001) We plot thehistograms of the spike-triggered and raw stimuli filterresponses (Figure 10 left) We observe the nonlinearity byexamining the ratio of these two histograms (Figure 10right) The instantaneous firing rate grows monotonicallyand asymmetrically that is increases for stimuli to whichthe filter responds strongly and positively

Figure 11 Nonlinearity for ideal complex cell model This corresponds to eigenvalue and eigenvector example of Figure 6 Left Scatterplots of stimuli projected onto estimated filters (ie filter responses) corresponding to first two eigenvalues (e1 and e2) Black pointsindicate the raw stimulus set White points indicate stimuli eliciting a spike Also shown are one-dimensional projections onto a single filterRight The quotient of the two-dimensional spike-triggered and raw histograms provides an estimate of the two-dimensional nonlinearfiring rate function This is shown as a circular-cropped grayscale image where intensity is proportional to firing rate Superimposedcontours indicate four different response levels Also shown are one-dimensional nonlinearities estimated for when the data are projectedonto each of the single filters


Note that the nonlinearity can be arbitrarily complicated(even discontinuous) The only constraint is that it mustproduce a change in the mean of the spike-triggeredensemble as compared with the original stimulus ensembleThus the interpretation of reverse correlation in the contextof the LNPmodel is a significant departure from theWienerVolterra series expansion in which even a simple sigmoidalnonlinearity would require the estimation of many terms foraccurate characterization (Rieke et al 1997)Next consider an ideal complex cell model neuron as in

Equation 6 The recovered eigenvalues indicate that twodirections within this space have substantially highervariance than the others (recall Figure 6) As before wecompute the raw and spike-triggered stimulus responsesfor each of the two filters A two-dimensional scatter plotof these filter responses is shown in Figure 11 (left) forboth the spike-triggered and raw stimuli This is a two-dimensional depiction of samples from the numerator anddenominator distributions in Equation 10 The scatter plotsare similar in essence to those described in Figure 4 butthe stimuli are projected onto the two filters recoveredfrom the analysis To estimate the two-dimensional non-linear firing rate function (Figure 11 right) we computethe two-dimensional histogram for the spike-triggered and

raw stimuli responses and calculate the ratio of thehistogram values in each bin This is analogous to theone-dimensional example shown in Figure 10 Similarpairs of excitatory axes and nonlinearities have beenobtained from STC analysis of V1 cells in cat (Touryanet al 2002) and monkey (Rust et al 2004 RustSchwartz et al 2005)Finally consider the divisive normalization model in

Equation 7 for which the eigenvalues and eigenvectors areshown in Figure 7 Figure 12 (left) shows a scatter plot ofthe STA filter response versus a suppressive filter responseThe two-dimensional nonlinearity is estimated by takingthe quotient as before This reveals a saddle-shapedfunction indicating the interaction between the excitatoryand suppressive signals (Figure 12 right) Similar suppres-sive filters have been obtained from STC analysis of retinalganglion cells (in both salamander and monkey Schwartzet al 2002) and simple and complex cells in monkey V1(Rust Schwartz et al 2005)For some systems such as H1 of the blowfly (Bialek amp de

Ruyter van Steveninck 2005 Brenner Bialek amp de Ruytervan Steveninck 2000) the dimensionality of STA andSTC filters is sufficiently low (and the data set sufficientlylarge) to calculate the quotient of Equation 10 directly (as

Figure 12 Nonlinearity for divisive normalization model This corresponds to the eigenvalue and eigenvector example of Figure 7Left Scatter plots of stimuli projected onto estimated filters (ie filter responses) corresponding to STA and last suppressiveeigenvector Black points indicate the raw stimulus set White points indicate stimuli eliciting a spike Also shown are one-dimensional projections onto a single filter Right The quotient of the two-dimensional spike-triggered and raw histograms providesan estimate of the two-dimensional nonlinear firing rate function This is shown as a circular-cropped grayscale image whereintensity is proportional to firing rate Superimposed contours indicate four different response levels Also shown are one-dimensional nonlinearities onto a single filter


we have shown in the simulation examples) and thusestimate the nonlinearity But what happens when thereare more than two significant filters derived from the STAand STC analyses There is not one single recipe ratherthere are a number of ways to try and approach this

problem and the answer depends on the particular systemand data at handOne approach is to consider specific classes of LNP

models that might be suitable for the particular neural areaunder study For instance in retinal ganglion cell data it

Figure 13 Interpretation issues and sum of half squares LNP model filters (A) Left Model filter responses are half squared (negativevalues set to zero) and then added together Note that this is different from the full squaring of the ideal complex cell Right Geometry ofthe STA and STC analysis The STA is a vector average of the model filters The STC is forced to be 90 deg away from the STA Althoughthe STA and STC filters do not equal the model filters they do span the same subspace (B) Example of spatially shifted model filtersBoth STA and STC analysis reveal filters that are quite different from the model but span the same subspace (C) Example of orientedfilters We extend the two-filter model to four filters that are each half squared and then added together The STA is the average of all fourfilters and has a centersurround appearance rather than an oriented one The other three STC filters are orthogonal (D) The modelneuron includes five spatially overlapping filters The filter responses undergo a weighted sum of half squares followed by addition of a(negative) linear surround (gray curve) The STA is a vector average of the linear filters and the STC filters are orthogonal


was shown that fitting a divisive normalization model to thefilters recovered from STA and STC provided a reasonablecharacterization of the data (Schwartz et al 2002) Inanother study in area V1 the dimensionality of the filtersfrom STA and STC was too high for computing thenonlinearity within the full recovered subspace (RustSchwartz et al 2005) The form of nonlinearity wasrestricted by first computing squared sums of excitatoryfilter responses and squared sums of suppressive filterresponses and only then was the nonlinearity betweenthese pooled excitatory and suppressive signals deter-mined This simplification could be made because it wasobserved that projections of stimuli onto the recoveredfilters within the excitatory or suppressive pools alwaysresulted in elliptical contoursVsuggesting sum of squaresoperations governing the combination within each poolAn alternative approach published in this special issueassumes that the nonlinearity takes the form of a ratio ofGaussians (Pillow amp Simoncelli 2006)

Limitations and potential failures

The STA and STC estimates depend critically on thedistribution of input stimuli and on the particular non-linearity of the neuron For an LNP model with a singlelinear filter the consistency of the STA estimator isguaranteed (eg irrespective of the neural nonlinearity)only if the distribution of input stimuli are sphericallysymmetric that is any two stimulus vectors with equalvector length have an equal probability of being presented(Chichilnisky 2001) If one aims to recover a set of filtersusing both STA and STC then the consistency of theestimator is guaranteed under the more stringent conditionthat the stimuli be Gaussian distributed (Paninski 2003)The estimator is also guaranteed for elliptically symmet-ric Gaussian stimuli in which the covariance matrix isnot equal to the identity (see Appendix) For exampleeven if the raw stimuli are constructed as sphericalGaussian a finite number of stimuli will by chanceproduce some axes that have (slightly) higher variancethan othersNote that non-Gaussian stimulus distributions can lead to

artifacts in the spike-triggered analysis and the artifacts aredependent on how the nonlinear response properties of theneuron interact with the distribution In the Experimentalissues section we show simulated examples with non-Gaussian stimuli demonstrating how this could poten-tially impact the STA and STC in a model neuron Theseexamples do not indicate that experiments with non-Gaussian stimuli and STASTC analysis will necessarilylead to artifacts but because there is no general solutionfor eliminating artifacts that can arise from non-Gaussianstimuli it is advisable to run experimental controls withGaussian stimuli

Even if one is careful to design an experiment and dataanalysis methodology that leads to accurate and artifact-free estimates a spike-triggered analysis can still fail if themodel assumptions are wrong Two examples of failure ofthe LNP model are as follows (1) there is no low-dimensional subspace in which the neural response maybe described or (2) the neural response has a strongdependence on spike history (eg refractoriness burstingadaptation) that cannot be described by an inhomogeneousPoisson process STASTC analysis of data simulated usingmore realistic spike generation models such as HodgkinndashHuxley (Aguera y Arcas amp Fairhall 2003 Aguera y ArcasFairhall amp Bialek 2001 2003 Pillow amp Simoncelli2006) and integrate-and-fire (Pillow amp Simoncelli 2003)produces biased estimates and artifactual filters Althoughthe STASTC filters might in some cases still provide areasonable description of a neuronrsquos response it isimportant to recognize that the LNP model provides onlya crude approximation of the neural response (seeInterpretation issues section)

Interpretation issues

There are a number of important issues that arise ininterpreting the spike-triggered analysis First the numberof filters recovered by STA and STC provides only a lowerbound on the actual number of filters The neural responsemay be dependent on mechanisms not identified by theSTC analysis (1) Other filters might affect the responsebut the dependence is too weak and buried in the statisticalerror (a possibility with any experimental methodVrecallFigure 9) or (2) The neural response nonlinearities maynot lead to a change in the mean or variance It should benoted that although such a nonlinearity is theoreticallypossible most known physiological nonlinearities doaffect the mean the variance or bothNext the recovered filters cannot be taken literally as

physiologically instantiated mechanisms The STC filterstogether with the STA form an orthogonal basis for thestimulus subspace in which the responses are generatedThe analysis does not yield a unique solution A wholefamily of equivalent models can be constructed (bytransforming to alternative sets of filters using aninvertible linear transformation) which given the samestimulus produce the same response Thus even if aneuronrsquos response is well described by an LNP model wecannot claim to recover the actual filters that the neuron isusing to compute its response Rather the goal is to find aset of filters that span the proper subspace that is withthis set of filters one can compute the same responses aswith the actual setFigure 13 shows a simulation for an example of model

neuron in which the STA and STC do not recover theactual model filters but do span the same subspace The


model neuron responds with a rate proportional to a sumof half squares as opposed to the sum of squares typicalof the ideal complex cell

gethsYTHORN frac14 r )kY

1 sY

22 thorn )k

Y

2 sY

22

The simulation results for different input filters are shownin Figure 13 Now the STA does not result in a zero-weighted filter because the filter responses are notsymmetric as in the ideal complex cell The STA is notequal to either of the two excitatory filters of the modelrather it is a vector average of the two filters STCanalysis on the stimuli perpendicular to the STA revealsan additional excitatory filter Note that the two recoveredfilters together span the excitatory subspace of the originalmodel filters Figure 13C shows an example with fourinput filters of different orientations whose responses arehalf squared and summed the STA takes on a more centerndashsurround unoriented appearance Figure 13D shows anexample of five overlapping spatial filters These can bethought of as subunits as has been proposed for retina(Hochstein amp Shapley 1976 see also Rust Schwartz etal 2005 for cortical data) The nonlinear combination ofthese filters is followed by a subtraction of a linearsurround The resulting STA takes on the well-knownspatial profile of retinal ganglion cells and the STC filtersare forced to be orthogonal and similar to what is foundexperimentally (Pillow Simoncelli amp Chichilnisky2004) The two-dimensional depiction of the nonlinearityfor the above examples is interesting The spike-triggered

stimuli form a shape that resembles a portion of anannulus (Figure 14) Neurons with nonlinearities of thisflavor can be seen in area V1 of the macaque (RustSchwartz et al 2005) and in retinal ganglion cells(Schwartz amp Simoncelli 2001)Another reason why the recovered filters should not be

interpreted as a physiological mechanism is that the LNPmodel assumes Poisson spiking A number of authors havedemonstrated that these Poisson assumptions do notaccurately capture the statistics of neural spike trains(Berry amp Meister 1998 Keat Reinagel Reid ampMeister 2001 Pillow Shlens Paninski Chichilniskyamp Simoncelli 2005a Reich Victor amp Knight 1998)The dependence of neural responses on spike history(eg refractoriness bursting adaptation) may be cap-tured only indirectly in the LNP model through time-delayed suppressive STC filters (Aguera y Arcas ampFairhall 2003 Aguera y Arcas et al 2003 Schwartzet al 2002) For instance during a refractory period aneuron will not spike and this can be captured by anLNP model with a set of suppressive STC filters in timeThe suppressive filters may still provide a reasonablyaccurate description of the neural response but do notreveal the mechanism of refractorinessFinally the labeling of whether a filter is excitatory or

suppressive is crudely based on the net change in the meanor variance and may not correspond physiologically toexcitation or suppression A given filter can indeed be bothexcitatory and suppressive For example a filter might behalf square rectified yielding a positive increase in themean but also include a compressive squared nonlinearity(as in divisive normalization) Because the STA and STC

Figure 14 Interpretation issues and sum of half squares LNP model nonlinearity Nonlinearity is shown for model simulation of filters inFigure 13B (almost identical plots are found for Figures 13C and 13D) Left Scatter plots of stimuli projected onto estimated filters (iefilter responses) corresponding to STA and first eigenvector Black points indicate the raw stimulus set White points indicate stimulieliciting a spike Also shown are one-dimensional projections onto a single filter Right The quotient of the two-dimensional spike-triggered and raw histograms provides an estimate of the two-dimensional nonlinear firing rate function This is shown as a circular-cropped grayscale image where intensity is proportional to firing rate Superimposed contours indicate four different response levels Alsoshown are one-dimensional nonlinearities onto a single filter


filters are orthogonal the analysis will extract a singlefilter and label it as excitatory As before the analysis stillfinds the right subspace one can then analyze theinteraction and aim to estimate a model within thesubspace

Experimental issues

We now discuss issues that arise when designing andinterpreting spike-triggered experiments

Stimulus choiceStimulus space

The stimuli in a spike-triggered experiment need to berestricted to lie in a finite-dimensional space and theexperimentalist must choose the fundamental components(ie the axes) of this space At any moment in time theneuron is exposed to a linear combination of this set ofstimulus components In many published examples (as wellas the examples shown in this article) the axes of thestimulus space corresponds to pixel (or stixel) intensitiesHowever the stimulus may be described in terms of othercomponents such as the amplitudes of a particular setof sinusoids (Ringach Sapiro amp Shapley 1997) the

velocities of a randomly moving spatial pattern (BairCavanaugh amp Movshon 1997 Borghuis et al 2003Brenner Bialek amp de Ruyter van Steveninck 2000 deRuyter van Steveninck amp Bialek 1988) or any other fixedset of functions More generally it is possible to do theanalysis in a space that is a nonlinear function of the inputstixels (David Vinje amp Gallant 2004 Nishimoto Ishidaamp Ohzawa 2006 Theunissen et al 2001) This is usefulwhen one believes that the cellsrsquo response is LNP on theseinputs (Rust Simoncelli amp Movshon 2005) although itmay then be more difficult to interpret the results Thefundamental constraints on the choice of these compo-nents are that (1) the neuron should respond reasonably tostochastically presented combinations of these compo-nents and (2) the neuronrsquos response should be wellapproximated by an LNP model operating in the spaceof these componentsThe choice of a finite-dimensional stimulus space places

a restriction on the generality of the experimental resultsThe response of the cell will only be characterized withinthe subspace spanned by the stimulus components(Ringach et al 1997) Stated differently without furtherassumptions the model one constructs with STC can onlypredict stimuli responses that lie in the space defined bythe experimental stimulus ensemble For example onecannot predict the responses to chromatic stimuli whenusing achromatic stimuli or to a full two-dimensionalspace when probing the neuron with only a single spatialdimension (as in the case of bars) Similarly one cannot

Figure 15 Simulations of an LNP model demonstrating bias in the STA for two different nonspherical stimulus distributions The linearstage of the model neuron corresponds to an oblique axis (line in both panels) and the firing rate function is a sigmoidal nonlinearity (firingrate corresponds to intensity of the underlying grayscale image in the left panel) In both panels the black and white Btarget[ indicates therecovered STA Left Simulated response to sparse noise The plot shows a 2-dimensional subspace of a 10-dimensional stimulus spaceEach stimulus vector contains a single element with a value of T1 whereas all other elements are zero Numbers indicate the firing rate foreach of the possible stimulus vectors The STA is strongly biased toward the horizontal axis Right Simulated response of the samemodel to uniformly distributed noise The STA is now biased toward the corner Note that in both examples the estimate will not convergeto the correct answer regardless of the amount of data collected


use the model to predict responses to stimuli that have afiner spatial or temporal resolution than that used in thecharacterizationTo obtain a more general characterization one needs to

increase the stimulus resolution Unfortunately thisincreases the dimensionality of the stimulus space andthus requires more spikes to achieve the same quality offilter estimation At the same time the increase inresolution typically reduces the responsivity of the cell(eg because the effective contrast is reduced) thusmaking it more difficult to obtain the needed spikesRecall that the error in filter estimation is a direct

consequence of the ratio of the number of spikes tostimulus dimensionality as in the example model neuronsimulation shown in Figure 9 Therefore it is useful to runa pilot experiment to determine the proper balance betweennumber of spikes (eg duration of the experiment) tostimulus dimensionality for a particular class of neurons Inpractice it useful for a physiologist to adopt a rule of thumbfor the particular system at hand In the V1 experimentsRust Schwartz et al (2005) found that at least 100 spikesper dimension were typically needed to obtain a goodcharacterization Other experimental methodologies orsettings (eg recordings from an awake behaving animal)

Figure 16 STC artifacts with binary stimuli The model neuron is the same as in Figure 7 but the Gaussian stimuli were replaced withbinary stimuli (A) Left There are four eigenvalues significantly below what one would expect by chance Two of the correspondingeigenvectors correspond to the real model filter subspace but two of them are artifactual Right Projection onto one of the artifactualfilters versus the STA The raw stimuli are nonspherical and have lower horizonal variance at the top than in the middle Although thevariance of the raw and spike-triggered stimuli is the same when confined to this corner the variance of the spike-triggered stimuli issignificantly smaller than the variance of the entire raw ensemble and this generates the artifactual suppressive filter (e45) (B) Left Afterconditional whitening (see main text) there are only two significantly low eigenvalues corresponding to the model neuron subspaceRight Projection onto the same eigenvalue as the artifactual filter above as against the STA The raw stimuli are now not perfectly circularbut have roughly equal variance in all directions


and other classes of neurons may be more limited in thenumber of spikes that can be collected

Stochastic stimulus distribution

As stated earlier the STC portion of the spike-triggeredanalysis is only guaranteed to work for Gaussian stimuliThe use of non-Gaussian white noise stimulus distributions(eg uniform binary sparse) is quite common experimen-tally as the samples are easy to generate and the highercontrast of the stimuli generally leads to higher averagespike rates In practice their use is often justified byassuming that the linear filters are smooth relative to thepixel sizeduration (eg Chichilnisky 2001) Naturalsignal stimuli (such as visual scenes and auditory vocal-izations) are also non-Gaussian (Daugman 1989 Field1987) but their use is becoming increasingly popular(David amp Gallant 2005 David et al 2004 Felsen amp Dan2005 Ringach Hawken amp Shapley 2002 Sen et al2000 Smyth Willmore Baker Thompson amp Tolhurst2003 Theunissen et al 2001 for recent perspectives seeFelsen amp Dan 2005 Rust amp Movshon 2005) Naturalsignals can reveal response properties that occur lessfrequently under Gaussian white noise stimulation such asbursting in the LGN (Lesica amp Stanley 2004) and theyare often more effective in driving higher neural areasHowever nonspherical stimuli can produce artifacts in

the STA filters and non-Gaussian stimuli can produceartifacts in the STC filters Figure 15 shows two simu-lations of an LNP model with a single linear filter and asimple sigmoidal nonlinearity each demonstrating thatnonspherical stimulus distributions can lead to poorestimates of the linear stage The examples are meant toemphasize the potential for bias but do not necessarilymean that an artifact will occur in experiment Indeed theinteraction between the stimulus distribution and theparticular nonlinear behaviors of the neural response willdetermine if and how much of a bias occurs Because wedo not know the nonlinearity a priori the safest approachis to compare the experimental linear filter estimate to acontrol using spherically symmetric stimuliThe first example shows a Bsparse noise[ experiment in

which the stimulus at each time step lies along one of theaxes As shown in the figure the nonlinearity can result inan STA that is biased toward an axis of the space Thesecond example uses stimuli in which each component isdrawn from a uniform distribution which produces anestimate biased toward the Bcorner[ of the space Notehowever that the estimate will be unbiased in the case of apurely linear neuron or of a half-wave-rectified linearneuron (Ringach et al 1997)Non-Gaussian stimuli can produce strong artifacts in the

STC analysis Figure 16A (left) shows an examplesimulation of the divisive normalization model withbinary stimuli Note that in addition to the two Breal[suppressive filters of the model the analysis also finds twosignificant artifactual suppressive filters these have a few

high-intensity stixels Similar artifacts have been found inexperimental circumstances (Rust Schwartz et al 2005)More intuition for the artifacts can be gained by examin-ing two-dimensional scatter plots that include anartifactual filter response versus the STA filter response(Figure 16A right) The raw binary stimuli are clearly notspherical in this two-dimensional view Specifically theset tapers as one moves in the direction of the STA Thisreduction in variance of the raw stimulus happens tocoincide with the stimuli that elicit spikes (ie those thathave a large STA component) Thus the spike-triggeredanalysis reveals the artifactual filter as an axis ofsignificantly reduced variance although it is actually notreduced relative to the raw stimuliThere is unfortunately no generic recipe for reducing

artifacts From our experience with binary stimuli we havefound that the artifacts can be partially corrected byadjusting the raw stimulus such that the covarianceestimated at each value of the STA is the same as it wouldbe in the case of a Gaussian stimulus distribution (condi-tional whitening Rust Schwartz et al 2005) Specificallywe partition the stimuli of Figure 16A (right) intohorizontal slabs according to the value of the excitatoryfilter response and compute the covariance matrix for eachsubset (Cn for the nth subset) The stimuli in each subsetare whitened by multiplying them by

EeETe thorn E0EnD

j12

n ETnE0 eth11THORN

where Ee is a matrix containing the (orthogonal) excita-tory filters (only one in this exampleVthe STA) E0

contains an orthogonal basis for the remainder of thestimulus space and En and Dn are the eigenvectors andeigenvalues for the remainder of the conditional covari-ance matrix Cn respectively The first term serves topreserve the component of the stimulus in the direction ofthe STA while the second term depicts a whitening (bythe inverse of the raw stimuli in that slice) in the otherdimensionsAfter this conditional whitening the stimuli are recom-

bined and STC analysis is applied on the spike-triggered setto reestimate the filters Figure 16B shows that followingconditional whitening there are only two significantsuppressive eigenvalues corresponding to the real modelfilter subspaceWe have described an example of binary stimulus

artifacts and partially correcting for those artifacts Thereis generally no known fix for artifacts but there are severalthings that can be done to check for artifacts

1 It is helpful to examine the projection of the rawstimuli onto pairs of filters recovered from theanalysis if these are not spherical then the filterscan include artifacts However it is important toremember that the stimulus space is huge andprojection onto two dimensions might appear spheri-


cally symmetric but does not guarantee sphericalsymmetry in the full space

2 It is sometimes useful to run a model neuronsimulation with the given stimuli and see if artifactualfilters emerge The simplest simulation one can run isan LNP model with a single linear filter If asignificant STC filter is found this is indicative of anartifactual axis in simulation Here we have demon-strated a slightly more involved example of a divisivenormalization simulation However it is important torealize that we have control only over the stimulusensemble we have no control over the nonlinearbehaviors of the neural response and the artifactsdepend on these nonlinearities We can explore insimulation nonlinearities that have been attributed toneurons and this has proved helpful in some cases

3 It is recommended to compare experimentally thefilter subspace recovered with a given stimulusensemble with that recovered with Gaussianstimuli (recording from the same neuron) differ-ences in the outcome between the two stimulustypes could indicate estimation biases or failuresof the model

Touryan Felsen and Dan (2005) compared STCanalysis in area V1 for white noise and natural imagesTo partially correct for the natural image stimuli they firstwhitened the stimuli in the ensemble Although thiscannot correct for the nonspherical nature of the stimulithey showed that the first two eigenvectors (representingcomplex cells in their data) were similar for white noiseand natural images The natural images required far fewerraw stimuli to achieve the same result probably becausethey are more effective at eliciting spikes They also foundadditional significant (and artifactual) filters that werecompared with artifactual filters arising in a simulationwith natural imagesOther techniques have been designed to cope directly

with non-Gaussian input such as images and thus bypassthis limitation of the STC approach The basic idea is quitesimple Instead of relying on a particular statistical moment(eg mean or variance) for comparison of the spike-triggered and raw stimulus distributions one can use a moregeneral comparison function that can identify virtually anydifference between the two distributions A natural choicefor such a function is information-theoretic One cancompare the mutual information between a set of filterresponses and the probability of a spike occurring(Paninski 2003 Sharpee et al 2003 2004) Thisapproach is promising because it places essentially norestriction on the stimulus ensemble A drawback is thatthe estimation problem is significantly more complicatedit is more expensive to compute and may get trapped inlocal optima However it has been successfully applied toestimate one- or two-dimensional subspace models insimulation and from physiological data in response tonatural images (Paninski 2003 Sharpee et al 2003 2004

2006) Other techniques based on artificial neural net-works (Lau Stanley amp Dan 2002 Lehky Sejnowski ampDesimone 1992) have also been developed and applied tonatural images (Prenger Wu David amp Gallant 2004)

Validation

Validation is useful to evaluate the degree to which therecovered model is an accurate description of the neuralresponse At the very least it is worthwhile verifying that themodel when fit to one run of white noise stimulation canthen predict responses to another run Because the model is arate model this is most directly done by measuringresponses to repeated stimuli and comparing their average(the PSTH) against that predicted from the model Anotherpossibility is to Bplay back[ as stimuli the eigenvectors thatwere found in the spike-triggered analysis to verify that theyaffect the neuronrsquos response as expected (Rust Schwartzet al 2005 Touryan et al 2002) This requires that oneperform the analysis and stimulus generation online duringthe experiment Playing back the eigenvectors is alsohelpful for determining the importance of the individualmodel components that are recovered from the analysis forexample the weakest components might have only a minorimpact on the neural responseIt is also of interest to test howwell the model generalizes

to other stimuli If one characterizes the model with a set ofbars how well does the model predict the response to asingle bar If one characterizes the model with highcontrast stimuli how well does it predict the response tolow contrast stimuli Ultimately we would like a modelthat predicts the response to any arbitrary stimulusValidating the model on different stimuli can help assessthe robustness of the model and when it breaks and in turncan identify the need for further improving spike-triggeredanalysis techniques

Discussion

We have described a set of spike-triggered techniquesfor characterizing the functional response properties ofneurons using stochastic stimuli In general there is atradeoff between restricting the subspace dimensionality(as in the STA and STC approaches) versus restricting thenonlinearity (as in the WienerVolterra approaches) Herewe have focused specifically on STA and STC analysesThese methods rely on an assumption that the response ofthe neuron is governed by an initial linear stage that servesto reduce the dimensionality of the stimulus space Thelinear stage is followed by a nonlinearity upon which weplace fairly minimal constraints Having worked withthese methods in both retina and V1 we have found that


many experimental and analysis issues are quite trickyWe have presented examples with model neuron simu-lations highlighting similarities with experiments wherepossibleEstimation of the linear subspace can be corrupted by

three distinct sources of error which we have discussed inthis article First there are errors due to the finiteness of thedata The rate at which these decrease with increasing datais given in Equation 8 and illustrated in Figure 9 Secondthere are biases that can arise from the interaction of theneural nonlinearities and use of non-Gaussian stimuliExamples are shown in Figure 15 Finally there are errorsdue to model failureThere are a number of interesting directions for future

research First the LNP model can be extended toincorporate some spike history dependence by recursivelyfeeding back the spiking output into the linear input stageThis Brecursive LNP[ model (also referred to as a generallinear model [GLM]) has appeared in recent literature(Pillow Paninski Uzzell Simoncelli amp Chichilnisky2005 Truccolo Eden Fellows Donogue amp Brown2005) and may allow the introduction of some adaptationeffects as well as shorter timescale effects such asrefractoriness bursting or rapid gain adjustments Thismodel can no longer be directly fit to data with STA andSTC and requires more complex fitting procedures Inaddition the techniques described here can be adjusted forthe analysis of multineuronal interactions (eg Nykamp2003 Okatan Wilson amp Brown 2005 Pillow ShlensPaninski Chichilnisky amp Simoncelli 2005b) Suchmethods have been applied for example in visual cortex(Tsodyks Kenet Grinvald amp Arieli 1999) motor cortex(Paninski Fellows Shoham Hatsopoulos amp Donoghue2004) and hippocampus (Harris Csicsvari HiraseDragoi amp Buzsaki 2003) Also neurons adapt to stimuliover multiple timescales (Brenner Bialek amp de Ruytervan Steveninck 2000 Fairhall Lewen Bialek amp deRuyter van Steveninck 2001) and it would be interestingto extend current approaches to incorporate adaptationFinally it would be desirable to develop techniques thatcan be applied to a cascaded series of LNP stages Thiswill be essential for modeling responses in higher ordersensory areas which are presumably constructed frommore peripheral responses Specifically if the afferentresponses that arrive in a particular neural area arereasonably understood then one may be able to arrangeto perform the spike-triggered analysis in the space of theafferents (Rust Simoncelli et al 2005)

Appendix

We describe how to compute STA and STC forelliptically symmetric Gaussian stimuli If the distributionof stimuli is elliptically symmetric then a modified

STA can be computed as follows (eg Theunissen et al2001)

Apara frac14 Cj1 A ethA1THORN

where

C frac14~n

sYethtnTHORNsYTethtnTHORN ethA2THORN

is the covariance matrix of the raw stimuli (we assumethat the mean stimulus is zero) Note that this solution is aregression estimate for a linear mapping from stimuli tospikes The surprising result is that one can use linearregression on a one-dimensional LN model if the inputvectors are elliptically distributedAs in the case of STA STC can be generalized to the case

of an elliptically symmetric stimulus distribution Here thenatural choice is to solve for stimulus dimensions in whichthe ratio of variances of the spike-triggered and rawstimulus ensembles is either large or small Mathemati-cally we write this ratio in a direction specified by unitvector u as

r ueth THORN frac14 uTCu

uTCu ethA3THORN

The solution to this problem can be computed directlyusing a generalized eigenvector analysis Specifically wefirst solve for the whitening transform in the deno-minator computing the eigenvalues D and eigenvectorsV of the covariance matrix of the raw stimuli We setX frac14 Veth

ffiffiffiffiD

pTHORNj1

and u frac14 Xv obtaining

r ueth THORN frac14 vTXTCVv

vT v ethA4THORN

This is now equivalent to solving a standard eigenvectorproblem calculating the eigenvalues and eigenvectors ofXTCX

Acknowledgments

We are very grateful to E J Chichilnisky TonyMovshon and Liam Paninski for helpful discussions

Commercial relationships noneCorresponding author Odelia SchwartzEmail odeliasalkeduAddress The Salk Institute 10010 North Torrey PinesRoad La Jolla CA 92037 USA


Footnotes

1It should be noted that a WienerVolterra approach has

also been applied within a subspace but under theassumption of a lowltorder polynomial nonlinearity (egEmerson et al 1987 1992 Szulborski amp Palmer 1990)

2Note that the STA estimate is unbiased but it does not in

general correspond to a maximum likelihood estimate(Dayan amp Abbott 2001)

3Note that recent work (Pillow amp Simoncelli 2006)

suggests an informationlttheoretic objective that combinesthe STA and STC optimally

References

Adelson E H amp Bergen J R (1985) Spatiotemporalenergy models for the perception of motion Journalof the Optical Society of America A Optics andImage Science 2 284ndash299 [PubMed]

Aguera y Arcas B amp Fairhall A L (2003) What causes aneuron to spikeNeural Computation 15 1789ndash1807[PubMed]

Aguera y Arcas B Fairhall A L amp Bialek W (2001)What can a single neuron compute In T K LeenT G Dietterich amp V Tresp (Eds) Adv NeuralInformation Processing Systems (NIPS00) (Vol 13pp 75ndash81) Cambridge MA MIT Press

Aguera y Arcas B Fairhall A L amp Bialek W (2003)Computation in a single neuron Hodgkin and Huxleyrevisited Neural Computation 15 1715ndash1749 [PubMed]

Albrecht D G amp Geisler W S (1991) Motionsensitivity and the contrast-response function ofsimple cells in the visual cortex Visual Neuroscience7 531ndash546 [PubMed]

Anzai A Ohzawa I amp Freeman R D (1999) Neuralmechanisms for processing binocular informationI Simple cells Journal of Neurophysiology 82891ndash908 [PubMed] [Article]

Bair W Cavanaugh J R amp Movshon J A (1997)Reconstructing stimulus velocity from neuronalresponses in area MT In M C Mozer M I Jordanamp T Petsche (Eds) Adv Neural Information Process-ing Systems (NIPS96) (Vol 9 pp 34ndash40) Cam-bridge MA MIT Press

Berry M J II amp Meister M (1998) Refractorinessand neural precision The Journal of Neuroscience18 2200ndash2211 [PubMed] [Article]

Bialek W amp de Ruyter van Steveninck R (2005)Features and dimensions Motion estimation in flyvision httparxivorgabsq-bio0505003

Borghuis B G Perge J A Vajda I van Wezel R Jvan de Grind W A amp Lankheet M J (2003) Themotion reverse correlation (MRC) method A linearsystems approach in the motion domain Journal ofNeuroscience Methods 123 153ndash166 [PubMed]

Brenner N Bialek W amp de Ruyter van Steveninck R(2000) Adaptive rescaling maximizes informationtransmissionNeuron 26 695ndash702 [PubMed] [Article]

Bussgang J J (1952) Cross-correlation functions ofamplitude-distorted gaussian signals RLE TechnicalReports 216 Cambridge MA Research Laboratoryof Electronics

Chichilnisky E J (2001) A simple white noise analysisof neuronal light responses Network 12 199ndash213[PubMed]

Daugman J G (1989) Entropy reduction and decorrela-tion in visual coding by oriented neural receptivefields IEEE Transactions on Bio-medical Engineer-ing 36 107ndash114 [PubMed]

David S V amp Gallant J L (2005) Predicting neuronalresponses during natural visionNetwork 16 239ndash260[PubMed]

David S V Vinje W E amp Gallant J L (2004)Natural stimulus statistics alter the receptive fieldstructure of v1 neurons The Journal of Neuroscience24 6991ndash7006 [PubMed] [Article]

Dayan P amp Abbott L F (2001) Theoretical neuro-science Computational and mathematical modelingof neural systems Cambridge MA MIT Press

DeAngelis G C Ohzawa I amp Freeman R D (1993)Spatiotemporal organization of simple cell receptivefields in the catrsquos striate cortex II Linearity oftemporal and spatial summation Journal of Neuro-physiology 69 1118ndash1135 [PubMed]

deBoer E amp Kuyper P (1968) Triggered correlationIEEE Transactions on Biomedical Engineering 15169ndash179 [PubMed]

de Ruyter van Steveninck R amp Bialek W (1988)Coding and information transfer in short spike sequen-ces Proceedings of the Royal Society of London BBiological Sciences 234 379ndash414

Eggermont J J Johannesma P M amp Aertsen A M(1983) Reverse-correlation methods in auditoryresearch Quarterly Reviews of Biophysics 16341ndash414 [PubMed]

Emerson R C Bergen J R amp Adelson E H (1992)Directionally selective complex cells and the compu-tation of motion energy in cat visual cortex VisionResearch 32 203ndash218 [PubMed]

Emerson R C Citron M C Vaughn W J amp KleinS A (1987) Nonlinear directionally selective sub-


units in complex cells of cat striate cortex Journal ofNeurophysiology 58 33ndash65 [PubMed]

Fairhall A L Lewen G D Bialek W amp de RuyterVan Steveninck R R (2001) Efficiency andambiguity in an adaptive neural code Nature 412787ndash792 [PubMed]

Felsen G amp Dan Y (2005) A natural approach tostudying vision Nature Neuroscience 8 1643ndash1646[PubMed]

Field D J (1987) Relations between the statistics ofnatural images and the response properties of corticalcells Journal of the Optical Society of America AOptics and Image Science 4 2379ndash2394 [PubMed]

Geisler D (1992) Two-tone suppression by a saturatingfeedback model of the cochlear partition HearingResearch 63 203ndash211

Harris K D Csicsvari J Hirase H Dragoi G ampBuzsaki G (2003) Organization of cell assembliesin the hippocampus Nature 424 552ndash556[PubMed]

Heeger D J (1992) Normalization of cell responses incat striate cortex Visual Neuroscience 9 181ndash197[PubMed]

Hochstein S amp Shapley R M (1976) Linear andnonlinear spatial subunits in Y cat retinal ganglioncells The Journal of Physiology 262 265ndash284[PubMed] [Article]

Horwitz G D Chichilnisky E J amp Albright T D (2005)Bluendashyellow signals are enhanced by spatiotemporalluminance contrast in macaque V1 Journal of Neuro-physiology 93 2263ndash2278 [PubMed] [Article]

Jones J P amp Palmer L A (1987) The two-dimensionalspatial structure of simple receptive fields in cat striatecortex Journal of Neurophysiology 58 1187ndash1211[PubMed]

Keat J Reinagel P Reid R C amp Meister M (2001)Predicting every spike A model for the responses ofvisual neurons Neuron 30 803ndash817 [PubMed][Article]

Korenberg M J Sakai H M amp Naka K (1989)Dissection of the neuron network in the catfish innerretina III Interpretation of spike kernels Journal ofNeurophysiology 61 1110ndash1120 [PubMed]

Lau B Stanley G B amp Dan Y (2002) Computationalsubunits of visual cortical neurons revealed byartificial neural networks Proceedings of theNational Academy of Sciences of the United Statesof America 99 8974ndash8979 [PubMed] [Article]

Lehky S R Sejnowski T J amp Desimone R (1992)Predicting responses of nonlinear neurons in monkeystriate cortex to complex patterns Journal of Neuro-science 12 3568ndash3581 [PubMed] [Article]

Lesica N A amp Stanley G B (2004) Encoding ofnatural scene movies by tonic and burst spikes in thelateral geniculate nucleus The Journal of Neuro-science 24 10731ndash10740 [PubMed] [Article]

Marmarelis P Z amp Marmarelis V Z (1978) Analysisof physiological systems The white noise approachLondon Plenum Press

McLean J amp Palmer L A (1989) Contribution oflinear spatiotemporal receptive field structure tovelocity selectivity of simple cells in area 17 of catVision Research 29 675ndash679 [PubMed]

MeisterM Pine JampBaylorDA (1994)Multi-neuronalsignals from the retina Acquisition and analysisJournal of Neuroscience Methods 51 95ndash106[PubMed]

Nishimoto S Ishida T amp Ohzawa I (2006) Receptivefield properties of neurons in the early visual cortexrevealed by local spectral reverse correlationThe Journal of Neuroscience 26 3269ndash3280[PubMed]

Nykamp D (2003) White noise analysis of coupledlinear-nonlinear systems SIAM Journal on AppliedMathematics 63 1208ndash1230

Nykamp D Q amp Ringach D L (2002) Full identi-fication of a linear-nonlinear system via cross-correlation analysis Journal of Vision 2(1) 1ndash11httpjournalofvisionorg211 doi101167211[PubMed] [Article]

Okatan M Wilson M A amp Brown E N (2005)Analyzing functional connectivity using a networklikelihood model of ensemble neural spiking activityNeural Computation 17 1927ndash1961 [PubMed]

Paninski L (2003) Convergence properties of somespike-triggered analysis techniques Network 14437ndash464 [PubMed]

Paninski L Fellows M Shoham S Hatsopoulos N ampDonoghue J (2004) Nonlinear population modelsfor the encoding of dynamic hand position signals inMI Neurocomputing Paper presented at Computa-tional Neuroscience Alicante Spain July 2003

Pillow J W Paninski L Uzzell V J SimoncelliE P amp Chichilnisky E J (2005) Prediction anddecoding of retinal ganglion cell responses with aprobabilistic spiking model The Journal of Neuro-science 25 11003ndash11013 [PubMed] [Article]

Pillow J W Shlens J Paninski L Chichilnisky E J ampSimoncelli E P (2005a) Modeling multineuronalresponses in primate retinal ganglion cells In Compu-tational and Systems Neuroscience (CoSyNe) SaltLake City UT

Pillow J W Shlens J Paninski L Chichilnisky E J ampSimoncelli E P (2005b) Modeling the correlatedspike responses of a cluster of primate retinal ganglion


cells In Annual Meeting Neurosciences WashingtonDC

Pillow J W amp Simoncelli E P (2003) Biases in whitenoise analysis due to non-Poisson spike generationNeurocomputing 52ndash54 109ndash115

Pillow J W amp Simoncelli E P (2006) Dimension-ality reduction in neural models An information-theoretic generalization of spike-triggered averageand covariance analysis Journal of Vision 6(4)414ndash428 httpwwwjournalofvisionorg649

Pillow J W Simoncelli E P amp Chichilnisky E J(2003) Characterization of nonlinear spatiotemporalproperties of macaque retinal ganglion cells usingspike-triggered covariance In Annual Meeting Neu-rosciences New Orleans

Pillow J W Simoncelli E P amp Chichilnisky E J(2004) Characterization of macaque retinal ganglioncell responses using spike-triggered covariance InComputational and Systems Neuroscience (CoSyNe)Cold Spring Harbor NY

Prenger R Wu M C David S V amp Gallant J L(2004) Nonlinear V1 responses to natural scenesrevealed by neural network analysis Neural Networks17 663ndash679 [PubMed]

Reich D S Victor J D amp Knight B W (1998) Thepower ratio and the interval map Spiking models andextracellular recordings The Journal of Neuro-science 18 10090ndash10104 [PubMed] [Article]

Reid R C amp Alonso J M (1995) Specificity ofmonosynaptic connections from thalamus to visualcortex Nature 378 281ndash284 [PubMed]

Rieke F Warland D de Ruyter van Steveninck R R ampBialek W (1997) Spikes Exploring the neural codeCambridge MA MIT Press

Ringach D L (2004) Mapping receptive fields inprimary visual cortex The Journal of Physiology558 717ndash728 [PubMed] [Article]

Ringach D L Hawken M J amp Shapley R (2002)Receptive field structure of neurons inmonkey primaryvisual cortex revealed by stimulation with naturalimage sequences Journal of Vision 2(1) 12ndash24httpjournalofvisionorg212 doi101167212[PubMed] [Article]

Ringach D L Sapiro G amp Shapley R (1997) A subspacereverse-correlation technique for the study of visualneurons Vision Research 37 2455ndash2464 [PubMed]

Rust N C amp Movshon J A (2005) In praise of artificeNature Neuroscience 8 1647ndash1650 [PubMed]

Rust N C Schwartz O Movshon J A amp SimoncelliE P (2004) Spike-triggered characterization of

excitatory and suppressive stimulus dimensionsin monkey V1 Neurocomputing 58ndash60C 793ndash799

Rust N C Schwartz OMovshon J A amp Simoncelli E P(2005) Spatiotemporal elements of macaque V1 recep-tive fields Neuron 46 945ndash956 [PubMed] [Article]

Rust N C Simoncelli E P amp Movshon J A (2005)How macaque MT cells compute pattern motion InAnnual Meeting Neurosciences Washington DC

Sakai H M amp Naka K (1987) Signal transmission inthe catfish retina V Sensitivity and circuit Journalof Neurophysiology 58 1329ndash1350 [PubMed]

Sakai K amp Tanaka S (2000) Spatial pooling in thesecond-order spatial structure of cortical complexcells Vision Research 40 855ndash871 [PubMed]

Schwartz O Chichilnisky E J amp Simoncelli E P(2002) Characterizing neural gain control usingspike-triggered covariance In T G Dietterich SBecker amp Z Ghahramani (Eds) Adv NeuralInformation Processing Systems (NIPS01) (Vol 14pp 269ndash276) Cambridge MA MIT Press

Schwartz O amp Simoncelli E P (2001) A spike-triggered covariance method for characterizing divi-sive normalization models In First Annual MeetingVision Sciences Society Sarasota

Sen K Wright B D Doupe A J amp Bialek W (2000)Discovering features of natural stimuli relevant toforebrain auditory neurons In Society for NeuroscienceAbstracts (Vol 26 p 1475) Washington DC

Sharpee T Rust N C amp Bialek W (2003) Max-imizing informative dimensions Analyzing neuralresponses to natural signals In S Becker S Thrun ampK Obermayer (Eds) Adv Neural Information Pro-cessing Systems (NIPS02) (Vol 15 pp 261ndash268)Cambridge MA MIT Press

Sharpee T Rust N C amp Bialek W (2004) Analyzingneural responses to natural signalsMaximally informa-tive dimensions Neural Computation 16 223ndash250[PubMed]

Sharpee T O Sugihara H Kurgansky A V RebrikS P Stryker M P amp Miller K D (2006)Adaptive filtering enhances information transmissionin visual cortex Nature 439 936ndash942 [PubMed]

Simoncelli E P Pillow J Paninski L amp Schwartz O(2004) Characterization of neural responses withstochastic stimuli In M Gazzaniga (Ed) Thecognitive neurosciences (3rd ed pp 327ndash338) Cam-bridge MA MIT Press

Smyth D Willmore B Baker G E Thompson I D ampTolhurst D J (2003) The receptive-field organizationof simple cells in primary visual cortex of ferrets undernatural scene stimulation The Journal of Neuro-science 23 4746ndash4759 [PubMed] [Article]


doi101167649 [PubMed] [Article]

Szulborski R G amp Palmer L A (1990) The two-dimensional spatial structure of nonlinear subunits inthe receptive fields of complex cells Vision Research30 249ndash254 [PubMed]

Theunissen F E David S V Singh N C Hsu AVinje W E amp Gallant J L (2001) Estimatingspatio-temporal receptive fields of auditory and visualneurons from their responses to natural stimuliNetwork 12 289ndash316 [PubMed]

Touryan J Felsen G amp Dan Y (2005) Spatialstructure of complex cell receptive fields measuredwith natural images Neuron 45 781ndash791 [PubMed][Article]

Touryan J Lau B amp Dan Y (2002) Isolation ofrelevant visual features from random stimuli forcortical complex cells The Journal of Neuroscience22 10811ndash10818 [PubMed] [Article]

Truccolo W Eden U T Fellows M R DonoghueJ P amp Brown E N (2005) A point processframework for relating neural spiking activity tospiking history neural ensemble and extrinsiccovariate effects Journal of Neurophysiology 931074ndash1089 [PubMed] [Article]

Tsodyks M Kenet T Grinvald A amp Arieli A (1999)Linking spontaneous activity of single cortical neu-rons and the underlying functional architectureScience 286 1943ndash1946 [PubMed]

Volterra V (1959) Theory of functionals and of integroand integro-differential equations New York Dover

Wiener N (1958) Nonlinear problems in random theoryNew York Wiley


triggered ensemble are then two clouds of points in thisspace Intuitively the task of estimating the neuralresponse function corresponds to describing the ways inwhich these two clouds differ In practice when the inputstimulus space is of high dimensionality one cannotestimate the neural response function without furtherassumptionsSpike-triggered analysis has been employed to estimate

the terms of a WienerVolterra expansion (KorenbergSakai amp Naka 1989 Marmarelis amp Marmarelis 1978Volterra 1959 Wiener 1958) in which the mapping fromstimuli to firing rate is described using a low-orderpolynomial (see Dayan amp Abbott 2001 Rieke Warlandde Ruyter van Steveninck amp Bialek 1997 for a review)Although any reasonable function can be approximated asa polynomial the firing rate nonlinearities found in theresponses of sensory neurons (eg half-wave rectifiedrapidly accelerating and saturating) tend to require apolynomial with many terms (see eg Rieke et al 1997)However the amount of data needed for accurateestimation grows rapidly with the number of termsTherefore in an experimental setting where one canestimate only the first few terms of the expansion thepolynomial places a strong restriction on the nonlinearityAs an alternative to the polynomial approximation one

can assume that the response function operates on a low-dimensional linear subspace of the full stimulus space(Bialek amp de Ruyter van Steveninck 2005 de Ruyter van

Steveninck amp Bialek 1988) That is the response of aneuron is modeled with a small set of linear filters whoseoutputs are combined nonlinearly to generate the instanta-neous firing rate This is in contrast to the WienerVolterraapproach which in general does not restrict the subspacebut places a restriction on the nonlinearity1 By concen-trating the data into a space of reduced dimensionality theneural response can be fit with less restriction on the formof the nonlinearityA number of techniques have been developed to estimate

the linear subspace and subsequently the nonlinearity Inthe most widely used form of this analysis the linear frontend is limited to a single filter that serves as an explicitrepresentation of the Breceptive field[ of the neuron but thenonlinearity is essentially unrestricted With the rightchoice of stimuli this linear filter may be estimated bycomputing the spike-triggered average (STA) stimulus (iethe mean stimulus that elicited a spike) The STA has beenwidely used in studying auditory neurons (eg EggermontJohannesma amp Aertsen 1983) In the visual system STAhas been used to characterize retinal ganglion cells(eg Meister Pine amp Baylor 1994 Sakai amp Naka1987) lateral geniculate neurons (eg Reid amp Alonzo1995) and simple cells in primary visual cortex (V1eg DeAngelis Ohzawa amp Freeman 1993 Jones ampPalmer 1987 McLean amp Palmer 1989) Given the STAfilter one typically has enough experimental data toconstruct a nonparametric estimate of the nonlinearity

Figure 1 The spike-triggered stimulus ensemble (A) Discretized stimulus sequence and observed neural response (spike train) On eachtime step the stimulus consists of an array of randomly chosen values (eight for this example) These could represent for example theintensities of a fixed set of individual pixels on the screen or the contrast of each of a set of fixed sinusoidal gratings that are additivelysuperimposed The neural response at any particular moment in time is assumed to be completely determined by the stimulus segmentthat occurred during a prespecified interval in the past In this figure the segment covers six time steps and lags three time steps behindthe current time (to account for response latency) The spike-triggered ensemble consists of the set of segments associated with spikes(B) Geometric (vector space) view of the spike-triggered ensemble Stimuli (here 48-dimensional) are projected onto two spacendashtimevectors In this example each of the two vectors contained 1 stixel (spacendashtime pixel) set to a value of 1 and the other 47 stixels were setto 0 For these given vectors the projection is equivalent to the intensity of the corresponding stixel in the stimulus More generally onecan project the stimuli onto any two 48-dimensional vectors The spike-triggered stimulus segments (white points) constitute a subset ofall stimulus segments presented (black points)















YethtTHORNIkY




Y


kY







Y

0IkY











A frac14 1

N~N

nfrac141










C frac14 1

Nj1~N

nfrac141

sYtneth THORNj A

sYtneth THORNj A

T eth3THORN








sY frac14 s









Y

22

eth5THORN







YTHORN2h i

eth6THORN




1thorn )kY

1 sY

22


YTHORN2 eth7THORN



















MAE kY

frac14 A

BethkYTHORN

ffiffiffiffid

N

r eth8THORN












and therefore

































1 sY

22 thorn )k

Y

2 sY

22








Experimental issues





















EeETe thorn E0EnD

j12

n ETnE0 eth11THORN













Validation



Discussion






Appendix




where

C frac14~n





uTCu ethA3THORN


ffiffiffiffiD

pTHORNj1



vT v ethA4THORN


Acknowledgments




Footnotes







References








































































































YethtTHORNIkY




Y


kY







Y

0IkY











A frac14 1

N~N

nfrac141










C frac14 1

Nj1~N

nfrac141

sYtneth THORNj A

sYtneth THORNj A

T eth3THORN








sY frac14 s









Y

22

eth5THORN







YTHORN2h i

eth6THORN




1thorn )kY

1 sY

22


YTHORN2 eth7THORN



















MAE kY

frac14 A

BethkYTHORN

ffiffiffiffid

N

r eth8THORN












and therefore

































1 sY

22 thorn )k

Y

2 sY

22








Experimental issues





















EeETe thorn E0EnD

j12

n ETnE0 eth11THORN













Validation



Discussion






Appendix




where

C frac14~n





uTCu ethA3THORN


ffiffiffiffiD

pTHORNj1



vT v ethA4THORN


Acknowledgments




Footnotes







References


































































































A frac14 1

N~N

nfrac141










C frac14 1

Nj1~N

nfrac141

sYtneth THORNj A

sYtneth THORNj A

T eth3THORN








sY frac14 s









Y

22

eth5THORN







YTHORN2h i

eth6THORN




1thorn )kY

1 sY

22


YTHORN2 eth7THORN



















MAE kY

frac14 A

BethkYTHORN

ffiffiffiffid

N

r eth8THORN












and therefore

































1 sY

22 thorn )k

Y

2 sY

22








Experimental issues





















EeETe thorn E0EnD

j12

n ETnE0 eth11THORN













Validation



Discussion






Appendix




where

C frac14~n





uTCu ethA3THORN


ffiffiffiffiD

pTHORNj1



vT v ethA4THORN


Acknowledgments




Footnotes







References





























































































C frac14 1

Nj1~N

nfrac141

sYtneth THORNj A

sYtneth THORNj A

T eth3THORN








sY frac14 s









Y

22

eth5THORN







YTHORN2h i

eth6THORN




1thorn )kY

1 sY

22


YTHORN2 eth7THORN



















MAE kY

frac14 A

BethkYTHORN

ffiffiffiffid

N

r eth8THORN












and therefore

































1 sY

22 thorn )k

Y

2 sY

22








Experimental issues





















EeETe thorn E0EnD

j12

n ETnE0 eth11THORN













Validation



Discussion






Appendix




where

C frac14~n





uTCu ethA3THORN


ffiffiffiffiD

pTHORNj1



vT v ethA4THORN


Acknowledgments




Footnotes







References




























































































sY frac14 s









Y

22

eth5THORN







YTHORN2h i

eth6THORN




1thorn )kY

1 sY

22


YTHORN2 eth7THORN



















MAE kY

frac14 A

BethkYTHORN

ffiffiffiffid

N

r eth8THORN












and therefore

































1 sY

22 thorn )k

Y

2 sY

22








Experimental issues





















EeETe thorn E0EnD

j12

n ETnE0 eth11THORN













Validation



Discussion






Appendix




where

C frac14~n





uTCu ethA3THORN


ffiffiffiffiD

pTHORNj1



vT v ethA4THORN


Acknowledgments




Footnotes







References






























































































YTHORN2h i

eth6THORN




1thorn )kY

1 sY

22


YTHORN2 eth7THORN



















MAE kY

frac14 A

BethkYTHORN

ffiffiffiffid

N

r eth8THORN












and therefore

































1 sY

22 thorn )k

Y

2 sY

22








Experimental issues





















EeETe thorn E0EnD

j12

n ETnE0 eth11THORN













Validation



Discussion






Appendix




where

C frac14~n





uTCu ethA3THORN


ffiffiffiffiD

pTHORNj1



vT v ethA4THORN


Acknowledgments




Footnotes







References








































































































MAE kY

frac14 A

BethkYTHORN

ffiffiffiffid

N

r eth8THORN












and therefore

































1 sY

22 thorn )k

Y

2 sY

22








Experimental issues





















EeETe thorn E0EnD

j12

n ETnE0 eth11THORN













Validation



Discussion






Appendix




where

C frac14~n





uTCu ethA3THORN


ffiffiffiffiD

pTHORNj1



vT v ethA4THORN


Acknowledgments




Footnotes







References






























































































and therefore

































1 sY

22 thorn )k

Y

2 sY

22








Experimental issues





















EeETe thorn E0EnD

j12

n ETnE0 eth11THORN













Validation



Discussion






Appendix




where

C frac14~n





uTCu ethA3THORN


ffiffiffiffiD

pTHORNj1



vT v ethA4THORN


Acknowledgments




Footnotes







References



















































































































1 sY

22 thorn )k

Y

2 sY

22








Experimental issues





















EeETe thorn E0EnD

j12

n ETnE0 eth11THORN













Validation



Discussion






Appendix




where

C frac14~n





uTCu ethA3THORN


ffiffiffiffiD

pTHORNj1



vT v ethA4THORN


Acknowledgments




Footnotes







References




























































































Experimental issues





















EeETe thorn E0EnD

j12

n ETnE0 eth11THORN













Validation



Discussion






Appendix




where

C frac14~n





uTCu ethA3THORN


ffiffiffiffiD

pTHORNj1



vT v ethA4THORN


Acknowledgments




Footnotes







References








































































































EeETe thorn E0EnD

j12

n ETnE0 eth11THORN













Validation



Discussion






Appendix




where

C frac14~n





uTCu ethA3THORN


ffiffiffiffiD

pTHORNj1



vT v ethA4THORN


Acknowledgments




Footnotes







References

































































































Validation



Discussion






Appendix




where

C frac14~n





uTCu ethA3THORN


ffiffiffiffiD

pTHORNj1



vT v ethA4THORN


Acknowledgments




Footnotes







References






























































































Appendix




where

C frac14~n





uTCu ethA3THORN


ffiffiffiffiD

pTHORNj1



vT v ethA4THORN


Acknowledgments




Footnotes







References



























































































Footnotes







References



























































































spike-triggered neural characterization - princeton...

Documents