p. c. bressloff et al- dynamical mechanism for sharp orientation tuning in an integrate-and-fire...

8/3/2019 P. C. Bressloff et al- Dynamical Mechanism for Sharp Orientation Tuning in an Integrate-and-Fire Model of a Cortical

1/39

ARTICLE Communicated by Carl van Vreeswijk

Dy namical Mechanism for Sharp Orientation Tunin g in an

Integrate-and-Fire Mod el of a Cortical Hy percolu mn

P. C. Bress lof f

Nonlinear and Complex Syst ems Group, Department of M athematical Sciences, Lough-

borough Un iversity, Loughborough, Leicestershire LE11 3TU, U .K.

N. W. Bressloff

Computational and Engineering Design Centre, Department of Aeronautics and As-

tronautics, University of Southampton, Southampton, SO17 1BJ, U.K.

J. D. Cowan

Department of Mathematics, University of Chicago, Chicago, IL 60637, U.S.A.

Orientation tuning in a ring of pulse-coupled integrate-and-fire (IF) neu-

rons is analyzed in terms of spontaneous pattern formation. It is shown

how the ring bifurcates from a synchronous state to a non-phase-locked

state who se sp ike trains are characterized b y clustered but irregu lar fluc-

tuations of the interspike intervals (ISIs). The separation of these clusters

in phase space results in a localized peak of activity as measured by the

time-averaged firing rate of the ne urons. This g enerates a s harp o rienta-

tion tuning curve that can lock to a slowly rotating, weakly tuned exter-

nal stimulus. Under certain conditions, the peak can slowly rotate even

to a fixed external stimulus. The ring also exhibits hysteresis due to thesubcritical nature of the bifurcation to sharp orientation tuning. Such be-

havior is shown to be consistent with a corresponding analog version of

the IFmo del in the limit of slow synaptic interactions. For fast synapses,

the deterministic fluctuations o f the ISIs associated w ith the tuning curve

can su pport a coefficient of variation of order unity.

1 Introduction

Recent studies of the formation of localized spatial patterns in one- and

two-dimensional neural netw orks have been used to investigate a variety

of neuronal processes including orientation selectivity in primary visual

cortex (Ben-Yishai, Bar-Or, Lev, & Somp olinsky, 1995; Ben-Yishai, Ha nsel,

& Sompolinsky, 1997; H ansel & Sompolinsky, 1997; Mu nd el, Dimitrov, &

Cowan, 1997), the coding of arm movements in motor cortex (Lukashin &Georgopolous, 1994a, 1994b; Georgopolous, 1995), and the control of sac-

cadic eye movements (Zhang, 1996).The networ ks considered in these stud -

ies are based on a simplified rate or analog model of a neuron, in which

Neural Computation 12, 24732511 (2000) c 2000 Massachusetts Institute of Technology


2/39

2474 P. C. Bressloff, N . W. Bressloff, an d J. D. Cow an

the state of each neuron is characterized by a single continuous variable

that determines its short-term average output activity (Cowan, 1968; Wil-

son & Cowan , 1972). All of these mod els involve the sam e basic dynam ical

mechanism for the formation of localized patterns: spontaneous symmetry

breaking from a uniform resting state (Cowan, 1982). Localized structures

consisting of a single peak of high activity occur when the maximum (spa-

tial) Fourier component of the combination of excitatory and inhibitory

interactions between neurons has a w avelength comparable to the size of

the n etwork. Typically such networks have periodic boundary conditions

so that unraveling the network results in a spatially periodic pattern of ac-

tivity, as stud ied p reviously by Ermen trout and Cowan (1979a, 1979b) (see

also the review by Ermentrou t, 1998).

In contrast to mean firing-rate models, there has been relatively little

analytical work on pattern formation in more realistic spiking mod els. Anum ber of nu merical stud ies have shown th at localized activity profiles can

occur in networ ks of Hod gkin-Huxley neurons (Lukashin & Georgopolous,

1994a, 1994b; Hansel & Sompolinsky, 1996). Moreover, both local (Somers,

Nelson, & Sur, 1995) and global (Usher, Stemmler, Koch, & Olami, 1994)

patterns of activity have been found in integrate-and-fire (IF) networks.

Recently a dynamical theory of global pattern formation in IF networks

has been developed in terms of the nonlinear map of the neuronal firing

times (Bressloff & Coombes, 1998b, 2000). A linear stability analysis of this

map shows how, in the case of short-range excitation and long-range in-

hibition, a network that is synchronized in the weak coupling regime can

destabilize as the strength of coupling is increased, leading to a state char-

acterized by clustered but irregular fluctuations of the interspike intervals

(ISIs). The separation of these clusters in phase-space results in a spatially

periodic pattern of mean (time-averaged) firing rate across the network,which is modu lated by d eterministic fluctuations in the instantaneous fir-

ing rates.

In this article we app ly the theory of Bressloff and Coombes (1998b, 2000)

to the analysis of localized pattern formation in an IFversion of the model of

sharp orienta tion tun ing developed by Han sel and Sompolinsky (1997) and

Ben-Yishai et al. (1995, 1997). We first study a corresponding analog model

in which the outputs of the neurons are taken to be mean firing rates. Since

the resulting firing-rate function is nonlinear rather th an semilinear, it is not

possible to construct exact solutions for the orientation t unin g curves along

the lines of Hansel and Sompolinsky (1997). Instead, we investigate the

existence and stability of such activity p rofiles using bifurcation th eory. We

show how a uniform resting state destabilizes to a stable localized pattern

as the strength of neuron al recurrent interactions is increased. This localized

state consists of a single narrow peak of activity whose center can lock toa slowly rotating, weakly tuned external stimulus. Interestingly, we find

that the bifurcation from the resting state is subcritical: the system jumps

to a localized activity profile on destabilization of the resting state, and


3/39

In tegrate-an d-Fire Mod el of Orien tation Tu nin g 2475

hysteresis occurs in the sense that sh arp or ientation tun ing can coexist with

a stable resting state.

We then turn to the full IFmod el and show how an analysis ofthe nonlin-

ear firing time map can serveas a basis for und erstanding orientation tuning

in networks of spiking neurons. We derive explicit criteria for the stability

of phase-locked solutions by considering the propagation of perturbations

of the firing times th roughout the n etwork. Our analysis identifies regions

in para meter sp ace where instabilities in the firing tim es cause (subcritical)

bifurcations to n on-phase-locked states that support localized patterns of

mean firing rates across the network similar to the sharp orientation tun-

ing curves found in the corresponding analog m odel. However, as found

in the case of global pattern formation (Bressloff & Coombes, 1998b, 2000),

the tun ing curves are m odulated by d eterministic fluctuations of the ISIs

on closed quasiperiod ic orbits, which grow w ith the speed of synapses. Forsufficiently fast synapses, the resulting coefficient of variation (CV) can be

of order unity and the ISIs appear to exhibit chaotic motion due to breakup

of the quasiperiodic orbits.

2 The Model

We consider an IF version of the neural network model for orientation tun-

ing in a cortical hyp ercolum n developed by H ansel and Sompolinsky (1997)

and Ben-Yishai et al.(1997). This is a simplified version of the mod el stud ied

num erically by Somers et al. (1995). The netw ork consists of two su bpop u-

lations of neurons, one excitatory and the other inhibitory, which code for

the orientation of a visual stimulus appearing in a common visual field (see

Figure 1).The ind exL = E,Iwill be used to distinguish the two populations.Each n euron is parameterized by an angle , 0 < , which representsits orientation preference. (The angle is restricted to be from 0 to since

a bar that is oriented at an angle 0 is indistinguishable from one that has

an orientation .) Let UL(, t) denote the membrane potential at time t of a

neuron of type L and orientation preference . The neurons are m odeled as

IF oscillators evolving according to the set of equations

0 UL(, t)

t= h0 UL(, t) +XL(, t), L = E,I, (2.1)

where h0 is a constant external inpu t or bias, 0 is a membrane time constant,

an d XL(, t) denotes the totalsynap ticinpu t to a neuron oftypeL.Periodic

boundary conditions are assumed so that UL(0, t) = UL(, t).Equation 2.1issupplemented by the condition that whenever a neuron reaches a threshold

, it fires a spike, and its membrane potential is immediately reset to zero.

In other words,

UL( , t+) = 0 w h en ev er UL( , t) = , L = E,I. (2.2)


4/39


I

E

WEI

WIE WEE

WII

Figure 1: Network architecture of a ring model of a cortical hypercolumn (see

Hansel & Sompolinsky, 1997).

For concreteness, we set th e threshold = 1 and take h0 = 0.9 < , so thatin the ab sence of any synap tic inputs, all neuron s are qu iescent. We also fix

the fundamental unit of time to be of the order 510 msec by setting 0 = 1.(All results concerning firing rates or interspike intervals presented in this

article are in units of0.)

The total synaptic inpu t XL(, t) is taken to be of the form

XL(, t) =

M=E,I

0

d

WLM( )YM( , t) + hL(, t), (2.3)

where WLM( ) denotes the interaction between a p resynaptic neuron of type M and a postsynaptic neuron of type L, and YM(, t) is the

effective input at time tinduced by the incoming spike train from the presy-naptic neuron (also known as the synaptic drive; Pinto, Brumberg, Simons,

& Ermentrout, 1996).The term hL(, t) represents the inpu ts from the lateral

geniculate nucleus (LGN). The weight functions WLM are taken to be even


5/39


an d -periodic in so that they have the Fourier expansions

WLE() = WLE0 + 2

k=1WLEk cos(2k) 0 (2.4)

WLI() = WLI0 2

k=1WLI

k cos(2k) 0. (2.5)

Moreover,the interactionsare assumed to depend on the degree ofsimilarity

of the p resynaptic and postsynaptic orientation preferences, and to be of

maximum strength w hen they have the same preference. In order to make

a direct comparison with the results of Hansel and Sompolinsky (1997),

almost all our numerical results will include only the first two harmonics

in equations 2.4 and 2.5. Higher harmonics (WLMn , n 2) generate similar-looking tuning curves.

Finally, we take

YL(, t) =

0

d()fL(, t ), (2.6)

where () represents some delay distribution and fL(, t) is the output

spike train of a neuron of type L. Neglecting the p ulse shape of an indi-

vidual action potential, we represent the ou tput spike train as a sequence

of impulses,

fL( , t) =kZ

(t TkL()), (2.7)

where TkL

(), integer k, denotes the kth firing time (threshold-crossing time)

of the given neuron. The delay distribution () can incorporate a number

of possible sources of delay in neural systems: (1) discrete delays arising

from finite axonal transmission times, (2) synaptic processing delays asso-

ciated with the conversion of an incoming spike to a postsynap tic potential,

or (3) dend ritic processing delays in wh ich the effects of a postsynap tic po-

tential generated at a synap se located on th e dend ritic tree at some distance

from the soma are mediated by diffusion along the tree. For concreteness,

we shall restrict ou rselves to synaptic d elays and take (t) to be an alpha

function (Jack, N oble, & Tsien, 1975; Destexhe, Mainen , & Sejnow sky, 1994),

() = 2e(), (2.8)

where is the inverse rise time of a postsynaptic potential, and () = 1if 0 and is zero otherwise. We expect axonal delays to be small withina given hypercolumn . (For a review of the d ynamical effects of dendritic

structure, see Bressloff & Coombes, 1997.)


6/39


It is important to see how the above spiking version of the ring model is

related to the rate models considered by Hanseland Sompolinsky (1997)and

Ben-Yishai et al. (1997). Suppose the synaptic interactions are sufficiently

slow that the output fL( , t) ofa neuron can be characterized reasonably well

by a mean (time-averaged) firing rate (see, for example, Amit & Tsodyks,

1991; Bressloff & Coombes, 2000). Let us consider the case in which () is

given by the alpha function 2.8 with a synaptic rise time 1 significantlylonger than all other timescales in the system. The total synaptic input to

neuron of type L will then be d escribed by a slowly time-varying function

XL( , t), such that the actual firing rate will quickly relax to approximately

the stead y-state value, as determ ined b y equa tions 2.1 and 2.2. This implies

that

fL( , t) = f(XL(, t)), (2.9)with the steady-state firing-rate function f given by

f(X) =

ln

h0 +X

h0 +X 1

1(h0 +X 1). (2.10)

(For simplicity, we shall ignore the effects of refractory period, which is

reasonable when the system is operating well below its maximal firing rate.)

Equation 2.9 relates the dynamics of the firing rate directly to the stimulus

dynamics XL(, t) throu gh th e steady-state response fun ction. In effect, the

use of a slowly varying distribution () allows a consistent definition of

the firing rate so that a dynamical network model can be based on the

steady-state properties of an isolated neuron.

Substitution of equa tions 2.6 and 2.9 into 2.3 yields the extended Wilson-

Cowan equations (Wilson & Cowan , 1973):

XL(, t) =

M=E,I

0

d

WLM( )

0

d()f(XM(, t ))

+ hL(, t). (2.11)An alternative v ersion of equation 2.11 may be obtained for the alph a func-

tion delay distribution, 2.8, by rewriting equation 2.6 as the differential

equation

1

2

2YL

t2+ 2

YL

t+ YL = f(XL), (2.12)

with XL given by equation 2.3 and YL( , t), YL( , t)/ t 0 as t .Similarly, taking ( t)

=et generates the par ticular version ofth e Wilson-

Cowan equations studied by Hansel and Sompolinsky (1997),

1YL

t+ YL = f(XL). (2.13)


7/39


There are a number of differences, however, between the interpretation of

YL in equation 2.13 and the corresponding variable considered by H ansel

and Somp olinsky (1997). In the former case, YL(, t) is thes ynapticdriveof a

single neuron with orientation preference , and f is a time-averaged firing

rate, whereas in the latter case, YL represents the activity of a population of

neurons forming an orientation column , and f is some gain function. It

is possible to introduce the n otion of an orientation column in our model

by partitioning the d omain 0 < into N segments of length /N suchthat

mL(k, t) = N(k+1)/N

k/N

d

fL(, t), k= 0, 1, . . . ,N 1 (2.14)

represents the population-averaged firing rate within the kth orientationcolumn, k = k/N. (Alternatively, we could reinterpret the IF model as acaricature of a synchronized column of spiking neurons.)

Hansel and Sompolinsky (1997) and Ben-Yishai et al. (1995, 1997) have

carried out a detailed investigation of orientation tuning in the mean firing-

rate version of the ring m odel defined by equations 2.3 and 2.13 with f a

semilinear gain function. They consider external inputs of the form

hL(, t) = CL[1 + cos(2( 0(t)))] (2.15)

for 0 0.5. This represents a tuned input from the LGN due to avisual stimulus of orientation 0. The parameter C denotes the contrast

of the stimulus, L is the transfer function from the LGN to the cortex,

an d determines the angular anisotropy. In the case of a semilinear gain

function, f(x) = 0 if x 0, f(x) = x if x 0, and f(x) = 1 for x 1,it is possible to derive self-consistency equations for the activity profile

of the network. Solving these equations shows that in certain parameter

regimes,localcorticalfeedback can generatesharp orientation tuning curves

in which only a fraction ofn eurons are active. The associated activity profile

consists of a single peak centered about 0 (Hansel & Sompolinsky, 1997).

This activity peak can also lock to a rotat ing stimulu s 0 = tprovided that is not too large; if the inhibitory feedback is su fficiently strong, then it is

possible for spontaneous wave propagation to occur even in the absence of

a rotating stimulus (Ben-Yishai et al., 1997).

The idea that local cortical interactions play a central role in generating

sharp orientation tuning curves is still controversial. The classical model

of Hu bel and Wiesel (1962) proposes a very different m echanism, in w hich

the orientation p reference of a cell arises p rimarily from the geometrical

alignment of the receptive fields of the LGN neurons projecting to it. Anum ber of recent experiments show a significant correlation between the

alignment of receptive fields of LGN neu rons and the orientation prefer-

ence of simple cells functionally connected to them (Chapman, Zahs, &


8/39


Stryker, 1991; Reid & Alonso, 1995). In ad dition , Ferster, Chun g, and Wh eat

(1997) have show n th at cooling a p atch of cortex and therefore presumab ly

abolishing cortical feedback does not totally abolish the orientation tuning

exhibited by excitatory postsyna ptic poten tials (EPSPs) generated by LGN

input. H owever, there is also growing experimental evidence suggesting

the im portan ce of cortical feedback. For example, the blockage of extracel-

lular inhibition in cortex leads to considerably less sharp orientation tun ing

(Sillito, Kemp , Milson, & Berad i, 1980; Ferster & Koch, 1987; Nelson , Toth,

Seth, & Mu r, 1994). Moreover, intracellular m easuremen ts ind icate that di-

rect inputs from the LGN to neurons in layer 4 of the primary visual cortex

provide only a fraction of the total excitatory inputs relevant to orientation

selectivity (Pei, Vidyasagar, Volgushev,& Creutzseldt, 1994; Douglas, Koch,

Mahow ald, Martin, & Suarez, 1995; see also Somers et al.,1995). In add ition,

there is evidence that orientation tuning takes about 50 to 60 msec to reachits peak, and that the dynamics of tuning has a rather complex time course

(Ringach, Haw ken, & Shapley, 1997),su ggesting some cortical involvement.

The dynamical mechanism for sharp orientation tuning identified by

Hansel and Sompolinsky (1997) can be interpreted as a localized form of

spontaneous pattern formation (at least for strongly modulated cortical in-

teractions).For generalspatially distributed systems,pattern formation con-

cerns the loss of stability of a spatially uniform state through a bifurcation

to a sp atially periodic state (Murr ay, 1990). The latter may be stationary or

oscillatory (time periodic). Pattern formation in neural networks was first

studied in detail by Ermentrout and Cowan (1979a, 1979b). They showed

how competition between short-range excitation and long-range inhibition

in a two-dimensional Wilson-Cowan network can indu ce periodic striped

and hexagonal patterns of activity. These spontaneous patterns provided a

possible explanation for the generation of visual hallucinations. (See alsoCowan , 1982.)At first sight, the analysis of orientation tuning by Han sel and

Sompolinsky (1997) and Ben-Yishai et al. (1997) appears to involve a differ-

ent dynamical mechanism from this, since only a single peak of activity is

formed over the domain 0 rather than a spatially repeating pattern.This apparen t difference vanishes, however, once it is realized that spatially

periodic patterns would be generated by unraveling the ring. Thus, the

one-dimensional stationary and propagating activity p rofiles that Wilson

and Cowan (1973) found correspond, respectively, to stationary and time-

periodic patterns on the periodic ring. In the following sections, we stu dy

orientation tuning in both the an alog and IF models from the viewpoint of

spontaneous pattern formation.

3 Orientation Tuning in Analog Model

We first consider orientation tuning in the analog or rate model described

by equation 2.11 with the firing-rate function given by equation 2.10. This

may be viewed as the 0 limit of the IF model. We shall initially restrict


9/39


ourselves to the case of time-independent external inputs hL(). Given a -

independent input hL() = CL, the hom ogeneous equ ation 2.11 has at leastone spatially uniform solution XL( , t) = XL, where

XL =

M=E,IWLM0 f(XM) + CL. (3.1)

The local stability of the homogeneous state is found by linearizing equa-

tion 2.11 abou t XL. Setting xL( , t) = XL(, t) XL and expanding to firstorder in xL gives

xL(, t) =M=E,I

M

0

d

WLM( )

0

d()xM(, t ), (3.2)

where M = f(XM) an d f df/dX. Equation 3.2 has solutions of the form

xL(, t) = ZLete2in . (3.3)

For each n, the eigenvalues and corresponding eigenvectors Z = (ZE,ZI)trsatisfy the equation

Z = ()Wn Z, (3.4)where

()

0

e()d=

2

( + )2(3.5)

for the alpha function 2.8, and

Wn =

EWEEn IWEIn

EWIEn IWIIn

. (3.6)

The state XL will be stable if all eigenvalues have a negative real part. It

follows that the stability of the homogeneous state d epends on the partic-

ular choice of w eight parameters WLMn and the external inputs CL, which

determine the factors L. On the other hand, stability is independ ent of the

inverse rise time . (This independence will no longer hold for the IF

mod el; see section 4.)

In order to simplify matters further, let us consider a symmetric two-

popu lation mod el in which

CL = C, WEEn = WIEn WEn , WEIn = WIIn WIn . (3.7)


10/39


Equation 3.1 then has a solution XL = X, where X is the unique solution ofthe equation X = W0 f(X) + C. We shall assume that C is sup erthreshold,h0 + C > 1; otherwise X = Csuch that f(X) = 0. Moreover, L = f(X)for L = E,Iso that Wn = Wn . Equat ions 3.4 through 3.6 then red uce to theeigenvalue equation

+ 1

2= n (3.8)

for integer n, where n are the eigenvalues of the weight matrix Wn withcorresponding eigenvectors Zn :

+n = Wn WEn WIn , n = 0, Z+n =

1

1

, Zn =

1/WEn1/WIn

. (3.9)

It follows from equation 3.8 that X is stable with respect to excitation of

th e () modes Z0 , Zn cos(2k), Zn sin (2k). If Wn < 1 for all n then Xis also stable with respect to excitation of the corresponding (+) modes.This implies that the effect of the superthreshold LGN input hL() = Cis to switch the network from an inactive state to a stable homogeneous

active state. In this parameter regime, sharp orientation tuning curves can

be generated only if there is a sufficient degree of angular anisotropy in

the afferent inputs to the cortex from the LGN, that is, in equation 2.15

mu st be su fficiently large (Hansel & Sompolinsky, 1997). This is the classical

mod el of orientation tun ing (Hu bel & Wiesel, 1962).

An alternative mechanism for sharp orientation tuning can be obtained

by having strongly modulated cortical interactions such that Wn < 1 for

all n = 1 and W1 > 1. The homogeneous state then destabilizes due toexcitation of the first harmonic m odes Z+1 cos(2), Z

+1 sin (2). Since these

modes have a single maximum in the interval (0, ), we expect the network

to sup port an activity profile consisting of a solitary peak centered about

some angle 0, which is the same for both the excitatory and inhibitory

populations. For -independ ent external inpu ts ( = 0), the location of thiscenter is arbitrary, which reflects the und erlying translational invariance of

the n etwork. Following Ha nsel and Somp olinsky (1997), we call such an ac-

tivity profile a marginal state. The presence of a small, angu lar anisotropy in

theinputs (0 < 1 in equation 2.15) breaksthe translationalinvarianceofthe system and locks the location ofth e center to the orientation correspond-

ing to the peak of the stimulus (Hansel & Sompolinsky, 1997; Ben-Yishai et

al., 1997). In contra st to afferent mechan isms of sharp orientation tu ning,

can be arbitrarily small.

From the symmetries imposed b y equations 3.7, we can restrict ourselves

to the class of solutions XL(t)

=X(t), L

=E,I, such that equation 2.11 re-

duces to the effective one-population model

X(, t) =

0

d

W( )

0

d()f(X(, t )) + C, (3.10)


11/39


2 4 6 8

0.2

1

0.6W1

-W0

H

M

0.4

0.8

Figure 2: Phase bound ary in the (W0, W1)-plane (solid curve) between a stable

homogeneous state (H) and a stable marginal state (M) of the analog model for

C= 1. Also shown are data p oints from a direct num erical simulation of a sym-metric two-population network consisting of N = 100 neurons per populationand sinusoidal-like weight kernel W() = W0 + 2W1 cos(2). (a) Critical cou-pling for destabilization of homogen eous state (solid diam ond s). (b) Lower crit-

ical coupling for persistence of stable marginal state d ue t o hy steresis (crosses).

with W() = W0 + 2n=1 Wn cos(2n) an d Wn fixed such that Wn < 1for all n = 1. Treating W1 as a bifurcation parameter, the critical couplingat which the marginal state becomes excited is given by W1 = W1c, whereW1c = 1/f(X) with X dependent on W0 an d C. This generates a phaseboundary curve in the (W0, W1)-plane that separates a stable homoge-

neous state and a marginal state. (See the solid curve in Figure 2.) When

the p hase bound ary is crossed from below, the network jump s to a stable

marginal state consisting of a sharp orientation tuning curve whose w idth

(equal to /2) is invariant across the parameter d omain over w hich it ex-

ists. This reflects the fact that the underlying pattern-forming mechanism

selects out the first harmonic modes. On the other hand, the height and

shape of the tuning curve will depend on the particular choice of weight

coefficients Wn as well as other parameters, such as the contrast C. Note

that sufficiently far from the bifurcation point, additional harmonic modes

will be excited, which may modify the width of the tuning curve, lead-ing to a dependence on the weights Wn . Indeed, in the case of a semilin-

ear firing-rate function, the width of the sharp tuning curve is found to

be dependent on the weights Wn (Hansel & Somp olinsky, 1997), reflect-


12/39


ing the absence of an additional nonlinearity selecting out the fundamental

harmonic.

Examples of typical tuning curves are illustrated in Figure 3a, wh ere we

show results ofa direct numericalsimulation ofa symmetrictwo-population

analog network. The steady-state firing rate f() of the marginal state is

plotted as a fun ction of orientation preference in the case ofa cosine weight

kernel W() = W0+2W1 cos(2). It can be seen that the height of the activityprofile increases with the w eight W1. Note, however, that ifW1 becomes too

large, then the marginal state is itself destroyed by amplitude instabilities.

The height also increases with contrast C. Consistent with the results of

Han sel and Somp olinsky (1997) and Ben-Yishai et al. (1997), the center of

the activity p eak can lock to a slowly rotating, weakly tuned stimulus as

illustrated in Figure 3b. Also shown in Figure 3a is the tuning curve for a

Mexican hat weight kernel W() = A 1e2

/221 A 2e

2

/22

2 over the range/2 /2. The constan ts i,A i are chosen so that the correspondingFourier coefficients satisfy W1 > 1 and Wn < 1 for all n = 1. The shapeof the tuning curve is very similar to that obtained using the simple cosine.

To reproduce more closely biological tuning curves, which have a sharper

peak and broader shoulders than obtained here, it is necessary to introduce

some noise into the system (Dimitrov & Cowan, 1998). This smooths the

firing r ate fun ction, 2.10, to yield a sigmoid-like nonlinearity.

Interestingly, the m arginal state is foun d to exhibit hysteresis in the sense

that sharp orientation tuning persists for a range of values of W1 < W1c.

That is, over a certain param eter regime, a stable hom ogeneous state and

a stable marginal state coexist (see data points in Figure 2). This suggests

that the bifurcation is subcritical, which is indeed found to be the case, as

we now explain. In standard treatments of pattern formation, bifurcation

theory is u sed to derive nonlinear ordinary differential equations for theamplitude of the excited modes from which existence and stability can be

determ ined, at least close to the bifurcation point (Cross &H ohenberg, 1993;

Ermentrout, 1998).Suppose that the firing-rate function f is expanded about

the (nonzero) fixed point X,

f(X) f(X) = (X X) + g2(X X)2 + g3(X X)3 + , (3.11)

where = f(X), g2 = f(X)/2, g3 = f(X)/6. Introduce the small param-eter according to W1 W1c = 2 and substitute into equation 3.10 theperturbation expansion,

X(, t) X = Z(t)e2in +Z(t)e2in+O(2), (3.12)where Z(t),Z(t) denote a complex conjugate pair ofO() amplitudes.


13/39


W1 = 1.2

W1 = 1.5W1 = 1.8

orientation (in units/100)

f()

0

1

2

3

4

0 20 40 60 80 100

(a)

0

0.5

1

1.5

2

2.5

0 20 40 60 80 100

f()

(b)

Figure 3: Sharp orientation tuning curves for a symm etric two-pop ulation ana-

log network (N = 100) with cosine weight kernel W() = W0 + 2W1 cos(2).(a) The firing rate f() is plotted as a function of orientation preference for

various W1 with W0 = 0.1, C= 0.25 and = 0.5. Also show n (solid curve) isthe corresponding tuning curve for a Mexican hat w eight kernel with 1 = 20degrees, 2 = 60 degrees, and A i = A /2

2i . (b) Locking of tuning curve to

a slowly rotating, weakly tuned external stimulus. Here W0 = 0.1, W1 = 1.5,C = 0.25, = 1.0. The degree of angular anisotropy in the input is = 0.1,and the angular frequency of rotation is = 0.0017 (which corresponds toapp roximately 10 degrees per second ).


14/39


A standard perturbation calculation (see appendix A and Ermentrout,

1998) shows that Z(t) evolves according to an equation of the form

dZ

dt= Z

W1 W1c +A |Z|2

, (3.13)

where

A = 3g32

+ 2g22

2

W2

1 W2+ 2W0

1 W0

. (3.14)

Since W1 an d A are all real, the phase ofZ is arbitrary (reflecting a ma rginal

state), whereas the amp litude is given by |Z| = |W1 W1c|/A . It is clearthat a stable marginal state will bifurcate from the homogeneous state ifand only if A < 0. A quick numerical calculation shows that A > 0 for

all W0, W2, and C. This implies that the bifurcation is subcritical, that is,

the resting state bifurcates to an unstable broadly tuned orientation curve

whose amplitude isO(). Since the bifurcating tuning curve is unstable, it is

not observed in pr actice; rather, one find s that the system jum ps to a stable,

large-amplitude, sharp tuning curve that coexists with the resting state and

leads to h ysteresis effects as illustrated schematically in Figure 4a. Note th at

in the case of sigmoidal firing-rate functions f(X), the bifurcation may be

either sub- or supercritical, depend ing on the relative strengths of recurren t

excitation and lateral inhibition (Ermentrout & Cowan, 1979b). If the latter

is sufficiently strong, then supercritical bifurcations can occur in w hich the

resting state changes smoothly into a sharply tuned state. However, in the

case studied h ere, the nonlinear firing-rate function f(X) of equation 2.10 is

such tha t the bifurcation is almost alw ays subcritical.The und erlying translational invariance of the network m eans that the

center of the tuning curve remains undetermined for a -independent LGN

input (marginal stability). The center can be selected by including a small

degree of angular anisotropy along the lines of equation 2.15. Suppose, in

particular, that we replace C on the right-hand side of equation 3.10 by an

LGN input of the form h(, t) = C[1 + cos(2[ t])] for some slowfrequency . If we take = 3, then the amplitude equation 3.13 becomes

dZ

dt= Z

W1 W1c +A |Z|2

+ Ce2it. (3.15)

Writing Z = re2i(+t) (with the phase defined in a rotating frame), weobtain the pair of equations:

r= r(W1 W1c + A r2) + Ccos 2 (3.16)

= C2r

sin (2 ). (3.17)


15/39


|Z |

H

M+

M-

W1c

d

0

|Z |

W10

(a)

(b)

Figure 4: (a) Schematic diagram illustrating hysteresis effects in the analog

model. As the coupling parameter W1 is slowly increased from zero, a critical

point c is reached where the homogeneous resting state (H) becomes unstable,

and there is a subcriticalb ifurcation to an unstable, broadly tuned m arginal state

(M). Beyond the bifurcation point, the system jumps t o a stable, sharp ly tunedmarginal state (M+), which coexists with the other solutions. If the couplingW1 is now slowly reduced, the marginal state M+ persists beyond the originalpoint c so that there is a range of values W1(d) < W1 < W1(c) over which a

stable resting state coexists w ith a stable marginal state (bistability). At p oint d,

the states M

annihilate each other, and the system jumps back to the resting

state. (b) Illustration of an imperfect bifurcation in the presence of a small de-

gree of anisotropy in the LGN inp ut. Thick solid (dash ed) lines represent stable

(unstable) states.


16/39


Thus, provided that is sufficiently sm all, equation 3.17 will have a stable

fixed-point solution in which the phase of the pattern is entrained to th e

signal.Another interesting consequence ofth e anisotropy is that it generates

an imp erfect bifurcation for the (real) amplitud e rdue to the presence of the

r-independ ent term = Ccos(2 ) on the right-hand side of equation 3.16.This is illustrated in Figure 4b.

4 Orientation Tuning in IF Model

Our analysis ofth e analog mod el in section 3ignored an y details concerning

neural spike trains by taking the ou tput of a neuron to be an average firing

rate. We now return to the more realistic IF model of equ ations 2.1 through

2.3 in which the firing times of individual spikes are specified. We wish to

identify the analogous mechanism for orientation tuning in the model withspike coding.

4.1 Existence and Stability of Synchronous State. The first step is to

specify what is meant by a homogeneous activity state. We define a phase-

locked solution to be one in which every oscillator resets or fires with the

same collective period Tthat mu st be determined self-consistently. The state

of each oscillator can then be characterized by a constant phase L() with

0 L() < 1and 0 < .The corresponding firingtimes satisfy TkL() =(kL())T,integer k.Integratingequation 2.1between two successivefiringevents and incorpor ating the reset cond ition 2.2lead s to the phase equations

1 = (1 eT)(h0 + CL)

+ M=E,I

0

d

WLM( )KT(M() L()), L = E,I (4.1)

where WLM() is given by equations 2.4 and 2.5, and

KT() = eTT

0

dt etkZ

( t+ (k+ )T). (4.2)

Note that for any phase-locked solution of equation 4.1, the distribution of

network output activity across the network, as specified by the ISIs

kL() = Tk+1L () TkL(), (4.3)

is homogeneou s since kL() = Tfor all k Z, 0 < ,L = (E,I). In otherword s, each phase-locked state plays an analogous role to the homogeneou sstate XL of the mean firing-rate mod el. To simplify our analysis, we shall

concentrate on instabilities of the synchronous state L() = , where isan arbitrary constant. In order to ensure such a solution exists, we impose


17/39


1

2

3

4

5

6

7

8

T

-W0

C = 1

C = 0.5 = 0.5

= 2

Figure 5: Variation of collective period T of synchronous state as a function of

W0 for d ifferent contrasts Cand inverse rise times .

the conditions 3.7 so that equation 4.1 reduces to a single self-consistency

equation for the collective period T of the synchronous solution

1 = (1 eT)C0 + W0KT(0), (4.4)where we have set C0 = h0 +Can d W0 = WE0 WI0. Solutions ofequ ation 4.4are plotted in Figure 5.

The linear stability of the synchronous state can be determined by con-

sidering perturbations of the firing times (van Vreeswijk, 1996; Gerstner,

van Hem men , & Cow an, 1996; Bressloff & Coombe s, 1998a, 1998b, 2000):

TkL() = (k )T+ kL(). (4.5)Integrating equ ation 2.1 between tw o successive firing events yields a non-

linear map ping for the firing times that can be expanded as a p ower series

in the p erturbations kL(). This generates a linear d ifference equa tion of the

form

A T

k+1

L() kL()

=

l=0

GT(l)

0

d

M=E,I

WLM( )

klM

() kL()

(4.6)

with

GT(l) = eTT

0

dt et (t+ lT) (4.7)


18/39


A T = C0 1 + W0kZ

(kT). (4.8)

Equation 4.6 has solutions of the form

kL() = ek e2in L, L = E,I, (4.9)

where C and 0 Im < 2 . The eigenvalues and associated eigen-vectors Z = (E, I)tr for each integer n satisfy the equation

A T[e 1]Z =

GT()Wn GT(0)DZ, (4.10)where

GT() = k=0

GT(k)ek (4.11)

an d

Wn =

WEEn WEInWIEn WIIn

, D =

WEE0 WEI0 0

0 WIE0 WII0

. (4.12)

In order to simplify the subsequent analysis, we imp ose the same sym-

metry conditions 3.7 as used in the study of the analog model in section 3.

We can then diagona lize equation 4.10 to obtain the result

A T[e

1] = GT()n GT(0)W0, (4.13)where n are the eigenvalues of the weight m atrix Wn (see equation 3.9).Note that one solution of equation 4.13 is = 0 and n = 0 associated withexcitation of the uniform (+) mode. This reflects invariance of the systemwith respect to u niform p hase shifts in th e firing times. The synchronous

state will be stable if all other solutions of equation 4.13 satisfy Re < 0.

Following Bressloff and Coombes (2000), we investigate stability by first

looking at the weak coupling regime in which Wn = O() for some 1.Performing a perturbation expansion in , it can be established that the

stability of the synchronou s state is determined by the nonzero eigenvalues

in a small neighborhood of the origin, and these satisfy the equation (to

first order in ) A T = (n W0)

GT(0). For the alpha function 2.8, it can

be established that A T > 0, whereas

GT(0) KT(0)/T < 0. Therefore,

the synchronous state is stable in the w eak coupling regime p rovided thatWn > W0 for all n = 0 and W0 < 0. Assuming that these two conditionsare satisfied, we n ow investigate what h appens as the strength of coupling

W1 is increased for fixed Wn , n = 1. First, it is easy to establish that the


19/39


synchronous state cannot destabilize due to a real eigenvalue crossing th e

origin. In particular, it is stable with respect to excitations of the () modes.Thus, any instability will involve one or more complex conjugate pairs of

eigenvalues = i crossing th e imagina ry a xis, signaling (for = ) theonset of a d iscrete Hop f bifurcation (or N eimark-Sacker bifurcation) of the

firing times due to excitation of a (+) mode kL() = ei0kei2 , L = E,I. Inthe special case = , this reduces to a subharmonic or p eriod doublingbifurcation since ei = 1.

4.2 Des ynchronization Leading to Sharp Orientation Tuning. In order

to investigate the occurrence of a Hopf (or period-doubling) bifurcation in

the firing times, substitute = i into equation 4.13 for n = 1 with +1 = W1and equate real and imaginary parts to obtain the pair of equations

A T[cos() 1] = W1C() W0C(0), A T sin () = W1S(), (4.14)where

C() = Re GT(i), S() = Im GT(i). (4.15)Explicit expressions for C() an d S() in the case of the alpha functiondelay distribution 2.8 are given in appendix B. Suppose that , W0, and

C0 are fixed, with T = T(, W0, C0) the unique self-consistent solution ofequation 4.4 (see Figure 5). The smallest value of the coupling parameter,

W1 = W1c, for which a nonzero solution, = 0 = 0, of the simultaneousequations 4.14 is then sought. This generates a phase boundary curve in

th e (W0, W1) plane, as illustrated in Figure 6 for various C an d . Phase

boundaries in the (, W1) plane and the (C, W1) plane are show n in Figures 7and 8, respectively.

The cusps of the bound ary curves in these figures correspond to points

wh ere two separate solution bra nches of equation 4.14 cross; only the lower

branch is shown since this determines the critical coupling for destabiliza-

tion of the synchronous state. The region below a given boundary curve

includes the origin. Weak coupling theory shows that in a neighborhood of

the origin, the synchronous state is stable. Therefore, the boun dary curve

is a locus of bifurcation points separating a stable homogeneous state from

a marginal non-phase-locked state. It should also be noted that for each

boundary curve in Figures 6 and 7, the left-hand branch signals the onset

of a (subcritical) Hopf bifurcation (0 = ), whereas the right-hand branchsignals the onset of a (subcritical) period dou bling bifurcation (0 = );theopp osite holds tru e in Figure 8. However, there is no qu alitative difference

in the observed behavior induced by these two types of instability.Direct numerical simulations confirm that when a phase bound ary curve

is crossed from below, the IF networ k jump s to a stable margina l state con-

sisting of a sharp orientation tuning curve as determined by the spatial


20/39


21/39


1

2

3

4

5

W0 = - 0.5

W0 = - 2W0 = - 0.1

C = 0.25

W1

Figure 7: Phase boundary in the (, W1) plane between a stable homogeneous

state and a marginal state ofa symmetrictwo-population IFnetwork. Bound ary

curves are shown for C= 0.25 and various values ofW0.

agreement between the two models only when (1) there is an ap proximate

balance between the mean excitatory and inhibitory coupling (W0 0) and(2) synap tic interactions are sufficiently slow ( 1). Both conditions holdin Figures 9. Outside this parameter regime, the higher value of W1c for the

IF model leads to relatively high levels of mean firing rates in the activity

profile of the marginal state, since large values of W1 imply strong short-range excitation. The condition W0 0 can be und erstood in terms of theself-consistency condition for the collective period T of the synchronous

state given by equation 4.4. As W0 becomes more negative, the period T

increases (see Figure 5) so that T > 1 and the reduction to the analog

model is no longer a good approximation. (In Bressloff & Coombes, 2000,

the p eriod T is kept fixed by varying the external input.)

4.3 Quasiperiodicity and Interspike Interval Variability. We now ex-

plore in m ore detail the n ature of the spatiotemporal d ynamics of the ISIs

occurring in the marginal state of the IF model. For concreteness, we con-

sider a symm etric two-population network w ith Wn = 0 for all n 2. Weshall proceed by fixing the parameters W0, W1, and C such that a sharp

orientation tuning curve exists and considering what happens as is in-

creased. In Figure 10 we plot the ISI pairs (

n

1

(),

n

()), integer n, forall the excitatory neurons in the ring. It can be seen from Figure 10a th at

for relatively slow synapses ( = 2), the temporal fluctuations in the ISIsare negligible. There exists a set of spatially separat ed p oints reflecting the


22/39


0.2 0.4 0.6 0.8 1

1

2

3

4 = 8 = 0.5 = 0.25

W1

C

= 20

W0 = -0.1

Figure 8: Phase boundary in the (C, W1) plane between a stable synchronous

state and a marginal state ofa symm etric two-population IF network. Bound ary

curves are shown for W0 = 0.1 and various values of the inverse rise time .The solid curve corresponds to the analog m odel, whereas the d ashed curves

correspond to the IF model.

-dependent variations in the mean firing rates a(), as shown in Figure 9.

However, as is increased, the system bifurcates to a state w ith period ic or

quasiperiodic fluctuations of the ISIs on spatially separated invariant cir-

cles (see Figure 10b). Althou gh the av erage beh avior is still consistent w ith

that found in the corresponding analog model, the additional fine structureassociated w ith these orbits is not resolved by the analog model. In order

to characterize the size of the ISI fluctuations, we define a deterministic

coefficient of variation CV() for a neuron according to

CV() =

() ()2

(), (4.17)

with averages defined by equation 4.16.

The CV() is plotted as a function of for various valuesof in Figure 11a.

This shows that the relative size of the deterministic fluctuations in the

mean firing rate is an increasing function of . For slow synapses ( 0),th e CV is very small, indicating an excellent match between the IF and

analog mod els. How ever, the fluctuations become much more significantwhen the synapses are fast. This is further illustrated in Figure 11b, where

we plot the variance of th e ISIs against the m ean. The variance increases

monotonically w ith the m ean, which shows w hy group s of neurons close


23/39


a()

0

1

2

3

4

0 20 40 60 80 100

W1 = 1.2W1 = 1.5

W1 = 1.8

0

0.5

1

1.5

2

2.5

0 20 40 60 80 100

a()

Figure 9: Sharp orientation tuning curves for a symmetric two-popu lation IF

network (N = 100) and cosine weight kernel. (a) The mean firing rate a() isplotted as a fun ction of orientation preference and various W1.Sameparameter

values as Figure 3a. (b) Locking to a slowly rotating, w eakly tuned external

stimulus. Same parameters as Figure 3b.

to the edge of the activity profile in Figure 11a (so that the mean firing rate

is relatively small) have a larger CV . An interpretation of this behavior is

that neurons with a low firing rate are close to threshold (i.e., there is an

approximate balance between excitation and inhibition), which means that

these neurons are more sensitive to fluctuations. Another observation fromFigure 11b is that the flu ctuations decrease with the size of the network; this

appears to be a finite-size effect since there is only a weak dependence on

N for N > 200.


24/39


0.2 0.4 0.6 0.8 1 1.2 1.4 1.6

n-1()

(b)

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

0.2 0.4 0.6 0.8 1 1.2 1.4 1.6

n()

n-1()

(a)

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

n()

Figure 10: Separation of the ISI orbits for a symmetric two-population IF net-

work (N = 100) with W0 = 0.5, W1 = 2.0, C = 0.25. (a) = 2. (b) = 5.The (projected) attractors of the ISI pairs with coordinates (n1(), n ()) areshown for all excitatory neurons.

In Figure 12 we plot (n1(), n ()) for two selected neu rons: one witha low CV/ fast firing rate and the other with a high CV/ low firing rate. Re-

sults are shown for = 8 and = 20. Corresponding ISI histograms arepresented in Figure 13. These figures establish that for fast synapses, the ISIs

display highly irregular orbits with the invariant circles of Figure 10b no

longer p resent. An interesting qu estion that we h ope to pursue elsewhereconcerns whether the underlying attractor for the ISIs supports chaotic dy-

namics, for it is well known that the breakup of invariant circles is one

possible route to chaos, as has previously been demonstrated in a num-


25/39


CV()

= 8

= 14

= 20

0

0.2

0.4

0.6

0.8

1

1.2

1.4

0 20 40 60 80 100

0

0.5

1

1.5

2

2.5

3

0.2 0.6 1 1.4 1.8

(20,200)

(20,50)

(20,100)

(14,100)

(8,100)

variance

mean

(b)

(a)

+

Figure 11: (a) Plot of the coefficient of variation CV() as a function of for

various values of the inverse rise time . (b) Plot of variance against the mean of

the ISIs for various values of inverse rise tim e and n etwork size N. All other

param eter values are as in Figure 10.

ber of classical fluid d ynam ics experimen ts (Berge, Du bois, & Vidal, 1986).

As partial evidence for chaotic ISI dynamics, we plot the power spectrum

|h(p)|2 of a neuron with high CV in Figure 14. We consid er a seq uen ce of ISIsover M + 1 firing events {k, k= 1, . . .M} and define

h(p) =1

M

Mk=1

e2ikpk, (4.18)

where p = m/M for m = 1, . . . ,M. Although the nu merical data are rather


26/39


0.2

0.6

1

1.4

1.8

0.2 0.6 1 1.4 1.8

high CV

low CV

n-1()

n()

0.2

0.6

1

1.4

1.8

0.2 0.6 1 1.4 1.8

high CV

low CV

n()

n-1()

(a)

(b)

Figure 12: Same as Figure 10 except th at (a) = 8 and (b) = 20. The attractoris shown for two excitatory neurons: one with a low CV and the other with a

high CV.

noisy, it does appear that th ere is a major difference between the qu asiperi-

odic regime shown in Figure 10 and the high-CV regime of Figure 12. The

latter possesses a broadband spectrum, which is indicative of (but not con-

clusive evidence for) chaos.

There is currently considerable interest in possible mechanisms for the

generation of high CVs in networks of cortical neu rons. This follows therecent observation by Softky and Koch (1993) of high variability of the ISIs

in data from cat V1 and MT neurons. Such variability is inconsistent with

the standard notion that a neu ron integrates over a large number of small


27/39


0.28 0.285 0.29 0.295 0.30

10

20

30

40

N

ISI

(a)

0.5 1 1.5 2 2.50

20

40

60

80

(b)

ISI

0.28 0.32 0.360

10

20

30

40

N

(c)

0

50

100

150

200250

300(d)

0 2 4 6 8 10

ISI ISI

Figure 13: Histograms of 1000 firing events for selected excitatory neu rons in a

symmetric two-population IF network: (a) = 8, low CV. (b) = 8, high CV.(c) = 20, low CV. (d) = 20, high CV. All other parameter values are as inFigure 10.

(excitatory) inputs, since this w ould lead to a regular firing pattern (as a

consequence of the law of large nu mbers). Two alternative ap proaches to

resolving this dilemm a have been prop osed. The first treats cortical neuron s

as coincidence detectors that fire in respon se to a relatively small num ber of

excitatory inputs arriving simultaneously at the (sub)millisecond levela

suggested m echanism for the amp lification of a neurons response is active

dendrites (Softky & Koch, 1993). The second approach retains the picture

of a neuron as a synaptic integrator by incorporating a large number of

inhibitory inputs to balance the effects of excitation so that a neuron oper-

ates close to threshold (Usher et al., 1994; Tsodyks & Sejnowski, 1995; van

Vreeswijk & Somp olinsky, 1996, 1998). These latter stud ies incorporate a de-

gree of randomness into a network either in the form of quenched disorder

in the coupling or through synaptic noise (see also Lin, Pawelzik, Ernst, &Sejnowski, 1998). We have shown that even an ordered network evolving

determ inistically can sup port large fluctua tions in the ISIs provided th at the

synapses are sufficiently fast. For the activity profile shown in Figure 11a,


28/39


0.1 0.2 0.3 0.4 0.510-1

10

100

1

frequency p

|h(p)|2

10-6

10-4

10-2

1

0.1 0.2 0.3 0.4 0.5frequency p

(b)(b)

(a)

|h(p)|2

Figure 14: (a)Spectrum of a neuron in the quasiperiodicregime( = 5).(b)Spec-trum of a high-CV neuron with = 20. All other parameter values are as inFigure 10.

only a subset of neurons have a CV close to u nity. In order to achieve a con-

sistently high CV across the w hole network, it is likely that one wou ld need

to take into account various sources of disorder present in real biological

networks.

5 Intrins ic Traveli ng Wave Profiles

So far w e have restricted our analysis of orientation tu ning to the case of

a symmetric two-population model in which the excitatory and inhibitorypopulations behave identically. Sharp orientation tuning can still occur if

the symmetry conditions 3.8 are no longer imposed, provided that the first

harmonic eigenmodes are excited w hen th e homogeneous resting state is


29/39


destabilized (n = 1 in equation 3.3 or 4.9). Now, however, the heights ofthe excitatory and inhibitory activity p rofiles will generally d iffer. More

significant, it is possible for traveling wave profiles to occur even in the

absence of a rotating external stimulus (Ben Yishai et al., 1997). In order

to illustrate this phenomenon, suppose that equation 3.7 is replaced by the

conditions

CL = C, W0 =

WE0 WI0WE0 WI0

, W1 =

W KK W

(5.1)

with K > W an d WLMn = 0 for all n 2. First consider the analog modelanalyzed in section 3. Under th e conditions 5.1, the fixed-point equ ation 3.1

has solutions XL = X,L = E,I, where X = W0 f(X)+ Cwith W0 = WE

0 WI

0.Linearization about this fixed point again yields the eigenvalue equation 3.8,

with th e eigenvalues of the w eight m atrices Wn of equation 5.1 now given

by

+0

= W0, 0 = 0, 1 = iW1, W1

K2 W2. (5.2)

Equations 3.8 and 5.2 imply that the fixed point X is stable with respect to

excitation of the n = 0 modes provided that W0 < 1. On the other hand,substituting for 2 using equ ation 5.2 shows th at there exist solutions for of the form

= 1 +

W1

2(1 i). (5.3)

Hence, at a critical coupling W1 = W1c 2/, the fixed point un dergoes aHopf bifurcation du e to the occurrence ofa p air ofp ure imaginary eigenval-

u es = i. The excited mod es take the form Zei(2t), where Z are theeigenvectors associated w ith 2 and correspond to traveling wave activityprofiles with rotation frequency .

Let us now turn to the case of the IF mod el analyzed in section 4. Linear

stability an alysis of the synchronous state generates the eigenvalue equa-

tion 4.13 with n given by equation 5.2. Following similar argum ents tosection 3, it can be established that the synchronous state is stable in the

weak coupling regime provided that W0 < 0. It is also stable with respect

to excitations of the n = 0 modes for arbitrary coupling strength . Therefore,we look for strong coup ling instabilities of the synchronous state by su bsti-

tuting = i in equation 4.13 for n = 2. Equating real and imaginary partsthen leads to the pair of equations

A T[cos() 1] = W1S() W0C(0), A T sin () = W1C(). (5.4)


30/39


1 2 3 4 5 6

1

2

3

4

5

6

7

8

= 0.5

= 2.0

= 0.1

W1

-W0

Figure 15: Phase boundary in the (W0, W1) plane between a stable synchronous

state and a traveling marginal state of an asymmetric two-population IF net-

work. Boun dary curves are show n for C= 0.25an d various values of the inverserise time . For each there are two solution branches, the lower one of which

determines the point ofinstability of the synchronous state.The bound ary curve

for the analog m odel is also shown (thick line).

Examples of boundary curves are shown in Figure 15 for W1 as a function

of W0 with fixed. For each , there are tw o solution branches, the lowerone of which determines the point at which the synchronous state desta-

bilizes. At first sight it would appear that there is a discrepancy between

the IF and analog models, for the lower bound ary curves of the IF model

do not coincide with the phase boundary of the analog model even in the

limit 0 and W0 0. (Contrast Figure 15 with Figure 8, for exam-ple.) However, numerical simulations reveal that as the lower boundary

curve of the IF model is crossed from below, the synchronous state desta-

bilizes to a state consisting of two distinct synchronized populationsone

excitatory and the other inhibitory. The latter state itself d estabilizes for

values ofW1 beyond the upper boundary curve, leading to the formation

of an intrinsically rotating orientation tun ing curve. In Figure 16 we p lot

the spike train of one of the IF neurons associated with such a state. The

neuron clearly exhibits approximately periodic bursts at an angular fre-

quency /60, where the time constant 0 is the fundamental unit oftime. This implies that orientation selectivity can shift over a few tens ofmilliseconds, an effect that has recently been observed by Ringach et al.

(1997).


31/39


100 102 104 106 108 110 112 114 116 118 120

time (in units of 0)

Figure 16: Oscillating spike train ofan excitatory neuron in a traveling marginal

state of an asymmetric two-population IF network. Here W0=

1.0, W1=

4.2,

WEE0 = 42.5, WEE1 = 20.0 = J, N = 50, C= 1.0, and = 2.0.

6 Discussion

Most analytical treatments of network dynamics in compu tational neuro-

science are based on analog mod els in which the outp ut of a neuron is taken

to be a mean firing rate (interpreted in terms of either population or time

averaging). Techniques such as linear stability analysis and bifurcation th e-

ory are used to investigate strong coupling instabilities in these networks,

which induce transitions to states with complex spatiotemp oral activity pat-

terns (see Ermentrout, 1998, for a review). Often a numerical comparison

is made w ith the behavior of a corresponding network of spiking neurons

based on t he IF model or, perha ps, a more d etailed biophysical model suchas H odgkin-Hu xley. Recently Bressloff and Coombes (2000) developed an-

alytical techniques for stud ying strong cou pling instabilities in IF networks

allowing a more direct comparison w ith analog networks to be mad e. We

have applied this dynamical theory of strongly coupled spiking neurons to

a simple computational model of sharp orientation tuning in a cortical hy-

percolumn. A num ber ofsp ecific results were obtained that raise interesting

issues for further consideration.

Just as in the an alog model, bifurcation theory can be u sed to studyorientation tu ning in th e IF mod el. The main results are similar in that

the coefficients of the interaction function W() = W0 + 2W1 cos(2)play a key role in determining orientation tuning. If the cortical in-

teraction is weakboth W0 an d W1 smallthen, as may be expected,

only biases in the geniculocortical map can give rise to orientationtuning. However, ifW1 is sufficiently large, then only a weak genicu-

locortical bias is needed to p rodu ce sharp orientation tu ning at a given

orientation.


32/39


Such a tuning is produced via a subcritical bifurcation, which im-plies that the switching on and off of cortical cells should exhibit

hysteresis (Wilson & Cowan, 1972, 1973). Thus, both tuned and un-

tuned states could coexist under certain circumstances, in both the

analog and the IF model. It is of interest that contrast and orientation-

dependent hysteresis has been observed in simple cells by Bonds

(1991).

Strong coupling instabilities in the IF model involve a discrete Hopfor period doubling bifurcation in the firing times. This typically leads

to non-phase-locked behavior characterized by clustered but irreg-

ular variations of the ISIs. These clusters are spatially separated in

pha se-space, which results in a localized activity pat tern for the m eanfiring rates across the network that is modulated by fluctuations in

the instantaneous firing rate. The latter can generate CVs of order

unity. Moreover, for fast synapses, the underlying attractor for the

ISIs as illustrated in Figure 12 appears to support chaotic dynam-

ics. Evidence for chaos in disordered (rather than ordered) balanced

networks has been presented elsewhere (van Vreeswijk & Sompolin-

sky, 1998), although caution has to be taken in ascribing chaotic be-

havior to potentially h igh-dimensional dynam ical systems. Another

interesting question is to what extent the modulations of mean fir-

ing rate observed in the responses of some cat cortical neurons to

visual stimuli by Gray and Singer (1989) can be explained by dis-

crete bifurcations of the fi ring tim es in a netw ork of s piking

neurons.

In cases where the excitatory and inhibitory populations comprisingthe ring are not ident ical, traveling wav e profiles can occur in response

to a fixed stimulus. This implies that peak orientation selectivity can

shift over a few tens of milliseconds. Such an effect has been observed

by Ringach et al. (1997) and again occurs in both the an alog and IF

models.

Certain care has to be taken in identifying parameter regimes overwhich p articular forms of network operation occur, since phase dia-

grams constructed for ana log networks d iffer considerably from those

of IF networks (see Figure 8). For example, good quantitative agree-

ment betw een the localized activity profiles of the IF and analog mod -

els holds only when there is an approximate balance in the mean

excitatory and inhibitory coupling and the synap tic interactions are

sufficiently slow.


33/39


Appendix A

Let X( , t) = X be a homogeneous fixed point of the one-population modeldescribed by equation 3.9, which we rewrite in the more convenient form:

1

2

2X( , t)

t2+ 2

X(, t)

t+X(, t)

=

0

d

W( )f(X(, t)) + C. (A.1)

Supp ose that the weight distribution W() is given by

W() = W0 + 2

k=1Wk cos(2k). (A.2)

Expand th e nonlinear firing-rate fun ction about the fixed point X as in equa-

tion 3.11. Define the linear op erator

LX = 12

2X

t2+ 2

X

t+X W X (A.3)

in terms of the convolution WX() = 0 W( )X()d/ . The opera-to r L has a discrete spectrum with eigenvalues satisfying (cf. equation 3.7)

1 +

2 = Wk (A.4)and corresponding eigenfunctions

X(, t) = zk e2ik +zk e2ik . (A.5)

Suppose that Wn for some n = 0 is chosen as a bifurcation parameteran d Wk < 1 for all k = n. Then Wn = Wn,c 1 is a bifurcation pointfor destabilization of the homogeneous state X due to excitation of the nth

mod es. Set Wn Wnc = 2, Wc() = W()|Wn =Wn,c , and perform the follow-ing p erturbation expansion:

X = X + X1 + 2X2 + (A.6)

Introduce a slow timescale = 2t and collect terms w ith equal pow ers of


34/39


. This leads to a hierarchy of equations of the form

O(1): X = W0 f(X) + C (A.7)

O(): LcX1 = 0 (A.8)

O(2): LcX2 = g2Wc X21 (A.9)

O(3):

LcX3

=g3Wc

X31

+2g2Wc

X1X2

2

X1

fn

X1 , (A.10)

where Lc = XWcX an d fn () = cos(2n).TheO(1) equation determinesthe fixed point X. The O() equation has solutions of the form

X1(, t) = z( ) e2in +z( ) e2in . (A.11)

We shall determine a dynamical equation for the complex amplitude z( )

by deriving so-called solvability conditions for the h igher-order equat ions.

We proceed by taking the inner product of equations A.9 and A.10 with

the linear eigenmode A.11.We define the inner produ ct oftw o periodic func-

tions U, V according to U|V = 0

U()V()d/ . The O(2) solvabilitycondition X1| LcX2 = 0 is autom atically satisfied since

X1|Wc X21 = 0. (A.12)

The O(3) solvability condition X1| LcX3 = 0 gives

X1| 2

X1

Wc X1 = g3X1|Wc X31 + 2g2X1|Wc X1X2. (A.13)

First, we have

e2in | 2

X1

fn X1 =

2

dz

d z (A.14)

an d

e2in |Wc X31 = Wn,c 0

dze2in +ze2in3 e2in

= 3Wn,cz|z|2. (A.15)


35/39


The next step is to determine U2. From equation A.9 we have

U2()

0

d

Wc( )U2( )

= g2

0

d

Wc( )U1()2

= g2z2W2n e

4in +z2W2n e4in + 2|z|2W0

. (A.16)

Let

X2() = V+e4in + Ve4in + V0 + X1(). (A.17)

The constant remains undetermined at this order of perturbation but doesnot appear in the amplitude equation for z( ). Substituting equation A.17

into A.16 yields

V+ =g2z

2W2n

1 W2n, V =

g2z2W2n

1 W2n, V0 = 2g2W0

1 W0. (A.18)

Using equation A.18, we find that

e2in |Wc X1X2 = Wn,c

0

d

ze2in +ze2in

V+e4in + Ve4in + V0 + X1()

e2in

=Wn,c[z

V+ +

zV0]

= g2Wn,cz|z|2 W2n

1 W2n+ 2W0

1 W0

. (A.19)

Combining equations A.14, A.15, and A.19, we obtain the Stuart-Landau

equation,

dz

d= z(1 + A |z|2), (A.20)

where w e have absorbed a factor /2 into an d

A = 3g32

+ 2g22

2

W2n

1 W2n+ 2W0

1 W0

. (A.21)

After rescaling, z = Z, = 2

t, this becomes

dZ

dt= Z(Wn Wn,c +A |Z|2). (A.22)


36/39


Appendix B

In this appendix we write explicit expressions for various quantities evalu-

ated in the p articular case of the alpha fun ction 2.8. First, the phase interac-

tion function defined by equation 4.2 becomes

KT() =2

1 1 eT

1 eT

a1(T)eT + TeT + a2(T)eT

(B.1)

a1(T) =TeT

1 eT 1

1 , a2(T) =1

1 1 eT1 eT . (B.2)

Second, equ ations 2.8 and 4.7 give

GT() = F0(T)1 eT

F1(T)eT

(1 eT )2 , (B.3)

where

F0(T) =2eT

(1 )2

(1 T+ 2T)e(1)T 1

(B.4)

F1(T) = T3eT

1

e(1)T 1

. (B.5)

Finally, setting

=i in equation B.3 and taking real and imaginary p arts

leads to the result

C() = a()C() b()

cos()[C()2 S()2] 2sin()C()S()

(B.6)

S() = a()S() b()

sin ()[C()2 S()2] + 2cos()C()S()

(B.7)

where

a() =F0(T)

C()2 + S()2 , b() =F1(T)e

T

[C()2 + S()2]2 (B.8)

an d C() = 1 eT cos(), S() = eT sin ().


37/39


Acknowledgments

This work w as sup ported by gran ts from the Leverhulme Trust (P.C.B.) and

the McDonnell Foundation (J.D.C.).

References

Amit, D. J., & Tsodyks, M. V. (1991). Quantitative study of attractor neural net-

works retrieving at low spike rates I: Substrate-spikes, rates and neuronal

gain. Network, 2, 259274.

Ben-Yishai, R., Bar-Or Lev, R., & Somp olinsky, H. (1995). Theory of or ientation

tuning in visual cortex. Proc. Natl. Acad. Sci. (USA), 92, 38443848.

Ben-Yishai, R., Hansel, D., & Sompolinsky, H. (1997). Traveling waves and the

processing of weakly tuned inputs in a cortical network module. J. Comput.

N eurosci., 4, 5777.

Berge, P., Dubois, M., & Vidal, C. (1986). Order within chaos. New York: Wiley.

Bonds, A. B. (1991). Temporal dynamics of contrast gain control in single cells

of the cat striate cortex. Visual Neuroscience, 6, 239255.

Bressloff,P.C., & Coombes, S.(1997).Ph ysics of the extended n euron. Int. J. Mod.

Phys., 11, 23432392.

Bressloff, P. C., & Coombes, S. (1998a). Desynchronization, mode-locking and

bursting in strongly coup led IF oscillators. Phys. Rev. Lett., 81, 21682171.

Bressloff, P. C., & Coomb es, S. (1998b). Spike t rain d yna mics u nd erlying pa ttern

formation in integrate-and -fire oscillator netw orks. Phys. Rev. Lett., 81, 2384

2387.

Bressloff, P. C., & Coombes, S. (2000). Dynam ics of stron gly coupled spiking

neurons. Neural Comput., 12, 91129.

Chapman, B., Zahs, K. R., & Stryker, M. P. (1991). Relation of cortical cell ori-entation selectivity to alignment of receptive fields of the geniculocortical

afferents that arborize within a single orientation column in ferret visual

cortex. J. Neurosci., 11, 13471358.

Cowan, J. D. (1968). Statistical mechanics of nervous nets. In E. R. Caianiello

(Ed.), Neural n etworks (pp . 181188). Berlin: Sprin ger-Verlag.

Cowan, J. D. (1982). Spontaneous symmetry breaking in large scale nervous

activity. Int. J. Quantum Chem., 22, 10591082.

Cross, M. C., & Hohen berg, P. C. (1993). Pattern formation outside of equilib-

rium. Rev. Mod. Phys., 65, 8511112.

Destexhe, A., Mainen, A. F., & Sejnowski, T. J. (1994). Synthesis of models for

excitable membranes synap tic transmission and neuromodulation u sing a

common kinetic formalism. J. Comp. Neurosci., 1, 195231.

Dimitrov, A., & Cowan, J. D. (1998). Spatial decorrelations in orientation-

selective cells. Neural Comput., 10, 17791796.

Douglas,R.J.,Koch,C.,Mahowald,M.,Martin,K.,&Suarez,H.(1995).Recurrentexcitation in neocortical circuits. Science, 269, 981985.

Ermentrout, G. B. (1998). Neural networks as spatial pattern forming systems.

Rep. Prog. Phys., 61, 353430.


38/39


Ermentrout, G. B., & Cowan, J. D. (1979a). A mathematical theory of visual

hallucination pattens. Biol. Cybern., 34, 137150.

Ermentrout, G. B., & Cowan, J. D. (1979b). Temporal oscillations in neuronal

nets. J. Math. Biol., 7, 265280.

Ferster, D., Chung , S., & Wheat, H . (1997). Orientation selectivity of thalamic

inpu t to simple cells of cat visual cortex. Natu re, 380, 249281.

Ferster, D., & Koch, C. (1987). Neuronal connections underlying orientation

selectivity in cat visual cortex. Trends in Neurosci., 10, 487492.

Georgopolous, A. P. (1995). Current issues in directional motor control. Trends

in Neurosci., 18, 506510.

Gerstner,W.,van Hemmen, J.L.,& Cowan, J. D.(1996).What m attersin neuronal

locking. Neural Comput., 94, 16531676.

Gray, C. M., & Singer, W. (1989). Stimulus-specific neuronal oscillations in ori-

entation column s of cat visual cortex. Proc. Nat. Acad. Sci., 86, 16981702.

Hansel, D., & Sompolinsky, H. (1996). Chaos and synchrony in a model of a

hypercolumn in visual cortex. J. Comput. Neurosci., 3, 734.

Hansel, D., & Sompolinsky, H. (1997). Modeling feature selectivity in local cor-

tical circuits. In C. Koch & I. Segev (Eds.), Methods of neuronal modeling (2nd

ed.) (pp. 499567). Cambridge, MA: MIT Press.

Hubel, D. H., & Wiesel, T. N. (1962). Receptive fields, binocular interaction and

functional architecture in th e cats visual cortex.J. Physiol. Lond., 160, 106154.

Jack, J. J. B., N oble, D., & Tsien , R. W. (1975). Electric current flow in excitable cells.

Oxford: Clarend on Press.

Lin,J. K., Pawelzik, K., Ernst, U.,& Sejnowski, T. J. (1998). Irregular synchronous

activity in stochastically-coupled networks of integrate-and-fire neurons.

Network: Comput . Neural Syst ., 9, 333344.

Lukashin, A. V., & Georgopolous, A. P. (1994a). A neural network for coding

trajectories by time series of neuronal p opu lation vectors. Neural Comput., 4,

5777.Lukashin, A. V., & Georgopolous, A. P. (1994b). Directional operations in the

motor cortex modeled by a neural network of spiking neurons. Biol. Cybern.,

71, 7985.

Mundel, T., Dimitrov, A., & Cowan, J. D. (1997). Visual cortex circuitry and

orientation t uning . In M. Mozer, M. Jordan, & T. Petsche (Eds.), Adv ances in

neural information processing systems, 9 (pp. 887893). Cambridge, MA: MIT

Press.

Murray, J. D. (1990). M athematical biology. Berlin: Sprin ger-Verlag.

Nelson , S., Toth, L., Seth, B., & Mur, S. (1994). Orientat ion selectivity of cortical

neurons during extra-cellular blockade of inhibition. Science, 265, 774777.

Pei, X., Vidyasag ar, T. R., Volgu shev, M., & Creu tzseld t, O. D. (1994). Recept ive

field analysis and orientation selectivity of postsynaptic potentials of simple

cells in cat visual cortex. J. Neurosci., 14, 71307140.

Pinto,D. J.,Brumberg,J.C.,Simons, D. J.,& Ermentrout, G.B.(1996).A quantita-

tive population model of whisker barrels: Re-examining the Wilson-Cowanequations. J. Comput. Neurosci., 6, 1928.

Reid, R.C., & Alonso, J. M. (1995). Specificity of monosyn aptic connections from

thalamus to visual cortex. Nature, 378, 281284.


39/39


Ringach, D.,H awken, M. J.,& Shapley, R.(1997).Dyn amics of orientation tun ing

in macaque primary visual cortex. Natu re, 387, 281284.

Sillito, A. M., Kemp, J. A., Milson, J. A., & Beradi, N. (1980). A re-evaluation

of the m echanisms un derlying simple cell orientation selectivity. Brain Res.,

194, 517520.

Softky, W. R., & Koch, C. (1993). The highly irregular firing of cortical cells

is inconsistent with temporal integration of random EPSPs. J. Neurosci., 13,

334-350.

Somers, D. C., Nelson, S. B., & Sur, M. (1995). An em ergent mod el of orienta tion

selectivity in cat visual cortical simple cells. J. Neurosci., 15, 54485465.

Tsodyks, M. V., & Sejnowski, T. J. (1995). Rapid switching in balanced cortical

network mod els. N etwork: Comput . Neural Syst., 6, 111124.

Usher, M., Stemmler, M., Koch, C., & Olami, Z. (1994). Network amplification

of local fluctuations causes h igh sp ike rate var iability, fractal firing patterns

and oscillatory local field p otentials. Neural Comput., 5, 795835.

van Vreeswijk, C. (1996). Partial synchronization in popu lations of pulse-

coupled oscillators. Phys. Rev. E., 54, 55225537.

van Vreeswijk, C., & Sompolinsky, H. (1996). Chaos in neuronal networks with

balanced excitatory and inhibitory activity. Science, 274, 17241726.

van Vreeswijk, C., & Sompolinsky, H. (1998). Chaotic balanced state in a model

of cortical circuits. Neural Comput., 10, 13211371.

Wilson, H . R., & Cowan, J. D. (1972). Excitatory and inhibitory interactions in

localized populations of model neurons. Biophys. J., 12, 124.

Wilson, H. R., & Cowan , J. D. (1973). A mathem atical theory of the functional

dynamics of cortical and thalamic nervous tissue. Kybernetik, 13, 5580.

Zhang, K.(1996). Representation of spatial orientation by the intrinsicd ynam ics

of the head-direction cell ensemble: A theory. J. N eurosci., 16, 21222126.

Received Apr il 15, 1999; accepted February 2, 2000.

p. c. bressloff et al- dynamical mechanism for sharp orientation tuning in an integrate-and-fire...

Documents