Synfire Braid Dynamics in Spatially Constrained Networks

Eric Jang, Chau Tran

May 16, 2013

Abstract

The presence of axonal conduction delays within spiking neural networks allowsfor complex dynamics in the form of out-of-phase, time-locked firing patterns. Thelengths of these delays partially depend on the distances between neurons. As theneuroanatomical layouts of animal brains are incredibly conserved, this leads us tosuspect that the physical arrangement of neurons may have conserved delays and con-nectivity, and by extension, unique dynamics. Here we simulate recurrently-connectednetworks of spiking neurons with synaptic delays that implicitly define topology. Weexamine spike rasters, firing rates, and detect synfire braids within various geometriclayouts. Although the overall stability and firing rates remain consistent regardless ofgeometry, we find that increasing “orderliness” in the network produces larger synfirebraids. These results suggest that low-dimensional, constrained topologies may actu-ally exhibit the richer dynamics necessary for sustaining complex phenomena such asattention and working memory.

1 Introduction

Synfire braids are emergent neural network patterns that occur when presynaptic neuronsfire out of phase with each other, but due to asymmetry in axonal conduction delays,the spikes may still arrive simultaneously at a postsynaptic neuron, resulting in a highprobability of firing. As these patterns are apparently self-selecting and self-sustainingvia spike-time-depdent plasticity (STDP), synfire braids, also referred to as polychromousgroups, have been proposed to serve as a mechanism for attention and working memory[1, 2].

Supposing that synaptic delays are proportional to the distances between point neurons,then the delays would implicitly define distances between neurons in the network[3]. Intypical computer models of spiking neural networks, delays are randomized, and it followsthat neurons would necessarily sit in a very high-dimensional space to avoid degeneratedistances (i.e. violations of the triangle inequality). However, the brain occupies 3D space;thus, if delays are proportional to distances, which cannot be degenerate in the space, then

1

delays are governed by these same constraints.1

Evolution tends to optimize form to best serve function, so we hypothesize that theneuroanatomy of the brain exists as it is to some computational advantage. In context ofspiking network dynamics, we seek to determine whether the spatially-constrained delaysgive rise to unique dynamics in particular geometries.

2 Methods

We simulated a network of 1000 recurrently-connected Izhikevich neurons, examining howsynfire braids and firing rhythms are affected by delays imposed by various network ge-ometries. We modified geometry in several ways: changing the distribution of delays perneuron, separating inhibitory and excitatory neurons, limiting connectivity between distantneurons, and using non-Euclidean metric spaces to compute delays. The network modelwas adapted from Izhikevich’s spnet.m code. All simulations and analyses were performedin MATLAB, using a cluster of workstation computers.

2.1 Modifying Delay Distributions

We have 8 models based on the 3D distribution of neurons (cube3d, torus3d, sulci3d,rectangle3d, lattice3d, sphere3d, segCube3d, segCheckerboard3d). For each model, we firstgenerate a cloud of points based on the predefined distribution of the model. The delaymatrix is then computed based on the distance matrix of the generated cloud of points.More specifically, the distance matrix is normalized by the largest distance, and discretizedinto 20 bins (1,2,..,20). For these models, the delay distributions are approximately normal.

We also have 3 models with uniform delay distributions: control, asymmetric1, andasymmetric2. The specific details about these models can be found in the table below. Wealso have a network arranged in a ’hypertetrahedron’, where the distances between anytwo neurons are equal (thus they all share the same conduction delay).

2.2 Spatial Separation by Neuron Type

The balance of excitation and inhibition is critically important to achieve useful compu-tation in spiking neural networks. Instead of scattering excitatory and inhibitory neuronsrandomly together in the same space, we wondered what would happen if we segregated

1Of course, the distance between neurons is not the only factor in deciding delays; myelination ofaxons results in faster conduction velocities. Then, the combination of delays AND myelination define thetrue geometry of the network, so that given an arbitrary delay matrix, one could still construct arbitrarynetwork layouts, provided that myelination accounts for degenerate situations as needed. However, in realbrains, myelination exists as a tradeoff between conduction speed and axon diameter [4], so yet again themyelination factor is partially restrained in 3D physical space. Summarily, the entries in the delay matrixshould not be independent random variables.

2

populations of excitatory and inhibitory neurons in their own separate clusters. These areexplored in the segCube3d and segCheckerboard3d models.

2.3 Restricted Connectivity

Although long-distance connections between cortical neurons in the brain are possible, suchsynapses are largely outnumbered by synapses to neighboring cells. It follows that actualbrain dynamics get most of their contributions from a structured connectome in which localconnectivity dominates long-distance projections. Such structures are explored in sulci3dand torus3d models.

2.4 Non-Euclidean Geometry

For the networks in which neurons were physically placed prior to generating delay matrices(3d networks), we replaced the Euclidean distance metric with a couple other functions,shown below. We are not suggesting that synaptic delays in actual brains are related inany way to the following metric spaces, but are rather constraining them out of the generalcuriosity of how braid organization might be impacted by unconventional constraints onphysical space (e.g. myelination).

d(P,Q) = max(|p1 − q1|, |p2 − q2|, |p3 − q3|) (1)

d(P,Q) = |p1 − q1|+ |p2 − q2|+ |p3 − q3| (2)

2.5 Summary of Network Geometries

Below is a summary of the different networks that we experimented with. Some delay ma-trices were generated using distance metrics between points placed in space, while otherswere generated directly from a distribution (control, asymmetric1&2, hypertetrahedron).

Experiment Description Layout Delay Histogram

control.m

Izhikevichs 2008code where eachpostsynaptic de-lay is uniformlychosen between[1,20].

N/A

3

cube3d.m

Neurons are gen-erated randomlyin a cube, anddelays derivedfrom normalizeddistances.

cube4d.mSame as cube3d,except 4-dimensions.

N/A

sphere3d.mNeurons arrangedin sphere

rectangle3d.m

Same as cube3d,except squasheddown. Ap-proaches 2D.

lattice3d.m

Instead of beingplaced randomly,neurons are posi-tioned in a lattice.

4

asymmetric1.m

All neuronsidentical, buteach neuronhas 5 of eachdelay value(oldIzhikevich model)

N/A

asymmetric2.m

Each neuron has100 projectionswith the samedelay, so thereare 5 types ofneurons.

N/A

hypertetrahedron.m

Sits on an 999-equilateral di-mension spacesuch that allneurons are thesame distancefrom each other.

N/A

segCube3d.m

Inhibitory neu-rons are sep-arated fromexcitatory neu-rons.

segCheckerboard3d.m

Excit. and inhib.neurons separatedby 3D checker-board.

5

sulci3d.m

Neurons arrangedalong foldedsheet. Connec-tions not allowedto cross troughs.

torus3d.m

Neurons arrangedin torus. Con-nections allowedif neurons arecloser than themaximum radiusof torus.

Table 1: Network Topologies. In the Layout column, neurons are shown in blue (excitatory:solid dots, inhibitory: open circles); subsets of synapses are shown in green. The delayhistogram column shows counts for each postsynaptic delay over all the neurons. Delaysprojecting from excitatory neurons are shown in grey, and delays projecting from inhibitoryneurons are shown in blue.

2.6 Braid-Detection

We made use of the braid_detection.m code provided by Dr. Elie Bienenstock, modifyingit to take in arbitrary delay distributions generated by our geometric models. The algo-rithm examines the delay matrix and synaptic weights at time t, and computes all pathsfrom neuron i to neuron j such that paths follow strong connections and that the pathsare the same length ddd. Although sampling one possible braid length (e.g. ddd = 35ms)out of N2 pairs of neurons for a single time point (3−9% of the simulation) is by no meansrepresentative of the overall braid distribution, repeating this procedure yielded fairly con-sistent results for each geometry.

6

(a) Delta rhythm, 1sec (b) Poisson regime, 251 sec (c) delta rhythm, 525sec

Figure 1: Spike rasters are all stable and fairly similar across geometries

3 Results

3.1 Characteristic patterns across all geometries

We observe fairly characteristic overall firing patterns in all experiments. They exhibit2-4Hz delta rhythm for 1 or 2 seconds, transitioning into Poisson-like firing activity forabout 8-10 minutes, and then re-entering the delta regime. The pattern persists for rad-ically different spatial distributions and distance functions, which means that excitationand inhibition are balanced, and that the dynamics observed by Izhikeivch 2006 are quiterobust. We did not observe any outwardly interesting behaviors in the firing rates either(not shown).

3.2 More orderly networks produce larger braids

Networks with lower dimensions, narrower delay histograms, and more restricted connec-tivities exhibited far larger braids than the original control groups. There seemed to be nodifference in networks that used non-Euclidean geometries (data not shown).

Because the network geometries are very diverse and difficult to parameterize (theylie in different dimensions and topologies), we loosely describe networks as ordered or un-ordered, based on the randomness of delays, neuron positions, and how many degrees offreedom are allowed in the neuron placement.2

2For example, the cube3d network is more ordered than cube4d because the triangle inequality mustbe satisfied in three dimensions as opposed to four, thereby imposing more constraints on possible networkdelays. However, the high-dimensional hypertetrahedron is even more ordered because the neurons areprecisely placed within a 999-dimensional space and are subject to less degrees of freedom than wouldbe imposed just by triangle inequality. The torus3d network (restricted connections, partially randompositions) is more ordered than the cube3d network but we cannot safely say whether it is more or lessordered than the lattice3d network (random connections, completely nonrandom points).

7

Experiment 1 hour run (ddd=20)

control.m

cube3d.m

cube4d.m

sphere3d.m

rectangle3d.m

lattice3d.m

8

asymmetric1.m

asymmetric2.m

hypertetrahedron.m

segCube3d.m

segCheckerboard3d.m

9

sulci3d.m

torus3d.m

Table 2: These figures depict sampled synfire braids with each path adding up to 20ms ofdelay. Due to computational limitations, we can only compute paths with four or fewerneurons. Observe that more “ordered” networks exhibit larger braids.

4 Discussion

Here we demonstrate that spatial organization of networks indirectly affects the local firingpatterns in the form of synfire braids. Curiously, the overall dynamics of the networkwere more or less consistent, but imposing more constraints (neuron positions, maximumaxon distances, fewer dimensions) increased the number of braids (we describe this asthe network becoming more“orderly”). It would be interesting to find out if there areany network dynamics that arise from exotic mathematical properties of low-dimensionalmanifolds [5].

It is important to note that the “orderliness” of the network was already very con-strained to begin with, due to the coarse discretization of synaptic delays. Had the delaysbeen more finely discretized, we would probably observe far fewer braids.

We were not able to find any glaring differences between spike rasters of different models,but we cannot rule out the possibility that the differences in dynamics were much moresubtle (such as braids). Indeed, there was variation in the intensities of delta rhythms, butwe were unable to quantify these at glance.

If synfire braids do indeed represent some sort of self-selecting, homeostatic atten-tion/memory mechanism, as suggested by Izhikevich and Edelman, it is somewhat con-cerning that we are only able to scratch the surface of this phenomenon with today’s com-puters. The current braid detection algorithm finds all anatomically plausible paths from astart to end neuron, but with a maximum of three neurons in between. The task becomesimpossible to compute when we consider all pairs of neurons, all possible path lengths,and arbitrary numbers of interneurons. Thus, one cannot help but wonder whether braidanalysis with our current computational capabilities is like feeling the leg of an elephant

10

and saying it is a tree.Perhaps it would be more efficient and meaningful to analyze braids based on the firing

history of the network – to find out which braids actually occur during the course of thesimulation rather than which ones are anatomically possible (especially considering that“ordered” networks have huge braids). This would also remove us from the assumptionthat STDP does not “break” certain braids while forming others, which we suspect is thecase.

5 Acknowledgements

We thank Dr. Elie Bienenstock for his braid detection code, and for his instruction andadvising on this project.

References

[1] Elie Bienenstock. A model of neocortex. Network: Computation in Neural Systems,6(2):179–224, 1995.

[2] Eugene M Izhikevich. Polychronization: Computation with spikes. Neural computation,18(2):245–282, 2006.

[3] H. A. Swadlow and S. G. Waxman. Axonal conduction delays. 7(6):1451, 2012.

[4] Samuel S.-H. Wang, Jennifer R. Shultz, Mark J. Burish, Kimberly H. Harrison,Patrick R. Hof, Lex C. Towns, Matthew W. Wagers, and Krysta D. Wyatt. Functionaltrade-offs in white matter axonal scaling. The Journal of Neuroscience, 28(15):4047–4056, 2008.

[5] Wikipedia. Low-dimensional topology — wikipedia, the free encyclopedia, 2013. [On-line; accessed 16-May-2013].

11