paper egr 183 modeling neural networks in silico

Modeling Neural Networks In Silico - A Neural Network Model and Novel Experimental Design using Light-

Directed Electrical Stimulation of Neurons for statistical refinement of Computational Neural Network Models

Daniel Carlin‡, B.S.E., Daniel Cook†, and Joshua E. Mendoza-Elias§

Fall 2007 Duke University

Engineering 183: Special Topics – Modeling Neural Networks In Silico Dr. David Needham, Ph.D. Pratt School of Engineering

Submitted: Sunday, December 16, 2007 Abstract Long‐term potentiation (LTP) and Long‐term depression (LTD) are the best understood mechanisms by which biological systems achieve the sculpting of neural pathways necessary for learning and memory. Foundational to the understanding of LTP and LTD is the model of Hebbian learning. Although research has demonstrated that neural networks can be developed and trained in silico to mimic the computational function of neural ensembles, the development of a truly realistic model remains elusive. We construct a simulation of Hebbian learning implementing the Hopfield network model. We then propose an experimental setup that will use the light‐directed electrical stimulation of neurons cultured on silicon wafers in order to gain information on the parameters of synaptic strength and firing frequencies for all neurons in the system simultaneously (determinants of LTP and LTD). This will allow the parallel running of neural networks in the in vivo and in silico setting, allowing statistical comparison between the biological model and computer simulation and additional refinement to create a more biologically relevant computational model.

Keywords: short and long-term potentiation (LTP), short and long-term depression (LTD), neural plasticity, Hebbian learning, Hopfield Network, parallel neural network training,

light-directed electrical stimulation (LDES).

Note: This study was submitted to Vertices, The Journal of Science and Technology at Duke University, which indicated an interest in publication of this paper. However, they decided that it was too technical for their audience and forwarded it to Jupiter, The Undergraduate Journal for Engineering, which was suggested as a better venue.

‡ Daniel Carlin, Pratt School of Engineering, Department of Biomedical Engineering. E-mail: [email protected]. † Daniel Cook, Trinity College of Arts and Sciences, Department of Computer Science. E-mail: [email protected]. Website: http://www.duke.edu~dkc6 § Joshua Mendoza-Elias, Trinity College of Arts and Sciences, Department of Biology. E-mail: [email protected]. Website: http://www.duke.edu/~jme17

Modeling Neural Networks In Silico Carlin, D., Cook, D., Mendoza-Elias, J.

of 25

Table of Contents: 1 Introduction ................................ ................................ ........... 3

1.1 Mechanisms Underlying LTP and LTD .... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 Figure 1: Molecular Mechanisms of LTP. ..............................................................4 Figure 2: Signaling Cascades activated by LTP and LTD. ...................................4 Figure 3: Short-term and Long-term changes to LTP. ..........................................5 Figure 4: Properties of LTP and LTD - Specifications and Associativity. ............5

1.2 Spike Timing Dependent Plasticity (STDP) ... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 Figure 5: The Biological Basis of our Model. .........................................................6

1.3 Theoretical Models of Neural Networks ... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 Figure 6: Abstract Representation of Model. . ........................................................8 Figure 7: Firing Relationships. ...............................................................................8

2 Mathematical Formulation ................................ .................... 10 2.1 Setup ... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 2.2 Hebbian Learning ... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 2.3 Inhibitory neurons and model training ... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

Supplement 1: Equation Summary. .....................................................................13 3 Computer Simulation ................................ ............................ 14

3.1 Setup ... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 3.4 Solving step ... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

Figure 8: Convergence of Firing Rates. ................................................................16 Figure 9: Rates of change in Firing Rates. ..........................................................16

3.5 Simulation Results .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 4 Future Directions ................................ ................................ . 18

Figure 10: Proposed Experimental Setup employing Light-Directed Electrical Stimulation (LDES) of Neurons cultured on Silicon Wafer. ...............................19 .

5 Acknowledgements ................................ ................................ 20 6 Appendix Simulation Code ................................ .................... 21

Code 1: model.m .....................................................................................................21 Code 2: normalize.m...............................................................................................22 Code 3: dT.m...........................................................................................................22 Code 4: alpha.m ......................................................................................................23 Code 5: beta.m ........................................................................................................23 Code 6: deadneurons.m ..........................................................................................23

7 References ................................ ................................ ............ 24

Carlin, D., Cook, D., Mendoza-Elias, J. Modeling Neural Networks In Silico

of 25

1 Introduction Synaptic connections between neurons form the basis of neural circuits that comprise neural networks. More importantly, it is the dynamic entity of synaptic connections that impart the property of plasticity that enable neural networks to "learn." Synaptic connections are in a constant state of change that is determined in response to neural activity and other mechanisms which operate in short‐term and long‐term time scales (milliseconds to years) [1, 8, 20]. In terms of neural ensembles, neurons are "trained" by mechanisms of potentiation and "detrained" by mechanisms of depression. In terms of short‐term potentiation, synapse connections are modulated using the mechanisms of facilitation, augmentation, and potentiation; all of which function by enhancing neurotransmitter release from presynaptic terminals. Complementary to this are mechanisms of short‐term depression, which decrease the amount of neurotransmitter released and appear to result from activity‐dependent depletion of synaptic vesicles primed for exocytosis. More pertinent to this research paper, are the mechanisms of Long‐Term Potentiation (LTP) and Long‐Term Depression. These biological mechanisms have provided the foundation of Hebbian Models of learning and the modeling of neural networks. Furthermore, LTP and LTD have been most thoroughly studied in the mammalian hippocampus and have a high degree of biological relevance for biomedical related studies and applications [4, 8]. 1.1 Mechanisms Underlying LTP and LTD LTP functions at time scales of 30 minutes or longer. LTP is accomplished by the release of glutamate from the presynaptic cleft and its binding to AMPA receptors on the postsynatpic cleft (See: Figure 1). AMPA receptors generate EPSPs (excitatory postsynaptic potential) that cause the expulsion of magnesium from the NMDA receptor in the postsynaptic cleft, thus allowing calcium to become permeable to the postsynaptic cleft (See: Figure 1). AMPA receptors generate EPSPs (excitatory postsynaptic potential) that cause the expulsion of magnesium from the NMDA receptor in the postsynaptic cleft, thus allowing calcium to become permeable to the postsynaptic cleft. In turn, these rapid and high calcium spikes cause the postsynaptic neuron to activate signal transduction cascades that include: Ca+2/calmodulin‐dependent protein kinase (CaMKII) and protein kinase C (PKC) (See: Figure 2) [2, 16]. These cascades are involved in short‐term changes that result in increased synaptic efficacy for the maintenance of LTP, such as an increase in AMPA receptors in the postsynaptic neuron (See: Figure 3) [6]. In terms of more long‐term changes, repeated stimulation causes increased activation of genetic programs involved in the upregulation of the transcription of AMPA receptors, tubulin, actin, signal cascade transduction networks ‐ essentially genes that will increase the size of the synaptic cleft, the radius of the axon and sensitivity to neurotransmitter glutamate by increasing AMPA receptors in the postsynaptic cleft. These long‐term changes in gene expression profiles are believed to be the basis by which essentially permanent modifications to brain function are achieved. LTD functions antagonistically to prevent synapses from attaining maximum synaptic efficacy as over time this would make the encoding of new information impossible. LTD functions in a similar fashion that essentially differs in the manner by which calcium levels rise in the postsynaptic cleft (which is modulated by stimulation from other neurons). In opposition to LTP, where EPSPs generate brief high frequency stimulation, LTD requires stimulation at a low rate (about 1 Hz) for long periods (10‐15 minutes). The low frequency stimulation then causes calcium levels to rise in a more steady and slow fashion. In turn, the rise in calcium levels causes the activation of Ca+2–dependent phosphatases that in the short‐term result in the loss of AMPA receptors and in the long‐term change gene expression to promote decrease in size of the synaptic button and radius of the axon [2, 3, 5, 10, 20].


of 25

Figure 1: Molecular Mechanisms of LTP. A: Resting Potential: EPSPs in the presynaptic neuron are insufficient to cause depolarization of the postsynaptic neuron and relieve the Mg+2 block on NMDA receptors. B: Postsynaptic Depolarization: Sufficient stimulation allows for the depolarization and entry of the Ca+2 ions, allowing the unblocking of NMDA receptors and activation of short-term LTP mechanisms. [20]

Reproduced from Purves et al (2008), Neuroscience: Fourth Edition (2008) [20].

Figure 2: Signaling Cascades activated by LTP and LTD. A: At resting potential: Signaling cascades in the postsynaptic cleft activated in response activation of LTP. B: Siganaling cascades in the postsynaptic cleft activated in response to activation and maintenance of LTD. [6, 20]

Reproduced from Purves et al (2008), Neuroscience: Fourth Edition (2008) [20].


of 25

Figure 3: Short-term and Long-term changes to LTP. Repeated stimulation causes short-term pathways (signaling cascades) to cause changes in gene expression that will serve to enhance the process (LTP or LTD) that is dominant. [20]

Reproduced from Purves et al (2008), Neuroscience: Fourth Edition (2008). [20]

Figure 4: Properties of LTP and LTD - Specifications and Associativity. Hebbian theory relies on Spike-Timing Dependent Plasticity (STDP). A: Strong activation of a neural pathway without adjacent pathways firing within an interval time-window causes the weaker pathway to not be strengthened. B: Strong activation of a neural pathway coupled with adjacent pathways firing within an interval time-window causes an already strong synapse to be strengthened and a weaker synapse to be strengthened. [5, 10, 20] Reproduced from Purves et al (2008), Neuroscience: Fourth Edition (2008) [20].


of 25

1.2 Spike Timing Dependent Plasticity (STDP) Central to both mechanisms is spike‐timing dependent plasticity (STDP). This imparts onto LTP and LTD the property of associativity1 (See: Figure 4). At a given low frequency of synaptic activity, LTD will occur if presynaptic activity precedes a postsynaptic action potential, while LTP occurs if the postsynaptic action potential follows presynaptic activity. The relationship between the time interval and the magnitude of synaptic change is a very sensitive function of the time internal, with no changes observed if the presynaptic activity and postsynaptic activity are separated by 100 milliseconds or longer. This requirement for precise timing between presynaptic and postsynaptic activity for the induction of these forms of long‐lasting synaptic plasticity is what drives the development of neural networks. Although the mechanisms involved are not yet understood, it appears that these properties of STDP arise from timing‐dependent differences in postsynaptic Ca+2 signals. Specifically, if a postsynaptic action potential occurs after presynaptic activity, the resulting depolarization will relieve the block of NMDA receptors by Mg+2, yielding LTP. In contrast, if the postsynaptic action potential occurs prior to the presynaptic action potential, then the depolarization associated the postsynaptic action potential will subside by the time that an EPSP occurs. This will reduce the amount of Ca+2 entry through the NMDA receptors, leading to LTD [5, 20].

Figure 5: The Biological Basis of our Model. A: Our model was inspired by the connectivity of neurocytes and development of motor coordination in the human cerebellum. B: Signal from the eye (red) is transmitted via climbing fibers (1–input neuron) which stimulate parallel fibers (interneurons: 2–excitatory or 3–inhibitory). Parallel fibers are believed to function as “timers.” Those with the correct “timing” (e.g., phase) in relation to the initial stimulus (climbing fibre) will serve as an excitatory neuron, and vice versa. C: Purkinje cells (4–output neuron) receive stimulus from Climbing fibers and Parallel fibers. The key serves to show the relationships between in vivo neurocytes and our in silico representation of them in Figure 6 (below).

A: Reproduced from http://www.mphy.lu.se/avd/nf/hesslow/neuroscience/classcond.html B, C: Reproduced from Purves et al (2008), Neuroscience: Fourth Edition (2008) [20]

1 Weak stimulation by itself will not trigger LTP, however if one pathway is strongly activated at the same time a neighboring pathway is weakly stimulated then both synaptic pathways will undergo LTP. LTP and LTD are restricted to the activated (stimulated) synapse. In short, cells that fire together, wire together, apart from specificity.


of 25

1.3 Theoretical Models of Neural Networks There are several different models of learning in neural networks, most of which originated from Hebb's postulate that "When an axon of cell A is near enough to excite a cell B and repeatedly or persistently takes part in firing it, some growth process or metabolic change takes place in one or both cells such that A's efficiency, as one of the cells firing B, is increased" [13]. These models are separable into two main classes: firing‐rate‐based and spike‐based. They differ in that firing‐rate models use the average firing rate of a neuron over a given unitary time period as the basis of computation, while spiking models use the spike train of a given neuron in a probabilistic fashion.

Our model was biologically inspired by the action of LTP and LTD in determining neural architecture in the cerebellum (See: Figure 5). Our model is a smoothed, synchronous version of the firing‐rate model based on the Hopfield attractor network (see Figure 6). This class of network stabilizes to an attractor (a stable pattern of neuron firing). The network can be considered a function of the input neurons based on the synaptic weights, so that training the network consists of defining this attractor (which can also be termed "memory"). This is done by reinforcing Hebbian learning proportionately to the difference between the emergent and desired output signals at each time step, given an input signal. We implement a slightly modified version of the model originally published by Hopfield [14], differing in two respects:

1. We do not update individual neurons asynchronously, but rather compute the firing rates of all neurons synchronously at each time step. This is justified because we are calculating averages over each time‐step, rather than trying to discretize a continuous process ‐ in other words, we are abstracting away from the mechanics of individual firing neurons in order to observe the state of the entire system at every time

€

(t) . 2. We use neurons with a continuous, graded‐response rather than two state McCullough‐Pitts neurons. Networks of graded‐response neurons have been shown to exhibit similar emergent computational properties to the original model [15]. Once again, this is justified because we are measuring the average firing rates of all neurons over a given time step rather than simply whether or not a neuron fired during that time step. Thus time steps in our simulation are several times longer than the refractory period of a single neuron. This allows computation of the simulation for a larger number of neurons, enabling a larger parallel experiment to be run.

We should note that networks with synchronous updating may have certain differences in emergent dynamical properties. These were beyond the scope of this study, but more information may be found in Little et al. [17]. (See Section 2 for a mathematical formulation of our firing‐rate‐based model of Hebbian learning implementing a Hopfield attractor network).

Spike‐based models are newer and not yet as well studied as firing‐rate models. These models assume that at a given time-step

€

(t) , there is some unknown probability distribution describing the likelihood of each neuron spiking at that time point. This distribution is parameterized around the hidden variable (the true input). Thus, rather than gradient descent toward a function as with firing‐rate based models, spiking models exhibit a gradient


of 25

Figure 6: Abstract Representation of Model. The diagram outlines the relationships between the different types of neurons: input neuron (1–Climbing fiber), excitatory interneuron (2–Parallel fiber firing in phase), inhibitory interneuron (3–Parallel fiber firing out of phase), and output neuron (4– Purkinje cell). Please note the key to the left corresponds to the key in Figure 5.

Figure 7: Firing Relationships. A: The equations governing the model: (i.) Firing Rate; (ii.) Synaptic Strength; (iii.) the “Learning Rule.” B: In phase – (i.) If two neurons are synchronous, then the synaptic weight will increase by 1.0. (ii.) The closer the pre-synaptic firing rate vi (phase = α) of two neurons (increasing synchronicity), the higher vj increases and synaptic weight wi,j in the next time-step. C: Out of phase – (i.) If two neurons completely asynchronous, then synaptic weight will increase by 0. (ii.) The further apart the pre-synaptic firing rate vi (phase = α) of two neurons (increasing asynchronicity), the higher vj decreases and synaptic weight wi,j in the next time-step.


of 25

descent towards a distribution parameterized around the attractor state (memory). These models then study the ability of a population of individually noisy neurons to reconstruct the hidden variable. Indeed, much of the work thus far focuses on coding of hidden variables in populations of afferent neurons. See Zemel et al. [22] for an explanation of population coding, and Deneve et al. [11] for a model showing that population of spiking neurons can perform Bayesian inference computations.

We chose to simulate the Hopfield network primarily due to practical concerns, namely that spike-based models are analytically and computationally intensive, and are difficult to train. They are also not yet well understood, thus there is uncertainty as to which model would be appropriate for our purposes here. Conversely, Hopfield attractors are theoretically well understood (see Ermentrout et al. [9], ch. 5). Our model allows us to compute a state of the entire network by solving a single system of linear differential equations and then updating synaptic weights appropriately, which is a faster computation than spike-based models allow.


of 25

2 Mathematical Formulation Note: See Supplement 1 (pp. 13) for an Equat ions Summary

2.1 Setup We define a vector V2 of dimension N (we are modeling N neurons), where vi represents the average firing rate of neuron i over a given time step. We also define weights wi,j, representing the efficacy of synaptic signal propagation (i.e., connection strength) between pre-synaptic neuron vi and post-synaptic neuron vj . Due to the directionality of synapses, wi,j does not necessarily equal wi,j, because these represent two different synapses. The rate of firing of a neuron is determined by the following:

€

v j = wi, jvii= 0

N

∑ (1)

which is a linear combination of the firing rates of all neurons connected to neuron vj , weighted by the strength of each synapse.

In a system with N neurons, this yields a system of N equations. We choose one neuron, vi=1 , to be the ”input” neuron. This can be achieved in a real system by voltage clamping the neuron. In this model we simply fix v1 at a specific rate during each time step. Say we choose the rate to be v1 = a, and assuming that the neuron does not feed back on itself (wi,j = 0) then the linear system of equations is:

€

v1 = av2 = w1,2v1 + w3,2 + ...+ wn, jvN vN = w1,Nv1 + w2,Nv2 + ...+ wN ,N−1vN−1

Let D be a column vector as follows:

,

and define T to be the following matrix:

. (2)

Then we have the matrix equation , (3)

where solving

(4) yields the average firing rates for all neurons over a given time step [5, 7, 12, 14, 19]. 2 This is a discrete model - for simplicity, time arguments are omitted where immaterial.


of 25

2.2 Hebbian Learning As in biological neurons, the simulated neurons “learn” by changing the synaptic

weight wi,j = 0. The rule by which these weights are changed classifies Hebbian learning. We

start by assuming that is a twice differentiable function of the two local variables, v i

and vj. Expanding in Taylor series, then:

Knowing that the change in weight should be dependent on the correlation between the two firing rates, the simplest learning rule that can be obtained is that c0 through c4 equal zero and c5 = A, where A is some positive constant, i.e.,

€

dwi, j

dt= A∗v1 ∗v2

This is the prototype for Hebbian learning, and is our representation of LTP.

However, we would also like to include a model for LTD, so we assume that the synaptic weights decrease in strength whenever they are not used (i.e. when vi ∗ vj = 0) at a constant rate, B. Then;

€

dwi, j

dt= A∗v1 ∗v2 − B

Additionally, we seek to bound the weights of the synapses between 0 (equivalent to

disconnected neurons) and 1 (a maximally efficient synapse). In order to do this, we choose and for positive constants and between 0 and 1. and

ca be viewed as the relative strengths of LTP and LTD in a single time step; respectively. We then have:

€

dwi, j

dt=α ∗ (1− wi, j )∗vi ∗v j −β ∗wi, j (5)

Furthermore, we want to bound the firing rates between 0 (not firing) and 1 (firing at

the maximally allowed rate for that neuron). In order to do this, the constants must be normalized by dividing by the sum of the weights of the most strongly connected neuron at every time step. To do this, we calculate the Hebbian learning change in synaptic strength first, and then normalize by the largest sum of the weights going into any one neuron.

While the equations above are continuous, our model is discrete. Thus at each time step the rule for obtaining from is therefore:

(6).

This introduces the important concept of competition between synapses. The idea is that


of 25

synaptic weights can only grow at the expense of others so that if a certain subgroup of synapses is strengthened, other synapses to the same post-synaptic neuron have to be weakened by dividing over a larger sum. 2.3 Inhibitory neurons and model training In order to implement learning reinforcement (this may be thought of as a dopamine drip3), we shall transform α into a function dependent on the firing rate of some arbitrary output neuron (let us use vN ). There are clearly a very large number of potential training functions that might be considered, depending on the desired behavior of the network. We shall seek here to obtain some degree of phase inversion between the input (voltage-clamped) neuron v1 and vN . To do this, we set

, (7) so that maximal promotion of synaptic efficacy would take place only during time steps when the input and output neurons had very different average firing rates. While this rule is sufficient to force the firing rate of the output neuron to zero for all input firing rates, in order to get opposing behavior, we must introduce inhibitory neurons into our model.

Inhibitory neurons are modeled similarly to the excitatory neurons; however, rather than a firing rate bound by [0, 1], we use [−1, 0]4. Thus, according to the learning rule above, a presynaptic inhibitory neuron vi simultaneously exhibiting a high firing rate as a post-synaptic excitatory neuron vj will cause the synaptic strength wi,j to be reinforced. However, the summation in equ. (1) means this reinforcement will result in a lower average firing rate for vj during future time steps (all else held equal). Using the same feedback rule as (7) and using the same update step as (6), we show that the output neuron’s firing rate reaches a local maximum when the input neuron’s rate is at a local minimum, yet also follows the input neuron to its maximum. Further refinement of the model would be required to show complete phase inversion between the input and output neurons [5, 7, 12, 14, 19]. 3 We define the reinforcement factor as a function of two neurons, one voltage-clamped and the other an ”output” neuron, the firing rate of which can be monitored by micro-electrode. Thus this form of learning reinforcement does not require monitoring of the firing rate of every single neuron, which in turn means that a biological reinforcement mechanism would only require a uniform concentration of dopamine across the entire network, rather than neuron-specific dosages.

4 Our model, though mathematically equivalent to this, differs slightly in implementation. Please refer to section 3 for details.


of 25

Supplement 1: Equation Summary. This supplementary figure accompanies Sections 2 and 3; and condenses the mathematical relationships describes in those sections. (A-D) accompanies Sections 2. (E) accompanies Section 3.


of 25

3 Computer Simulation Note: See Supplement 1 (pp. 13) for an Equat ions Summary

We used a computer simulation of the model described above in order to be able to generate testable predictions of neural network behavior. This simulation was written in MATLAB, and simulations were performed using Octave5, with graphical representations in Gnuplot6. After initialization, the simulation repeatedly normalizes, calculates the next step of the strength matrix, and then solves for the resulting vector of firing rates. 3.1 Setup The implementation is a direct representation of the mathematical formulation described above. We set N = 20, modeling a small network of 20 neurons for computational tractability, and the simulation is run for STEPS = 100 time steps. Additionally, we define 6 inhibitory neurons in our model. We begin by defining the stimulus pattern for our input neuron:

€

v1(t) =sin tΠ

8

+1

2 , (8)

using a periodic function bounded between 0 and 1. This allows us to observe the phasic relationship between v1 and vN , the output neuron. We then define the vector D:

7 (9)

Inhibitory neurons are stored in the V vector, and identified by a subset Inhib of the indices of V, where dim(Inhib) < N determines the number of inhibitory neurons in the system. All logic is performed identically for inhibitory neurons as for excitatory, with one fundamental difference: synaptic strengths wi,j between an inhibitory neuron

€

i and its postsynaptic neurons j are negated in (1). This means that during the solving of (4), wi,j terms are negated in the T matrix when

€

i ∈ I . All other processing of inhibitory neurons remains the same as for excitatory, and with the exception of the solving step, all weights are maintained in the system as positive floating point numbers in order to ease computation of the update matrix. We next initialize a random matrix T of dimensions (N, N ). We set T1,1 = 1 (the input neuron is entirely self-determined) and

€

Ti, j = 0, 1< j < N (the input neuron is not post-synaptic to any other). We additionally set all

€

Ti, j = −1, 1< i < N , such that no neuron is post-synaptic to itself. We then have 5 See: http://www.octave.org

6 See: http://www.gnuplot.info

7 We must additionally constrain a second neuron to fire constantly at a rate of 1.0. This is due to equation (4), which calculates each firing rate as a linear combination of the others. Each of these linear combinations is dependent on the value of v1 . Thus when v1 = 0, such that D = 0, the zero vector, the only possible solution to (4) is that V = 0 as well. This is an artifact of this model and is not biologically significant.


of 25

€

T =

1 0 0 0 …

0 1 0 0 …

w3,1 w3,2 −1 w3,4 …

… … … −1 …

… … … … …

. (10)

which the reader will notice differs from equation (2) in one respect: in that equation,

€

Ti, j = −1, 1< i < N . Those values are necessary in order to avoid solving for a vector of 0 firing rates in the solving step in (4). 3.2 Normalization step As presented in (2.2), matrix T, representing synaptic strengths in the network, must be normalized before the first time step, and then at every subsequent time step in order to maintain the bounding conditions (strengths and firing rates must remain both in the interval [0, 1]). This normalization proceeds as follows (where Tnorm represents the normalized strength matrix):

€

Ti,inorm = 0, i > 3

We then solve (4) once before the first time step in order to obtain an initial set of

firing rates. 3.3 Differential Step This step proceeds exactly as described in equation (5). See figure (9) for the implementation, and figures (10, 11) for implementations of the alpha and beta functions. During our simulation we simply used β = 0.1 (this parameter determines the effect of LTD in

comparison to LTP in the model). We set all , and use the learning rule in equation

(7). 3.4 Solving step The solving step proceeds as described in (4) (see: code in Code 7). However, the presence of inhibitory neurons raises a bounding problem in the firing rates V : a heavily inhibited excitatory neuron can occasionally display a negative average firing rate over a given time‐step, which is nonsensical (it would imply that an excitatory neuron is inhibiting other neurons, or that an inhibitory neuron is potentiating other neurons, by firing). This arises due primarily to rounding errors arising from the numerical nature of our simulation. It is remedied by simply setting all negative values in V to 0 at the end of each solving step.


of 25

Figure 8: Convergence of Firing Rates. Vertical axis: Averaged neuron firing rates. Horizontal Axis: Time progression (number of time-steps 0 to 100). Input neuron is BBB LLL UUU EEE. Output neuron is YYY EEE LLL LLL OOO WWW. Other neurons are represented by various colors. The graph displays the average firing rate at each time-step. Convergence (synaptic firing rates) of all neurons to the output neuron occurs after approximately the 50th time-step. This demonstrates in silico representation of our model can be trained to a firing pattern. [Network description: (20 neurons (N), 100 STEPS, 6 inhibitory neurons (inhib 3-9)].

Figure 9: Rates of change in Firing Rates. Vertical axis: Rates of change in neuron firing rates. Horizontal Axis: Time progression (number of time-steps 0 to 100). The rate of change of the input neuron is BBB LLL UUU EEE; represented

by the cosine function

€

v1'(t) =π16cos π

8t

; derivative of

€

v1(t) =sin t π

8

+1

2 . Output neuron is yellow. The rate of change of the input neuron is All other neurons are represented by various colors eventually converge to the YYY EEE LLL LLL OOO WWWcurve (output neuron), representing the time derivatives of all

€

vi, i >1.


of 25

3.5 Simulation Results We present the results of the simulation below. In Figure 8, we show the average firing rates of all the neurons at each time step.

We may observe several effects taking place. The non‐input neurons all slowly converge into a single firing pattern, and this convergence is complete after approximately 50 time steps. Additionally, we notice that the output neurons (as all non‐input neurons are synchronized in steady‐state, and we may consider them output neurons) converge to a periodic function containing two components: the input neuron’s sine wave, and a wave phase‐inverted to the input neuron’s. We notice that while the output neurons climb to reach the input neuron’s maxima, they also reach local minima very close to the input maxima.

This behavior can be further analyzed by observing the rates of change of neuron firing rates between each time step. Figure 9 shows a graph of these rates over the same time period.

We notice here that the rates of change of the output neurons’ firing rates are also described by a periodic function. This function’s frequency is approximately double that of the input neuron’s rate of change function. We can see that the output rates of change also reach local minima as the input reaches local maxima. Thus the output neurons have both phase‐inverted and frequency‐inverted components.

From these two results we can derive testable predictions as to the behavior of a biological neural network in similar conditions. Specifically:

• The firing rate of the output neuron will eventually stabilize to a periodic function containing components of both the input neuron firing pattern, and the learning rule reinforcement function (in this case, a phase‐inversion of the input neuron’s function). • The rate of change of the output neuron will also eventually stabilize to a periodic function such that local minima will be reached at maxima in the input neuron’s rate of change.


of 25

4 Future Directions Now that a robust, computable model has been developed we propose the following experimental design to test our computational model in the biological setting and allow its refinement8. Traditionally, disassociated neurons cultured in vitro have served as the model system for studying the dynamics of neural networks [3, 21]. One of the obstacles in studying neuron networks is the need for simultaneous recording of all neurons. Another major obstacle is the reproducibility of the shape, morphology, position, and state of the neurons which cannot be controlled with sufficient precision to ensure networks are identical. Current techniques employ multi-electrode arrays (MEAs) which can only stimulate and record electrical potentials of neurons at one place at a time. As a result, once neurons are cultured, studying the formation of neural networks requires installing MEAs in every neuron. Furthermore, the networks would never have the identical starting conditions. Therefore, we propose (see: Figure 10) using neurons (embryonic rat hippocampal neurons) cultured onto silicon wafers as described in Starovoytov et al. [21] (see: Supplement 2). The silicon wafer allows the selective stimulation of neurons by pulse laser, as well as the recording of electrical potential of neurons by detecting changes in electrical states on the silicon surface [21].

8 To date, no model exists that is truly capable of emulating the biological setting of LTP and LTD.

Supplement 2: Starovoytov Experimental Setup.

A: Culture Dish. Design of a culture dish used in electro-optical stimulation experiments. Silicon wafer is glued to the bottom of the plastic culture dish. Neurons are plated onto the exposed silicon surface in the middle opening of the dish. Extracellular solution is grounded using Ag/AgCl electrode, and the bottom of the dish is contacted by a copper electrode with InGa eutectic painted onto the back side of silicon wafer.

B: Equivalent Circuit. Ci and Ri represent, correspondingly, the capacitance and the resistance of the double layer formed at the interface of electrolyte and semiconductor. Vb denotes the applied bias voltage; Rs is the resistance of electrolyte. Additional series resistor Rm was used for total current measurements. Extracellular potential for the patched cell is determined by the conditions immediately above the silicon surface.

C: Optical arrangement. Collimating and steering optics, the AOD, and the output coupler of the fiber were all rigidly attached to the microscope and moved together when translation across the sample was required. This allowed us to keep the patch stationary at all times. The Tube and Scan lenses were introduced to lock the beam at the center of microscope objective for all deflection angles. Dash-dot and dotted lines show targeting different locations at the sample.

Reproduced from Starovoytov, et al. Journal of Neurophysiology. 93: 1090‐1098. (2005). [20]


of 25

Figure 10: Proposed Experimental Setup employing Light-Directed Electrical Stimulation (LDES) of Neurons cultured on Silicon Wafer.

1a Electrical Potential Scan: The bottom of the silicon wafer is fitted with Electric Potential (EP) probes. This informs the electrical potential states in the in silico neural network representation (2a).

1b-Visual Mapping Scan: The microscopy setup from Starovoytov et al. can be employed to enable positional mapping of the neurons in the network under study. This informs the in silico model (2a).

2a-Computer Model: Position and electrical states of neurons are compiled to make an in silico representation of the in vivo neural network. 2b Parallel Simulations: Extrapolated future states of the in silico model are correlated the real-time in vivo model. Statistical analysis is then used to refine rules governing the in silico model (2a) for prediction of future states.

2b-Parallel Simulations: 2b-Parallel Simulations: The in silico model (2a) can be copied and “stimulated” virtually to extrapolate future states. Extrapolated future states of the in silico model are correlated the real-time in vivo model. Statistical analysis is then used to refine rules governing the in silico model (2a) for prediction of future states. The stimulation should parallel stimulation that will, or is occurring in vivo by pulse laser. Statistical analysis can be performed on data sets to refine the rules governing the in silico model for future simulations (e.g., “noise” coefficients).

3-Trainig Output Data Set: The neural network must “trained.” A solution set is computed that will “push” the neuron firing rates to a desired output.

4-Modulatory Instructions: Pulse lasers are coordinated to stimulate neurons at the appropriate time.

5-Change in Neural Architecture: Modulation of in vivo neural network causes intercellular changes in gene expression in neurons from the activation of LTP and LTD and the intercellular development of a neuronal pathway. The neural architecture reflects output solution set in (3) (e.g., hand-eye coordination in cerebellum see Figure 5).

Once the neuron plates are produced, a scan over the silicon wafer can be made to determine the initial states (electrical potentials and firing rates) of the neurons. This data can then be used to supply our model with data on synaptic strengths and firing rates of the network. Then the training through stimulation of neural networks can happen both in silico and in vitro. This allows virtual and real training of neural networks to occur in parallel and the results from these trainings can be statistically analyzed between virtual predictions and real‐time data. In this way, refinements can be made to the model and new factors introduced that may alter the learning rules to include variables that are traditionally


of 25

“non‐Hebbian.” A caveat to this is that in the current model, there is no fixation of certain synaptic weights at 0 ‐ that is to say that we model all neurons as post‐synaptic to all the others, with the exception of a single (voltage‐clamped) input neuron. Thus an important future direction would be to enable a sparse matrix of weights (i.e., with some values fixed at 0) to be used in the computational simulation. In purely computational experiments, these would be distributed randomly in proportion to the ratio of synapses to pairs of neurons that occurs in biological neural networks. In parallel in vivo/in silico experiments, they would be set to emulate the in vivo network’s state of connectedness. Adding random variation or “noise” to our model ‐ either in the form of asynchronous updating of neurons as implemented in [14, 15] or as a random noise term in the synaptic weight updating rules ‐ would lend more biological fidelity to our model. In addition, the focus of our model was the generation of the rules that govern LTP and LTD with an emphasis on spike‐driven plasticity; however, this does not account for what truly occurs, at the nano‐level, in the short‐term and long‐term behavior of neural plasticity. In order to make the model more realistic, a model of the synapse should be added to include neurotransmitter, receptor, and Ca+2 levels. This would involve re‐designing the computational model to use the more modern spike‐based methods referred to in section (1.3) (For an example of this type of model and its performance, see [18], which implements a spiking neuron model allowing for complex behavior to be encoded).

Lastly, the LDES platform allows recording and modulation of neural activity of cultured neurons and thereby connectivity. Consequently, the use and development of this platform may enable the generation of artificial neural networks with tailored neural architecture for biomedical applications. 5 Acknowledgements We would like to thank the Dr. David Needham of the Pratt School of Engineering and the Center for Biologically Inspired Materials and Materials Systems (CBIMMS) for the helping to create this opportunity and making this research project possible. In addition, we would like to thank Dr. Ragachavari of the Duke Neurobiology Department for his consult and advice on the project.


of 25

6 Appendix Simulation Code

Code 1: model.m clear N=20 Inhib=zeros(N,N); Inhib(:,:)=1; Inhib(3:9,:)=-1; STEPS=100 % Connection Strengths T=normalize(rand(N,N),N,Inhib); % Values of neurons T(1,1)=1; T(1:2,2:N)=0; T(2,1)=0; T(2,2)=1; D=zeros(N,1); D(1)=.5; D(2)=.5; Delta = zeros(N,N); V=inv(T)*D; for s=1:STEPS D(1)=(sin(s*pi/(8))+1)/2; D(2)=1; Delta = dT(V,T,N,Inhib); T=T+Delta; T=normalize(T,N,Inhib); T0 = T .* Inhib; V=[V,inv(T0)*D]; V=deadneurons(V); end plot(V’);


of 25

Code 2: normalize.m % T (strength matrix) normalization function out = normalize(T,N,Inhib) Normed=T; for i=3:N % in order to avoid summing in w(i,i) terms: Normed(i,i)=0; end Rows = sum(abs(Normed’)); for i=3:N for j=1:N Normed(i,j) = Normed(i,j) / max(Rows(2:N)); end end for i=3:N % to avoid linear independence of strengths Normed(i,i)=-1; end out=Normed; end

Code 3: dT.m % T (strength matrix) differential function out = dT(V,T,N,Inhib) beta=.2; delta=zeros(N,N); for i=3:N for j=1:N alph = alpha(i,j,V,T,N,Inhib); delta(i,j)=alph*V(i,size(V,2))*V(j,size(V,2))*(1-T(i,j)) - (.1)*T(i,j); end end for i=3:N delta(i,i)=0; end out=delta; end


of 25

Code 4: alpha.m

function out = alpha(i,j,V,T,N,Inhib) % alpha=abs(V(1)-V(N)) rule out = abs(V(1,size(V,2))-V(N,size(V,2))); end

Code 5: beta.m function out = beta(alph) out = 1 - alph; end

Code 6: deadneurons.m function out = deadneurons(V) for i = 3:size(V,1) if (V(i,size(V,2)) < 0) Vend=size(V,2); V(i,Vend) = 0; endif endfor out = V; end


of 25

7 References [1] Abraham, W.C., Logan, B., Greenwood, J.M., and Dragunow, M. Induction and experience dependent consolidation of stable long‐term potentiation lasting months in the hippocampus. Journal of Neuroscience. 23: 9626‐9634. (2002). [2] Ahn, S., Ginity, D.D., Linden, D.J. A late phase of cerebellar long‐term depression requires activation of CaMKIV and CREB. Neuron. 26: 559‐568. (1999). [3] Bi, G., and Poo, M. Synaptic modifications in cultured hipppocampal neurons: dependence on spike timing, synaptic strength, and postynaptic cell type. The Journal of Neuroscience. 18(24): 10484‐10472. (1998). [4] Bi, G., and Poo, M. Distributed synaptic modification in neural networks induced by patterned stimulation. Nature. 401: 792‐796. (1999). [5] Brader, J.M., Senn, W., and Fusi, S. Learning real‐world stimuli in neural network with spikedriven synaptic dynamics. Neural Computation. 19: 2881‐2912. (2007) [6] Chung, H.J., Steinberg, J.P., Hungair, R.L., and Linden, D.J. Requirement of AMPA receptor GluR2 phosphorylation for cerebellar long‐term depression. Science. 300: 1751‐1755. [7] Dayan, P., Abbott., L.F. Theoretical Neuroscience. MIT Press: Massachusetts, USA. (2001). [8] Engert, F. and Bonohoeffer, F. Dendritic spine changes associated with hippocampal long‐term synaptic plasticity. Nature. 399: 66‐70. (1999). [9] Ermentrout, Bard. Neural networks as spatio‐temporal pattern‐forming systems. Reports on Progress in Physics. 61: 353‐430. (1998). [10] Fitzimonds, R.M., Song, H., Poo, M. Propogation of activity‐dependent synaptic depression in simple neural networks. Nature. 388: 439‐448. (1997). [11] Deneve, S. Bayesian Spiking Neurons I: Inference. Neural Computation. 20: 91‐117. (2008). [12] Gerstner W., Kistler, W. Neural networks and physical systems with emergent computational abilities. Spiking Neuron Models. Cambridge University Press, UK. (2002). [13] Hebb, Donald. O. The Organization of Behavior: A Neuropsychological Theory. Wiley – Interscience, New York. (1949). [14] Hopfield, J. J. Proceedings of the National Academy of Sciences, USA: Biophysics 79: 2554‐2558. (1982). [15] Hopfield, J. J. Proceedings of the National Academy of Sciences, USA: Biophysics 81: 3088. (1984). [16] Junge, H.J., Rhee, J., Jahn, O., Varoqueaux, F., Spiess, J., Waxham, M., Rosemund, N., and Brose, N.Calmodulin and Munc13 form a Ca+2 sensor/effect complex that controls short‐term synaptic plasticity. Cell. 118: 389‐401. (2004).


of 25

[17] Little W. A. The Existence of Persistent States in the Brain. Mathematical Biosciences 19: 101‐120. (1974). [18] Lovelace, J., Cios, K. A very simple spiking neuron model that allows for modeling of large, complex systems. Neural Computation. 20: 65‐90. (2008). [19] Mascagni, M., Sherman, A. Numerical methods for neuron modeling. Methods in Neuronal Modeling � Second Edition, ed. Koch C., Segev I. MIT Press: Massachusetts, USA. (2001). [20] Purves, D., Augustine, G.J., Fitzpatrick, D., Hall, W.C., LaMantia, A., McNamara, J.O., White, L.E. Neuroscience: Fourth Edition Sinauer Associates, Inc: Sunderland, Massachusetts, USA. (2008) [21] Starovoytov, A., Choi, J., Seung, H.S. Light‐directed electrical stimulation of neurons cultured on silicon wafers. Journal of Neurophysiology. 93: 1090‐1098. (2005). [22] Zemel, RS, Huys, QJM, Natara jan, R., Dayan, P. Probabilistic computation in spiking populations. Advances in Neural Information Processing Systems. MIT Press: Massachusetts, MA (2004). [23] Orlando, D.A., Lin, C.Y., Bernard, A., Wang, J.Y., Socolar, J.E.S, Iversen, E.S., Hartemink, A.J., and Haase, S. Global Control of cell‐cycle transcription by coupled CDK and network oscillators. Nature. 453: 944‐948. (2008). [24] Strange, K. The end of “naïve” reductionism”: rise of systems biology or renaissance of physiology? American Physiological Society Cell Physiology. 288: C968‐C974. (2005).

paper egr 183 modeling neural networks in silico

Health & Medicine