draft

1

Theoretical model for mesoscopic-level scale-freeself-organization of functional brain networks

Jaroslaw Piersa, Filip Piekniewski, Member, IEEE CIS and Tomasz Schreiber

Abstract—In this paper we provide theoretical and numericalanalysis of a geometric activity flow network model which is aimedat explaining mathematically the scale-free functional graph self-organization phenomena emerging in complex nervous systemsat a mesoscale level. In our model each unit corresponds to alarge number of neurons and may be roughly seen as abstractingthe functional behavior exhibited by a single voxel under thefMRI imaging. In the course of the dynamics the units exchangeportions of formal charge which correspond to waves of activity inthe underlying microscale neuronal circuit. The geometric modelabstracts away the neuronal complexity and is mathematicallytractable which allows us to establish explicit results on itsground states and the resulting charge transfer graph modelingfunctional graph of the network. We show that for a widechoice of parameters and geometrical set-ups the model yieldsa scale-free functional connectivity with exponent approaching2 in agreement with previous empirical studies based on fMRI.The level of universality of the presented theory allows us toclaim that the model does shed light on mesoscale functionalself-organization phenomena of the nervous system, even withoutresorting to closer details of brain connectivity geometry whichoften remain unknown. This is a continuation of our previouswork where a simplified mean-field model in a similar spirit wasconstructed, ignoring the underlying network geometry.

Index Terms—scale-free, geometric neural network, mesoscale,mesodynamics

I. INTRODUCTION

SCALE-FREE self-organization has attracted considerableamount of attention in statistical mechanics, physics and

mathematics. Power laws in degree distributions have beenreported in various networks ranging from the World WideWeb [1], science collaboration networks [2], citation networks[3], ecological networks [4], linguistic networks [5], cellularmetabolic networks [6], [7] to telephone call network [8],[9] etc. This subject is also tempting for neural network andneuroscience community. There are various reports suggestingperformance gains with artificial NN built on power law andsmall world graphs [10]–[12] and more research is in progress.The studies of biological neural network connectivity of C.elegans worm revealed exponential rather than polynomialdecay in degree distribution [13], [14], however the wormhas only about 300 neurons total and therefore the self-organization in the sense discussed in this paper is very limited

J. Piersa, F. Piekniewski and T. Schreiber are affiliated with the Faculty ofMathematics and Computer Science, Nicolaus Copernicus University, Torun,Poland. This research has been supported by the Polish Minister of ScientificResearch and Higher Education grant N N201 385234 (2008-2010). Thework of J. Piersa has also been supported by the European Social Fundand Government of Poland as a part of Integrated Operational Program forRegional Development, ZPORR, Activity 2.6 ”Regional Innovation Strategiesand Knowledge Transfer” of Kuyavian-Pomeranian Voivodeship in Poland.

Manuscript received ???; revised ???.

in this case. It would also be surprising to find a power lawconnectivity on the level of single neurons due to physicallimitations - similarly the Internet on the level of hardware isnot a scale-free network since there are no physical devicesthat could connect thousands of computers.

However, recent advances in medical imaging and dataprocessing techniques have provided very strong hints thatpower law connectivity migt constitute an important aspect ofself-organization and dynamics of far more complex (mam-malian like) nervous systems. Whereas we refer the readerto [15], [16] for an extensive review, here we only brieflyrecapitulate the general picture emerging from this interdis-ciplinary research. In general, both the structural (anatomic)and functional (based on observed activity profile) networks inhuman and higher animal brains are found to exhibit highlystructured hierarchic architecture of small-world type (smallmean path-lengths and high local clustering), see ibidem andthe references therein, yet not always are these networks scale-free (exhibiting power law in degree distribution). Among var-ious network types studied, depending on the medical imagingand data processing methodology, our interest in this paper isfocused on voxel-based functional networks arising in fMRIbrain observations [17]–[20] where scale-free architecture hasbeen reported, often with exponent approaching 2, both in therest state and in the course of some task execution. This powerlaw gets exponentially cut off or even breaks down on largerscales, for instance with anatomical regions playing the role ofnodes, [21], [22] which is apparently due to the scale differ-ence and to system’s physical limitations, see the discussionin [15, p.192]. In fact, it should not be considered surprisingthat the scale-freeness, whose mathematical definition involvessystem sizes tending to infinity, at the level of large but finitesystems is encountered only in approximate form, moreoverit is also very natural that the finite-size corrections arising(for instance the exponential cut-off) become more significantas the system size decreases, for instance due to decreasingresolution, as reported e.g. in [23]. This is a usual situationin statistical physics where the quality of approximation of acomplex system’s behavior by its mathematical idealizationderived in the thermodynamic limit essentially depends onsystem’s size and scale. However, determining the preciserange of characteristic spatial and temporal scales as well as ofdata processing schemes where the scale-free approximation ofbrain’s functional topology is valid, seems to be an importantopen question upon which many competing opinions exist, seeagain [15, p.192].

The research triggered by these and related empirical find-ings has often been complemented by computer simulations

draft

2

[24]–[26]. Interestingly, also in some simulation studies basedon phenomenological neuron model by E. Izhikevich [27], thepresence [25] or absence [28, Chapter 6] of scale-freenessstrongly depended on the definition of network’s constituentnodes.

An important step in theoretical understanding of thesescale-free self-organization phenomena was made in [29],where it is argued and shown by a numerical study thatmesoscopic scale brain functional networks exhibit in manyaspect behavior typical for the Ising model at criticality. Thisis further supported by numeric study in [30] where a furthermodel due to Kuramoto is also considered at criticality.

The empirical findings quoted above have so far lackeda satisfactory dedicated theoretical model, since the tradi-tional ways of constructing scale-free networks [31]–[33]were based on the concept of growth and preferential at-tachment/duplication - ideas well suited for networks like theInternet, citation graph or even the molecular networks, butnot necessarily appropriate for the nervous system. The meanfield spike flow model introduced in [34] and further studiedin [28], [35] was the first mathematically tractable approach toprovide such a theoretical background. The model is intendedto describe functional networks arising on mesoscopic spatialscales (at the level of fMRI voxels, possibly also on the finerscale of local assemblies of cortical microcolumns, but not onthe scale of individual neurons) and mesoscopic time scales,long enough to allow for system’s relaxation but short enoughto safely assume fixed environment of the dynamics (quencheddisorder). It is important to emphasize that on the chosenmesoscale each functional node is composed of thousandsof neurons and possibly a number of cortical minicolumns,never of just individual neurons. As we proved mathematicallyin [35], our model does indeed result in a power law graphwith exponent equal 2 (in agreement with many empiricalfindings as quoted above) based solely on its dynamicalfeatures which reflects the charge exchanges/activity flow ina neural network on a medium (meso) scale. In this paperwe present an important geometric extension to this spikeflow model, which goes significantly beyond the mean-fieldsetting and introduces geometric distances between the unitsthus making the model a lot more feasible for the descriptionof the real nervous system by embedding it in a physicalspace. We show mathematically and present numeric evidencethat also this spatially embedded model does give rise topower law connectivity. In the course of theoretical analysiswe also identify finite range distortions marking the validityregion of the power law in finite systems. Strikingly, notonly is the power law exponent again 2 here, but undercertain reasonable regularity conditions it does not dependon parameters of the geometric embedding! These criticalityand universality features stand in strong conceptual agreementwith [29], [30], although the models considered there pertainto a different spatial scale – an individual state variable inthe Ising model is {+1,−1}-valued, in Kuramoto’s model ittakes values in the unit circle, whereas in our model statevariables are unbounded and can be any natural numbers thuscorresponding to more complex entities. Also, the criticality ofour model is self-organized (not driven by parameter tuning) as

opposed to the cases discussed in [29], [30] where the dynamiccriticality is reached by manipulating the control parameters.Moreover, our model evolves under Kawasaki dynamics asopposed to the spin flip Glauber dynamics considered in [29].Some further discussion of these differences and relationshipsbetween our model and [29], [30] will be given at the endof Section IV although we will not go far beyond conceptualcomparisons there because these models work on essentiallydifferent scales.

The rest of the paper is organized as follows: in section IIwe describe our basic model which constitutes an importantenhancement of the spike flow model introduced previously, asbeing now equipped with geometrical connectivity of a rathergeneral type. We then analyze the ground states and provide asimple description of the asymptotic dynamics in section III.The crucial section IV contains the most important results ofthe paper related to the emerging scale-free connectivity. Insection V we equip the model with additional long range con-nections which might correspond to long myelinated neuronalfibers connecting remote sites of the cortical surface. SectionVII contains a number of numerical results supporting thediscussed claims. Finally we conclude the paper and providereferences.

II. THE BASIC ISOTROPIC MODEL

The spin glass type model constructed in this section isaimed at abstracting in a mathematically tractable way theessential features determining the mesoscopic level geometryof spiking activity in recurrent neural nets. This was firstdone in a mean-field setting in [35] where we studied amodel built on a complete graph, with all constituent unitsfully interconnected. Here we go significantly beyond themean-field analysis and introduce a rather flexible geometricset-up with a considerable amount of degrees of freedom.For definiteness we construct our system on a (unit) sphereSd−1 ⊂ Rd, referred to as the spatial domain of the modelin the sequel, and usually taken to be S2 in our simulations.For intensity parameter λ > 0 we write Pλ to denote thehomogeneous Poisson point process on Sd−1 with intensityλ, modeling the spatial locations of the units constituting oursystem. The intensity λ will always be taken large in oursimulations, which is reflected by letting λ → ∞ in ourtheoretical argument. Each unit x ∈ Pλ, sometimes also calleda neuron by a convenient abuse of terminology, is endowedwith its state variable σx ∈ N := {0, 1, 2, . . .}. In termsof the original neural network the σ′xs are to be regarded asan abstraction of activity levels of coarse-grained mesoscopicportions of the network at the corresponding spatial locationsdistributed according to Pλ. An alternative interpretation canbe also provided under the dynamical system description of theneurodynamics, where σx can be regarded, on a mesoscopictime scale, as abstracting the (fraction of) time spent by thesystem within a behavioral pattern locally manifested at x. Forterminological convenience, in the sequel we shall refer to σxas to the charge stored at x, which makes our developed modelinto a charge flow model. The charge conservation propertyenjoyed by the stochastic dynamics we construct below, admits

draft

3

a natural motivation in terms of both abstract interpretationsof σx discussed above, corresponding respectively to totalactivity conservation and the fact that system sojourn timessum up to a constant. To proceed, we consider a connectivityfunction g : R+ → [0, 1] and for each pair x, y ∈ Pλ weindependently place a connection (edge) between x and y withprobability g(|x− y|) and leave these units unconnected withthe complementary probability. We write x ∼ y to denote thefact that x and y are connected by an edge. Note that the so-constructed connectivity structure of our system corresponds toa classical random connection model considered in stochasticgeometry [36]. To each edge joining x ∼ y ∈ Pλ, denotedbelow by exy, we independently assign a standard Gaussianweight wxy ∼ N (0, 1), where, in intuitive terms, a positivewxy should be interpreted as a propensity of x and y tosynchronize and thus to exhibit similar activity levels whereas,conversely, a negative wxy indicates an inhibitory interactionbetween x and y. The energy function of the system (theHamiltonian) is given by

H(σ) :=∑x∼y

wxy|σx − σy|. (1)

The initial configuration of the network is constructed byendowing each unit with a constant initial charge α ∈ N. Next,the system evolves under a standard Kawasaki-type charge-conserving dynamics. At each step we randomly choose apair of connected units (σx, σy), x ∼ y ∈ Pλ and denoteby σ∗ the network configuration resulting from the originalconfiguration σ by decreasing σx by one and increasing σyby one, that is to say by letting a unit charge transfer fromσx to σy, whenever σx > 0. Next, if H(σ∗) ≤ H(σ) weaccept σ∗ as the new configuration of the network whereasif H(σ∗) > H(σ) we accept the new configuration σ∗

with probability exp(−β[H(σ∗) − H(σ)]), β > 0, andreject it keeping the original configuration σ otherwise, withβ > 0 standing for an extra parameter of the dynamics, inthe sequel referred to as the inverse temperature conformingto the usual language of statistical mechanics and assumedfixed and large (low temperature) throughout. Observe thatthe sum

∑x∈Pλ σx of neuronal charges is preserved by the

dynamics and that, in the course of dynamics with some initialconfiguration σ0, any other σ with

∑x∈Pλ σ

0x =

∑x∈Pλ σx

is eventually reached with positive (although possibly verysmall) probability. Recall that our default initial configurationchoice is σ0

x ≡ α. Consequently, upon standard verificationof the usual detailed balance conditions, we readily see thatthe collection of stationary states of the above dynamics areprecisely the distributions

Pn(σ) =

exp(−βH(σ))∑

σ′,∑

xσ′x=n

exp(−βH(σ′)), if

∑x σx = n,

0, otherwise(2)

and their convex combinations. In particular, upon forgettingits geometric structure, our model bears formal resemblanceto the usual stochastic Boltzmann machines [37] (underKawasaki dynamics) and inherits the usual intuitive interpreta-tion of their energy function, with the weights wxy indicatingthe extent to which the system favors the agreement (for

positive wxy) or disagreement (for negative wxy) of the activitylevels σx and σy as already mentioned above. In spite ofthese formal analogies our model does nevertheless differfrom Boltzmann machines in a very essential way due to theunbounded state space N for σ′xs. For the network dynamicsrunning during a long time period [0, T ] we are now in aposition to define the spike flow graph to be a directed graphwith vertices corresponding to the units σx, x ∈ Pλ and whoseedges carry numbers (edge multiplicities) Fx→y indicatinghow many times in the course of the dynamics the chargeflow occurred from σx to σy, x, y ∈ Pλ. As we will see, ifβ is very large, which is always going to be assumed in thispaper, after a long enough simulation run the system freezesin some ground state whereupon any further potential flowbecomes very rare and consequently the numbers Fx→y alsoeffectively freeze undergoing virtually no further changes. Wesay that the dynamics jams or saturates in this case and wealways consider Fx→y at saturation in the sequel, the way ofdetecting saturation is discussed at the very end of the nextSection III, see condition [Saturation] there. The in-degree ofa unit σx is now defined as din(x) :=

∑y∼x Fy→x

The crucial question considered in this paper is whether theso-defined spike flow graph is scale-free in that its in-degreedistribution follows a power law, that is to say P(din(x) ≈k) ∼ cink−γin for a randomly picked node x ∈ Pλ. Below weshall establish a positive answer to this question. It should benoted at this point, as will become clear from our discussionbelow, that the asymptotic behavior of the corresponding out-degree distribution is the same as that of the in-degrees.

III. WINNER-TAKE-ALL DYNAMICS AND GROUND STATES

It turns out that the long time scale dynamics of our basicmodel admits a very convenient approximation in terms of asimpler winner-take-all dynamics introduced in [35] as soonas λ is large enough, which mathematically corresponds totaking λ → ∞. Our argument here is a version of that givenin Section 3 of [35] specialized for our present setting, werefer the reader to [35] for further details. First, we define thesupport Sx of a unit at x ∈ Pλ

Sx := −∑y∼x

wxy. (3)

Assume now we run our charge-flow dynamics for some longenough amount of time to get close to equilibrium, whereuponwe consider a small number o(N) of units which store thehighest charge, considerably higher than the remaining units,and we call these elite units/neurons while granting the termbulk units/neurons to the remaining units in the system. Sincethe cardinality of elite is a negligible fraction of N, the formula(1) becomes then

H(σ) ≈ −∑x∈elite

σxSx +12

∑x,y∈bulk

wxy|σj − σl|.

Next, we note that whenever in the course of the networkdynamics a charge transfer is proposed from a bulk unit σyto an elite neuron σx, y ∼ x, the resulting energy change isseen to be well approximated by −Sx plus a term due to the

draft

4

interaction between σy and other bulk units. In general, wehave no control of this term, yet if σx is one of the neuronswith the highest support in the system, this offending termis very likely to be negligible compared to −Sx thus makingthe energy change strongly negative and the proposed transferextremely likely to be accepted. Clearly, the inverse transferbecomes then almost impossible. Consequently, whenever aunit with a very high support enters the elite, it virtually neverleaves it; moreover it continuously drains charge from the bulklosing it only to other elite members if at all. Furthermore,should a unit with a small support value happen to enter theelite at the early stages of the dynamics, it will soon leaveit having its charge drained by other higher supported units.Thus, after running our dynamics long enough we end up witha picture where the elite consists of units with the highestsupport. Although the elite neurons do struggle for chargebetween themselves, they cooperate in draining it from thebulk. Therefore eventually almost no charge will be present inthe bulk and hence the Hamiltonian will admit a particularlysimple approximation

H(σ) ≈ −∑x∈elite

σxSx (4)

and all further updates in the system will only happen dueto charge transfers within the elite. Note now that, sincethe cardinality of the elite is o(N), for each elite neuronits interactions with the remaining elite units are effectivelydominated by those with the bulk. Thus, keeping in mindthat the inverse temperature β is very large, the dynamicsbetween the elite neurons takes eventually a particularly simpleform: a pair x ∼ y of connected elite neurons is chosen byrandom and if the one with smaller support attempts to transfera unit charge to the one with higher support, the attemptis accepted, otherwise it is rejected. Clearly, this dynamicseffectively terminates by storing all available charge in neuronswhich are not connected to any higher support units, call theseground units. Although formally the only ground state of thesystem is obtained by putting all charge into the unit of thehighest support, which clearly is a ground neuron, we willslightly abuse the terminology and grant the name of a groundstate to each configuration where all charge is stored in groundunits – note that this makes a good sense in the λ → ∞asymptotics where the potential barriers between these groundstates become impassable for the dynamics. It should be notedthat at intermediate stages of the dynamics it may happen thatelite members show up with charges whose order is inverse tothat determined by the supports rather than consistent with it.This is a metastable artifact due to the fact that if we admittednegative charges here, a twofold sign-flip symmetry would bepresent in the system in full analogy to usual networks with noexternal field and such inverse ordering would compete withthe standard one on equal rights. This is not the case herethough because negative charges are not allowed and thereforesuch inversely ordered structures are temporarily stable and donot persist in the course of the dynamics.

In view of the above discussion, the highest in-degrees ofthe spike-flow graph are observed in elite units enjoying thehighest support from the system, and the corresponding charge

flows Fi→j are mainly due to the internal charge transferswithin the elite. Thus, our discussion shows that the asymptoticλ → ∞ behavior of the charge flow graph generated byour network model is accurately described by the followingsimplified winner-take-all (WTA) model:• The system consists of gas of elite units with spatial loca-

tions determined by homogeneous Poisson point processP% on Sd−1 of intensity 1� %� λ.

• Each unit located at x ∈ P% has its state variable denotedas before by σx.

• Each unit located at x ∈ P% has its support mark µxchosen uniformly in [0, 1] independently of the locationsand remaining supports of units constituting the system.The neuron σx is declared to have higher support thanσy, x, y ∈ P% iff µx > µy.

• α card[P%] = Poisson(α ς(Sd−1)) units of charge aresequentially introduced into the system, each time ac-cording to the following dynamics

– first, a unit charge is transferred to a randomlychosen neuron σx0 , x0 ∈ P%,

– thereupon it starts jumping to subsequent neuronsσx1 , σx2 , . . . where xl+1 is randomly chosen amongy with xl y, that is to say among y ∼ xl withµy > µxl . Whenever a jump from σxl to σxl+1

is performed, the charge transfer counter Fxl→xl+1

gets incremented by one. These charge countersapproximate the corresponding charge flow counts inthe original model. An additional counter F⊥→x isalso considered storing the number of units initiallyassigned to σx.

– eventually the unit charge reaches a ground unit andgets frozen there.

• the in-degrees of the elite neurons in the original network,given by din(x) =

∑y∼x Fy→x are approximated by the

numbers Dx = F⊥→x +∑y∼x Fy→x indicating how

many charge units have visited σx on their way to aground neuron.

In other words, in this model the charge transfers alwaysoccur from a neuron with smaller support to a randomlychosen better supported one to which a connection is available,whence the term winner-take-all dynamics. A few words aredue to explain the meaning and assignment mechanism of thesupport marks. Roughly speaking, our intention is to have µxsuch that

dµx card[P%]e (5)

stand for the number the unit σx occupies in the hierarchyof neurons ordered by ascending supports. Assigning i.i.d.uniform marks to units in P% is equivalent to making allsupport orderings equiprobable, which is natural in view of therotational symmetry of the system, implying equidistributionof support marks, and in view of the fact that the supports Sxand Sy as defined in (3) for two different units σx and σy canonly share wxy (if at all) among their constituent weights andthus are effectively independent given P%, likewise for higherbut fixed cardinality collections, which asymptotically grantseffective independence of the support marks. Clearly, this

draft

5

construction yields an approximation of the original dynamics,which is only statistically valid, but this is sufficient for ourneeds:• the winner-take-all rule happens to be violated, with very

small probability though,• the independence of supports given P% is only approx-

imate, but this approximation is very good in large %asymptotics,

• the supports Sx and hence µx are in general not indepen-dent of the spatial configuration P% as determining thenumber of connections at x, and thus µx should be madedepend on P%, however this effect can be neglected forour purposes in large % asymptotics as it vanishes alreadyupon local spatial averaging (mesoscopic-level coarse-graining) whereas we mainly consider global averagecharacteristics determining the scale-free geometry ofcharge flow graphs.

We conclude this section by noting that as a by-productof the discussion above we get a clear criterion determiningwhether the dynamics of our basic model has reached satura-tion. This is

[Saturation] The system has reached saturationwhen all charge is stored at ground units.

IV. SCALE-FREE SELF-ORGANIZATION

The purpose of the present section is to study the degreedistribution of the charge flow graph generated by our chargeflow model. To this end, we will analyse the behavior ofexceedance probabilities of the type

G(k) := P(din(x) ≥ k) (6)

where x is a randomly chosen point in Pλ and where both kand λ are large. Mathematically this corresponds to takingλ → ∞ and k → ∞ and thus all our theory below isto be understood as statements about the limits as suitableparameters tend to ∞. The requirement that k be large allowsus to restrict our attention to the elite units evolving under thewinner-take-all dynamics discussed in Section III. Indeed, thein-degrees of the bulk units are in usually relatively low andthus the bulk neurons can be neglected in our considerations.Consequently, it is enough to study the winner-take-all modelwith spatial locations distributed according to P%, as discussedabove. To proceed with our analysis, to each point x ∈ P% weascribe its visiting intensity Φ(x) set to be the expected valueof the number Dx of charge units visiting σx on their way to aground neuron, given the configuration P%. Then it is readilyseen by the definition of the WTA dynamics that

Φ(x) = α+∑y x

Φ(y)card[N↑(y)]

(7)

where y x stands for y ∼ x and µy < µx whereasN↑(y) := {z ∈ P%, y z}. The relation (7) allows usto evaluate Φ(x) recursively starting from the minimal pointsof the -induced order, that is to say points y for whichN↓(y) := {z ∈ P%, z y} is empty, and proceeding up-wards in the support hierarchy towards the maximal elementscorresponding to the ground neurons of the system. Writing

1x y for the random variable taking value 1 if x y and 0otherwise we can rewrite (7) as

Φ(x) = α+∑y∈P%

1y xΦ(y)∑z∈P% 1y z

and recalling that the random variables 1(·) (·) are all inde-pendent by the construction of the random-connection graph,we conclude that the collection of (Φ(x))x∈P% exhibits in large% asymptotics strong self-averaging properties which, togetherwith the spherical symmetry of the system, implies that thevalue of Φ(x) effectively only depends on the value of thesupport mark µx, that is to say

Φ(x) ≈ φ(µx) (8)

where φ(·) is some deterministic function. Moreover, for given(x, µx) and y ∈ P% with µy unknown we have

P(y x) = µxg(|x− y|), P(x y) = (1− µx)g(|x− y|)(9)

whereas for µy known we get

P(y x) = 1µy<µxg(|x− y|). (10)

Hence

card[N↑(x)] ≈ %(1−µx)∫Sd−1

g(|x−y|)ς(dy) = %(1−µx)γ

(11)with γ :=

∫Sd−1

g(|x − y|)ς(dy) not depending on x byisotropy of the system and where ς(·) stands for the sphericalsurface measure. Note that although (11) is valid for allµx ∈ (0, 1) in large % asymptotics, for a given % we shouldhave %(1 − µx) � 1 so that the self-averaging applies.Consequently, putting (7) together with (8,9,10) and (11) weare led to

φ(µx) = α+ %

∫Sd−1

∫ µx

0

g(|x− y|) φ(µy)%(1− µy)γ

dµyς(dy) =

α+∫ µx

0

φ(y)(1− µy)

dµy

[∫Sd−1

g(|x− y|)ς(dy)

γ

]for 1− µx � %−1. Recalling now the definition of γ we endup with

φ(s) = α+∫ s

0

φ(t)1− t

dt, 1− s� %−1, (12)

and thus, upon solving,

φ(s) =α

1− s, 1− s� %−1. (13)

To proceed, we use the fact that the average number of pointsx ∈ P% with µx ∈ [s, s + ds] is %ς(Sd−1)ds, whereas theoverall mean number of points in P% is %ς(Sd−1), whence theaverage fraction of points with support marks in [s, s + ds]is simply ds. Moreover, we note that by the self-averagingproperty of our system, as % → ∞ we have Dx ≈ Φ(x) ≈φ(µx). Therefore, using (13) and recalling (6) we come to

G(k) ≈ P(Dx ≥ k) ≈∫{s, φ(s)≥k}

ds = α/k. (14)

draft

6

This is the cumulative version of the statement thatP(din(x) ≈ k) ∝ k−2 with x standing for the random point inP%. Thus, we have established the first main theoretical resultof this paper.

Theorem 1: The charge flow graph induced by our basicisotropic model is, in large λ and % asymptotics, scale-freewith exponent 2, with overwhelming probability.

Note that this theorem supports the viewpoint presented in[29], [30] where universality classes of statistical mechanicalmodels at criticality have been advocated as providing possibleexplanation of crucial properties of brain functional networks.Clearly, there are important differences in mathematical detailsbetween our model and the approach presented there, whichare due to different spatial scales – the models in [29], [30]refer to simple state variables whereas we are dealing withunbounded state variables corresponding to more complicatedmesoscopic units abstracting neuronal ensembles (for instanceat the level of fMRI voxels, possibly also on a lower scaleof groups of cortical microcolumns) rather than individualneurons. Moreover, the models used in [29], [30] achievecriticality upon adjusting suitable order parameters whereasour model exhibits self-organized criticality as the power lawin Theorem 1 does not depend on any order parameters.However, at the conceptual level there is a good agreement.In this context it is worth noting for instance the presence ofmetastable inverted configurations in the course of the basicmodel’s dynamics as discussed just above the formal definitionof the WTA model in Section III. Likewise, it should beobserved that although the Winner-Take-All dynamics itselfis very robust, the notion of the winner is not – the systemwill unfold according to the asymptotic WTA description foroverwhelming majority of disorder (weights) configurations,but the support values determining the corresponding supportmarks and thus the WTA hierarchy are themselves extremelysensitive to the disorder. Moreover, as observed, discussed andpresented in figures in Subsection VII-A, the ground unitsgive rise to local basins of attraction which exhibit certainlevel of geometric contiguity and which consist of units whoseactivities correlate in the course of the dynamics. All thesefeatures are again in agreement with critical properties of brainfunctional networks in focus of [29]. Questions related to thisfact are the subject of our ongoing work in progress.

A careful reader might wonder at this point why we didnot mention above one further apparently crucial differencebetween our model and the approach in [29], namely that ourmodel here evolves according to charge-conserving Kawasakidynamics whereas there the Ising model is let evolve underGlauber spin flip dynamics. The point is that, as can be seen inthe proof of Theorem 2 and especially in (18), if we relax thecharge conservation the model still exhibits scale-free behavioralthough with a different exponent. In other words, withinreasonable limits the particular choice of the dynamics is notcrucial for the scale-freeness property, although clearly thevery nature of our charge-flow model requires that at eachstep two units be chosen, one as the source and the otheras the destination, which excludes for instance any kind ofGlauber-type dynamics.

We conclude this section by making one further remark. Allour results stated in this paper have the form of limit theorems,that is to say statements about limits when suitable parameterstend to ∞. Although we have some partial knowledge aboutthe speed of this convergence for WTA regime, obtained usingmeasure concentration techniques, see proofs of Corollaries 1and 2 in [38], and although we are able to estimate certainfinite size corrections, see (12) and (13), in general currentlywe do not know precisely how large λ should be so that ourasymptotic results provide a good approximation. However, avery practical answer to this question is given in SubsectionVII-A where the system sizes and simulation lengths turnedout to be large enough to ensure validity of our theoreticalapproximation, see Table I.

V. HANDLING LONG RANGE CONNECTIONS

Our basic model admits a natural extension to neatly dealwith long range random connections. These connections areassumed to occur along strongly elongated myelinated axons(covered by myelin sheaths) and their impulse propagationtime scales are larger than that characteristic for the meso-scopic self-organizing activity of the network considered inour basic model. Consequently, from the viewpoint of themesoscopic time scale in the focus of our attention, the spiketransfers along long myelinated axons do not participate inthe self-organizing activity of the network and are locallyfelt just as a random excitatory noise. Note however thattheir influence can be important and even crucial for somehigher order self-organization on larger time scales, which fallsoutside the scope of this paper though. In view of the above,the effect of long axonal connections in our mesoscopic settingis modeled by gradual fading or even suppressing the contribu-tion of these spike transfers to dynamic self-organization andregarding them just as a long range propagation mechanismfor excitatory noise. In terms of our models it amounts tothe following modification to their construction and dynamics,with the WTA version approximating the asymptotic behaviorof the original dynamics by argument fully analogous to thatgiven in Section III for our basic set-up:

• We fix a parameter δ > 0 and from each neuron σxin the system we emit a mean δ number of exceptionalconnections, modeling the long axons, with target unitsindependently chosen from law κ(|x − y|)ς(dy), where∫Sd−1

κ(|u|)ς(du) = 1. The newly created exceptionalconnection between two units already connected replacesthe previous usual connection. The fact that there isan exceptional connection between units σx and σy isdenoted by x⇔ y.

• In both the original model and its winner-take-all approx-imation, whenever a charge transfer is attempted fromx to y with x ⇔ y, this attempt is always succesful,respectively regardless of the sign of the resulting energychange and regardless of whether µy > µx or not.

Note that the standard way of constructing the exceptionalconnections with properties required above is to independentlyplace an exceptional connection between each two units σx, σy

draft

7

• with probability δκ(|x−y|)λς(Sd−1) for x, y ∈ Pλ in the basic

model,• with probability δκ(|x−y|)

%ς(Sd−1) for x, y ∈ P% in the WTAmodel.

In both cases, the obtained cardinality of exceptional connec-tions outgoing from an individual unit is precisely Poisson(δ)as required.

Unlike in our basic set-up, here with the exceptional con-nections present the dynamics in general does not terminate.Thus, to avoid infinite weights on the edges of the charge flowgraph, we resort to a technical trick: choose a constant ε� 1and, both for the original model and its WTA approximation,each time a charge transfer is attempted, prior to this anadditional survival test is applied, passed with probability(1 − ε) and failed with the complementary probability ε. Acharge unit having failed such a test is discarded from thesystem, whereas a unit having passed continues its evolution.The correct behavior of the system is recovered upon lettingε→ 0. Writing x⇒ y iff either x y or x⇔ y and puttingN⇑(y) := {z ∈ P%, y ⇒ z} we see that, in full analogy to(7), here we have

Φ(x) = α+ (1− ε)∑y⇒x

Φ(y)card[N⇑(y)]

. (15)

Since the difference of cardinalities card[N⇑(y)] andcard[N↑(y)] is of order O(δ) = O(1) by our construction(recall that δ stays fixed as % varies), their asymptotics areequivalent as long as both these expressions are � δ, thatis to say as long as %(1 − µx) � δ. Moreover, for similarreasons, summing over y ⇒ x rather than over y x as in (7)above, makes non-negligible difference only if %µx = O(δ)(recall that %µxγ is the expected cardinality of N↓(x), cf.(11)). Consequently, as in (12) we get

φ(s) = α+(1−ε)∫ s

0

φ(t)(1− t)

dt, %s� δ, %(1−s)� δ, (16)

whence

φ(s) =C

(1− s)1−ε , %s� δ, %(1− s)� δ, (17)

where the order of the constant C can be readily estimated asfollows. We check how many charge units may, on average,visit a minimal point x ∈ P%, that is to say a point x with noy x, y ∈ P%, whence in particular µx ≈ 0. Clearly, at theinitial stage of the dynamics we can expect visits of mean αcharge units with their initial assignment at x. Next, each ofthe O(α%) units present in the system survives on average 1/εjumps (geometric lifetime with mean 1ε), at each stage havingthe probability O(δ/%) of hitting a vertex connected to x byan exceptional connection, and the probability O(1/(γ%)) offollowing this connection. This yields C ≈ φ(0) = α+O(α% ·1/ε·δ/%·1/(γ%)) = α(1+O(δ/(εγ%))). Assuming that ε%� 1we end up with C ≈ α and thus, as in (14),

G(k) ≈∫{s, φ(s)≥k}

ds ≈ (α/k)1/(1−ε). (18)

Letting %→∞ and ε→ 0 yields therefore

Theorem 2: The charge flow graph induced by the modifiedmodel with long axonal delays is, in large λ, % and small εasymptotics, scale-free with exponent 2, with overwhelmingprobability.

VI. NON-HOMOGENEOUS SET-UP

Consider now a situation where the distribution of neuronalspatial locations is no more isotropic and neither is their con-nectivity function. We argue below that also in these cases thescale free behavior of the charge flow graph is usually presentand the power law exponent is again 2. Our argument is quitesketchy because the considered general scenario includes aplethora of specific models, yet for many of them the followingargument is valid in large density asymptotics. For simplicitywe do not introduce exceptional connections here and we placeourselves in context of the approximating WTA dynamics,usually available under reasonable regularity conditions butwith support marks no more identically distributed and, tothe contrary, with their laws strongly depending on spatiallocations. Requiring that (5) be asymptotically valid, we seethat we keep the property that the fraction of neurons withsupport marks in [s, s+ds] is ds. Next, we write γ(t, s), t < s,for the probability that for randomly chosen x with µx = tand y with µy = s we have x y – whereas in the isotropiccase we always had γ(t, s) ≡ γ/ς(Sd−1), here this is no morethe case and the function γ(s, t) may exhibit quite non-trivialbehaviors. Repeating the argument leading to (12) we get thefollowing equation, valid under many reasonable collectionsof regularity conditions:

φ(s) = α+∫ s

0

φ(t)γ(t, s)∫ 1

tγ(t, u)du

dt = α+∫ s

0

φ(t)η(t, s)

dt, (19)

where

η(t, s) :=

∫ 1

tγ(t, u)duγ(t, s)

, t < s.

In the isotropic case we always had η(t, s) ≡ 1− t, which isno more valid in the general non-homogeneous situation. Asimple analysis of the integral equation (19) shows that thebehavior of φ is usually determined by the properties of η,and thus of γ, in the vicinity of (1, 1). Further, the intuitivelynatural situation is that the limit lims,t→1 γ(s, t) exists and ispositive. We have then η(t, s) ≈ 1− t for t close to 1, whichdoes again yield G(k) ∝ k−1 and hence the charge flow graphis scale-free with exponent 2. Clearly, in general the equation(19) does also constitute a hint how to construct a ’anomalous’system where the scale free property of the charge flow graphfails. However, we believe that these are exceptional situationsand we are not aware of natural systems where such anomalieswould occur.

VII. NUMERICAL SIMULATIONS

A. Geometric model

In this subsection we present results of numerical simu-lations for the geometric spike flow model. We work withthe basic isotropic set-up introduced in Section II. Neuronswere randomly placed on two-dimensional sphere with radius

draft

8

varying up to 20. The expected density is 10 nodes per squareunit of surface, that gives up to 50000 neurons. The synapticconnectivity function g(·), as considered in Section II, isdefined here as

g(r) =

{1 r ∈ [0, 1)1rα r ≥ 1 (20)

where alpha is fixed at value α = 2.5. In particular, thisensures that the network is connected with overwhelmingprobability. The power-law form of the connectivity functionwas chosen motivated by qualitative empirical reports in [17].It should be recalled at this point that although the real-world form of the connectivity function is still a subjectof ongoing discussion and may depend on the particularanatomical region, see ibidem and [15], our theory does notdepend on a particular shape of g(·). The value of inversetemperature β has been set to 1000, which results in rejectingvast majority of changes leading to higher energy states.

The resulting final network state output by simulationincluded less than 400 units (out of 50000, that is about0.8 per cent) whose charge was not completely depleted andwhich are therefore natural candidates to be considered as theground units of the network, see Section III. Recall that ina fully connected network there is precisely one such groundunit, a single node storing all charge present in the network.In our present geometric situation though there are usuallyseveral ground units, enjoying exceptionally high supports(3). Formally to determine when to stop the simulation weshould use the criterion [Saturation] at the end of Section III,however for practical reasons we have fixed a large enoughnumber of iterations instead (up to 109), which allowed usto avoid the costly run-time checks and gave very goodconvergence to saturation, see the data below.

The cumulative log-log plot of spike flow graph in-degreesvs their frequencies, produced by simulation, is depicted in fig.1(a). We observe dramatic breakdown of occurrence frequencyfor high in-degree nodes. This behavior seems explicable whenwe take into account that high in-degree units are likely notto be connected, which makes the analysis leading to ourcrucial equation (13) fail to work for the highest support elite –effect discussed in the derivation of (13) and made explicit byvalidity bounds imposed on the support variable s there. In themiddle part of the plot strongly marked linear dependency canbe observed. Note that in terms of the original data this reflectsa power law with exponent given by the slope of the straightline in the log-log plot. Due to the afore-mentioned abnormalbehavior of the plot for highest support elite, a naive leastsquare estimate for this slope, attaining values a ' −1.2964, isdominated by outlying high in-degree values and also distortedby the statistics of low bulk neurons of supports too lowto guarantee WTA-type evolution. However, as discussed indetail in the theoretical part of this paper, we are interestedin the behavior of neurons exhibiting average characteristicsand thus we reject the highest elite and lowest bulk outliersand only concentrate on the statistics of the clearly pronouncedstraight line in the middle of the plot. In our case, due to finitesimulation size, quite modest compared to real-world neuralsystem sizes, this corresponds to rejecting about 40 per cent

degree

freq

101

102

103

104

105

101 102 103 104 105

(a) No cutoff. Slope ' −1.2964

degree

freq

101

102

103

104

105

101

102

103

104

105

(b) Cutoff 60 − 100% of the data.Slope ' −1.063

Fig. 1: Plot of degree distribution function and an approximat-ing line.

of the data with highest flow values, whereupon LS returnedslope parameter a ' −1.063 which is a good approximationof the theoretical value −1 for the slope of cumulative log-log plot. The cutoff range has been chosen ’manually’ to getthe best linear fit. Generally, it should be emphasized that thegreater network size is, the smaller cutoff is needed and betterapproximation is returned by the simulation.

Interesting structures, emerging during simulation, are high-est elite charge flow graphs centered at individual ground elitenodes, i.e. sets of edges, through which large amount of chargeflows towards a ground elite unit and nodes which transfertheir charges via those edges. It turns out that the entire flowgraph centered at a high-support unit usually covers nearly thewhole network (see fig. 2(a)). This confirms predictions aboutthe cooperation of elite in draining charge from the bulk units.Therefore, to get an interesting picture we should impose asize limit for a single elite node charge flow graph and onlyconsider edges of highest transfer counts. Having applied thesefor an individual graph we obtain an ,,attraction basin” of anelite node (figures 2(b), 2(c)). Basins tend to repel one another,but they are not completely separated because charge flowfrom bulk neurons branches out among elite units.

Fraction of accepted charge transfers can vary dependingon network size and iteration number, yet always rapidlydecreases in the course of simulation, marking the convergenceto equilibrium/saturation, and hardly ever exceeds two per centat later stages of network’s evolution. Figure 3(a) depicts thesefractions collected over simulation periods (one per cent ofconsecutive iterations) vs period number, this value rapidlyapproaches zero (in fact, interestingly enough, we observedit decays as a power function). Fluctuations observable on3(b) seem to originate from local metastable states visited bythe system in the course of the dynamics (see Section III) aswell as to zero-to-zero connections removal from the network(which speeds up computation while not affecting the WTAdynamics).

Due to high inverse temperature, number of transfers result-ing in energy increase is about few hundreds overall, which is0.01−0.1% of all accepted changes and thus can be considerednegligible.

draft

9

Neurons Connections Iterations Slope Geometry Units with charge9k 580k 70M -1.062 S2 0.00659k 560k 100M -1.060 S2 0.01

11k 700k 100M -1.100 S2 0.00912k 800k 100M -1.118 S2 0.00812k 800k 150M -1.100 S2 0.00912k 800k 150M -1.053 S2 0.008713k 960k 150M -1.066 (0..d)3 0.00819k 1.3M 150M -1.082 S2 0.01121k 1.7M 200M -1.170 (0..d)3 0.00821k 1.7M 200M -1.129 (0..d)3 0.0132k 2.1M 300M -1.102 S2 0.007633k 2.7M 200M -1.096 (0..d)3 0.01940k 3.4M 500M -1.066 (0..d)3 0.006545k 3.0M 500M -1.103 S2 0.008248k 4.2M 800M -1.063 (0..d)3 0.005648k 4.2M 600M -1.143 (0..d)3 0.01950k 3.4M 800M -1.080 S2 0.01650k 3.5M 800M -1.058 S2 0.007858k 4M 900M -1.091 S2 0.008258k 5M 1G -1.081 (0..d)3 0.006

TABLE I: Results of simulation. Table columns include number of neurons, number of connections, number of iterations,approximated slope value, network geometry (S2 is a sphere, (0..d)3 is three-dimensional cube) and fraction of nodes storingall network charge.

100 101 102 103 104100

101

102

103

Degree

Num

ber o

f nod

es

No E−R component estimated slope of CCDF=−1.014602

~1 E−R edge/node estimated slope of CCDF=−0.969015

~2 E−R edges/node estimated slope ofCCFD =−0.769935

~3 E−R edges/node estimated slope ofCCFD =−0.822615

~5 E−R edges/node estimated slope ofCCFD =−0.801819

100 101 102 103 104100

101

102

103

Degree

Num

ber o

f nod

es

No E−R component estimated slope ofCCFD= −0.966435

Out degree cumulative distributions, 1000 vertices

~1 E−R edge/node estimated slope ofCCFD = −0.898702

~2 E−R edges/node estimated slope ofCCFD = −0.759239

~3 E−R edges/node estimated slope of

CCFD −0.713454

~5 E−R edges/node estimated slope ofCCFD = −0.707421

~ 10 E−R edges/nodeestimated slope of CCFD = −0.793535

100 101 102 103 104 105100

101

102

103

104

Degree

Num

ber o

f nod

es

No E−R component, estimated slope ofCCFD=−0.976459

Out degree cumulative distributions, 5000 vertices

~1 E−R edge/node estimated CCFD slope = −0.976225

~2 E−R edges/node estimated CCFD slope= −0.917765

~5 E−R edges/node estimated CCFD slope= −0.659305

~10 E−R edges/node estimated CCDF slope = −0.683456

~3 E−R edges/node estimated CCFD slope = −0.839935

Fig. 4: Sample (out) degree distributions (complementary CDF, log-log plots) of mean field charge flow systems with variousdensity of Erdos-Renyi fractions. The figure shows results from simulation of 500,1000 and 5000 vertices from left to rightcorrespondingly. Clearly for the smaller system the presence of Erdos-Renyi fraction with approximately 2 edges/node alreadyseriously disturbs the power law. The bigger system maintains scale-free properties until the density of Erdos-Renyi fractionreaches approximately 5 edges/node.

B. Mean field model with long range connections

In this section we deal with long range connections, whosetheory has been given in Section V above, here restricting oursimulation to mean field set-up (connectivity function g ≡ 1)which was necessary at this stage of our work due to lack ofcomputational power to simulate significantly larger networksas needed to provide statistically relevant outcomes for set-ups where non-trivial connectivity function g and long rangeconnections co-exist (we believe the order of hundreds ofthousands would be required). The detailed numerical resultsfor the mean field model in its most basic form are availablein [28], [35]. The instances of the mean field model are muchsmaller that those presented in previous subsection due to all-to-all connectivity and memory limitations (5000 vertices isa reasonable limit for such simulations), yet this is alreadyenough to confirm our theoretical predictions because theconvergence of network’s asymptotic characteristics is much

faster here due to the simplicity of the model.

Below, we present the degree histograms of the meanfield model equipped with random long range connections asdiscussed in Section V. These connections can be regardedas an Erdos-Renyi random graph imposed on the original all-to-all connectivity structure, along whose edges the chargetransfers always get accepted (these exceptional edges alwaysconduct charge, neglecting the energy factor). Since in sucha case the simulation might not converge to a ground state(the charge might cycle forever, distorting the structure ofthe resulting charge flow graph), the amount of charge wasslightly reduced at every time step (each charge unit had afinite geometric lifetime) - this trick as mentioned in Sec-tion V ensures that the system does not over-saturate, whilepreserving all essential features of the dynamics. The figure4 shows complementary cumulative distributions (CCFD) ofsystems of three sizes (500, 1000 and 5000 units) under

draft

10

100 101 102 103 104100

101

102

103

104

Degree

Num

ber o

f nod

esOut degree distributions

200 vertices1000 vertices2000 vertices3000 vertices4000 vertices5000 vertices500 vertices

100 101 102 103 104 105100

101

102

103

104

Degree

Num

ber o

f nod

es

Out degree distributions (3 E−R edges/node)

200 vertices500 vertices1000 vertices3000 vertices5000 vertices2000 vertices

Fig. 5: Degree distributions (complementary CDF, log-log plots) of systems of various sizes plotted together. The left figureshows the original mean field spike flow model, where the power law is well pronounced for a variety of sizes. The right oneshows the model equipped with Erdos-Renyi fraction of approximately 3 edges/node. Note that the plot curvature decreasesas the number of vertices gets increased. While small systems clearly reveal light tailed decay in vertex degrees, the plotfor a system of 5000 vertices (red) is close to linear with the required slope 1 and thus resembles the predicted power lawdependency.

various densities of the exceptional connections. The powerlaws are fairly well preserved even if the exceptional edgesform a giant component (there is more than one edge pernode) and are nearly intact for sparser components (below thegiant component threshold). At some density the power lawbreaks down because the simulated system’s size is not highenough for the number of exceptional connections to remainnegligible. As expected, bigger systems can accept more denserandom components and still display power law connectivityas seen in figure 5. More details of the presented numericresults can be found in [39].

As already mentioned above, we haven’t run a combinedsimulation in which both geometry and exceptional connec-tions are present. The point is that we expect that, sinceboth these factors produce strong finite size effects (boundarydistortions to the range of node degrees obeying the powerlaw), the required size of simulated system that could revealthe scale-free connectivity in a statistically significant waymight be very large.

In particular this could be the reason why certain functionalnetworks inherit some structural properties usually found inscale-free networks [24], but lack a well pronounced powerlaw dependence in the degree distribution.

VIII. CONCLUSION

In this paper we have introduced a geometrically-embeddedspin glass type theoretical model for mesoscopic scale brainfunctional networks, essentially extending the mean-field set-up of [35]. We have proved mathematically and verifiednumerically that, in large system size asymptotics, our modelresults in scale-free functional networks in agreement withvoxel-level fMRI-based empirical findings [17]–[20]. The cor-responding power law exponent is 2, which is equal to or

not far from the exponents reported there. Moreover, wealso have shown that this exponent does not depend onparticular choice of the underlying structural connectivityfunction determining the embedding and even remains mostlyinvariant under additional modifications of model’s geometryand dynamics as discussed in Sections V, VI and SubsectionVII-B. These features are indicative of self-organized critical-ity and universality, in conceptual agreement with [29], [30],although their considerations involve essentially different scaleof network units. One further idea in common with [29], [30]is that whereas the dynamics underlying the real-world brainfunctional networks is extremely intricate and complicated, itsessential large scale emergent features can be described bya rather simply formulated mathematical model at criticality(a model in the same universality class in the language ofstatistical physics). In our theoretical study we have alsodetermined validity bounds of the power law connectivitystatistics in finite size systems. These bounds are felt inour simulations and result in cut-off power laws when thenumber of units is limited, which is reminiscent of the cut-offphenomena reported in [21]–[23]. It should be emphasized atthis point that the universality claim we make refers only to thepresence of scale-freeness and the corresponding exponent, itis clear other important asymptotic features of the network canand often do essentially depend on the underlying connectivityfunction. A number of further crucial properties have beenreported empirically for brain functional networks [15] and itis the subject of our further research in progress to verify theseproperties and their degree of universality for our model withvarious geometric embeddings.

draft

11

x

y

z

(a) Charge flow graph covering the whole network.

x

y

z

(b) Two graphs limited only to 200 most important edges per graph.

x

y

z

(c) Six graphs limited only to 100 most important edges per graph.

Fig. 2: Charge flow graphs.

100

1.0

0.1Simulation progress

Acceptedtransfers

(a) Overall plot.

100

0.03

3 · 10−3

Simulation progress

Acceptedtransfers

(b) Closer view at tail values.

Fig. 3: Fraction of accepted charge transfers in during simu-lation progress.

ACKNOWLEDGEMENTS

The authors gratefully acknowledge the support from thePolish Ministry of Scientific Research and Higher Educationgrant N N201 385234 (2008-2010). The work of J. Piersahas also been supported by the European Social Fund andGovernment of Poland as a part of Integrated OperationalProgram for Regional Development, ZPORR, Activity 2.6”Regional Innovation Strategies and Knowledge Transfer” ofKuyavian-Pomeranian Voivodeship in Poland.

REFERENCES

[1] R. Albert, H. Jeong, and A.-L. Barabasi, “Diameter of the world-wideweb,” Science, vol. 401, pp. 130–131, Septmeber 1999.

[2] A.-L. Barabasi, H. Jeong, Z. Neda, E. Ravasz, A. Schubert, andT. Vicsek, “Evolution of the social network of scientific collaborations,”Physica A, vol. 311, no. 4, pp. 590–614, 2002.

[3] S. Redner, “How popular is your paper? an empirical study of the citationdistribution,” European Physical Journal B, vol. 4, no. 2, pp. 131–134,1998.

[4] J. M. Montoya and R. V. Sole, “Small world patterns in food webs,”Journal of Theoretical Biology, vol. 214, no. 3, pp. 405–412, February7 2002.

[5] R. F. i Cancho and R. V. Sole, “The small-world of human language,”Proceedings of the Royal Society of London B, vol. 268, no. 1482, pp.2261–2265, November 7 2001.

[6] U. S. Bhalla and R. Iyengar, “Emergent properties of networksof biological signaling pathways,” Science, vol. 283, no. 5400, pp.381–387, January 1999. [Online]. Available: http://dx.doi.org/10.1126/science.283.5400.381

[7] H. Jeong, B. Tombor, R. Albert, Z. N. Oltvai, and A.-L. Barabasi, “Thelarge-scale organization of metabolic networks,” Nature, vol. 407, no.6804, pp. 651–653, October 5 2000.

[8] J. Abello, A. Buchsbaum, and J. Westbrook, “A functional approachto external graph algorithms,” ESA ’98: Proceedings of the 6th AnnualEuropean Symposium on Algorithms, pp. 332–343, 1998. [Online].Available: http://dx.doi.org/10.1007/3-540-68530-8 28

[9] W. Aiello, F. Chung, and L. Lu, “A random graph model for massivegraphs,” in STOC ’00: Proceedings of the thirty-second annual ACMsymposium on Theory of computing. New York, NY, USA: ACM,2000, pp. 171–180.

[10] J. I. Perotti, F. A. Tamarit, and S. A. Cannas, “A scale-free neuralnetwork for modelling neurogenesis,” Physica A Statistical Mechanicsand its Applications, vol. 371, pp. 71–75, Nov. 2006.

[11] D. Stauffer, A. Aharony, L. da Fontoura Costa, and J. Adler, “Efficienthopfield pattern recognition on a scale-free neural network,” The euro-pean physical journal B, no. 32, pp. 395–399, 2003.

[12] P. N. McGraw and M. Menzinger, “Topology and computationalperformance of attractor neural networks,” Physical Review E, vol. 68,p. 047102, 2003. [Online]. Available: http://www.citebase.org/abstract?id=oai:arXiv.org:cond-mat/0304021

[13] L. A. Amaral, A. Scala, M. Barthelemy, and H. E. Stanley,“Classes of small-world networks.” Proc Natl Acad Sci U S A, vol. 97,no. 21, pp. 11 149–11 152, October 2000. [Online]. Available:http://dx.doi.org/10.1073/pnas.200327197

draft

12

[14] C. Koch and G. Laurent, “Complexity and the Nervous System,”Science, vol. 284, no. 5411, pp. 96–98, 1999. [Online]. Available:http://www.sciencemag.org/cgi/content/abstract/284/5411/96

[15] E. Bullmore and O. Sporns, “Complex brain networks: graph theoreticalanalysis of structural and functional systems,” Nature Reviews Neuro-science, vol. 10, pp. 186–198, 2009.

[16] W. J. Freeman, “Scale-free neocortical dynamics,” Scholarpedia, vol. 2,no. 2, p. 1357, 2007. [Online]. Available: http://www.scholarpedia.org/article/Scale-free neocortical dynamics

[17] V. M. Eguıluz, D. R. Chialvo, G. A. Cecchi, M. Baliki, andA. V. Apkarian, “Scale-free brain functional networks.” PhysRev Lett, vol. 94, no. 1, January 2005. [Online]. Available:http://view.ncbi.nlm.nih.gov/pubmed/15698136

[18] R. Salvador, J. Suckling, M. R. Coleman, J. D. Pickard, D. Menon, andE. Bullmore, “Neurophysiological architecture of functional magneticresonance images of human brain,” Cerebral Cortex, vol. 15, pp. 1332–1342, 2005.

[19] G. A. Cecchi, A. R. Rao, M. V. Centeno, M. Baliki, A. V. Apkarian,and D. R. Chialvo, “Identifying directed links in large scale functionalnetworks: application to brain fmri,” BMC Cell Biology, vol. 8, Suppl.1, p. S5, 2007.

[20] M. P. Van den Heuvel, C. J. Stam, M. Boersma, and H. E. Hulshoff Pol,“Small-world and scale-free organization of voxel-based resting-statefunctional connectivity in the human brain.” Neuroimage, vol. 43, pp.528–539, 2008.

[21] S. Achard, R. Salvador, B. Whitcher, J. Suckling, and E. Bullmore,“A resilient, low-frequency, small-world human brain functionalnetwork with highly connected association cortical hubs,” J. Neurosci.,vol. 26, no. 1, pp. 63–72, January 2006. [Online]. Available:http://dx.doi.org/10.1523/JNEUROSCI.3874

[22] G. Gong, Y. He, L. Concha, C. Lebel, D. W. Gross, A. C. Evans,and C. Beaulieu, “Mapping anatomical connectivity patterns of humancerebral cortex using in vivo diffusion tensor imaging tractography,”Cerebral Cortex, vol. 19, pp. 524–536, 2009.

[23] S. Hayasaka and P. J. Laurienti, “Degree distributions in mesoscopicand macroscopic functional brain networks,” 2009. [Online]. Available:http://arxiv.org/abs/0903.4168

[24] M. Kaiser, R. Martin, P. Andras, and M. P. Young, “Simulationof robustness against lesions of cortical networks,” EuropeanJournal of Neuroscience, vol. 25, p. 3185, 2007. [Online]. Available:doi:10.1111/j.1460-9568.2007.05574.x

[25] F. Piekniewski, “Emergence of scale-free graphs in dynamical spikingneural networks,” in Proc. IEEE International Joint Conference onNeural Networks. Orlando, Florida, USA: IEEE Press, August 2007,pp. 755–759. [Online]. Available: http://ieeexplore.ieee.org/xpl/freeabsall.jsp?arnumber=4371052

[26] R. Galan, “On how network architecture determines the dominantpatterns of spontaneous neural activity,” PLoS ONE, vol. 3, p. e2148,2008.

[27] E. M. Izhikevich, “Simple model of spiking neurons,” IEEE Transactionson Neural Networks, no. 14, pp. 1569– 1572, 2003. [Online]. Available:http://www.nsi.edu/users/izhikevich/publications/spikes.pdf

[28] F. Piekniewski, “Spontaneous scale-free structures in spike flow graphsfor recurrent neural networks,” Ph.D. dissertation, Warsaw University,Warsaw, Poland, 2008. [Online]. Available: http://www.mat.umk.pl/∼philip/dissertation.pdf

[29] D. Fraiman, P. Balenzuela, J. Foss, and D. R. Chialvo, “Ising-likedynamics in large-scale functional brain networks,” Physical Review E,vol. 79, p. 061922, 2009.

[30] M. G. Kitzbichler, M. L. Smith, S. R. Christensen, and E. Bullmore,“Broadband criticality of human brain network synchronization,” PLoSComputational Biology, vol. 5, p. e1000314, 2009.

[31] R. Albert and A.-L. Barabasi, “Statistical mechanics of complexnetworks,” Reviews of modern physics, no. 74, pp. 47–97, January2002. [Online]. Available: http://arxiv.org/abs/cond-mat/0106096

[32] M. Newman, A.-L. Barabasi, and D. J. Watts, The Structure andDynamics of Networks: (Princeton Studies in Complexity). Princeton,NJ, USA: Princeton University Press, 2006. [Online]. Available:http://press.princeton.edu/titles/8114.html

[33] F. Chung and L. Lu, Complex Graphs and Networks (Cbms RegionalConference Series in Mathematics). Boston, MA, USA: AmericanMathematical Society, 2006.

[34] F. Piekniewski and T. Schreiber, “Emergence of scale-free spike flowgraphs in recurrent neural networks,” in Proc. IEEE Symposium Series inComputational Intelligence - Foundations of Computational Intelligence,Honolulu, Hawai USA, April 2007, pp. 357–362. [Online]. Available:http://doi.ieeecomputersociety.org/10.1109/FOCI.2007.371496

[35] ——, “Spontaneous scale-free structure of spike flow graphs inrecurrent neural networks,” Neural Networks, vol. 21, no. 10, pp. 1530– 1536, 2008. [Online]. Available: http://dx.doi.org/10.1016/j.neunet.2008.06.011

[36] R. Meester and R. Roy, Continuum percolation, ser. Cambridge Tractsin Mathematics. Cambridge: Cambridge University Press, 1996, vol.119.

[37] E. Aarts and J. Korst, Simulated annealing and Boltzmann machines: astochastic approach to combinatorial optimization and neural comput-ing. New York, NY, USA: John Wiley & Sons, Inc., 1989.

[38] T. Schreiber, “Spectra of winner-take-all stochastic neural networks,”2008. [Online]. Available: http://www.citebase.org/abstract?id=oai:arXiv.org:0810.3193

[39] F. Piekniewski, “Robustness of power laws in degree distributionsfor spiking neural networks,” in Proc. IEEE 2009 International JointConference on Neural Networks (IJCNN). Atlanta, Georgia, USA:IEEE Press, June 2009, pp. 2541–2546.