H. Guclu, G. Korniss (Rensselaer)B. Kozma (Rensselaer)M.B. Hastings, Z. Toroczkai (Los Alamos)M.A. Novotny (Mississippi State University)P.A. Rikvold (Florida State University)

Funded by NSF, Research Corporation

Extreme Fluctuations in Small Worlds

with Relaxational Dynamics

Synchronization in Coupled Multi-Component Systems

Collective dynamics on the network

Examples:!Internet (packet traffic/flux)!Load-balancing schemes (job allocation among processors)!Electric power grid (power transmission and phase synchronization)!High-performance or grid-computing networks (progress of the processors in distributed simulations)

Fluctuations of the �load� in the network!Average size of the fluctuations!Typical size of the largest fluctuationsfailures/delays are often triggered by extreme events

Critical phenomena in small-world networks

!Monasson (1999): diffusion on SW networks!Barrat&Weight (2000), Gitterman (2000), Kim et al. (2001),

Herrero (2002): Ising model on SW networks!Hong et al. (2002): XY-model and Kuramoto oscillators on

SW networks.!these systems typically exhibit (strict or anomalous)

mean-field-like phase transitions!Hastings (2003): general criterion for mean-field scaling

Synchronization in Parallel Discrete-Event Simulations (PDES)

Parallelization for asynchronous dynamics of continuum-time processes

Paradoxical task:! (algorithmically) parallelize (physically) non-parallel

dynamics

Examples:! Cellular communication networks (call arrivals)! Magnetization dynamics in condensed matter(Ising model: spin flips with Glauber dynamic)

!Spatial epidemic models, contact process (infections)

Must use local simulated times {hi} on each processor and a synchronization scheme to preserved causality

local synchronization rules for the processing elements (PE)

� one-site-per PE, NPE=N=Ld

� t=0,1,2,� parallel steps� hi(t) local simulated time� local time increments areiid exponential randomvariables (Poisson asynchrony)

� advance only if }min{ nnhhi ≤(nn: nearest neighbors)

i (PE index)

{ }ih

Lubachevsky (1987)

: �virtual time horizon�{ }ih

Synchronization landscape (virtual time horizon)

410=N �rough� landscape

N~ξ

ξαNw ~

w

correlation length

roughness exponent

2

1

2 )]()([1)( ththN

twN

ii∑

=

−= ∑=

=N

iih

Nh

1

1

width (measure of de-synchronization)

1=dprogress of the simulation

)()2( 11 thhhh iiiiit η++−+=∂ −+ K

∑=

+Γ−=∂N

jij

oijit thh

1)(η

)'(2)'()( tttt ijji −=⟩⟨ δδηηGaussian noise

α22 ~ Nw L⟩⟨2/1=α

roughness exponent

(Edward-Wilkinson model, kinetic roughening)prototypical synchronization problem for unbounded local variables with local relaxation and short tailed noise

local synchronization/communication rules in PDES(�microscopic dynamics�)

effective equation of motion form the virtual time horizon

G. K. et al. Phys. Rev. Lett. (2000).

Suppressing roughness in virtual time horizons

!Watts & Strogatz (1998): �� enhanced signal-propagation speed, computational power, and synchronizability�.

�soft� SW network (ER on top of ring)

−=

NpyprobabilitwithNpyprobabilitwith

Jij /10/1

�hard� SW network

[ ] pJJl

ill

il ==

∑∑ average degree(in addition to n.n.)

:or0 pJij =N/2 random links are selected,such that each site has exactly one random link of strength p(in addition to n.n.)

∑ =l

il pJ no fluctuations in the individual degree

average over network realizations

(Newman & Watts, Monasson, 1999)

Alternative SW constructions:

Edward-Wilkinson model on small-world networks

)()()2(1

11 thhJhhhh i

N

jjiijiiiit η+−−−+=∂ ∑

=−+

∑=

+Γ−=∂N

jijijit thh

1)(η

112 −+ −−=Γ jijijioij δδδ

ijl

ilijij JJV −= ∑δ -Laplacian on random part of the network

-Laplacian on regular network (ring)

Vo +Γ=Γ

[ ] ( )[ ]iiNw 12 −Γ=⟩⟨disorder-averaged width:

effective interaction due to random links (self-energy)

�soft� network:

�hard� network:

K+Σ 2~ p

K+−Σ 2/3

21~ pp

[ ]Σ

⟩⟨→∞

21~2

N

NwΣ

→∞ 1~N

ξ

finite widthfinite correlation length for any p>0

)()()2( 11 thhhhhh iiiiiit η+−Σ−−+=∂ −+

Impurity-averaged perturbation theory

SW network induces effective relaxation to the mean

B. Kozma, M.B. Hastings, and G.K, Phys. Rev. Lett. (2004).

(anomalous mean-field)

(asymptotically strict mean-field)

αNw ~ .~ constwN ∞→

0.0=p 1.0=p

random connections:used with probability p>0

410=N

p

regular lattice (ring) topology(�p=0�)

ww

Implementation to PDES:

Steady-state height structure factorsNhNhhkS kkk /|~|/~~)( 2⟩⟨=⟩⟨= −

2

1~)(k

kSΣ+2

1~)(k

kS

1d ring SW network

LNPE =0.0=p

10.0=p

roughnessutilization = fraction of non-idling PEs

utilization trade-off/scalable data management

speedup =utilization × NPE

G. K., Novotny, Guclu, Toroczkai, Rikvold, Science, (2003).fully scalable high-performance or grid-computing networks

Relationship between extremal statistics and universal fluctuations

!Bramwell et al. (1998): turbulence, XY, SOC, etc.!Watkins, Chapman, & Rowlands (2002)!Dahlstedt & Jensen (2001): SOC (Sneppen model)!Aji & Goldenfeld (2001)!Antal, Györgyi, Rácz ... (2001, 2003): 1/f and other types of interfaces

etc.),(),( 2wPmP

∑=

−+ ++−−−+=∂N

jijiijiiiit thhJhhhh

111 )()()2( ηK

2

1

)(1 hhN

wN

ii∑

=

−=

⟩−⟨=⟩∆⟨ hhmaxmax

Extreme height fluctuationsin synchronization landscape

αNw ~αN~max ⟩∆⟨

.~ constwN ∞→

δ/1max )][ln(~ Nw⟩∆⟨

regular 1d (ring) small world

Synchronizability of coupled multi-component systems with�local� relaxation and short-tailed noise

Raychaudhuri et al., PRL (2001) H. Guclu, and G.K., PRE (2004)

virtual time horizon: finite correlation length → quasi-independent blocks

ξ/NN →

( ) )ln(~)/ln(~ /1max NwNw

N ∞→>∆< δξ

1=δ

Galambos (1978)Bouchaud & Mézard (1997)Baldassarri (2000)

hhii −≡∆

(measured)

Extreme height fluctuationsin the virtual time horizon

hh −=∆ maxmax

αNw ~αN~max∆ )ln(~max N∆

ring:

)0.0( =p

small worlds:

w

⟩∆⟨ max

N

same exponent .~ constwN ∞→

max∆

)( max∆p

max∆

FTG

�soft� SW �hard� SW

Summary" SW links can facilitate synchronization of parallel

computing networks with many nodes" Relevant node-to-node process is relaxation in the

presence of short-tailed noise →FTG distribution for extreme fluctuations (weak logarithmic divergence for the typical size of the largest fluctuations)

" In progress: extreme fluctuations of the �load� in other types of networks with other types of noise

www.rpi.edu/~kornissH. Guclu, and G.K., PRE (in press, 2004); arXiv:cond-mat/0311575.B. Kozma, M.B. Hastings, and G.K, Phys. Rev. Lett. 92, 108701 (2004).G. K., M.A. Novotny, H. Guclu, Z. Toroczkai, and P.A. Rikvold, Science 299, 677 (2003).

G. K., Z. Toroczkai, M.A. Novotny, and P.A. Rikvold, Phys. Rev. Lett. 84, 1351 (2000).