applications of cellular neural/nonlinear networks in physics

Applications of Cellular Neural/Nonlinear Networks in

Physics

Mária-Magdolna Ercsey-RavaszScientific advisors:

Dr. Prof. Zoltán Néda

Dr. Prof. Tamás Roska

Babes-Bolyai UniversityPéter Pázmány Catholic University

Outline

• CNN computing• A realistic random number generator• Stochastic simulations on CNN computers

• The site-percolation problem• The two-dimensional Ising model

• Optimization of spin-glasses on a space-variant CNN

• Pulse-coupled oscillators communicating with light pulses

The standard CNN modelEach cell has a circuit with:➡ Input voltage: u➡ State voltage: x➡ Output voltage: y

1121

ijijij xxy

ijjiNlkC

kljiNlkC

klijij zulkjiBtylkjiAtxdt

tdx

rr

),(),(),(),(

),;,()(),;,()()(

Template:Template: , A= , B= , z

111011

010010

111011

aaa

aaa

aaa

111011

010010

111011

bbb

bbb

bbb

.

L. O. Chua , L. Yang, IEEE Transactions on

Circuits and Systems 35. No. 10, 1988

The CNN Universal Machine

ACE16K CNN chip: 128*128 cells Bi-i V2

• programmable

• parallel processing

• continuous in time

• continuous (analog) in values

• discrete in space

• Universal (in Turing sense) on integers and on analog array signals

T. Roska, L. O. Chua, IEEE Transactions on Circuits and Systems – II, 40, 1993

CNN computingName Year Size-- 1993 12*12

ACE440 1995 20*22

POS48 1997 48*48

ACE4k 1998 64*64

CACE1k 2001 32*32*2

ACE16k 2002 128*128

XENON 2004 128*96*2

EYE-RIS 2007 176*144

• image processing

• real-time algorithms

• fast and smart camera computer

• robot eyes, bionic eye-glass

• cellular automata models

• partial differential equations

Research goals:• applications in physics

• how should CNN computers be further developed? – from physicist perspectives

Generating realistic random numbers

• Chaotic cellular automaton perturbed

with the natural noise of the chip P’(t)=P(t) xor N(t) N(t): - very few black pixels

- strong correlations but real stochastic fluctuations

Yalcin, et alle., Int.J. Circ. Theor. Appl., 32, 591-607, 2004

),()1,(),1())1,(),1((),(1 jixjixjixjixjixjix tttttt

P(t) P’1(t) XOR P’2(t)

• A good pseudo-random generator

• a good random binary image in t=116 µs• 1 single random value:

• ACE16K 7ns• Pentium 4 at 2.8 GHz (Linux) 33ns

Increasing the size of the chip in the future will

assure even much bigger advantage for

CNN chips

Trend for the simulation time as a function of the chip size

M. Ercsey-Ravasz, T. Roska, Z. Neda, Int. J. of Modern Physics C, Vol. 17, No. 6, p. 909 (2006)

• generating random images with different p density of the black pixels --- using more images with ½ density

--- if p is an n bit number we need n images

p=0.25 p=0.375 p=0.03125P Measured density1/2= 0.5 0.4999529

1/4=0.25 0.254261

1/8=0.125 0.124140

1/16=0.0625 0.061423

1/32=0.03125 0.031561

1/64=0.015625 0.015257

1/128=0.0078125 0.007470

1/256=0.00390625 0.004154

1/4 + 1/8=0.375 0.377712

Correlations:

• in space (first neighbors): 0.05% - 0.4%

• in time (consecutive steps): 0.7% - 0.8 %


Stochastic simulations on CNN computersThe site percolation problem

Used for modeling:- conductivity or mechanical properties of composite materials

- magnetization of dilute magnets at low temperatures

- fluid passing through porous materials

- propagation of diseases

Probability of percolation - density of black pixels

• 2nd order geometrical phase-transition• With CNN: 1 single template detecting percolation

• input: the random image• initial state: the first row• output: the parts connected to the first row

5.05.05.05.045.05.05.05.0

A

000040000

B 3z ACE16k chip

Percolation probability- for each p: 10000 different initial conditions - results agree with the accepted critical value:

pc=0.407

• Time needed:• CNN: t ~ L• digital computers: t ~ L

• For L=128 CNN is slower• if L grows still promises

advantage

2

Trend for the simulation time in function of the chip size


The two-dimensional Ising model

1,

i

jji

i

S

SSJH

kTEepE

pE/0

10

Energy of the system:

On CNN: A parallel Metropolis algorithm is used– Because parallel computing we have to avoid flipping 2 neighbors

simultaneously chessboard mask– Odd (even) step : spins marked with black (white) are updated– Equivalent with a Metropolis algorithm in which spins are

chosen in a well defined order

Metropolis algorithm

- randomly choose a spin and flip it with p probability

• Algorithm scheme for 1 MC step:– Build 3 masks marking: -generate 2 random images

- Spins with 4 similar neighbors (E=8J): M1 ----AND------ P1 with exp(-8J/kT) - Spins with 3 similar neighbors (E=4J): M2 ----AND------ P2 with exp(-4J/kT)- Other spins (E0): M3

– Build the composed mask M=(M1 AND P1) OR (M2 AND P2) OR M3– Use the (inverse) chessboard mask: M’= M AND C in (even) odd steps– Flip the spins marked on M’

T=2 T=2.3 T=2.6 ( J/k=1)Movies obtained with the ACE16K chip

M. Ercsey-Ravasz, T. Roska, Z. Neda, Eur. Phys. J. B, Vol. 51, No. 3, p. 407, (2006)

• Initial state: homogeneous• Boundary conditions: fixed • 5000 transition MC steps• Averaging over 10000 MC steps

Results for the Ising model


Magnetization

Specific heat

Susceptibility

Time needed for 1 MC step (128*128 lattice):Time needed for 1 MC step (128*128 lattice): 4.3 ms4.3 ms on the ACE16K chipon the ACE16K chip

2.2 ms2.2 ms on a Pentium on a Pentium 4 at 2.4 GHz4 at 2.4 GHz


Optimization of spin-glasses on a space-variant CNN

same local minimasame local minima

1 operation 1 operation ↔↔ 1 local minimum 1 local minimum

0

0),;,(,),;,(

1),;,(),,;,(),;,(,1),;,(

z

lkjiBbjijiB

jijiAjilkAlkjiAlkjiA

only simulatedonly simulated

),(

,,,;,

,,,;,ji

jijilkji

lkji yubyylkjiAE

• final state after an operation: yyij ij = = 11

• Lyapunov function (energy) of the CNN:

• monotone decreasing

• final state: local minimum (dE/dt=0)CNN Spin-glassy[-1,1] y=±1

The stochastic optimization algorithm

1 cooling process

Same principles as in simulated annealing:

• noise: random input U

• b ↔ strength of noise

• b slowly decreases

• we choose:

b0=5

Δb=0.05

NP-hard problem

hard for p<0.6

Speed estimation:

• the A templates must be introduced only once for each problem

• we can use the characteristic parameters of the ACE16k chip

1000- 5000 steps / second1000- 5000 steps / secondIndependent of size !Independent of size !

Many applications: error-correcting codes, econophysics, computer science etc.

M. Ercsey-Ravasz, T. Roska, Z. Neda, Physica D: Nonlinear Phenomena, Special issue: “Novel computing paradigms: Quo vadis?, accepted, (2008),

http://dx.doi.org/10.1016/j.physd.2008.03.028

Pulse-coupled oscillators communicating with light pulses

Motivations: - studying a CNN with pulse-coupled oscillators

- communicating with light global coupling

- perspectives: separately programmable oscillators

- first part of the study: collective behavior of identical units

The oscillators:• “electronic fireflies”• simple integrate-and-fire type neurons

• Photoresistor (R,U) + LED

• light R U

• G: threshold

• if U>G LED fires

• not before Tmin

• not after Tmax

Tmin 800 ms

Tmax 2700 ms

Firing 200 ms

Reaction time of the photoresistor 40 ms

Deviations: 2-10 %

Collective behavior

Order parameter: - normalized phase-histogram: smoothing:

n

kiiiik HH

nr

,1

)min()max(1

1

)(ih )(iH

n

kkrn

r1

1

PerspectivesSeparately programmable oscillators:

•Tmin, Tmax, Tflash, light intensity A, threshold G

CNN model using pulse-coupled oscillators:

jilk lk

klklklij ztutyAtx,, ,

)()()(

Benefits: - global coupling

- dynamical inputs

- time-delays

No independent A(i,j;k,l)

- pattern recognition, detecting spatio-temporal events

- studying the role of reaction time Δ

- dynamically changing the parameters

Conclusion• Realistic random numbers• Stochastic simulations on lattice models

- The site-percolation problem- The two-dimensional ising model- many related problems could be also simulated

• Locally variant CNN – very fast stochastic optimization algorithm for spin-glass models

• Further motivating the development of CNN-UM hardwares

• CNN built up by pulse-coupled oscillators– Ineteresting collective behavior – further studies– Communication with light could be useful idea also in hardware

projects

Journal publications

1. M. Ercsey-Ravasz, T. Roska, Z. Néda, “Perspectives for Monte Carlo simulations on the CNN universal machine”, Int. J. of Modern Physics C, Vol. 17, No. 6, pp. 909-923, 2006

2. M. Ercsey-Ravasz, T. Roska, Z. Néda, “Stochastic simulations on the cellular wave computers”, Eur. Phys. J. B, Vol. 51, No. 3, pp. 407-412, 2006

3. M. Ercsey-Ravasz, T. Roska, Z. Néda, “Statistical physics on cellular neural network computers”, Physica D: Nonlinear Phenomena, vol. Special issue: “Novel computing paradigms: Quo Vadis?”, 2008, accepted, http://dx.doi.org/10.1016/j.physd.2008.03.28

International conferences1. M. Ercsey-Ravasz, T. Roska, and Z. Neda, “Random number generator and

monte carlo type simulations on the cnn-um,” in Proceedings of the 10th IEEE International Workshop on Cellular Neural Networks and their applications, (Istanbul, Turkey), pp. 47–52, Aug. 2006.

2. M. Ercsey-Ravasz, Z. Sarkozi, Z. Neda, A. Tunyagi, and I. Burda, “Collective behavior of ”electronic fireflies”, SynCoNet 2007: International Symposium on Synchronization in Complex Networks, July 2007.

3. M. Ercsey-Ravasz, T. Roska, and Z. Neda, “Statistical physics on cellular neural network computers.” International conference ”Unconventional computing: Quo vadis?”, Mar. 2007.

4. M. Ercsey-Ravasz, T. Roska, and Z. Neda, “Spin-glasses on a locally variant cellular neural network.” International Conference on Complex Systems and Networks, July 2007.

5. M. Ercsey-Ravasz, T. Roska, and Z. Neda, “Applications of cellular neural networks in physics.” RHIC Winterschool, Nov. 2005.

6. M. Ercsey-Ravasz, T. Roska, and Z. N´eda, “The cellular neural network universal machine in physics.” International Conference on Computational Methods in Physics, Nov. 2006.

7. M. Ercsey-Ravasz, T. Roska, and Z. Neda, “NP-hard optimization using locally variant CNN,” accepted in the Proceedings of the CNNA2008.

Thank You!

applications of cellular neural/nonlinear networks in physics

Documents